IPEDSuploadables: Transforms Institutional Data into Text Files for IPEDS Automated Import/Upload

Description

Starting from user-supplied institutional data, these scripts transform, aggregate, and reshape the information to produce key-value pair data files that are able to be uploaded to IPEDS (Integrated Postsecondary Education Data System) through their submission portal https://surveys.nces.ed.gov/ipeds/. Starting data specifications can be found in the vignettes. Final files are saved locally to a location of the user's choice. User-friendly readable files can also be produced for purposes of data review and validation.

Author(s)

Maintainer: Alison Lanski alanski@nd.edu

Authors:

• Shiloh Fling shilohfling@gmail.com

See Also

Useful links:

• https://github.com/AlisonLanski/IPEDSuploadables

• https://alisonlanski.github.io/IPEDSuploadables/

• Report bugs at https://github.com/AlisonLanski/IPEDSuploadables/issues

IPEDSuploadables package

Description

Tools to assist data formatting for upload to IPEDS surveys

Details

See the README on GitHub or view documentation at the pkgdown site

Shortcut function to turn a dataframe into key-value pairs

Description

Shortcut function to turn a dataframe into key-value pairs

Usage

apply_upload_format(df)

Arguments

Value

a dataframe with one column and upload-compatible rows

Dummy cip data for Completions functions

Description

Contains sample values for extra cip codes

Usage

com_cips

Format

A data frame with 3 rows and 10 columns

Details

See complete information by running ?create_dummy_data_com.R

Dummy student data for Completions functions

Description

Contains sample values for students

Usage

com_students

Format

A data frame with 105 rows and 13 columns

Details

See complete information by running ?create_dummy_data_com.R

Create dummy data for testing the completions functions

Description

Creates a prepared dataframe to test scripts related to IPEDS Completions reporting. Produces either a student/degree dataframe or a dataframe of cips previously reported but not in the current student data, depending on the argument you select

Usage

create_dummy_data_com(df_type = "student")

Arguments

Value

a dataframe ready for the rest of the comp scripts

Note

The final dataset has 60 students with 105 majors. Students 100-130, 140, 150 have 1 major for 1 degree (journalism) Students 131-139 have 2 majors for 1 degree (journalism + parks) Students 141-149 have 3 majors for 1 degree (journalism, parks, linguistics) Students 151-159 have 3 majors for 2 degrees (1 degree with journalism/parks, 1 MBA degree) Note: 1 student has a faulty birthdate; this will show the warning "1 failed to parse"

Two rows (level 18 linguistics) are flagged as distance education

To fully process completions, we will need to include an example of a CIP code that is a possible major but has no completers and a CIP code in an award level that is possible but has no completers This is the second piece of dummy df produced

Examples

set.seed(1892)

# one date fails to parse:
# this is to provide an example of missing
# data which is acceptable to IPEDS
students <- create_dummy_data_com()

additional_cips <- create_dummy_data_com(df_type = "cip")

Create dummy data for testing the completions functions

Description

Creates a prepared dataframe to test scripts related to IPEDS 12 Month Enrollment reporting. Produces either a student dataframe or a dataframe of instructional activity, depending on the argument you select

Usage

create_dummy_data_e1d(df_type = "student")

Arguments

Value

a dataframe ready for the rest of the e1d scripts

Note

The final dataset has 100 students 60 UG students (40 FT, 20 PT; 26 seeking degrees, 34 not) UG include: 20 first time, 20 transfer, 20 continuing/returning; 40 Grad Students (10 FT, 30 PT; 24 seeking degrees, 16 not)

For simplicity, only 1 race-ethnicity category is used 5 UG and 5 Grad are set to be fully distance ed 10 UG are set to be partially distance ed

Examples

set.seed(1892)

student_df <- create_dummy_data_e1d()

instr_df <- create_dummy_data_e1d(df_type = "instr")

Create dummy data for testing the fall enrollment functions

Description

Creates students and retention dataframes for use in parts A, B, C, D, E, G, H. Student-faculty ratio (part G) will ask for a number when the function is run and does not need to exist here. To create both dataframes, run the function twice with different arguments, and save results into separate objects.

Usage

create_dummy_data_ef1(df_type = "students", n = 100)

Arguments

Value

A dataframe ready for the rest of the ef1 scripts

Examples

set.seed(1234)

#default creates 100 students
students <- create_dummy_data_ef1()

#change the dataframe
retention <- create_dummy_data_ef1(df_type = "retention")

#change the population size
more_students <- create_dummy_data_ef1(df_type = "students", n = 250)

Create dummy data for testing the Grad Rates functions

Description

Creates dummy data for testing the Grad Rates functions

Usage

create_dummy_data_gr(n = 100)

Arguments

Value

a dataframe ready for the rest of the Grad Rates functions

Examples

#use this seed to reproduce the dummy data saved to the package
set.seed(4567)

#default makes 100 students
graduated <- create_dummy_data_gr()

more_graduated <- create_dummy_data_gr(n = 500)

Create dummy data for testing the Grad Rates 200 function

Description

Dummy data for Grad Rates 200 testing

Usage

create_dummy_data_gr200(n = 1000)

Arguments

Value

a dataframe ready for the rest of the Grad Rates 200 functions

Examples

set.seed(4567)

#default creates 1000 students
graduates <- create_dummy_data_gr200()
more_graduates <- create_dummy_data_gr200(n = 100)

Create dummy data for testing the hr functions

Description

to do: save this out into the package and make it accessible as package data

Usage

create_dummy_data_hr()

Value

a dataframe ready for the rest of the hr scripts

Examples

set.seed(4567)
hr_pop <- create_dummy_data_hr()

Create dummy data for testing the outcome measures functions

Description

Creates a prepared dataframe to test scripts related to IPEDS Outcome Measures reporting. Produces either a student/status dataframe

Usage

create_dummy_data_om()

Details

remember: want to save this data out into the package so it's available

Value

a dataframe ready for the rest of the om scripts

Note

The final dataset has 20 students covering most statuses

Examples

#creates a very specific population
#function does not allow for anything to be updated at time of run
#in other words: will always create a fixed-value dataframe
dat <- create_dummy_data_om()

Dummy aggregated data for 12 Month Enrollment part B

Description

Contains sample values for credit hours generated and doctors-professional FTE

Usage

e1d_instr

Format

A data frame with 1 row and 5 columns

Details

See complete information by running ?create_dummy_data_e1d.R

Dummy student-level data for 12 Month Enrollment parts A, C, D, E, and F

Description

Contains 100 fictional student records with all required data

Usage

e1d_students

Format

A data frame with 100 rows (students) and 14 columns

Details

See complete information by running ?create_dummy_data_e1d.R

Dummy student retention data for Fall Enrollment scripts part E

Description

This data provides aggregated counts in a dataframe suitable for use in the retention component of the Fall Enrollment survey.

Usage

ef1_retention

Format

A data frame with 2 rows and 6 columns

Dummy student data for Fall Enrollment scripts

Description

Using the default number of students, this data provides a population that touches most available categories of student reporting. Some columns use only a selection of possible values to reduce complexity.

Usage

ef1_students

Format

A data frame with 100 rows and 25 columns

Note

To recreate the saved dataframe exactly, use seed 1234 with 100 students.

Grab institution's UNITID from supplied data to populate missing-data rows

Description

Grab institution's UNITID from supplied data to populate missing-data rows

Usage

get_ipeds_unitid(df)

Arguments

Value

a character unitid

Dummy student data for Graduation Rates 200 functions

Description

Contains sample values for students

Usage

gr200_students

Format

A data frame with 1000 rows and 5 columns

Details

See complete information by running ?create_dummy_data_gr200.R

Dummy student data for the Graduation Rates scripts

Description

Dummy student data for the Graduation Rates scripts

Usage

gr_students

Format

A data frame with 101 rows and 13 columns

Details

Includes only 3 Race/Ethnicity categories [6, 7, 8] for simpler code; one student (a program-switcher) has a 4th category [1] for easy tracking

Dummy staff data for Human Resources functions

Description

Contains sample values for staff

Usage

hr_staff

Format

A data frame with 3600 rows and 13 columns

Details

See complete information by running ?create_dummy_data_hr.R

Make Completions Part A

Description

Make Completions Part A

Usage

make_com_part_A(df, extracips = NULL)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Completions Part B

Description

Make Completions Part B

Usage

make_com_part_B(df, extracips = NULL)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Completions Part C

Description

Make Completions Part C

Usage

make_com_part_C(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Completions Part D

Description

Make Completions Part D

Usage

make_com_part_D(df, extracips = NULL)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Completions Part E (gender details)

Description

Make Completions Part E (gender details)

Usage

make_com_part_E(df, ugender, ggender)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make 12 Month Enrollment Part A

Description

Make 12 Month Enrollment Part A

Usage

make_e1d_part_A(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make 12 Month Enrollment Part B

Description

Make 12 Month Enrollment Part B

Usage

make_e1d_part_B(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make 12 Month Enrollment Part C

Description

Make 12 Month Enrollment Part C

Usage

make_e1d_part_C(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make 12 Month Enrollment Part D (gender details)

Description

Make 12 Month Enrollment Part D (gender details)

Usage

make_e1d_part_D(df, ugender, ggender)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make 12 Month Enrollment Part E

Description

R/E and Gender counts for dual enrollment (high school students)

Usage

make_e1d_part_E(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make 12 Month Enrollment Part F

Description

Flag questions about high school students enrolled for credit

Usage

make_e1d_part_F(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Fall Enrollment Part A

Description

Breakdown of students level and demographics; also by designated CIPs in required years

Usage

make_ef1_part_A(df, cips = TRUE)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Fall Enrollment Part B

Description

Student Counts by Age/gender

Usage

make_ef1_part_B(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Fall Enrollment Part C

Description

State of origin for first time students

Usage

make_ef1_part_C(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Fall Enrollment Part D

Description

Count of new non-degree students

Usage

make_ef1_part_D(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Fall Enrollment Part E

Description

Retention counts

Usage

make_ef1_part_E(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Fall Enrollment Part F

Description

Student Faculty Ratio

Usage

make_ef1_part_F(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Fall Enrollment Part G

Description

Distance Ed counts

Usage

make_ef1_part_G(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Fall Enrollment Part H (gender details)

Description

Make Fall Enrollment Part H (gender details)

Usage

make_ef1_part_H(df, ugender, ggender)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Graduation Rates 200

Description

Make Graduation Rates 200

Usage

make_gr200(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Graduation Rates Part B

Description

Make Graduation Rates Part B

Usage

make_gr_part_B(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Graduation Rates Part C

Description

Make Graduation Rates Part C

Usage

make_gr_part_C(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Graduation Rates Part E (gender details)

Description

'r lifecycle::badge("deprecated")'

This section was removed by IPEDS in the 2024-2025 reporting cycle, after being required for the 2023-2024 reporting cycle. It remains here in case it is re-added in a future year. I have updated testing/produce code so nothing else calls it. By making it internal, it won't be listed but remains available.

Usage

make_gr_part_E(df, ugender)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Human Resources Part A1

Description

Part A1 — COUNT of FT INSTRUCTIONAL staff by tenure status, academic rank, and race/ethnicity/gender

Usage

make_hr_part_A1(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Human Resources Part A2

Description

Part A2 — COUNT of FT instructional staff by tenure status, medical school, and function

Usage

make_hr_part_A2(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Human Resources Part B1

Description

HR Part B1 — COUNT of FT Non-instructional staff by occupational category

Usage

make_hr_part_B1(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Human Resources Part B2

Description

Part B2 — Full-time non-instructional staff by tenure, medical school, and occupational category

Usage

make_hr_part_B2(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Human Resources Part B3

Description

Part B3 — Full-time non-instructional staff by medical school, and occupational category

Usage

make_hr_part_B3(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Human Resources Part D1

Description

Part D1 — Part-time staff by occupational category

Usage

make_hr_part_D1(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Human Resources Part D2

Description

Part D2 — Graduate assistants by occupational category and race/ethnicity/gender

Usage

make_hr_part_D2(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Human Resources Part D3

Description

Part D3 — Part-time staff by tenure, medical school, and occupational category

Usage

make_hr_part_D3(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Human Resources Part D4

Description

Part D4 — Part-time Non-instructional staff by medical school, and occupational category

Usage

make_hr_part_D4(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Human Resources Part G1

Description

Part G1 — Salaries of INSTRUCTIONAL staff

Usage

make_hr_part_G1(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Human Resources Part G2

Description

Part G2 — Salaries of non-instructional staff

Usage

make_hr_part_G2(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Human Resources Part H1

Description

Part H1 — Full-time new hire instructional staff by tenure status and race/ethnicity/gender

Usage

make_hr_part_H1(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Human Resources Part H2

Description

Part H2 — New hires by occupational category, Race/Ethnicity/Gender

Usage

make_hr_part_H2(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Outcome Measures Part A

Description

Establishing the Outcome Measures cohorts

Usage

make_om_part_A(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Outcome Measures Part B

Description

Award Status at Four Years after Entry

Usage

make_om_part_B(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Outcome Measures Part C

Description

Award Status at Six Years after Entry

Usage

make_om_part_C(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Make Outcome Measures Part D

Description

Award Status and Enrollment at Eight Years after Entry

Usage

make_om_part_D(df)

Arguments

Value

A dataframe with the required IPEDS structure for this survey part

Dummy data for Outcome Measures functions

Description

Contains sample values for students

Usage

om_students

Format

A data frame with 20 rows and 9 columns

Details

See complete information by running ?create_dummy_data_om.R

Output files from the make functions using the package dummy data (for testing)

Description

Contains a list of output files from each "make" script which can be called by testthat tests to ensure refactoring doesn't change output inappropriately

Usage

part_outputs

Format

A list of dataframes: one dataframe per make script

Details

General list-creation code is commented out in the test-part-outputs.R script; uncomment and run/add as needed if a df needs a legitimate update (e.g. IPEDS rules change, dummy data includes new use case)

Examples

part_outputs$com_partD

Some initial recoding for Completions

Description

Some initial recoding for Completions

Usage

prep_com_data_frame(df)

Arguments

Value

A dataframe ready for the make_com scripts

Some initial recoding for Fall Enrollment

Description

Some initial recoding for Fall Enrollment

Usage

prep_ef1_data_frame(df)

Arguments

Value

A dataframe ready for the make_ef1 scripts

Some initial recoding for Human Resources

Description

Some initial recoding for Human Resources

Usage

prep_hr_data_frame(df)

Arguments

Value

A dataframe ready for the make_hr scripts

Set up extra_awards df for Outcome Measures part B, C, D

Description

Select correct year, ensure all award levels end up with a column

Usage

prep_om_awards(df, award)

Arguments

Value

A dataframe pivoted and prepared for use within the make_om_part functions B-D

Some initial recoding for OutcomeMeasures

Description

Some initial recoding for OutcomeMeasures

Usage

prep_om_data_frame(df)

Arguments

Value

A dataframe ready for the make_om scripts

Shortcut function with all steps to provide a Completions report

Description

Shortcut function with all steps to provide a Completions report

Usage

produce_com_report(
  df,
  extracips = NULL,
  part = "ALL",
  format = "uploadable",
  ugender = TRUE,
  ggender = TRUE
)

Arguments

Value

A txt or csv file at the path of your choice

Examples

#entire report
produce_com_report(com_students, com_cips)

#one part as csv instead of key-value
produce_com_report(com_students, com_cips, part = "A", format = "readable")

Shortcut function with all steps to provide a 12 Month Enrollment report

Description

Shortcut function with all steps to provide a 12 Month Enrollment report

Usage

produce_e1d_report(
  df,
  hrs,
  part = "ALL",
  format = "uploadable",
  ugender = TRUE,
  ggender = TRUE
)

Arguments

Value

A txt or csv file at the path of your choice

Examples

#entire report
produce_e1d_report(e1d_students, e1d_instr)

#one part, as csv instead of key-value file
produce_e1d_report(e1d_students, part = "A", format = "readable")

Shortcut function with all steps to provide a Fall Enrollment report

Description

Shortcut function with all steps to provide a Fall Enrollment report

Usage

produce_ef1_report(
  students,
  retention,
  part = "ALL",
  include_optional = FALSE,
  format = "uploadable",
  ugender = TRUE,
  ggender = TRUE
)

Arguments

Value

A txt or csv file at the path of your choice

Examples

#entire report
produce_ef1_report(ef1_students, ef1_retention)

#entire report with optional sections
produce_ef1_report(ef1_students, ef1_retention, include_optional = TRUE)

#one part as csv instead of key-value
produce_ef1_report(ef1_students, part = 'D', format = 'readable')

Shortcut function with all steps to provide a Grad Rates 200 report

Description

Shortcut function with all steps to provide a Grad Rates 200 report

Usage

produce_gr200_report(df, format = "uploadable")

Arguments

Value

A txt or csv file at the path of your choice

Examples

#entire report
produce_gr200_report(gr200_students)

Shortcut function with all steps to provide a Graduation Rates report

Description

Shortcut function with all steps to provide a Graduation Rates report

Usage

produce_gr_report(
  df,
  part = "ALL",
  format = "uploadable",
  ugender = lifecycle::deprecated()
)

Arguments

Value

A txt or csv file at the path of your choice

Examples

#entire report
produce_gr_report(gr_students)

#one part in csv format instead of key-value
produce_gr_report(gr_students, part = "B", format = "readable")

Shortcut function with all steps to provide a Human Resources report

Description

Shortcut function with all steps to provide a Human Resources report

Usage

produce_hr_report(df, part = "all", format = "uploadable")

Arguments

Value

A txt or csv file at the path of your choice

Examples

#entire report
produce_hr_report(hr_staff)

#subsection with csv output instead of key-value txt
produce_hr_report(hr_staff, part = "A1", format = "readable")

Shortcut function with all steps to provide an Outcome Measures report

Description

Shortcut function with all steps to provide an Outcome Measures report

Usage

produce_om_report(df, part = "ALL", format = "uploadable")

Arguments

Value

A txt or csv file at the path of your choice

Examples

#entire report
produce_om_report(om_students)

#one part with csv output instead of key-value
produce_om_report(om_students, part = 'A', format = 'readable')

Produce an upload-compatible txt file from pre-aggregated files

Description

Use this function to create a key-value pair uploadable file from your own prepared dataframes, instead of using a different (provided) produce function. Your dataframes must be prepped to match final submission requirements as laid out by IPEDS (or whatever survey you will use this for. Use this function for one survey at a time, and add a separate dataframe for each part to the ... argument. See vignette for more details.

Usage

produce_other_report(..., survey = "MySurvey", part = "AllParts")

Arguments

Value

txt file on your computer with the title [survey]_[part]_[today's date].txt

Note

You must name the arguments for survey and part if using non-default value. If the arguments are unnamed, the function will assume their values are additional dataframes.

Examples

#With built-in R data
produce_other_report(mtcars[1:5,], iris[1:5,], ToothGrowth[1:5,], survey = 'FakeSurvey')


#Will not execute properly (argument unnamed)
#produce_other_report(mtcars[1:5,], iris[1:5,], ToothGrowth[1:5,], 'FakeSurvey')

Set the path for where the reports will be saved to.

Description

Set the path for where the reports will be saved to.

Usage

set_report_path()

Value

path

Table of data requirements for Completions starting dataframe

Description

Table of data requirements for Completions starting dataframe

Usage

specs_COM

Format

A data frame with 21 rows and 4 columns

Table of data requirements for 12 Month Enrollment starting dataframes

Description

Table of data requirements for 12 Month Enrollment starting dataframes

Usage

specs_E1D

Format

A data frame with 19 rows and 4 columns

Table of data requirements for Fall Enrollment starting dataframes

Description

Table of data requirements for Fall Enrollment starting dataframes

Usage

specs_EF1

Format

A data frame with 23 rows and 4 columns

Table of data requirements for Graduation Rates starting dataframe

Description

Table of data requirements for Graduation Rates starting dataframe

Usage

specs_GR

Format

A data frame with 13 rows and 3 columns

Table of data requirements for Grad Rates 200 starting dataframe

Description

Table of data requirements for Grad Rates 200 starting dataframe

Usage

specs_GR200

Format

A data frame with 5 rows and 3 columns

Table of data requirements for HR starting dataframe

Description

Table of data requirements for HR starting dataframe

Usage

specs_HR

Format

A data frame with 13 rows and 3 columns

Table of data requirements for OM starting dataframe

Description

Table of data requirements for OM starting dataframe

Usage

specs_OM

Format

A data frame with 9 rows and 3 columns

Write the prepared data to a txt file in key-value format

Description

Write the prepared data to a txt file in key-value format

Usage

write_report(..., survey, part, output_path)

Arguments

Value

a txt file (at the path location)

Note

All arguments for this function are required and must be named. Dataframes must have the key as the column name (with appropriate capitalization) and the value in the cells

Write the prepared data to a csv file

Description

Write the prepared data to a csv file

Usage

write_report_csv(df, survey, part, output_path)

Arguments

Value

a csv file (at the path location)

Note

All arguments for this function are required. The dataframe must have the key as the column name (with appropriate capitalization) and the value in the cells