Example 1: National Population Trends
Real-world motivation: Understanding population growth patterns is fundamental for policy planning, from infrastructure investments to social security projections. Let’s examine U.S. population trends to see how growth has changed over time.
# Get U.S. population data
# We found these identifiers using the Data Commons website:
# - Variable: "Count_Person" (from Statistical Variable Explorer)
# - Place: "country/USA" (from Place Explorer)
us_population <- dc_get_observations(
variable_dcids = "Count_Person",
entity_dcids = "country/USA",
date = "all",
return_type = "data.frame"
)
# Examine the structure
glimpse(us_population)
#> Rows: 490
#> Columns: 8
#> $ entity_dcid <chr> "country/USA", "country/USA", "country/USA", "country/US…
#> $ entity_name <chr> "United States of America", "United States of America", …
#> $ variable_dcid <chr> "Count_Person", "Count_Person", "Count_Person", "Count_P…
#> $ variable_name <chr> "Total population", "Total population", "Total populatio…
#> $ date <chr> "1900", "1901", "1902", "1903", "1904", "1905", "1906", …
#> $ value <int> 76094000, 77584000, 79163000, 80632000, 82166000, 838220…
#> $ facet_id <chr> "2176550201", "2176550201", "2176550201", "2176550201", …
#> $ facet_name <chr> "USCensusPEP_Annual_Population", "USCensusPEP_Annual_Pop…
# Notice we have multiple sources (facets) for the same data
unique(us_population$facet_name)
#> [1] "USCensusPEP_Annual_Population"
#> [2] "CensusACS5YearSurvey_AggCountry"
#> [3] "CensusACS5YearSurvey"
#> [4] "USDecennialCensus_RedistrictingRelease"
#> [5] "WorldDevelopmentIndicators"
#> [6] "USCensusPEP_AgeSexRaceHispanicOrigin"
#> [7] "OECDRegionalDemography_Population"
#> [8] "CensusACS5YearSurvey_SubjectTables_S0101"
#> [9] "CensusACS5YearSurvey_SubjectTables_S2601A"
#> [10] "CensusACS5YearSurvey_SubjectTables_S2602"
#> [11] "CensusACS5YearSurvey_SubjectTables_S2603"
#> [12] "CDC_Mortality_UnderlyingCause"
#> [13] "CensusPEP"
#> [14] "CensusACS1YearSurvey"
#> [15] "WikidataPopulation"Handling Multiple Data Sources
Data Commons aggregates data from multiple sources. This is both a strength (comprehensive data) and a challenge (which source to use?). Let’s handle this:
# Strategy 1: Use the official Census PEP (Population Estimates Program)
us_pop_clean <- us_population |>
filter(facet_name == "USCensusPEP_Annual_Population") |>
mutate(
year = as.integer(date),
population_millions = value / 1e6
) |>
filter(year >= 2000) |>
select(year, population_millions) |>
arrange(year)
# Visualize the clean data
ggplot(us_pop_clean, aes(x = year, y = population_millions)) +
geom_line(color = "steelblue", linewidth = 1.5) +
geom_point(color = "steelblue", size = 3) +
scale_y_continuous(
labels = label_number(suffix = "M"),
limits = c(280, 350)
) +
scale_x_continuous(breaks = seq(2000, 2025, 5)) +
labs(
title = "U.S. Population Growth",
subtitle = "Using Census Population Estimates Program data",
x = "Year",
y = "Population (millions)",
caption = "Source: U.S. Census Bureau via Data Commons"
)
# Calculate average annual growth
growth_rate <- us_pop_clean |>
mutate(
annual_growth =
(population_millions / lag(population_millions) - 1) * 100
) |>
summarize(
avg_growth = mean(annual_growth, na.rm = TRUE),
pre_2020_growth = mean(annual_growth[year < 2020], na.rm = TRUE),
post_2020_growth = mean(annual_growth[year >= 2020], na.rm = TRUE)
)
message(
"Average annual growth rate: ", round(growth_rate$avg_growth, 2), "%\n",
"Pre-2020 growth rate: ", round(growth_rate$pre_2020_growth, 2), "%\n",
"Post-2020 growth rate: ", round(growth_rate$post_2020_growth, 2), "%"
)
# Note for statistics enthusiasts: We're using arithmetic mean for simplicity.
# For compound growth rates, geometric mean would be more precise, which you can
# calculate as geometric_mean_growth = (last_value/first_value)^(1/n_years) - 1.
# But given the small annual changes (~0.7%), the difference is negligible.Key insight: Notice the marked change in growth
patterns around 2020. While pre-2020 showed steady growth, the post-2020
period reveals a significant slowdown. This dramatic shift raises
important policy questions: How much is due to COVID-19 mortality versus
changes in immigration patterns or birth rates? The
datacommons package makes it easy to pull in additional
variables like death rates, immigration statistics, and birth rates to
investigate these questions further.
Alternative: Aggregate Across Sources
# Strategy 2: Take the median across all sources for each year
us_pop_aggregated <- us_population |>
filter(!is.na(value)) |>
mutate(year = as.integer(date)) |>
filter(year >= 2000) |>
group_by(year) |>
summarize(
population_millions = median(value) / 1e6,
n_sources = n_distinct(facet_name),
.groups = "drop"
)
# Show which years have the most sources
us_pop_aggregated |>
slice_max(n_sources, n = 5) |>
kable(caption = "Years with most data sources")| year | population_millions | n_sources |
|---|---|---|
| 2017 | 325.0538 | 12 |
| 2019 | 328.2395 | 12 |
| 2018 | 326.6875 | 11 |
| 2020 | 329.8250 | 11 |
| 2021 | 331.9714 | 10 |
| 2022 | 332.1845 | 10 |
| 2023 | 333.6512 | 10 |
