U.S. State and County Estimates Resources

Small area estimation methodology was used to produce the U.S. PIAAC model-based estimates for literacy and numeracy at the state and county levels, as well as for age and education groups within states. These estimates were then used to produce the U.S. PIAAC model-based estimates for age and education groups for counties, using an allocation method.

State and county estimates

The U.S. PIAAC state and county estimates are based on area-level hierarchical Bayes linear three-fold models. A bivariate model was used to estimate the percentages at or below Level 1 and at or above Level 3, which were then used to derive the percentage at Level 2. A univariate model was used to estimate the average. Separate models were produced for literacy and numeracy. All models included three levels of random effects: county, state, and census division. Each model used the following seven county-level covariates:

• percentage of population age 25 and over with less than high school education;
• percentage of population age 25 and over with more than high school education;
• percentage of population below 100 percent of the poverty line;
• percentage of Black or African American population;
• percentage of Hispanic population;
• percentage of civilian noninstitutionalized population who have no health insurance coverage; and
• percentage of population age 16 and over with service occupations.

The source for the model covariates was the Census Bureau's 2013–2017 American Community Survey data. Further details are provided in the PIAAC State and County Estimation Methodology Report for the state and county estimates.

State estimates for age and education groups

Area-level hierarchical Bayes linear univariate models with state-level random effects were used to produce the U.S. PIAAC state estimates for age and education groups. Separate models were produced for each of the six age groups (16-24, 25-34, 35-44, 45-54, 55-64 and 65-74) and four education groups (less than high school, high school diploma or GED, some college, and bachelor’s degree or higher), for the average scores and percentages at or below Level 1 and at or above Level 3, for literacy and numeracy. A total of 60 separate models were produced. A benchmarking adjustment was applied to align the aggregated, model-based estimates for groups to the state model-based estimates. The results were then combined to estimate the percentages at Level 2.

In the age group models, the model covariates included:

• percentage of population age 16 and over with service occupations;
• percentage of population with less than high school education;
• percentage of population with more than high school education; and
• percentage of population below 100 percent of the poverty line.

In the education group models, the model covariates included:

• percentage of civilian noninstitutionalized population who have no health insurance coverage;
• percentage of population below 100 percent of the poverty line; and
• percentage of population age 16 to 24.

The source for the model covariates was the Census Bureau's 2013–2017 American Community Survey data. Further details are provided in the PIAAC State-Level Estimation for Age and Education Methodology Report for the state estimates for age and education groups.

County estimates for age and education groups

The U.S. PIAAC county estimates for age and education groups are based on an allocation method. This method consists of using the county, state, and state by group estimates and allocating them to the county by group level via a deterministic model. The allocation method was applied separately to each of the six age groups (16-24, 25-34, 35-44, 45-54, 55-64 and 65-74) and four education groups (less than high school, high school diploma or GED, some college, and bachelor’s degree or higher), to the average scores and percentages at or below Level 1 and at or above Level 3, to literacy and numeracy. The results were then combined to estimate the percentages at Level 2. Further details are provided in the PIAAC County-Level Estimation for Age and Education Methodology Report.

The seven covariates used in the area-level hierarchical Bayes linear three-fold models to produce county- and state-level model estimates were selected from a large pool of county- and state-level variables that were collected or derived from large surveys and official statistics. The process of selecting covariates was conducted in two phases. In the first phase, the list of all the county- and state-level variables was considered as fixed effect and reduced through various methods (1) correlation matrix between the predictor variables and the outcome variables; (2) tree search algorithm for each outcome variable; and (3) least absolute shrinkage and selection operator (LASSO (Tibshirani 1996)) method for each outcome model. Once the list of covariates were reduced, the second phase evaluated the selected covariates using a cross-validation process adding the random effect estimation to arrive at the final list of covariates.

To produce the state-level estimates for age and education groups, the general strategy was to build on the previous research to the extent feasible, but also to modify the details where necessary. For example, the covariates used for the state and county estimates were modified to reflect age-specific or education-specific rates, rather than the original overall rate. For the educational groups, additional covariates include the percentage of population in the six age groups within the educational group. To avoid fitting models with a high number of parameters to a limited number of areas, several fixed-effect regression models weighted by the state sample size were suggested and fitted to determine the final set of model covariates through comparisons based on the Akaike information criterion (AIC) (Akaike 1974).

Sound research on reliable national PIAAC data have shown that several variables (e.g., gender, age, race/ethnicity, education attainment, employment status, nativity, health status) have interesting associations with proficiency scores. For example, the literacy skills increase through middle age adults and then tend to decrease in the older age groups, while the literacy skills tend to have a strong increasing relationship for education attainment. Availability of state and county estimates for age and education groups allows for further exploration of these associations and use of these estimates toward making effective evidence-based policies and laws relating to adult education and the workforce.

The U.S. PIAAC estimates are based on models which are in turn based on assumptions. During the model development process, the model assumptions and fitting algorithms were tested iteratively with the model specification. The predictive power of the nearly final model candidates was evaluated using cross-validation techniques. Upon finalization of the models structures, the resulting model-based estimates were compared against the survey direct estimates, when available, and they were in close agreement when the survey direct estimates were based on large sample sizes. In addition, model-based estimates were also compared across levels of aggregation and estimates at higher levels of aggregation were more precise than estimates at lower levels of aggregation, e.g., state versus county, or state versus state by group levels. When aggregated within state, the state by group model-based estimates matched exactly the state estimates, since they were explicitly benchmarked during the modeling process. Even if the county by group estimates could not be benchmarked to exact county estimates (due in part to challenges related to availability of consistent aggregation weights), they aggregated closely upon aggregation. Finally, county by group estimates were also compared against auxiliary data and similar relationships between them were observed for both county by group domains with sample data and county by group domains without sample data.

A number of federal statistical agencies have developed small area estimation small area estimation (SAE) methods and have published SAE estimates. For example, the Census Bureau's Small Area Income and Poverty Estimates (SAIPE) provides annual estimates of income and poverty for states, counties, and school districts. Indirect estimates are also produced for the National Survey on Drug Use and Health. Other examples can be found in Czajka, Sukasih, and Maccarone (2014). For more information on the PIAAC state and county estimates, please see the PIAAC State and County Estimation Methodology Report.

The previously published PIAAC estimates were computed directly using PIAAC survey data. Those estimates are representative of the national household population or large subgroups of the national household population. Unlike the previously published PIAAC estimates, U.S. PIAAC state and county estimates are model based, using a statistical technique called small area estimation (SAE) to provide valid estimates for U.S. states and counties. SAE refers to a variety of methods to estimate information for subpopulations or smaller areas of interest. SAE uses survey data, in combination with correlated data at the small-area level from other sources, to model the estimates of interest. In this case, the small areas are states and counties in the United States and the estimates of interest are skills proficiencies of the population.

The model-dependent approach was used to produce estimates for states and counties, for which PIAAC sample sizes were too small for direct estimation. The models used the combined 2012/2014/2017 U.S. PIAAC data in conjunction with data from the U.S. Census Bureau's 2013–2017 American Community Survey (ACS) to produce reliable estimates. The estimates are thus predictions of how the adults in the whole state or county would have performed had they been administered the PIAAC assessment.

The U.S. PIAAC state and county estimates are not directly comparable to the previously published PIAAC estimates for the United States because of the following differences:

1. The U.S. PIAAC state and county data are from the combined 2012/2014/2017 data collections whereas the previously published PIAAC estimates are for either 2012/14 or 2017.
2. The average scores and proficiency levels include literacy-related nonrespondents (adults whose English language skills were too low to participate in the study) whereas the previous PIAAC estimates exclude them. The U.S. PIAAC state and county estimates assume this population as performing at or below Level 1 for proficiency levels.
3. The national-level estimates in the PIAAC Skills Map are weighted aggregates (based on the population of the counties) of the county-level estimates.

The PIAAC state and county estimates are likewise not directly comparable to previous PIAAC estimates in international reports published by the Organization for Economic Cooperation and Development (OECD) for the following reasons:

1. The target population for U.S. reporting was 16- to 74-year-olds whereas the OECD published results are for 16- to 65-year-olds.
2. The U.S. average scores and proficiency levels include literacy-related nonrespondents whereas the OECD international reporting exclude them.

The U.S. PIAAC state and county estimates are not measuring the same population as the previous estimates for PIAAC countries that are reported by the OECD. Specifically, the U.S. PIAAC state and county estimates (1) represent adults ages 16 to 74, whereas the OECD's estimates for participating countries represent adults ages 16 to 65; and (2) include “literacy-related non-respondents” (i.e., adults whose English language skills were too low to participate in the study), whereas the OECD's estimates for countries exclude this group.

The proficiency assessment instruments and scales used in the 2003 National Assessment of Adult Literacy (NAAL) and the 1992 National Adult Literacy Survey (NALS) were different from those used in PIAAC, and thus the estimates for counties and states from NAAL and NALS small area estimation (SAE) are not comparable to the corresponding estimates from PIAAC. The table below summarizes other differences between NAAL and PIAAC state and county estimates.

	NAAL and NALS State and County Estimates	U.S. PIAAC State and County Estimates
Age range	16 years and older	16- to 74-year-olds
Estimates	Percentage of adults lacking basic prose literacy skills	For literacy and numeracy separately: Percentage of the population at or below Level 1 Percentage of the population at Level 2 Percentage of the population at or above Level 3 Average scale score
Covariates used in SAE model	2003 NAAL model: Percentage of population who were foreign born and had stayed in the United States for 20 years or less Percentage of population age 25 and older with a high school education or less Percentage of population who Black or African American or were who were Hispanic Percentage of population in households with incomes below 150% of poverty level Indicator variable identifying the New England and North Central census divisions Indicator variable identifying the NAAL states 1992 NALS model: Percentage of population for whom English was not a native language Percentage of population age 25 and older with only a high school education or less Percentage of population who were Black or African American Percentage of population who were Hispanic Indicator variables identifying the New England and North Central census divisions Indicator variable for counties in a NALS state	PIAAC model: Percentage of population age 25 and older with less than a high school education Percentage of population age 25 and older with more than a high school education Percentage of population below 100% of the poverty line Percentage of population who were Black or African American Percentage of population who were Hispanic Percentage of civilian noninstitutionalized population who had no health insurance coverage Percentage of population age 16 and older with service occupations

The U.S. PIAAC state- and county-level estimates are reported for both literacy and numeracy for the percentage of the population at or below Level 1 (low proficiency), at Level 2 (medium proficiency), and at or above Level 3 (high proficiency), as well as the average scale score for the population in a county or a state. The table below provides definitions for the proficiency levels.

U.S. PIAAC Proficiency Measures	Literacy	Numeracy
At or below Level 1 0–225 points	Adults at this level can be considered at risk for difficulties using or comprehending print material. Adults at the upper end of this level can read short texts, in print or online, and understand the meaning well enough to perform simple tasks, such as filling out a short form, but drawing inferences or combining multiple sources of text may be too difficult. Adults who are below Level 1 may only be able to understand very basic vocabulary or find very specific information on a familiar topic. Some adults below Level 1 may struggle even to do this and may be functionally illiterate.	Adults at this level can be considered at risk for difficulties with numeracy. Adults at the upper end of this level can understand how to add, subtract, multiply, and divide and can perform basic one-step mathematical operations with given values or common spatial representations (e.g., calculate how many bottles of soda are in a full box with two levels when only the top level can be seen). Adults who are below Level 1 may only be able to count, sort, and do basic arithmetic operations with simple whole numbers and may be functionally innumerate.
Level 2 226–275 points	Adults at this level can be considered nearing proficiency but still struggling to perform tasks with text-based information. Such adults may be able to read print and digital texts, relate multiple pieces of information within or across a couple of documents, compare and contrast, and draw simple inferences. They can navigate in a digital environment to access key information, such as finding two main benefits of one product over another. However, more complex inferencing and evaluation may be too difficult.	Adults at this level can be considered nearing proficiency but still struggling to perform numeracy tasks. Such adults can successfully perform tasks requiring two or three steps involving calculations with whole numbers and common decimals, percentages, and fractions. They can conduct simple measurement and interpret relatively simple data and statistics in texts, tables, and graphs. However, more complicated problem solving (where the information is not explicit or is in an unfamiliar context) may be too difficult.
At or above Level 3 276 points or more	Adults at this level can be considered proficient at working with information and ideas in texts. Their higher literacy skills range from the ability to understand, interpret, and synthesize information across multiple, complex texts to the ability to evaluate the reliability of sources and infer sophisticated meanings and complex ideas from written sources.	Adults at this level can be considered proficient at working with mathematical information and ideas. Their higher numeracy skills range from the ability to recognize mathematical relationships and apply proportions to the ability to understand abstract representations of mathematical concepts and engage in complex reasoning about quantities and data.

Coefficients of variation (CV) and credible intervals are common ways of measuring precision. The coefficient of variation is the ratio of the standard error to the value of the estimate. The credible interval is the range of the upper and lower point of the estimate in which there is a 95 percent chance that the true value of the estimate could be found. The length of the credible interval corresponds to the precision of the estimate: narrow credible intervals correspond to estimates of high precision and wide credible intervals correspond to estimates of low precision. In general, one can have high confidence in an estimate when the CV is less than 30 percent, moderate confidence where the CV is between 30 percent and 50 percent, and low confidence when the CV is greater than 50 percent.

Although not always, in general, estimates at higher levels of aggregation are more precise than estimates at lower levels of aggregation. That is, state estimates are more precise than either county, county by group, or state by group estimates; state by group estimates are more precise than county by group estimates; and county estimates are more precise than county by group estimates. Also, estimates for areas with persons in the PIAAC 2012/2014/2017 household sample tend to be more precise than the estimates for areas without persons in the PIAAC sample. Furthermore, the estimates of average scores are much more precise than the percentage estimates in general.

• The county-level estimates for the percentage of adults at or below Level 1 in literacy have a median CV of 10 percent. Estimates with CVs of this magnitude are considered to be precise (i.e., at a high confidence level). Meanwhile, there are a small number of county estimates that have CVs larger than 50 percent, which are imprecise and indicated as low confidence estimates in the U.S. PIAAC Skills Map. The state predictions are more precise, with a median CV of 8.1 percent. For state estimates for age and education groups, the median (computed for each of the 10 groups across states) CV ranges from 11.7 percentage points to 32.6 percentage points. For county estimates for age and education groups, the median (computed for each of the 10 groups across counties) CV ranges from 12.7 percentage points to 28.3 percentage points.
• For the percentage of adults at or below Level 1 in literacy, the median credible interval width for county estimates is 8.0 percentage points, while for the state estimates the median is 6.1 percentage points. For state estimates for age and education groups, the median (computed for each group across states) credible interval width ranges from 4.8 percentage points to 21.1 percentage points. For county estimates for age and education groups, the median (computed for each of the 10 groups across counties) credible interval width ranges from 5.0 percentage points to 20.7 percentage points. Note that these county and state ranges for the median credible interval widths (computed over the geography and within the age/education group) are not comparable across geography. For example, the county with the median credible interval width of 20.7 percentage points isn’t necessarily nested in the state with the median credible interval width of 21.1 percentage points, or the group for which the county credible interval width is 20.7 percentage points isn't necessarily the same as the group for which the state credible interval width is 21.1 percentage points.
• The accuracy level for the other five types of percentage estimates (i.e., the percentage at Level 2, the percentage at or above Level 3 in literacy, and the three percentages in numeracy) is at a comparable level to that for the percentage at or below Level 1 in literacy, for the county, county by group, state, and state by group estimates.

The U.S. PIAAC Skills Map can make comparisons of two areas at a time. For example, a county can be compared to another county or to the state it is in, or a state could be compared to another state or to the nation as a whole. For a given pair of areas — (county1, county2) or (county1, state1) or (state1, state2) or (state1, nation) — the hypothesis testing of the difference between the two areas being equal to zero is performed for all eight measures (percentages of the area's populations at or below Level 1, at Level 2, and at or above Level 3, as well as average scores in both literacy and numeracy domains). In the same manner for a selected age or education group, comparisons can be made between pairs of states, between counties within the same state, between a county and a state, or between a state and the nation. Note that comparisons between groups within a county or a state are not supported.

The results are reported as “statistically” different when differences are significant at the critical significance level that was adjusted using the Bonferroni method, an adjustment to account for the additional statistical error that occurs when multiple comparisons are conducted simultaneously. The results are reported as “notably” different when differences that are not significant under the Bonferroni-adjusted critical significance level are significant under a single test at the α (type 1 error) = .05 level.

Reporting on PS-TRE skills is problematic at the county level because the percentage of people who completed PIAAC on a computer is likely to vary widely across counties. It could be misleading to compare average PS-TRE scores or the percentage of the population at a certain PS-TRE proficiency level, because they may represent different percentages of the population as well as different populations (e.g., just the most computer savvy in one county versus a cross-section of computer users in another). Without knowing whether 90 percent or 15 percent of a county's population completed the assessment on a computer, a score of, say, 500 is both not meaningful and misleading. While respondents were encouraged to use the computer for the assessment if they were able, the percentage using the computer could be influenced by personal preference as well. Therefore, the meaning of the percentage at each proficiency level is also influenced by the percentage of respondents who used the computer to complete the assessment. In addition, complete covariance information is not available for respondents who did not use the computer for the assessment. The estimates of PS-TRE performance also have large standard errors that depend on estimates of computer use per county, which also have large standard errors that need to be factored in. For these reasons, it was determined that PS-TRE was not a solid domain for reporting state- and county-level estimates.

Yes. The estimates are available for download in the U.S. PIAAC Skills Map. See the last link on the left-hand side of the Skills Map (click the button “Download Data” under the bigger button “Compare Counties”). To access the Skills Map, please visit https://nces.ed.gov/surveys/piaac/skillsmap. Access the Guidance for Using State and County Estimates of Adult Skills for how to use the data from the PIAAC Skills Map.

The PIAAC state and county estimates can be used to estimate the total population at each proficiency level, and to estimate percentages or averages for groups of counties or states. Beyond reporting such estimates, the PIAAC state and county estimates allow interested researchers to conduct various statistical analyses that combine the model-based skills estimates with data from other sources at the state or county level. However, it is important to consider the uncertainty associated with the PIAAC state and county estimates in analyses. If not accounted for, then analysis results may be misleading due to overly precise results, and one may arrive at statistical significance when there is none. Guidance for Using State and County Estimates of Adult Skills provides examples and discussions to guide data users through these analyses. The released model estimates for age and education groups cannot be crossed to arrive at estimates for combinations of age groups and education groups. To do so one would require a methodology that relies on strong assumptions, and the model estimates would rely much less on PIAAC data for those groups due to a small number PIAAC sample cases (if any), and much more on models.