The U.S. PIAAC Skills Map is an interactive data tool that provides state- and county-level data on the literacy and numeracy proficiency of adults ages 16–74 in all 50 states, all 3,141 counties, and the District of Columbia. The Skills Map also includes state-level model-based estimates for six age groups and four education groups.
The Skills Map reports estimates of adult skills as
The Skills Map allows users to compare the estimates of any county to those of the state it is in or to compare the estimates of any state to national estimates. It also allows users to compare the estimates for any two counties or any two states.
To help users to better understand the results produced for each state or county, the Skills Map gives users access to selected demographic variables from the American Community Survey (ACS). The ACS variables provide context for the estimates by presenting basic demographic characteristics for the state or county, such as educational attainment, race/ethnicity, nativity, employment status, and poverty level.
Additionally, the set of model-based estimates in the Skills Map includes state-level estimates for six age groups and four education groups. The age groups are 16–24, 25–34, 35–44, 45–54, 55–64, and 65–74. The education groups are less than high school, high school diploma or GED, some college (no degree or attained associate’s degree), and bachelor’s degree or higher.
These state and county estimates are based on the combined PIAAC data collected in 2012, 2014, and 2017 and are produced using an advanced statistical method generally referred to as small area estimation (SAE). For more information about the SAE techniques used in the development of these estimates, please refer to the SAE methodology report.
For more detailed information on the state and county estimates, see the Frequently Asked Questions (FAQ) below.
For more information about technique and the development of the Skills Map estimates, see the State and County Estimation Methodology Report.
This report describes the statistical methodology used to produce estimates of average scores and high, middle, and low proficiency levels of adult skills for every state and county in the United States.
This report expands the previous report to include the statistical methodology used to produce estimates of average scores and high, middle, and low proficiency levels of adult skills on a state-level for six age groups (16-24, 25-34, 35-44, 45-54, 55-64, and 65-74) and four education groups (less than high school, high school diploma or GED, some college (no degree or attained associate’s degree), and bachelor’s degree or higher) in the United States.
The previously published PIAAC estimates were computed directly using PIAAC survey data. Those estimates are representative of the national household population or large subgroups of the national household population. Unlike the previously published PIAAC estimates, U.S. PIAAC state and county estimates are model based, using a statistical technique called small area estimation (SAE) to provide valid estimates for U.S. states and counties. SAE refers to a variety of methods to estimate information for subpopulations or smaller areas of interest. SAE uses survey data, in combination with correlated data at the small-area level from other sources, to model the estimates of interest. In this case, the small areas are states and counties in the United States and the estimates of interest are skills proficiencies of the population.
The model-dependent approach was used to produce estimates for states and counties, for which PIAAC sample sizes were too small for direct estimation. The models used the combined 2012/2014/2017 U.S. PIAAC data in conjunction with data from the U.S. Census Bureau's 2013–2017 American Community Survey (ACS) to produce reliable estimates. The estimates are thus predictions of how the adults in the whole state or county would have performed had they been administered the PIAAC assessment.
The U.S. PIAAC state and county estimates are not directly comparable to the previously published PIAAC estimates for the United States because of the following differences:
The PIAAC state and county estimates are likewise not directly comparable to previous PIAAC estimates in international reports published by the Organization for Economic Cooperation and Development (OECD) for the following reasons:
The U.S. PIAAC state- and county-level estimates are reported for both literacy and numeracy for the percentage of the population at or below Level 1 (low proficiency), at Level 2 (medium proficiency), and at or above Level 3 (high proficiency), as well as the average scale score for the population in a county or a state. The table below provides definitions for the proficiency levels.
|U.S. PIAAC Proficiency Measures||Literacy||Numeracy|
At or below Level 1
|Adults at this level can be considered at risk for difficulties using or comprehending print material. Adults at the upper end of this level can read short texts, in print or online, and understand the meaning well enough to perform simple tasks, such as filling out a short form, but drawing inferences or combining multiple sources of text may be too difficult. Adults who are below Level 1 may only be able to understand very basic vocabulary or find very specific information on a familiar topic. Some adults below Level 1 may struggle even to do this and may be functionally illiterate.||Adults at this level can be considered at risk for difficulties with numeracy. Adults at the upper end of this level can understand how to add, subtract, multiply, and divide and can perform basic one-step mathematical operations with given values or common spatial representations (e.g., calculate how many bottles of soda are in a full box with two levels when only the top level can be seen). Adults who are below Level 1 may only be able to count, sort, and do basic arithmetic operations with simple whole numbers and may be functionally innumerate.|
|Adults at this level can be considered nearing proficiency but still struggling to perform tasks with text-based information. Such adults may be able to read print and digital texts, relate multiple pieces of information within or across a couple of documents, compare and contrast, and draw simple inferences. They can navigate in a digital environment to access key information, such as finding two main benefits of one product over another. However, more complex inferencing and evaluation may be too difficult.||Adults at this level can be considered nearing proficiency but still struggling to perform numeracy tasks. Such adults can successfully perform tasks requiring two or three steps involving calculations with whole numbers and common decimals, percentages, and fractions. They can conduct simple measurement and interpret relatively simple data and statistics in texts, tables, and graphs. However, more complicated problem solving (where the information is not explicit or is in an unfamiliar context) may be too difficult.|
At or above Level 3
276 points or more
|Adults at this level can be considered proficient at working with information and ideas in texts. Their higher literacy skills range from the ability to understand, interpret, and synthesize information across multiple, complex texts to the ability to evaluate the reliability of sources and infer sophisticated meanings and complex ideas from written sources.||Adults at this level can be considered proficient at working with mathematical information and ideas. Their higher numeracy skills range from the ability to recognize mathematical relationships and apply proportions to the ability to understand abstract representations of mathematical concepts and engage in complex reasoning about quantities and data.|
The U.S. PIAAC state and county estimates are based on area-level hierarchical Bayes linear threefold models. A bivariate model was fit for the percentages at or below Level 1 and at or above Level 3, which were then used to derive the percentage at Level 2. A univariate model was fit for the average. Separate models were produced for literacy and numeracy. All models include three levels of random effects: county, state, and census divisions. Each model used the following seven county-level covariates:
The source for these covariates was the Census Bureau's 2013–2017 American Community Survey data. Further details are provided in the PIAAC State and County Estimation Methodology Report.
The U.S. PIAAC state and county estimates are not measuring the same population as the previous estimates for PIAAC countries that are reported by the OECD. Specifically, the U.S. PIAAC state and county estimates (1) represent adults ages 16 to 74, whereas the OECD's estimates for participating countries represent adults ages 16 to 65; and (2) include “literacy-related non-respondents” (i.e., adults whose English language skills were too low to participate in the study), whereas the OECD's estimates for countries exclude this group.
A number of federal statistical agencies have developed small area estimation small area estimation (SAE) methods and have published SAE estimates. For example, the Census Bureau's Small Area Income and Poverty Estimates (SAIPE) provides annual estimates of income and poverty for states, counties, and school districts. Indirect estimates are also produced for the National Survey on Drug Use and Health. Other examples can be found in Czajka, Sukasih, and Maccarone (2014). For more information on the PIAAC state and county estimates, please see the PIAAC State and County Estimation Methodology Report.
The PIAAC small area estimation (SAE) program employed an extensive model evaluation process, as models are never a perfect fit to the data and systematic errors can manifest themselves. It was especially important for PIAAC to conduct a thorough evaluation of the model development because over 90 percent of the county estimates rely solely on the model predictions. A full range of model diagnostics, sensitivity analyses, and evaluation were conducted. For example, the model predictions were compared to direct estimates. Model predictions were generally close to direct estimates for counties with the largest PIAAC sample sizes, as one would expect for valid models. In addition, the modeling approach improved on the associated statistical error; for instance, the standard error of model predictions was substantially smaller than the standard error of direct estimates. The evaluation results confirmed the validity of the U.S. PIAAC state and county estimates.
Overall, the state-level estimates are more precise than the county-level estimates, and, to a lesser extent, the estimates are more precise in counties and states that had persons sampled in the PIAAC household sample than in those that had no persons sampled.
The coefficient of variation (CV) is a common way of measuring precision. The county-level estimates for the percentage of adults at or below Level 1 in literacy have a median CV of 10 percent. Estimates with CVs of this magnitude are considered to be precise (i.e., at a high confidence level). Meanwhile, there are a small number of county estimates that have CVs larger than 50 percent, which are imprecise and indicated as low confidence estimates in the U.S. PIAAC Skills Map. The state predictions are more precise, with a median CV of 8.1 percent.
Another way of measuring precision is to report credible intervals. For example, for the percentage of adults at or below Level 1 in literacy, the median credible interval width for county estimates is 8.0 percentage points, while for the state estimates the median is 6.1 percentage points. The median credible interval width is 7.2 percentage points for county estimates that had persons in the PIAAC 2012/2014/2017 household sample and 8.0 percentage points for county estimates that had no persons in the PIAAC household sample.
The accuracy level for the other five types of percentage estimates (i.e., the percentage at Level 2, the percentage at or above Level 3 in literacy, and the three percentages in numeracy) is similar to that for the percentage at or below Level 1 in literacy, with slightly larger credible intervals and slightly smaller CVs.
Yes. The estimates are available for download in the U.S. PIAAC Skills Map. See the last link on the left-hand side of the Skills Map (click the button “Download Data” under the bigger button “Compare Counties”). To access the Skills Map, please visit https://nces.ed.gov/surveys/piaac/skillsmap. Access the Guidance for Using State and County Estimates of Adult Skills for how to use the data from the PIAAC Skills Map.
The PIAAC state and county estimates can be used to estimate the total population at each proficiency level, and to estimate percentages or averages for groups of counties or states. Beyond reporting such estimates, the PIAAC state and county estimates allow interested researchers to conduct various statistical analyses that combine the model-based skills estimates with data from other sources at the state or county level. However, it is important to consider the uncertainty associated with the PIAAC state and county estimates in analyses. If not accounted for, then analysis results may be misleading due to overly precise results, and one may arrive at statistical significance when there is none. In the user’s guide called PIAAC SAE: Guidance on Use of State and County Estimates, five guidelines provide examples and discussions to guide data users through these analyses. Three of the guidelines are for reporting: (1) reporting model-based skills estimates; (2) estimating a skills population total using model-based estimates; and (3) estimating skills percentages or averages for groups of counties or states using model-based skills estimates. Guidelines 4 and 5 extend beyond the reporting of the PIAAC model-based skills estimates to dive into research analyses: (4) relating the model-based estimates to data from external sources; and (5) using contextual variables to describe groups of counties or states.
The U.S. PIAAC Skills Map can make comparisons of two areas at a time. For example, a county can be compared to another county or to the state it is in, or a state could be compared to another state or to the nation as a whole. For a given pair of areas—(county1, county2) or (county1, state1) or (state1, state2) or (state1, nation)—the hypothesis testing of the difference between the two areas being equal to zero is performed for all eight measures (percentages of the area's populations at or below Level 1, at Level 2, and at or above Level 3, as well as average scores in both literacy and numeracy domains).
The results are reported as “statistically” different when differences are significant at the critical significance level that was adjusted using the Bonferroni method, an adjustment to account for the additional statistical error that occurs when multiple comparisons are conducted simultaneously. The results are reported as “notably” different when differences that are not significant under the Bonferroni-adjusted critical significance level are significant under a single test at the α (type 1 error) = .05 level.
Reporting on PS-TRE skills is problematic at the county level because the percentage of people who completed PIAAC on a computer is likely to vary widely across counties. It could be misleading to compare average PS-TRE scores or the percentage of the population at a certain PS-TRE proficiency level, because they may represent different percentages of the population as well as different populations (e.g., just the most computer savvy in one county versus a cross-section of computer users in another). Without knowing whether 90 percent or 15 percent of a county's population completed the assessment on a computer, a score of, say, 500 is both not meaningful and misleading. While respondents were encouraged to use the computer for the assessment if they were able, the percentage using the computer could be influenced by personal preference as well. Therefore, the meaning of the percentage at each proficiency level is also influenced by the percentage of respondents who used the computer to complete the assessment. In addition, complete covariance information is not available for respondents who did not use the computer for the assessment. The estimates of PS-TRE performance also have large standard errors that depend on estimates of computer use per county, which also have large standard errors that need to be factored in. For these reasons, it was determined that PS-TRE was not a solid domain for reporting state- and county-level estimates.
The proficiency assessment instruments and scales used in the 2003 National Assessment of Adult Literacy (NAAL) and the 1992 National Adult Literacy Survey (NALS) were different from those used in PIAAC, and thus the estimates for counties and states from NAAL and NALS small area estimation (SAE) are not comparable to the corresponding estimates from PIAAC. The table below summarizes other differences between NAAL and PIAAC state and county estimates.
|NAAL and NALS State and County Estimates||U.S. PIAAC State and County Estimates|
|Age range||16 years and older||16- to 74-year-olds|
|Estimates||Percentage of adults lacking basic prose literacy skills||For literacy and numeracy separately:
|Covariates used in SAE model||2003 NAAL model: