​​​NAEP Technical DocumentationStratification of Schools for the 2019 Twelfth-Grade Public School National Assessment

The purpose of school stratification is to increase the efficiency and ensure the representativeness of school samples in terms of important school-level characteristics, such as geography (e.g., census division), urbanicity, and race/ethnicity composition. NAEP school sampling utilizes two types of stratification: explicit and implicit.

​

Explicit stratification partitions the sampling frame into mutually exclusive groupings called strata. The systematic samples selected from these strata are independent, meaning that each sample is selected with its own unique random start.

Implicit stratification involves sorting the sampling frame, as opposed to grouping the frame. For NAEP, schools are sorted in serpentine fashion by key school characteristics within sampling strata and sampled systematically using this ordering. This type of stratification ensures the representativeness of the school samples with respect to the key school characteristics.

The sampling of public schools for the grade 12 assessments in mathematics, reading, and science did not involve any explicit stratification, but it involved six dimensions of implicit stratification. The frames were hierarchically sorted by the following in the order shown to create the implicit strata:


AIAN Composition
​

For the twelfth-grade mathematics, reading, and science assessments in the national public school sample, we created implied strata by first classifying schools on the sampling frame  as either low AIAN or high AIAN based on the percentage of AIAN students in the targeted grade (the cutoff was 5 percent AIAN students). This is the first time AIAN classification was used to implicitly stratify the national NAEP public school sampling frames. It is part of an oversampling scheme to ensure sufficient numbers of AIAN students are present in the student samples. Grouping high AIAN schools together in a sampling stratum helps bring schools with relatively large numbers of AIAN students into the school sample. In turn, schools with more AIAN students improve the chance that sufficient numbers of AIAN students are included in the student samples.

Census Division

Within each of the low and high AIAN classifications, schools were further classified into groups based on census division. A census division-based grouping can consist of a single census division, a set of neighboring census divisions, or a part of an individual census division. When census divisions are combined to form implied sampling strata, it is done generally within census regions. Because there are so few high AIAN schools, the census division grouping within the high AIAN stratum consisted of several neighboring census divisions.

Within the low AIAN stratum, each census division, except the Pacific Census Division, constituted a separate census division grouping. The Pacific Census Division was split into two parts: California in one part and Alaska, Hawaii, Oregon, and Washington in the other part. This was done purposely so that California could use achievement data as the last stratification variable instead of median income. See last paragraph for more detail.

Urbanicity Status

The urbanicity classification strata were derived from the NCES urban-centric locale variable from the Common Core of Data (CCD), which classifies schools based on location ([1] city, [2] suburb, [3] town, [4] rural) and proximity to urbanized areas. Urban-centric locale has 12 possible values.

The urbanicity classification cells were created by starting with the original 12 NCES urban-centric locale categories within each AIAN classification-by-census division grouping. Any cell with an expected school sample size less than four was combined with a neighboring cell within the same census division grouping. Collapsing was first done among the subcategories within a location class. (For example, the subcategories for location class city are (1) large, (2) mid-size, and (3) small. If one of these subcategories was deficient then either 1 was collapsed with 2; 3 collapsed with 2; or 2 collapsed with the smaller of 1 or 3.) If the collapsed cell was still too small, all three subcategories within a location class were combined.

If a collapsed location class still had an expected school sample size less than four, then it was collapsed with a neighboring collapsed location class. That is, 1 would be collapsed with 2 or 3 would be collapsed with 4. If additional collapsing was necessary, all location classes were combined. No collapsing across census division strata was allowed or necessary.

The result of this was a set of sampling strata defined by AIAN classification, census division strata, and urbanicity classification having expected school sample sizes of at least four schools.

No further implicit strata for High AIAN schools were formed beyond urbanization classification.

Black/Hispanic Composition

Low AIAN schools within the nested urbanicity classification strata were further stratified into Black/Hispanic classification strata. The first division was the classification of schools as either low Black/Hispanic schools or high Black/Hispanic schools based on the percentage of Black or Hispanic students in the target grade (the cutoff was 15 percent Black and Hispanic students). Within the high Black/Hispanic classification, the number of substrata was based on the expected school sample size.

  • If the expected school sample size of resultant strata was less than or equal to 8.0, then this was the final urbanicity-Black/Hispanic stratum;
  • if the expected sample size was greater than or equal to 8.0 and less than 12.0, there were two substrata;
  • if the expected sample size was greater than or equal to 12.0 and less than 16.0, there were three substrata; and
  • if the expected sample size was greater than or equal to 16.0, there were four substrata.

The substrata were defined by percentage of Black and Hispanic students, with the cutoffs for substrata defined by weighted percentiles (with the weight equal to expected hits for each school).

  • For two substrata, the cutoff was the weighted median;
  • for three substrata, the weighted 33rd and 67th percentiles; and
  • for four substrata, the weighted median and quartiles.

For the low Black/Hispanic classification, there were six urbanicity strata that had a large enough expected school sample size, and these were split into groups of states. Two or three state groups were formed using adjacent states if possible, while maintaining an expected school sample size of at least four for each state group for each of these six urbanicity strata.

School Type​​​

The next implicit stratification variable was school type. School type takes on values of public, BIE, and DoDEA.

Median Income/Achievement 


The last implicit stratification variable was median income of the ZIP code area containing the school, except in California, where student achievement data was used. Schools in California contain more than 12 percent of the grade 12 students in the nation. Using achievement data provides a benefit. Achievement is a better sort variable than median income when ordering schools within a state because it is direct measure of student performance. However, when ordering schools across state, median income is better than achievement because states generally use different achievement measures while median income is a standard measure across states. 


Last updated 18 December 2023 (PG)