​NAEP Technical DocumentationNew-School Sampling Frame for the 2020 Public School Long-Term Trend Assessment

The most current Common Core of Data (CCD) file available was used to construct the public school frame for NAEP 2020. However, the information on that file was two years out of date by the time of the NAEP assessment. During that two-year period, some schools closed, others changed grade span, and still others came into existence.

One can improve coverage by asking districts to provide information on currently open schools that were not listed in the 2017–2018 CCD file used to create the NAEP public school frame, and also to report grade span changes that may have caused a CCD-listed school to become newly eligible by age population (9 or 13). Asking all districts to do this would have imposed an undue burden, so instead, a random sample of districts was contacted to obtain lists of new and newly eligible schools. The goal was to allow every new or newly eligible school a chance of selection, thereby fully covering the target population of schools in operation during the 2019–2020 school year.

The first step in this process was the development of a new-school frame through the construction of a district-level file from the CCD school-level file. Since the long-term trend assessment was to be conducted within a total of 105 primary sampling units (PSUs), only districts that fell within the boundaries of those PSUs were eligible for sampling. Once the district-level file was subset to just the targeted PSUs, it was divided into three files: the first containing state-operated and charter school districts, the second containing small districts, and the third containing large districts.

State-operated districts and districts containing no schools other than charter schools require special handling. In survey years when state-level assessments are conducted, NAEP State Coordinators are asked to provide the names of all new charter-only and state-run schools. However, these types of school districts tend not to be geographically compact, and it is not feasible to link such a district to a single PSU, except at the individual school level. The smaller the proportion of a state’s population falling within sampled PSUs, the less likely that a specific new school of this type will be added to the frame and the more likely that state personnel will have expended unnecessary effort in providing updated information that will not be used. For this reason, for the NAEP 2020 assessment, the charter-only and state-run district component of the new school procedure was implemented only in states where more than 60 percent of youth fell within sampled PSUs. This meant that this component of the new-school sampling frame procedure was implemented in 13 states plus the District of Columbia, which taken together contain about 47 percent of the nation's youth.

The remaining districts were classified as small or large. A small district usually contains no more than three schools on the frame in total, with no more than one school at each targeted grade (fourth, eighth, and twelfth). However, for NAEP 2020 new schools were selected for age-based assessments. Therefore, for NAEP 2020, a small district contains no more than one school with any of grades 2-12 on the frame in total. New schools in small districts were identified during school recruitment and added to the sample if the frame school in the same district was sampled. From a sampling perspective, the new school was viewed as an “annex” to the sampled school that had a well-defined probability of selection equal to that of the frame school. Thus when the frame school was sampled in a small district for age 9 (for example), any age-eligible new school was automatically sampled for age 9 (for example) as well.

Large districts were divided into 77 strata based on the NAEP 2020 PSU sampling strata, with districts in certainty PSUs grouped together in a single stratum. The district sample was allocated to each of the 77 strata proportional to the percent of the age-eligible U.S. population contained in that stratum, with the caveat that each stratum had to be allocated at least one district. This allocation was then adjusted because it resulted in too many districts in the certainty strata and not enough in the noncertainty strata. Once the allocation to each stratum had been fixed, districts were sampled from a sorted list using systematic sampling with probability proportional to size and a random start, with the measure of size being the number of age-eligible students enrolled in the district. Within the certainty PSU stratum, districts were sorted in a serpentine manner by state and measure of size prior to sampling. In all other strata the districts were sorted by measure of size alone. District selection probabilities were retained and used in all subsequent stages of sampling and weighting.

The selected districts were then sent a listing of all their schools that appeared on the 2017–2018 CCD file and were asked to provide information about any schools missing from CCD, and grade span changes of existing schools. This information provided by the sampled districts was used to construct sampling frames for the selection of new or newly eligible public schools and also for updating the status of existing schools (e.g., school closings). This process was conducted through the NAEP State Coordinator in each jurisdiction. The coordinators were sent the information for all sampled districts in their respective states and were responsible for returning the completed updates.

The eligibility of a school was determined based on the grade span and whether it was located in a sampled PSU. A school was also classified as "newly eligible" if a change of grade span had occurred such that the school status changed from ineligible to eligible for a particular age population.

This process yielded 210 schools on the age 9 new school sampling frame and 313 new schools on the age 13 new school sampling frame. The schools on the age 9 frame contained an estimated 6,623 nine year olds, and the schools on the age 13 frame contained an estimated 6,713 thirteen year olds. 

Printer-friendly Version