​​​​​NAEP Technical DocumentationNew-School Sampling Frame for the 2019 Fourth- and Eighth-Grade Public School National Science Assessment

The primary sampling frames for the 2019 fourth- and eighth-grade public school samples for the national assessment in science were constructed using the most current Common Core of Data (CCD)  file available from NCES. This file contained schools that were in existence during the 2016-2017 school year (i.e., it was two years out of date). During the subsequent 2-year period, undoubtedly some schools closed, some changed structure (one school becoming two schools, for example), some newly opened, and still others changed their grade span.

A supplemental sample was selected from a list of schools that were new or had become newly eligible sometime after the 2016-2017 school year. The goal was to allow every new school a chance of selection, thereby fully covering the target population of schools in operation during the 2018-2019 school year. It was infeasible to ask every school district in the United States to provide a supplemental school frame, so a two-stage procedure was employed. First, a sample of school districts was selected within each state. Then each State or Trial Urban District Assessment (TUDA) Coordinator was sent a list of the schools within their sampled districts that had been present on the 2016-2017 CCD file. The Coordinators were asked to add in any new schools and identify any schools on this list that had become newly eligible for grades 4, 8, or 12.

The new-school process began with the preparation of a district-level frame. The starting point was a file containing every public school district in the United States.

Specific districts were designated as in sample with certainty. They included the following districts:

• districts in jurisdictions where all schools were selected for sample at either grade 4 or 8;

• state-operated districts;

• districts in states with fewer than 10 districts;

• charter-only districts (that is, districts containing no schools other than charter schools); and

• TUDA districts.

Then noncertainty districts were classified as small, medium, or large based on the number of schools and student enrollment of schools from the CCD-based public school frame.

A district was considered to be small if it contained no more than one school at each targeted grade (4, 8, and 12). During school recruitment, the coordinators were asked to identify schools within their district that newly offered the targeted grade. Every identified new school was added to the sample. From a sampling perspective, the new school was viewed as an “annex” to the sampled school which meant that it had a well-defined probability of selection equal to that of the sampled school. When a school in a small district was sampled from the CCD-based frame, its associated new school was automatically sampled as well.

Within each jurisdiction, districts that were neither certainty selections nor small were divided into two strata, one containing large-size districts and a second containing medium-size districts. These strata were defined by computing the percentage of jurisdiction grade 4, 8, and 12 enrollment represented by each district, sorting in descending order, and cumulating the percentages. All districts up to and including the first district at or above the 80th cumulative percentage were defined as large districts. The remaining districts were defined as medium districts.

A simplified example is given below. The state's districts are ordered by descending percentage enrollment. The first six become large districts and the last six become medium districts.

The target sample size for each jurisdiction was 10 districts total across the medium-size and large-size district strata. Where possible, eight districts were selected from the large-size district stratum and two districts from the medium-size district stratum. However, in the example above, since there are only six large districts, all of the districts in the large district stratum and four districts from the medium district stratum would have been selected for the new-school inquiry.

If sampling was needed in the medium-size district stratum, districts in this stratum were selected with equal probability. If sampling was needed in the large-size district stratum, the districts in this stratum were sampled with probability proportional to enrollment. These probabilities were retained and used in later stages of sampling and weighting of new schools..

The selected districts in each jurisdiction were then sent a listing of all their schools that appeared on the 2016-2017 CCD file and were asked to provide information about the new schools not included in the file and grade span changes of existing schools. These listings provided by the selected districts were used as sampling frames for selection of new public schools and updates of existing schools. This process was conducted through the NAEP State or TUDA Coordinator in each jurisdiction. The Coordinators were sent the information for all sampled districts in their respective states and were responsible for returning the completed updates.

The following table presents the number and percentage of schools and average estimated grade enrollment for the fourth- and eighth-grade new-school frame by census region.

Printer-friendly Version