The schools and students participating in NAEP assessments are selected to be representative of the target populations for which results are reported. While national and regional results reflect the performance of students in public schools, Bureau of Indian Education (BIE) schools, Department of Defense schools, and private schools, state-level results reflect the performance of public and BIE school students only. For comparison purposes within the state results section, the national sample is composed of public and BIE school students only.
The samples of American Indian/Alaska Native (AI/AN) students participating in the 2009 NAEP reading and mathematics assessments represent augmentations of the sample of AI/AN students who would usually be selected by NAEP. This allows more detailed reporting of performance for this group. AI/AN students are one of the six mutually exclusive race/ethnicity categories reported by NAEP. The other five are White (non-Hispanic), Black (non-Hispanic), Hispanic, Asian/Pacific Islander, and Unclassified. Unclassified students are those whose school-reported race/ethnicity was “other” (i.e., two or more races), unavailable, or was missing. NAEP race/ethnicity categories are based on official school records, as reported by participating schools at the time of data collection.
Prior to 2005, BIE schools were identified as part of the national sample, and the resulting number of participating schools was usually small, fewer than five per grade. In 2005, BIE schools were sampled as a part of each state sample, at the same rate as public schools in a given state. That means, roughly speaking, that a BIE student had the same probability of selection as a public school student in the same state. As a result, about 30 BIE schools were included per grade, thereby increasing the number of AI/AN students in the sample. In 2007 and 2009, there were even larger samples of BIE schools than in 2005; all BIE schools and students were included in the 2007 and 2009 samples. The BIE population represents approximately 130 schools at grade 4 and 110 schools at grade 8. In terms of the number of students, the BIE population represents approximately 2,900 students at grade 4 and 2,500 students at grade 8.
In 2005, seven states had sufficient samples of AI/AN students to report state-level data. In 2007, a total of 11 states had sufficiently large samples, with Minnesota, North Carolina, Oregon, and Washington being added to the original 7 selected states from 2005. In 2009, results are also reported for Utah, resulting in state-level reporting for a total of 12 states. While 6 of the 12 states had sufficient AI/AN students without oversampling, schools in 6 states were oversampled in 2009: Arizona, Minnesota, North Carolina, Oregon, Utah, and Washington. Schools with relatively large percentages (at least 5 percent) of AI/AN students were oversampled by factors ranging from 2 to 6 based on state and grade. When AI/AN students are widely dispersed among schools, school oversampling is not effective. The basic approach taken was to create a new stratum in each state that contains schools with a high percentage of AI/AN students, and then to increase the measure of size of these schools by an oversampling factor, thereby increasing their probability of selection. The increase in the expected sample size of AI/AN students was then calculated.
Using different sampling weights for different subgroups of the population, and consequently applying different weights, is generally not as efficient as a sampling scheme that gives each unit in the population an equal chance of selection. The precision achieved by a sample selected in this way could be achieved by a smaller sample size (typically called the “effective” sample size) if sampling rates were the same for each subgroup. However, sampling different subgroups at different rates provides more accurate estimates of target population characteristics and reduces the costs associated with collecting data in the field.
Each school that participated in the assessment, and each student assessed, represents a portion of the population of interest. Results are weighted to make appropriate inferences between the student samples and the respective populations from which they are drawn. Sampling weights account for the disproportionate representation of the selected sample. This includes the oversampling of schools with high proportions of students from certain race/ethnicity groups and lower sampling rates of students who attend very small nonpublic schools. All population and subpopulation characteristics based on the assessment data were estimated using sampling weights. These weights included adjustments for school and student nonresponse.
