NAEP Technical Documentation NAEP 2011 Sample Design

			2011 State Assessment Sample Design 2011 National Assessment Sample Design

The sample design for NAEP 2011 included samples for various operational, special study, and pilot test assessments. Representative samples were drawn for the following operational assessments:

• national assessments in mathematics and reading in public and private schools at grades 4 and 8;
• national assessments in computer-based writing (WCBA) in public and private schools at grades 8 and 12;
• national assessments in science in public and private schools at grade 8;
• state-by-state and Trial Urban District Assessments (TUDA) assessments in mathematics and reading in public schools at grades 4 and 8; and
• state-by-state assessments in science in public schools at grade 8.

Representative samples were drawn for the following special study and pilot test assessments:

• mathematics computer-based study (MCBS) in public schools at grade 8;
• study to examine a direct link between NAEP and Trends in International Mathematics and Science Study (TIMSS) in public schools at grade 8;
• Special mathematics assessment in Puerto Rico in public and private schools at grade 4 and in public schools at grade 8; and 
• pilot tests in reading and mathematics in public and private schools at grade 4, in reading and mathematics in public schools at grade 8, and in economics in public schools at grade 12.

The samples for the operational assessments were organized into four distinct components and sampled separately. The samples for the special studies and pilot tests were integrated into these various components.

• mathematics, reading, and science assessments of fourth- and eighth-grade students in public schools;
• mathematics, reading, and science assessments of fourth-grade and eighth-grade students in private schools;
• computer-based writing assessments and mathematics study of eighth-grade and twelfth-grade students in public schools; and
• computer-based writing assessments of eighth-grade and twelfth-grade students in private schools.

The national assessments were designed to achieve nationally representative samples of public and private school students in the fourth, eighth, and twelfth grades. Their target populations included all students in public, private, Bureau of Indian Education (BIE), and Department of Defense Education Activity (DoDEA) schools, who were enrolled in fourth, eighth, and twelfth grades, respectively, at the time of assessment.

For the fourth- and eighth-grade mathematics, reading, and science  assessments in public schools, the NAEP state student samples and assessments constitute the NAEP national student samples and assessments. Nationally representative samples were drawn for the remaining populations of private school students in fourth and eighth grades.

The TUDA samples formed part of the corresponding state public school samples, and the state samples formed the public school grade 4 and 8 part of the national sample.   

The mathematics, reading, and science samples were based on a two-stage sample design:

• selection of schools within strata, and
• selection of students within schools.

The computer-based writing and mathematics samples were based on a three-stage sample design:

• selection of primary sampling units (PSUs),
• selection of schools within strata, and
• selection of students within schools.

In the three-stage design, schools were stratified and selected within the sampled PSUs. The samples of schools were selected with probability proportional to a measure of size based on the estimated grade-specific enrollment in the schools for both designs.

The state assessments were designed to achieve representative samples of students in the fourth and eighth grades. Their target populations included all students in each participating jurisdiction, which included states, District of Columbia, BIE, DoDEA, and school districts chosen for the TUDA assessments. Each sample was designed to produce aggregate estimates with reliable precision for all the participating jurisdictions, as well as estimates for various student subpopulations of interest. 

At grades 4 and 8, all BIE schools were included in the mathematics, reading, and science assessments. Also, public schools with relatively high American Indian/Alaska Native populations were oversampled in six states (Arizona, Minnesota, North Carolina, Oregon, Utah, and Washington). This was designed to enhance the reporting of results for American Indian students at the state level in those states with a sizable proportion of the nation's American Indian students for the National Indian Education Study (NIES), which was conducted in conjunction with NAEP. 

All states participated in the mathematics, reading, and science assessments. By design, only BIE schools did not participate in the state science assessment, as it lacked the required number of students for the state science assessment. A small portion of students received the science assessment in BIE schools in science to supplement the national science sample.

The District of Columbia, which generally does not have enough students for an assessment in a third subject, also participated in the grade 8 science assessment. To accomplish this, each student in the District of Columbia was assigned to two of the three assessment subjects and thus tested twice over two days.

The figure below illustrates the various sample types and subjects.

Components of the NAEP 2011 samples, by assessment subject, grade, and school type: 2011

NOTE: View an accessible version of this figure.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2011 Assessments.

Printer-friendly Version