NAEP Technical DocumentationDescribing NAEP Scale Score Distributions
|
A goal of NAEP has been to provide information to the public about what students in U.S. schools know and can do in various subject areas. While the NAEP scales in each subject area provide information about the distributions of scale scores for various student groups of interest, they do not directly provide information about the meaning of specific points on the scale.
There are two approaches of attaching meaning to selected points on the scale of assessments: norm-referencing and criterion-referencing. Traditionally, in educational assessments, meaning has been attached to educational scales by norm-referencing—that is, by comparing students at a particular scale level to other students (e.g., percentile rank in SAT or ACT). However, NAEP uses a criterion-referencing approach to attach meaning to selected points on its educational scales. Scale anchors and achievement levels provide descriptions of the types of skills that are likely to be exhibited at those levels.
One way that scale anchoring is accomplished in NAEP is through the process of item mapping. In item mapping, individual NAEP assessment items are mapped to a point on their corresponding scale, so that the content of each item provides information about what students at each score level can do in a probabilistic sense.
A second way that scale anchoring is accomplished is through the process of
performance-level setting. For the reading and mathematics long-term trend assessments, results are reported on separate scales ranging from 0 to 500. Each scale was divided into five successive levels of performance, and the scale anchoring process was used to define what it meant to score in each of these levels. Questions were identified that were likely to be answered correctly by students performing at a particular level on the scale and much less likely to be answered correctly by students performing at the next lower level. Scale anchoring for reading long-term trend was based on the 1984 assessment, and scale anchoring for mathematics long-term trend was based on the 1986 assessment.
NAEP achievement-level setting is accomplished through a process of expert judgment, matching performance on assessment items to the descriptions of ideal performance expected by the National Assessment Governing Board framework committees at the
NAEP Basic,
NAEP Proficient, and
NAEP Advanced levels. The descriptions of performance at each achievement level should be used with caution, according to independent evaluations of those processes. Achievement levels have been defined for the NAEP economics, geography, mathematics, reading, and U.S. history composite scales and for univariate scales in civics, science, technology and engineering literacy (TEL), and writing. The achievement levels were set for civics in 1998, economics in 2006, geography in 1994, mathematics in 1990 for grades 4 and 8 and 2005 for grade 12, reading in 1992, science in 2009, TEL in 2014, U.S. history in 1994, and writing in 2002 for grade 4 and 2011 for grades 8 and 12.
Last updated 24 July 2024 (SK)