NAEP Technical DocumentationScience Achievement Levels

The process of developing achievement levels involves the judgments of informed, well-qualified people from throughout the nation and its territories. Approximately 20 persons served on each of three grade-level panels to develop NAEP science achievement levels. These 60 people included teachers (about 55 percent), other educators (about 15 percent), and members of the general public (about 30 percent). To the extent possible, the panels were proportionally representative of the nation's population with respect to region, race/ethnicity, and gender.

The panels for science were convened in 1996, however, before setting the science NAEP achievement levels, the National Assessment Governing Board reviewed achievement level data from other sources. The 1997 College Board Advanced Placement results in science for twelfth-grade students, and information about eighth-grade student performance on the Third International Mathematics and Science Study (TIMSS) were examined. The Board concluded that the recommendations based on the work of the achievement-level setting panels were not reasonable. The Board judged that some levels were set too high and some were set too low. The Board analyzed the relationship of cutscores to percentages of students scoring at or above different levels and adopted levels that it judged to be reasonable. The achievement levels set for the science NAEP were used in reporting results for the 1996 assessment and subsequent assessments. They will be used until the Board determines that a new framework is needed to guide the development of the assessment. At that time, new achievement levels may be developed and set.

Two reports provide details on the science achievement levels:

The Board's complete report on achievement levels for science includes descriptions of achievement levels and cutscores, sample items illustrating achievement levels, and performance data.
Appendix G of the 1996 NAEP Technical Report (Allen, Carlson, and Zelenak 1999) provides detailed information related to the process for developing the achievement levels, selecting exemplar items, evaluating and validating the levels, mapping the levels onto the NAEP scale, as well as panel meeting participants and affiliations.

NOTE: At this time, these achievement levels are used on a trial basis.

Last updated 18 March 2009 (RF)

Printer-friendly Version