Interpreting NAEP Science ResultsOverview of the Assessment Overview of the AssessmentAs the only federally-funded ongoing assessment of student science achievement on a national scale, the NAEP science assessment reflects the content framework and expert perspectives about science and its measurement. Read more about what the assessment measures, how it was developed, who took the assessment and how the assessment was administered. Reporting the Assessment - Scale Scores and Achievement LevelsThe results of student performance on the NAEP science assessment are presented on this Web site in two ways: as average scores on the NAEP science scale and as the percentages of students attaining NAEP science achievement levels. The average scale scores represent how students performed on the assessment. The achievement levels represent how that performance measured up against set expectations for achievement. Thus, the average scale scores represent what students know and can do, while the achievement-level results indicate the degree to which student performance meets expectations of what they should know and be able to do. The national results for 1996 and 2000 are summarized on the NAEP composite science scales, which are composed of subscales for each of the three fields of science (i.e., Earth, physical, and life). The subscales summarize student performance across all three types of questions in the assessment (multiple-choice, short constructed-response, and extended constructed-response). The composite scales were developed at each of grades 4, 8, and 12 by weighting the separate subscales based on the relative importance of each content area in the NAEP science framework. The composite scale at each grade ranges from 0 to 300. While the scale score ranges are identical, the scale was derived independently at each grade. Also, scales were weighted differently at different grades in determining the overall scale. Therefore, average scale scores across grades cannot be compared. For example, equal scale scores on the grade 4 and grade 8 scales do not imply equal levels of science achievement. Achievement-level results are presented in terms of science achievement levels as adopted by the National Assessment Governing Board (NAGB), and are intended to measure how well students' actual achievement matches the achievement desired of them. For each grade tested, NAGB has adopted three achievement levels: Basic, Proficient, and Advanced. For reporting purposes, the achievement-level cut scores are placed on the science scales, resulting in four ranges: below Basic, Basic, Proficient, and Advanced. The Board established its achievement levels in 1996 based upon the science content framework and standard-setting process involving a cross section of educators and interested citizens from across the nation who were asked to judge what students should know and be able to do relative to the content reflected in the NAEP science framework. As provided by law, the Acting Commissioner of Education Statistics has determined that the achievement levels are to be considered developmental and should be interpreted and used with caution. However, both the Acting Commissioner and the Board believe these performance standards are useful for understanding trends in student achievement. Description of Science Performance by Item Maps for Each GradeThe performance of fourth-, eighth-, and twelfth-graders can be illustrated by maps that position question descriptions along the NAEP science scale. The descriptions used on these maps focus on the science skill or knowledge needed to answer the question. For multiple-choice questions, the description indicates the skill or knowledge demonstrated by selection of the correct option; for constructed-response questions, the description takes into account the skill or knowledge specified by the different levels of scoring criteria for that question. Approximately 25 science questions per grade have been selected and placed on an item map for grade 4, grade 8, and grade 12. Results are EstimatesThe average scores and percentages presented on this Web site are estimates because they are based on representative samples of students rather than on the entire population of students. Moreover, the collection of subject-area questions used at each grade level is but a sample of the many questions that could have been asked. As such, NAEP results are subject to a measure of uncertainty, reflected in the standard error of the estimates. The standard errors for the estimated scale scores and percentages in the figures and tables presented on this Web site are available through the NAEP Data Tool. Statistical SignificanceThe differences between scale scores and between percentages discussed in the results on this Web site take into account the standard errors associated with the estimates. Comparisons are based on statistical tests that consider both the magnitude of the difference between the group average scores or percentages and the standard errors of those statistics. Throughout the results, differences between scores or between percentages are discussed only when they are significant from a statistical perspective. All differences reported are significant at the 0.05 level with appropriate adjustments for multiple comparisons. The term "significant" is not intended to imply a judgment about the absolute magnitude of the educational relevance of the differences. It is intended to identify statistically dependable population differences to help inform dialogue among policy makers, educators, and the public. Cautions in InterpretationsUsers of this Web site are cautioned against interpreting NAEP results in a causal sense. Inferences related to subgroup performance or to the effectiveness of public and nonpublic schools, for example, should take into consideration the many socioeconomic and educational factors that may also impact performance. Return to main results.
|
