Interpreting NAEP Geography ResultsOverview of the Assessment Overview of the AssessmentAs the only federally funded ongoing assessment of student geography achievement on a national scale, the NAEP geography assessment reflects the content framework developed under the guidance of the National Assessment Governing Board. Read more about what the assessment measures, how it was developed, who took the assessment and how the assessment was administered. Reporting the Assessment--Scale Scores and Achievement LevelsThe results of student performance on the NAEP geography assessment are presented here in two ways: as average scores on the NAEP geography scale and as the percentages of students attaining NAEP geography achievement levels. The average scale scores represent how students performed on the assessment. The achievement levels represent how that performance measured up against set expectations for achievement. Thus, the average scale scores represent what students know and can do, while the achievement-level results indicate the degree to which student performance meets expectations of what they should know and be able to do. The national results for 1994 and 2001 are summarized on the NAEP composite geography scales, which are composed of subscales for each of the 3 geography content areas (i.e., Space and Place, Environment and Society, and Spatial Dynamics and Connections). The subscales summarize student performance across all three types of questions in the assessment (multiple-choice, short constructed-response, and extended constructed-response). The composite scales were developed at each of grades 4, 8, and 12 by weighting the separate subscales based on the relative importance of each content area in the NAEP geography framework. The composite scale at each grade ranges from 0 to 500. While the scale score ranges are identical, the scale was derived independently at each grade. Also, scales were weighted differently at different grades in determining the overall scale. Therefore, average scale scores across grades cannot be compared. For example, equal scale scores on the grade 4 and grade 8 scales do not imply equal levels of geography achievement. Achievement-level results are presented in terms of geography achievement levels as adopted by the National Assessment Governing Board (NAGB), and are intended to measure how well students' actual achievement matches the achievement desired of them. For each grade tested, NAGB has adopted three achievement levels: Basic, Proficient, and Advanced. For reporting purposes, the achievement-level cut scores are placed on the geography scales, resulting in four ranges: below Basic, Basic, Proficient, and Advanced. The Board established the achievement levels in 1994 based upon the geography content framework and standard-setting process involving a cross section of educators and interested citizens from across the nation who were asked to judge what students should know and be able to do relative to the content reflected in the NAEP geography framework. As provided by law, the Deputy Commissioner of Education Statistics has determined that the achievement levels are to be used on a trial basis and should be interpreted and used with caution. However, both the Deputy Commissioner and the Board believe these performance standards are useful for understanding trends in student achievement. Description of Geography Performance by Item Maps for Each GradeThe performance of fourth-, eighth-, and twelfth-graders can be illustrated by maps that position question descriptions along the NAEP geography scale. The descriptions used on these maps focus on the geography skill or knowledge needed to answer the question. For multiple-choice questions, the description indicates the skill or knowledge demonstrated by selection of the correct option; for constructed-response questions, the description takes into account the skill or knowledge specified by the different levels of scoring criteria for that question. Approximately 25 geography questions per grade have been selected and placed on an item map for grade 4, grade 8, and grade 12. Results are EstimatesThe average scores and percentages presented on this web site are estimates because they are based on representative samples of students rather than on the entire population of students. Moreover, the collection of subject-area questions used at each grade level is but a sample of the many questions that could have been asked. As such, NAEP results are subject to a measure of uncertainty, reflected in the standard error of the estimates. The standard errors for the estimated scale scores and percentages in the figures and tables presented on this web site are available through the NAEP Data Tool. Statistical SignificanceThe differences between scale scores and between percentages discussed in the results on this web site take into account the standard errors associated with the estimates. Comparisons are based on statistical tests that consider both the magnitude of the difference between the group average scores or percentages and the standard errors of those statistics. Throughout the results, differences between scores or between percentages are discussed only when they are significant from a statistical perspective. All differences reported are significant at the 0.05 level with appropriate adjustments for multiple comparisons. The term "significant" is not intended to imply a judgment about the absolute magnitude of the educational relevance of the differences. It is intended to identify statistically dependable population differences to help inform dialogue among policymakers, educators, and the public. Cautions in InterpretationsUsers of this web site are cautioned against interpreting NAEP results in a causal sense. Inferences related to subgroup performance or to the effectiveness of public and nonpublic schools, for example, should take into consideration the many socioeconomic and educational factors that may also impact performance. Return to main results.
|
