Skip Navigation
small NCES header image

Interpreting NAEP Geography Results

Overview of the Assessment
Reporting the Assessment—Scale Scores and Achievement Levels
Description of Geography Performance by Item Maps for Each Grade
Results Are Estimates
NAEP Reporting Groups
Exclusion Rates
Statistical Significance
Cautions in Interpretations

Overview of the Assessment

NAEP assesses student performance in geography by administering assessments to samples that are representative of the nation's students. The content of the NAEP geography assessment is determined by a framework developed with the help of researchers, policymakers, and the interested public, including those with expert perspectives about geography and its measurement. Read more about what the assessment measures, how it was developed, who took the assessment, and how the assessment was administered.

The 2010 geography results presented on the website are based on representative samples of students for the nation. Approximately 30,000 students from more than 1,400 schools participated in the NAEP geography assessment at grades 4, 8, and 12. The national results reflect the performance of students attending both public and nonpublic schools. Results from 2010 are compared to results from the previous geography assessments administered in 1994 and 2001. In 1994, students were not offered accommodations as part of the NAEP assessments; however, in 2001, NAEP administered the assessment with both a sample who were allowed accommodations and those who were not allowed accommodations. Changes in student performance across years or differences between groups of students in 2001 are discussed only if they have been determined to be statistically significant.

Reporting the Assessment—Scale Scores and Achievement Levels

The results of student performance on the NAEP geography assessment are presented on this website in two ways: as average scores on the NAEP geography scale and as the percentages of students attaining NAEP geography achievement levels. The average scale scores represent how students performed on the assessment. The achievement levels represent how that performance measured up against set expectations for achievement. Thus, the average scale scores represent what students know and can do, while the achievement-level results indicate the degree to which student performance meets expectations of what they should know and be able to do. In addition to reporting an overall geography score for each grade, scale scores are reported at five percentiles to show trends in results for students performing at lower (10th and 25th percentiles), middle (50th percentile), and higher (75th and 90th percentile) levels.

Average geography scale score results are based on the NAEP geography scale, which ranges from 0 to 500.  Average scale scores are computed for groups of students; NAEP does not produce individual student scores. The average scores are based on analyses of the percentages of students who answered each item successfully. While the score ranges at each grade in geography are identical, the scale was derived independently at each grade. Therefore, average scale scores across grades cannot be compared. For example, equal scale scores on the grade 4 and grade 8 scales do not imply equal levels of geography achievement.

Achievement-level results are presented in terms of geography achievement levels adopted by the National Assessment Governing Board, and are intended to measure how well students' actual achievement matches the achievement desired of them. For each grade tested, the Governing Board has adopted three achievement levels: Basic, Proficient, and Advanced. For reporting purposes, the achievement-level cut scores are placed on the geography scales, resulting in four ranges: below Basic, Basic, Proficient, and Advanced. As provided by law, the achievement levels are to be used on a trial basis and should be interpreted and used with caution.

Description of Geography Performance by Item Maps for Each Grade

Item maps illustrate the knowledge and skills demonstrated by students performing at different scale points on the 2010 NAEP geography assessment. In order to provide additional context, the cut points for the three NAEP achievement levels are marked on the item maps. The map location for each question represents the probability that, for a given score point, 65 percent of the students for a constructed-response question and 74 percent of the students for a four-option multiple-choice question answered that question successfully. For constructed-response questions, responses may be completely or partially correct; therefore, different types of responses to the same question could map onto the scale at different score levels.

Approximately 20 to 30 geography questions per grade have been selected and placed on an item map for each grade.

Top

Results Are Estimates

The average scores and percentages presented on this website are estimates because they are based on representative samples of students rather than on the entire population of students. Moreover, the collection of subject-area questions used at each grade level is but a sample of the many questions that could have been asked. As such, NAEP results are subject to a measure of uncertainty, reflected in the standard error of the estimates. The standard errors for the estimated scale scores and percentages in the figures and tables presented on this website are available through the NAEP Data Explorer.

NAEP Reporting Groups

Results are provided for groups of students defined by shared characteristics—gender, race/ethnicity, eligibility for free/reduced-price school lunch, students with disabilities, and students identified as English language learners. Based on participation rate criteria, results are reported for various student populations only when sufficient numbers of students and adequate school representation are present. The minimum requirement is at least 62 students in a particular group from at least five primary sampling units (PSUs). However, the data for all students, regardless of whether their group was reported separately, were included in computing overall results. Explanations of the reporting groups are presented below.

Gender

Results are reported separately for males and females.

Race/Ethnicity

In all NAEP assessments, data about student race/ethnicity is collected from two sources: school records and student self-reports. Before 2002, NAEP used students' self-reports of their race and ethnicity on a questionnaire as the source of race/ethnicity data. In 2002, it was decided to change the student race/ethnicity variable highlighted in NAEP reports. Starting in 2002, NAEP reports of students' race and ethnicity are based on the school records, with students' self-reports used only if school data are missing. Information based on student self-reported race/ethnicity will continue to be reported in the NAEP Data Explorer.

In order to allow comparisons across years, assessment results presented are based on school-reported information for six mutually exclusive racial/ethnic categories: White, Black, Hispanic, Asian/Pacific Islander, American Indian (including Alaska Native), and Other. Students who identified with more than one of the first five categories or had a background other than the ones listed were categorized as Unclassified.

Eligibility for National School Lunch Program

NAEP collects data on student eligibility for the National School Lunch Program (NSLP) as an indicator of low family income. Under the guidelines of NSLP, children from families with incomes below 130 percent of the poverty level are eligible for free meals. Those from families with incomes between 130 and 185 percent of the poverty level are eligible for reduced-price meals. (For the period July 1, 2009, through June 30, 2010, for a family of four, 130 percent of the poverty level was $28,665, and 185 percent was $40,793.) Some schools provide free meals to all students irrespective of individual eligibility, using their own funds to cover the costs of noneligible students. Under special provisions of the National School Lunch Act intended to reduce the administrative burden of determining student eligibility every year, schools can be reimbursed based on eligibility data for a single base year. Participating schools might have high percentages of eligible students and report all students as eligible for free lunch.

Because of the improved quality of the data on students' eligibility for NSLP, the percentage of students for whom information was not available has decreased in 2010 in comparison to 2001. While results for NSLP eligibility for 2001 are not shown in the print report, all results are available on the website beginning in 2001--the earliest year in which NAEP collected data by this variable as part of the geography assessment.

Students with Disabilities (SD)

Results are reported for students who were identified by school records as having a disability. A student with a disability may need specially designed instruction to meet his or her learning goals. A student with a disability will usually have an Individualized Education Program (IEP) which guides his or her special education instruction. Students with disabilities are often referred to as special education students and may be classified by their school as learning disabled (LD) or emotionally disturbed (ED).

Most figures in the geography web results show two data points in 2001—one in which accommodations were permitted and one in which they were not. While results for both 2001 samples are presented, comparisons between 2001 and 2010 are based on the accommodated samples of both years. When results for students with disabilities are presented, only the accommodated sample data are shown. Accommodations were not offered prior to 2001.

No accommodations were provided in the 1994 NAEP geography assessment. Providing accommodations represented a change in testing conditions that could potentially affect the measurement of changes over time, so split samples of students were assessed in 2001. One sample permitted accommodations and the other did not. Results for both samples are shown in the tables and figures, but comparisons to 2001 in the text are based on just the accommodated samples.

English Language Learners (ELL)

Results are reported for students who were identified by school records as being English language learners. When results are presented for English language learners, only the accommodated sample is presented as the 2001 starting point in the trend line. (Note that English language learners were previously referred to as limited English proficient [LEP].)

Type of School

The national results are based on a representative sample of students in both public schools and nonpublic schools. Nonpublic schools include private schools, Bureau of Indian Affairs schools, and Department of Defense schools. Private schools include Catholic, Conservative Christian, Lutheran, and other private schools. Results are reported for private schools overall, as well as disaggregated by Catholic and other private schools, in 2010 at grade 8 only due to low participation rates at other grade levels.

Parental Education

Parents' highest level of education is defined by the highest level reported by eighth-graders and twelfth-graders for either parent. Fourth-graders' replies to this question were not reported because their responses in previous studies were highly variable, and a large percentage of them chose the "I don't know" option.

Top

Exclusion Rates

NAEP has established policies and procedures to maximize the inclusion of all students in the assessment. Every effort is made to ensure that all selected students who are capable of participating meaningfully in the assessment are assessed. While some students with disabilities (SD) and/or English language learners (ELL) can be assessed without any special procedures, others require accommodations to participate in NAEP. Still other SD and/or ELL students selected by NAEP may not be able to participate. Local school authorities determine whether SD/ELL students require accommodations or should be excluded because they cannot be assessed. The percentage of SD and/or ELL students who are excluded from NAEP assessments varies from one jurisdiction to another and within a jurisdiction over time.

See additional information about the percentages of special-needs students identified, excluded, and assessed at the national level.

Statistical Significance

Differences between scale scores and between percentages that are discussed in the results on this website take into account the standard errors associated with the estimates. Comparisons are based on statistical tests that consider both the magnitude of the difference between the group average scores or percentages and the standard errors of those statistics. Throughout the results, differences between scores or between percentages are discussed only when they are significant from a statistical perspective.

All differences reported are significant at the 0.05 level with appropriate adjustments for multiple comparisons. The term "significant" is not intended to imply a judgment about the absolute magnitude or the educational relevance of the differences. It is intended to identify statistically dependable population differences to help inform dialogue among policymakers, educators, and the public.

Cautions in Interpretations

Users of this website are cautioned against interpreting NAEP results as implying causal relations. Inferences related to student group performance or to the effectiveness of public and nonpublic schools, for example, should take into consideration the many socioeconomic and educational factors that may also have an impact on performance.

The NAEP geography scale makes it possible to examine relationships between students' performance and various factors measured by NAEP. However, a relationship that exists between achievement and another variable does not reveal its underlying cause, which may be influenced by a number of other variables. Similarly, the assessments do not reflect the influence of unmeasured variables. The results are most useful when they are considered in combination with other knowledge about the student population and the educational system, such as trends in instruction, changes in the school-age population, and societal demands and expectations.

Top

View more geography subject information.


Last updated 31 July 2012 (NB)
Would you like to help us improve our products and website by taking a short survey?

YES, I would like to take the survey

or

No Thanks

The survey consists of a few short questions and takes less than one minute to complete.
National Center for Education Statistics - http://nces.ed.gov
U.S. Department of Education