Interpreting NAEP Economics Results

Overview of the Assessment
Reporting the Assessment—Scale Scores and Achievement Levels
Description of Economics Performance by Item Maps for Each Grade
Results Are Estimates
NAEP Reporting Groups
Exclusion Rates
Statistical Significance
Cautions in Interpretations

Overview of the Assessment

NAEP administered its second economics assessment at grade 12 in 2012. The first assessment was administered in 2006. The 2012 assessment was administered to a representative sample of 10,900 students at almost 480 schools throughout the nation. Read more about who took the assessment. The content of the NAEP economics assessment is determined by a framework, which was developed under the guidance of the National Assessment Governing Board with the help of researchers, policymakers, and the interested public, including those with expert perspectives about economics and its measurement. Read more about what the assessment measures, and how it was developed.

Reporting the Assessment—Scale Scores and Achievement Levels

The results of student performance on the NAEP economics assessment are presented as average scores on the NAEP economics scale, as the percentages of students attaining NAEP economics achievement levels, and as percentile scores. The average scale scores represent how students performed on the assessment. The achievement levels represent how that performance measured up against set expectations for achievement. Thus, the average scale scores represent what students know and can do, while the achievement-level results indicate the degree to which student performance meets expectations of what they should know and be able to do. Percentile scores provide a score location below which a specified percentage of the population falls.

Average economics scale score results are based on the NAEP economics scale, which ranges from 0 to 300. Average scale scores are computed for groups of students; NAEP does not produce individual student scores. The average scores are based on analyses of the percentages of students who answered each item successfully.

Achievement-level results are presented in terms of economics achievement levels adopted by the National Assessment Governing Board and are intended to measure how well students' actual achievement matches the achievement desired of them. For each grade tested, the National Assessment Governing Board has adopted three achievement levels: NAEP Basic, NAEP Proficient, and NAEP Advanced. For reporting purposes, the achievement-level cut scores are placed on the economics scales, resulting in four ranges: below NAEP Basic, NAEP Basic, NAEP Proficient, and NAEP Advanced. As provided by law, the achievement levels are to be used on a trial basis and should be interpreted and used with caution.

Description of Economics Performance by Item Maps for Each Grade

An item map illustrates the knowledge and skills demonstrated by grade 12 students performing at different scale points on the 2012 NAEP economics assessment. In order to provide additional context, the cut points for the three NAEP achievement levels are marked on the item maps. The map location for each question represents the probability that, for a given score point, 65 percent of the students for a constructed-response question and 74 percent of the students for a four-option multiple-choice question answered that question successfully. For constructed-response questions, responses may be completely or partially correct; therefore, different types of responses to the same question could map onto the scale at different score levels.

Approximately 20 to 30 economics questions have been selected and placed on the item map for grade 12.

Results Are Estimates

The average scores and percentages presented on this website are estimates because they are based on representative samples of students rather than on the entire population of students. Moreover, the collection of subject-area questions used at each grade level is but a sample of the many questions that could have been asked. As such, NAEP results are subject to a measure of uncertainty, reflected in the standard error of the estimates. The standard errors for the estimated scale scores and percentages in the figures and tables presented on the website are available through the NAEP Data Explorer.

Top

NAEP Reporting Groups

Results are provided for groups of students defined by shared characteristics—gender, race/ethnicity, eligibility for free/reduced-price school lunch, students with disabilities (SD), and students identified as English learners (EL). Based on participation rate criteria, results are reported for various student populations only when sufficient numbers of students and adequate school representation are present. The minimum requirement is at least 62 students in a particular group from at least five primary sampling units (PSUs). However, the data for all students, regardless of whether their group was reported separately, were included in computing overall results. Explanations of the reporting groups are presented below.

Gender

Results are reported separately for males and females.

Race/Ethnicity

Prior to 2011, student race/ethnicity was obtained from school records and reported for the six mutually exclusive categories shown below:

• White
• Black
• Hispanic
• Asian/Pacific Islander
• American Indian/Alaska Native
• Other or unclassified

Students identified with more than one of the other five categories were classified as “other” and were included as part of the "unclassified" category along with students who had a background other than the ones listed or whose race/ethnicity could not be determined.

In compliance with standards from the U.S. Office of Management and Budget for collecting and reporting data on race/ethnicity, additional information was collected beginning in 2011 so that results could be reported separately for Asian students, Native Hawaiian/Other Pacific Islander students, and students identifying with two or more races. All of the students participating in the NAEP assessments from 2011 forward were identified as belonging in one of the seven racial/ethnic categories listed below:

• White
• Black or African American
• Hispanic
• Asian
• Native Hawaiian or other Pacific Islander
• American Indian or Alaska Native
• Two or more races

As in earlier years, students identified as Hispanic continued to be classified as Hispanic even if they were also identified with another racial/ethnic group. Students identified with two or more of the other racial/ethnic groups (e.g., White and Black) would have been classified as “other” and reported as part of the "unclassified" category prior to 2011, and classified as “two or more races” beginning in 2011.

When comparing the 2012 economics results for racial/ethnic groups with results from 2006, the 2012 data for Asian and Native Hawaiian/Other Pacific Islander students were combined into a single Asian/Pacific Islander category. Information based on student self-reported race/ethnicity will continue to be reported in the NAEP Data Explorer.

Parental Education

Parents' highest level of education is defined by the highest level reported by twelfth-graders for either parent. Students were asked to indicate their mother's and father's education level with one of the following categories: she/he did not finish high school; she/he graduated from high school; she/he had some education after high school; she/he graduated from college; or, I don't know.

Student with Disabilities (SD)

Results are reported for students who were identified by school records as having a disability. A student with a disability may need specially designed instruction to meet his or her learning goals. A student with a disability will usually have an Individualized Education Program (IEP) which guides his or her special education instruction. Students with disabilities (SD) are often referred to as special education students and may be classified by their school as learning disabled (LD) or emotionally disturbed (ED). The goal of NAEP is that students who are capable of participating meaningfully in the assessment are assessed, but some students with disabilities selected by NAEP may not be able to participate, even with the accommodations provided. Beginning in 2009, NAEP disaggregated students with disabilities from students who were identified under section 504 of the Rehabilitation Act of 1973; however, trend results dating back to 2006 are available in economics for the SD variable that includes section 504 students.

English Learners (EL)

Results are reported for students who were identified by school records as being English learners. (Note that English learners were previously referred to as limited English proficient (LEP).)

Type of School

The national results are based on a representative sample of students in both public schools and nonpublic schools. Nonpublic schools include private schools, Bureau of Indian Affairs schools, and Department of Defense schools. Private schools include Catholic, Conservative Christian, Lutheran, and other private schools.

Type of Location

NAEP results are reported for four mutually exclusive categories of school location: city, suburb, town, and rural. The categories are based on standard definitions established by the Federal Office of Management and Budget using population and geographic information from the U.S. Census Bureau. Schools are assigned to these categories in the NCES Common Core of Data based on their physical address.

In 2007, the classification system was revised; therefore, trend comparisons to previous years are not available. The new locale codes are based on an address's proximity to an urbanized area (a densely settled core with densely settled surrounding areas). This is a change from the original system based on metropolitan statistical areas. To distinguish the two systems, the new system is referred to as "urban-centric locale codes." The urban-centric locale code system classifies territory into four major types: city, suburban, town, and rural. Each type has three subcategories. For city and suburb, these are gradations of size—large, midsize, and small. Towns and rural areas are further distinguished by their distance from an urbanized area. They can be characterized as fringe, distant, or remote.

Top

Exclusion Rates

NAEP has established policies and procedures to maximize the inclusion of all students in the assessment. Every effort is made to ensure that all selected students who are capable of participating meaningfully in the assessment are assessed. While some students with disabilities (SD) and/or English learners (EL) can be assessed without any special procedures, others require accommodations to participate in NAEP. Still other SD and/or EL students selected by NAEP may not be able to participate. Local school authorities determine whether SD/EL students require accommodations or should be excluded because they cannot be assessed. The percentage of SD and/or EL students who are excluded from NAEP assessments varies from one jurisdiction to another and within a jurisdiction over time.

See additional information about the percentages of special-needs students

• identified, excluded, and assessed at the national level and
• the types of accommodations permitted for students with disabilities and/or English learners.

Statistical Significance

Differences between scale scores and between percentages that are discussed in the results take into account the standard errors associated with the estimates. Comparisons are based on statistical tests that consider both the magnitude of the difference between the group average scores or percentages and the standard errors of those statistics. Throughout the results, differences between scores or between percentages are discussed only when they are significant from a statistical perspective.

All differences reported are significant at the 0.05 level with appropriate adjustments for multiple comparisons. The term "significant" is not intended to imply a judgment about the absolute magnitude or the educational relevance of the differences. It is intended to identify statistically dependable population differences to help inform dialogue among policymakers, educators, and the public.

Cautions in Interpretations

Users of this website are cautioned against interpreting NAEP results as implying causal relations. Inferences related to student group performance or to the effectiveness of public and nonpublic schools, for example, should take into consideration the many socioeconomic and educational factors that may also have an impact on performance.

The NAEP economics scale makes it possible to examine relationships between students' performance and various factors measured by NAEP. However, a relationship that exists between achievement and another variable does not reveal its underlying cause, which may be influenced by a number of other variables. Similarly, the assessments do not reflect the influence of unmeasured variables. The results are most useful when they are considered in combination with other knowledge about the student population and the educational system, such as trends in instruction, changes in the school-age population, and societal demands and expectations.

Top

See more economics subject information.