- Surveys & Programs
- Data & Tools
- Fast Facts
- News & Events
- Publications & Products
- About Us
Table of Contents | Search Technical Documentation | References
NAEP Technical DocumentationDevelopment of the Mathematics Cognitive Items and Instruments
The instruments used in the NAEP mathematics assessment are composed of
blocks of cognitive
items from the previous NAEP assessments, as well as blocks that are newly developed for the current assessment. Administering the same blocks of items across years allows for the reporting of trends in mathematics performance. At the same time, developing new items makes it possible to release some items for public use. In some assessment years, one or more blocks at each grade are released to the public and can be accessed via the
NAEP Questions Tool.
The
NAEP mathematics framework and specifications documents guide the item development efforts. A new framework was introduced for 2009. Other than some minor reorganization, the 2009 framework for grades 4 and 8 remained largely unchanged from 2005. The framework for the 2019 and 2022 assessments at grades 4 and 8 was the same as that used in 2009. The framework for grade 12 was updated in 2005. Changes were made to reflect adjustments in curricular emphasis and to ensure an appropriate balance of content. Due to these changes, the twelfth-grade mathematics results in 2005 and subsequent years cannot be compared to previous assessments, necessitating the start of a new trend line beginning with the 2005 mathematics assessment. The 2009 framework for grade 12 underwent some changes to help NAEP report how well prepared twelfth-grade students are for postsecondary education and training. The results of a special analysis (2009 Mathematics Trend Study at Grade 12) determined that the trend line could be continued after the changes introduced in the 2009 framework. Read more about the differences between the frameworks.
The NAEP mathematics framework specifies that each question in the assessment measure one of five mathematical content areas at grades 4 and 8.
- Number properties and operations measures students’ understanding of ways to represent, calculate, and estimate with numbers.
- Measurement assesses students’ knowledge of units of measurement for such attributes as capacity, length, area, volume, time, angles, and rates.
- Geometry measures students’ knowledge and understanding of shapes in two and three dimensions, and relationships between shapes such as symmetry and transformations.
-
Data analysis, statistics, and probability measures students’ understanding of data representation, characteristics of data sets, experiments and samples, and probability.
- Algebra measures students’ understanding of patterns, using variables, algebraic representation, and functions.
The grade 12 assessment measures four content areas; because most measurement topics suitable for twelfth-grade students are geometric in nature, geometry and measurement are combined into one content area.
Many of the mathematics tables that are included in the Technical Documentation on the Web present data for the number properties and operations; measurement; geometry; data analysis, statistics, and probability; and algebra subscales, which correspond to the content-related dimensions described in the mathematics framework. At grade 12, the tables present data for a combined measurement and geometry subscale. When there are several content-related subscales defined for a subject area, the subscales are combined into a composite scale used to describe the overall attainment of students in that subject area. The weight of each subscale in the composite reflects the relative importance of the scale as specified in the framework developed by the National Assessment Governing Board. The weights are approximately proportional to the number of items in each scale at a given grade level.
Assessment items are written by NAEP item development staff. All assessment materials are reviewed by members of the Mathematics Standing Committee, as well as other specialists in education and assessment development. The cognitive items are assembled into blocks containing a range of questions. Following approval from the
National Center for Education Statistics, the blocks are packaged as digital test forms for digital delivery of the assessment.
See what the NAEP mathematics assessment measures for more information.
NAEP officially transitioned from a paper-based assessment (PBA) to a digitally based assessment (DBA) in mathematics at grades 4 and 8 in 2017 in order to keep pace with the new generation of classroom environments in which digital technology has become an increasing part of students' learning. A bridge study was conducted in 2017 and involved randomly equivalent samples receiving the NAEP mathematics assessment in either the paper or the digital format administered on tablets. For the digitally based assessment, previously used paper-based assessment questions were adapted to fit a tablet screen but the construct being measured did not change. In accordance with the principles of bridge studies historically employed by NAEP, the 2015 operational NAEP mathematics assessment was re-administered in 2017 as a paper-based assessment at grades 4 and 8. The 2017 paper-based assessment was identical to the 2015 operational assessment in terms of the instruments, including all of the items, with the same design of the test booklets which divided cognitive testing time into two 25-minute blocks. Approximately two-thirds of the questions from the 2015 paper-based assessment were adapted to the 2017 digitally based assessments. The 2017 digitally based assessment divided the cognitive testing time into two 30-minute blocks.
At both grades 4 and 8, six of the ten assessment blocks used only questions that had been adapted from the 2015 paper-based assessment and were assembled to be as similar as possible to corresponding paper-based blocks. Four of the ten blocks consisted of new questions developed for digital administration in 2017.
For more information, see the
mode evaluation study on the 2017 NAEP transition to digitally based assessments. Since 2017, the reported results at grades 4 and 8 are based on the performance of students who took the assessment on tablets.
The NAEP mathematics assessments at grades 4 and 8 in 2019 were administered for the second time as digitally based assessments. Because the transition that placed the DBA results onto the trend line took place in 2017, the usual NAEP procedure of common item linking (also known as common calibration linking) was used to calculate the trend line from 2017 to 2019. The 2019 results at grades 4 and 8 are based on the performance of students who took the assessment on tablets. Approximately two-thirds of the questions from the 2019 DBA were also administered in the 2017 DBA.
In 2022, the mathematics assessments at grades 4 and 8 were administered digitally for the third time. All of the content at grades 4 and 8 in 2022 was also used in the 2019 DBA. A total of fourteen blocks of questions were administered in both years at grades 4 and 8. Common item linking was used to calculate the trend line from 2019 to 2022.
The grade 12 mathematics assessment was administered for the first time as a digitally based assessment in 2019. Just as was done for the grades 4 and 8 assessments in 2017, a multistep process was used for the transition from PBA to DBA in order to preserve trend lines that show student performance over time. The transition process involved administering the assessment in both the DBA and PBA formats to randomly equivalent groups of twelfth-grade students in 2019. The results from DBA can therefore be compared to those from previous years, showing how students' performance in mathematics has changed over time. Read more about the
grade 12 transition and mode evaluation.
At grade 12, approximately 60 percent of the questions from the 2015 PBA were adapted to the 2019 DBA. Seven of the eleven assessment blocks administered in 2019 used only questions that had been adapted from PBA, and six of those seven blocks were assembled to be as similar as possible to their corresponding paper versions. Four of the eleven blocks at grade 12 consisted of questions newly developed for the 2019 digital administration.
In addition to the DBA, random subsamples of students were administered the complete 2015 paper-based version of the grade 12 assessment in 2019. NCES administered the assessment in both modes—paper-based and digitally based—in all the sampled schools to investigate potential differences in performance between students taking the assessment on a tablet and students taking the PBA. Each participating student took the assessment in only one mode.
After the administration of the assessment at grade 12, NCES conducted rigorous analyses of the data and aligned the 2019 results to previous assessment years using a two-step process. The first step used common item linking followed by
common population linking in the second step. These analyses—common item linking based on paper results and common population linking of paper results to digital results—enabled NCES to successfully maintain the mathematics trend line while transitioning to the digital assessment in 2019. The reported 2019 grade 12 mathematics assessment is based on the combined performance of students who took the assessment on paper and students who took the assessment on tablets.
Last updated 20 September 2024 (PG)
