​​​​​​​​​​​​​​​​​​​​​​​NAEP Technical DocumentationDevelopment of the U.S. History Cognitive Items and Instruments

The instruments used in the NAEP U.S. history assessment are composed of blocks of cognitive items from the previous NAEP assessments, as well as blocks that are newly developed for the current assessment. Administering the same blocks of items across years allows for the reporting of trends in U.S. history performance. At the same time, developing new items makes it possible to release some items for public use. In some assessment years, one or more blocks at each grade are released to the public and can be accessed via the NAEP Questions Tool.

The NAEP U.S. history framework and specifications documents guide the item development efforts. The same framework that has guided assessment development since 1994, including 2006 modifications to add clarifications and to delete outdated information, was used to guide the development of the 2018 and 2022 digitally based assessments, ​as this continuity enables reporting of student achievement trends over time.

The NAEP U.S. history framework specifies four historical themes that make up the core structure of the ​​assessment.

• The Democracy theme focuses on change and continuity in American democracy: ideas, institutions, events, key figures, and controversies.
• The Culture theme focuses on the gathering and interactions of peoples, cultures, and ideas.
• The Technology theme focuses on economic and technological changes and their relationship to society, ideas, and the environment.​
  
• The World Role theme focuses on the changing role of America in the world.
  

Many of the U.S. history tables that are included in the Technical Documentation on the Web present data for the democracy, culture, technology, and world role​ subscales, which correspond to the content-related ​dimensions described in the U.S. history framework.​ ​When there are several content-related subscales defined for a subject area, the subscales are combined into a composite scale used to describe the overall attainment of students in that subject area​. The weight of each subscale in the composite reflects the relative importance of the scale as specified in the framework developed by the National Assessment Governing Board. The weights are approximately proportional to the number of items in each scale at a given grade level.

​Assessment items are written by NAEP item development staff. All assessment materials are reviewed by members of the U.S. History Standing Committee, as well as other specialists in education and assessment development. The cognitive items are assembled into blocks containing a range of questions​. ​Following approval from the National Center for Education Statistics (NCES), the blocks are packaged as digital test forms for digital delivery of the assessment.

See what the NAEP U.S. history assessment measures for more information.

NAEP transitioned from a paper-based assessment (PBA) to a digitally based assessment (DBA) in U.S. history at grade 8 in 2018 in order to keep pace with the new generation of classroom environments in which digital technology has become an increasing part of students' learning. To investigate potential differences in performance that might be due to the digital transition, randomly equivalent groups of students were administered the NAEP U.S. history assessment in either the paper or the digital format.

The 2014 operational NAEP U.S. history assessment was re-administered in 2018 as a PBA. The PBA was identical to the 2014 operational assessment in terms of the instruments, including all of the items, with the same design of the test booklets which divided cognitive testing time into two 25-minute blocks.

In addition to the 2018 PBA, a digitally based U.S. history assessment which divided cognitive testing time into two 30-minute blocks was also administered. Most of the content administered in the 2018 DBA was also used in the 2014 PBA. The previously used questions were adapted to fit a tablet screen, but the U.S. history content itself did not change. Of the nine DBA blocks administered at grade 8, three blocks were newly developed for the 2018 digital administration.

After the administration of the assessment, NCES conducted rigorous analyses of the data and aligned the 2018 results to previous assessment years. These evaluations supported making trend comparisons between the DBA and previous PBAs, enabling NCES to maintain the U.S. history trend line while transitioning to a DBA. The procedures for estimating the error variance to allow for comparisons between the DBA and previous PBAs are explained here. The 2018 U.S. history assessment results are based on the combined performance​ of students who took the assessment on paper and students who took the assessment on tablets.​

In 2022, the U.S. history assessment at grade 8 was administered digitally for the second time. All of the content at grade 8 in 2022 was also used in the 2018 DBA. A total of nine​ blocks of questions were administered in both years. The transition from PBA to DBA​ was completed in 2018. Because the transition that placed the DBA results onto the trend line took place in 2018, the usual NAEP procedure of common item linking ​(also known as common calibration linking) was used to calculate the trend line from 2018 to 2022. Read more about the social science assessment transition and mode evaluation.

Printer-friendly Version