- Surveys & Programs
- Data & Tools
- Fast Facts
- News & Events
- Publications & Products
- About Us
Table of Contents | Search Technical Documentation | References
NAEP Technical DocumentationDevelopment of the Reading Cognitive Items and Instruments
The instruments used in the NAEP reading assessment are composed of blocks containing reading passages and items from previous NAEP assessments, as well as blocks that are newly developed for the current assessment year. Administering many of the same blocks of items across years allows for the reporting of trends in reading performance. At the same time, developing new sets of items makes it possible to release some blocks to the public. Released blocks can be accessed via the
NAEP Questions Tool.
The
NAEP reading framework and specifications documents guide the item development efforts. In 2009, a new reading framework was introduced, and all of the blocks were newly developed. A special
trend study was conducted at that time to determine whether the results of the 2009 reading assessment could be compared to those from earlier assessment years developed under the previous framework. Overall, the results of the special analyses suggested that the old and new assessments were similar in terms of their item and scale characteristics and the results they produced for important demographic groups of students. It was determined that the results of the 2009 reading assessment could still be compared to those from earlier assessment years, thereby maintaining the trend lines established in 1992. The framework for the 2022 assessment and all previous assessments since 2009 was the same.
The NAEP reading framework specifies the use of literary and informational texts in the assessment.
- Literary texts include fiction, literary nonfiction, and poetry.
-
Informational texts include exposition, argumentation and persuasive texts, and procedural texts and documents.
Many of the reading tables that are included in the Technical Documentation on the Web present data for the literary and informational subscales, which correspond to the content-related dimensions described in the reading framework. When there are several content-related subscales defined for a subject area, the subscales are combined into a composite scale used to describe the overall attainment of students in that subject area. The weight of each subscale in the composite reflects the relative importance of the scale as specified in the framework developed by the National Assessment Governing Board. The weights are approximately proportional to the number of items in each scale at a given grade level.
The framework also calls for a systematic assessment of vocabulary. Vocabulary questions measure students' knowledge of specific words as used in the passages they are asked to read for the assessment. To answer these questions, students integrate their understanding of the word with their passage comprehension.
Items in the reading blocks are written by NAEP item development staff and reviewed by members of the Reading Standing Committee. All assessment materials are reviewed by specialists in reading education, measurement, and assessment development for accessibility and potential bias. The reading passages and associated cognitive items are assembled into blocks containing a range of questions. Following approval from the National Center for Education Statistics, the blocks are packaged as digital test forms for digital delivery of the assessment.
See
what the NAEP reading assessment measures for more information.
NAEP officially transitioned from a paper-based assessment to a digitally based reading assessment at grades 4 and 8 in 2017. In 2017, a bridge study was conducted which involved randomly equivalent samples receiving the NAEP reading assessment in either the paper or the digital format.
For more information, see the mode evaluation study on the 2017 NAEP transition to digitally based assessments. Since 2017, the reported results at grades 4 and 8 are based on the performance of students who took the assessment on tablets.
In 2019, the reading assessments at grades 4 and 8 were administered digitally for the second time. The transition from paper-based assessment (PBA) to digitally based assessment (DBA) was completed in 2017. Most of the content at grades 4 and 8 in 2019 was also used in the 2017 digitally based assessment and in the 2015 paper-based assessment. In 2017, ten blocks were administered at grade 4 and 13 blocks were administered at grade 8. Four blocks were grade 4/grade 8 cross-grade blocks. Of the 19 unique passages and question sets administered across grades 4 and 8, one set at each grade was newly developed for 2017. For the 2019 digitally based assessment, 12 blocks were administered at grade 4 and 15 blocks were administered at grade 8. Three blocks were grade 4/grade 8 cross-grade blocks. Four new passages and question sets were developed at each grade. Because the transition which placed the DBA results onto the trend line occurred in 2017, the usual NAEP procedure of common item linking (also known as common calibration linking) was used to calculate the trend line from 2017 to 2019. The 2019 results at grades 4 and 8 were based on the performance of students who took the assessment on tablets.
In 2022, the reading assessments at grades 4 and 8 were administered digitally for the third time. All of the content at grades 4 and 8 in 2022 was also used in the 2019 DBA. At grade 4, a total of 11 blocks of questions were administered in both years, and at grade 8, a total of 14 blocks of questions were administered in both years. Common item linking was used to calculate the trend line from 2019 to 2022.
The assessment at grade 12 was administered digitally for the first time in 2019. Most of the content at grade 12 was also used in the 2015 paper-based assessment. Of the 15 passage and question sets administered at grade 12 in 2019, eleven were transitioned from the 2015 paper-based assessment and four sets were newly developed.
As with the grades 4 and 8 assessments in 2017, a multistep process was used for the transition from the paper-based assessment to a digitally based assessment in order to preserve trend lines that show student performance over time. In addition to the digitally based assessment, a random subsample of students was administered the complete 2015 paper-based version of the grade 12 assessment in 2019. NCES administered the assessment in both modes—paper-based and digitally based—in all the sampled schools to investigate potential differences in performance between students taking the assessment on a tablet and students taking the paper-based assessment. Each participating student, however, took the assessment in only one mode. After the administration of the assessment at grade 12, NCES conducted rigorous analyses of the data and aligned the 2019 results to previous assessment years using a two-step process: common item linking and
common population linking.
These analyses—common item linking based on paper results and common population linking of paper results to digital results—enabled NCES to successfully maintain the reading trend line while transitioning to the digital assessment at grade 12 in 2019. The 2019 grade 12 reading assessment results were based on the combined performance of students who took the assessment on paper and students who took the assessment on tablets.
For more information, see the
grade 12 transition and mode evaluation.
Last updated 19 September 2024 (SK)
