Skip to main content
Skip Navigation

Data Collection

sampling

Sampling

The 2018 NAEP ORF studyClick to open pdf. includes fourth-graders attending public schools. About 2,000 students from 220 schools from across the country were sampled. A nationally representative sample of 1,800 fourth-graders from 180 public schools participated, with a response rate of 88 percent. Public schools with over 75 percent of students eligible for the National School Lunch Program (NSLP) were oversampled. The sample was weighted to represent the total population of U.S. public school fourth-graders.

assessment

Assessment Administration

The 2018 NAEP Oral Reading Fluency assessment was administered using the new eNAEP platform on tablets to small groups of students at the same time in a single room, rather than with one-on-one administration. This enabled the assessment to be self-administered by the students in a group setting, with students wearing headsets with microphones. The group administration increased the efficiency of administration, yielded high-quality recordings, and provided a more relaxed testing atmosphere with fewer possible inconsistencies due to administrator interactions during the assessment.

The flowchart to the right shows the 2018 NAEP ORF study assessment administration procedure. The assessment started with a NAEP grade 4 reading assessment tutorial, followed by a reading assessment consisting of 8 out of 10 blocks of assessment items from the 2017 grade 4 NAEP reading assessment (two blocks were released). As is typical in the NAEP reading assessment, ORF students received two blocks of reading items. Then they responded to the ORF study task module with four ORF feedback questions. Lastly, they answered the NAEP fourth-grade student questionnaire for the reading assessment.

The ORF study task module consisted of four types of tasks in the following order: (1) repeating four sentences aloud to assess students’ speaking skills, (2) reading a word list aloud, (3) reading a pseudoword list aloud, and (4) reading four passages aloud with a single “yes/no” passage meaning question for each passage to encourage students to read for meaning. Before students were administered each task type, they were shown a tutorial that provided instructions with task-specific examples of a student demonstrating each upcoming task. After completing the four tasks, students answered the following four ORF feedback questions: (1) In this school year, how often have you read out loud in school or at home, or anywhere? (2) Who did you read to? (3) Where were you? and (4) How difficult was this reading out loud test?

Student recordings were transferred from the tablet to a NAEP database to be processed using a speech analysis system to produce a profile of scores to be analyzed for the study. For more information about the speech analysis system, see Scoring.

NAEP Oral Reading Fluency study assessment administration procedure: 2018
NAEP Reading Assessment
NAEP Grade 4
Reading Assessment
Welcome Screen and Tutorial
NAEP Grade 4
Reading Assessment
(2 reading blocks)
NAEP ORF Module 1
Sentence Repeat Task
Word Reading Task
Pseudoword Reading Task
Passage Reading Task with Passage Meaning Question
ORF Feadback Questions
NAEP Grade 4 Student Questionnaire
Thank You Message

Last updated 19 January 2023 (DS)