Survey Instruments -> Assessment Booklets

Of Special Note

The primary goal for developing 7 new blocks was to update the pool of questions to reflect NAAL's new assessment design. For example, 5 oral tasks were eliminated as were 13 multiple-choice questions (the 2003 NAAL is 100 percent open-ended, requiring short-answer responses).

For those blocks that were repeated from the 1992 NALS, the order of tasks within those blocks remained fixed in order to preserve trend data. If the order was changed, it would influence a respondent's performance on the particular block of questions.

One goal of the 2003 NAAL was to capture the full content domain of adult literacy. As a result, the main assessment had a large number of assessment questions. Of the 152 questions, 65 were repeated from the 1992 NALS (to allow for trend comparisons) and 87 were newly created for the 2003 NAAL.

The 152 NAAL assessment questions were assembled into 26 booklets, each utilizing a Balanced Incomplete Block (BIB) spiraling approach. Also known as matrix sampling, this approach broke a large set of tasks into smaller sets or blocks. Each of these blocks contained a balanced sample of tasks from each of the three types of literacy.

• There were 13 blocks in all: seven new blocks of 2003 questions; and 6 blocks of 1992 questions.
• The 13 blocks were spiraled together into 26 booklets (view block design for 26 booklets). Most of the booklets included at least 1 block from the 2003 NAAL and at least 1 block from the 1992 NALS (for trend analysis).
• Each respondent was administered one booklet, which contained 3 blocks of tasks, in addition to the same set of 7 relatively easy core screening questions (which were the first block in every assessment booklet).
• Each block had 7 to 11 tasks and 4 to 6 stimulus materials and took about 15 minutes to complete.

---

SOURCE: U.S. Department of Education, National Center for Education Statistics.

Top