Skip to main content
Skip Navigation

Table of Contents | Search Technical Documentation | References

NAEP Processing Assessment Materials → Processing of Student Booklets and Questionnaires

NAEP Technical DocumentationProcessing of Student Booklets and Questionnaires

       

Receiving Student Booklets and Questionnaires

There are several stages of work involved in receiving and processing the documents used in NAEP paper-based assessment (PBA) and digitally based assessment (DBA). 

Starting in 2017, the DBA response data and questionnaires (received through the online questionnaire system) have been securely transferred to the materials, distribution, processing, and scoring (MDPS) staff. The materials distribution technology team processes the data, validates that the responses are provided as expected, and converts the responses into a more easily consumable format for the design, analysis, and reporting (DAR) staff. 

The NAEP MDPS staff create a set of predetermined rules and specifications for its processing departments to follow the PBA session materials. A variety of procedures are performed on materials received from the sampling and data collection (SDC) administrators before releasing these materials into their processing system. Control systems are used to monitor all NAEP materials returned from the field. The NAEP Process Control System (PCS) contains the status of sampled schools for all sessions and their scheduled assessment dates. As materials are returned, the PCS is updated to indicate receipt dates and to document any problems discovered in the shipments. As documents are processed, the system is updated to reflect processed counts. The materials report programs are utilized to allow NAEP MDPS staff to monitor the progress of the receipt control operations.

An alert process is used to record, monitor, and categorize all discrepant or problematic situations. Throughout the processing cycle, alert situations are either flagged by computer programs or identified during clerical check-in procedures.

Certain alerts, such as changes to administration codes on the administration form, are resolved by staff changing information due to notes written on booklet covers within the SDC's electronic preassessment visit system. These alerts, known as "Information Alerts," are recorded directly into the PCS system, thereby eliminating the need for paper documentation. Since these problem situations are categorized and tallied as they are entered in the PCS system, project staff are able to provide timely reporting on clerical-type errors made during test administration.

In cases where an alert situation cannot be resolved by the staff who first receive the materials, the alert form is forwarded to project personnel for resolution. An example of this type of alert would be a school that returned a shipment with no administration codes on either the booklet covers or the administration form. Once resolved, the problems and resolutions are recorded online by MDPS staff in the PCS system.

The Work Flow Management System (WFM) created for NAEP is used to track batches of student booklets through each processing step, allowing the MDPS staff to monitor the status of all work in progress. WFM is also used to analyze the current workload, by project, across all workstations. Through consistent monitoring of these data, the MDPS staff are able to assign priorities to various components of the work and to monitor all phases of the data receipt and processing.

After the PBA student booklets and questionnaires are processed, they are stored securely in the MDPS warehouse until NCES provides approval to securely destroy them. The electronic data (multiple-choice, open-ended responses, and human scores) captured from PBA and DBA are stored on a secure server until given approval by NCES to be securely destroyed.


Last updated 05 June 2023 (SK)