There are several stages of work involved in receiving and processing the documents used in NAEP assessments. The NAEP materials staff creates a set of predetermined rules and specifications for its processing departments to follow. Project staff perform a variety of procedures on materials received from the assessment administrators before releasing these materials into its processing system. Control systems are used to monitor all NAEP materials returned from the field. The NAEP Process Control System (PCS) contains the status of sampled schools for all sessions and their scheduled assessment dates. As materials are returned, the PCS is updated to indicate receipt dates and to document any problems discovered in the shipments. As documents are processed, the system is updated to reflect processed counts. The materials report programs are utilized to allow NAEP staff to monitor the progress of the receipt control operations.
An alert process is used to record, monitor, and categorize all discrepant or problematic situations. Throughout the processing cycle, alert situations are either flagged by computer programs or identified during clerical check-in procedures.
Certain alerts, such as changes to administration codes on the administration schedule, are resolved by staff changing information due to notes written on booklet covers. These alerts, known as "Information Alerts," are recorded directly into the PCS system by opening personnel, eliminating the need for paper documentation. Since these problem situations are categorized and tallied as they are entered in the PCS system, project staff are able to provide timely reporting on clerical-type errors made during test administration.
In cases where an alert situation cannot be resolved by the staff who first receive the materials, the alert form is forwarded to project personnel for resolution. An example of this type of alert would be a school that returned a shipment with no administration codes on either the booklet covers or the administration schedule. Once resolved, the problems and resolutions are recorded online by project staff in the PCS system.
The Work Flow Management System (WFM) created for NAEP is used to track batches of student booklets through each processing step, allowing project staff to monitor the status of all work in progress. WFM is also used by the materials staff to analyze the current workload, by project, across all workstations. Through consistent monitoring of these data, management staff is able to assign priorities to various components of the work and to monitor all phases of the data receipt and processing.