Item-level schedules are developed collaboratively by item developers and subject area coordinators. The schedules are flexible and include
The item-level scoring schedules for the operational subjects to be scored are developed at a meeting several months prior to the administration and scoring of the assessment. Historical data for scoring speed and perceived difficulty of items and scoring are used to develop the item-level schedule. The number of scoring staff and scoring locations needed are estimated at this time, subject to revision as actual scoring nears. When possible, the schedule is made with open days to accommodate potential rescoring. NCES holds a scoring debriefing and planning meeting in Washington, DC to review the previous assessment scoring and to plan for the upcoming assessment scoring. Subject area specialists review a sample of papers for all new constructed-response items. Scoring staff print images of student responses. Across all subjects, 200 sample responses are provided for each NAEP item.
These sample responses are used for the selection of anchor papers, practice sets, and qualifying sets, and to assemble consensus sets. The training set samples are reviewed during standing committee meetings (a standing committee is composed of educators, subject matter experts, and curriculum specialists who oversee the development of an assessment) prior to scoring and are used to further refine final scoring guides prior to NAEP training and scoring.
The following table provides information about the scoring of NAEP assessments.
|Year||Subject area||Scoring center location||Start date||End date||Number of scoring supervisors||Number of scorers|
|Reading||Virginia Beach, VA||3/21/2011||5/6/2011||19||220|
|Virginia Beach, VA||5/9/2011||5/18/2011||4||53|
|2010||Civics||Mesa, AZ and Virginia Beach, VA||3/22/2010||4/28/2010||23||153|
|Geography||Mesa, AZ and Virginia Beach, VA||3/22/2010||4/28/2010||23||153|
|U.S. history||Mesa, AZ and Virginia Beach, VA||3/22/2010||4/28/2010||23||153|
|Reading||Virginia Beach, VA and Columbus, OH||3/23/2009||4/25/2009||30||336|
|2008||Arts||Iowa City, IA||6/2/2008||6/25/2008||6||57|
|Reading||Virginia Beach, VA and Iowa City, IA and Tucson, AZ||3/19/2007||4/21/2007||51||362|
|Writing||Virginia Beach, VA and Iowa City, IA and Tucson, AZ||4/3/2007||7/2/2007||50||328|
|2006||U.S. history||Mesa, AZ||3/27/2006||5/1/2006||21||65|
|Reading||Virginia Beach, VA and Iowa City, IA||3/21/2005||4/22/2005||36||363|
|Reading||Virginia Beach, VA||3/22/2004||4/9/2004||30||170|
|Reading||Virginia Beach, VA||3/17/2003||4/18/2003||27||340|
|2002||Reading||Virginia Beach, VA||3/18/2002||4/19/2002||20||200|
|Reading||Iowa City, IA||3/18/2002||3/25/2002||13||130|
|Writing||Virginia Beach, VA||4/22/2002||7/26/2002||20||180|
|Writing||Iowa City, IA||5/6/2002||5/31/2002||9||90|
|2001||Geography||Iowa City, IA||5/7/2001||5/23/2001||9||81|
|U.S. history||Iowa City, IA||5/7/2001||5/25/2001||9||81|
|Science||Iowa City, IA||3/13/2000||6/4/2000||16||115|
|NOTE: Data for 2011 mathematics, reading, science, and writing represent national and state assessments. Data for 2010 civics, geography and U.S. history were combined into one social sciences assessment; trainers and teams scored a combination of items from all three subject areas.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), Various Assessments, 2000-2012.
The following table provides information about scoring of NAEP's Long-Term Trend assessment.
|Year||Subject area||Age||Scoring center location||Start date||End date||Number of scoring supervisors||Number of scorers|
|2012||Reading long-term trend||9||Virginia Beach, VA||4/2/2012||4/13/2012||2||22|
|13||Virginia Beach, VA||3/05/2012||3/15/2012||2||20|
|17||Virginia Beach, VA||6/11/2012||6/22/2012||2||27|
|Mathematics long-term trend||9||Mesa, AZ||3/19/2012||4/3/2012||5||24|
|2008||Reading long-term trend||9||Virginia Beach, VA||3/24/2008||3/28/2008||2||14|
|13||Virginia Beach, VA||1/14/2008||1/25/2008||2||13|
|17||Virginia Beach, VA||6/9/2008||6/20/2008||2||16|
|Mathematics long-term trend||9||Mesa, AZ||3/28/2008||4/9/2008||1||12|
|2004||Reading long-term trend||9||Virginia Beach, VA and Iowa City, IA||3/3/2004||4/2/2004||3||27|
|13||Iowa City, IA||1/7/2004||1/23/2004||3||27|
|17||Iowa City, IA||5/17/2004||6/18/2004||3||27|
|Mathematics long-term trend||9||Iowa City, IA||3/3/2004||4/2/2004||4||35|
|13||Iowa City, IA||1/13/2004||1/23/2004||6||68|
|17||Iowa City, IA||5/17/2004||6/18/2004||4||34|
|SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2004, 2008, and 2012 Long-Term Trend Assessments.|