Welcome to NCES StatChat Live. This is your entry point into a live online question and answer session.

The Nation?s Report Card — Results from the 2006 NAEP U.S. History and Civics Assessments

Dr. Peggy G. Carr

Good afternoon, and welcome to our Stat Chat on two new NAEP reports -- U.S. History 2006 and Civics 2006. I hope you?ve had time to examine the results and that I can answer any questions you may have. There are many findings in both reports for all three grades and I?m interested to hear which ones you want to talk about. So, let?s get right to your questions?

Lindsey from New York asked:

Why are there no state-level results for these reports?

Dr. Peggy G. Carr 's response:

The enabling legislation for NAEP calls for state-level reporting of student results in reading, mathematics, science, and writing. Large samples, representative of each state and the nation, are drawn when these subjects are assessed. In NAEP's other subjects, such as civics, U.S. history, geography, and the arts, smaller samples are drawn, which are representative of the nation, but not of individual states. Therefore, results for individual states are not available for either U.S. history or civics in these reports.

Heather from Maryland asked:

Hi there. I'm wondering how the decision was made to make two reports on seemingly similar subjects- civics and history. How did you differentiate the two initially?

Dr. Peggy G. Carr 's response:

Hello Heather. The National Assessment Governing Board considered these two subjects, U.S. history and civics, to be important enough to keep as distinct assessments. The 2006 NAEP U.S. history assessment measures how well students know the specific facts of American history, how well they evaluate historical evidence, and how well they understand changes and continuity over time. Four themes were measured: Democracy, Culture, Technological and Economic Change, and U.S. Role in the World. The 2006 NAEP civics assessment evaluated students? knowledge and skills in areas deemed important for citizenship in our constitutional democracy. The civics assessment had five content areas covering civic life in general, the values and principles underlying the American political system, the structure and functions of government under the Constitution, the relationships between the U.S. and other nations, and the roles of citizens in American democracy.

Rhonda from Sacramento, California asked:

What cities were the History/Civics samples drawn from?

Dr. Peggy G. Carr 's response:

Hi Rhonda, the samples of schools and students that participated in the assessments were drawn from throughout the 50 states and the District of Columbia, and included private as well as public schools. The sample is designed to represent the entire U.S. student population at each of the grades reported.

Jon from San Diego, CA asked:

Hello, Can you please explain what "Civics" encompasses? It seems like a dated term that has since been replaced with "Social Studies," which is also a cross-over term for "History." Why is it important that 4th-graders have an understanding of "Civics," if that subject is not explicitly included in most curricula?

Dr. Peggy G. Carr 's response:

NAEP civics in 2006 encompasses the following content areas as specified by the National Assessment Governing Board. The current framework was used to guide the 1998 and 2006 assessments. It does not have separately reportable subscales but requires assessment in five content areas. The framework poses the knowledge component as questions, reflecting the position that civic knowledge encompasses not just factual knowledge, but a broader and deeper understanding of the meaning of citizenship. The five civics content areas are: I. What are civic life, politics, and government? II. What are the foundations of the American political system? III. How does the government established by the Constitution embody the purposes, values, and principles of American democracy? IV. What is the relationship of the United States to other nations and to world affairs? V. What are the roles of citizens in American democracy? Civics concepts are taught within social studies beginning even before the 4th grade. It is important to begin the understanding of American democracy and the working of government even in these early grades. More complex concepts and details can then be covered in the upper grades. History emphasizes the chronology and development of concepts of American government. Civics emphasizes the actual working concepts and details of government at all levels.

Nadia from Roanoke, VA asked:

How can I see the questions asked? Some of them are in the report cards, but are there more that I can use in my classroom?

Dr. Peggy G. Carr 's response:

Nadia, All of our released questions are available at http://nces.ed.gov/nationsreportcard/itmrls/startsearch.asp. But, please note that these questions do not represent complete coverage of the content, cognitive skills, and range of difficulty in the NAEP assessment for any particular subject area. Some of the questions must be kept secure for use in future NAEP assessments.

Melissa from Toronto asked:

Hi, where can I find more sample questions for these subjects? I'd like to see more of what Civics entails.

Dr. Peggy G. Carr 's response:

Hi Melissa, please see my response to Nadia's question.

Kelly from Tampa, FL asked:

Why are there no content subscale results for the civics assessment?

Dr. Peggy G. Carr 's response:

The NAEP civics framework does not specify the creation of subscales in civics. However, the Governing Board?s framework does specify five content areas for the assessment, as shown in a previous question. Although there are five content areas for the civic assessment, items classified within the content areas were not scaled as a unit as items are when they comprise a subscale. This means that content area scores are not available for civics.

Juan from Des Moines, IA asked:

Who determines the content area questions for the assessment of students?

Dr. Peggy G. Carr 's response:

The objectives for each assessment were based on the framework developed and approved by the National Assessment Governing Board prior to the 1998 assessment.

Alex from East Orange, NJ asked:

Where can you find out more about the types of questions that are asked of our students in regard to history or civics? As a teacher, how deeply can I delve into your data tools to obtain information that might be useful to me?

Dr. Peggy G. Carr 's response:

Hi Alex, please see my response to Nadia's question.

Paul from Lancaster, PA asked:

It was 2001 to 2006 between the US History Assessments... and prior to 2006 Civics was last administered in 1998. When is the next History and Civics assessment scheduled, and will it be at the state level like Reading and Math?

Dr. Peggy G. Carr 's response:

Paul, the next civics and history assessment is scheduled for 2010, but no state level data will be reported.

Alka from Washington DC asked:

When will the next assessments in these subjects be given? When will the frameworks be revised for these subjects?

Dr. Peggy G. Carr 's response:

Hi Alka, please see my response to Paul.

John from Washington asked:

Does NAEP have any plans to make History and/or Civics mandatory, like Mathematics and Reading?

Dr. Peggy G. Carr 's response:

NAEP has no current plans to make these assessments mandatory. U.S. history and civics are scheduled to be assessed again in 2010. States are required to participate in the reading and mathematics assessments as a condition for receiving Title I funds, as part of the The No Child Left Behind legislation. All NAEP assessments, including reading and mathematics, are voluntary for students.

Ann from Olney MD asked:

Were you surprised by any of the findings?

Dr. Peggy G. Carr 's response:

Actually, the results for 4th-graders on these assessments were very similar to what was recently reported for the 2005 NAEP reading, mathematics, and science assessments. On the other hand the 12th-grade improvements in history represent a finding not seen in recent NAEP assessments.

Ava from Arlington, VA asked:

How will the results of the test be used in national and/or state policies?

Dr. Peggy G. Carr 's response:

I can't say exactly how, or even if, the results will be used in making policy. NAEP's role, and that of the National Center for Education Statistics, is to provide data to government and the public that they will hopefully find useful as they consider their educational priorities.

Tony from Arlington Virginia asked:

Why are the tests given only at grades 4, 8 and 12? Isn't it important for the nation to know how students are performing in other grades?

Dr. Peggy G. Carr 's response:

Legislation authorizing NAEP specified the grade levels to report on student achievement at key points during elementary, middle and high school. While monitoring achievement at more grade level could provide useful information, results from these three key points can inform decision-makers without overburdening schools and students.

Carol from Nebraska asked:

Are teachers involved in developing or reviewing the test questions?

Dr. Peggy G. Carr 's response:

Hello Carol, the simple answer is "yes." But let me describe how teachers fit into the assessment development process. Like every NAEP assessment, the U.S. history and civics assessments were based on a framework, or ?blueprint,? for the assessment. The National Assessment Governing Board, through a process of consultation with subject experts, school administrators, teachers, state education experts, and others, develops the framework and lays out test specifications based on the framework. The framework and test specifications are revised or replaced periodically to reflect what students are learning in schools. The actual questions used on NAEP assessments are guided by the test specifications. The questions are developed in consultation with classroom teachers, curriculum specialists, and test development specialists and are reviewed by the Governing Board. Before an actual assessment is given, the questions are field-tested. I hope this helps!

Tim from Macon, GA asked:

Hi. I have a question about subscales. I read on your website that U.S. history has four subscales. Is the increase from 2001 to 2006 similar across these scales? Where can I find more information about specific scales? Is there a way to compare these scales?

Dr. Peggy G. Carr 's response:

U.S history has 4 subscales by historical themes: Democracy; World Role; Technology; and Culture. Compared to 2001, in Democracy and World Role the mean scale scores at all three grade levels were higher in 2006. In Technology, scores at grade 12 are higher in 2006 than in 2001-in the other grades the scores are not statistically significantly different. In Culture, scores at grades 8 and 12 were higher in 2006 than in 2001. The content of the subscales is described in the assessment framework, available at http://www.nagb.org/frameworks/history_06.pdf. The subscale scores within a grade from year to year can be compared. However, because subscales were set separately for each theme, comparisons should not be made from one subscale to another.

Jessica from Palo Alto, CA asked:

Just a compliment: your Test Yourself feature is very nice (educational and fun!) Do you have Test yourself for other subjects?

Dr. Peggy G. Carr 's response:

Jessica, yes, we do have the Test Yourself feature for different subjects. Please see all Report Cards released on http://nationsreportcard.gov. Each Report Card features the Test Yourself feature on the Sample Questions page.

Ron from Louisville, KY asked:

I read in the NY Times that the US History shows far worse results in terms of percentage students getting a score below or at basic compared to other assessments (I think they quoted reading, math, science). To what extent can you compare between these assessments? Does a below basic level mean the same in History as in Reading or Math?

Dr. Peggy G. Carr 's response:

No, it does not mean the exact same thing in U.S. history as in the other fields. In each subject, the general meaning of "Proficient" is performance that indicates competency over challenging subject matter. The "basic" performance generally means what the content experts judge to be partial mastery of the knowledge and skills needed for the proficient level. However, translating the general, policy meaning of these terms into the specific terms of each subject depends on the judgments of experts in each field who evaluate performance on each test question. In each Report Card, fuller descriptions of the achievement levels are provided (see pages 19 for history and page 17 for civics). These should help.

Larry from Kansas City, MO asked:

Why have 12th grades students shown more improvement in the U.S. History assessment, but not on other NAEP assessments?

Dr. Peggy G. Carr 's response:

These NAEP reports do not give us a clear answer to this question. However, the 2005 High School Transcript Study, released a few months ago. It indicates that seniors have been taking more credits in social studies courses, which include history.

Lindsey from Castro Valley, CA asked:

It was reported today that there were gains in civics and history for 12th graders which had not been seen in some time for several NAEP subjects, what do you think that's attributable to?

Dr. Peggy G. Carr 's response:

Please see my response to Larry's question.

Emily from Wisconsin asked:

Is the test available to the public? How can we be sure that it isn't biased?

Dr. Peggy G. Carr 's response:

As with all NAEP assessments, there are a number of safeguards in place to assure that these assessments are of the highest possible quality and provide unbiased results. NAEP questions undergo an extensive review process, and are seen by a variety of program stakeholders and state personnel before they are considered for use on the assessment. All questions are reviewed and approved by staff at the National Center for Education Statistics and members of the National Assessment Governing Board. Once approved for use, all questions are tried out on small samples of students to evaluate the performance of the questions before being used in an assessment from which results are reported. The questions used on the final form of a NAEP assessment are carefully selected from a larger set that was tried out. As with every NAEP assessment, items and associated performance data are released from the assessments today along with the results, and there are items available that were released with the previous assessments as well. These are available online at http://nces.ed.gov/nationsreportcard/itmrls/.

Frank from Carolina Beach, NC asked:

Why did so many more 8th graders take US History, compared to 4th grade?

Dr. Peggy G. Carr 's response:

The U.S. history assessment at 8th grade has more extensive content than the 4th grade assessment. As a result, there are more assessment question and booklets at 8th grade than 4th grade, and the sample of students in the 8th grade must be larger than that at grade 4.

Nadine from Gilroy, CA asked:

Who decided what content should be tested at each grade level? How do I know that California students cover the same curriculum?

Dr. Peggy G. Carr 's response:

Congress decides what grade levels should be assessed. Please see my response to Juan's question as well.

Jack from Naples, FL asked:

Some years ago two bills were filed in Congress-s2721 in the 108th and s860 in the 109th Congress-that would have provided for the first state comparative date on how students performed in U.S. History. Neither bill passed, and, as a result, no state is ever held accountable for how well they prepare our students to be citizens, despite the fact they ARE compared in Reading, Math, Writing, Science and even in the effectiveness of their anti-tobacco, drug and alcohol programs. To what extent is not holding states accountable for the civic and historical mission of our schools causing student performance to lag in this vital area?

Dr. Peggy G. Carr 's response:

Last year, Senators Lamar Alexander, R-Tenn., and Edward M. Kennedy, D-Mass., sponsored legislation that would require that the history NAEP be expanded to include state-by-state results, starting with a 10-state pilot program. This would create a pilot program in 10 states to test 8th-and 12th-graders in U.S. history and civics. The legislation was offered in similar form last year but failed to advance. History and civics are periodically tested on NAEP now, but only as part of a more limited, nationwide sample of students in 4th, 8th, and 12th grades. Under the bill, the National Assessment Governing Board, which sets policy for NAEP, would determine 10 states that would participate in the pilot program.

Daniel from Washington DC asked:

There seems to be a wide range of competence - e.g., only 14% of seniors could explain why the US was involved in the Korean War, vs 70% could analyze a historical text. How do you explain this unevenness of knowledge?

Dr. Peggy G. Carr 's response:

NAEP assesses different item types. Some questions, such as multiple-choice questions, are scored as correct or incorrect. Other questions allow students to receive partial credit for their responses. The 14% you mention were the ones who received full credit (i.e., had a complete response). Other students received partial credit on the question. Therefore, it is difficult to compare across individual questions of different types. In addition, while information on individual questions is interesting and informative, we should not draw too many conclusions from specific questions. Instead, the individual questions serve as an example of the content that is assessed.

Robert from Brooklyn asked:

Is there score gap information for Black and Hispanic students for History and Civics?

Dr. Peggy G. Carr 's response:

There is information on score gaps in both of the reports and on the Web (see http://nces.ed.gov/nationsreportcard/). Gaps can also be explored using the NAEP Data Explorer (see http://nces.ed.gov/nationsreportcard/naepdata/). In 2006 in U.S. history we found that the score gap between White and Black fourth-graders decreased from the 1994 assessment, and the score gap between White and Hispanic fourth-graders also decreased in that time period. Score gaps between student groups for grades 8 and 12 didn't show any significant change. For Civics, the score gap between White and Hispanic fourth-grade students decreased since 1998. As with U.S. history, score gaps between student groups for grades 8 and 12 didn't show any significant change.

Lisa from Burlington, NJ asked:

Nice job on presenting the information. And I know that a lot of work went into producing these national reports for History and Civics--which were published this morning. Will school district results be released soon? I'm especially interested in results from students tested in my county. thanks.

Dr. Peggy G. Carr 's response:

Thanks for the kind words! Unfortunately NAEP cannot provide results for schools districts in these assessments, as they are based on a national sample. This sample is not representative of individual school districts.

Josiah from Miami, FL asked:

How are the 3 levels - Basic, Proficient, and Advance - that are used to report student achievement derived? So when you say, 73% of American students at the fourth grade achieved at or above Basic, how do we know what they have achieved?

Dr. Peggy G. Carr 's response:

NAEP achievement levels reflect what students should know and be able to do. They are set by the National Assessment Governing Board for each subject area and each grade- based on recommendations by a broad range of experts. They are reported as percentages of students who reach each level: Basic, Proficient and Advanced. Basic means partial mastery of the knowledge and skills that are fundamental to proficient work at each grade. Proficient means mastery of challenging subject matter. And Advanced means superior performance. You can get a good idea of what types of questions a student at any given achievement level should be able to answer by exploring the questions tool on our website, or on the tables pages 21, 25, and 29 of the U.S. history report and pages 19, 25, and 31 of the civics report. Both reports are also on the site. For example, in civics, 12th-grade students in the Basic achievement range should be able to identify the protection provided by the First Amendment, while students in the Proficient range should be able to apply a constitutional principle to a community conflict scenario.

Louise from Redwood City, CA asked:

How often are Civics and History assessed? Will you ever assess world history? Are state results available?

Dr. Peggy G. Carr 's response:

With respect to civics, here's the story: The first national assessments, administered by NAEP in 1969-70, were in science, writing, and citizenship (civics was considered part of citizenship). Civics has been assessed, either as part of a social studies assessment or solely on civics and government, in 1972, 1976, 1982, 1988, and 2006. The next civics assessments are planned for 2010 and 2014. With respect to U.S. history, the story is similar. NAEP carried out the U.S. history assessment in 1986, 1988, 1994, 2001, and 2006. As a result of the development and modification of the framework, comparisons of student performance can be made only for the 1994, 2001, and 2006 assessments. The next U.S. history assessments are planned for 2010 and 2014. An updated or new framework is planned for the 2014 assessment. The schedule calls for a world history assessment in 2012, but the framework has not yet been specified. There are no state data for U.S. history and civics. Congress determines through legislation that there will be state-level reporting of student results in reading, mathematics, science, and writing.

Lisa from Rhode Island asked:

Thank you for answering questions today. Does NAEP assess students' knowledge of world history?

Dr. Peggy G. Carr 's response:

The assessment results released today are for U.S. history only. However, NAEP has a world history assessment scheduled in 2012-the first time this assessment will be given. Keep an eye out for the results then!

Lou from Carrboro, NC asked:

I work with students with disabilities. I am concerned about how they can take these NAEP tests. How many do you have to exclude, and what accommodations can you offer?

Dr. Peggy G. Carr 's response:

Lou, I share your concern. We include as many students with disabilities who are selected into the NAEP sample as possible. In the U.S. history assessment, about 2-3 percent of all students were excluded because their schools indicated they could not participate meaningfully in NAEP because of their disabilities. NAEP allows students to use nearly all the testing accommodations they receive in their usual classroom testing, such as extended time, use of a scribe, and small-group testing (a complete list of accommodations can be found on the NAEP website, http://nces.ed.gov/nationsreportcard).

Adam from Brookeville, MD asked:

Why isn't History and Civics, or History, Civics, and Economics combined into one Social Studies Assessment?

Dr. Peggy G. Carr 's response:

Please see my response to Heather's question. Hope that helps!

Alisha from Baltimore, MD asked:

Who selects the questions for the assessment? Do students in the same grade answer the same questions?

Dr. Peggy G. Carr 's response:

NCES develops and selects the questions for each assessment based a specific framework developed by the National Assessment Governing Board. The Governing Board then approves the final set of questions.

Brett from Green Bay, WI asked:

As more data points come for History and Civics, will this be included in long term trends with new material being added to curriculum and classroom textbooks every year?

Dr. Peggy G. Carr 's response:

As additional assessments are conducted in History and Civics, these data points will be added to the history and civics trend lines. To address the second part of your question, about 1/3 of the test questions each year are released to the public and are replaced with new test questions. The new test questions will reflect more recent historical topics as long as they fit within the assessment framework developed by the National Assessment Governing Board. The assessment frameworks can be found at http://www.nagb.org. Also, the Governing Board has a goal of revising the frameworks about once every decade so that the assessments stay current with what is being taught.

Shannon from Seattle WA asked:

What suggestions do you have for local assessments of students in the social studies? How can we assess the students' ability to use critical thinking - to be able to apply concepts and facts they know? What alternatives do you see to the multiple choice type of test?

Dr. Peggy G. Carr 's response:

NAEP has several resources available for local teachers: one is the framework that indicates the content that the NAEP assessment covers, available from the National Assessment Governing Board (www.nagb.org). Another is the Questions Tool on the NAEP website (http://nces.ed.gov/nationsreportcard/itmrls/), which provides sample test questions that teachers can use. NAEP uses a variety of question formats, including multiple choice, short-constructed response, and extended-constructed response, in order to assess the full range of student content knowledge and critical thinking skills. All of these item types can be used to evaluate students' complex skills and critical thinking, if they are well constructed. It is important that the content and skills being assessed be matched with an item type that is best suited to allow students to demonstrate what they know and can do. In the social sciences, there are many exciting opportunities for students to use real historical source materials and visit historical sites. Role playing activities such as moot court, mock elections, cooking authentic foods from time gone by, or wearing costumes from historical periods all engage students' imagination and enthusiasm. Performance assessment components can be included in these types of activities, and students may not even notice they are being tested during the excitement!

Alan from Falls Church Virginia asked:

How were the questions for the U.S. History and Civics assessments chosen?

Dr. Peggy G. Carr 's response:

NAEP questions are developed by professional item writers according to the assessment framework and specifications set by the National Assessment Governing Board, available at http://www.nagb.org/. NAEP questions undergo an extensive review process, and are seen and approved by content experts, state personnel, NCES staff, and the Governing Board staff and board members before they are used with students. Once approved for use, all questions are tried out on small pilot samples of students to evaluate the performance of the questions. The questions used on the final form of a NAEP assessment are carefully selected from all the items that were tried out. All NAEP questions are evaluated for indicators of differential performance across student groups and the quality of statistical performance in addition to the content requirements of the framework. NAEP items are reviewed, tried out, evaluated, and tested themselves before they reach the stage of use in an operational assessment, undergoing more than 100 review stages before they are seen by students.

Frank from Salt Lake City, UT asked:

During the release this morning, Dr Schneider showed some scores for students with and without test accommodations. Are these scores adjusted based on the type of accommodation the student receives? Are the scores adjusted for the percentage of students included (with and without test accommodations) in the assessment? It doesn't seem fair otherwise. Thanks for helping me understand this better.

Dr. Peggy G. Carr 's response:

Frank, the results on the NAEP assessments are not adjusted for the percentage of students receiving accommodations. The purpose of the accommodations is to allow students to overcome the barriers that would otherwise exist to their being able to demonstrate the knowledge and skills. Through the provision of appropriate accommodations, the goal is to place all students-accommodated or not- on an equal footing with the assessment, with a comparable opportunity to demonstrate what they know and can do. Since the intention is to equalize the opportunity for student performance through the provision of appropriate assessment accommodations for each student according to their needs, there should be no need to adjust the scores in addition. In both U.S history and civics, there were few students who were excluded from participation in the assessment generally about 2-3%.

Dr. Peggy G. Carr :
Thanks for all the excellent questions. Unfortunately, I could not get to all of them, but please feel free to contact NAEP staff or myself for more information. I hope that you found this chat helpful and the reports interesting. Visit our web site at http://nces.ed.gov/nationsreportcard/ for the latest information on upcoming assessments and releases.