Welcome to NCES StatChat Live. This is your entry point into a live online question and answer session.

The Nation?s Report Card ? Results from the 2006 NAEP Economics Assessment

Dr. Peggy G. Carr

Good afternoon, and welcome to our StatChat on the NAEP 2006 Economics report. I hope you?ve had time to examine the results and that I can answer any questions you may have. There were some interesting findings from our first economics assessment, so I?d like to get right to your questions.

George from Boston, MA asked:

Will this discussion be captured and made available at a later date for those of us unable to login to this chat live?

Dr. Peggy G. Carr 's response:

Yes, a transcript will be available later today.

Kelly from Rockville, MD asked:

Hi, I'm wondering when the next economics assessment will be conducted? Thanks.

Dr. Peggy G. Carr 's response:

The next economics assessment is scheduled for 2012, at which time we will have some notion of the trends in performance since 2006.

Sam from Cinnaminson, NJ asked:

When will the next Economics assessment be administered so that we can see if students are making any gains in this subject area?

Dr. Peggy G. Carr 's response:

Hi Sam, please see my response to Kelly from Rockville.

Joe from Pittsburgh, PA asked:

Could you explain the scoring system? Is it accurate to say that the average overall score was 150 on a 300-point scale?

Dr. Peggy G. Carr 's response:

Joe, your statement is accurate. The results are reported on a 0-300 scale. The average scale score was set at 150 for this, the first economics assessment.

Amanda from Satsuma, FL asked:

How are students chosen to be in the NAEP assessment?

Dr. Peggy G. Carr 's response:

Thanks for the question, Amanda. NAEP uses a fairly sophisticated set of procedures to select schools and students to participate in our assessments. The sample for the 12th-grade economics assessment was selected through a stratified, three-stage design that involved sampling students from selected schools within selected geographic areas across the country. In identifying the sample and determining which schools and students should be asked to participate, we were very careful to ensure that the final sample of over 11,000 students from nearly 600 schools was representative of 12th-graders nationwide--that way, we can be sure that the performance results we are reporting today truly reflect the performance of high school seniors across the country.

Joe from New York, NY asked:

What insight is there into the boy/girl disparity?

Dr. Peggy G. Carr 's response:

The 4 point difference between average scale scores of males and females might be the result of a variety of factors. Students completed background questionnaires that asked about themselves and their educational experiences. Researchers who are interested in examining this difference can explore the data by using the NAEP Data Explorer at http://nces.ed.gov/nationsreportcard/nde/

Yolanda from Los Angeles, CA asked:

Besides trying to improve reading, math and other general education aspects, what could be done to reduce the gap on economic knowledge that minority students suffer according to the study?

Dr. Peggy G. Carr 's response:

The answer to your question is in principal a matter of policy on educational resources geared towards minority students or, more to the point, low income and disadvantaged students regardless of race. Specific economics related efforts aimed at closing the gap could entail increasing exposure to economics principles and data both in and outside the classroom and to increased participation of disadvantaged students in economics education.

Sarah from Kensington, MD asked:

What percentage of students performed very well on this assessment?

Dr. Peggy G. Carr 's response:

The National Assessment Governing Board sets achievement levels for what students should know and be able to do. For the Advanced level, students should be able to identify and recognize an extensive set of economics concepts and relationships that are important for a thorough understanding of the market economy, national economy, and international economy. Three percent of 12th-graders scored at the Advanced level. Forty-two percent of high school seniors performed at or above the Proficient level, which is defined as identifying and recognizing a broader set of economic concepts and relationships that are important for a thorough understanding of the martket economy, national economy, and international economy.

Lucia from New York, NY asked:

What kind of launch will you have for this report? Are you planning on a formal presentation or press release?

Dr. Peggy G. Carr 's response:

Lucia-- We held a live release event this morning in Washington, DC. You can view the live event later this afternoon, by clicking on the webcast archive link on either http://nces.ed.gov/nationsreportcard/ or http://nationsreportcard.gov. The webcast archive link should be available after 4 today.

Martha from Indianapolis, IN asked:

How many and what types of questions were students asked on this test?

Dr. Peggy G. Carr 's response:

There are a total of 187 questions on the NAEP economics assessment. Of these, 32 questions required students to provide a written response--the others were formatted as multiple-choice questions. In addition, our economics assessment covers three assessment areas--market economy, national economy, and international economy. Of the total number of questions, 45 percent asked about the market economy, 40 percent were about the national economy, and 15 percent were about the international economy. Please keep in mind, however, that no student takes the whole test. Instead, we divide the entire set of questions in 25-minute timed sections that contain approximately 20 questions each. Each student that participated in the assessment received two of these sections.

Joe from Pittsburgh, PA asked:

Is there publicly available list of states and districts that took part in the assessment?

Dr. Peggy G. Carr 's response:

No, there is not a publicly available list of states and districts that took part in the assessment. This is because this particular assessment is a national sample, and is selected to represent the entire nation, not individual states or districts. States and districts were not sampled directly. Rather schools were selected randomly within geographic areas called PSUs. A list of the states and districts in which the sample schools are located is not available publicly for confidentiality reasons.

Jon from Irvine, CA asked:

When I took Economics in Grade 12, it was only a semester, and it was a complimentary course with Political Science. If I were to have taken Economics in my second semester (beginning in February), would I be categorized by NAEP as having taken a general course, or not?

Dr. Peggy G. Carr 's response:

One of the background questions asked students to indicate any economics-related course they had taken or were taking in Grades 9-12. So, in this case, the NAEP classification for your situation in twelfth grade would be both a combined course and a general economics course.

Jeff from Pleasanton, CA asked:

How were the achievement levels set?

Dr. Peggy G. Carr 's response:

The achievement levels in the Economics Report Card were adopted by the Governing Board based on a standard-setting process designed and conducted under a contract with ACT, Inc. for the 2006 assessment. To develop these levels, ACT convened a cross section of educators and interested citizens from across the nation and asked them to judge what students should know and be able to do relative to performance level descriptions developed as part of the framework.

Rachel from Denver, CO asked:

What is the potential impact of exclusion rates on scores?

Dr. Peggy G. Carr 's response:

I only have five more minutes for discussion... To answer your question, Rachel, the exclusion rate for economics is only 3 percent of the national sample. Therefore, the potential impact on national results of not testing those students is small.

Louise from Menlo Park, CA asked:

On the http://nationsreportcard.gov website, Sample Questions/Test Yourself is very helpful in understanding what seniors know about economics. Do you have plans to release more questions, and questions for other subjects?

Dr. Peggy G. Carr 's response:

After each NAEP assessment, about 1/3 of the test questions are released to the public. The remainder are kept secure so that changes in the performance can be measured over time. There are no plans to release additional economics questions until after the next additional economics assessment, which is planned for 2012. Additional reading and mathematics questions for grades 4 and 8 will be available after the next release this October. The next release of test questions for 12th-graders will be in writing in Spring, 2008.

Beck from Arlington, VA asked:

It seems that this Economics report covered a wide definition of ?Economics.? What types of Economics courses did students say they had taken?

Dr. Peggy G. Carr 's response:

The student background questionnaire in the assessment booklet includes a series of questions on student's economics course-taking. The courses included (a) Economics course (general), (b) Government and economics course, (c) Consumer economics or personal financial course, (d) Advanced Placement Economics course, (e) International Baccalaureate Economics course, (f) Honors economics course,(g) Business course such as entrepreneurship. (h) Any other economics course, and (i) any other course that includes an extended (at least 8-week long) unit on economics. Students are asked to indicate if they took the specific course, and if so, in which grades they took the class. Results for Economics course-taking is summarized in the Report Card (see page 8) available on the NCES/NAEP website. Complete information for economics course-taking can be found on the NAEP Data Explorer (NDE) at http://nces.ed.gov/nationsreportcard/nde/

Bob from Brookeville, MD asked:

Based on the percentage of 12th graders that scored in the proficient range in Economics in 2006, compared to say Math or Reading results from 2005, is it accurate to say that 12th graders know more Economics than Math?

Dr. Peggy G. Carr 's response:

Bob, we actually establish separate scales for each subject area that we assess. In addition, when the National Assessment Governing Board set its achievement levels (Basic, Proficient, and Advanced), they also set them separately for each subject area. Therefore, it is not really possible to directly compare mathematics or reading results to the results we released today in economics. That being said, it is true that all of the Governing Board's achievement levels are initially informed by the generic policy descriptions the Board has established for each of the three levels. (For example, Basic denotes partial mastery of the knowledge and skills that are fundamental for Proficient work at a given grade.) So--it is possible to make some very general comparisons across subjects--but, please keep in mind the problems in making direct comparisons I described above.

Kurt from Denver, CO asked:

With 42% of 12th graders at or above proficient in Economics in 2006, compared to less than 25% of 12th graders in Math (back in 2005)... do you think the 12th graders were more motivated to do well on the Economics assessment in 2006 than they are for Math in 2005?

Dr. Peggy G. Carr 's response:

Kurt, please see my response to Bob. Thanks.

Patrick from San Francisco, CA asked:

How would you explain the fact that students who didn't take any economics courses in school scored about the same as those who did?

Dr. Peggy G. Carr 's response:

Economics course-taking was asked in different questions. In one of the questions the results are as you describe ? students who did not take any economics courses performed about the same as students who did. The results of the other course-taking variable indicated that students who did take at least one course have an average score of 152, which is significantly greater than those who did not take an economics course, scoring, on average, 144. Therefore, the course-taking results are not straightforward, depending on the specific question. Given the data is self-reported, we cannot verify the accuracy of the reported values. In addition, some of the factors that might dilute a possible effect are that students who answered that they took an economics course might have done that in grade 9, 10, 11, or 12. Complete course-taking results can be found on the NAEP Data Explorer at: http://nces.ed.gov/nationsreportcard/nde/

Dr. Peggy G. Carr :
Thanks for all the excellent questions. Unfortunately, I could not get to all of them, but please feel free to contact NAEP staff or myself for more information. I hope that you found this chat helpful and the report interesting. Please visit the NAEP web site in the coming months for more information on our upcoming 2007 Reading and Mathematics reports for the nation and all 50 states.