Hello, this is Dr. Peggy Carr, Associate Commissioner of the National Center for Education Statistics (NCES). Welcome to Ask NAEP, an online Q&A session about the findings for the NAEP Trial Urban District Assessment (TUDA) 2009 Reading report. I hope you've had time to look at the results on the Nation's Report Card website, and that I can answer any questions you may have about them. I have a team of NAEP specialists here with me to make sure your questions are addressed thoroughly, so there may be a slight delay between the time you submit your question and the time the answer appears.
We are usually able to post answers for all questions, even after the Ask NAEP session ends at 3:30. If you cannot stay online for the full time, a complete Ask NAEP transcript will be available online after our discussion. I'm interested to hear which results you want to talk about.
bill from north kingstown rhode island asked:
- Dr. Carr
I am a first time participant and evaluator of NAEP data.
I would like to know what you consider as a significant difference in achievement (a half point, one point etc) when comparing data from one grade or class to another, school to school, district to district and/or even state to state. I am perplexed as to the indicator's that should be used to drive or mandate educational change.
I guess I'm saying "why should a successful education program that isn't broken be fixed" and what is an acceptable metric/yard stick (+/-) indicator to drive a small change or total change in regard to fixing a broken program.
Surely, you don't have to throw out the baby with the bath water.
Too often community change freaks (mostly those seeking political office) point to the need for change by comparing schools and/or school districts achievement data that in fact may not be sufficiently relevant.
Former School Committee Member
Dr. Peggy G. Carr's response:
- Bill, thanks for your interest in NAEP. It's important to keep in mind that we use the term "significant" in only a statistical sense. It is an indication of how reliable the results are — not an indication of their practical or educational significance. Whether two scores are significantly different is determined by conducting statistical tests. For more information about the statistical procedures used, you can refer to http://nces.ed.gov/nationsreportcard/tdw/analysis/infer.asp.
It may be important to keep in mind that NAEP results are just one indicator available to policymakers in districts and states for evaluating students' academic progress and making decisions about the need for educational change. It does play a unique role, however, by serving as a common yardstick across districts and states. State and district assessments, while not able to provide comparisons of results with other jurisdictions, in most cases produce scores for individual students and schools, which NAEP is not designed to provide. Therefore, NAEP results may be used in conjunction with other indicators to make educational policy decisions.
Joanne from Philadelphia PA asked:
- Why is it called the trial urban district assessment? What makes it a trial? Does that mean the results shouldn't be trusted as much as other NAEP results?
Dr. Peggy G. Carr's response:
- Hi Joanne- Thank you for your question. "Trial Urban District Assessment" is the term used for NAEP's focus on large school districts. The term "trial" is used because the NAEP program considers that some of its procedures may need further development. For example, this year we changed our sampling and reporting procedures to reflect that charter schools are accountable only to some urban school districts. In other districts, charter schools are completely independent of the urban district and should not be included in TUDA scores. Eventually, I expect the term "trial" will be dropped, which will be when the NAEP program reaches the point that its procedures have been fully worked out. I would also add that the NAEP state assessment program started out as a trial until independent evaluations of its procedures were deemed successful. As for whether you should trust the results, the analyses and procedures are conducted with the same rigor and attention to industry standards as the other assessments. So, the results from TUDA are both valid and reliable.
Charlie from DC asked:
- Can you explain why charter schools aren't included in NAEP any more?
Dr. Peggy G. Carr's response:
- As public schools, charter schools are included in NAEP results. For example, the state results that we released last month represented performance of students in all public schools—including students attending charter and non-charter schools.
Before 2009, we included charter schools in the TUDA samples if they were listed in NCES's Common Core of Data as part of the district, regardless of whether or not they contributed to the district's AYP results. Beginning in 2009, however, we only included charter schools that do contribute to AYP reporting for that district.
As we end our chat, I'd like to take this opportunity to thank all of the students, teachers and district staff that made the 2009 assessments possible. As we saw at this morning's press conference, the data we've released today certainly give our participating districts and the nation quite a bit to consider as they work to increase the educational achievement of our students.
Much more data and information is available on our website nationsreportcard.gov—including the NAEP Data Explorer, which allows you to run your own analyses on the data, and the NAEP Questions Tool, where you can explore thousands of released NAEP questions and associated information. You can also always contact us at http://nces.ed.gov/nationsreportcard/contactus.asp.
Dr. Peggy G. Carr:
Thanks for all the excellent questions. Unfortunately, I could not get to all of them, but please feel free to contact NAEP staff or myself for more information. I hope that you found this chat helpful and the report interesting.
Back to StatChat Home