Commissioner, National Center for Education Statistics
National Assessment of Educational Progress
The Nation's Report Card: Trial Urban District Assessment: 2007 Reading & Mathematics Results at Grades 4 and 8
November 15, 2007
Today I am releasing the results of the 2007 reading and mathematics assessments from the National Assessment of Educational Progress for 11 urban school districts. We call these Trial Urban District Assessments, or TUDAs. These assessments were given earlier this year to 4th- and 8th-grade students at the same time as the national- and state-level assessments.
Background of the 2007 TUDA Mathematics and Reading Assessments
The TUDA program is a collaboration involving the National Center for Education Statistics, the National Assessment Governing Board, and the Council of the Great City Schools. TUDA was designed to measure performance of public school students at the school district level. TUDA is not yet a permanent part of NAEP.
In 2007, 11 districts from around the country were invited to participate in the program. The TUDA assessments began in 2002 with the participation of Atlanta, Chicago, Houston, Los Angeles, and New York City, as well as the District of Columbia. In 2003, Boston, Charlotte-Mecklenburg, Cleveland, and San Diego joined, and Austin was added in 2005. Participation in TUDA is voluntary.
Overview of the Trial Urban District Assessment (TUDA)
TUDA was administered as part of the regular NAEP assessment. Samples that ranged from about 900 to 2,800 students per district per grade participated in the assessments. Unlike the national assessment, which includes public and private school students, TUDA assesses public school students only.
Average scores for both the fourth and eighth grades are presented on a single 0 to 500 point scale for each assessment. These scores measure what students know and can do.
Results are also reported according to three achievement levels established by the National Assessment Governing Board. These levels set standards for what students should know and should be able to do. The three achievement levels are Basic, Proficient, and Advanced. Ultimately, the goal is to have all students performing at or above the Proficient level.
In reporting these results, we compare the districts to each other and to both public school students nationally and to those in large central cities-those having a population of 250,000 or more. We make this comparison because the demographic makeup of these 65 large central cities is closer to the TUDA district populations than is the nation as a whole.
TUDA districts vary widely in demographic composition, both from each other and the nation. For example, in the reading assessment, Black students made up about 17 percent of fourth-graders in public schools across the nation, but in the districts the percentages range from 11 percent in Los Angeles and San Diego to 86 percent in the District of Columbia. Hispanic students, in turn, made up about 20 percent of the fourth-grade population nationally, but ranged from 4 percent in Atlanta to 75 percent in Los Angeles.
Urban districts also tend to have larger percentages of lower-income students, as measured by eligibility for the National School Lunch Program. The percentage nationally was 45 percent, but for the districts it ranged from 48 percent in Charlotte to 100 percent in Cleveland. The urban districts also differ from the nation in the percentages of students who are English language learners or students with disabilities, two historically lower-performing student groups. For example, the percentage of fourth-grade students who were English language learners nationally in 2007 was 11 percent, but in the TUDA districts it ranged from 3 percent in Atlanta to 48 percent in Los Angeles. For students with disabilities, the national percentage was 14 percent, while the district range was 10 percent in Atlanta to 21 percent in Boston.
NAEP allows accommodations for both English language learners and students with disabilities, if required to participate. Even with accommodations, some students are excluded from NAEP, based on the decision of school officials, relying on NAEP guidelines and local practices. While the effect of exclusion on average performance is not precisely known, comparisons of performance results across districts could be affected if exclusion rates are comparatively high or vary widely over time. Exclusion rates for the districts vary. For example, the national exclusion rate for fourth-grade reading in 2007 was 6 percent, but in the TUDA districts exclusion rates ranged from 3 to 20 percent. Appendix tables A-2 and A-3 in the Report Cards display the percentages of students accommodated and excluded in each district.
In considering NAEP results in reading and mathematics, it is important to keep in mind that all NAEP scores are based on samples of students. For this reason, there is a margin of error associated with each score or percentage. When comparing scores and other NAEP results, I only cite differences that are larger than the margin of error.
Trend in Reading Scores, Grade 4
In the nation as a whole and in large central city schools across the nation, scores were higher in 2007 than in either the first assessment in 2002 or the last assessment in 2005. Since the first assessment, reading scores for fourth-graders increased in four of the six districts that participated in both years-Atlanta, Chicago, the District of Columbia, and New York City.
Comparing 2007 scores to 2005, Atlanta and the District of Columbia showed gains. In Houston, the average score declined. Scores in the other eight districts did not change significantly from 2005 to 2007.
In terms of percentages of students at or above the Basic achievement level, in comparison to large central city students nationally, the percentages in three districts were higher, three were not significantly different, and five were lower. These percentages ranged from a high of about 66 percent in Charlotte-Mecklenburg to a low of about 39 percent in Cleveland. The percentages at or above Proficient in these districts ranged from about 35 percent in Charlotte-Mecklenburg to about 9 percent in Cleveland.
Nation - District Gaps Narrower for Lower-Income Students, Grade 4
As noted earlier, NCES uses eligibility for the National School Lunch Program as a measure of income. Students eligible for free or reduced-price lunches are considered lower-income.
Unlike the nation as a whole, lower-income students constitute a majority of students in almost all the TUDA districts, frequently a large majority. Comparing the performance of lower-income students in the districts with those in the nation and large central cities can be more meaningful than comparisons of results for all students.
The performance gaps between the nation and the urban school districts tend to be smaller when only lower-income students are compared. In Cleveland, for example, the gap for lower-income students was 15 points less than the gap for all students. In New York City, lower-income students in that district performed above lower-income students in the nation as a whole.
White - Black Score Gaps, Grade 4
At the national level, the reading performance gap between Black and White students was 27 points in 2007. Among the 11 districts, the gaps range from 23 points in Cleveland to 67 points in the District of Columbia. The largest gaps tend to occur in those districts where White students outperform White students nationally.
White - Hispanic Score Gaps, Grade 4
The White-Hispanic score gap in the nation was 26 points in 2007, while the district gaps ranged from 15 points in Cleveland to 52 points for the District of Columbia. (In Atlanta, the Hispanic sample size at grade 4 was not large enough to report results.)
Trend in Reading Scores, Grade 8
Nationally, the average score in 2007 at grade 8 was higher than in 2005, while the large central city average did not show a significant difference.
At the district level, reading scores were higher in two of the five districts that participated in the first assessment-Atlanta and Los Angeles. Four districts scored higher since the last assessment-Atlanta, Cleveland, the District of Columbia, and Houston. In the other seven districts, there were no significant changes since 2005.
When we examine percentages of students at or above Basic in 2007, two districts had higher percentages than in large central cities as a whole, six were not significantly different, and three were lower. The district percentages at or above Basic ranged from a high of about 69 percent in Charlotte-Mecklenburg to a low of about 48 percent in the District of Columbia.
In terms of percentages of students at or above Proficient in 2007, the pattern of results is similar. The percentages ranged from about 29 percent at or above Proficient in Charlotte-Mecklenburg to a low of about 11 percent in Cleveland.
White - Black Score Gaps, Grade 8
Nationally, the performance gap between Black and White students was 26 points, while the district gaps ranged from 20 points in Cleveland to 46 points in Austin. In Atlanta and the District of Columbia, there were not enough White students in the eighth grade to report results.
White - Hispanic Score Gaps, Grade 8
The national gap between White and Hispanic performance was 25 points and the district gaps ranged from 11 points in Chicago to 40 points in Austin. These results are not reportable for the District of Columbia because of the small number of White students. In Atlanta, both the White and Hispanic eighth-grade sample sizes are too small to report results.
Trend in Mathematics Scores, Grade 4
There have been three TUDA mathematics assessments, in 2003, 2005, and 2007. All 11 districts participated in 2003 except Austin, which joined in 2005.
Performance in mathematics at grade 4 improved in the nation and in the large central cities in 2007, compared to the two prior assessments. At the district level, scores increased since the first assessment in 2003 for 8 of the 10 participating districts-all except Charlotte-Mecklenburg and Cleveland. Since the last assessment, in 2005, four districts had higher scores-Atlanta, Boston, the District of Columbia, and New York City-and in one district, Cleveland, the average score declined. For the remaining six districts, there was no significant change.
Comparing percentages of students at or above Basic shows six districts higher than the large central city percentage, and five lower. The range was from about 85 percent at or above Basic in Charlotte-Mecklenburg to about 49 percent in the District of Columbia. In six districts, at least 74 percent of students were at or above Basic. Charlotte-Mecklenburg's percentage at or above Basic was higher than the national percentage but most of the districts were below it.
Comparing percentages of students at or above Proficient shows four districts higher than the large central city percentage, two not significantly different, and five lower. In the top four districts-Charlotte-Mecklenburg, Austin, San Diego, and New York City-over one-third of students were at or above Proficient.
National-District Score Gap, Grade 4
Comparing the gaps between national and district scores shows smaller gaps when scores for lower-income students only are compared. In Boston, Houston, and New York City, student performance shifts from below the national average when all students are included to above the nation when only lower-income students are considered.
Score Changes by Race/Ethnicity, Grade 4: 2003 and 2007
In Boston and New York City, White, Black, Hispanic, and Asian and Pacific Islander students (the race/ethnic groups for which there were sufficient samples to report results) all had higher average scores in 2007 than in 2003. The other districts showed either increases or no significant change for the various race/ethnic groups. (No trend comparison could be made for Austin because that district did not participate in 2003.)
Trend in Mathematics Scores, Grade 8
At grade 8, in 2007 both the national and the large central city average scores were higher than in either the 2003 or 2005 assessments. Eight urban districts showed higher scores since the first assessment-all except New York City and Cleveland. Since the last assessment, six districts achieved higher scores. There were no declines for any district.
Comparing percentages of students at or above Basic, five districts were higher than the large central city percentage, one not significantly different, and five lower. The top five districts had at least 62 percent of students at or above Basic, while the lower five had no more than 49 percent. The percentages of students at Basic and above in Austin and Charlotte-Mecklenburg were not significantly different from the nation's, while the rest of the districts were below the nation.
Comparing percentages of students at or above Proficient shows three districts higher than the large central city percentage, three not significantly different, and five lower. The top three-Austin, Charlotte-Mecklenburg, and Boston-had at least 27 percent of students at or above Proficient, while the lower five had no more than 14 percent. Austin's percentage at or above Proficient was above the nation's, Charlotte-Mecklenburg's was not significantly different, and the percentages in the other districts were lower.
Score Changes by Race/Ethnicity, Grade 4: 2003 and 2007
Examining score changes since 2003 by race/ethnicity, Boston, Houston, Los Angeles, and San Diego all showed improvement for three of the four race/ethnic groups for which there were sufficient samples to report results (White, Black, Hispanic, and Asian and Pacific Islanders). Chicago, Cleveland, and New York City did not show any increases by race/ethnicity. However, no district showed a decline for any group. Austin did not participate in 2003.
Both the reading and mathematics TUDA Report Cards feature profiles of student performance for each district, by both grade and subject. These profiles include average scores in each district and its state over time; district scores according to racial/ethnic group; scores for lower-income students for the district and the nation; and achievement-level performance over time for the district, and for both the nation and large central cities in 2007.
For More Information
There is much more information in the Report Cards themselves. In addition, the initial release website provides:
Commissioner Mark Schneider's Powerpoint Presentation:
The Nationís Report Card: Trial Urban District Assessment Mathematics and Reading 2007 (1.70 MB)
In closing, I would like to thank all the students and schools that participated in these assessments.