Acting Commissioner, National Center for Education Statistics
National Assessment of Educational Progress
The Nation's Report Card: Mathematics 2009
Trial Urban District Assessment
December 8, 2009
Today I am releasing the results of the NAEP 2009 Trial Urban District Assessment for mathematics—what we call TUDA.
TUDA assesses the performance of public school students at the district level. It is a collaboration involving the National Center for Education Statistics, or NCES, along with the National Assessment Governing Board, and the Council of the Great City Schools.
Participation in TUDA is voluntary. In 2009, 18 districts from around the country were invited to participate, and all 18 agreed to do so. TUDA is not yet a permanent part of NAEP.
Because the NAEP assessments are the same for the nation, the states, and the urban districts, NAEP serves as a common yardstick for comparison.
Our list of participating districts has been growing steadily since the first TUDA in 2002. The first TUDA mathematics assessment occurred in 2003. In that year, 10 urban school districts participated:
Our samples for the 2009 mathematics assessment ranged from about 900 to 2,200 students per district per grade. The variation in sample size among districts is large because the districts themselves differ dramatically in size. Nevertheless, for all of our districts, regardless of size, we have samples that provide reliable results for comparisons both over time and among districts.
There is one change affecting the TUDA samples for 2009. In the past, charter schools located in a district were included in the district's results if the schools were listed as being under the district's jurisdiction according to the Common Core of Data maintained by NCES. But beginning in 2009, the TUDA samples only include those charter schools that each district includes for the purposes of reporting Adequate Yearly Progress (AYP) to the Department of Education under the Elementary and Secondary Education Act. This change has had minimal impact on the results for most of the TUDA districts.
We report student performance in two ways: scale scores and percentages at or above achievement levels. NAEP scale scores indicate what students know and can do, and are reported on a scale ranging from 0 to 500 for mathematics. The achievement levels were developed by the National Assessment Governing Board, which sets standards for what students should know and be able to do. For each subject and for each grade, the Governing Board has established standards for Basic, Proficient, and Advanced performance. Ultimately, the goal is to have all students performing at or above the Proficient level.
When comparing scale scores and other NAEP results we only discuss differences that are statistically significant.
For the most part, I will compare urban districts' performance to the national average score for public school students and the average score for public school students in large cities—defined as those cities with a population of at least 250,000. We compare the TUDA results to the large city average because the student populations of our large urban districts are more similar in composition to these populations than to students nationally.
TUDA districts often vary from one another, as well as from the national population. For example, while about 16 percent of the nation's students at grade 4 were Black, in the 18 TUDA districts the percentages ranged from 7 percent to 87 percent, in Los Angeles and Baltimore, respectively. Nationally, about 22 percent of 4th-graders were Hispanic. But the district percentages ranged from 3 percent in Baltimore to 77 percent in Los Angeles.
Also in the 4th grade, about 10 percent of students nationally were English language learners. The range in the districts was from 2 percent in Atlanta and Baltimore, to 41 percent in Los Angeles. And while the national average for students from lower-income families was 48 percent, the range in the districts was from 47 percent in Charlotte to 100 percent in Cleveland.
The large city percentages for these four groups of students were higher than the national percentages, reflecting the heavy concentration of these students in the TUDA districts.
Grade 4 Results
In describing performance in mathematics for grade 4, I'll first discuss trend results for the 11 TUDA districts that participated in previous assessments.
When comparing scores from 2009 to the scores from 2003 and 2007, two districts—Boston and the District of Columbia—had significantly higher scores in 2009 than in 2007, with increases of 3 and 6 points respectively. The remaining nine districts showed no significant change. When compared to 2003, scores were higher for 8 of the 10 districts for which we have data. Charlotte and Cleveland are the two districts that did not have an increase. The large city average also increased since 2007, while the national (public) average did not.
When examining results at different percentiles (the 10th, 25th, 50th, 75th, and 90th), scores for grade 4 public school students nationally showed no change from 2007 to 2009. Results for public school students in large cities showed a significant increase at the 10th and 25th percentiles since 2007. Thus, while scores in grade 4 mathematics did not increase nationally for low-performing students, they did increase for those students in large cities.
Now I'll discuss the 4th grade score results for all 18 districts in 2009. One district, Charlotte, had a higher average score than the nation. Seven districts (Austin, Boston, Charlotte, Houston, Miami-Dade, New York City, and San Diego) had scores higher than large cities. One district, Jefferson County, had an average score that was not significantly different from the large city average, but was below the national average. The remaining 10 districts had scores lower than both the nation and large cities.
Turning to the 4th grade achievement level results, we see that seven districts had larger percentages of students at or above Basic than in large cities as a whole. They were Charlotte, Austin, Houston, Miami-Dade, Boston, New York City, and San Diego. In one district—Jefferson County—the percentage of students at or above Basic was not significantly different from the large city percentage. The remaining 10 districts were lower than the large city percentage. It should be kept in mind that all of the districts had some students who performed at Proficient and Advanced, ranging from about 45 percent in Charlotte to about 3 percent in Detroit.
When examining the score results for an urban district as a whole, it is also informative to consider the performance of particular demographic groups within the district as well. We most often examine performance by racial / ethnic group and eligibility for the free/reduced-price lunch program, which is used as an indicator of low income. Results for these groups sometimes differ from the overall district results.
For example, in three districts—Charlotte, Houston, and New York City—scores for White, Black, and Hispanic students were all above the national averages for those groups of students. However, only Charlotte's overall average score was above the national average. Houston's overall score was below the national average, and New York City's overall average score was not statistically different from the national average.
The result for Houston may seem particularly counter-intuitive, because all major racial/ethnic groups, including Asian/Pacific Islander students, had higher average scores than their peers nationally. How could Houston score below the nation overall when all the racial/ethnic groups were above average? The answer is that Houston has high proportions of Black and Hispanic students. These students made up about 89 percent of Houston's 4th-graders compared to 38 percent nationally. Since these students, on average, tend to score lower than White and Asian/Pacific Islander students, when the student group performance is aggregated, Houston's overall score was lower than the national average.
For Austin, Boston, and Miami-Dade, we also see how higher performance for individual groups can still add up to performance that is comparable to or lower than the national average overall. White and Hispanic students in Austin and Miami-Dade, and Black and Hispanic students in Boston, outperformed their peers nationally, even though the overall average for Austin was comparable to the national average, and the overall scores for Miami-Dade and Boston were below it. In these three districts, Black and Hispanic students made up at least 70 percent of the grade 4 student populations.
In four districts—Cleveland, Detroit, Fresno, and Philadelphia—students in all major groups scored below their peers nationally.
It should be noted that even in the districts where scores for Black or Hispanic students were high, the performance gaps for these students, as compared to White and Asian/Pacific Islander students, remained large. And in no district were the scores for Black or Hispanic students above the overall national average.
Grade 8 Results
Now I'll describe performance in mathematics at grade 8.Scores were higher in 2009 than in 2003, the year of the first TUDA mathematics assessment, for 9 of the 10 districts for which we have data for both years. Cleveland did not show an increase.
Over the more recent period, two districts' scores were higher in 2009 than in 2007—Austin with a 5-point increase, and San Diego with an 8-point increase. The large city average and the national average also increased over those two years.
For the District of Columbia TUDA, we did not see an increase since 2007, even though we reported an increase for the District in our 2009 Mathematics Report Card, released earlier this year, where the District of Columbia was treated as a state. The reason is that, beginning with 2009, there are two different samples for the District of Columbia, one for state NAEP and one for TUDA. I mentioned earlier that charter schools not included in the districts' AYP reporting were not part of the samples for the 2009 TUDAs. Thus, all of the District of Columbia's charter schools were excluded from their 2009 sample in the TUDA assessment. But when defined as a state, the charter schools were included in the District of Columbia's results. For the District of Columbia TUDA, we compared a 2007 sample that contained charter schools with a 2009 sample that does not contain them. If the charter schools were removed from the 2007 sample as well, the District of Columbia would have shown an increase in average score from 2007 to 2009, and the District of Columbia would be the only TUDA to show increases in both grades 4 and 8 since 2007.
Examining the 8th grade score results for all 18 districts in 2009, we note that one district, Austin, had a higher average score than the nation. Five districts (Austin, Boston, Charlotte, Houston, and San Diego) had scores higher than large cities. Three districts—Jefferson County, Miami-Dade, and New York City—had an average score that was not significantly different from the large city average. The remaining 10 districts had scores lower than both the nation and large cities.
Comparing the districts to the large cities in their percentage at or above Basic, six districts had higher percentages than the large cities. They were Austin, Boston, Charlotte, Houston, Miami-Dade, and San Diego. Two districts—Jefferson County and New York City—were not significantly different from the large city percentage. The remaining 10 districts were lower than the large city percentage.
As with grade 4, all the districts had some students performing at Proficient and Advanced, ranging from 39 percent in Austin to 4 percent in Detroit.
I discussed the 4th grade results by race/ethnicity, which are also available for 8th grade in the report. I'd like to highlight the 8th grade results for lower-income students, defined by their eligibility for free or reduced price school lunches. (The income eligibility level was $39,000 for a family of four.) I'll compare the 8th grade scores of lower-income students in each district with the average score for these students nationally. As with the race/ethnicity comparisons at grade 4, these comparisons illustrate that performance patterns for lower-income students do not always mirror overall results.
The grade 8 national overall score, including both eligible and ineligible students, was 282. Austin's overall score was higher than this national average, while the scores for Boston, Charlotte, and San Diego were comparable to it, and the scores for the remaining 14 districts were below it.
The average national score for lower-income students was 266. In Austin, Boston, Houston, and New York City, scores were higher for lower-income students than the average score nationally for these students. In Charlotte, Miami-Dade, and San Diego, scores were comparable to the national average, while in the remaining 11 districts, scores were lower. Note that these school districts vary in their concentration of lower-income students.
In all of the 18 districts, the percent of eligible students was higher than the national percentage of 43 percent. In Charlotte, for example, about 46 percent of students were eligible for free or reduced-price lunch, while in Chicago, Fresno, and Cleveland, over 85 percent of students were eligible.
For More Information
That completes my overview of results from the 2009 TUDA mathematics assessment. There is much more information available in the Report Card. In addition, the initial release website will give you extensive information on the performance of students in each district and access to released assessment questions using the NAEP Questions Tool, and to the NAEP Data Explorer, our online data-analysis tool.
In closing, I would like to thank the students, schools, and school districts that participated in these assessments.