Common Core of Data (CCD)

5. Data Quality and Comparability

The data in the CCD are obtained from the universe of SEAs, which are provided with a common set of definitions for all data items requested. In addition, for the CCD fiscal surveys, NCES provides crosswalk software that converts a state’s existing accounting reports to the federal standard, as indicated in Financial Accounting for Local and State School Systems, 2009 Edition (Allison, Honegger, and Johnson 2009). This ensures the most comparable and comprehensive information possible across states. As with any survey, however, there are possible sources of error, as described below.

Sampling Error

Because the CCD is a universe survey, its data are not subject to sampling errors.

Nonsampling Error

Coverage error. An NCES report by Owens and Bose (1997), found that overall coverage in the 1994–95 Local Education Agency Universe Survey was 96.2 percent of that in state education directories. “Regular” agencies—those traditionally responsible for providing public education—had almost total coverage. Most coverage discrepancies were attributed to nontraditional agencies that provide special education, vocational education, and other services.

Nonresponse error

Unit nonresponse. The unit of response in the CCD is the SEA. Under current NCES standards, the regular components of the CCD are likely to receive at least partial information from every state, resulting in a 100 percent unit response rate.

Item nonresponse. Any data item missing for one school district is generally missing for other districts in the same state. The following items have higher than normal nonresponse: free–lunch–eligible students by school; nontraditional agencies; and dropouts. Some states assign all ungraded students to one grade and therefore do not report any ungraded students.

Several items have shown marked improvement in response during recent years. Student enrollment was only reported for 80 percent of the districts in 1986–87, but is now available for almost 100 percent. Reports of student race/ethnicity at the school level has increased from 63 percent in the 1987–88 school year (when first requested) to nearly 100 percent today.

Measurement error. Measurement error typically results from varying interpretations of NCES definitions, differing record keeping systems in the states, and failures to distinguish between zero, missing, and inapplicable in the reporting of data. NCES attempts to minimize these errors by working closely with the state CCD coordinators.

Definitional differences. Although states follow a common set of definitions in their CCD reports, the differences in how states organize education lead to some limitations in the reporting of data, particularly regarding dropouts. CCD definitions appear to be less problematic in the NPEFS, although data on average daily attendance in this survey are not comparable across states. States provide figures for average daily attendance in accordance with state law; NCES provides a definition for states to use in the absence of state law. Because of this lack of comparability, student membership counts from the State Nonfiscal Survey are used as the official state counts.

Because not all states follow the CCD dropout definition and reporting specifications, dropout counts cannot be compared accurately across states. For states that do not comply with the CCD definition, the dropout count is blanked out in the database and considered missing. Currently, there is considerable variation across local, state, and federal data collections on how to define dropouts. The CCD's definition differs from that in other data sources, including the Education Longitudinal Study of 2002, the High School Longitudinal Study of 2009, and the Current Population Survey (CPS), conducted by the Census Bureau. Although the collection of dropout information in the CCD is designed to be consistent with procedures in the CPS, differences remain. CCD dropout data are obtained from state administrative records (whereas the CPS obtains this information from a household survey). The CCD includes dropouts in grades 7 through 12 (whereas the CPS includes only grades 10 through 12).

States also vary in the kinds of high school completion credentials on which they collect data. Some states issue a single diploma regardless of the student's course of study. Others award a range of different credentials depending upon whether the student completed the regular curriculum or addressed an individualized set of education goals. Unreported information is shown as missing in CCD data files and published tables unless it is possible to impute or adjust a value.

Changes in state reporting practices. The basic characteristics of a school or district do not change frequently. However, a minor change in local or statewide reporting practices (such as two or three coordinators instructing schools to review all of their general information) can have a large impact on the reliability and validity of CCD items. In the 1990–91 school year, a significant proportion (7 percent) of schools, primarily in three states, reported a change in locale code from the prior survey. While this undoubtedly provided better information on school locales in these states, data became less comparable across years. Such changes are rare, however, and tend to be clustered by state and year.

Data Comparability

Most CCD items can be used to assess changes over time by state, district, and school. However, checks of the prevalence and patterns of nonresponse should be performed to assess the feasibility of any analysis. There may also be discontinuities in the data resulting from the introduction of new survey items, changes in state reporting practices, etc., and there may be inconsistencies across reporting levels in the numbers for the same data element (e.g., number of students).

Content changes. As new items are added to the CCD, NCES encourages states to incorporate into their own survey systems the items they do not already collect so that these data will be available in future rounds of the CCD. Over time, this has resulted in fewer missing data cells in each state’s response, thus reducing the need to impute data. Users should keep in mind, however, that while the restructuring of data collection systems can produce more complete and valid data, it can also make data less comparable over time. For example, prior to FY 89, public revenues were aggregated into four categories and expenditures into three functions. Because these broad categories did not provide policymakers with sufficient detail to understand changes in the fiscal conditions of states, the survey was expanded in 1990 to collect detailed data on all public revenues and expenditures within states for regular education in prekindergarten through grade 12.

Comparisons within the CCD. A major goal of the CCD is to provide comparable information across all surveys. The surveys are designed so that the schools in the Public School Universe survey are reflected in the Local Agency Universe survey and so that the data from these surveys are reflected in the State Nonfiscal survey. While counts may not always be equal across reporting levels or even within the same level, differences should be consistent and explainable.

For the most part, the total number of students in a regular district is close to the aggregated number of students in all of the district’s schools. Since 1990, there has typically been agreement between these counts in at least 85 percent of the districts. Membership numbers in the Public School Universe and Local Agency Universe surveys may legitimately differ if (1) there are students served by the district but not accounted to any school (e.g., hospitalized or homebound students); or (2) there are schools operated by the state Board of Education rather than by a local agency. To avoid confusion, NCES publishes the numbers of students and staff from the State Nonfiscal Survey as the official counts for each state.

Top