View Quarterly by:
This Issue | Volume and Issue | Topics
|
|||
| |||
This commentary represents the opinions of the authors and does not necessarily reflect the views of the National Center for Education Statistics. | |||
Greater Expectations and Needs for Education Data
With renewed emphasis on accountability in public education, it appears that the more we prove ourselves capable of doing—even with finite resources—the more we are expected to do. This is not a complaint. The expectation to do more is justifiably applied to teachers, students, education administrators, and education agencies alike, and it is necessary if the coming generation of Americans is to have the requisite skills to thrive in an increasingly high-tech and intricately networked world. The expectation of doing more with what’s available is therefore founded at least as much on need as on hope. And the fulfillment of such need can only be accomplished through effective partnering of education leadership at national, state, and local levels. This is especially true for those of us who work with education management information systems (MIS).
Uses and benefits of education data
The needs and expectations for more and better education data to support informed decision making at all levels also underlie the importance of National Center for Education Statistics (NCES) publications such as Projections of Education Statistics to 2010. In education planning and policymaking, we need to see where we’re heading in order to steer the course. The annual Projections report features the kinds of data (on enrollment, graduates, teachers, and expenditures) that are fundamental to assessing areas of greatest need and planning how best to allocate education resources in the near future. We need to know, for example, where growth in enrollment is most likely to occur (which grades or regions), whether there will be enough teachers to go around (and whether any shortages will be regional or nationwide, generalized or concentrated in specialized fields), to what extent overcrowded classrooms are likely to present problems, how much the costs of education are likely to rise, who will be able to afford higher education, and whether teachers will be adequately compensated for their work and performance (i.e., whether we will be able to attract and retain professionals for a competent teacher workforce). Moreover, states need comparative state-level data produced through a consistent methodology, such as the data provided by the maps and state-level tables in the Projections of Education Statistics series and several other NCES publications. State education agencies rely on a supply of such data to understand where they stand in relation to other states (for instance, to assess how states with similar demographics and/or economies have addressed common challenges). Further, state education agencies have already begun to benefit from cross-state information-sharing practices, from ongoing expansion of networking capabilities, and from movement toward greater compatibility among information management systems. For example, many states and districts have borrowed from other states in researching interactive Web-site report designs, distance learning initiatives, and other areas in education services, including database design, MIS, and comprehensive statewide school-level reporting. Hence, states are able to derive tangible benefits from comparable and comparative state-level data. As technological advances continue, state education agencies will increasingly be expected to use data from or about other states in modeling programs and initiatives to improve education locally. Advances in technology not only increase our ability to manage data, but also promote an even greater need for data. Success in answering a technically challenging question often generates a greater challenge to our capabilities.
Desire for accurate universe data
The public’s trust of education data, including statistical projections, may be affected by factors such as type of survey and level of error. Some staff in education information management have the sense that, for some policymakers, appreciation of the statistical usefulness of sampling research is subordinated to a desire to "have it all," statistically speaking. That is, these policymakers prefer data from universe surveys over sample surveys. They prefer not to deal with sampling errors or even the possibility of sampling errors. They prefer to conceive of school data as providing a kind of snapshot of conditions as they actually exist across actually exist across the universe of the school system at a given point in time. It is apropos that, in Florida and a few other states, our capability to provide accurate universe survey data has increased with the development of an integrated education information system that allows for the tracking of individuals by data elements (such as unique student I.D. numbers that students retain as they move from grade to grade, school to school, or district to district). This technology has already had positive applications in many areas within Florida’s public education system, including improved accuracy of reported graduation and dropout rates, as well as improved accuracy of baseline data for long-range projections. With recognition of these increased data management capabilities comes increased intolerance of nonsampling errors, however. As noted earlier, once the ability to execute is demonstrated, the expectation of flawless execution becomes ingrained—necessarily so. This is the attitude we must strive to uphold in preparing data that will affect major decisions about the allocation of resources toward education. Procedures to Ensure More Accurate Baseline Data
The greater the length of the forecast horizon, the greater the need to eliminate potential errors in baseline data, because any inaccuracies at the baseline perpetuate and even amplify themselves throughout the forecast. The effectiveness of the demographic and economic assumptions factored into NCES projections for enrollment, graduates, postsecondary enrollment, college degrees, teachers, and expenditures depends on the accuracy of the actual compiled data that form the starting point for the calculated projections. So, in a sense, data now being reported by school districts and compiled at the state level—especially if they are to be used for Common Core of Data (CCD) reporting—carry multiple burdens of responsibility: they have to be accurate for both the present and the future, for the benefit of one’s own state and other states as well. What may not be readily apparent are the often tedious but necessary quality-control processes that must occur at both state and local levels before data are submitted to federal statistical agencies, which in turn provide some of the source data for Projections of Education Statistics.
Automated quality-control measures
As states are collecting more data from schools and districts, we’ve been compelled to implement comprehensive automated quality-control measures in the data-reporting process. That is, data submitted from school districts to the state education agency’s database must comply with a series of edit rules to ensure that erroneously formatted data are not entered into the state system. The erroneous data are rejected (via reject rules), and the submitting district receives electronic notification of the rejected data so that the data may be resubmitted in correct format. At this level of reporting, greater efficiencies are achieved when (1) data elements have been adequately defined at the national and state levels and adequately communicated to local districts and (2) guidelines for data submission have likewise been adequately developed and communicated. While extensive guidelines and edit rules for data submission can eliminate errors in the formatting of data (a missing digit in a school number, a transposed digit in a birth date, etc.), other measures are required to ensure the accuracy of records that make it through the edit rules to reside in the state education agency’s database.
Review of data
In a sense, the accuracy of data—and hence, its utility—is only as good as the weakest link in the chain of reporting. Weakness that goes unchecked at any level of data transmission is perpetuated at every subsequent level. For instance, inaccurate enrollment data reported from a school to the district and from the district to a state-level database, if left uncorrected, may then affect the accuracy of the aggregated state enrollment data, which then are sent to a federal statistical agency to be compiled for the universe data on states. And these data may later be returned to the state in the federal agency’s published compilations and projections that include comparative state-level data. Strengthening the links between each level of data reporting and the next is therefore critical to improving the quality of data from which comparative results as well as projections are derived. At the same time, with the increasing volume and types of school data being processed, there must be some selectivity in determining how data submitted by schools and districts should be reviewed at each level prior to passing them to the next level and/or incorporating them in published reports. The selection of data for review depends on several factors, including the data’s impact on other forms of data, the expected visibility of the data, the audience for the data, and the data’s expected impact on decision making. Effective data review is dependent on communication between state and district personnel and typically includes (or should include) a process whereby targeted types of data (for example, indicators used in annual school reports) are compiled at the state level and then presented or made accessible to MIS staff, program staff, and principals in all school districts for verification prior to public reporting. In essence, we (at the state level) are saying to school districts: "This is what you sent us. Are you sure it’s correct?" At some point, however, there has to be a level of trust regarding the quality of data being reported, and that trust is fostered within the public education system by having MIS and other staff communicate effectively at and between the local and state levels. Within Florida, a new initiative is underway to conduct a series of data-review workshops in individual school districts for school district staff, with the objectives of troubleshooting reporting problems and increasing both the efficiency of reporting and the quality of initial data submissions. Shared Responsibility for and Benefits From High-Quality Data
At the national level, progress has been made in reducing nonsampling errors in data from universe surveys, thus increasing the scope and accuracy of national statistical data that can be used by policymakers and researchers at all levels. To the extent that we (at the state level) are able to provide more accurate universe survey information for national statistical surveys, which may in turn provide source data for projections, we may be able to contribute in some small way to the public acceptance and usefulness of projections of education statistics. Not only do the projections provide us with a complement to trend data we prepare for our own state’s schools, but they help us appreciate objectively where we reside among all states and where we are heading. |