The comforting adage, "the data speak for themselves," is often untrue in real life (see exhibit 5.1). In the complicated world of education data, answers to even apparently straightforward questions often depend on complex data. The "simple" question in chapter 1 illustrates this point: How many eighth grade English teachers are in your schools? This type of inquiry prompts a prudent school leader to ask other, even tougher, questions, such as:
Some organizations rely on the experience of their data steward(s) as the primary source of information for understanding and interpreting their data. In these organizations, that staff member's mind is the metadata system—the resource that describes, explains, locates, or otherwise makes it possible to retrieve, use, or manage data. But in this era of data-driven decisionmaking, the sheer volume of data collected for administrative, instructional, and management purposes complicates data systems beyond the management capacity of even the most experienced professional. There are simply more data to organize, access, and understand than ever before; and no data steward's mind, however powerful, is up to the task of managing all that data about data. A metadata system is not only a better and more reliable alternative—it is the only realistic way to effectively accomplish this vital information management task.
Metadata systems are critical components of an effective information management system. The benefits of properly implementing a robust metadata system include
While metadata cannot eliminate every opportunity for improperly collecting, using, or reporting data, a sound metadata system provides a framework for better understanding data and, therefore, minimizes the likelihood of data misuse. Cost- benefit and return-on-investment analyses ensure that both positive and negative implications have been considered prior to making a significant investment in time and resources to introduce a metadata system. In fact, many organizations find that the potential improvements to data quality and use are well worth the costs of developing, implementing, and supporting a robust metadata system.
Nevertheless, despite the potential value of metadata systems, many organizations have yet to develop and implement robust metadata systems. Leaders in these organizations may make this decision passively (they do not think about it) or actively (they reject the notion as too costly). When education leaders make an intentional decision not to implement a metadata system, they often do so because they believe that developing metadata items and systems:
All of these reasons for not developing metadata systems are valid—up to a point. Developing a metadata system is a substantial undertaking that requires significant time, expertise, commitment, and money. But like other time-, staff-, and resource-intensive initiatives (e.g., installing new networking systems, building new facilities, and introducing new professional development programs), metadata systems should yield benefits that far outweigh their costs.
No matter what the anticipated benefits are, a decision on whether to proceed with a metadata system will eventually be based on the relative costs and benefits of the proposed system. Planners engage in cost-benefit analysis to ensure that both the positive and negative implications of a metadata system have been considered. In addition to many readily measurable benefits, less quantifiable benefits exist as well. These are sometimes called "soft" or "intangible" benefits, and include improved data use when, for example, teachers can identify potential dropouts; improved staff morale, such as employees trusting that the organization maintains accurate human resources files; and more effective auditing procedures, such as error checking to confirm calculations. While placing a dollar value on these "soft" benefits is difficult, they are nevertheless real and could be estimated for the purposes of cost-benefit analysis. Many organizations have found that the potential improvements to data quality and use are well worth the costs of developing, implementing, and supporting a robust metadata system (see exhibit 5.2).
Given the complexities involved in designing and developing a metadata system, an education organization cannot decide it needs metadata, develop and institute a system, and expect it to be operational that same day. A thorough and realistic project plan is critical to implementing the effort effectively and getting the job done efficiently. A robust metadata system will provide context for a single data item, serve as the backbone for efficient data management, and improve the use, analysis, and management of any body of data. The result will be improved accuracy, utility, and comparability of elementary and secondary education data at local, state, and national levels.