Forum Guide to Data Quality

Part One: Data Quality—Chapter 2: Building a Culture of Data Quality

A culture of data quality is, among everyone who has a role in student outcomes, a shared understanding of the importance of data quality and commitment to ensuring data quality. In an agency with a strong culture of data quality, staff members and stakeholders demonstrate a common belief that good data provide critical support for teaching, learning, planning, governance, and strategy. The culture stems from an agency’s intentional and integrated strategy to ensure that stakeholders understand the purpose of the data and to give stakeholders a chance to review, discuss, and support quality data and the role of data in evaluation and decision-making. These stakeholders can include students and parents, school office staff, teachers, data stewards or coordinators, technology support personnel, principals, school board members, superintendents, U.S. Department of Education (ED) staff, and staff of state and local education agencies (SEAs and LEAs). In the culture of data quality, these stakeholders all play a part in supporting data integrity, which allows trust in the data standards, credibility, and security of the data.

Key Steps to Build a Culture of Data Quality

1. Engage all stakeholders.
2. Establish a data quality team whose members possess the qualifications necessary to manage the agency’s data collections and have access to effective data tools.
3. Offer ongoing professional development so that all staff stay informed about current requirements, changes, and new collections or procedures.
4. Support school-level staff who enter data into established systems but do not work with data regularly.
5. Regularly demonstrate the importance of quality data, for example, by demonstrating how quality data are tied to program funding.

To promote a culture of data quality, it is useful to provide agency staff with answers to and additional information about the following questions:

• How will the data be used? Demonstrating the connection between data collections and the day-to-day operations of classrooms and the school helps stakeholders who are involved in data work recognize how high-quality information supports their work and improves education outcomes.
• What data are collected? An overarching understanding of the data collections helps contextualize the data and identify potential quality issues, such as duplicate or outdated data collections and fields.

To avoid inaccuracies in data once it moves beyond its origin in the LEA, LEAs can clarify how the data will be maintained within external systems after being pulled from their student information system (SIS). For example, one LEA asks how data such as “name” and “gender” will be maintained, and only provides these data if a system can accept corrections to these elements (and others) that are not static. The LEA’s reasoning is that incorrect or invalid data are typically worse than no data: outdated data is invalid but may be treated like it is valid if this issue is not considered.

• When are the data entered, checked, and validated? Data staff often need to meet multiple reporting deadlines throughout the year. Ensuring that these staff know when the tasks related to each data collection must occur helps them to meet deadlines and control their workflow. General information about the data entry, review, and validation process can help staff understand how data are processed.
• Who enters, validates, and manages the data? Clear roles and responsibilities around data collection help ensure that data are entered accurately and on time. Similarly, clarity around how data will be validated and managed moving forward is critical. Leaders should ensure that staff members understand how their diligence and careful practice when entering and reporting data help the agency provide services and meet reporting requirements.
• What are the consequences for poor data quality or inaccurate reporting? Clear processes for auditing data and known consequences for inaccurate reporting are essential for dedicating sufficient resources to quality reporting. For example, questionable data may be suppressed in public reporting or not used for making important performance decisions such as accreditation ratings.

By knowing this key information, agencies create confidence in their data and lay the groundwork for implementing the best practices described in Chapter 3.

Data Quality Culture Throughout the Education System

Data quality must begin at the data’s sources, the LEAs and the schools from which the data are collected. LEAs often take the first steps toward creating and ensuring quality education data. LEA staff, including school staff, initially collect the data that are used for administration, instruction, and operations. These data are entered in local data systems and may be shared later with other LEAs, the SEA, and federal agencies for additional planning and reporting.

Agencies with a culture of data quality understand that data quality can be strengthened by strong collaboration within and across local, state, and federal levels. High-quality, valid national or state level data are crucial for education leaders who make decisions about programs, policies, and funding. Leaders also need to understand how local results compare nationally or at the state level, as well as how current data compare to those collected earlier or under different circumstances.⁵ Though federal, state, and local education agencies serve different roles in the data process, they have responsibilities that should complement and support one another.⁶

Agencies support data quality when

• data definitions are aligned, understood, and consistent between LEA, SEA, and federal education entities;
• SEA staff work with LEA data coordinators to establish sound, practical procedures for collecting, managing, and reporting data while remaining sensitive to data burdens;
• LEA and SEA offices responsible for data reporting cooperate to identify and remove redundancies and to consolidate requests;
• LEAs and SEAs work together to create new data elements and indicators when needed;
• LEAs and SEAs work together to retire the collection of data elements when they are no longer needed or relevant;
• ED works closely with SEAs to plan and improve data collection processes to ensure quality data at the federal, state, and local levels, while simultaneously aiming to reduce data burdens on SEAs and LEAs;
• LEA leaders work with their data coordinators to make sure that modifications to data collections (such as definitions, calendars, or reporting) align with mandated reporting requirements and support timelines, are well understood, and are properly communicated; and
• LEA data coordinators collaborate with school staff responsible for data entry to provide input to student information system (SIS) vendors and identify efficiencies in data collection.

School Level

The data collection process technically begins at the school level, with data collected and entered by school staff, clerks, teachers, and others. The foundation of data quality includes effective training that ensures that those entering data understand the requirements for doing so accurately, as well as validation edits and local review. Best practices at the school level include

• developing tools to enable efficient review of data (especially during collections), which can identify data that do not conform to the business rules or expected values and formats;
• implementing controls to prevent common errors during data entry (for example, specifying the preferred format for date fields rather than allowing for free-response data entry);
• including training on the business rules under which data are collected as part of the professional learning provided on using data systems;
• working with district staff to regularly review aggregate data—at least in key areas like enrollment, attendance, discipline, and finance—to monitor for anything that looks anomalous or out of place;
• making data quality monitoring and feedback tools available so school staff notice when data quality issues occur and correct them quickly;
• assessing the root causes of data quality issues, with a goal of preventing them from reoccurring, rather than just fixing them; and
• cross-training staff on data systems and collections so that when schools are facing staff absences or attrition, they are still able to collect and input data.

LEA Level

To develop a culture of data quality in an LEA, it is best practice that district staff work with schools to help staff understand the purpose of the data collected and, where possible, connect the data to a specific instructional program or goal. It is also important for school personnel to understand the reasons behind the collection for data that may not be related to instruction, such as data required to qualify for financial resources or data required for the Civil Rights Data Collection (CRDC). Program funding is tied to accountability measures that, in turn, are based on data collected by schools and districts. Therefore, it is critical that districts provide staff with the resources needed to produce quality, on-time data.

Districts must adhere to deadlines set by other agencies as well as their own data needs and schedules. Creating and communicating a district data calendar will help track the times when reports are due, when the corrections windows open and close, and when schools must provide data to meet these deadlines. A district data calendar will also communicate timelines for making modifications to collections to district leadership. Additionally, LEAs often do a certain amount of “managing upward” when working with their SEA. SEAs may not be directly familiar with the front-facing data entry systems in LEAs, and managing expectations and communicating with the SEA can contribute to an improved culture of data quality.

Districts also must ensure that data and data systems adhere to policies and regulations set by state and federal authorities as well as their own internal policies. District staff can more easily meet these policy and regulatory requirements if they are confident in the data they receive. To that end, district personnel often are responsible for training data collectors and for ensuring that the data collected are of high quality. Districts can help schools identify the authoritative sources for data and reduce data collection and reporting burdens by determining when data are duplicated in different school and district systems. Additionally, districts can implement automated quality controls to ensure data accuracy by helping staff members identify and correct data errors while entering them.

Districts can benefit from including school personnel in the creation of district data policies, regulations, and processes. For example, a process for transferring data from a school to a district should involve

• staff responsible for developing relevant reports and information;
• representatives from the IT and data teams;
• representatives from the schools involved; and
• data stewards at multiple levels.

By including everyone involved in a data collection in the planning stage, district data stewards can more effectively create a collaborative environment that fosters a culture of quality data. In this environment, the people responsible for all aspects of the reporting cycle will carry out their work with a full understanding of required tasks and why they are important. Collaboration and communication are the foundation of a culture of data quality both within and across agencies. When guidelines are developed within a process that considers multiple viewpoints and needs, districts and schools can create real-world procedures that enhance the quality of information across the state as well as within the district.

SEA Level

SEAs provide the foundation for a culture of data quality statewide. Their roles and responsibilities in collaborating with LEAs are critical to achieving quality data. Given the complexity of collecting data from multiple LEAs and reporting data to ED, SEAs play a key role in fostering collaboration among stakeholders at all levels and implementing data-quality controls and processes.

SEAs set standards and expectations for data collection and reporting among LEAs and SEA offices. Standards define data elements and outline proper data entry procedures and business rules, and they should reflect state and federal requirements as well as local and state information needs. SEAs can help LEAs adhere to standards in a number of ways, including

• providing tools such as data standards⁷ that help LEAs follow data collection procedures;
• publishing a calendar of data collection due dates, including data correction opportunities;
• conducting audits to ensure the accuracy and completeness of data;
• establishing a data governance process that includes LEA input in the development process for new and modified collections;
• working with LEAs to locate and correct data errors within data correction windows;
• offering training materials and manuals covering data definitions and collection procedures;
• providing "train-the-trainer" opportunities to create local experts on the use and functionality of the SEA data system;
• facilitating helpdesks to respond to LEA questions and issues;
• sharing data collection changes, deadlines, and details through newsletters⁸ or other existing communications;

Data leaders in Milwaukee Public Schools (WI) have monthly meetings with the SEA to remain up-to-date on data quality, learn about changes, and troubleshoot issues. Leaders also attend a quarterly meeting that is statewide for large districts.

• providing clarity on how data quality impacts agency processes, functions, and responsibilities; and
• working with districts to resolve data quality issues that originate from other districts and may cause conflicts when reported to the SEA level.

The SEA also is an important resource for districts. Like schools and districts, the SEA relies on high-quality data and can support districts with information on data elements, collection timelines, and alignment across collections.

Federal Level

ED’s Data Strategy⁹, published in 2020, establishes strategic goals for advancing data capabilities and envisions agency-wide outcomes. It establishes an ambitious vision as a point on the horizon: to realize the full potential of data to improve education outcomes and lead the nation in a new era of evidence-based policy insights and data-driven operations.

In its 2023 update, the Strategy included 16 new guiding principles within four key areas:

• Strengthen Agency-Wide Data Governance
• Build Human Capacity to Leverage Data
• Advance the Strategic Use of Data
• Improve Data Access, Transparency, and Privacy

Like both ED’s inaugural Data Strategy and the ED’s updated Data Strategy, all work carried out under the updated Data Strategy will be informed by the three guiding principles of ethical governance, conscious design, and learning culture.¹⁰

Proposed changes to data collections are shared through the U.S. Office of Management and Budget (OMB) to allow SEAs and LEAs an opportunity to comment on the visibility and impact of changes. ED encourages comments and considers impact on quality of data when reviewing.

ED encourages comments and considers impact on quality of data when reviewing.

As ED collects data from states and districts, it utilizes multiple processes and tools to ensure data quality. These data quality efforts include those targeted toward the largest collections, CRDC and EDFacts. These ED collections have allowed states and districts to report data through file uploads with data notes, receive data quality feedback from the system, and allow for states and districts to revise files and resubmit. Recently, the EDFacts modernization process was launched to bring the reporting system in line with technology, data tools, and data needs that had changed significantly since EDFacts’ inception. Modernization will improve ED reporting systems to support the growth of state data systems, provide business rules to be tested ahead of reporting, and provide immediate feedback in a more detailed and yet easy to use interface, thus allowing these data to be submitted, reviewed, and released quicker and with more accuracy.

⁵ Spiegelman, M., & Merlin, J. (2023, August 30). Using Federal Education Data to Inform Policymaking: Part 1–Benefits and Advantages. NCES Blog. U.S. Department of Education, National Center for Education Statistics. Retrieved from https://nces.ed.gov/blogs/nces/.
⁶ Spiegelman, M., & Merlin, J. (2023, August 30). Using Federal Education Data to Inform Policymaking: Part 2–Challenges and Opportunities. NCES Blog. U.S. Department of Education, National Center for Education Statistics. Retrieved from https://nces.ed.gov/blogs/nces/.
⁷ For an example of state data standards, see https://education.ky.gov/districts/tech/sis/Pages/KSIS-Data-Standards.aspx.
⁸ For an example of a state data newsletter, see https://education.ky.gov/districts/tech/sis/Pages/KSIS-Newsletters.aspx.
⁹ U.S. Department of Education. (2020). Data Strategy. Retrieved September 1, 2023, from https://www.ed.gov/sites/default/files/cdo/ed-data-strategy.pdf.
¹⁰ U.S. Department of Education. (2023). Data Strategy. Retrieved November 8, 2023, from https://www2.ed.gov/about/offices/list/opepd/ocdo/ed-data-strategy.pdf.