Skip Navigation
small NCES header image
The Forum Guide to Data Ethics
NCES 2010-801
March 2010

6. Promote data quality by adhering to best practices and operating standards


All students enrolling in the district were flagged in the student information system (SIS) as either "eligible" or "ineligible" for free- and reduced-price meals. In addition to these data, food services maintained its own record of students receiving free- and reduced-price meals to support the daily management of the cafeterias. Because of these redundant sources for similar data ("eligible" and "participating" counts were easily confused), the district was erratic in its free- and reduced-price meals reporting—the number of students reported for the program varied depending on whether the data were reported by the SIS or food services staff. Moreover, because there were two different counts, district staff faced the temptation to use the number that better met their reporting needs. Sometimes the count of participating students made the district look better, and sometimes the count of eligible students was beneficial. It didn't take long for the staff to realize that this confusion between eligible and participating counts was leading to ethical dilemmas.

Two new data governance policies were enacted to remedy the situation. The first stated that the student information system was the authoritative source for all data in the district. The second policy was an offshoot of the first, declaring that only data staff could respond to data requests, and program or service staff were no longer permitted to provide data independently. Thus, if a count of students receiving free- or reduced-price meals was needed, the report would clearly distinguish it from the number of students eligible for this program.

Effective instruction, efficient school management, and quality data are related. The importance of quality in the information used to develop an instructional plan, run a school, create a budget, or place a student in a class cannot be overvalued. Most data experts would agree that the following characteristics are critical to this essential component of good schools:

  • Utility: Data should provide information that is useful to the organization in a practical way. If data are not useful, there is no reason to collect them.
  • Accuracy and validity: Data should measure what they purport to measure. In other words, data values should be correct and free of bias.
  • Reliability: Data should be consistent, reproducible, and dependable. Data are not reliable if the values would change if they were collected more than once, or by more than one person. Properly documenting revisions to data values, definitions, and other characteristics is also necessary to ensure data reliability.
  • Timeliness: Data should be readily available for decisionmaking. For example, do teachers and curriculum developers receive test results in time to inform instructional planning? If data are not available when they are needed, they lack in value and quality.
  • Cost-effectiveness: Collecting and maintaining data requires resources, and this burden should be evaluated against the data's utility and necessity. In other words, the value to instructional and non-instructional decisionmaking and reporting should outweigh the costs of collection and storage.
For more information about supporting data quality, see the Forum Guide to Building a Culture of Data Quality, listed in appendix A.

Just as canon 3 asserts that data handlers are ethically responsible for knowing the laws, regulations, and policies that govern access to the information for which they are responsible, this canon asserts that they are ethically responsible for ensuring high data quality to the best of their ability.

There are many accepted practices and operating standards that promote good
data quality. For example,

  • office staff should set aside a regular time for data entry tasks, which should be conducted in an area free from distractions such as foot traffic and noise;
  • technologists should offer a help desk and/or an online help area for data entry staff;
  • data stewards should resolve data discrepancies before reports are forwarded to senior staff for review and approval;
  • principals should develop a calendar for data reporting deadlines;
  • teachers should ask for instructions and guidance for improving data use in the classroom;
  • superintendents should support a culture of quality data through a robust professional development program; and
  • school board members should recognize data collection, management, and use as a routine cost of doing business—and, as much as possible, provide the resources for this work.

Recommended Practices and Training

  1. Use best practice resources to design your data systems and train data handlers. The topic is beyond the scope of this Guide, but there are a number of Forum publications that may be especially helpful (see appendix A):
    1. Forum Curriculum for Improving Education Data: A Resource for Local Education Agencies—for lesson plans, instructional handouts, and related resources for helping schools develop a culture of data quality.
    2. Managing an Identity Crisis: Forum Guide to Implementing New Federal Race and Ethnicity Categories—to guide implementation of the new federal race and ethnicity categories.
    3. Every School Day Counts: The Forum Guide to Collecting and Using Attendance Data—to improve the quality, comparability, and utility of attendance data.
    4. Accounting for Every Student: A Taxonomy for Standard Student Exit Codes—to improve the quality, comparability, and utility of exit data.
    5. Forum Guide to the Privacy of Student Information: A Resource for Schools—to help apply federal, state, and local privacy laws.
    6. Forum Guide to Education Indicators—to help construct and apply commonly used education measures.
    7. Forum Guide to Decision Support Systems: A Resource for Educators—to develop a robust decision support system in an education organization.
    8. Creating a Longitudinal Data System—to learn more about the ten essential elements of a state longitudinal data system, as available from the Data Quality Campaign.
  2. Data system improvement is more than this Guide can address, but organizations are encouraged to develop a comprehensive and coordinated plan for improving and ensuring data quality, addressing issues such as
    1. assigning unique student identifiers;
    2. using prepopulated forms for assessments when appropriate;
    3. performing data verification, validation, editing, as necessary to assess and improve data quality;
    4. integrating systems to reuse data for multiple appropriate purposes (payroll, human resources, etc.) rather than collecting and rekeying information already owned by the organization;
    5. incorporating industry standards across disparate systems, such as the Schools Interoperability Framework standards;
    6. utilizing applications that allow for easy import and export of existing student data; and
    7. identifying the authoritative source of data items when multiple systems include the same data items.
  3. Update data practices to reflect changing policy needs. For example, the routine use of a surname as a family identifier may not be sufficient when the traditional family unit definition is expanded to include other family members, such as parents, stepparents, grandparents, or child advocates.
  4. Train all data users about the concepts and practices surrounding the generation, maintenance, and use of high quality education data.

Top

Would you like to help us improve our products and website by taking a short survey?

YES, I would like to take the survey

or

No Thanks

The survey consists of a few short questions and takes less than one minute to complete.