4. Report information accurately and without bias

With improved technologies for generating and presenting data, policymakers and the general public have become more number-savvy and more accustomed to looking for statistics to back up a position. Our ability to provide data that are more accurate and more relevant is stronger than it ever has been. As organizations focus on producing high-quality data, they should also raise expectations for high-quality data reporting and presentation. In the story that introduces this canon, a principled school administrator uncovered a mistake caused by faulty information. However, even after data quality has been established, data can still be subject to misuse. One common way of distorting data is by manipulating the way in which information is presented.

Data are often presented in graphs or charts that demonstrate, for example, if test scores are improving, administrative costs are decreasing, or student populations are changing. These visuals are effective when they summarize a host of numbers and complex analyses into a single, immediately understandable, "information snapshot." However, unethical (or incompetent) data handlers can bias perception with formatting tricks that change the way data look in graphic form. One way to accomplish this, for example, is to change scales on a graph to influence interpretation (see figure 1).

Not all data users possess the same level of analytical expertise. Community members, for example, may want a single graph, or a table that presents the findings as simply and straightforwardly as possible, while statistical researchers would probably prefer to see all the data and analyses laid out in detail. Despite the need to tailor reports to its audiences, organizations will benefit from adopting standard protocols to guide data reporting and minimize the potential for biased presentation that might skew interpretation.

Policies governing operations can also affect data accuracy. For example, a district rule that says "all students are considered present if a teacher fails to collect attendance" could tempt some school staff members to "forget" to take roll. Such a policy might lead to the appearance that student attendance rates were improving over time (as teachers continue to "forget" to take roll), even though the data collection policy had no meaningful effect on actual student attendance.

Finally, and very importantly, this canon comes into play when data represent bad news. People are rarely tempted to skew data that already make them and their organization look good. This ethical principle is likely to be tested when data show that something is not working well or is behind schedule. When this happens—and the news is bad—data handlers find out how strong their commitment to ethical behavior really is.

Recommended Practices and Training

1. Develop a standard reporting framework to improve the consistency of reports and minimize the likelihood of bias in data presentation.
   1. Develop and apply statistical standards and guidelines for writing data reports and presenting data tables. For example, many issues related to axis scales, years reported, and graphing style (e.g., line chart, pie chart, or stacked bars) can be standardized regardless of data values.
   2. Include, in all reports, an explanation of how past reporting was presented and any changes in data methods (e.g., an assessment tool) or presentation (e.g., a report format); state the reasons why changes were made.
   3. To the extent possible, use the same presentation standards in all report products. If information is presented in a standard format across years and across studies, audiences will learn to read reports easily—they can concentrate on the information rather than on the way in which it is presented.
   4. Before releasing a report, undertake an independent review to assess whether the data are presented objectively and without bias, especially when they describe a situation that is not favorable to those responsible for producing the report.
The National Center for Education Statistics (NCES), the principal statistical agency within the U.S. Department of Education, publishes statistical standards and guidelines to encourage the generation of high-quality, reliable, useful, and informative statistical information for the use of public policy decisionmakers and the general public.
2. Incorporate improvements to research design and methodology in data reporting, but do not let these changes mislead data interpretation. For example, if your dropout formula is adjusted and you see a correction in one direction or another because of the modification, do not declare that there was a meaningful change in your school's retention efforts unless you can crosswalk or otherwise compare the two methods. Similarly, a recalibrated assessment may offer neither positive nor negative evidence of change in student performance.
3. Correct errors that are identified in previously reported data. If published data are discovered to be inaccurate and correcting the data is feasible, the data should be corrected with an explanation and documentation in subsequent reports or releases of the data. Establish procedures for making corrections to ensure that revised releases include clear statements about the impact the revisions may have on previously reported statistics.
4. Train data reporters and users to follow standard data preparation and presentation methodologies so that data are presented as accurately and consistently as possible. Customize training efforts by job type as appropriate for communicating concepts and translating instruction into practice. For example, one exercise for technical staff would be to present them with an array of tables or graphs representing the same information but presented in ways that would mislead a reader. The task would be to find the "bad" presentations and talk about how each biased the data. Non-technical users, such as instructional staff or board members, could discuss these same tables or graphics after the trainer has highlighted the flaws in presentation. Another approach that might help staff who prepare presentations would be an open discussion (without names!) of situations in which they have been encouraged, or tempted, to bend the rules in showing data a little more favorably.

Top