4. Report information accurately and without bias
The community was thrilled to learn that the local high school had been named one of the top 10 schools in the
country by a major news magazine. However, when examining the methodology behind the award, the district
superintendent questioned the finding and decided that she needed to know how the rankings were determined. An
inquiry to the magazine found that the data had been "checked and double-checked," but no one at the publication
was willing to divulge what data were used to determine the rankings. Additional investigation by district staff revealed
that the magazine had used an incorrect enrollment figure, causing the participation percentage on a national test to
be tremendously inflated. The superintendent understood that, if she reported this to the magazine, the high school
would surely drop from the top tier to the second tier of "best schools." Still, the error had to be corrected—it was
the right thing to do. Despite the decline in national prominence, the superintendent was surprised to learn that her
community—including parents, students, alumni, and the local media—were very proud that the school district chose
to report the error rather than receive recognition it didn't deserve. Ensuring accuracy over fame had actually confirmed
to community members that they really did have one of the top school systems in the country.
With improved technologies for generating and presenting data, policymakers and
the general public have become more number-savvy
and more accustomed to looking
for statistics to back up a position. Our ability to provide data that are more accurate
and more relevant is stronger than it ever has been. As organizations focus on
data, they should also raise expectations for high-quality
reporting and presentation. In the story that introduces this canon, a principled school
administrator uncovered a mistake caused by faulty information. However, even after
data quality has been established, data can still be subject to misuse. One common
way of distorting data is by manipulating the way in which information is presented.
Data are often presented in graphs or charts that demonstrate, for example, if
test scores are improving, administrative costs are decreasing, or student populations
are changing. These visuals are effective when they summarize a host of numbers and
complex analyses into a single, immediately understandable, "information snapshot."
However, unethical (or incompetent) data handlers can bias perception with formatting
tricks that change the way data look in graphic form. One way to accomplish this, for
example, is to change scales on a graph to influence interpretation (see figure 1).
Not all data users possess the same level of analytical expertise. Community
members, for example, may want a single graph, or a table that presents the findings as
simply and straightforwardly as possible, while statistical researchers would probably
prefer to see all the data and analyses laid out in detail. Despite the need to tailor
reports to its audiences, organizations will benefit from adopting standard protocols
to guide data reporting and minimize the potential for biased presentation that might
Policies governing operations can also affect data accuracy. For example, a
district rule that says "all students are considered present if a teacher fails to collect
attendance" could tempt some school staff members to "forget" to take roll. Such a
policy might lead to the appearance that student attendance rates were improving over
time (as teachers continue to "forget" to take roll), even though the data collection
policy had no meaningful effect on actual student attendance.
Finally, and very importantly, this canon comes into play when data represent
bad news. People are rarely tempted to skew data that already make them and their
organization look good. This ethical principle is likely to be tested when data show
that something is not working well or is behind schedule. When this happens—and the
news is bad—data handlers find out how strong their commitment to ethical behavior
Recommended Practices and Training
- Develop a standard reporting framework to improve the consistency of reports and
minimize the likelihood of bias in data presentation.
- Develop and apply statistical standards and guidelines for writing data reports
and presenting data tables. For example, many issues related to axis scales, years
reported, and graphing style (e.g., line chart, pie chart, or stacked bars) can be
standardized regardless of data values.
- Include, in all reports, an explanation of how past reporting was presented and
any changes in data methods (e.g., an assessment tool) or presentation (e.g., a
report format); state the reasons why changes were made.
- To the extent possible, use the same presentation standards in all report
products. If information is presented in a standard format across years and
across studies, audiences will learn to read reports easily—they can concentrate
on the information rather than on the way in which it is presented.
- Before releasing a report, undertake an independent review to assess whether
the data are presented objectively and without bias, especially when they
describe a situation that is not favorable to those responsible for producing the
- Incorporate improvements to research design and methodology in data reporting,
but do not let these changes mislead data interpretation. For example, if your
dropout formula is adjusted and you see a correction in one direction or another
because of the modification, do not declare that there was a meaningful change in
your school's retention efforts unless you can crosswalk or otherwise compare the
two methods. Similarly, a recalibrated assessment may offer neither positive nor
negative evidence of change in student performance.
- Correct errors that are identified in previously reported data. If published data are
discovered to be inaccurate and correcting the data is feasible, the data should be
corrected with an explanation and documentation in subsequent reports or releases
of the data. Establish procedures for making corrections to ensure that revised
releases include clear statements about the impact the revisions may have on
previously reported statistics.
- Train data reporters and users to follow standard data preparation and presentation
methodologies so that data are presented as accurately and consistently as possible.
Customize training efforts by job type as appropriate for communicating concepts
and translating instruction into practice. For example, one exercise for technical staff
would be to present them with an array of tables or graphs representing the same
information but presented in ways that would mislead a reader. The task would
be to find the "bad" presentations and talk about how each biased the data. Non-technical users, such as instructional staff or board members, could discuss these
same tables or graphics after the trainer has highlighted the flaws in presentation.
Another approach that might help staff who prepare presentations would be an
open discussion (without names!) of situations in which they have been encouraged,
or tempted, to bend the rules in showing data a little more favorably.