Contact:
Ghedam Bairu (202) 502-7304 It is important to ensure confidentiality and protect student and staff privacy. |
Chapter 2: Meeting the Challenges of Data CollectionAs discussed in the preceding chapter, a safe environment, one that is free of threats and intimidation, provides a climate conducive
to learning and teaching. Data collection is an important component in
creating such an environment. Nonetheless, there are challenges to data
collection. One challenge is accuracy-do the data represent what they
are intended to represent? Other challenges include how interpretation
of the data affects school image, especially how data are reported by
the media, and costs of collecting data.
It is important to ensure confidentiality and protect student and staff privacy. It is necessary to exercise caution when sharing information with staff and other professionals serving schools. In addition, it is especially important to exercise caution when working with the public. The Federal Family Education Rights and Privacy Act (FERPA) prohibits schools and school districts from releasing most types of information about students. Many states also have legislation regarding the confidentiality of student records and information. The purpose of data collection is to support the mission of schools (i.e., to continue efforts to secure a safe and orderly learning place), and not to create a repressive environment. Security is a more complicated issue now that many records are stored and transmitted electronically. Schools and school districts need to be sure that their computerized data are appropriately protected from unauthorized electronic access. Records can no longer be secured solely by locking the filing cabinet. As previously described, incident data can be used to ensure the following:
To create safe and orderly places for learning, incident data must be accurate. Staff training and other efforts to ensure the reliability and validity of data are critical. Chapter 4, "Collecting Data, " provides
specific details on the Task Force recommendations for the elements
to be included in an incident database. In addition to listing types
of incidents, variables are identified (e.g., time, locale) and defined.
A key feature of the incident database is the ability to interface with
student and staff databases. |
||||||||||||||||||||||||||||
It is important that the database records incidents beyond state-mandated or legal violations. | It is important that discipline referrals and disruptive incidents
be recorded even when they do not cross the threshold for state-mandated
reporting or legal violations. This information can be used to identify
trends and link incidents to grade level, location, time of year, or
other potentially useful factors (Cornell et al., 1999).
When designing a database system or purchasing one that is commercially available, users need to evaluate key features, such as whether users will be able to produce all reports necessary to identify problem areas. Problem reports and trend data can be used to set specific goals for preventing problem behavior, develop school improvement programs, and facilitate documentation of progress in addressing school needs (Cornell et al., 1999). Training is a critical element of data collection. It is important to train all school staff on what kinds of problem behavior are to be reported to administrators. It is important for administrators to be clear on how incidents are defined, and for both administrators and data entry personnel to understand the specifics of reporting procedures. Cornell and colleagues (1999) advocate statewide training for all school divisions on the implementation of an incident-based data collection and reporting procedure, as well as training on the appropriate use of such information for school safety planning and procedures. School level administrators will be better prepared to guide their
staff in district discipline policy if they themselves receive training
from qualified instructors. District level administrators can educate
school level administrators more effectively regarding state mandates
if they have received guidance on the subject. A coordinated, integrated
training program will provide both clarity of policy goals and consistency
of implementation.
|
||||||||||||||||||||||||||||
Even with clear and concise definitions, training at all levels is crucial. | Providing guidance on discipline to administrators takes two forms:
(1) training on recordkeeping and (2) instruction regarding definitions
and classification of incidents and disciplinary actions. If school
district administrators wish to compare incident data across schools, they need to do more than leave it to untrained school staff to simply
choose the incident code that most nearly matches the infraction. Doing
so may produce inconsistent data. Even with clear and concise definitions, training at all levels is crucial to ensure data comparability. Districts
may want to consider forming discipline committees to review data, develop
and implement plans for training, and revise codes as necessary. Including
practice on identifying the types of misbehavior to be recorded would
be beneficial. Examples of what does and does not meet the standard
for each type of infraction will be important to differentiate among
definitions (e.g., Battery (code 1300) versus Fighting (code 1700) versus
Physical Altercation, Minor (code 2500) [See
Chapter 4]).
Technical assistance staff in Florida provide guidance on incident reporting during regional workshops. School and district administrators are trained on how to categorize incidents and what level of problem behavior meets the threshold for state-mandated reporting. Before developing this program, state staff worked to identify reporting problems, revise and clarify definitions, and develop technical assistance materials such as video clips of examples and "nonexamples" of each type of incident. School and district administrators and law enforcement officials contributed to this process. The state also provides grants to school districts that need additional support to work with their schools on data collection issues. Many districts in Florida follow this model (See Figure 2-1.). Training is provided to district staff, school administrators, and data clerks. This training addresses:
For example, "disorderly conduct" that rose to the level of "state reportable" was a significant problem one year in one district. District staff felt that school staff were coding many incidents as "disorderly conduct" when a less serious code would have been appropriate. Training focused on the definition of "disorderly conduct, " and the number of reported incidents decreased substantially. Another district found that some principals were recording fights as multiple incidents-one incident for each participant. They held training on the structure of their database, which has separate modules for incident and perpetrator information; thus, the incident itself is recorded in one data file and the information on each perpetrator is recorded in a separate data file. Documentation within schools is most likely to be accurate and consistent if school administrators or their assistants code infractions, but it is still very important that all school staff are trained regarding prohibited behaviors. Training within schools could follow the state and district model provided in figure 2-1. School staff members need to be trained on what to report. Training will reduce the likelihood of inconsistent reporting. In addition, training can help ensure that minor problem behaviors are reported before they escalate. Including information on incident definitions and reporting procedures in teacher handbooks is recommended. Efforts to ensure consistency among staff members in the level of misbehavior that will be tolerated are also important. In addition, regular review of incident reporting and discipline problems at faculty meetings is useful. 2.1.3 Ensuring Reliability
and Validity
Reliability and validity are important components of data collection. The following components ensure that reporting is consistent (reliable) and that variables measure what they were intended to (valid):
Defining problem behavior. Chapter 4 describes the types of incidents to track and elements to record. The list of incidents in Chapter 4 is extensive; however, not all incidents will be appropriate for all schools and school districts. In addition, the list may miss incidents important to particular jurisdictions or schools. Decisions on which incidents to include need to be made by the people
they affect. Involving stakeholders in the decisionmaking process will
generate support for data collection efforts-no effort can succeed without
the support of front-line individuals. If data collection decisions
are being made at the school level, it is critical to include teachers
and other staff on the team that designs the incident-reporting system.
If the decisions are being made at the district level, it is necessary
to include school administrators and school staff in the process. If
decisions are being made at the state level, it is important to include
school and school district administrators in the process. It may also
be helpful to include parents and community members on the incident
committee. Information that is useful to individual schools may not
be useful to the school district as a whole. For example, it may be
important for a school to know whether an incident occurred in the locker
room or in the cafeteria, but the school district may not need to know
this. Similarly, a state may not need to know whether an attendance
policy violation consisted of skipping class or forging an absence note.
|
||||||||||||||||||||||||||||
Communication is essential to ensure reliability and validity. |
Communication is essential to ensure reliability and validity. It is
critical to clearly describe behaviors that have been defined as infractions
in student, staff, and parent handbooks. Students need to know what
behaviors are unacceptable before they can be expected to behave appropriately.3
Staff must know the behaviors for which they are expected to discipline
students before they can do so.4
Finally, parents must be informed about how their children are expected
to behave.
Data checks. It is crucial that data be reviewed regularly
for completeness and accuracy. It is also necessary to verify that all
data elements are being recorded for each incident. During the data
review process it is important to examine incident elements being tracked
(e.g., time, perpetrator).
|
||||||||||||||||||||||||||||
Assigning a key person the responsibility for oversight of all aspects of incident reporting can improve accuracy. | At the school level, all incidents recorded on paper need to be entered
into the incident database. Staff need to note on each paper reporting
form that the incident has been recorded before the form is filed. It
is important to establish a system for recording this information (i.e., who enters what information when) to increase the likelihood that information
will be recorded as intended. Reviewing paper files to determine whether
or not incidents were recorded appropriately may also be useful in managing
data entry systems. Assigning a key person the responsibility for oversight
of all aspects of incident reporting can improve accuracy.
One way to assess accuracy is to compare a sample of incident forms to what has been recorded in the database for those incidents. When assessing accuracy, it is also important to consider differences from month to month or week to week in the number of each type of incident. If significantly more or fewer "bullying" incidents occurred in one time period than another, it is important to investigate why that is so. If a new bullying prevention program has been implemented, it would be logical for the number of incidents to increase just after the program takes hold. At that time staff will be more sensitive to the kinds of behavior that are unacceptable, and for incidents to decrease after a program has been in place for some time, as students learn more positive ways of interacting. The data must also "make sense." In other words, did a school really have 10 battery incidents last month, or is someone miscoding another type of infraction? Other basic data checks include assessing what percentage of students committed rule violations and which violations are the most frequent. An effective database has built-in checks for consistency of data. For example, if a weapon-related incident is recorded, the computer program will automatically prompt for type of weapon. When analyzing schools within a district, district staff need to ask whether it is reasonable that School A has more "fights" than School B, given differences in enrollment. If not, more training may be required. Similarly, it will be useful for district and state staff to ask whether it is reasonable that District A has the same number of "fights" as District B, given differences in enrollment. Compare and contrast. It is useful to compare incident data to other sources. Serious infractions (e.g., illegal behaviors) can be compared to police reports. Incidents involving injury can be compared to hospital records. Levels of problem behavior identified in self-reported data from student and teacher surveys can be contrasted with the level of problem behavior being recorded in incident databases. For many reasons, rates reported from these sources will differ from school or district databases. However, it is important to investigate significant disparities. For example, if students are reporting very high rates of drug use but there are very few drug infractions recorded, it will be valuable to learn why the difference exists. |
||||||||||||||||||||||||||||
Problem behavior will not go away if ignored. | It is reasonable to be concerned about how data will be used. Such
concerns should not become barriers to effective data collection. Problem
behavior will not go away if ignored. Without information on what kinds
of problem behavior exist, schools will not be able to take steps to
address their problems. It is important that information lead to action, not frustration. One use of data is to identify students, schools, and
school districts in need of additional supports. Schools cannot demonstrate
improvement without a way to establish a starting point-an incident
database can provide this.
Parents and the public want schools to produce well-educated students.
Given this focus, it is important to educate the public on the need
to invest energy in creating safe and orderly schools. The premise that
an orderly environment is an important factor in student achievement, and that students cannot be successfully engaged in academic work in
a disorderly environment, must be communicated to parents, taxpayers, and policymakers (Squires, Huitt, and Segars, 1983). One way to do this
is to develop strong media relations.
|
||||||||||||||||||||||||||||
Approach problems as "needs"to let the public know what is needed to accomplish specific goals. |
Positive media relations. Working with
the media can help schools, districts, and states present data in ways
that will help strengthen public support for their data collection and
prevention efforts. In "A Short Guide to School Public Relations, " Kinder
(2000) explains that when the facts are known, the vast majority of people
are supportive of the efforts of their school leaders. He notes that the
time to launch a public relations effort is not during a crisis. A well-designed
public relations program aims to prevent problems, not to solve them.
Sound public relations efforts can be used to build public support for
schools (Muir, 1999). The National
School Public Relations Association recommends enlisting the media
to publicize efforts to promote school safety (Price, 1999). Another important
strategy is to ensure that all communication contains some form of success
story, even when problems are being identified, because this conveys the
message that something can be done to improve matters (California Department
of Education, 1997). Other experts on school public relations recommend
that, when school officials are talking with the public, they approach
problems as "needs"-that is, to let the public know what is needed to
accomplish specific goals, and what the community can expect to gain for
its support (Muir, 1999; Gaskins, 1999). If the school needs money to
implement prevention programs, the public must know how effective these
efforts can be. For example, Patterson Elementary School used the Effective
Behavior Support Program to halve the school's office referrals. The program
is aimed at teaching students appropriate conduct and involved tracking
the number of students referred to the office, for what offenses, where
the violations took place, and concentrated on the most disruptive students.
(Sack, 1999). Such success can be very persuasive when asking the public's
help in raising the money to implement a program in a specific set of
schools. Public perception. Before sharing data with people outside the school building, school staff need to be aware of this information. Communicating with school staff is a vital part of safety planning and discipline improvement, and staff members will particularly appreciate receiving timely information directly from administrators. Providing staff members with data on behavior improvement gives them an opportunity to be proud of their accomplishments. In addition, it is important that they know what problems need to be addressed and that they are involved in deciding how the problems will be addressed (Bernhardt, 1998; Ross, 1999). Data can easily be shared at faculty meetings or through school improvement teams. Next, share information with parents. Presentations to parent/teacher associations and school newsletters are two possible communication tools. Reporting mechanisms and presentations are described in Chapter 4. The California Department of Education (2001) suggests that, when reporting to the public, administrators stress the positive messages illustrated by the data (i.e., safety instead of crime) and how data have been used to develop successful programs. When administrators communicate statistical data in such a positive way, the public is presented with a solvable problem, not an insurmountable dilemma. Presenting data offers an opportunity to form a partnership with law enforcement or other community agencies-another potential resource for schools. |
||||||||||||||||||||||||||||
Implementing a system that allows for early identification and intervention could result in substantial cost savings. |
It is likely that there will be costs associated
with efforts to clarify definitions; aligning handbooks, rules, policies, and practices; and training staff in what and how to report. However, these initial investments can have great payoffs. One of the most important
results would be fewer incidents of problem behavior and a net gain in
time spent on the task of learning. There is a strong association between
students having academic and behavioral problems (e.g., Hinshaw, 1992;
Yoshikawa, 1995). High levels of problem behavior are associated with
early grade retention and grade retention is costly, adding at least the
expense of an additional year of school to a child's education (Byrd and
Weitzman, 1994). According to the National
Center for Education Statistics (2001), per-pupil spending averaged
nearly $6,500 nationally for the 1998-99 school year. Implementing a system
that allows for early identification of students with serious behavior
problems and for early intervention (thereby supporting the prevention
of grade retention and remediation of other problems) could result in
substantial cost savings. The implementation of a system that assists
with the prevention of violence and vandalism will also result in cost
savings. In addition, schools with low rates of violence and vandalism
are more effective places where more learning occurs (Squires, Huitt,
and Segars, 1983). Thus, the rationale for data collection meets educational
goals of a safe climate for academic achievement.
|