 |
|
|
Chapter 7
Evaluating Facilities Maintenance Efforts
Goals:
| |
 |
To communicate the importance of regular facilities
maintenance program evaluation
|
| |
|
To recommend best practice strategies for
evaluating facilities maintenance efforts |
Program evaluation allows planners to see which initiatives are working,
which are not working, and which strategies need to be reconsidered. There
is simply no substitute for good data when making evaluation and program
decisions.
| When the Going Gets Tough, the Tough Get Evaluating
Nick had pretty much staked his reputation, and perhaps his job, on a preventive
maintenance program. He'd championed the idea, recommending it in no uncertain
terms to the superintendent and school board. So when the money was earmarked
for a preventive maintenance program, everyone congratulated him. But Nick knew
that getting the money and implementing the initiative was only the start of the job.
He had to show that the program was working-or at least find out where it wasn't
working and then reassess his strategy as needed.
When the next year's budget cuts came down from the top, the assistant
superintendent tried to reassure Nick that the maintenance program
would survive the cut. "Look, Nick, you could make up nearly a third
of the cut if you just reassign your program evaluation funds back
into the maintenance budget." Nick looked at his boss with surprise.
"Ted, in ten years of working together, I've never heard such a
bad idea come out of your mouth." Ted was taken aback by the  reply,
"But Nick, I just want to make sure that you're getting the biggest
bang for your buck out of the budget." Nick laughed, "So do I, and
that's why we've got to evaluate our work. Otherwise, we'll have
no way of knowing what the 'buck' is really buying us. We won't
know what we're doing right, or doing wrong, or where we needed
to improve our performance. I'm telling you, Ted, when the budget
gets lean-that's when we really need to stay serious about evaluating
our work so that we can determine our priorities and allocate those
tight dollars." Ted scratched his head, "I hadn't thought of it
that way, Nick. I bet the same is true in the rest of the district
as well, huh?" Nick cracked a smile, "I bet it is."
|
|
|
Reasons for evaluating the facilities maintenance program include:
internal management control
school board requests
state reporting mandates
regulatory inspections (e.g., EPA)
|
Evaluating
Your Maintenance Program
 |
This Planning Guide provides a framework for proactively
developing a comprehensive, district-wide facility maintenance plan.
The preceding chapters address the primary elements of maintenance
planning, including recognizing the need for effective maintenance
programs, planning maintenance programs, performing facilities audits
(i.e., data collections), ensuring environmental safety, maintaining
grounds and facilities, and managing staff and contractors. One other
vital component of adequate school facilities maintenance is periodic
evaluation to assess the success of these efforts at a program level.
To realize the full potential of a comprehensive preventive maintenance
system, school staff, the school board, and town planners must incorporate
maintenance priorities into all
|
modernization goals, objectives,
and budgets. However, it is also fair for stakeholders to expect the maintenance program to yield results-namely: clean, orderly, safe, cost-effective, and instructionally supportive school facilities that enhance the educational experience of all
students. But stakeholders also need to demonstrate patience because the only thing that takes more time than implementing changes to a maintenance program is waiting to see the improvements bear fruit.
Considerations When Planning Program Evaluations
Evaluation doesn't have to mean more dollars and more surveys. Many of
the day-to-day activities or systems used to plan and operate a maintenance
program also generate the types of information needed to evaluate the
program's effectiveness. These can include:
|
Program
success can only be evaluated relative to program objectives. In other words,
measuring "success" means answering the question: Are we reaching our goals
and objectives?
|
| |
|
Physical inspections: Records of physical
inspections are good evaluative material. To care for buildings and
grounds, staff must observe and assess their condition on a regular
basis. Inspections should be both visual (i.e., how things look) and
operational (i.e., how things work), and should result in work orders
for items requiring service or repair.
|
| |
|
Work order systems: An effective work order
system, as explained in Chapter 5, is a good tool for identifying,
monitoring, and projecting future maintenance needs. All maintenance
work should be recorded on work orders, which then provide valuable
quantitative information for evaluations.
|
| |
|
User feedback/customer satisfaction surveys:
There are many ways to gather information from users/customers (i.e.,
the people who benefit from the maintenance activities), including
collecting satisfaction surveys and convening advisory committees
of stakeholders. The value of user perception should not be overlooked
as an evaluation tool. Appendix I provides a sample customer survey
form used to request feedback relating to custodial and maintenance
work.
|
| |
|
Audits: Performance audits, commissioning,
retro-commissioning, comparisons with peer organizations, benchmarking,
and annual reviews of accomplishments provide important data for the
facility plan and ensuing evaluation.
|
| |
|
Alternative resources: Maintenance staff
need not reinvent the wheel when it comes to evaluations. Maintenance
and operations manuals, vendor expertise, warranties, and other resources
(e.g., Web sites) can be sources of benchmarking data or evaluation
standards.
|
| |
|
Regulatory activities: Appropriately trained
staff or contractors must be assigned to determine whether applicable
public safety and environmental regulations are followed. These staff
must be responsible for documenting inspection activities and reports,
notifying appropriate oversight organizations of deficiencies, developing
strategies for remedying deficiencies, and verifying compliance to
applicable laws and regulations. Documentation of these activities
can be used in program evaluation.
|
|
A Note about Budgets
Even with
the best planning, budget cutbacks are sometimes unavoidable. This
may force planners to reprioritize their operational objectives-which
can affect the goals of an evaluation effort as well. For example,
under shortfall conditions, evaluators might be asked to assess
whether budget cutbacks have prevented the department from reaching
one or more of its goals. Or the evaluation effort might be used
to identify mission-critical components of the maintenance plan
in the event of ongoing program cuts.
|
|
Questions to Drive Evaluation Efforts
A simple
evaluation program can be implemented by answering these four questions:
Step 1:
What is the purpose of the evaluation? That is, what decisions need to be made and by whom?
Example: The facilities maintenance director and the
school business official want to know whether the new work order
system is worth the money that was invested in its purchase, installation,
and staff training.
Step 2:
What questions need to be answered to make an informed decision, as identified in Step 1?
Example: Is the new work order system accomplishing
all that we had hoped it would? Is the new work order system running
more efficiently than the old system?
Step 3:
What information needs to be available to answer
the questions identified in Step 2?
Example: What was the total cost for purchasing and
installing the work order system? What was the total cost to train
staff to use the work order system? Are staff time and materials
accounted for in the work order system? Does the system maintain
historical data about maintenance at each site? Does the system
track all purchases, from ordering through delivery, installation,
and storage? Does the system document all preventive maintenance
activities? Has the response time for work order requests decreased?
If so, by how much? Has the number of work orders accomplished
increased? If so, by how much?
Step 4:
What is the best way to capture the information needs identified in Step 3?
Example: Accounting and management audits, work order
system user surveys, current work order system reports, and data
from previous work order requests.
|
|
|
|
The only thing worse
than no data is misplaced confidence in bad data. Decisions are bound to
be bad if the data used to inform them are of poor quality.
|
Collecting
Data to Inform a Comprehensive Evaluation
 To
evaluate a facilities management program, the district must collect and
maintain accurate, timely, and comprehensive data about its facilities.
After all, responsible decision-making requires good data and documentation.
Before assessing maintenance improvements, it is necessary to identify
the baseline against which progress will be measured (see Chapter 3).
In other words, will the organization compare its current status against
its previous status, against peer organizations, or relative to commonly
accepted norms and best-practice standards?
The graph shows the number of days it took for a work order to get
completed in a school district before and after the process was streamlined. Although the amount of time it takes for the actual work to be accomplished has not changed, two significant time-saving approaches have been adopted:
1) the number of people handling the work order has been cut, and 2) the parts and materials procurement system has been linked to the work order system. This type of streamlining not only increases efficiency with respect to getting work accomplished, but also decreases unnecessary administrative costs.
|
Document all "lessons learned" to keep a record
of things that didn't work as planned so that mistakes can be avoided the
next time around.
|
Collecting data may require substantial effort, but it is a necessary
task all the same. Proven sources of information about the condition of
school facilities and the impact of a facility maintenance program include:
| |
|
number of work orders completed
|
| |
|
changes in maintenance costs
|
| |
|
major incident reviews (e.g., number of school
shutdowns, safety events, etc.) |
| |
|
"customer" feedback (e.g., the opinions of principals
and other occupants)
|
| |
|
visual inspections by supervisors and managers
|
| |
|
comprehensive management audits
|
| |
|
performance audits
|
| |
|
organizational studies
|
| |
|
annual snapshots (e.g., maintenance/operations
cost per square foot or per student)
|
| |
|
facility report cards or other summaries
|
| |
|
comparisons with "peer" organizations
|
| |
|
benchmark performance
|
| |
|
trend analysis (e.g., progress toward the organization's
long-range plans)
|
| |
|
external audits/peer reviews
|
| |
|
weekly foreman's meetings
|
| |
|
staff turnover rates
|
| |
|
public opinion (e.g., newspaper articles, etc.)
|
|
Pitfalls to Avoid When Interpreting Maintenance Evaluations
Stakeholders should not assume that improvements to a maintenance
program will always yield cost savings in real dollars. To obtain
an accurate assessment of maintenance initiatives, evaluators must
also look for:
| |
|
Cost avoidance rather than direct savings
(e.g., well-maintained equipment tends not to wear out or need
to be replaced as quickly as poorly maintained equipment)
|
| |
|
Fewer service interruptions resulting from
better maintained, and better performing, equipment.
|
Moreover, improving facilities maintenance requires patience. A
comprehensive, proactive program takes resources, energy, and time
to initiate-and even more time before results are realized.
|
|
