Key Questions and Indicators

Four of the five key questions for this chapter address a single issue: Are software and applications systems appropriately addressing important school management and instructional functions?

The first two questions deal with teaching and learning standards and basic technology tool skills; the next one addresses the use of (primarily computer-based) technology in support of communications. The fourth question deals with technology-based support of key administrative functions.

The fifth question addresses a different issue entirely: the existence of a process to assess software and applications. Software and applications, ideally, have a life cycle: specifically, there is a point in their installed life at which they need to be upgraded or replaced. Making this process systematic, or at least transparent to the community, is important for the intelligent use of technology. At this point, the only indicators suggested for this question deal with the existence of an evaluation process and the ratings categories used in performing the evaluation.

Key Question 1. Do the school or district's instructional applications support teaching and learning standards across the curriculum?

This key question relates to software packages or applications (for instance, web-based applications) that directly support teaching and learning. Software packages such as those that support the teaching of reading or writing, or those that relate to specific mathematics or science skills and knowledge, are currently the ones most likely to have external ratings. There are commercial enterprises that rate software in terms of alignment with state and national standards (see sidebar "Commercial Sites Rating Software Alignment to Standards" on the facing page). It is also clearly possible for state or district groups to rate software packages in terms of their alignment with teaching and learning standards.

"Usage Tip" If you are in the process of developing or purchasing a database or information management system for tracking technology in your school or district, and are not sure what entries should be included, you may sample the unit record lists of data elements in each chapter to decide what will best support the critical decisions you must make.

INDICATORS

Alignment with teaching and learning standards	Existence and current status of software alignment plan (most likely at district or state level; may be for all subjects or for a specific curriculum area or grade).
	Alignment rating (for each application).
	Alignment measure (for each standard).
	Percentage of applications aligned to teaching and learning standards.
Approved instructional applications in regular use	Number of approved instructional applications in regular use,by subject area, grade, and type.
	Number of approved teacher-support instructional applications in regular use, by type.

TERM CATEGORIES

Existence and current status of software/applications alignment plan: 0=no alignment plan exists; 1=an alignment plan is being developed; 2=an alignment plan has been approved; 3=an alignment plan is approved and is being implemented.

Alignment rating (for a single software/applications package; may be for alignment with standards for all subjects, or for a specific curriculum area or grade): 0=no alignment to standards exists for this application; 1=a weak alignment to standards exists for this application; 2=this application is somewhat aligned to standards; 3=a strong alignment to standards exists for this application; 4=this application is fully aligned to standards. [Note: judgment about relative strength of alignment to standards might be in terms of the extent of fit between software goals and one or more standards, or in terms of the number of standards addressed by a software package or application, or even in terms of the credibility of the rating organization.]

Alignment measure (for a single teaching and learning standard; may be for all subjects, or for a specific curriculum area or grade): 0=none of the applications in use in the school/district have been rated as fully aligned to a learning standard; 1=one application has been rated as fully aligned to a learning standard; 2=more than one application has been rated as fully aligned to a learning standard.

Percentage of instructional applications aligned with one or more learning standards: Number of applications with ratings of 3 or 4 (see "Alignment rating," above) divided by the total number of instructional applications and converted to a percentage. May be for alignment with standards for all subjects, or for a specific curriculum area or grade.

Instructional application types: Instructional process support (see also "Teacher-support instructional applications," immediately below); learning support (practice drills; problem solving; data analysis; simulation/demonstration; research; distance learning); assessment.

Teacher-support instructional applications: Software or applications specifically designed to support teachers in instructional processes. Examples include attendance software; grading applications; testing systems; student work or portfolio systems; lesson planning software.

Key Question 2. Is there software support for technology tool skill development?

The basic tool skills that everyone involved in schools-from the principal to the teacher to the student-must have in order to be able to use technology range from simple keyboarding to computer programming. While some skills obviously vary in complexity and may not be needed until higher education, basic capabilities have become part of a new literacy. Knowing them is akin to learning how to read in an earlier age: the mark of an educated, competent person. Assessing the extent to which the school technology offerings teach these tool skills is very important and will no doubt become even more so as technology becomes more transparent in the classroom.

INDICATORS

Application support for technology tool skill development

Count of applications, by type, in use in instructional settings in the school or district that support technology tool skill development.

TERM CATEGORIES

Examples of application types that support technology tool skill development in instructional settings: word processors; spreadsheets; database software; desktop publishing; process writing software; keyboarding training software; telecommunications; web browsers and search engines; web authoring; presentation development software; programming tools, including compilers and interpreters.

As applications and devices in use in classrooms continue to evolve, and as technology becomes further integrated into education, what are considered "basic tool skills" can be expected to evolve. Five years from now, the above list will be quite different from what it is now, even if it is not much longer.

[back to top]

Key Question 3. Does the school/district use technology applications to improve communication?

There are a number of important communication functions that schools and districts carry out which technology can enable or improve. Communication between school staff and parents of students can be supported through class homework assignment web pages or telephone hotlines. E-mail can support direct parent communication with teachers or the principal. Communication with the larger community is enabled by school web pages or online school "report cards"; the latter often become part of real estate agencies' electronic brochures for the community.

Schools use computer-based networks to conduct significant business with their districts, other institutions, and with state authorities. Data are sent to state education agencies; requests for purchases go back and forth from schools to central districts. Staff attendance records are forwarded to district payroll systems. Transcripts are sent to other schools or districts when students move, or to colleges when students graduate.

The current list of indicators below focuses on electronic mail and web use. Indicators related to telephone use could also be developed.

INDICATORS

Applications for communication	E-mail type: School or district-wide, non-Internet; Internet-based; none.
	E-mail: Percentage of teachers with active accounts.
	E-mail: Percentage of students with active accounts.
	Existence of active district/school web site.
	School web site usage.
	Percentage of classrooms with active web pages.
	Percentage of teachers with active web pages.
	Use of Internet telephony ("voice over IP," or voIP) in the school or district.

TERM DEFINITIONS AND CATEGORIES

Active account (electronic mail): At least one message sent in the current academic year.

Active web site/page: At least one update in the past 90 days.

School/district web site usage: Count of user sessions in a month, converted to a daily average.

Key Question 4. Does the school/district have appropriate software and systems to support primary administrative functions?

Although most of the public and policy focus on computing in schools addresses instructional uses, large gains in efficiency and effectiveness in schools can be made by using computers and network systems in administrative applications. Although such applications are decidedly less glamorous and less likely to be visible or appear reasonable to anxious parents worried about their children's future, they can play a critical role in making food services, transportation or records management work better and in reducing the management burden on senior administrators so they can maintain their focus on students' education.

Administrative software is also essential to the gathering, processing, and transmission of critical education data. Applications at the district level can be integrated into decision support tools.

Strategic partnerships between schools and private corporations have resulted in technology forums, initiatives, and criteria to assist superintendents in choosing models to follow.

INDICATORS

Applications for core administrative activities	Availability of applications to support core administrative activities, by activity type.
Applications for decision support	Decision support tools are available to administrators.
	Decision support tools are available to teachers and curriculum groups.
	Decision support tools are available to parents and the public.
	Availability of decision support tools, by information category.
Applications for policy support	Availability of applications that support school/district Security and Acceptable Use Policies.
	Applications in regular use that support the school/district Security and Acceptable Use Policies, by application type.

TERM DESCRIPTIONS AND CATEGORIES

Software/application types to support core administrative activities: Capital improvements (building and grounds); financial (accounting, budgeting, payroll, human resources); food services; inventory control; library services (cataloguing, circulation); network security (firewall, filtering, secure data transmission, Acceptable Use Policy enforcement); office applications; student materials (purchasing, inventory); student records management (attendance, assessment, grading); teacher records management (attendance, assessment, certification); transportation; other software and applications.

Key Question 5. Are the applications in use evaluated for effectiveness?

Standards for educational technology applications are being developed, with the potential to be used for assessment of instructional software and applications. There are already many databases available that offer thousands of reviews of educational technology, including instructional software (see the Resources section at the end of this chapter). Technical coordinators and other evaluators will have to develop criteria to evaluate the effectiveness of products beyond "best-of-breed" determinations.

The situation is more problematic for evaluation of administrative software and applications. Standards for such applications do not exist or at least represent an entirely different framework. Schools and districts need to evaluate administrative software in terms of functional effectiveness (i.e., do they accomplish the goals they are intended to accomplish?). Review databases may be helpful in this context.

INDICATORS

Software evaluation	Existence of software evaluation plan.
	Evaluation ratings by software category.

School districts have evaluation procedures in place that may include reviewing the content of software and web sites for alignment with state or local learning standards (see the sidebar "Evaluating Web-Based Products and Standards" on the next page), accuracy, grade and ability levels, special student needs, teacher training requirements, and more. Others take advantage of services such as EvaluTech (http://www.evalutech.sreb.org) or states such as Florida (http://www.doe.firn.edu/edtech/it/esc.shtml) and California (http://www.clrn.org/home/), which have developed extensive evaluation web sites to assist educators in selecting high-quality electronic learning resources.

Unit Record Structure

Although information about technology applications is typically analyzed at the school or district level, the relevant unit for data element definitions is the software program or application. Unit records for software form the basic elements for a comprehensive technology application database.

A system based on unit records would meet day-to-day administrative needs and support overall assessment and planning. As noted in other chapters, these suggested records are not intended to define data collection. Instead, they illustrate basic units of a data system from which data elements and indicators can be extracted to answer important policy and planning questions.

The data elements listed below, along with others defining basic school components such as classrooms and adapted from other NCES handbooks, can be used to create the indicators listed in this chapter. The complete list of data elements for this guide can be found in Appendix A; a number of detailed examples illustrating the creation of indicators from data elements can be found in Appendix B.

LIST OF POTENTIAL DATA ELEMENTS FOR A UNIT RECORD: SAMPLE UNIT RECORD FOR ALL SOFTWARE AND APPLICATIONS OFFICIALLY INSTALLED IN SCHOOL SYSTEM EQUIPMENT

For each software or application listing:

General information:

For instructional and instructional support software:

Descriptive information about educational function, if applicable:

For administrative software:

For each machine or workstation:

Records kept and aggregated at building or district level.

[There also exists a database of all instructional machines in use (see unit records for Chapter 3), in which there will be a record of all software and applications installed.]