Many states jump head first into developing or purchasing an LDS (or certain components) before spending time to think about exactly why they are doing so, what their stakeholders need, and what it will take to get the job done. Experience suggests that successful LDS development is roughly 80 percent planning and only about 20 percent building.* Careful planning can make the difference between a clumsy belly flop and a graceful dive into system development.
During the needs assessment phase, policymakers should establish the business justification for what kind of LDS they want and why. In addition, stakeholders should define their requirements so that the resulting system reflects their needs. Needs assessment is also another step in creating early buy-in for the project, getting everyone—both internal and external stakeholders who use, or would like to use, education data—involved in creating a vision for a system that they will one day find useful.
Before the developers start defining requirements and building the system, some fundamental questions should be answered about why the system is being built and what it will do for the education community. Early on, decisionmakers should ask themselves the following questions:
|“Why do we want an LDS?”|
|"What are our ultimate goals for the system?"|
|Preferably, the answers will not be:|
|"To check off the 12 elements of the America COMPETES Act."|
|"To keep up with other agencies."|
|Better answers are:|
|"To improve instruction by helping teachers identify student needs, discover and practice the most effective teaching strategies, and tailor instruction; by helping administrators target teachers for professional development; and by helping researchers conduct more informative studies to identify effective strategies."|
|"To inform policy and resource allocation decisions at the state and local levels with better information, and to help state program staff target district and school improvement needs."|
|"To calculate academic growth; and follow students and maintain their academic histories as they advance to higher grade levels, transfer across districts within the state, drop out or transfer to private school or another state and come back into the system, and move into higher education."|
|"To track staff and maintain their professional histories as they enter and progress through teacher preparation programs, receive professional development, and transfer among schools."|
|"To streamline operations and improve data quality by automating processes such as data entry and loading, making data collection more efficient; by helping state staff produce federally- and state-mandated reports; and by conforming to broadly shared data standards."
Once decisionmakers and other stakeholders are familiar with the LDS concept (see Book One: What is an LDS?), the benefits it can provide (see chapter 5 of Book One) what their current system looks like (see chapters 6, 7, and 9), and why they need such a system, they can create a vision of what they want their own LDS to do and what functionalities they want it to have.
Determine system requirements early so that expectations can be adjusted for everyone involved in the development process. Establishing goals and expectations for the system is also a good practice. This will help focus system design and keep IT staff, vendors, legislators, and others on the same page in terms of what needs to be done and what will actually be available in the future.
Everyone wants to improve student achievement, but the important question is, “How do you want to do it?” Policymakers and other stakeholders need to determinewhat functionalities they want and establish requirements for the developers. If system developers have clear requirements to fulfill, policymakers and other stakeholders are much more likely to get what they need.
You may want a souped-up, cutting edge system that will cost a lot. Or you may just want a basic, no-frills economical system. Either way, you must be clear about what you need and what you can afford. Without careful reflection and planning, your organization may find itself with a poorly designed system that does not really do what it needs it to do.
The National Center for Analysis of Longitudinal Data in Educational Research offers links to several LDS datasets along with descriptions of their source systems. The State Data page is a good place to research other agencies' LDSs.
It is sometimes difficult to figure out what you need because you may not be familiar with what you do not have. You may not realize that certain functionalities might make your job easier. To get started, you might compare your current system, as depicted in the self-assessment, to the ideal LDS outlined in chapter 3 of Book One: What is an LDS? Researching other data systems or visiting an education agency with a more sophisticated system can also provide context and insight into available options (see NCES's Personnel Exchange Network). Also, review the software applications on the market. Consider the functionalities they offer and determine if they would work in your environment and meet your stakeholders' needs.
When working with stakeholders, try to operate in concrete rather than in abstract terms. Instead of asking vague, open-ended questions about what they need, it may be helpful to start with questionnaires to gauge interest in certain LDS components and capabilities. For instance, an agency might survey staff and other stakeholders with a list of questions that the new system could be designed to answer. The staff could then rate the questions in terms of the value they think the answers would provide, thus allowing you to gauge stakeholder interest in a systematic fashion. And, since some stakeholders will inevitably be more outspoken in stating their needs in meetings and other venues, such surveys are a good way of leveling the playing field so prioritization of needs can be done fairly.
Specific questions included in such a survey might be drawn from the stakeholders themselves, and the rating process could provide a basis for prioritizing stakeholder needs. Surveys like this might be written for different groups. For instance, teachers might be asked to evaluate questions about curricula or student achievement, while other staff might be asked to assess operational functionalities such as data entry automation and data sharing between databases. Based on the answers provided and the resources available, a state might decide which data to collect, which tools and reports to offer, and which capabilities to prioritize.
To help guide discussions, develop some questions that should be asked of the various audiences. Some examples are listed in table 3.
Documenting requirements in some form, such as in a needs statement, is very important. This is "a description of the functional needs, technical requirements, and security and ethical standards that need to be met by a technology solution" (National Forum on Education Statistics 1998). An agency should take its findings, categorize them, and try to boil them down to discrete needs. The resulting set of requirements can help guide the work of in-house system developers or help an agency find a product on the market that will meet its needs. If you are planning to hire a vendor (or several vendors), your findings will help in the creation of an RFP (see chapters 13–14). Either way, be sure your identified needs can be measured by tangible criteria, so that success in attaining the organization's goals can be assessed during evaluation of the system later on (see chapter 15).
After figuring out what you have and what you want, a "gap" analysis should be done to identify the discrepancies between "here" and "there," and determine what work will be necessary to achieve the desired system. For instance, if the agency does not currently have a student identifier system, but the needs assessment calls for one, a "gap" has been found that needs to be addressed.
Building an LDS with stakeholders' input is an iterative process. Give it time and be persistent. Expect needs assessment to be ongoing, rather than completed once at the beginning of the project. As parts of the system are developed and go live, continue to get feedback to find out what stakeholders think and what new needs they might have. New ideas often arise from new developments.
|District difference: District-level LDS considerations
Most of the discussions about LDSs focus on state-level systems. But state agencies are not the only ones building these systems—many school districts and regional agencies are as well. While fewer people may work on a local-level LDS, the same need exists and the same mistakes are often made. Below are questions and considerations for school-district staff to ponder before building their LDS. (Many of these are also applicable to a state-level effort.)
When assessing the need for a district-level LDS, ask
If the decision has been made that a local LDS is necessary, then