Problem Solving in Technology-Rich Environments

The Problem Solving in Technology-Rich Environments (TRE) study is one of three field investigations in NAEP's Technology-Based Assessment Project, which explores the use of new technology in NAEP. The TRE study was designed to demonstrate and explore innovative uses of computers in NAEP by developing two extended problem-solving scenarios.

Download the Problem Solving in Technology-Rich Environments report for a printed version of the results and to read about the implications of the study for NAEP.

TRE focuses on the physical science associated with helium gas balloons used for space exploration. Both scenarios—a Search scenario and a Simulation scenario—were developed for grade 8 students.

The TRE study was guided by the following principles:

• TRE should use the computer to do what cannot be done easily on paper.
  • The TRE scenarios allow students to explore the domain of physical science in a dynamic environment. All student actions are captured in computer files for later scoring, allowing for evaluation of the processes used in problem solving.
• TRE should represent the type of problem solving done with computers in educational and work environments.
  • TRE attempts to capture the multidimensionality characteristic of problem solving with technology by requiring students to demonstrate both science skills and basic facility with the computer. Technology in TRE is a means of solving specific problems, rather than an end in itself.
• To the degree possible, TRE should allow the disentangling of component skills.
  • While the assessment was designed to measure both basic computer skills and science skills in an integrated way, several steps were taken to ensure that deficiencies in one skill did not prevent the successful demonstration of other skills. For instance, tutorials were provided to help students understand the scenario interfaces, and science help tools were created to give access to basic information. See the description of the TRE scenarios for more information on the help tools provided to students.

Data were collected from a nationally representative sample of grade 8 students in the spring of 2003. Over 2,000 public school students participated, with approximately 1,000 students receiving each assessment scenario. Within each school, students were assigned randomly to one of the scenarios, which were presented on school computers via the World Wide Web or on laptop computers taken into the schools by NAEP staff.

For both scenarios, data were collected about student demographics; students' access to computers, their use of computers and attitudes toward them; and students' science coursetaking and activities in school. Additionally, at the beginning of each scenario, students answered prior knowledge questions designed to determine if they already had the computer and/or science knowledge and skills being assessed.