​​​​​​​​​​​​​​​NAEP Technical DocumentationDevelopment of the Mathematics Indices

For the 2015 mathematics assessment, several indices of policy interest were developed that satisfied both theoretical criteria based on content analysis, and empirical criteria based on multivariate statistical techniques. This resulted in the creation of several new reporting elements. The development of the 2015 mathematics indices can be summarized in three main steps:

1. Question selection. A content analysis of the existing subject-specific 2015 mathematics survey questionnaires was conducted to select sets of the questions that were theoretically interpretable and meaningful as a conceptual unit, as potential indices to measure specific constructs of interest.

2. Examination of empirical relationships. Factor analysis was used to explore and verify the empirical properties of the data. Construct validity of the potential indices was evaluated through factorial validity with respect to the survey question responses, and the convergent and discriminant validity of the factor with respect to other factors. If the factor had the expected pattern of relationships and non-relationships, the construct validity of the factor as representing the intended index was supported.

3. Index scoring. The partial credit item response theory (IRT) model was used to scale the indices. Scaling of the index items was first conducted to get the item parameters and was based on marginal maximum likelihood methodologies. After the parameters were estimated, expected a posteriori (EAP) scores were calculated as the estimate of the index score. Then, the EAP scores were transformed to have a mean of 10 and a standard deviation of 2 on a scale from 0–20.

Indices were formed for grades 8 and 12 mathematics, but the grade 4 survey questionnaire did not include a sufficient number of questions that would allow for the creation of indices.

Index of Students' Views on Mathematics

The tables below present the items forming the index of students' views on mathematics at grades 8 and 12, respectively. This index was designed to measure students' interest in and enjoyment of mathematics. At grade 8, students were asked to indicate how often they agreed with two statements about their mathematics class (never or hardly ever, sometimes, often, or always or almost always). They were also asked to indicate the extent to which they agreed with two statements about mathematics (strongly disagree, disagree,agree, or strongly agree). At grade 12, students were asked to indicate the extent to which they agreed with two statements about mathematics (strongly disagree, disagree, agree, or strongly agree). They were also asked to indicate how often they agreed with three statements about their mathematics class (never or hardly ever, sometimes, often, or always or almost always).

Index of Students' Computer Use in Mathematics

The table below presents the items forming the index of students' computer use in mathematics at grade 8. This index was designed to measure students' use of computers to complete mathematics-related activities. Ten items asked students how often they used a computer for math at school, how often they used different types of computer programs when doing math for school or homework, and how often they used computers to get help with mathematics (never or hardly ever, once every few weeks, about once a week, two or three times a week, or every day or almost every day).

The partial credit IRT model was used to scale the indices. Scaling of the index items was first conducted to get the item parameters and was based on marginal maximum likelihood methodologies. ​The following tables show the IRT parameters for the 2015 mathematics indices.

After the parameters were estimated, EAP scores were calculated as the estimate of the index score, and transformed to a reporting scale of 0–20.

​For each index item, the response categories were scored as numerical values (e.g., the first response category strongly disagree was scored as 1; the second response category disagree was scored as 2).

As a reporting aid, cut points were used to divide the grades 8 and 12 students' views on mathematics index into low, moderate, and high classifications, and the grade 8 students' computer use in mathematics index into never, once every few weeks, about once a week, and twice a week or more classifications. The cut points selected to divide the index scores into meaningful categories were based on the distributions of the response average of each index.

As an example, for the index of students' views on mathematics, grade 8 students were classified as follows:

• Students with a response average less than the second response category were classified as low on the index. That is, students who on average responded less than disagree on a question with response options strongly disagree, disagree, agree, and strongly agree generally reported that mathematics was not fun for them and that they did not like mathematics, and were classified as low on the index.
• Students with a response average greater than or equal to the second response category but less than the third response category were classified as moderate on the index.
• Finally, students with a response average greater than or equal to the third response category were classified as high on the index. That is, students who on average responded agree or higher generally reported that mathematics was fun for them and that they liked mathematics, and were classified as high on the index.

The tables below show the transformed scale scores and score classifications for the mathematics indices.

Printer-friendly Version