Table of Contents | Search Technical Documentation | References

NAEP Instruments → Contextual Items (Survey Questionnaires) → Indices Based on Questionnaire Data → Development of the 2019 Grade 12 Mathematics Indices

NAEP Technical DocumentationDevelopment of the 2019 Grade 12 Mathematics Indices

For the 2019 grade 12 mathematics assessment, several indices of policy interest were developed that satisfied both theoretical criteria based on content analysis, and empirical criteria based on multivariate statistical techniques. This resulted in the creation of several new reporting elements. The development of the 2019 grade 12 mathematics indices can be summarized in three main steps:

1. Question development. New sets of contextual items, such as those exploring students’ persistence in learning and their enjoyment of complex problems, were developed and included in the mathematics student questionnaire. Through content analysis as part of the item development process, only sets of items that were theoretically interpretable and meaningful as a conceptual unit were included as potential indices to measure specific constructs of interest.

2. Examination of empirical relationships. Factor analysis was used to explore and verify the empirical properties of the data. Construct validity of the potential indices was evaluated through factorial validity with respect to the survey question responses, and the convergent and discriminant validity of the factor with respect to other factors. If the factor had the expected pattern of relationships and non-relationships, the construct validity of the factor as representing the intended index was supported.

3. Index scoring. The partial credit Item Response Theory (IRT) model was used to scale the indices. Scaling of the index items was first conducted to get the item parameters and was based on marginal maximum likelihood methodologies. After the parameters were estimated, expected a posteriori (EAP) scores were calculated as the estimate of the index score. When a student does not provide a valid response to a contextual item, the student will not receive an index score for the index containing that item. In the first administration of an index, the EAP scores were transformed to have a mean of 10 and a standard deviation of 2 on a scale from 0–20. In subsequent administrations, EAP scores for established (i.e., trend) indices are estimated using concurrent calibration together with data from the previous year and then transformed using linear transformation (Kolen and Brennan 2004) to be on the same scales created in the initial administration. Transformation constants and are calculated as follows:

$a equals fraction with numerator standard deviation, sub target, end sub, and denominator standard deviation, sub provisional, end sub$

b equals mean sub target, end sub, minus open parenthesis a times mean sub provisional, end sub, close parenthesis

where Standard deviation sub target and Mean sub target are the target standard deviation and mean of the transformed scores from the previous year, and Standard deviation sub provisional and Mean sub provisional are the provisional standard deviation and mean of the previous year EAP scores in the current year concurrent calibration.

Index of Students' Persistence in Learning

The table below presents the items forming the index of students' persistence in learning. This index was designed to measure students' tendency to persevere and work hard in the face of challenges. Grade 12 students were asked to indicate how much each of the four statement items described them (not at all like me, a little bit like me, somewhat like me, quite a bit like me, or very much like me).

Items in the index of twelfth-grade students' persistence in learning, grade 12 mathematics national assessment: 2019
How much does each of the following statements describe you? Select one answer choice on each row.
Response categories
Item			Not at all like me	A little bit like me	Somewhat like me	Quite a bit like me	Very much like me
B034901	a.	I finish whatever I begin.	A	B	C	D	E
B034902	b.	I try very hard even after making mistakes.	A	B	C	D	E
B034903	c.	I keep working hard even when I feel like quitting.	A	B	C	D	E
B034904	d.	I keep trying to improve myself, even when it takes a long time to get there.	A	B	C	D	E
NOTE: Persistence in learning was a new index in the 2019 grade 12 mathematics assessment. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2019 Mathematics Assessment.

Index of Students' Enjoyment of Complex Problems

The table below presents the items forming the index of students' enjoyment of complex problems. This index was designed to measure students' enjoyment of problems and activities that challenge them to think. Grade 12 students were asked to indicate how much each of the four statement items described them (not at all like me, a little bit like me, somewhat like me, quite a bit like me, or very much like me).

Items in the index of twelfth-grade students' enjoyment of complex problems, grade 12 mathematics national assessment: 2019
How much does each of the following statements describe you? Select one answer choice on each row.
Response categories
Item			Not at all like me	A little bit like me	Somewhat like me	Quite a bit like me	Very much like me
B035101	a.	I like complex problems more than easy problems.	A	B	C	D	E
B035102	b.	I like activities that challenge my thinking abilities.	A	B	C	D	E
B035103	c.	I enjoy situations where I will have to think about something.	A	B	C	D	E
B035104	d.	I enjoy thinking about new solutions to problems.	A	B	C	D	E
NOTE: Enjoyment of complex problems was a new index in the 2019 grade 12 mathematics assessment. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2019 Mathematics Assessment.

Index of Students' Academic Self-Discipline

The table below presents the items forming the index of students' academic self-discipline. This index was designed to measure students' tendency to pay attention and stay on task when learning. Grade 12 students were asked to indicate how often they have done each of the four statement items (never or hardly ever, less than half of the time, about half of the time, more than half of the time, or all or almost all of the time).

Items in the index of twelfth-grade students' academic self-discipline, grade 12 mathematics national assessment: 2019
In this school year, how often have you done each of the following? Select one answer choice on each row.
Response categories
Item			Never or hardly ever	Less than half of the time	About half of the time	More than half of the time	All or almost all of the time
B035001	a.	I started working on assignments right away rather than waiting until the last minute.	A	B	C	D	E
B035002	b.	I paid attention and resisted distractions.	A	B	C	D	E
B035003	c.	I stayed on task without reminders from my teacher.	A	B	C	D	E
B035004	d.	I paid attention in class even when I was not interested.	A	B	C	D	E
NOTE: Academic self-discipline was a new index in the 2019 grade 12 mathematics assessment. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2019 Mathematics Assessment.

Index of Students’ Performance Goals in Mathematics

The table below presents the items forming the index of students' performance goals in mathematics. This index was designed to measure students' motivation to show others that they are good at math and get better grades than other students in their math class. Grade 12 students were asked to indicate how much each of the four statement items described them (not at all like me, a little bit like me, somewhat like me, quite a bit like me, or exactly like me).

Items in the index of twelfth-grade students' performance goals in mathematics, grade 12 mathematics national assessment: 2019
How much does each of the following statements describe you? Select one answer choice on each row.
Response categories
Item			Not at all like me	A little bit like me	Somewhat like me	Quite a bit like me	Exactly like me
M831501	a.	I want other students to think I am good at math.	A	B	C	D	E
M831502	b.	I want to show others that my math schoolwork is easy for me.	A	B	C	D	E
M831503	c.	I want to look smart in comparison to the other students in my math class.	A	B	C	D	E
M834004	d.	I want to get better grades than most other students in my math class.	A	B	C	D	E
NOTE: Performance goals in mathematics was a new index in the 2019 grade 12 mathematics assessment. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2019 Mathematics Assessment.

Index of Students’ Mastery Goals in Mathematics

The table below presents the items forming the index of students' mastery goals in mathematics. This index was designed to measure students' motivation to learn as much as possible and master new skills in their math class. Grade 12 students were asked to indicate how much each of the four statement items described them (not at all like me, a little bit like me, somewhat like me, quite a bit like me, or exactly like me).

Items in the index of twelfth-grade students' mastery goals in mathematics, grade 12 mathematics national assessment: 2019
How much does each of the following statements describe you? Select one answer choice on each row.
Response categories
Item			Not at all like me	A little bit like me	Somewhat like me	Quite a bit like me	Exactly like me
M831504	a.	I want to learn as much as possible in my math class.	A	B	C	D	E
M834102	b.	I want to master a lot of new skills in my math class.	A	B	C	D	E
M831505	c.	I want to become better in math this year.	A	B	C	D	E
M831506	d.	I want to understand as much as I can in my math class.	A	B	C	D	E
NOTE: Mastery goals in mathematics was a new index in the 2019 grade 12 mathematics assessment. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2019 Mathematics Assessment.

Index of Students’ Confidence in Mathematics Knowledge and Skills

The table below presents the items forming the index of students' confidence in their mathematics knowledge and skills at grade 12. This index was designed to measure students' belief in their abilities to do various math-related tasks. Grade 12 students were asked to indicate their confidence in doing the task described in each of eight items (I definitely can’t, I probably can’t, maybe, I probably can, or I definitely can).

Items in the index of twelfth-grade students' confidence in mathematics knowledge and skills, grade 12 mathematics national assessment: 2019
Thinking about math, do you think that you would be able to do each of the following? Do not actually solve the problems. Select one answer choice on each row.
Response categories
Item			I definitely can't	I probably can't	Maybe	I probably can	I definitely can
M831408	a.	Create an expression that represents the average number of miles you run in a week if you run 100 miles in w weeks	A	B	C	D	E
M831403	b.	Determine a 20 percent tip of a 67-dollar restaurant dinner bill	A	B	C	D	E
M831404	c.	Describe the properties shared by every isoceles right triangle	A	B	C	D	E
Thinking about math, do you think that you would be able to do each of the following? Select one answer choice on each row.
Response categories
Item			I definitely can't	I probably can't	Maybe	I probably can	I definitely can
M831412	a.	Decide if your classmate's math work is correct	A	B	C	D	E
M831409	b.	Give an example to show that a math statement is false	A	B	C	D	E
M831410	c.	Explain to your classmate how you solved a math problem	A	B	C	D	E
M831411	d.	Use correct mathematical words and symbols when showing your work	A	B	C	D	E
M831413	e.	Use definitions of geometric shapes to support an argument	A	B	C	D	E
NOTE: Confidence in mathematics knowledge and skills was a new index in the 2019 grade 12 mathematics assessment. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2019 Mathematics Assessment.

Index of Students’ Interest/Enjoyment in Mathematics

The table below presents the items forming the index of students' interest/enjoyment in mathematics. This index was designed to measure students' interest in and enjoyment of math, and their views of the importance of math. Grade 12 students were asked to indicate how much each of the seven statement items described them (not at all like me, a little bit like me, somewhat like me, quite a bit like me, or exactly like me).

Items in the index of twelfth-grade students' interest/enjoyment in mathematics, grade 12 mathematics national assessment: 2019
How much does each of the following statements describe you? Select one answer choice on each row.
Response categories
Item			Not at all like me	A little bit like me	Somewhat like me	Quite a bit like me	Exactly like me
M831901	a.	I enjoy doing math.	A	B	C	D	E
M831902	b.	I look forward to my math class.	A	B	C	D	E
M831903	c.	I am interested in the things I learn in math.	A	B	C	D	E
M831904	d.	I think making an effort in math is worthwhile.	A	B	C	D	E
M831905	e.	I think math will help me even when I am not in school.	A	B	C	D	E
M831906	f.	I think it is important to do well in math.	A	B	C	D	E
M831907	g.	I take mathematics because it will help me in the future.	A	B	C	D	E
NOTE: Interest/enjoyment in mathematics was a new index in the 2019 grade 12 mathematics assessment. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2019 Mathematics Assessment.

Index Item Scoring

Analyses were conducted separately for the paper-based assessment (PBA) and digitally based assessment (DBA) samples. The 2019 grade 12 mathematics indices are reported based on both PBA and DBA data. Although the scaling of each index was performed by mode, measurement invariance was established between the PBA indices and the DBA indices.

For each index item, the response categories were scored as numerical values (e.g., for an item with five response categories, category A was scored as 1, B was scored as 2, C was scored as 3, D was scored as 4, and E was scored as 5). For five of the 2019 grade 12 mathematics indices, item response categories were collapsed; scores for a five-category item thus became 1, 2, 3, and 4 after collapsing. The first two tables on the page Index Scoring for the 2019 Grade 12 Mathematics Indices describe the treatment of the index items.

IRT Parameters

The partial credit IRT model was used to scale the indices. Scaling of the index items was first conducted to get the item parameters and was based on marginal maximum likelihood methodologies. The tables on the page Index Scoring for the 2019 Grade 12 Mathematics Indices show the IRT parameters for the 2019 grade 12 mathematics indices.

Response Averages and Transformed Scores

After the parameters were estimated, EAP scores were calculated as the estimate of the index score, and transformed to a reporting scale of 0–20. The tables on the page Index Scoring for the 2019 Grade 12 Mathematics Indices show the response averages and transformed scale scores for each of the grade 12 mathematics indices. Note that response averages represent the average scored responses after score collapsing (if there was any collapsing). Each response average corresponds to one transformed score. The increment used for increases in the response average is determined by the number of items that form an index. For example, an index with four items would have response averages that increase by an increment of 1 / 4 or 0.25, while an index with five items would have response averages that increase by an increment of 1 / 5 or 0.20.

As a reporting aid, index scores were divided into a range of categories or classifications (e.g., low, moderate, high). The cut points selected to divide the index scores into meaningful categories were based on the distributions of the response average of each index.

As an example, for the index of persistence in learning, grade 12 students were classified as follows:

Students with a response average less than 2 were classified as low on the index. That is, students who on average responded not at all like me or a little bit like me on a question with the five response options not at all like me, a little bit like me, somewhat like me, quite a bit like me, and very much like me were classified as having a low level of persistence in learning.
Students with a response average greater than or equal to 2 to less than 3 were classified as moderate on the index. That is, students who on average responded somewhat like me were classified as having a moderate level of persistence in learning.
Finally, students with a response average greater than or equal to 3 were classified as high on the index. That is, students who on average responded quite a bit like me or very much like me were classified as having a high level of persistence in learning.

Last updated 31 March 2025 (SK)

Printer-friendly Version

​​​NAEP Technical DocumentationDevelopment of the 2019 Grade 12 Mathematics Indices

NAEP Technical DocumentationDevelopment of the 2019 Grade 12 Mathematics Indices