|
Chapter 3. Mathematical Content of Lessons Any observer of classroom instruction is struck by its complexity. Instruction is multidimensional. Instruction also unfolds quickly in real time, comprised as it is of an unceasing flow of events. To focus on one dimension is to lose sight of the others, which is why video is so useful in the study of classroom instruction. With video, we can make multiple passes through the lesson, focusing in each pass on a particular dimension or layer of description. For purposes of analysis, we can describe instruction in layers. Although these layers may differ in character across cultures, the layers themselves as analytical points of reference have validity across cultures. These layers include the following:
GENERAL DESCRIPTIONS OF CONTENT Our first step in describing the mathematical content of each lesson was to apply the TIMSS content coding system. The complete system, which included 44 categories, is available in Robitaille, McKnight, Schmidt, Britton, Raizen, and Nicol (1993). All coding was done from the lesson tables. For each segment in the table, the coder wrote down the TIMSS code that best described the mathematical content. Each lesson was thus described using one or more content codes. TIMSS content coding was checked by independent coders at Michigan State University (MSU)--the same coders who had done the textbook coding for the TIMSS curriculum analysis. They, like the UCLA coders, based their analysis on the video lesson tables. There was perfect agreement between coders at UCLA and at MSU. In general, more topics were represented in the sample of U.S. videotapes (24 topics) than in the samples of German (18 topics) or Japanese (13 topics) tapes. The 44 TIMSS content coding system categories were further grouped into 10 major categories:
Percentage of lessons in each country in which content belonged to each of the ten major content categories
NOTE: 1.1=Numbers; 1.2=Measurement; 1.3=Geometry (Position, Visualization, Shape); 1.4=Geometry (Symmetry, Congruence, Similarity); 1.5=Proportionality; 1.6=Functions, Relations, Equations; 1.7=Data Representation, Probability, Statistics; 1.8=Elementary Analysis; 1.9=Validation and Structure; 1.10=Other. SOURCE: U.S. Department of Education, National Center for Education Statistics, Third International Mathematics and Science Study, Videotape Classroom Study, 1994-95.
HOW ADVANCED IS THE CONTENT BY INTERNATIONAL STANDARDS? It is not possible, a priori, to say that one topic is more complex than another. However, it is possible to make an empirical judgment of how advanced a topic is based on its placement in mathematics curricula around the world. We were able to make use of the TIMSS curriculum analyses, conducted by William Schmidt and his colleagues at MSU, to make such a judgment. The TIMSS content codes for each lesson (Robitaille et al., 1993) were assigned a number indicating the modal grade level at which the majority of the 41 countries studied gave the most concentrated curricular attention to the topic. Average level for each lesson was obtained by averaging the MSU index for all topics coded for the lesson. The average grade level of topics covered in the video sample, as indicated by the MSU index, is shown in figure 10. In terms of this index, the average grade level of topics covered in the U.S. sample was significantly different than in Germany and Japan. Based on the MSU index of international standards, the mathematical content of the U.S. lessons in the videotape study was at a seventh-grade level, whereas the German and Japanese lessons fell at the high eighth- or even ninth-grade level.
Average grade level of content by international standards
We begin a more detailed look at content by examining teachers' responses on the teacher questionnaire. We asked teachers whether the content of the lesson was all review, all new, or somewhere in between. Responses to this question are shown in figure 11. Analyses indicated that the distribution of responses in Japan differed significantly from that in both Germany and the United States. Teachers' description of the content of the videotaped lesson on a continuum from "all review" to "all new"
We also found significant differences when we asked teachers what main thing they wanted students to learn from the lesson. Responses were coded into five categories:
The teachers' responses are shown in figure 12. There was a significant difference between the reported goals of teachers in Japan and teachers in the other two countries. A majority of Japanese teachers reported that thinking was the main goal/skill to be learned from their lessons, while 55 percent of German teachers and 61 percent of U.S. teachers reported that skills were the main thing to be learned.
Teachers' responses, on the questionnaire, to the question, "What was the main thing you wanted students to learn from today's lesson?"
Number of Topics and Topic Segments per Lesson Having determined which topics are taught in each country, we proceeded to divide lessons into topic segments (i.e., the points in the lesson at which the topic shifts from one to another). By topic, we here refer to the TIMSS content topics, which are broadly defined. A shift in topic is a clear shift in the primary content of the lesson and is usually marked by an announcement by the teacher. For example, in lesson GR-080 (00:28:48) the teacher says: "Okay let's still start something new today. Your school exercise book please." This is a clear signal that the topic is about to change, thus marking the beginning of a new topic segment. If a lesson has only one topic then it will, by definition, have only one topic segment. However, many lessons had more than one topic. In this case, the number of topic segments might equal the number of topics (i.e., there could be one segment for each topic) or it might exceed the number of topics if, for example, the lesson alternated from a segment of one topic, to a segment of another, to another segment of the original topic. US-003 is an example of a lesson that contains two topics and three topic segments. The teacher begins the lesson with proportionality, assigning students to work on the following problem:
At 22:05, the teacher shifts to focus on a new topic, inequalities, with the following statement:
Students work on discussing the inequalities they had done for homework until, at 35:12, the teacher changes the focus back to proportionality, assigning exercises from the textbook until the end of class. Thus, the first and third topic segments focused on proportionality, the second segment, on inequalities. Average number of topics and topic segments per videotaped lesson in each country
As shown in figure 13, U.S. lessons contained significantly more topics and topic segments than did Japanese lessons. Also, German lessons contained more topics and topic segments than did Japanese lessons. We next examined, for each topic covered in the lesson, whether the topic included concepts, applications, or both. We defined these mutually exclusive categories broadly to catch all instances in which students might have constructed concepts or learned to apply them.
An example of a topic coded as CONCEPT can be seen in GR-074. The lesson deals with calculations involving polygons. CONCEPT was coded because solution methods were stated abstractly, as formulas, rather than using the specific numbers of a particular problem. Here is an excerpt from the lesson transcript:
Another example of a topic coded as CONCEPT is found in JP-040. The topic of this lesson is similarity. The teacher first presents the definition of similar figures and, together with the students, generates examples. The teacher then demonstrates that operations of multiplication and division can be used but not addition and subtraction. Finally, the students derive a triangle's similarity conditions from its congruence conditions. Students are asked to memorize the conditions for similarity. APPLICATIONS are coded when there is no explicit mention of mathematical concepts. In GR-096, for example, the topic is representation of data. The teacher reviews different types of charts by using illustrations as examples (figure 14).
Pictures of the chalkboard from GR-096
Teacher and students talk briefly about the names of different kinds of charts, as well as what different values in the charts represent. The application begins when the teacher hands out data in tabular form and asks students, working in groups, to create their own charts from the data. As shown in figure 15, U.S. lessons had significantly lower percentages of topics that consisted of concepts only than did either German or Japanese lessons. And, U.S. and German lessons had a higher percentage of topics that consisted of applications only than did Japanese lessons.
Average percentage of topics in each lesson that include concepts, applications, or both
WERE CONCEPTS STATED OR DEVELOPED? When concepts were included in the lesson, they could be stated or they could be developed. A concept was coded as STATED if it was simply provided by the teacher or students but not explained or derived. For example, the teacher, in the course of solving a problem at the board, might simply remind the students of the Pythagorean theorem (e.g., "The formula for finding the length of the hypotenuse of a right triangle is a2 +b2 = c2") in order to guide the solution of the problem. The focus here is on the mathematical information itself rather than on the process of deriving it. A concept was coded as DEVELOPED when it was derived and/or explained by the teacher or the teacher and students collaboratively in order to increase students' understanding of the concept. The form of the derivation could be through proof, experimentation, or both. US-068 provides an example of a concept being developed through experimentation. The topic of the lesson is the relationship between circumference and Pi. The teacher begins the lesson by defining terms such as circumference and diameter. Students then break into groups to work with circular objects. They measure the objects' circumferences (C) and diameters (D) with a measuring tape (figure 16). They then divide C by D and examine their answers. In a subsequent class discussion, the teacher uses the commonality across answers as a basis for defining Pi.
Materials used in US-068
US-061 also engages students in a concrete example. This time, however, the concept is coded as STATED. The topic is data representation. The teacher asks all students to write their shoe size on a piece of paper and asks them to stand in order according to shoe size from smallest to largest (figure 17). The teacher then defines several statistical terms (e.g., range and mean) and asks the students to calculate the statistics using their shoe sizes. This is not coded as development because the concepts are not derived, only stated and applied in the example.
A view of the classroom in US-061
The results of coding concepts as STATED versus DEVELOPED are shown in figure 18. Concepts were significantly more likely to be STATED in the U.S. lessons than in both Germany and Japan. Conversely, concepts were more likely to be DEVELOPED in German and Japanese lessons than in the U.S. lessons.
Average percentage of topics in eighth-grade mathematics lessons that contained concepts that were stated or developed
Did Applications Increase in Complexity? Finally, in topics that contained applications, we were interested in the relationship among the application problems. Specifically, when a topic contained more than one problem, were the problems just multiple examples of the same level of complexity, or did they increase in complexity over the course of the lesson? Change in complexity--categorized as either INCREASE or SAME/DECREASE--was coded only if there was more than one application problem within the same topic in a lesson. Increase in complexity was coded when either a procedural or conceptual difficulty was added from one application problem to another. Increasing procedural difficulties generally consisted of additional operations (i.e., calculations previously executed separately were now combined in one application). Increasing conceptual difficulties generally consisted of added mathematical information (i.e., previously learned concepts now had to be modified for a new application using more mathematical information). US-018 provides a good example of increasing complexity across two application problems. The lesson deals with area and circumference of a circle. In the first problem students are asked to find the area of the shaded region between the circle and the square (figure 19). In order to solve this problem the students had to find the area of the circle and then subtract that from the area of the polygon.
Drawing from chalkboard of first problem in US-018
Students next were asked to find the area of a semicircle in which a triangle was inscribed (figure 20) and then the area of the region not covered by the triangle. This problem, although it employed some of the same concepts of the previous one, is clearly more complex. To solve it, students had to find the hypotenuse of the triangle (which is also the diameter of the circle), calculate the area of the semicircle, calculate the area of the triangle, and then subtract the area of the triangle from the area of the semicircle.
Drawing from chalkboard of second problem in US-018
The results of coding this distinction are shown in figure 21. The Japanese applications were significantly more likely to increase in complexity than those in Germany. There was no difference in this regard between the United States and the other two nations.
Average percentage of topics in each lesson that contained applications that increased in complexity vs. stayed the same or decreased over the course of the lesson
Often the teacher's goal is to teach students how to solve a specific type of mathematics problem (e.g., an algebraic equation or a geometric construction). This can be accomplished by presenting one solution method and asking students to use it on similar problems or by encouraging the development of different methods and examining their relative advantages. We coded whether an alternative solution method was presented by the teacher, or by students, during the course of each lesson. Panel (a) of figure 22 shows the percentage of lessons that included alternative solution methods of each type; Panel (b) shows the average number of alternative solution methods of each type presented in the lessons of the three countries. U.S. lessons included significantly more teacher-presented alternative solution methods than did Japan. Japanese lessons included significantly more student-presented alternative solution methods than did either German or U.S. lessons.
(a) Percentage of lessons that included teacher-presented and student-presented alternative solution methods; (b) average number of teacher- and student-presented alternative solution methods presented per lesson
Principles, Properties, and Definitions As described earlier, when describing the content of each lesson we not only recorded tasks and situations but also the PRINCIPLES, PROPERTIES, and DEFINITIONS (PPD) that were stated in the lesson. Although these PPDs filled a relatively small percentage of lesson time, they nevertheless could be crucial for moving along the content of the lesson. We were not able to reliably differentiate PRINCIPLES and PROPERTIES, but we were able to differentiate between these and DEFINITIONS. To be coded as a DEFINITION, a statement of mathematical information had to include a general statement (generic term) followed by the defining characteristics or properties. For example: A ray is a straight line (generic term) that has a beginning point but no end point (defining characteristics). A statement that was not a complete definition in this sense was coded as PRINCIPLE/PROPERTY. Any mathematical information stated in the lesson that was not coded as a DEFINITION was coded as PRINCIPLE/PROPERTY. An example of a DEFINITION is seen in the following excerpt from GR-022. The topic of this lesson is congruence of triangles. Here is a part of the lesson transcript:
Another example is from the Japanese lesson JP-035, which deals with the topic of similarity of geometric figures. The teacher writes the word "similarity" on the board, then says:
An example of PRINCIPLE/PROPERTY is from JP-039, which dealt with parallel lines and similarity. The teacher first reviewed four theorems on this topic that were covered in a previous lesson, then introduced the fifth theorem, namely, the midpoint connection theorem (figure 23). The theorem is clearly stated on the chalkboard. (Some of the writing on the chalkboard has been digitally enhanced to improve readability.)
Excerpt from chalkboard from JP-039, with English translation In the triangle ABC, if we make M and N the midpoints of lines AB and AC respectively, MN//BC MN = 1/2 BC
A few other examples of PRINCIPLES/PROPERTIES are found in the following teacher statements:
The average number of PRINCIPLES/PROPERTIES and DEFINITIONS in each lesson is graphed in figure 24. There was no difference across countries in the number of PRINCIPLES/PROPERTIES per lesson. U.S. lessons, however, contained more DEFINITIONS per lesson than did German lessons. The U.S. and Japan, and Germany and Japan, did not differ from each other in this regard.
Average number of principles/properties and definitions in each German, Japanese, and U.S. eighth-grade mathematics lesson
Constructing proofs is an important mathematical activity because it provides a reasoned method of verification based on the accepted assumptions and observations of the discipline. Many reform documents, for example, those by the National Council of Teachers of Mathematics (1989, 1991), recommend that students should have increasing opportunities to examine and construct mathematical proofs. We coded a lesson as including a proof if an assumption was presented, a proof executed, and the assumption confirmed as correct. The proof could be presented by the teacher, by a student, or worked out collaboratively during classwork. As long as an assumption was presented and the strategy for proving it was discussed, it was still considered a proof, even if the proof was not executed. Likewise, if a proof was started but not completed due to running out of time in the lesson, we still coded the lesson as including a proof. Analysis revealed that a greater percentage of the Japanese lessons included proofs than either the German or U.S. lessons. Indeed, 10 percent of German lessons included proofs while 53 percent of Japanese lessons included proofs. None of the U.S. lessons included proofs.1
FINDINGS OF THE MATH CONTENT GROUP We turn now to present the findings of the independent Math Content Group. Recall that this group analyzed the content of 30 lessons from each country, 15 algebra and 15 geometry. The Math Content Group based its analyses on the detailed descriptions of mathematical content contained in the lesson tables, as previously described. To reduce the likelihood of bias, tables were disguised (e.g., references to currency and other country-specific contents were altered) so that it was not possible to tell for certain which country the lessons came from. After analyses were complete, results were then tabulated by country. Content descriptions in the lesson tables were subjected to a detailed series of analyses. The first step was to construct a directed graph representation of each lesson. The purpose of the directed graph was to show the content and flow of the lesson in shorthand notation so that patterns within and among lessons would become more apparent. In this graph, the content of each lesson was represented as a set of nodes (depicted by circles) and links among the nodes (depicted by arrows), depending on relationships existing among the nodes. Both nodes and links were then labeled according to the coding system developed by the group. Definitions of each code will be presented later. For now, however, it is useful to go through one lesson in detail, showing how the Math Content Group transformed the content description in the table into a directed graph. For convenience, we will use a Japanese lesson (JP-012) to illustrate this process. The lesson table for JP-012 was presented earlier. The directed graph produced by the Math Content Group is shown in figure 25.
Directed graph representation of a Japanese lesson (JP-012) as constructed by the Math Content Group
For purposes of content analysis, lesson JP-012 was divided into three content segments. The homework segment at the end was not counted as a separate segment. Each segment became a node on the graph. We will now review the construction of the directed graph segment by segment.
The directed graphs took a number of different forms. One additional example is shown in figure 26. Our purpose in presenting this graph is simply to illustrate the general level of complexity of the graphs. We will forgo a detailed explanation of exactly how this graph was derived.
Analyses of the Directed Graphs The first analysis based on the directed graphs was simply to count the number of nodes and links of each lesson. The average number of nodes and links for lessons in each country are presented in figure 27. There was no significant difference in the number of nodes or links across countries.
Average number of nodes and links on the directed graph representations of lessons in each country
When we examine the structure of the directed graphs, however, we begin to see cross-national differences. Two indicators in particular attracted our interest:
These indicators seem to measure the coherence of the lesson content because they measure the interconnectedness of different content segments. The distribution of lessons by number of components and number of leaves is shown in figure 28. The distribution in Japan was significantly different than that in the United States. For example, Japanese and German lessons were more likely than U.S. lessons to contain only one component. Japanese lessons were also more likely than lessons in the United States to contain one leaf. This suggests that the content of lessons is more coherent in Japan than in the United States.
(a) Percentage of lessons that included one, two, or more than two components; (b) percentage of lessons that included one, two, or more than two leaves
Further Analyses of Nodes and Links Having examined the number of nodes and links and the structure of how nodes are connected in the lessons of each country, we turn now to examine the characterizations of nodes and links provided by the Math Content Group. A number of characteristics were coded for nodes and for links. For nodes, the following codes were applied when the group of coders judged them present:
For links, the following codes were applied:
The percentage of lessons that included nodes coded as ILLUSTRATION, MOTIVATION, INCREASE IN COMPLEXITY, or DEDUCTIVE REASONING is shown in figure 29. (Few nodes were coded as INDUCTIVE REASONING, and, therefore, were not included in the following analysis.) There was no significant difference across countries in the percentage of lessons containing ILLUSTRATION or INCREASE IN COMPLEXITY nodes. However, Germany and Japan had significantly more lessons containing MOTIVATION nodes than did the United States. And Japan had the largest percentage of lessons containing DEDUCTIVE REASONING nodes, while the United States had the smallest percentage.
Percentage of lessons with nodes coded to include illustrations, motivations, increase in complexity, and deductive reasoning
Some indication of how content was developed over the course of each lesson was gained by coding the kinds of links that connected parts of the lesson together (see figure 30). Increasing complexity between two nodes was coded more often in Japanese and German lessons than U.S. lessons. Japanese lessons contained significantly more links coded as necessary than did U.S. lessons.
Percentage of lessons containing links coded as increase in complexity and necessary result/process
An overall summary of the Math Content Group's coding can be obtained by adding up the total number of positive characteristics coded for each lesson. Thus, for each directed graph, we simply added up the number of codes that were attached to nodes (i.e., Motivation + Illustration + Deductive Reasoning + Inductive Reasoning + Increase in Complexity), and the number of codes that were attached to links (i.e., Necessary + Helpful + Similar + Deductive/Inductive Reasoning + Increase in Complexity). Figure 31 shows the average number of codes per node and per link for lessons in the three countries. Japanese lessons contained significantly more codes per node than either German or U.S. lessons; and U.S. lessons contained significantly fewer codes per link than lessons in the other two countries.
Average number of codes per node and per link in German, Japanese, and U.S. lessons
The Math Content Group developed two more codes to indicate the kinds of tasks which were engaged in during the lesson. The first of these additional codes was Task Complexity. Each task was categorized as either single-step or multi-step. Lessons were then categorized as containing mostly single-step tasks, equal number of single- and multi-step tasks, or mostly multi-step tasks. The results are shown in figure 32. None of the pairwise differences was significant.2
Percentage of lessons in each country containing mostly single-step, mostly multi-step, or equal numbers of the two types of tasks
The second coding of tasks was for what the Math Content Group called LOCUS OF CONTROL. What level of choice did students have in determining how to perform the task? Were they all controlled by the task, or was some of the control left up to the student? For example, if the teacher had just demonstrated how to solve a problem, then asked the students to try applying the same method to a similar problem, it was coded as TASK CONTROLLED. This is because the students were not asked to make any decisions about how to approach the problem, only to follow the exact procedure demonstrated by the teacher. On the other hand, if the teacher asked students to see if they could think of another method for solving a problem it was coded as SOLVER CONTROLLED, because the student had the freedom to decide which of several possible approaches they would take. In figure 33 we show the percentage of lessons that contained all Task Controlled tasks, all Solver Controlled, or a mixture of the two. Seventeen and 48 percent of Japanese and German lessons contained all task controlled tasks, respectively, while the share was 83 percent for U.S. lessons.
Percentage of lessons containing task controlled tasks, solver controlled tasks, or a combination of task and solver controlled tasks
In addition to constructing the directed graph representation of each lesson and coding the nodes and links, the members of the Math Content Group also assigned a rating to each lesson that reflected their overall judgment of the quality of the mathematical content in the lesson. They rated each lesson on a three-point scale: low, medium, or high. (Where raters disagreed on this judgment, the disagreement was resolved by discussion so that, in the end, all four raters agreed.) Although the measure is subjective, there was high agreement among the independent raters. A summary of these ratings is presented in figure 34. The distributions of ratings differed significantly between the United States and the other two countries; German and Japanese lessons were rated higher in quality than U.S. lessons.3 Twenty-eight and 39 percent of German and Japanese lessons, respectively, received the highest rating; none of the U.S. lessons did. Eighty-nine percent of U.S. lessons received the lowest rating.
Percentage of lessons rated as having low, medium, and high quality of mathematical content
How well do the specific codes we have described relate to these more global judgments? Correlations were calculated between specific indicators constructed by the Math Content Group and the group's overall ratings of content quality. The single highest predictor (Pearson r = .66) was the total number of codes on both nodes and links in a lesson divided by the total number of nodes in the lesson. The average of this index for Japanese lessons was 1.17, for German lessons, .60, and for U.S. lessons, .25. The differences between all pairs of countries were statistically significant.4 In summary, we found a number of differences in the mathematical content of lessons across the three countries. Japanese lessons, in general, were found to be more advanced by international standards, and richer in content on several dimensions, than were U.S. lessons. German lessons tended to fall between the Japanese and U.S. lessons on some dimensions, or differed from the United States on one measure and from Japan on the next. We move, now, to consider how teachers presented content to students in the three countries.
1 Standard errors for German, Japanese, and U.S. lessons are 4.32, 3.45, and 0, respectively. 2 On a three-point scale where 1 indicates "More single-step" and 3 indicates "More multi-step," the averages (with standard errors) for German, Japanese, and U.S. lessons were 2.3 (0.17), 2.7 (0.11), and 2.2 (0.19), respectively. 3 On a three-point scale where 1 indicates "low" and 3, "high" quality, estimates (and standard errors) for German, Japanese, and U.S. lessons were 1.9 (0.14), 2.3 (0.12), and 1.1 (0.06), respectively. 4 Standard errors for Germany, Japan, and the U.S. are 0.07, 0.16, and 0.05, respectively. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Theorem of midpoint connection:
 |
 |
 |
 |
Questions, problems or comments with this website? Contact timss@ed.gov