Experimental Impacts of Learning Trajectory–Oriented Formative Assessment on Student Problem-Solving Accuracy and Strategy Sophistication

View More View Less
  • 1 University of Pennsylvania
  • | 2 American Institutes for Research

In this article, we use a two-dimensional assessment to examine the experimental impacts of a mathematics learning trajectory–oriented formative assessment program on student strategies for problems involving multiplication and division. Working from the theory that the development of students’ multiplicative reasoning involves improvements in both problem-solving accuracy and sophistication of strategies used to solve problems, we designed an assessment instrument to measure both dimensions of student learning. The instrument was used to measure the impact of the Ongoing Assessment Project (OGAP), which develops teachers’ capacity to regularly assess student thinking in relation to a learning progression to develop instructional responses that are based on evidence of student thinking. The results showed significant impacts of OGAP on both students’ problem-solving accuracy and the sophistication of their strategy. The findings suggest that capturing both dimensions of students’ multiplicative reasoning offers important information for researchers and program designers who seek to understand different dimensions of student mathematics performance.

Footnotes

This research was funded by the National Science Foundation (Grant No. DRK-12 1316527). Any opinions, findings, and conclusions or recommendations expressed in this article are those of the authors and do not necessarily reflect the views of the National Science Foundation.

The guest editor for this article was Marcy Wood.

Contributor Notes

Jonathan A. Supovitz, Consortium for Policy Research in Education, University of Pennsylvania, 3440 Market Street Suite 560, Philadelphia, PA 19104; jons@upenn.edu

Caroline B. Ebby, Consortium for Policy Research in Education, University of Pennsylvania, 3440 Market Street Suite 560, Philadelphia, PA 19104; cbe@upenn.edu

Janine T. Remillard, Consortium for Policy Research in Education, University of Pennsylvania, 3440 Market Street Suite 560, Philadelphia, PA 19104; janiner@gse.upenn.edu

Robert Nathenson, American Institutes for Research, 1400 Crystal Drive, 10th Floor, Arlington, VA 22202; rnathenson@air.org

Journal for Research in Mathematics Education
  • Ball, D. L., Thames, M. H., & Phelps, G. (2008). Content knowledge for teaching: What makes it special? Journal of Teacher Education, 59(5), 389407. https://doi.org/10.1177/0022487108324554

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 774. https://doi.org/10.1080/0969595980050102

    • Search Google Scholar
    • Export Citation
  • Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21(1), 531. https://doi.org/10.1007/s11092-008-9068-5

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bunker, J. (2007). The case for ongoing assessment based on cognitive research [Unpublished manuscript]. University of Vermont.

  • Carpenter, T. P., Fennema, E., & Franke, M. L. (1996). Cognitively guided instruction: A knowledge base for reform in primary mathematics instruction. The Elementary School Journal, 97(1), 320.https://doi.org/10.1086/461846

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Carpenter, T. P., Fennema, E., Franke, M. L., Levi, L., & Empson, S. B. (1999). Children’s mathematics: Cognitively guided instruction. Heinemann.

    • Search Google Scholar
    • Export Citation
  • Carpenter, T. P., Franke, M. L., Jacobs, V. R., Fennema, E., & Empson, S. B. (1998). A longitudinal study of invention and understanding in children’s multidigit addition and substraction. Journal for Research in Mathematics Education, 29(1), 320. https://doi.org/10.2307/749715

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Carr, M., & Jessup, D. L. (1997). Gender differences in first-grade mathematics strategy use: Social and metacognitive influences. Journal of Educational Psychology, 89(2), 318328. https://doi.org/10.1037/0022-0663.89.2.318

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Che, M., Wiegert, E., & Threlkeld, K. (2012). Problem solving strategies of girls and boys in single-sex mathematics classrooms. Educational Studies in Mathematics, 79(2), 311326. https://doi.org/10.1007/s10649-011-9346-x

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Clements, D. H., & Sarama, J. (2007). Effects of a preschool mathematics curriculum: Summative research on the Building Blocks project. Journal for Research in Mathematics Education, 38(2), 136163.

    • Search Google Scholar
    • Export Citation
  • Clements, D. H., & Sarama, J. (2008). Experimental evaluation of the effects of a research-based preschool mathematics curriculum. American Educational Research Journal, 45(2), 443494. https://doi.org/10.3102/0002831207312908

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Clements, D. H., & Sarama, J. (2009). Learning and teaching early math: The learning trajectories approach. Routledge. https://doi.org/10.4324/9780203883389

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Clements, D. H., Sarama, J., Spitler, M. E., Lange, A. A., & Wolfe, C. B. (2011). Mathematics learned by young children in an intervention based on learning trajectories: A large-scale cluster randomized trial. Journal for Research in Mathematics Education, 42(2), 127166. https://doi.org/10.5951/jresematheduc.42.2.0127

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Clements, D. H., Sarama, J., Wolfe, C. B., & Spitler, M. E. (2013). Longitudinal evaluation of a scale-up model for teaching mathematics with trajectories and technologies: Persistence of effects in the third year. American Educational Research Journal, 50(4), 812850. https://doi.org/10.3102/0002831212469270

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Confrey, J., Maloney, A. P., Nguyen, K. H., & Rupp, A. A. (2014). Equipartitioning, a foundation for rational number reasoning: Elucidation of a learning trajectory. In A. P. Maloney, J. Confrey, & K. H. Nguyen (Eds.), Learning over time: Learning trajectories in mathematics education (pp. 6196). Information Age.

    • Search Google Scholar
    • Export Citation
  • Daro, P., Mosher, F. A., & Corcoran, T. (2011). Learning trajectories in mathematics: A foundation for standards, curriculum, assessment, and instruction (Research Report No. RR-68). Consortium for Policy Research in Education. https://doi.org/10.12698/cpre.2011.rr68

    • Search Google Scholar
    • Export Citation
  • Downey, D. B., & Vogt Yuan, A. S. (2005). Sex differences in school performance during high school: Puzzling patterns and possible explanations. The Sociological Quarterly, 46(2), 299321. https://doi.org/10.1111/j.1533-8525.2005.00014.x

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ebby, C. B., & Nathenson, R. (2019, April 1–2). Beyond correctness: Strategy use in multiplicative reasoning performance [Paper presentation]. National Council of Teachers of Mathematics Research Conference, San Diego, CA, United States.

    • Search Google Scholar
    • Export Citation
  • Ebby, C. B., & Petit, M. (2018). Using learning trajectories to elicit, interpret, and respond to student thinking. In E. A. Silver & V. L. Mills, A fresh look at formative assessment in mathematics teaching (pp. 81101). National Council of Teachers of Mathematics.

    • Search Google Scholar
    • Export Citation
  • Ebby, C. B., Remillard, J., & D’Olier, J. H. (2019). Pathways for analyzing and responding to student work for formative assessment: The role of teachers’ goals for student learning [Working paper]. Consortium for Policy Research in Education, University of Pennsylvania.

    • Search Google Scholar
    • Export Citation
  • Edgington, C. P. (2012). Teachers’ uses of a learning trajectory to support attention to students’ mathematical thinking [Doctoral dissertation, North Carolina State University]. University Libraries. https://repository.lib.ncsu.edu/handle/1840.16/8087

    • Search Google Scholar
    • Export Citation
  • Edgington, C., Wilson, P. H., Sztajn, P., & Webb, J. (2016). Translating learning trajectories into useable tools for teachers. Mathematics Teacher Educator, 5(1), 6580. https://doi.org/10.5951/mathteaceduc.5.1.0065

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ellenberg, J. H. (1996). Intent-to-Treat Analysis versus As-Treated Analysis. Drug Information Journal, 30(2), 535544. https://doi.org/10.1177/009286159603000229

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fennema, E., Carpenter, T. P., Franke, M. L., Levi, L., Jacobs, V. R., & Empson, S. B. (1996). A longitudinal study of learning to use children’s thinking in mathematics instruction. Journal for Research in Mathematics Education, 27(4), 403434. https://doi.org/10.2307/749875

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fennema, E., Carpenter, T. P., Jacobs, V. R., Franke, M. L., & Levi, L. W. (1998). A longitudinal study of gender differences in young children’s mathematical thinking. Educational Researcher, 27(5), 611. https://doi.org/10.3102/0013189X027005006

    • Search Google Scholar
    • Export Citation
  • Furtak, E. M. (2009, June 24–26). Toward learning progressions as teacher development tools [Paper presentation]. Learning Progressions in Science Conference, Iowa City, IA, United States.

    • Search Google Scholar
    • Export Citation
  • Gallagher, A. M., & De Lisi, R. (1994). Gender differences in Scholastic Aptitude Test: Mathematics problem solving among high-ability students. Journal of Educational Psychology, 86(2), 204211. https://doi.org/10.1037/0022-0663.86.2.204

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gallagher, A. M., De Lisi, R., Holst, P. C., McGillicuddy-De Lisi, A. V., Morely, M., & Cahalan, C. (2000). Gender differences in advanced mathematical problem solving. Journal of Experimental Child Psychology, 75(3), 165190. https://doi.org/10.1006/jecp.1999.2532

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Goodchild, S., & Grevholm, B. (2009). An exploratory study of mathematics test results: What is the gender effect? International Journal of Science and Mathematics Education, 7(1), 161182. https://doi.org/10.1007/s10763-007-9114-7

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Greer, B. (1992). Multiplication and division as models of situations. In D. A. Grouws (Ed.), Handbook of research on mathematics teaching and learning (pp. 276295). National Council of Teachers of Mathematics.

    • Search Google Scholar
    • Export Citation
  • Hackenberg, A. J. (2010). Students’ reasoning with reversible multiplicative relationships. Cognition and Instruction, 28(4), 383432. https://doi.org/10.1080/07370008.2010.511565

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Routledge. https://doi.org/10.4324/9780203887332

    • Search Google Scholar
    • Export Citation
  • Heritage, M. (2008). Learning progressions: Supporting instruction and formative assessment. Council of Chief State School Officers.

  • Heritage, M. (2013). Formative assessment in practice: A process of inquiry and action. Harvard University Press.

  • Kimball, M. M. (1989). A new perspective on women’s math achievement. Psychological Bulletin, 105(2), 198214. https://doi.org/10.1037/0033-2909.105.2.198

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kingston, N., & Nash, B. (2011). Formative assessment: A meta-analysis and a call for research. Educational Measurement: Issues and Practice, 30(4), 2837. https://doi.org/10.1111/j.1745-3992.2011.00220.x

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119(2), 254284. https://doi.org/10.1037/0033-2909.119.2.254

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Klute, M., Apthorp, H., Harlacher, J., & Reale, M. (2017). Formative assessment and elementary school student academic achievement: A review of the evidence (Report No. 2017-259). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Central. https://ies.ed.gov/ncee/edlabs/regions/central/pdf/REL_2017259.pdf

    • Search Google Scholar
    • Export Citation
  • Kouba, V. L., & Franklin, K. (1995). Multiplication and division: Sense making and meaning. Teaching Children Mathematics, 1(9), 574577. https://doi.org/10.5951/TCM.1.9.0574

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lobato, J., & Walters, C. D. (2017). A taxonomy of approaches to learning trajectories and progressions. In J. Cai (Ed.), Compendium for research in mathematics education (pp. 74101). National Council of Teachers of Mathematics.

    • Search Google Scholar
    • Export Citation
  • National Governors Association Center for Best Practices & Council of Chief State School Officers. (2010). Common core state standards for mathematics. http://www.corestandards.org

    • Search Google Scholar
    • Export Citation
  • National Research Council. (2001). Adding it up: Helping children learn mathematics. National Academies Press. https://doi.org/10.17226/9822

    • Search Google Scholar
    • Export Citation
  • National Research Council. (2007). Taking science to school: Learning and teaching science in grades K–8. National Academies Press. https://doi.org/10.17226/11625

    • Search Google Scholar
    • Export Citation
  • Nesher, P. (1988). Multiplicative school word problems: Theoretical approaches and empirical findings. In J. Hiebert & M. Behr (Eds.), Number concepts and operations in the middle grades (pp. 1940). Erlbaum.

    • Search Google Scholar
    • Export Citation
  • Ongoing Assessment Project. (2005). Exploratory study of impact on student work [Unpublished raw data]. Consortium for Policy Research in Education, University of Pennsylvania.

    • Search Google Scholar
    • Export Citation
  • Panorkou, N., & Kobrin, J. L. (2017). Enhancing teachers’ formative assessment practices through learning trajectory-based professional development. Mathematics Teacher Educator, 5(2), 178201. https://doi.org/10.5951/mathteaceduc.5.2.0178

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pepper, K. L., & Hunting, R. P. (1998). Preschoolers’ counting and sharing. Journal for Research in Mathematics Education, 29(2), 164183. https://doi.org/10.2307/749897

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rathbun, A., West, J., & Germino Hausken, E. (2004). From kindergarten through third grade: Children’s beginning school experiences (Report No. 2004-007). U.S. Department of Education, National Center for Education Statistics. https://nces.ed.gov/pubs2004/2004007.pdf

    • Search Google Scholar
    • Export Citation
  • Reardon, S. F., Kalogrides, D., & Shores, K. (2019). The geography of racial/ethnic test score gaps. American Journal of Sociology, 124(4), 11641221. https://doi.org/10.1086/700678

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Riggan, M., & Ebby, C. B. (2013). Enhancing formative assessment in elementary mathematics: The Ongoing Assessment Project [Working paper]. Consortium for Policy Research in Education, University of Pennsylvania.

    • Search Google Scholar
    • Export Citation
  • Robinson-Cimpian, J. P., Lubienski, S. T., Ganley, C. M., & Copur-Gencturk, Y. (2014). Teachers’ perceptions of students’ mathematics proficiency may exacerbate early gender gaps in achievement. Developmental Psychology, 50(4), 12621281. https://doi.org/10.1037/a0035073

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18(2), 119144. https://doi.org/10.1007/BF00117714

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Schoenfeld, A. H., Hulbert, E. T., Petit, M. M., Ebby, C. B., Cunningham, E. P., & Laird, R. E. (2017). A focus on multiplication and division: Bringing research to the classroom. Routledge. https://doi.org/10.4324/9781315163611

    • Search Google Scholar
    • Export Citation
  • Sherin, B., & Fuson, K. (2005). Multiplication strategies and the appropriation of computational resources. Journal for Research in Mathematics Education, 36(4), 347395.

    • Search Google Scholar
    • Export Citation
  • Shulman, L. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57(1), 123. https://doi.org/10.17763/haer.57.1.j463w79r56455411

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Silver, E. A., & Mills, V. L. (Eds.). (2018). A fresh look at formative assessment in mathematics teaching. National Council of Teachers of Mathematics.

    • Search Google Scholar
    • Export Citation
  • Steffe, L. P. (1992). Schemes of action and operation involving composite units. Learning and Individual Differences, 4(3), 259309. https://doi.org/10.1016/1041-6080(92)90005-Y

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Supovitz, J. (2012). Getting at student understanding—The key to teachers’ use of test data. Teachers College Record, 114(11), 129. https://www.tcrecord.org/content.asp?contentid=16804

    • Search Google Scholar
    • Export Citation
  • Supovitz, J., & Sirinides, P. (2018). The Linking Study: An experiment to strengthen teachers’ engagement with data on teaching and learning. American Journal of Education, 124(2),161189. https://doi.org/10.1086/695610

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Supovitz, J. A., Ebby, C. B., Remillard, J., & Nathenson, R. A. (2018). Experimental impacts of the Ongoing Assessment Project on teachers and students. Consortium for Policy Research in Education. https://repository.upenn.edu/cpre_researchreports/107

    • Search Google Scholar
    • Export Citation
  • Sztajn, P., Confrey, J., Wilson, P. H., & Edgington, C. (2012). Learning trajectory based instruction: Toward a theory of teaching. Educational Researcher, 41(5), 147156. https://doi.org/10.3102/0013189X12442801

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tzur, R. (2007). Fine grain assessment of students’ mathematical understanding: Participatory and anticipatory stages in learning a new mathematical conception. Educational Studies in Mathematics, 66(3), 273291. https://doi.org/10.1007/s10649-007-9082-4

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ulrich, C. (2015). Stages in constructing and coordinating units additively and multiplicatively (Part 1). For the Learning of Mathematics, 35(3), 27.

    • Search Google Scholar
    • Export Citation
  • Ulrich, C. (2016). Stages in constructing and coordinating units additively and multiplicatively (Part 2). For the Learning of Mathematics, 36(1), 3439.

    • Search Google Scholar
    • Export Citation
  • Wilson, P. H. (2009). Teachers’ uses of a learning trajectory for equipartitioning [Doctoral dissertation, North Carolina State University]. University Libraries. https://repository.lib.ncsu.edu/handle/1840.16/2994

    • Search Google Scholar
    • Export Citation
  • Wilson, P. H., Mojica, G. F., & Confrey, J. (2013). Learning trajectories in teacher education: Supporting teachers’ understandings of students’ mathematical thinking. The Journal of Mathematical Behavior, 32(2), 103121. https://doi.org/10.1016/j.jmathb.2012.12.003

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wilson, P. H., Sztajn, P., Edgington, C., & Myers, M. (2015). Teachers’ uses of a learning trajectory in student-centered instructional practices. Journal of Teacher Education, 66(3), 227244. https://doi.org/10.1177/0022487115574104

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wilson, P. H., Sztajn, P., Edgington, C., Webb, J., & Myers, M. (2017). Changes in teachers’ discourse about students in a professional development on learning trajectories. American Educational Research Journal, 54(3), 568604. https://doi.org/10.3102/0002831217693801

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Winkelmann, H., van den Heuvel-Panhuizen, M., & Robitzsch, A. (2008). Gender differences in the mathematics achievements of German primary school students: Results from a German large-scale study. ZDM, 40(4), 601616. https://doi.org/10.1007/s11858-008-0124-x

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zhu, Z. (2007). Gender differences in mathematical problem solving patterns: A review of literature. International Education Journal, 8(2), 187203. https://files.eric.ed.gov/fulltext/EJ834219.pdf

    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 35 35 0
Full Text Views 22 22 0
PDF Downloads 23 23 0
EPUB Downloads 0 0 0