Understanding Math Teachers (and Students) Through Prototype Testing
By Wendy McMahon
%20Through%20Prototype%20Testing)
In October 2019, grade-three teacher Noralee Yarra participated in a round of prototype testing for Pearson Mathology, a process she appreciated and learned from.
“I don’t know of any other education companies that do this,” explains Yarra, who teaches in the Dufferin-Peel Catholic District School Board. “How valuable for teachers and school boards to have this validity from teachers who’ve tested beforehand—before they launch the program.”
“When the resources arrived, it was like Christmas in a box. There were so many resources. And they’re all designed with a seven-or-eight-year-old in mind; there are pops of color, lots of animals and game-based learning”
Noralee Yarra
Why Prototype Testing?
Christine Chea, Director of Research and Innovation for Pearson Canada, says the prototyping process is part of the design thinking approach that Pearson uses to create the digital and print components of its K-8 Math resource, Mathology.
“Design thinking starts with gaining a deep understanding of the end-user,” explains Chea. “It’s about trying to develop empathy by really understanding their context. It helps us understand the problems teachers may be facing, the different jobs they may be doing as part of their math teaching, and the challenges they may be experiencing.”
Using that knowledge, the team first determines which problems they can help address. Then they consider solutions that help address those problems or help educators have a better experience in their math lesson planning and teaching. Then they create prototypes of those solutions.
“Those prototypes then go to educators so they can test them and provide feedback which we then use to make improvements and optimizations. And the process repeats until we validate that we are actually supporting them with the challenges they face,” explains Chea.
The prototypes Yarra tested included a variety of new Mathology hands-on resources for grade three classrooms along with new additions to Mathology.ca (a digital planning platform for teachers).
Yarra was one of the 33 grade three teachers Pearson worked with from across the country. “The group of teachers represented a range of experiences and comfort-levels in teaching mathematics,” explains Miranda Kus, a Math Consultant for Pearson Canada who worked on the prototype testing project. As Kus explains, Yarra and the other teachers each received a package containing all the prototype materials. They also took part in training, video calls, and webinars, introducing them to the resource before they began testing.
Then, Yarra and the other teachers used both the digital and print prototypes as their primary math resource for three weeks.
Throughout the process, teachers provided feedback to the Pearson team through weekly logs, detailed surveys, and in some cases, in-depth interviews. Pearson representatives also visited various classrooms during the process to observe and gather feedback from teachers.
Through each of these feedback vehicles, Chea and Kus say the team gathered vital insights. First, they learned how well the resources supported teachers in addressing specific pain points when teaching math. Teachers also shared how satisfied they were with the materials, how they actually used the materials, and their suggested changes or improvements.
The team also reviewed analytics from the mathology.ca digital platform to better understand how the teachers used the platform and if they were using key features.
“The whole process gave us a very complete picture of the overall experience for teachers and students, and a good view into how the prototypes were used and incorporated into math lessons. It provided great insights that are helping us optimize the solutions so that we can increase our likelihood of having a positive impact among teachers and students. Product efficacy is at the heart of what we do,” explains Chea.
"Product efficacy is at the heart of what we do"
Christine Chea
Mathology Prototype Testing: A Teacher’s Perspective
For Yarra, the process was positive from start to finish for her and her students. Having never participated in prototype testing before she says she was intimidated, but once she began, the process went smoothly. And she felt confident sharing all of her feedback, including constructive feedback such as when she felt the language in some activities was too advanced for her students.
“When the resources arrived, it was like Christmas in a box. There were so many resources. And they’re all designed with a seven-or-eight-year-old in mind; there are pops of color, lots of animals and game-based learning,” explains Yarra.
In her feedback, Yarra shared that she felt there was a good balance between activities that used technology and those that were technology-free. She also related that her students responded well to lessons that helped make learning more authentic through real-life connections. “For example, they loved the Little Books and thought they were great. They loved how the little books incorporated adventures and facts about animals. It was relatable to them at their age.”
Yarra says she was impressed with how well-thought-out the Mathology resources and activities she tested were. “They seemed to really think about how they can respond to what students need. And through avenues like game-based learning, the students absolutely reacted well to what I was teaching.”
For Yarra, the process was well worth the effort, and she’d gladly participate in prototype testing again. “My grade threes loved this whole experience, they were so engaged. And I liked it because they were learning math in a fun way and were able to retain more information and make connections to their real life.”