More Students Earn A’s After Playing Triseum’s Calculus Game Variant: Limits

Shelley Kaufeld May 2, 2019

Research Links Gameplay to Higher Scores and Stronger Learning Outcomes at Texas A&M University

College Station, TX – May 2, 2019 – A calculus game developed by game-based learning company Triseum and Texas A&M is moving the needle on student achievement. Earlier this year, a study released by the university showed that students who played Variant: Limits^®scored higher on their exams than students who did not play the calculus game. Newly released data concludes that the percentage of A’s as final grades among those students who played the calculus game was 10.8 percent higher compared to students who did not play the game. Additionally, the DFQW rate (those students who received a D, F or withdrew) for those students who played the calculus game was 3.1 percent lower than those who did not play.

“Game-based learning increases motivation and engagement, and if students are more motivated and engaged, they are more likely to stick with it and perform better,” said André Thomas, CEO of Triseum and director of the LIVE Lab at Texas A&M. “We know students have fun playing Variant: Limits and we know they connect more deeply with the content, but now we have empirical evidence that directly ties gameplay to stronger learning outcomes.”

Calculus is foundational to STEM degrees. As STEM graduates continue to be in high demand, and as shortages are predicted, higher education needs to put proven practices in place to help students succeed with this notoriously challenging subject. According to the Mathematical Association of America, only about 65 percent of students succeed in calculus, yet the math department at Texas A&M has been working hard to implement innovative tactics that have helped bring its student success rate to well above 90 percent.

Triseum worked closely with Texas A&M’s math department on the game’s proof of concept. “The math department gets it. They were instrumental in helping create the vision for what Variant: Limits could achieve and they are committed to making sure students have the tools and technologies necessary to help them succeed,” said Thomas.

In a recent on-air interview, Thomas talked about the importance of calculus in STEM curriculum and the need for more STEM graduates to fulfill millions of well paying jobs. “Limits are foundational for calculus, and understanding limits contributes to deeper learning that students can carry with them in other calculus and STEM courses,” said Thomas.

Game-based learning is gaining traction, and for good reason. Where more traditional learning resources like tutoring centers leave very little to the imagination, they also have high fees and are difficult to scale across multiple campuses and programs. Games are the opposite. Triseum’s academic games excite, leveraging sophisticated entertainment design principles that engage students, so much so that students often forget they are learning. Students can visualize the challenges and apply their knowledge, not to mention the company’s games offer an affordable and scalable solution for big and small student populations alike.

Texas A&M’s “Variant: Limits” study involved more than 2,000 students in Fall 2018. Those in the experimental group played the game on their own outside of class and scored higher on their first exams than students in the control group who did not play the game. Data did not take into account student motivation between the experimental group and the control group.

Data Analytics, Academic Gaming and the Student Experience

bhhobson March 8, 2019

By André Thomas, CEO of Triseum

It’s rare to look at a higher education coverage these days and not find references to the importance of data analytics. Across the student lifecycle from marketing and enrollment, to the learning experience, to alumni relations, data seems to be behind it all.

According to a blog post on ACE, “over the last decade, data analytics has evolved from a buzzword to a multibillion-dollar business, and it has begun to permeate higher education.” What’s more, eCampus News talks about the data-enabled institution among the issues that will shape 2019 for higher-ed IT leaders, and the 2018 NMC Horizon Report published by EDUCAUSE named analytics technologies an important development in higher education.

This same Horizon Report goes on to say that a “growing focus on measuring learning is an accelerating trend in educational settings.” Faculty want to know when and how their students are engaging with the course and its content, not to mention how this engagement is impacting outcomes. And so here we are, back to data.

As an educator and game designer, I can’t help but think about the ways that data analytics in academic video games, such as Variant: Limits or ARTé: Mecenas, can be used to measure learning, even more so than a book. Like traditional textbooks, these video games are another medium to help students learn a subject. But unlike books, immersive video games have the power to analyze student progress and mastery along the way, giving faculty important insights.

Intelligent game analytics enable faculty to monitor student engagement and advancement made toward learning objectives. Teachers can see where students are thriving, if they might get hung up on certain concepts, whether or not the majority of students are gravitating toward a particular incorrect answer, and how students are navigating through the content. Teachers can also use this data to intervene with individual students if they see patterns that require additional attention or clarification.

Data analytics in higher education isn’t just a thing, it’s a big thing. It’s becoming the basis for which institutions connect with and support students, and it’s a critical piece of the learning experience.

Games in Every Classroom

André Thomas December 12, 2018

Books have long been an accepted part of every classroom. I have visited schools all around the world and see the same picture everywhere. Books are a common staple in the educational experience for students and teachers alike.

I don’t just see textbooks in classrooms, but rather all different types of books. For example Tom Sawyer, a work of fiction, has now become a standard in many classrooms in Europe and the U.S. From art books and reference books, to law reviews, business case studies, policy books and of course biographies, to history books and non-fiction books, they have become trusted resources.

When I went to school, our teachers would augment books with television and movies as a tool to drive engagement and make course content more relevant and exciting to us. At the same time they were trying to help us as students achieve a higher level of content mastery.

Teachers all over the world have but one goal–to empower students to succeed. Everywhere I look I see teachers trying to enhance the classroom experience for their students so that they can become more engaged and better retain the information they are learning. I regularly hear comments made about what teachers should do or what tools they should use, yet most of these recommendations come from people who aren’t teachers and have never been in a classroom.

As teachers can attest, you can’t just try something and hope it works. Imagine the conversation with the parents: “Sorry your child flunked math this year, we tried this new xyz… and it just didn’t work for our students. We will try something different next year and see if it works better.”

Instead of trying to prescribe something to teachers and tell them what to do, we should work with them to understand what they are already doing, how they are doing it, and what they really need. For example, teachers don’t just have one book for any given subject. There are different books for the same curriculum, giving teachers a choice in how they approach the subject in a way that they think will be most impactful for students.

Games for the classroom should be no different; we should make a variety of games available to teachers and students so they have a choice. Where once we relied on TV and movies to offer another avenue of content delivery, today games can meet that need.

What would a world look like where every classroom and every teacher from pre-K to university would have games available to use as part of the teaching and learning process? In my opinion that would mark a monumental step towards utilizing a proven medium to enhance and improve the education experience and outcomes for teachers and students.

However having said that, mass acceptance of games requires several considerations. First, there is testing. While I have never seen a study that demonstrates the efficacy of the book, testing game scenarios with teachers is important. When testing and validating games, we need to keep in mind that, just like books, games can be used in a variety of ways. Testing every possible scenario might be difficult, but working directly with teachers who are using games can help other educators gauge effectiveness of the games and spur ideas to present content in a different and engaging way.

Another consideration is support and maintenance. Something as simple as an update to an operating system can cause a game to no longer function. Fixing the problem shouldn’t be left to teachers or students, not to mention it likely requires funding. Who will help a student or teacher on a Sunday at 2am when they are trying to use the game and have a simple support question? What about the analytics data, bandwidth and/or cloud storage so students can play anywhere? What about supporting materials like lesson plans, textbook mapping and classroom exercises? How about efficacy studies?

Lastly, funding is critical. A well-designed game-based curriculum requires someone to oversee it, as well as additional technology and that requires a mechanism to fund it. Too often I hear that learning games should be free, and while I find great merit in that philosophy, I have yet to see an effective system that allows those who develop, maintain and support learning games to be compensated for their work and expenses by giving their games away. I believe they deserve to be fairly compensated for their work.

While great strides are being made on the game-based learning front, it will take collaboration on the part of teachers, parents, designers, developers, publishers and policy makers to come up with a system that addresses development, testing, support, maintenance and funding for new innovative education technology. Only then can we hope to put games in every classroom.

Video Games Teach Us More than Just their Content

André Thomas November 28, 2018

“A book or movie can show us what it is like to be in a character’s shoes, but it is the video game that can put us into those shoes.” – Robert B. Marks, Ph.D., Whittier College

Generally speaking, active learning fosters far greater curriculum engagement than say listening to a ‘sage on the stage’ or memorizing stats from a textbook. Fortunately, modern technologies, specifically video games, are making this kind of immersive, hands on learning a greater reality. Game-based learning is enabling students to walk in the shoes of both real life and imaginary characters, empowering them to interact with content in an entirely new way.

But it doesn’t end there. Because video games inspire active learning, students are able to gain skills beyond the knowledge of the content itself.

Video games teach the art of persistence. Students are encouraged to try again until they succeed. There is less of a stigma around failure as it becomes fun to practice and attempt new strategies to see what works. For example, in our calculus game, Variant: Limits, students can visually see how manipulating equations impacts graphs, charts and their course of action, and therefore they are encouraged to use a trial and error approach.

Video games can help shape students’ decision making and critical thinking skills. It is exploration and experimentation at its best. Students begin to think outside the box, act on their curiosities, and look at different ways to get the correct answer. Once they start to understand what works, they apply that knowledge to help the story unfold.

Video games inspire social interaction and are a great tool to promote teamwork. According to instructor Anita Streich, “ARTé: Mecenas taught my students the importance of collaboration, a great example of active learning. Students were willingly working in pairs and in groups. They were having a dialogue with each other not only about the game’s content, but also its strategy.”

Being immersed in a video game, walking in the character’s shoes, and having a stake in the character’s outcome keeps students invested. Whether they know it or not, they are not just learning the content, but they are acquiring accompanying life skills–the kinds of skills that will serve them over the long term over every aspect of their life. I ask, could a talking head or textbook do that?

Game-Based Learning: What do Students and Faculty Have to Say?

André Thomas September 11, 2018

For those of us who are at Triseum and the LIVE Lab at Texas A&M, we can attest to the countless hours of research, collaboration and testing that go into our game development processes. And while all of the results and interactions during development lead us to believe we are producing impactful games, the ultimate testament to the effectiveness of our games comes from those on the front lines–the instructors and students who are interacting with our games as part of the teaching and learning experience. We recently had a chance to sit down with some of our users and their feedback was inspiring. The opportunity for game-based learning and faculty is very promising and this is what faculty have to say.

ARTé: Mecenas encourages art history exploration and application:

“I think that the direct benefit that students received was learning without realizing they were learning.”
-Kelly Donahue-Wallace, Art History Professor, University of North Texas

https://www.youtube.com/watch?v=0aVYHV2SNHU

“We started off introducing [the game] in classes in lieu of quizzes and also other assignments, and [students] really appreciated having that at the end of each week as opposed to other assignments they found to be less engaging and exciting. They really enjoyed it and seemed to apply the material better in class.”
-James Hutson, Associate Professor Art & Design, Lindenwood University

https://www.youtube.com/watch?v=-aE3gMQCGyI

“It stuck in a different way and not in a ‘regurgitate on a test’ way.”
-Cynthia Meersman, Art History Student, Texas A&M University

https://www.youtube.com/watch?v=bdArHDLf2fE

Variant: Limits gives students a fresh new perspective on calculus:

“People in general are willing to bang their heads in a game where they generally roll their eyes and groan in a classroom. It’s stretching the students and challenging them to realize where the concepts can show up.”
-Rob Eby, Professor of Mathematics, Blinn College

https://www.youtube.com/watch?v=u4rcvZw19TU

“Homework doesn’t give you a second chance, but in the game, if you put in one wrong [answer] and change it, you can actually see the change in the game.”
-Alexander Rosen, Chemistry Student, Texas A&M University

https://www.youtube.com/watch?v=Dc8cuv0RVTk

“I’ve taken courses for limits and calculus and pre-calculus math, and I learned more from this game than I ever did in those. I saw in the lectures how they did it and then doing it in the video game really immersed me in it and allowed me to learn and think about it in lots of different ways.”
-Ethan Ritchie, Physics Student, Texas A&M University

https://www.youtube.com/watch?v=PISWU0ud_yU

“Because I was playing a game it helped me retain the information. I wasn’t just sitting in a lecture, I was actually engaging and looking at the game and finding out the information for myself. It definitely stuck with me more, definitely – I remember playing a game, I don’t remember the lecture of limits.”
– Jack Norman, Software Engineering Student, University of Texas

https://www.youtube.com/watch?v=nWP0N_Jc-z8

Hearing first-hand how our games are being used in the classroom and how they are helping boost engagement validates game-based learning and the incredible impact it is having on the education experience. Bring on the game-based learning revolution!

According to Teachers, Art History Game – ARTé: Mecenas™ Boosts Decision Making Skills, Creativity and Collaboration

André Thomas July 12, 2018

The second webinar in our game-based learning validation study series recently aired, giving teachers a platform to share their insights from implementing our art history game, ARTé: Mecenas™, into their classrooms. The study followed classes in Norway, Poland, Portugal, Italy and Greece and was designed to empower participating teachers to consider new trends in learning models and investigate innovative technologies, all without losing site of learning outcomes.

The webinar was hosted by European Schoolnet and the replay is available here.

More than 470 students in 19 classes played ARTé: Mecenas over the 2017-2018 academic year. Collectively, teacher feedback revealed the interest and enjoyment students found in art history as a result of playing the game. The vast majority of participating teachers agreed that game had a positive impact on student motivation and engagement and one teacher even noted that the game was used an accelerator to learn the content.

During the webinar, instructors were asked the following questions and provided some keen conclusions:

What competencies did you look to foster in your students?

Students gained research and problem-solving skills, critical and reflective thinking skills, and communicative skills.
One of the most surprising take-aways was the creativity ARTé: Mecenas inspired in students, whereby they not only created instructional videos, but also their own imaginative scenarios, artworks and storylines.

What were students’ reactions to playing ARTé: Mecenas?

Students were surprised at first as games are not a typical medium in their classrooms; they quickly grew to appreciate the teamwork aspect.
Students loved that they could combine what they learned with each other, which made the learning experience more motivating and stimulating.

What was the impact of the game on student knowledge acquisition?

ARTé: Mecenas not only improved student decision making, it proved to be a valid approach to global knowledge.
Students learned through teamwork, were active in discussions, and supported each other to level-up in the game.
Students shared both knowledge and strategic thinking, as well as engaged in self-assessment.

What changed in the classroom after the implementation of ARTé: Mecenas?

Students realized they can learn, compete and have fun simultaneously.
Students gained an appreciation for the complexities of life, strategic thinking in a business setting, and the importance of collaboration.
Students engaged in a dialog about more than just the classroom content, but also the strategy behind the game.

Prior to our ARTé: Mecenas webinar, teachers also participated in a webinar in which they shared their insights from implementing our calculus game, Variant: Limits, into their classroom. The replay of that webinar is available here.

Both the executive summary and the complete report can be downloaded here. The study was conducted in partnership with Triseum and European Schoolnet and the results were evaluated by the University of Würzberg.

Four Studies That Make the Case for Game-Based Learning

André Thomas June 18, 2018

The case for integrating game-based learning into school curriculum is not new. Over the last 30 years, there has been an immense amount of game based learning research demonstrating the benefits of educational games. Here are a few studies that have uncovered a variety of advantages associated with using games to improve student performance and achievement.

Digital Games, Design and Learning: A Systematic Review and Meta-Analysis In 2014, SRI Education, an organization that analyzes different approaches to education, conducted a meta-analysis of studies on the use of educational computer games between 2000-2012. Although studies on educational video games began as early as the 1980’s, the SRI researchers focused on the first decade of the 21st century due to the tremendous advances in gaming technology that occurred.

The SRI meta-analysis found that “digital games significantly enhanced student learning.” The game based learning research report also highlighted evidence that the more sessions that students play educational games, the greater the advantage they gain over their peers who are not provided games as part of their course.

The analysis also investigated the hypothesis that games encouraging multi-player collaboration would be more effective than single-player games. The analysis did not find strong evidence that collaborative games were more effective than single-player games. It found, however, that collaborative competitive games and single-player games that lacked a competitive element were both more effective than single-player competitive games.

While other game based learning research studies have found that “non-game instruction” would enhance the educational effects of the gameplay, the meta-analysis did not find evidence that additional instruction made a significant difference.

Clark, D., Tanner-Smith, E., & Killingsworth, S. (2014). Digital games, design and learning: A systematic review and meta-analysis [Executive Summary]. Menlo Park, CA: SRI International. Retrieved from https://www.sri.com/sites/default/files/publications/digital-games-design -and-learning-executive_summary.pdf.

The National Survey of Digital Game Use Among Teachers A survey of 513 K-8 teachers conducted in 2014, showed that a majority (55 percent) use games at least once a week to teach their curriculum. When asked what the teachers believed was the most valuable effect of games, 55 percent cited that games have ability to motivate struggling, low-performing or special education students. In addition, nearly half (47 percent) of teachers reported that low-performing students benefited the most from using games while only 15 percent reported that high-performing students benefited from the games. Overall, 30% of the teachers said all students benefit equally.

The survey not only showed that teachers are already seizing opportunities to introduce game-based learning to students, but that they wish more opportunities existed. Teachers agree (45 percent) that even commercial games, that are not specifically designed with educational outcomes in mind, can serve an educational purpose. However, 80 percent of the surveyed teachers would like more games that align with their curriculum standards. A good example is Variant: Limits which aligns with standard pre-calculus and calculus curriculum and National standards.

Joan Ganz Cooney Center (2013). Teachers surveyed on using digital games in class: A games and learning research report. New York, NY.

Substantial Integration of Typical Educational Games into Extended Curricula: Researchers at Vanderbilt conducted a study on more than 1,000 students in 13 schools across in seven different U.S. states (Alabama, Kentucky, Maryland, Missouri, New Jersey, Rhode Island and Washington D.C.) that produced compelling evidence that games can be helpful in teaching history. As part of the study, each teacher was asked to teach at least one classroom about Jacksonian democracy without the assistance of games and one classroom with the help of 55 different educational video games.

Post-tests featured four multi-choice questions and seven short essay questions about the content covered in the course. The student responses on multi-choice questions were evaluated based on the percentage correct, while the essays were assessed based on both a 0-10 scale of “evidentiary depth and contextualization” about the Jackson presidency as well as the length of the response. Separately, students took an engagement survey that examined both their interest in the subject and their confidence in their understanding of the content.

The greatest observed difference between students in the control group and those who learned the content through games was in reported interest in the subject. Those who played games exhibited much greater enthusiasm for the curriculum.

At first, the results of the post-test did not appear to show a significant difference between those who played games and those who didn’t. However, the study authors noted that the average performance of the game-playing students was brought down dramatically by one outlier classroom that performed much worse than every other classroom (in both conditions). When that classroom was excluded from the evaluation, the advantage gained by game-playing students over the control group was statistically significant.

Clark D.B., Tanner-Smith E., Hostetler A., Fradkin A., & Polikov V. (2017). Substantial integration of typical educational games into extended curricula. Journal of the Learning Sciences. 27(2), 265-318. https://doi.org./10.1080/10508406.2017.1333431

The Power of Play: The Effects of Portal 2 and Lumosity on Cognitive and Noncognitive Skills: In this study, researchers at Florida State University found evidence that a popular video game (Portal 2) did more to bolster certain cognitive skills than a computer program (Luminosity) designed specifically for cognitive training. The study highlights the fact that, like any other type of learning strategy, there is a right way and a wrong way to do game-based learning.

The study featured 77 college students, all of whom were randomly assigned to play one of the two games for eight hours. All of the participants completed pretest questions that assessed problem-solving, spatial skills and persistence. After playing the game for eight hours, they took post-test evaluations.

The study authors hypothesized that players of both games would show improved performance on two distinct problem-solving measures: rule application and cognitive flexibility. They also hypothesized those who played Portal 2 would improve “all three facets of spatial skill: “figural, vista and environmental,” while those who played Luminosity would “only improve their figural and vista skills.”

The study actually found that neither game produced gains in problem-solving skills. However, it did find that those who played Portal 2 showed measurable improvement in two of the three spatial tests (mental rotation test and a virtual spatial navigation assessment), while those who played Luminosity did not display any significant improvement.

Shute, V. J., Ventura, M., Ke F., (2014). The power of play: The effects of Portal 2 and Lumosity on cognitive and noncognitive skills. Computers & Education. 80, 58-67. https://doi.org/10.1016/j.compedu.2014.08.013

Game based learning research for Variant is available here. Additionally a year long game based learning research project that took place in 5 countries in the European Union yielded great results and can be found here.

Triseum’s ARTé: Mecenas™ Game Boosts Students’ Knowledge Gain by Nearly 25 Percent

André Thomas March 7, 2017

March 7, 2017 – Bryan, TX – Triseum’s game based learning development practices and rigorous design methods are creating a measurable impact on student outcomes. Newly released results from Triseum’s Fall 2016 research study aimed at measuring the learning efficacy of its ARTé: Mecenas™ game indicate that, after approximately two hours of game play, art history students who participated in the experimental group had a knowledge gain of 24.7 percent (p < .0001) from pre-test to post-test. “Too often in art history we see students memorizing and regurgitating facts, but with ARTé: Mecenas, they are empowered to be a more active and engaged participant in the learning process,” said André Thomas, CEO of Triseum. “In developing this Renaissance learning game, we relied heavily on faculty, subject matter experts and instructional designers, in tandem with research, to not only understand how to teach the concepts, but also how to test them so we can make sure students are comprehending and retaining the material.” “As faculty, we have limited classroom time to thoroughly convey the complexities that surround the works of art and the broader societal norms of the time period, yet by playing ARTé: Mecenas, students have the opportunity to understand the ideas and their relevance more deeply,” said Carey Rote, Professor of Art History at Texas A&M University-Corpus Christi. “ARTé: Mecenas gets students excited about the subject matter, enabling them to learn in a fun and competitive manner and truly experience the concepts that might be harder to grasp in a lecture format. My students’ enthusiasm for the game has been tremendous and they have achieved greater knowledge.” ARTé: Mecenas is imaginative and sophisticated, yet also maintains academic rigor to enhance art history courses. The game transports students into the Renaissance era where they take on the role of a member of the merchant/banking Medici family, one of the most influential families of the time. True to the life of the Medici, students must balance relationships with powerful city-states, merchant factions and the Catholic Church to build and maintain a financial empire. “We have incorporated ARTé: Mecenas into our curriculum again for the Spring 2017 term and the feedback we are getting from students and faculty is inspiring,” said Kelly M. Quintanilla, Ph.D., Interim President & CEO at Texas A&M University-Corpus Christi. “By allowing students to explore and develop innate insights through the game, they are more connected to and familiar with the subject.” Triseum is committed to pushing the boundaries of educational games, making them as sophisticated and imaginative as their entertainment counterparts, yet preserving learning efficacy supported by data. ARTé: Mecenas adheres to rigorous scientific standards and the latest research. Purposeful sampling was used in Triseum’s recent study, which is detailed on its website. Participants were recruited from a pool of students in college-level art history survey courses (the target audience for the game). In total, 184 participants consented to join the study, and 173 participants completed the study and earned extra credit in their course for participating. The nearly 25 percent gain is practically significant (Cohen’s d = 1.14, large effect) in an educational setting for a single unit of complementary instruction. Additionally, in Spring 2017, Triseum is conducting additional research studies within the state of Texas. Triseum is also seeking independent researchers to replicate the study. If interested in conducting research on the Renaissance learning game: ARTé: Mecenas, please contact learning@triseum.com.