Evaluating the Effectiveness of Games and Simulations in Online Teaching

Adult learner reviewing an online simulation activity

Integrating games and simulations into online learning environments can be an effective way to engage learners, promote critical thinking, and provide an authentic learning environment (Bertiz, 2020). However, not all games or simulations are equally effective. To ensure that these tools genuinely support learning outcomes, instructors need a clear framework for evaluation (Petri & Gresse von Wangenheim, 2016; Ward, 2015). Below is a checklist of criteria for evaluating the effectiveness of games and simulations, followed by a reflection on how one simulation I have previously used aligns with these standards.

Checklist for Evaluating Games and Simulations

Criteria	Effectiveness	Met	Not Met	Feedback
Alignment with Learning Objectives	The game or simulation must directly support course outcomes. If objectives are not aligned, learners might not glean the required knowledge from the content.
Cognitive Engagement	Learners should be challenged to think critically, solve problems, and/or apply knowledge. Games that test recall without deeper engagement are less effective.
Authenticity and Real-World Relevance	Effective simulations mirror real-world contexts, helping learners transfer skills beyond the classroom. Authenticity increases motivation and perceived value.
Accessibility and Inclusivity	The tool should be usable by learners with diverse abilities and technological access. Accessibility ensures equity and supports the principles of Universal Design for Learning (UDL).
Feedback and Assessment Integration	Games should provide timely, meaningful feedback that helps learners understand mistakes and improve.
Ease of Use and Technical Reliability	If learners have difficulties with navigation or encounter frequent glitches, cognitive load shifts from learning. Usability is essential for sustained engagement.
Collaboration and Social Interaction	Opportunities for teamwork, collaboration, or peer dialogue enhance learning by placing knowledge in social contexts. Simulations that support collaboration often lead to deeper understanding.
Motivation and Engagement	The game should sustain learner interest through sound preparation and clear goals, rewards, or narrative. Engagement is a foundational for persistence and learning.
Scalability and Adaptability	Effective tools can be adapted for different class sizes, learner levels, or instructional contexts. Flexibility ensures broader applicability.
Evidence of Effectiveness	The game or simulation has been studied or piloted with positive learning outcomes. Research-based tools provide greater confidence in their impact.

Simulation Checklist Test

A well-constructed simulation I have previously used is called Budget Challenge (ProperLiving LLC, n.d.), which is a financial literacy and capability simulation focused on young adult learners. With this simulation, learners manage a virtual household budget over a number of weeks. The simulation effectively meets the criteria outlined in the checklist in the following ways:

• Alignment with Learning Objectives. The Budget Challenge simulation aligns well with financial literacy outcomes where learners engage in practical exercises, including budgeting, bill management, and making financial decisions. The activities help learners develop essential skills for managing their personal finances effectively.

• Cognitive Engagement: Learners must exhibit higher cognitive engagement for the simulation. Skills include trade-off analysis, anticipating consequences of financial decisions, and using math to manage budgets. This level of learner engagement helps both deepen understanding of financial concepts and improve problem-solving abilities.

• Authenticity and Real-World Relevance: Simulation tasks represent real-world financial responsibilities, resulting in authenticity. This helps learners transfer skills from the classroom, which increases motivation and perceived value.

• Accessibility and Inclusivity: An area for improvement is with accessibility and the complexity of the web-based interface. Accessibility features could be improved to support equity and Universal Design for Learning (UDL) principles.

• Feedback and Assessment Integration: The simulation's feedback mechanism is immediate, as learners receive substantial consequences for late payments or poor decisions, allowing for improvement once a mistake is made.

• Ease of Use and Technical Reliability: The simulation’s initial learning curve is steep and requires mastery to improve usability. If learners have difficulties with navigation or glitches their cognitive load will focus away from learning.

• Collaboration and Social Interaction: The focus for the simulation is primarily for individual learners, however reflection and discussion could facilitate collaboration. The simulation could be adapted to foster teamwork and cooperation, thereby enhancing the learning environment from a social-context perspective.

• Motivation and Engagement: There are elements of investment and motivation created by overcoming the potential of financial pitfalls. The simulation is able to maintain learner interest through sound preparation, clear learner goals, rewards, and narrative.

• Scalability and Adaptability: Budget Challenge operates well across differing class sizes, learner levels, and instructional contexts, however instructor monitoring is intensive.

• Evidence of Effectiveness: The tool is supported by research in financial education contexts, though more longitudinal studies would be useful.

  
  

For the most part, the simulation passes my checklist, with strengths in authenticity, engagement, and alignment. Areas for improvement include accessibility and collaborative opportunities. Overall, the Budget Challenge is an effective tool for online teaching when paired with structured reflection activities.

Games and simulations can transform online learning by making abstract concepts concrete and engaging (Shaiakhmetov et al., 2024). However, effectiveness depends on careful evaluation. My checklist emphasizes alignment, engagement, authenticity, accessibility, feedback, usability, collaboration, motivation, adaptability, and evidence. By applying these criteria, educators can make informed choices that maximize learning outcomes while minimizing distractions.

References

Petri, G., & Gresse von Wangenheim, C. (2016). How to evaluate educational games: A systematic literature review. Journal of Universal Computer Science, 22(7), 992–1021.

Bertiz, R. (2020). Evaluation in simulation-based education. Nurse Educators’ Guide to Simulation-Based Education.

ProperLiving, LLC. (n.d.). Budget Challenge® – Financial literacy and capability + investing

Shaiakhmetov, D., ReshidMekuria, R., Gimaletdinova, G., Gulbarga, M. I., Orozakhunov, Z., & Khan, M. T. (2024). Criteria for assessing the quality of educational games and their impact on student learning outcomes. ACM CompSysTech Conference.

Ward, T. (2015). Evaluating instructional games and simulations. Ward Archives.