Scientists Discover Real Job Applicants Respond Better to Gamified Hiring Tests
Imagine taking a job assessment that feels more like a game than an exam – would that change how you feel about the hiring process? And if the result could actually impact your career, would you see the test differently? A recent study explores how gamified hiring tests are experienced by real applicants versus volunteers. The results reveal that when the stakes are high, people see these assessments in a more positive light.
Note: This article is intended for general information and educational purposes. It summarizes scientific research in accessible language for a broad audience and is not an official scientific press release.
Gamification – the integration of game – like features such as storylines, digital guides, and playful tasks into serious activities – is quickly becoming part of the modern hiring process. Today’s job seekers might find themselves not only answering questions, but also exploring virtual workplaces, solving challenges within storylines, or receiving encouragement from digital colleagues. These innovations promise a more engaging application experience. But do they truly change how applicants feel about assessments? And does that effect depend on whether the applicant’s future actually hinges on the result?
Scientists Marie L. Ohlms of Albert Ludwig University of Freiburg (Germany) and Klaus G. Melchers of University of Ulm (Germany) published a study in the Journal of Intelligence (March 2025) to address these questions. They set out to discover: Do real job applicants respond to gamified cognitive ability tests differently than research volunteers, and does the testing context change the entire experience? Their findings offer a new perspective on how context shapes perceptions of hiring assessments.
What Did the Researchers Set Out to Discover?
The central aim of the study was to find out if it makes a difference when a person takes a gamified hiring test as a genuine candidate, compared to someone participating for research purposes.
To answer this, the scientists designed a direct comparison between two groups:
- The first was a “high-stakes” group: real candidates who had applied for apprenticeships or dual study programs at a well-known German insurance company and were taking the test as part of the actual selection process.
- The second was a “low-stakes” group: students and young adults who volunteered to take the same test, with only a small prize available for top performance and no consequences for their careers.
After completing the test, both groups filled out a detailed questionnaire measuring their reactions to the experience. The researchers wanted to know: Did people see the process as fair? Did they feel motivated and engaged? Did they enjoy the assessment? And, crucially, did their impressions of the hiring organization change depending on whether the test mattered for their future?
How the Study Was Designed and Conducted
Who participated in the research?
- High-stakes group: 104 real job applicants (ages 15–37, average 19), all of whom completed the test as part of an authentic hiring process at a large German insurance company.
- Low-stakes group: 106 student volunteers (ages 18–59, average 22), recruited through schools and universities. For these participants, the test was simply a research activity.
What did the gamified test involve? The assessment was no ordinary quiz. It invited participants into a storyline where they helped fictional colleagues with work tasks, guided by digital avatars, while also learning about the company in interactive ways. The test was specifically designed for the insurance company’s recruitment process and lasted around 55 minutes.
How were reactions measured? After completing the test, all participants rated their experience on standardized scales, evaluating:
- Procedural fairness: Did the process feel just and unbiased?
- Job-relatedness: Did the test seem relevant to the role?
- Opportunity to perform: Did they feel able to show their skills?
- Test motivation: How motivated were they?
- Organizational attractiveness: Did the company seem appealing?
- Behavioral intentions: Did participants want to pursue further opportunities?
- Organizational image, clarity of work activity, and enjoyment: What was their overall impression?
The researchers analyzed the differences between the two groups using robust statistical methods to ensure meaningful results.
What Makes This Study Stand Out
While many earlier studies on applicant reactions relied on “low-stakes” volunteer participants – people who were not real candidates for a job, but simply took part in a research exercise – few have directly compared real job candidates to these volunteers experiencing the exact same test. As the authors note, there is a risk that findings from simulated studies do not reflect the true experiences of people whose futures are actually affected by the outcome.
This research stands out because it directly contrasts real job applicants with research volunteers using the same gamified assessment. According to the authors, previous research had not directly compared applicant reactions to the same cognitive ability test in both high-stakes (real application) and low-stakes (simulated) settings. They highlight that, specifically in the context of gamified cognitive ability tests, there had been no studies assessing applicant reactions in a real-world, high-stakes environment or directly comparing them to reactions in simulated scenarios. This direct approach provides important new insights into how real-world consequences can transform perceptions of hiring assessments.
Key Findings: High Stakes Mean Higher Marks
- Real applicants report more positive experiences. Real job applicants in the high-stakes group consistently rated the gamified assessment more favorably than the volunteers. According to the study, “the means of all applicant reaction variables were higher in the high-stakes sample than in the low-stakes sample.”
- Significant differences were observed in participants’ levels of engagement and perceptions of organizational appeal. The most pronounced differences appeared in measures of test motivation, enjoyment, and organizational attractiveness. Real job applicants reported higher motivation, found the test more enjoyable, and viewed the organization more positively compared to volunteers in the low-stakes group.
- Perceptions of fairness and job relevance less influenced by context. While real applicants perceived the process as fairer, the difference was less dramatic. Notably, there were no significant differences in perceived job-relatedness or the opportunity to demonstrate ability between the groups.
- Generalizability from low-stakes to high-stakes settings is limited. The authors note that findings from laboratory or simulated conditions do not always translate directly to real-world hiring scenarios, particularly in terms of how strongly applicants react to assessments when actual job opportunities are at stake.
- Not all aspects are equally affected. Gamification influenced enjoyment and motivation much more than perceptions of test relevance or fairness.
Why Does This Matter?
The study demonstrates that the assessment context – specifically, whether applicants are facing real-world consequences or simply participating in a research scenario – significantly influences how individuals experience and evaluate gamified hiring tests. According to the authors, real job applicants tend to report higher motivation, greater enjoyment, and a more positive perception of the employer compared to volunteers in low-stakes situations.
The researchers also point out that organizations relying solely on feedback from research volunteers or simulated settings may not gain a complete understanding of how actual candidates perceive the hiring process. They recommend that employers and researchers interested in accurately evaluating assessment methods should seek feedback directly from real applicants, as findings from laboratory or low-stakes studies may not fully capture the reactions and attitudes present in genuine hiring contexts.
Authors’ Conclusions and Future Directions
According to the study, “applicants in a high-stakes setting show more positive reactions to a gamified cognitive ability test than participants taking the same test in a low-stakes simulated selection scenario.” The authors recommend caution when applying laboratory findings to real-world selection, because “contextual factors appear to influence perceptions of gamified cognitive ability tests.”
Limitations of the research include its focus on a single company and type of assessment, and the fact that participants’ actual test scores were not compared. The authors also point out that not all gamified assessments are alike, so reactions may differ across companies and industries.
Future studies could expand to different tests, organizations, and applicant groups to better understand how context influences the experience of gamified hiring.
The information in this article is provided for informational purposes only and is not medical advice. For medical advice, please consult your doctor.
Full citation:
Ohlms, M. L., & Melchers, K. G. (2025). Do Applicant Reactions to Gamified Cognitive Ability Tests Differ Between High- Versus Low-Stakes Settings? Journal of Intelligence, 13(3), 33. https://doi.org/10.3390/jintelligence13030033
