it is the game plan of your research

Find the right market research agencies, suppliers, platforms, and facilities by exploring the services and solutions that best match your needs

list of top MR Specialties

Browse all specialties

Browse Companies and Platforms

by Specialty

by Location

Browse Focus Group Facilities

Manage your listing

Follow a step-by-step guide with online chat support to create or manage your listing.

About Greenbook Directory

IIEX Conferences

Discover the future of insights at the Insight Innovation Exchange (IIEX) event closest to you

IIEX Virtual Events

Explore important trends, best practices, and innovative use cases without leaving your desk

Insights Tech Showcase

See the latest research tech in action during curated interactive demos from top vendors

Stay updated on what’s new in insights and learn about solutions to the challenges you face

Greenbook Future list

An esteemed awards program that supports and encourages the voices of emerging leaders in the insight community.

Insight Innovation Competition

Submit your innovation that could impact the insights and market research industry for the better.

Find your next position in the world's largest database of market research and data analytics jobs.

For Suppliers

Directory: Renew your listing

Directory: Create a listing

Event sponsorship

Get Recommended Program

Digital Ads

Content marketing

Ads in Reports

Podcasts sponsorship

Run your Webinar

Host a Tech Showcase

Future List Partnership

All services

Dana Stanley

Greenbook’s Chief Revenue Officer

Research Methodologies

October 29, 2018

How to Design Research Game Concepts

Gamifying research offers a look into players’ implicit and system 1 thinking

by Emma Galvin

Creative Designer at Northstar Research

In last month’s Monthly Dose of Design, we examined how to effectively brand your research projects. This month we are looking at the key principles of conceptualising and designing a game concept that can be used within a research methodology.

But why should you use games within a research methodology?

Games are an effective way to engage research participants. This is because often people don’t know what they think or over rationalise their decisions. This means it can be hard to get a true reflection of their attitudes and behaviour. To overcome this, researchers often try to better understand attitudes and behaviours on an implicit/system 1 level (anything automatic, habitual, rapid or unconscious).

Gamifying research allows us to better understand implicit/system 1 thinking. It also increases participant engagement and increases motivation by using competition.

Having a game concept is the first step in designing a research game. A game concept is the idea that the game is built around. E.g. the concept behind Pokemon is ‘a world full of little monsters’ and the concept behind Monopoly is ‘property ownership and entrepreneurialism’.

Before you develop your concept you need to ask yourself the following key questions: What are the research objectives? Who is playing? What is the scene the game is set in? What is valuable in the game? What motivates players? What kind of problem-solving does the game use?

If you ask yourself these questions, you will be able to effectively brainstorm ideas that will help you generate a viable research game concept.

Concept Foundations

The foundations of your concept will be based on the story and aesthetics.

The story describes the world of your game. This is paramount at the start of the game as the copy and tone of voice sets the agenda for the rest of the game. The aesthetics encompass the visuals, sounds and animation. Both story and aesthetics are closely tied together and are integral to generating an engaging player experience.

Personalisation

Personalisation allows a player to become immersed – and therefore engaged – within a game. You can allow players to personalise the game experience by building personalised avatars to represent themselves.

Game Mechanics

Mechanics are the rulebook of the game. They set out the steps a player must take to achieve the goals of the game. When considering game mechanics, you don’t have to reinvent the wheel, common game mechanics include:

• Capture (gaining territory or possessions) e.g. Risk
• Exploration (discovering new places or unlocking experiences) e.g. Warcraft
• Role Playing (being someone else) e.g. Zelda
• Puzzle-Solving (matching objects or solving riddles) e.g. Solitaire
• Dice Rolling (luck based-instructions) e.g. Backgammon
• Co-operative Play (collaborating with other players) e.g. Worms

Success Criteria

There needs to be some way of knowing when players have met the objectives of the game. Clear success visuals, sound and animations help establish a sense of success and completion. These can then lead on to reward screens. Clear communication of success criteria will avoid confusion and motivate players to continue participating.

Rewarding Players

It is integral to consider how you will reward your players. Rewards help to encourage game participation and enhance the game experience. This will provide richer results and insights. There are two ways to reward players – intrinsically and extrinsically.

Intrinsic rewards consist of in-game rewards. For example, earning points, getting recognition and encouragement for earning those points or unlocking other areas/elements of the game.

Extrinsic rewards are external rewards. For example, offering vouchers or cash incentives for scoring high points or completing a certain amount of levels of the game.

Competition

Competition is an interesting element to introduce into the game. It can thrive by encouraging better engagement. If your game is team-based, it can encourage richer collaboration within the teams. This can be done by using scoring systems that compare results across different players and leadership boards.

However, only introduce competition if it aids your game objectives, otherwise, it can create an unhealthy obsession with winning at any cost i.e. What is the most effective way to overtake my competitors? Which could compromise the quality of your results?

If you follow these rules you will have the foundations to develop a research game that will engage players and allow you to better understand implicit/system 1 thinking.

Emma Galvin

37 articles

The views, opinions, data, and methodologies expressed above are those of the contributor(s) and do not necessarily reflect or represent the official policies, positions, or beliefs of Greenbook.

Comments are moderated to ensure respect towards the author and to prevent spam or self-promotion. Your comment may be edited, rejected, or approved based on these criteria. By commenting, you accept these terms and take responsibility for your contributions.

More from Emma Galvin

Insights Industry News

The Fundamentals of Layout Design for Insights

Data visualization is easy with basic design layout design principles.

September 1, 2022

Read article

Monthly Dose of Design

Innovation Management: Barriers to Innovation (Part Five)

The five main barriers to innovation.

June 29, 2022

Innovation Management: Disruptive Innovation (Part Four)

Disruptive innovation: What it is and why it’s important to MR.

May 30, 2022

Innovation Management: Breakthrough/Radical Innovation (Part Three)

Diving into breakthrough/radical innovation and how it applies to MR.

April 29, 2022

Top in Quantitative Research

Moving Away from a Narcissistic Market Research Model

Why are we still measuring brand loyalty? It isn’t something that naturally comes up with consumers, who rarely think about brand first, if at all. Ma...

Devora Rogers

Chief Strategy Officer at Alter Agents

May 31, 2023

Qualitative Research

The Stepping Stones of Innovation: Navigating Failure and Empathy with Carol Fitzgerald

Natalie Pusch

Senior Content Producer at Greenbook

March 16, 2023

Play Episode

Sign Up for Updates

Get content that matters, written by top insights industry experts, delivered right to your inbox.

67k+ subscribers

Weekly Newsletter

Greenbook Podcast

Event Updates

I agree to receive emails with insights-related content from Greenbook. I understand that I can manage my email preferences or unsubscribe at any time and that Greenbook protects my privacy under the General Data Protection Regulation.*

Get the latest updates from top market research, insights, and analytics experts delivered weekly to your inbox

Your guide for all things market research and consumer insights

Create a New Listing

Manage My Listing

Find Companies

Find Focus Group Facilities

Tech Showcases

GRIT Report

Expert Channels

Get in touch

Marketing Services

Future List

Publish With Us

Privacy policy

Cookie policy

Terms of use

Copyright © 2024 New York AMA Communication Services, Inc. All rights reserved. 234 5th Avenue, 2nd Floor, New York, NY 10001 | Phone: (212) 849-2752

User Research Throughout the Game Development Process

Chris Hugelmann

Contributing author at PlaytestCloud. Currently playing: Lost Sphear & Destiny 2.

As the prominence of user research in the tech industry continues to rise, more and more game development companies are looking to user researchers to improve the player experience within their title libraries. We recently spoke with two research strategists at Outfit7, Anja Arhar and Nives Đorđević. They confirmed for us that user research’s place within the game development life cycle has become much more involved throughout the whole process. What’s more, they said user researchers are better able to enact change throughout the course of a game’s development. This is in stark contrast to how such planning was previously established, whereby user research was generally considered quality assurance (QA), completed at the end of the development cycle, and only as a final pass.

Here, we will discuss how user research plays a critical role in the development of digital games, how that role has changed, and what types of research can be done at each step of the process to benefit your game’s first time user experience (FTUE) and beyond!

In the beginning: User research at the concept stage of development

According to the Nielsen Norman Group [1] , when an organization — or in this case, a game development studio — has reached a certain level of UX maturity, they have successfully integrated a user-centered design process within their UX or user research departments, and throughout their culture and at all leadership levels. This means they have placed adequate trust in user researcher involvement and initiative early on in a game’s development life cycle, have supported their integration into the development team, and have helped them enact positive change at all steps of the process.

As a user researcher, being involved early and often is ideal, and offers up a chance for researchers to assist with decision-making processes, and to steer your game towards better UX through their user research expertise.

Nives and Anja, for instance, pointed out that early on, methods like focus groups [2] allowed their user research team to better understand how the game concept was being perceived, and whether that perception aligned with expectations from the game development team. This was also done with a convenience sample [3] of players who had played previous titles of the game in the past. This allowed the research team to gather qualitative information about the general sentiment of a potential player’s reception to the concept being proposed, allowing the user research team to incorporate feedback from participants such as “needs” and “wants” into categories for further study.

Testing before the build: The importance of pre-production user research

Generally at this stage of the process, ideas are still amorphous and flexible, so being open-minded is key, as is testing all sorts of different concepts, as well as keeping lines of communication between the user research and development teams open.

Additionally, as a game is still in the pre-production state, this is an ideal time to complete a competitor analysis [4] of games that may have similar mechanics or audiences. This exploration allows the design team to catch any potential issues of incongruence between audience and product, as well as to gather feedback on what players want or expect to see, even before they have a working build!

Finally, testing ideas to solidify them into assets in-game is an ideal target for this step, as there is still plenty of flexibility and opportunity for change. As Nives and Anja relayed in their talk, even testing icons with paper prototypes was found to be valuable (not to mention cost effective, thereby improving feedback accessibility) and will continue to aid designers further on in the development process as they learn what works best and what needs improvement.

Research while creating the game: User research during the production phase

While games are being produced, there are usually several benchmarks or milestones that production and development teams utilize to ensure the game is on track to release, and that each step of the development process is up to a satisfactory standard before moving on. Obviously, this method offers user researchers an opportunity to intervene and help the team better understand the overall game experience, knowledge, and usability from a player’s perspective. Nives and Anja described the ways in which detailed, iterative playtesting offers fantastic results during the production phase. Indeed, much of the game will have been fully formulated at that point, with the builds of the game offering players a real glimpse into what the final game will look and feel like.

This phase is important for user researchers on the team, since they can test several different elements of gameplay, including game mechanics, visual design, usability and accessibility issues, iconography, and interface design. As these elements become solidified within different builds, testing early and often becomes vital, and impacts the extent to which valuable feedback makes its way from players to the development team.

One important aspect Outfit7 chose to highlight is the tutorial process: a necessary but sometimes overlooked element of digital games that scaffolds necessary knowledge from the game to the player. In addition to gathering useful qualitative data through a think-out-loud protocol [5] , Nives and Anja mentioned that quantitative data, such as time-on-task to complete tutorials, can be helpful in discern player behavior such as whether players are rushing through the tutorial and will inevitably churn [6] . Additionally, the use of A/B testing [7] when implementing changes to specific parts of the build will allow players to compare and contrast those alterations, and ultimately express their preferences.

Out in the players’ hands: What user research can do during release and post-release

As with many mobile games, Nives and Anja - and the entire team at Outfit7 - focused on the first-time user experience (FTUE) during the production phase of My Talking Tom Friends , to ensure that players understood the game and would continue to play. While FTUE is vital to the success of many digital games, it is also important to understand how players are playing your game in the real-world, and how their play behaviors can change throughout the gameplay process. While many games use data analytics and telemetry data [8] to understand what players are doing, an improved understanding of the qualitative side of how players are feeling is essential.

Anja and Nives discussed their usage of diary studies [9] , which, according to their practice, was a new methodology for them. This also spoke to their commitment to iteratively learning more about the method while also learning through the method. As is usually the case, their diary study was meant to look at potential reasons for player churn. Nives and Anja thus created a test plan which involved five days of gameplay at some point over a ten-day timeframe. Participants then recorded their thoughts and feelings during play sessions.

Anja and Nives’ diary study was used in combination with short surveys and wrap-up interviews at the end of the full study protocol. This exploration enabled in-house researchers to better understand player stories, their different playstyles, their feelings surrounding the game, and their motivations behind displayed behaviors, specifically from a player’s perspective.

As a result, developers were then better at referring to specific player needs, not just in terms of more skilled/less skilled players, but in terms of what motivates players, what keeps them engaged, what sorts of time commitments players are willing to use in the game, in addition to other important aspects which can ultimately guide development. These insights can inform future game updates, especially in games as a service (GaaS) [10] models, where player retention is especially important.

Concluding thoughts: integrating user research from the start within the game development process

Game development can be a lengthy, arduous experience, yet it offers a plethora of opportunities for user researchers to improve the experience players have in your game. As Nives & Anja have discussed, user researchers can have a dramatic impact on a game’s development cycle, and they can play an influential role in elevating a game to be the best it can be.

As user experience research becomes more common in the tech industry, it is vital that every part of the team is working cohesively, and that they are communicating every step of the way - ideally starting this process as early as possible. This more than anything will have the greatest impact on both the product and on overall player experience.

Want to start using user research in your game development? sign-up now and start playtesting!

https://www.nngroup.com/articles/ux-maturity-model/ ↩︎

a type of group interview with participants who have similar experiences or behaviors, to gain an understanding of their reactions and responses to predetermined research questions. ↩︎

where a sample is drawn from part of a population because they are easily accessible, though not always representative of the total population. ↩︎

an assessment of strengths and weaknesses, especially in comparison to your own product, of current or potential competitors in the same market space. ↩︎

a methodology that asks the user to engage with the system while actively verbalizing their thoughts and ideas as they are using the system. ↩︎

churn rate or churning is the act of users playing your game and then stopping, ultimately never returning. This is especially notable in the mobile and free-to-play markets. It is the opposite of a retention rate. ↩︎

a type of experiment where two different variants of a similar object/screen are shown to users at random, where the variant is a simple change that might affect user behavior. ↩︎

data that is remotely sent back to the developer from folks playing the game, such as time spent in specific areas, latency between players, and so on. ↩︎

collecting qualitative data from individuals over the course of an extended time period -- called longitudinally -- logging specific information about the activity that is being studied. ↩︎

where developers and publishers provide game content on a continual revenue model, monetizing video games after the initial purchase, or as a way to make money within a free-to-play model. ↩︎

Moderate your first Live playtest

When to use moderated and unmoderated research

Analyzing player interviews - Making sense of the data

Running great player interviews - Making the most of your time with players

Overcome the fear - preparing for your first player interview

User Research in Games: What is it and why does it matter?

User Research is all about investigating the different aspects of the user experience: how games fit into our lives, why we play them, what encourages us to keep playing, and what makes us stop. In this article we join User Researcher Project Manager Ahmad Azadvar in a deep dive into what it means to be a User Researcher and how our team at Massive works to support the creation of fulfilling experiences for players.

Making games is a complex process and part of that process is to make sure that there is a bridge between game creators and players. That means we believe that empowering game development teams with information about their players, be it studying player interaction with the product or general trends on players’ motivations, will result in better games. At Massive User Research, we are dedicated to providing actionable insights that support design and production teams through reliable data collection and analysis. Our team includes psychologists, human-computer interaction (HCI) experts, data scientists among other researchers who are interested in why we play games, how we interact with them, and the range of emotions we experience about them, even when we are not directly engaging with the game.

Verification and exploration

Typically, it starts with an idea, a question, or an assumption about players’ intended actions, emotional state, or markers of their behavior like personae, motivations, and preferences. Then our experts will use their toolbox to design a study most suitable to inform, respond to, and examine these questions. Not only do we want to verify that our games are enjoyable while meeting the standards of Usability and Accessibility, but we are also interested in exploring players’ choices and behavior as well as their relationship with concepts like psychological well-being or mood regulation.

USABILITY AND ACCESSIBILITY

Usability is one of the fundamental aspects of user experience and it refers to how pleasant, easy, and intuitive it is to interact with different elements of the game. First, the player must learn the basics of interaction and logic that connects different parts of the game. Learning is enjoyable when its function follows the context of the game and is coupled with consistent feedback and intuitive use of symbols. Of course, the player would need enough opportunities to practice and master the newly learned skill. A usable game is also efficient (no extra steps) and accounts for players’ potential errors, long-term interaction, and when they come back after a period of not playing. In other words, the game is aware of the kind of errors that a player would encounter and provides necessary information after returning to the game, to support players in recalling previous knowledge.

Accessibility then, is to ensure that the game usability is as inclusive and flexible as possible for accommodating a range of disability conditions, diversity needs, and customization options. Massive is one of the studios in the forefront in this field, with Gameslab’s Accessibility initiative that directly involves players with different types of disabilities in our in-house playtests , as well as contributions to understanding and advocating gaming needs for different generations, specifically older adults .

combining methods

Understanding the experience of playing games, far beyond functionality and ease of use, also involves the study of players’ psychology. What motivates us to play games and spend time on them? What makes one game worth playing over another? How are games able to provoke certain emotions? How and why do opinions about games change over time? These questions require User Researchers to combine methods used in social sciences with statistical models and AI techniques to unravel relationships between player behaviors, perceptions, and emotional states. For example, in one study we were able to predict players’ motivations of The Division players by their gameplay data with 94% accuracy.

The User Researchers’ toolbox, or the methods chosen to address research questions, has two main data categories: self-reported and sampled data.

User Researchers are trained to ask players to voice their opinions, expectations, ratings, and descriptions of what they do and how they feel while interacting with the game in presence of a researcher (think-aloud study) or after the play-session via surveys and interviews. They may also ask the player to keep a journal (player diaries) of certain topics or discuss them in groups with other players (focus groups, forum discussions). Researchers may also sample player behavior by observing patterns and key elements in gameplay, especially when they provide context to or contradict self-reported data. Another unintrusive method is to monitor the physiological properties of the player that are known to be connected to markers of excitement, attention, and cognitive load. Generally referred to as Biometrics, tools like Eye tracking  (gaze patterns, pupil dilation),  Galvanic Skin Response  (conductivity of the skin, a measure of arousal),  Heart Rate Variability (HRV) ,  Facial Expression recognition  and  Electromyography  (EMG)  provide a moment-by-moment picture of how players react to different beats of the game.

Although telemetry engines record a bulk of players’ behavioral markers such as performance, progression, economy interactions, sequence of actions and much more, User Researchers need to utilize their knowledge of the game to select the relevant parts of that data as well as the appropriate method to make sense or connect the telemetry data to player behavior and opinions. In an innovative example, the Massive User Research and Data Analytics teams joined forces to create a new method of categorizing character builds in Tom Clancy’s The Division 2 ( see the published paper here ).

Whether it is an intimate one-on-one, think-aloud play-session or a standardized mass-survey linked to granular gameplay metrics, part of the analysis is to figure out if aggregations of trends and behavior types (e.g. clustering, segmentation) would best answer the research question or if emphasizing on individual quotes and instances provides more context.

In any case, the User Research team is equipped and eager to attend to the games’ research needs, collect and clarify players’ perspective and contribute to the practice of player centered design. You can help us in doing so by signing up here to participate in our routine playtests of a diverse set of Ubisoft games here in Malmö.

Want to know more?

• Read more about Playtesting and sign up to become a playtester.
• Read Ahmad’s previous article about how social network analysis and data science can improve social experiences in games .
• Want to join Massive? Check out our career page !

More to read

Discover more articles

Ray Tracing Audio in Snowdrop: Creating a Living Pandora

Snowdrop’s Ray Tracing Shines a Light on Pandora

An Evolving Pandora: Snowdrop’s Dynamic World

Crafting Pandora’s Breathtaking Landscape With Snowdrop

Happy Birthday, The Division 2!

How Multiplayer Was Developed for Far Cry 3

Crafting the Storyline of Tom Clancy’s The Division’s Faye Lau

Creating Cosmetic Gear for the Fairview Crew Apparel Event

Working as a Community Developer on Tom Clancy’s The Division 2

C++, Game Development and Me: a Talk by Olafur Waage

The History of Snowdrop: From R&D Concept to AAA Engine

Collaboration and community: the snowdrop summit 2020, become a playtester.

How to write a research plan: Step-by-step guide

Last updated

30 January 2024

Reviewed by

Short on time? Get an AI generated summary of this article instead

Today’s businesses and institutions rely on data and analytics to inform their product and service decisions. These metrics influence how organizations stay competitive and inspire innovation. However, gathering data and insights requires carefully constructed research, and every research project needs a roadmap. This is where a research plan comes into play.

Read this step-by-step guide for writing a detailed research plan that can apply to any project, whether it’s scientific, educational, or business-related.

• What is a research plan?

A research plan is a documented overview of a project in its entirety, from end to end. It details the research efforts, participants, and methods needed, along with any anticipated results. It also outlines the project’s goals and mission, creating layers of steps to achieve those goals within a specified timeline.

Without a research plan, you and your team are flying blind, potentially wasting time and resources to pursue research without structured guidance.

The principal investigator, or PI, is responsible for facilitating the research oversight. They will create the research plan and inform team members and stakeholders of every detail relating to the project. The PI will also use the research plan to inform decision-making throughout the project.

• Why do you need a research plan?

Create a research plan before starting any official research to maximize every effort in pursuing and collecting the research data. Crucially, the plan will model the activities needed at each phase of the research project .

Like any roadmap, a research plan serves as a valuable tool providing direction for those involved in the project—both internally and externally. It will keep you and your immediate team organized and task-focused while also providing necessary definitions and timelines so you can execute your project initiatives with full understanding and transparency.

External stakeholders appreciate a working research plan because it’s a great communication tool, documenting progress and changing dynamics as they arise. Any participants of your planned research sessions will be informed about the purpose of your study, while the exercises will be based on the key messaging outlined in the official plan.

Here are some of the benefits of creating a research plan document for every project:

Project organization and structure

Well-informed participants

All stakeholders and teams align in support of the project

Clearly defined project definitions and purposes

Distractions are eliminated, prioritizing task focus

Timely management of individual task schedules and roles

Costly reworks are avoided

• What should a research plan include?

The different aspects of your research plan will depend on the nature of the project. However, most official research plan documents will include the core elements below. Each aims to define the problem statement , devising an official plan for seeking a solution.

Specific project goals and individual objectives

Ideal strategies or methods for reaching those goals

Required resources

Descriptions of the target audience, sample sizes , demographics, and scopes

Key performance indicators (KPIs)

Project background

Research and testing support

Preliminary studies and progress reporting mechanisms

Cost estimates and change order processes

Depending on the research project’s size and scope, your research plan could be brief—perhaps only a few pages of documented plans. Alternatively, it could be a fully comprehensive report. Either way, it’s an essential first step in dictating your project’s facilitation in the most efficient and effective way.

• How to write a research plan for your project

When you start writing your research plan, aim to be detailed about each step, requirement, and idea. The more time you spend curating your research plan, the more precise your research execution efforts will be.

Account for every potential scenario, and be sure to address each and every aspect of the research.

Consider following this flow to develop a great research plan for your project:

Define your project’s purpose

Start by defining your project’s purpose. Identify what your project aims to accomplish and what you are researching. Remember to use clear language.

Thinking about the project’s purpose will help you set realistic goals and inform how you divide tasks and assign responsibilities. These individual tasks will be your stepping stones to reach your overarching goal.

Additionally, you’ll want to identify the specific problem, the usability metrics needed, and the intended solutions.

Know the following three things about your project’s purpose before you outline anything else:

What you’re doing

Why you’re doing it

What you expect from it

Identify individual objectives

With your overarching project objectives in place, you can identify any individual goals or steps needed to reach those objectives. Break them down into phases or steps. You can work backward from the project goal and identify every process required to facilitate it.

Be mindful to identify each unique task so that you can assign responsibilities to various team members. At this point in your research plan development, you’ll also want to assign priority to those smaller, more manageable steps and phases that require more immediate or dedicated attention.

Select research methods

Once you have outlined your goals, objectives, steps, and tasks, it’s time to drill down on selecting research methods . You’ll want to leverage specific research strategies and processes. When you know what methods will help you reach your goals, you and your teams will have direction to perform and execute your assigned tasks.

Research methods might include any of the following:

User interviews : this is a qualitative research method where researchers engage with participants in one-on-one or group conversations. The aim is to gather insights into their experiences, preferences, and opinions to uncover patterns, trends, and data.

Field studies : this approach allows for a contextual understanding of behaviors, interactions, and processes in real-world settings. It involves the researcher immersing themselves in the field, conducting observations, interviews, or experiments to gather in-depth insights.

Card sorting : participants categorize information by sorting content cards into groups based on their perceived similarities. You might use this process to gain insights into participants’ mental models and preferences when navigating or organizing information on websites, apps, or other systems.

Focus groups : use organized discussions among select groups of participants to provide relevant views and experiences about a particular topic.

Diary studies : ask participants to record their experiences, thoughts, and activities in a diary over a specified period. This method provides a deeper understanding of user experiences, uncovers patterns, and identifies areas for improvement.

Five-second testing: participants are shown a design, such as a web page or interface, for just five seconds. They then answer questions about their initial impressions and recall, allowing you to evaluate the design’s effectiveness.

Surveys : get feedback from participant groups with structured surveys. You can use online forms, telephone interviews, or paper questionnaires to reveal trends, patterns, and correlations.

Tree testing : tree testing involves researching web assets through the lens of findability and navigability. Participants are given a textual representation of the site’s hierarchy (the “tree”) and asked to locate specific information or complete tasks by selecting paths.

Usability testing : ask participants to interact with a product, website, or application to evaluate its ease of use. This method enables you to uncover areas for improvement in digital key feature functionality by observing participants using the product.

Live website testing: research and collect analytics that outlines the design, usability, and performance efficiencies of a website in real time.

There are no limits to the number of research methods you could use within your project. Just make sure your research methods help you determine the following:

What do you plan to do with the research findings?

What decisions will this research inform? How can your stakeholders leverage the research data and results?

Recruit participants and allocate tasks

Next, identify the participants needed to complete the research and the resources required to complete the tasks. Different people will be proficient at different tasks, and having a task allocation plan will allow everything to run smoothly.

Prepare a thorough project summary

Every well-designed research plan will feature a project summary. This official summary will guide your research alongside its communications or messaging. You’ll use the summary while recruiting participants and during stakeholder meetings. It can also be useful when conducting field studies.

Ensure this summary includes all the elements of your research project . Separate the steps into an easily explainable piece of text that includes the following:

An introduction: the message you’ll deliver to participants about the interview, pre-planned questioning, and testing tasks.

Interview questions: prepare questions you intend to ask participants as part of your research study, guiding the sessions from start to finish.

An exit message: draft messaging your teams will use to conclude testing or survey sessions. These should include the next steps and express gratitude for the participant’s time.

Create a realistic timeline

While your project might already have a deadline or a results timeline in place, you’ll need to consider the time needed to execute it effectively.

Realistically outline the time needed to properly execute each supporting phase of research and implementation. And, as you evaluate the necessary schedules, be sure to include additional time for achieving each milestone in case any changes or unexpected delays arise.

For this part of your research plan, you might find it helpful to create visuals to ensure your research team and stakeholders fully understand the information.

Determine how to present your results

A research plan must also describe how you intend to present your results. Depending on the nature of your project and its goals, you might dedicate one team member (the PI) or assume responsibility for communicating the findings yourself.

In this part of the research plan, you’ll articulate how you’ll share the results. Detail any materials you’ll use, such as:

Presentations and slides

A project report booklet

A project findings pamphlet

Documents with key takeaways and statistics

Graphic visuals to support your findings

• Format your research plan

As you create your research plan, you can enjoy a little creative freedom. A plan can assume many forms, so format it how you see fit. Determine the best layout based on your specific project, intended communications, and the preferences of your teams and stakeholders.

Find format inspiration among the following layouts:

Written outlines

Narrative storytelling

Visual mapping

Graphic timelines

Remember, the research plan format you choose will be subject to change and adaptation as your research and findings unfold. However, your final format should ideally outline questions, problems, opportunities, and expectations.

• Research plan example

Imagine you’ve been tasked with finding out how to get more customers to order takeout from an online food delivery platform. The goal is to improve satisfaction and retain existing customers. You set out to discover why more people aren’t ordering and what it is they do want to order or experience. 

You identify the need for a research project that helps you understand what drives customer loyalty . But before you jump in and start calling past customers, you need to develop a research plan—the roadmap that provides focus, clarity, and realistic details to the project.

Here’s an example outline of a research plan you might put together:

Project title

Project members involved in the research plan

Purpose of the project (provide a summary of the research plan’s intent)

Objective 1 (provide a short description for each objective)

Objective 2

Objective 3

Proposed timeline

Audience (detail the group you want to research, such as customers or non-customers)

Budget (how much you think it might cost to do the research)

Risk factors/contingencies (any potential risk factors that may impact the project’s success)

Remember, your research plan doesn’t have to reinvent the wheel—it just needs to fit your project’s unique needs and aims.

Customizing a research plan template

Some companies offer research plan templates to help get you started. However, it may make more sense to develop your own customized plan template. Be sure to include the core elements of a great research plan with your template layout, including the following:

Introductions to participants and stakeholders

Background problems and needs statement

Significance, ethics, and purpose

Research methods, questions, and designs

Preliminary beliefs and expectations

Implications and intended outcomes

Realistic timelines for each phase

Conclusion and presentations

How many pages should a research plan be?

Generally, a research plan can vary in length between 500 to 1,500 words. This is roughly three pages of content. More substantial projects will be 2,000 to 3,500 words, taking up four to seven pages of planning documents.

What is the difference between a research plan and a research proposal?

A research plan is a roadmap to success for research teams. A research proposal, on the other hand, is a dissertation aimed at convincing or earning the support of others. Both are relevant in creating a guide to follow to complete a project goal.

What are the seven steps to developing a research plan?

While each research project is different, it’s best to follow these seven general steps to create your research plan:

Defining the problem

Identifying goals

Choosing research methods

Recruiting participants

Preparing the brief or summary

Establishing task timelines

Defining how you will present the findings

Should you be using a customer insights hub?

Do you want to discover previous research faster?

Do you share your research findings with others?

Do you analyze research data?

Start for free today, add your research, and get to key insights faster

Editor’s picks

Last updated: 18 April 2023

Last updated: 27 February 2023

Last updated: 6 February 2023

Last updated: 6 October 2023

Last updated: 5 February 2023

Last updated: 16 April 2023

Last updated: 7 March 2023

Last updated: 9 March 2023

Last updated: 12 December 2023

Last updated: 11 March 2024

Last updated: 6 March 2024

Last updated: 5 March 2024

Last updated: 13 May 2024

Latest articles

Related topics, .css-je19u9{-webkit-align-items:flex-end;-webkit-box-align:flex-end;-ms-flex-align:flex-end;align-items:flex-end;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-box-flex-wrap:wrap;-webkit-flex-wrap:wrap;-ms-flex-wrap:wrap;flex-wrap:wrap;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;row-gap:0;text-align:center;max-width:671px;}@media (max-width: 1079px){.css-je19u9{max-width:400px;}.css-je19u9>span{white-space:pre;}}@media (max-width: 799px){.css-je19u9{max-width:400px;}.css-je19u9>span{white-space:pre;}} decide what to .css-1kiodld{max-height:56px;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}@media (max-width: 1079px){.css-1kiodld{display:none;}} build next, decide what to build next.

Users report unexpectedly high data usage, especially during streaming sessions.

Users find it hard to navigate from the home page to relevant playlists in the app.

It would be great to have a sleep timer feature, especially for bedtime listening.

I need better filters to find the songs or artists I’m looking for.

Log in or sign up

Get started for free

How To Conduct User Research In The Prototype Stage For Your Game

Testing ideas in the prototype stage is a vital step in your game’s development. But what should you test? And who to? In this post from GameAnalytics, they answer just that…

Editor’s note:  This post was originally published by Sarah Impey,  Content Marketing Manager at GameAnalytics . With several years of experience digging into industry trends, Sarah knows how to boil down complicated topics into key insights.

Whether it’s your game concept or your core game features, prototyping and testing ideas at an early stage can you let you know a lot about the health of your game, and more specifically, if people are actually going to enjoy it.

Ideally, you’ll want to make sure your title is popular the moment it hits the market. Not only is testing in the early stages of development good for getting your game out there, but it can also help you focus your production process on features that will make the most impact on your intended audience.

In this post, you’ll find the steps you need to take when conducting user research during that all-important prototype phase. GameAnalytics included a bunch of different examples of specific test methods (with pros and cons of each) as well as advice for pulling together your very own test plan.

This is a bit of a lengthy post, so feel free to skip to a specific section if you’re looking for something in particular:

Step 1: Plan out your approach

Step 2: make sure you know what you’re testing, and why.

• Step 3: Choose the right testing method

Step 4: Reviewing the results

• Top takeaways (TL;DR)
• Useful links
• Your User Research Checklist  (useful guide that you can download)

Testing is great. It can tell you things about your game that you’re blind to, or just confirm suspicions that may have been playing on your mind. However, before you test anything about your game, you’ll need to take a moment to reflect on the purpose of your prototype stage.

This stage is about figuring out exactly what you need to test in order to improve your game, but also laying out the steps you need to take to move forward. A few questions you’re going to need to answer are:

• What is your  ultimate objective ?
• What are your  plans post-test ?
• What  resources  do you have?
• How much time  can you dedicate to test?
• Which  features  do you need to test?

Now is the time to define your strategy for your user research. The approach you take should be tailored to the data that matters to your game development, whether it’s feedback on game angles, ideas, concepts, gameplay, or, simply just the design.

For example, if I’m creating a hyper-casual title with simple mechanics, it would be foolish to test narrative. A better use of time here would be the games core loop, difficulty or level design.

To help you understand what this requires, GameAnalytics created a useful document with the essential questions and checklists you’ll need when building a prototype test plan. You can download a copy here .

What on earth are you testing?

So you have your approach all together, and you’re ready to take the next step. At this stage, you’re going to have to look at your title and decide exactly on what you’re going to test. Depending on the approach, this could either be a particular in-game feature, aesthetics, the overall UX, or even a monetization strategy.

Knowing exactly what you’re testing will inform the whole structure of the user research. If you’re searching for feedback on your game’s concept, for example, you might want to mock-up an app store screen with descriptions and images to see how new players respond.

Who should you be testing?

Once you know what you’re testing, you’ll have a better idea of who you should be testing it with. This refers to the number of participants you need to gather results from, as well as the demographics and  player personas  you need to target.

If you want to gain more qualitative feedback on whether a specific kind of player will enjoy your game, playtesting sessions that involve face-to-face interviews might be the best option for you. To do this, you’ll need to be familiar with the type of players your game is targeting, considering things like age, gender, platform, player experience levels and interests.

To give you an idea of what we mean, here’s a few different examples:

• Specific audience:  Hard-core gamers who play for 20+ hours a week, aged 20 – 40, lives in London
• Less specific:  Women, aged 50+ who are interested in puzzle games
• Open : 10 Beta players

Why are you doing this test?

You should have a pretty good idea what the answer is to this question, but it’s important to write this down. It’s easy to get lost in all of the nitty gritty details when conducting your user research, so this all-important ‘why’ will bring you back to Earth, and keep your vision and goal clear.

A few examples of ‘why’ you’ll be testing could be:

• See if players simply  enjoy the core features
• Understand first impressions  of the game in the app store
• Find out  if levels are too difficult  or too easy
• Learn  how players interact  with the core game

How are you going to test?

There’s an appropriate tool for every job. Once you’ve answered the above questions, you should be in a good place to decide on which testing methods are the best fit for your purpose. For instance, your decision will vary massively depending on whether you’re searching for in-depth qualitative feedback on player experiences, or if you’re searching for more general feedback from a large sample of participants.

GameAnalytics actually listed some of the most popular testing methods below to aid your user research during the prototype stage…

Step 3: Choose the right testing methods

A/b testing.

As you may already be aware, A/B testing lets you test two or more things at the same time in a controlled environment. These types of tests normally are an iteration of one aspect, as it’s important to make sure you’re testing apples against apples.

For example, you can test a red icon colour against a green icon colour. But you can’t test a red icon colour against a different type of font – the results of this will just confuse you and lead to inconclusive results.

Online surveys

The use of online market research tools has come a long way in the last decade. There are now several game-specific platforms (like  Playtest Cloud ,  Sekg ,  Player Research , and more) that are set up to offer you easy survey functionality when it comes to testing elements of your game.

Diary studies

Diary studies are a form of long-term research that takes place over a set duration of time. Typically, users report their activities at regular intervals to create a record that developers can study. It’s up to you how this is structured, but it can include player’s thoughts, frustrations or enjoyability levels over a number of sessions.

Expert reviews

Mobile game experts can offer something unique when it comes to prototype testing. Whether it’s a pro gamer or a professional prototype tester, sometimes you need feedback from someone who has a firm grasp of the medium to inform your development strategy. This should help offer actionable development advice that can help you transform specific features of your game.

Face-to-face interviews

There are a few variations for how this test method works. A basic model is having a group of testers play your game for a set period of time, allowing them test out different features and experience the game as a whole. Following this, an interview can be performed to gather specific feedback related to their experience.

When reviewing your test results, everything should tie back in to your test objectives as outlined in your test plan. In the end, your interviews, studies or surveys are all designed to improve your overall development process, so make sure you use the results to inform your strategy going forward.

The data you receive should also be reviewed as objectively as possible. If your team are too close to the situation to judge it fairly, then bring in some fresh eyes to help you understand your next course of action. It’s important that you don’t let emotions get in the way. If you receive some negative feedback, you simply need to suck it up and adapt your future approach – there’s no two ways about it.

If your results confirm your initial ideas then hooray! You can now act with the confidence you need to invest additional resources into further developing and refining your game, bringing it that one step closer to the finish line.

Here’s an example…

As we are a strong advisor of ‘show don’t tell’, here’s an example of how you can conduct your user research for your next project. Also, don’t forget to download GameAnalytics’ template , which should help you cut some corners when setting up.

Example: We have an unproven game concept in mind, but we don’t know if players will like it

• We need to find out if our concept will resonate with potential players. To achieve this, we’ll need to mock-up an app store page, with fake images, text and gameplay. Our resources are limited and budget tight, so this route will make sure we don’t exhaust ourselves in case we need to do more testing.
• What: A fake app store page, with images, text and mock gameplay
• Who: Men aged 20 – 35 who work in London and play mobile games (roughly 300 invitations.)
• Why: To find out if our game’s concept is interesting enough for players to want to download
• How: Online survey
• Create mock assets for testing
• Decide and set up online survey questions
• Identify audience and send out invitation
• Decide on closing date
• At this stage, we should have a collection of results that will help decide our next steps.

Due to the high costs associated with developing and launching new games, identifying failing ideas as fast as possible is crucial. This could save you a ton of time in the long run, as well as make sure you’re on the path to success.

When figuring out your approach, don’t get stuck within a rigid test format. What might work for one studio might not necessarily work for you, so tailor your user research to suit your target audience, genre and the development stage of your game (as well as the resources that you have).

Finally, use a collection of different test methods for more reliable results (if you can). Any areas that overlap between different methods will give you more solid conclusions, rather than relying on the success of one method alone.

More on game testing

For more information and advice on game testing, GameAnalytics collected some great links that can help you while you’re conducting research during the prototype phase:

• How To Do An Efficient Game Test
• A/B testing with Custom Dimensions
• UX Walkthrough: Anatomy of a Usability Test in Video Games (Part-1)

Related posts

All You Need to Know About the Chinese Mobile Games Market: Genres, Publishers, Brands, and Features

Overview of the Current Mobile Sports & Racing Games Market

Get our latest news in a monthly digest into your inbox!

Trusted by leading game studios and publishers.

UX Research Plan Cheat Sheet to Understanding Games User Research

The Acagamic Tip Tuesday #56

In today’s newsletter, you’re going to learn how to navigate the basics of games user experience.

Understanding how to bridge the gap between design intent and user needs for the created artifact can immensely benefit your game design process and you stakeholders. This knowledge empowers designers to make informed decisions and evaluate the intended effect of their decisions. It ultimately enhances the gaming experience for users and can significantly boost the overall success of a game.

Unfortunately, many people underestimate the importance of user research in games or struggle to make actionable progress due to several barriers.

Lack of Domain Knowledge

Often, people don’t fully comprehend the unique nuances and requirements of the gaming industry and its stakeholders. Here are 4 primary reasons development teams tend to falter in game UX design and user experience research:

• Lack of familiarity with gaming : Understanding the language and mechanics of games is vital. You can’t effectively research without engaging in the medium you’re studying. UX practitioners, expert user researchers, UX researchers, and UX teams focused on research and design of games, all ask research questions focused on this relevant product or service. Your background may be in user interfaces, user behaviours, or human-computer interaction (HCI), maybe even gamification, but it’s essential to understand experience design in games.
• Neglecting the emotional aspect of gaming : Unlike usability research for applications where efficiency is key, games require a balance of challenge and enjoyment. Overlooking this can lead to ineffective research.
• Underestimating the importance of designer’s intent : Successful user research in games requires a deep understanding of the designer’s vision and goals. Analytics (or even if you organize content in heat maps) don’t provide insights on game mechanics without this.
• Challenges of the secretive nature of the gaming industry : The highly confidential nature of game development can hinder the application of conventional user research methods.

Worry not, my friend. You can overcome these hurdles in every project and become proficient in user-centered UX practices. Here’s how UX research helps, step by step:

1. Immerse yourself in the world of gaming with user research methods

Understanding the language and mechanics of games is crucial to conducting effective user research. By immersing yourself in the gaming world, you get a firsthand experience of what gamers go through.

For instance, if you’re studying an adventure game, play it. Understand the challenges players face (maybe even collect user feedback, a researcher might use surveymonkey for popular forms, do some qualitative research project or playtests), the rewards they seek, and the journey they undertake. If working with other groups of people playing, ask them questions. This first-hand experience will provide invaluable insights that can be a way to learn and guide your research. Keep in mind that every project is different.

2. User research helps understand the emotional UX aspects of gaming

A common mistake researchers make is treating game UX research like usability testing for web applications. The objective of a game isn’t necessarily efficiency—it’s enjoyment. Wireframes (or even information architecture and card sorting) can help make better game UIs, but interaction design to test assumptions for actual users with different goals is more important to build the right thing.

To help you understand, consider a scenario where a player is battling a dragon in a game. The player might not want to slay the dragon quickly because the process—casting spells, using weapons—is enjoyable (for a range of people). As a researcher, understanding this emotional aspect of gaming is critical to capturing the user’s experience accurately.

3. Develop a UX research plan of the designer’s intent

The key to successful player experience research is understanding the designer’s intent. This understanding is the bridge between design intent (the problem space of design decisions) and the created product or feature.

Let’s take an example: a designer may intentionally create a difficult to find passage in a game to challenge the player. As a user researcher, you may flag this as a problem, thinking players are unable to find it. However, if this difficulty aligns with the designer’s intent, it’s not a problem. Understanding the designer’s intent can help you better identify actual issues that hinder the gaming experience.

By following these steps, you’ll be well on your way to conducting effective games user research, improving the gaming experience, and ultimately contributing to the success of the game.

Lennart Nacke, PhD

Hey there, I am a Professor and the Research Director of the HCI Games Group at the University of Waterloo in Canada. I am a world-leading expert on what makes games engaging and how we can use them to improve products, systems, and services. My research is widely discussed and recognized by the New Yorker, Forbes, MIT Technology Review, CTV News, New Scientist, The Daily Mail, PC Gamer Magazine, and elsewhere. I have edited a textbook on Games User Research and authored hundreds of academic articles in gamification, user experience research, human-computer interaction, and game design.

Privacy Policy · Terms

• Reviews / Why join our community?
• For companies
• Frequently asked questions

Games User Research

What is games user research.

Games User Research focuses on understanding players' behavior, interactions, and experiences in video games. Researchers use methodologies like observations, interviews, and surveys to gather valuable data. This data helps improve games, remove bugs, and increase player experience.

Steve Bromley, a games user research expert who has worked with companies such as Sony Interactive Entertainment and EA, gives an overview of Games User Research:

• Transcript loading…

The Role of a Games User Researcher

Games user researchers combine principles of psychology, human-computer interaction , and UX design to study how players interact with video games. Their studies uncover potential issues in the game mechanics, user interface , or any other aspect that could negatively affect the player's experience.

Games user researchers work closely with game designers, producers, and the UX team to ensure the game aligns with the designer’s vision and meets the player's expectations.

Researchers analyze the collected data to identify patterns, trends, and potential usability issues. Then, they communicate these findings to the design team in a way that is easy to understand. Their conclusions create empathy between the design team and the players. Researchers also provide actionable recommendations to improve the game's design and enhance the player experience.

Differences between User Research and Games User Research

While games user research and user research are similar, there are a few key differences.

User Needs: Tasks vs Entertainment

User researchers focus on user needs —what users want to achieve, their challenges, etc.

For example, the design team for an e-commerce website creates a mega menu to help users see all product categories at once. However, the research team finds that their users find it challenging to find the products because of the way they are ordered on the menu. The design team incorporates this insight to restructure the navigation. The navigation is a functional element to help users complete a goal, in this case, to find products quickly.

On the other hand, games are a form of entertainment, an art form, like movies. Scriptwriters write movies with the intention of creating an emotional response in the viewer. Game designers do the same but for games. Therefore, in games, user researchers approach their work from the designer’s point of view. Unlike other products where designers rely on user research to decide what to create, game designers begin with a vision, and researchers evaluate if users' experience matches the designers’ vision . 

For example, if the purpose of a horror game is to make players feel scared and nervous, the research team’s goal is to find out how the players are feeling. If the players are not “scared enough,” the designer can use research insights to reach their vision.

User researchers ask users, “What difficulties do you encounter while using our platform/product/service?” Designers use this research to remove these difficulties.

However, in games, difficulty improves the experience . It's essential as it keeps players interested while enhancing their skills. As players overcome challenges, they have more fun.

An excellent example of difficulty in games is the popular game series Dark Souls. The Dark Souls games are famous for being incredibly difficult to complete. However, this is what has made them popular. Players enjoy the satisfaction of overcoming the challenges of the game.

© Bandai Namco, Fair Use

Researchers need to identify when difficulty is intentional or accidental:

When difficulty is intentional, it elevates the gaming experience. As players move through the game, they will become bored if it does not get progressively more difficult. As intentional difficulty increases, so does the skill of the player. Equally, If level 5 is easy compared to level 3, this will not meet the players’ expectation of increasing difficulty through the game.

When difficulty is accidental, it reduces the player's enjoyment level. The game does not immerse the player, who may eventually give up. Examples of unintentional difficulty include glitches and bugs, poor balancing, or illogical changes in gameplay.

In the massively multiplayer online game World of Warcraft, there was an incident known as the “Corrupted Blood Incident.” The game's developers introduced a new enemy who cast a spell on players, giving them a contagious disease. Only when a player defeated the enemy was the disease healed. Many players left the area where the enemy existed without beating it and spread the disease throughout the game. This unintended consequence resulted in many players having difficulty playing the game. Their characters would get infected, die, respawn, and then catch the disease again, repeating the cycle—an example of accidental difficulty.

© Blizzard Entertainment, Fair Use

Game researchers need to understand secrecy's crucial role in game development. Marketing and advertising are critical to a game’s commercial success. Game studios usually protect their games to prevent leaks that could disrupt marketing strategies.

Secrecy directly impacts research methods, especially those involving the public. Some research methods, like public surveys, are not usable when you must preserve secrecy. Therefore, the research methods in games user research can differ from those used in other industries.

Why Is Games User Research Important?

“Games user researchers bring structure to the playtesting process so that game developers are confident that the game they’re making is experienced by players in the way they want them to be experiencing it.”  – Steve Bromley, Games user research expert

The video game industry is highly competitive. Studios aim to create games that captivate players and keep them engaged so they don’t switch to a competitor’s game. Games user researchers help achieve this by identifying issues to fix throughout development.

Steve Bromley explains the concept of playtesting and why applying user research practices to the process is important:

Let’s look at a simple scenario: a game studio is building a game. They know what type of game they want to make and begin developing it. 

1. What Happens If You Don’t Use Games User Research?

Only the game studio employees play the game throughout the development process. Since they’ve built it, they are biased and only identify and fix issues they find themselves. This approach saves them time and money and means they can launch the game sooner. 

When the game launches, they look at the user feedback and discover their game is filled with bugs. Players also find the game too complicated and don’t understand how to play. Ultimately, the game doesn't sell many copies, and the industry considers it a failure.

2. What Happens If You Do Use Games User Research?

Throughout the development process, the game studio employs user research. Since the users testing the game are new to it, they highlight and identify many issues the studio hadn’t noticed.

Using these findings, they fix the issues and continue to employ user research to identify and fix further issues. This approach adds extra time to the development process and costs money, but the studio feels more confident their game will succeed. 

When the game launches, the feedback is positive, and the players enjoy the game. Ultimately, the game sells many copies, and the industry considers it successful.

Games User Research Sets You Up for Success

The process of game design is, of course, much more complex than this. However, these scenarios show how vital games user research is to game development.

Researchers can provide important information to guide the design team when they understand the players' preferences and behaviors. This results in better game experiences, higher player retention rates, and commercially successful games.

Games User Research Methods

Games user researchers use many of the same user research methods employed in other industries, such as observation, interviews, surveys and analytics.

Once the development team has a working version or demo of the game, researchers will do most of their user research through playtesting . Playtesting is where users play the game, and researchers collect data using qualitative and quantitative approaches.

Researchers conduct playtesting with one-to-one or small groups and large groups of 20+ players (also known as mass playtesting or multi-seat testing).

Typically, researchers employ qualitative methods for small groups and quantitative methods for mass playtesting.

Qualitative Research 

Researchers use qualitative research to observe and talk with players in smaller groups to understand them better. Qualitative research helps us discover how players feel, act, and think, leading to better designs.

Qualitative research methods include:

User interviews – It's helpful to talk to players throughout game development. In the ideation phase, you can understand their behaviors and preferences. Later on, through playtesting, you can ask about their experience with the game and how it makes them feel.

Observation – Watch your players as they play. Record their choices, successes, and struggles.

Diary Studies – Participants record their thoughts, feelings, and experiences. This approach allows you to gain insights into users' experiences over a period of time.

Quantitative Research

Researchers use quantitative research to study people's attitudes and behaviors based on statistical data. Due to the importance of secrecy in games, quantitative research methods like public surveys may not be available during game development. This research method is more likely to result in someone leaking the game than if you work directly with a smaller group of users.

However, games have many variables and require many users to test them. For this reason, game studios employ mass playtesting, where dozens of players test the game simultaneously. Given these large user groups, researchers can use quantitative methods to collect data. 

Quantitative research methods include:

Analytics – Monitor players' actions while they play. Examples of analytics include:

Time spent playing.

How many times players had to restart a level.

How many items players have collected.

Surveys and Questionnaires – Gather information from players after they have played the game. Use carefully crafted questions to understand how players felt and what they experienced when they played the game.

When Does Games User Research Begin?

Researchers conduct research throughout the whole game development process. It is essential to keep the bridge between designers and players strong. This empathy avoids scenarios where designers build features and find out too late that players don’t like them.

In the ideation phase, game designers want to understand:

Do users find this idea fun?

Is it worth building a game from this idea?

Does this idea give users the emotional response and reaction we want?

To answer these questions, researchers speak to target users of the game. Since secrecy is crucial in game design, researchers typically use qualitative methods such as interviews and diary studies . This research helps design teams create mental models of their target users to understand their expectations and how they think.

For example, a design team plans to build a platform game like Super Mario. Before they begin development, they interview users who love platform games. Through this research, they discover players prefer platform games with strong narratives. When the character grows, learns, and changes over the story's progression, it makes it worth investing their time in the game.

Pre-production

As soon as designers have a playable demo, they playtest it. The players will reveal if the basics of the game are fun.

At this stage, researchers can begin using the observation method. By watching how users play and approach the game, they can understand essential information to inform game development.

For example, in the platform game, researchers might discover that players try to explore areas of the level that don’t exist. This discovery can translate into a design element where the team includes secret and hidden areas to add further engagement and player retention to the game.

The production phase is where studios build the whole game. Game studios conduct playtesting regularly to ensure that the game meets the designers' vision and that players enjoy it.

Since mass testing is now possible, researchers can employ quantitative methods like analytics to understand where designers can improve the game.

For example, players may spend more time on a particular level than others. These findings show that the level's objectives need to be clearer and that it should be easier for players to achieve them.

Post-Production

During post-production, studios balance and tune the game. Researchers gather much information from playtests at this stage to inform final design decisions.

For example, a game studio builds a game where you can play multiple characters. The majority of players are choosing one character in particular. Researchers discover that this is because, compared to the other characters, this specific character is much stronger. These findings inform the design team that they must refine this character to make them equally powerful as the other characters.

Post-Launch

Before the widespread use of the internet, video game developers could not update games after releasing them. Occasionally, developers released updated versions of popular games with improvements, but this was rare.

Most game developers now have the option to update their games after launch. Since secrecy is no longer a concern, researchers can use quantitative methods such as public surveys and questionnaires to gather vast amounts of player feedback and data. With this new data, studios can fix previously undiscovered issues and improve the player experience.

Some game developers also employ early access. Early access is when developers release a game unfinished before they complete development. Indie developers and crowdfunded games often use this tactic to fund the game's completion. Early access contradicts the need for secrecy in game development but ultimately can be more beneficial to some game studios.

Early access is a particularly effective way to gather study participants. Since players are usually fans of the game genre or developer, they will happily play an unfinished game.

An example of a successful early access release is the role-playing game Baldur’s Gate 3. Developer Larian Studios released the early access version of the game in October 2020. Baldur’s Gate 3 is a sprawling, incredibly complex game with many paths players can take. The early access period lasted almost three years until Larian Studios released the game in September 2023. Early access allowed Larian Studios to collect vast user feedback to improve the game. Baldur’s Gate 3 became one of the best-selling, top-rated games of 2023.

© Larian Studios, Fair Use

How to Approach Games User Research

Researchers use a 4-step process to carry out their study.

The first step in the research process is to define the objectives— what is the purpose of the research? What do you want to find out? Talk to the game designers, producers, and UX designers to understand what they want to discover. 

Researchers may also read through previous research and analysis to understand what others have uncovered. It is also essential they play the current version of the game, as well as similar games from competitors.

Armed with this knowledge, researchers can answer the following questions:

What goals do we want to reach with our research, and what do we want to learn from it?

What plan do we have to carry out the study?

When are we scheduling the study, and who are our targeted participants?

Which stakeholders will benefit from the study results? 

Those working individually or on a small design team may already know the purpose of their research. However, anyone conducting research must ask themselves these questions to ensure they get the data they need.

Depending on the research objectives and the game development stage, researchers choose the appropriate research method(s). 

If the research aims to understand how a game level makes your players feel, they may use a qualitative method like user interviews. On the other hand, they might use a quantitative method like analytics to learn how long players spend on a certain level.

Researchers also need to gather research participants. Design consultancy IDEO's uses a method to recruit “Extremes” and “Mainstreams.” This method enables researchers to cover the entire spectrum of the target group. 

Extremes are users who, for example: 

Have minimal gaming experience.

Prefer other genres.

If these participants enjoy the game, most other players will, too. 

If you use the extremes and mainstreams methods, remember to include the mainstream users as well. Mainstream users are the ones who represent the majority of your target group.

Always include a small number of extreme users in your study. They are more likely to highlight issues only newcomers will encounter with your game.

© Interaction Design Foundation, CC BY-SA 4.0

Many researchers also act as moderators when using qualitative research methods like interviews or observations. Moderators help their research participants feel comfortable. A solid ability to empathize and be inquisitive is beneficial to being a moderator.

Once researchers have gathered their data, they analyze it and present their findings. How researchers present their data will depend on the research purpose and methods. However, it is essential to present data in a way the design and development teams can easily understand.

Learn More about Games User Research

Take our course User Research – Methods and Best Practices to build your foundational knowledge of user research.

Watch Steve Bromley’s Master Class, How to Become a Games User Researcher , for insights from a GUR expert.

You can also read Steve’s blog to learn about GUR in more detail.

Learn about the differences and similarities between game and mainstream user research.

Access a library of presentations from the Games User Research Summit , presented by industry experts.

Questions about Games User Research

Some highly cited research on games user research and related topics include:

Desurvire, H., & El-Nasr, M. S. (2013). Methods for Game User Research: Studying Player Behavior to Enhance Game Design . IEEE Computer Graphics and Applications, 33(4), 82-87. 

Mirza-Babaei, P., Nacke, L., & Drachen, A. (2018). Games User Research Methods . In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts (CHI PLAY '18 Extended Abstracts) (pp. 1-4). Association for Computing Machinery. 

Nacke, L. E. (2017). Games user research and gamification in human-computer interaction . XRDS, 24(1), 48–51.

Smeddinck, J., Krause, M., & Lubitz, K. (2020). Mobile Game User Research: The World as Your Lab? [Epub ahead of print]. arXiv. 2012.00378

Shin, Y., Kim, J., Jin, K., & Kim, Y. B. (2020). Playtesting in Match 3 Game Using Strategic Plays via Reinforcement Learning . IEEE Access, 8, 51593-51600.

Lee, I., Kim, H., & Lee, B. (2021). Automated Playtesting with a Cognitive Model of Sensorimotor Coordination . In Proceedings of the 29th ACM International Conference on Multimedia (MM '21) (pp. 4920–4929). Association for Computing Machinery.

Mirza-Babaei, P., Stahlke, S., Wallner, G., & Nova, A. (2020). A Postmortem on Playtesting: Exploring the Impact of Playtesting on the Critical Reception of Video Games . In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20) (pp. 1-12). Association for Computing Machinery.

Ariyurek, S., Surer, E., & Betin-Can, A. (2022). Playtesting: What is Beyond Personas [Epub ahead of print]. arXiv. 2107.11965

Sevin, R., & DeCamp, W. (2020). Video Game Genres and Advancing Quantitative Video Game Research with the Genre Diversity Score . Computational and Game Journal, 9, 401–420.

Díaz, C., Ponti, M., Haikka, P., et al. (2020). More than data gatherers: Exploring player experience in a citizen science game . Qualitative User Experience, 5, 1.

If you’d like to cite content from the IxDF website , click the ‘cite this article’ button near the top of your screen.

Some recommended books on games user research and related topics include:

Bromley, S. (2021). How to be a games user researcher .

This book provides deep insights into contemporary topics important for games user researchers and UX designers. Steve Bromley covers a wide range of approaches and challenges in the industry. The book teaches how to develop essential skills like moderating, biometrics, and analytics.

Drachen, A., Mirza-Babaei, P., & Nacke, L. E. (2018). Games user research . Oxford University Press.

This book is essential for anyone working with players and games or other interactive entertainment products. Whether you are new to Games User Research or have plenty of experience, this book is suitable.

Hodent, C. (2018). The Gamer's Brain: How neuroscience and UX can impact video game design . CRC Press.

Celia Hodent provides an overview of cognitive psychology and neuroscience research and how to apply it to video game design. It covers attention, perception , memory , motivation , and emotion and provides practical advice on designing more engaging and enjoyable games for players.

Playtesting, a crucial practice in game design, offers researchers insights into real users' interactions in a game. You should follow these best practices when you conduct playtesting:

Define Objectives: Determine your playtest learning goals.

Choose Participants: Select a representative target audience.

Prepare Materials: Ensure a testable game/product state.

Create Environment: Set a conducive play/observation space.

Observe and Note: Watch and record participant interactions.

Ask Questions: Encourage open feedback from participants.

Analyze Feedback: Review and apply session insights.

Iterate, Repeat: Continue to improve via multiple tests.

Researchers will often use interviews to gain insights from users after playtesting. Ann Blandford, Professor of Human-Computer Interaction at University College London, explains the pros and cons of user interviews in this video:

Read our article How to Conduct User Interviews to learn more.

In games user research, researchers employ various tools to understand player behavior, preferences, and experiences. These tools include:

Analytics Software: Track in-game player behavior and metrics.

Survey Tools: Gather feedback on game design via surveys.

Game Testing Platforms: Collect player data using specialist software.

Eye Tracking Technology: Track player gaze for insight into attention focus.

Physiological Measurement Tools: Measure heart rate, brain activity, and signals for emotional and physical engagement insights.

Heat Maps: Discover a game's most interacted areas using user data.

Audio and Video Recording Tools: Capture player reactions during playtesting sessions.

William Hudson, User Experience Strategist, teaches you how to ensure you collect good quality data from your research:

Read our Ultimate Guide to learn how, when, and why to use surveys.

Researchers employ playtesting, surveys, and behavior observation in games user research to identify accessibility barriers, like issues faced by players with visual or hearing impairments.

They use these insights to develop game accessibility features that benefit all players. Features include settings that can be customized, such as text size and control schemes.  

You should integrate accessibility early in game development, involve players with disabilities, and iterate based on their feedback.

Accessibility is a key consideration in inclusive design. Inclusive design is an approach to creating accessible products and experiences that are usable and understandable by as many people as possible. Katrin Suetterlin, UX Content Strategist, provides an overview of inclusive design:

Learn more in our course Accessibility: How to Design for All .

Recruitment for games user research studies is a crucial task. Researchers identify and engage with individuals who represent their target audience. To successfully recruit participants, you should follow these steps:

Define the Target Audience: Age, experience, preferences.

Recruitment Appropriate Users: Tap into social media, forums, and your existing user base.

Encourage Participation: Offer game credits, merchandise, and money.

Create a Concise Survey: Screen participants efficiently.

Ensure Ethical Practices: Transparency, data use, and confidentiality.

Plan for Dropouts: Over-recruit for reliability.

Maintain Communication: Inform participants about progress.

Seek Feedback: Improve future recruitment efforts.

William Hudson, User Experience Strategist, offers practical tips to ensure the effectiveness and efficiency of your recruitment process:

Learn more about qualitative user research in User Research – Methods and Best Practices .

Enroll in Data-Driven Design: Quantitative Research for UX for more on quantitative user research methods.

Games user researchers must prioritize ethical considerations to ensure respect for players. They must maintain integrity in research practices and responsibly use data. Key ethical considerations include:

Consent and Privacy: Obtain and respect player consent and privacy.

Transparency and Honesty: Ensure openness and honesty in research practices.

Data Security: Safeguard data confidentiality and security rigorously.

Avoid Bias: Strive to eliminate biases in research.

Minimize Harm: Design research to minimize participant harm.

Beneficence and Respect: Respect participant autonomy, dignity, and benefits.

Some games feature violence and other potentially distressing elements. Ann Blandford, Professor of Human-Computer Interaction at University College London, discusses how she asks research participants about emotionally charged and critical incidents:

To learn more, watch the Master Class Webinar Ethics In Design: A Practical Guide from Guthrie Weinschenk, COO of The Team W, Inc.

Games user research (GUR) uses crucial insights from psychology about player behaviors and motivations. Researchers use these insights to enhance gaming experiences.

Researchers focus on player psychology in GUR to improve game design and engagement. They understand player motivation through theories like flow theory, a state of mind where a person is completely absorbed and focused in an activity.

You can use psychology to create intuitive interfaces and positive game mechanisms. To do this, you must study players’ behavior and how they learn.

GUR includes usability testing, where researchers analyze player interactions to refine game mechanics and improve the player experience.

Psychology in GUR leads to more engaging, rewarding games, offering players more profound, meaningful experiences.

Alan Dix, Professor and Expert in Human-Computer Interaction, teaches you how psychological principles can enhance player engagement:

Celia Hodent, Ph.D., Game UX Strategist and author of The Gamer's Brain, teaches the cognitive science and psychology that support developing engaging video games in her Master Class, How to Design Engaging Products: Insights from Fortnite's UX .

A researcher must understand that game design user personas represent various player types. These personas focus design efforts on a relatable character set and avoid the complexity of endless player possibilities.

To develop user personas: 

Gather user data through surveys for rich insights.

Analyze data to form main player archetypes.

Create detailed, engaging personas for each archetype.

Define goals and scenarios for each persona.

Reference personas in shaping game design elements.

Validate design choices and refine them based on feedback.

Frank Spillers, CEO of Experience Dynamics, teaches you how personas can guide design decisions:

Learn How To Create Actionable Personas in this Master Class from Daniel Rosenberg, UX Professor, Designer, Executive and Early Innovator in HCI.

Career paths in games user research (GUR) are diverse and dynamic. A career in GUR will offer you exciting opportunities if you are passionate about combining the science of user experience with the art of game design. Key career paths in GUR include:

User Research Analysts conduct user research and analyze player behavior and feedback. They provide insights to improve game design and player experience.

User Experience (UX) Designers for Games focus on designing user interfaces and creating seamless user experiences. They ensure the game is intuitive and enjoyable.

Game Data Analysts analyze in-game data to understand player behavior, preferences, and trends. This analysis informs game development and marketing strategies.

Playtest Coordinators organize playtesting sessions, gather feedback, and ensure the game meets user expectations.

Accessibility Specialists ensure that people with disabilities can play video games. They help to make game design more inclusive.

Learn more about accessibility in Elena’s Master Class Introduction to Digital Accessibility .

Developers employ games user research (GUR) to enhance gaming experiences when developing virtual reality (VR) games. Researchers use various methods like playtesting and biometric analysis to understand player interactions in VR.

Researchers observe player behavior and reactions in VR playtesting to identify usability issues and engagement levels. They use surveys, interviews, and biometric tools like eye tracking. These qualitative research methods help us understand player emotions.

Researchers apply GUR methods in VR to make informed decisions that improve user interfaces and game immersion. Insights from playtesting and biometric analysis help us to:

Refine game mechanics.

Enhance player emotional engagement.

In this video, researcher and professor Mel Slater explores design considerations for VR. He provides valuable context for applying games user research in VR game development:

Enroll in UX Design for Virtual Reality to develop your understanding of UX in VR.

You don't need a PhD to become a user researcher. Practical skills and experience are often more valued than a PhD.

User researchers use diverse methods to understand behaviors, needs, and motivations. Essential skills include empathy, observation, critical thinking, and communication.

Ann Blandford, Professor of Human-Computer Interaction at University College London, offers valuable insights into effective user research techniques:

To learn more about being a user researcher, read our 15 Guiding Principles .

To become a UX researcher with no experience, you should focus on UX research fundamentals. Learn UX design principles, user-centered processes, and research methodologies to understand user needs and behaviors.

Engage in self-learning with courses and materials on:

UX research methods.

Empathy skills.

Critical thinking skills. 

Development in these areas is vital in UX research.

Volunteer, intern, or carry out research projects to gain practical experience. A portfolio displaying your projects and findings is vital if you aspire to be a UX researcher.

Networking is crucial for entering the UX field. Join communities, attend events, and seek mentorship from experienced UX researchers for industry insights and career advice.

A perfect place to start is the IxDF course, User Research – Methods and Best Practices .

Answer a Short Quiz to Earn a Gift

What is the main goal of Games User Research (GUR)?

• To create marketing strategies for games and understand gaming trends
• To design game graphics and create game story plots
• To understand player behavior and improve the gaming experience

How is games user research different from traditional user research?

• It does not use qualitative methods with users.
• It evaluates player experiences against the game designer’s vision rather than just usability.
• It focuses only on aesthetic and story aspects of the game.

Why is playtesting important in games user research?

• It creates game trailers from the gamer's point of view.
• It helps identify and fix problems from a player's perspective.
• It is the final step before game release.

What distinguishes intentional difficulty from accidental difficulty in games?

• Accidental difficulty improves the gaming experience, while intentional difficulty makes each game level impossible to beat.
• Intentional difficulty challenges players, while accidental difficulty arises from bugs or poor design.
• Intentional difficulty is unplanned and disruptive.

Why are ethical considerations important in games user research?

• To improve game graphics
• To guarantee respectful treatment of participants and maintain their trust
• To speed up the development process for an earlier launch date

Better luck next time!

Do you want to improve your UX / UI Design skills? Join us now

Congratulations! You did amazing

You earned your gift with a perfect score! Let us send it to you.

Check Your Inbox

We’ve emailed your gift to [email protected] .

Literature on Games User Research

Here’s the entire UX literature on Games User Research by the Interaction Design Foundation, collated in one place:

Learn more about Games User Research

Take a deep dive into Games User Research with our course User Research – Methods and Best Practices .

How do you plan to design a product or service that your users will love , if you don't know what they want in the first place? As a user experience designer, you shouldn't leave it to chance to design something outstanding; you should make the effort to understand your users and build on that knowledge from the outset. User research is the way to do this, and it can therefore be thought of as the largest part of user experience design .

In fact, user research is often the first step of a UX design process—after all, you cannot begin to design a product or service without first understanding what your users want! As you gain the skills required, and learn about the best practices in user research, you’ll get first-hand knowledge of your users and be able to design the optimal product—one that’s truly relevant for your users and, subsequently, outperforms your competitors’ .

This course will give you insights into the most essential qualitative research methods around and will teach you how to put them into practice in your design work. You’ll also have the opportunity to embark on three practical projects where you can apply what you’ve learned to carry out user research in the real world . You’ll learn details about how to plan user research projects and fit them into your own work processes in a way that maximizes the impact your research can have on your designs. On top of that, you’ll gain practice with different methods that will help you analyze the results of your research and communicate your findings to your clients and stakeholders—workshops, user journeys and personas, just to name a few!

By the end of the course, you’ll have not only a Course Certificate but also three case studies to add to your portfolio. And remember, a portfolio with engaging case studies is invaluable if you are looking to break into a career in UX design or user research!

We believe you should learn from the best, so we’ve gathered a team of experts to help teach this course alongside our own course instructors. That means you’ll meet a new instructor in each of the lessons on research methods who is an expert in their field—we hope you enjoy what they have in store for you!

All open-source articles on Games User Research

The 'rules of play': directing gamer and user behaviour.

• 8 years ago

Open Access—Link to us!

We believe in Open Access and the  democratization of knowledge . Unfortunately, world-class educational materials such as this page are normally hidden behind paywalls or in expensive textbooks.

If you want this to change , cite this page , link to us, or join us to help us democratize design knowledge !

Privacy Settings

Our digital services use necessary tracking technologies, including third-party cookies, for security, functionality, and to uphold user rights. Optional cookies offer enhanced features, and analytics.

Experience the full potential of our site that remembers your preferences and supports secure sign-in.

Governs the storage of data necessary for maintaining website security, user authentication, and fraud prevention mechanisms.

Enhanced Functionality

Saves your settings and preferences, like your location, for a more personalized experience.

Referral Program

We use cookies to enable our referral program, giving you and your friends discounts.

Error Reporting

We share user ID with Bugsnag and NewRelic to help us track errors and fix issues.

Optimize your experience by allowing us to monitor site usage. You’ll enjoy a smoother, more personalized journey without compromising your privacy.

Analytics Storage

Collects anonymous data on how you navigate and interact, helping us make informed improvements.

Differentiates real visitors from automated bots, ensuring accurate usage data and improving your website experience.

Lets us tailor your digital ads to match your interests, making them more relevant and useful to you.

Advertising Storage

Stores information for better-targeted advertising, enhancing your online ad experience.

Personalization Storage

Permits storing data to personalize content and ads across Google services based on user behavior, enhancing overall user experience.

Advertising Personalization

Allows for content and ad personalization across Google services based on user behavior. This consent enhances user experiences.

Enables personalizing ads based on user data and interactions, allowing for more relevant advertising experiences across Google services.

Receive more relevant advertisements by sharing your interests and behavior with our trusted advertising partners.

Enables better ad targeting and measurement on Meta platforms, making ads you see more relevant.

Allows for improved ad effectiveness and measurement through Meta’s Conversions API, ensuring privacy-compliant data sharing.

LinkedIn Insights

Tracks conversions, retargeting, and web analytics for LinkedIn ad campaigns, enhancing ad relevance and performance.

LinkedIn CAPI

Enhances LinkedIn advertising through server-side event tracking, offering more accurate measurement and personalization.

Google Ads Tag

Tracks ad performance and user engagement, helping deliver ads that are most useful to you.

Share Knowledge, Get Respect!

or copy link

Cite according to academic standards

Simply copy and paste the text below into your bibliographic reference list, onto your blog, or anywhere else. You can also just hyperlink to this page.

New to UX Design? We’re Giving You a Free ebook!

Download our free ebook The Basics of User Experience Design to learn about core concepts of UX design.

In 9 chapters, we’ll cover: conducting user interviews, design thinking, interaction design, mobile UX design, usability, UX research, and many more!

100 research rules of the game: How to make your research world class; how to successfully publish in top international refereed journals

Accounting, Auditing & Accountability Journal

ISSN : 0951-3574

Article publication date: 28 May 2019

Issue publication date: 28 May 2019

There are several basic, and at times minor, pedantic principles required to successfully publish in good-quality international peer-reviewed journals. These are what the author calls the “rules of the game”. Many are so basic, so taken-for-granted, tacit knowledge, that at times supervisors do not tell their students about them. The paper aims to discuss this issue.

Design/methodology/approach

The author has assembled 100 research rules of the game from her work over many years with doctoral students and early career researchers. Each rule is accompanied by short advice. Additional citations are included directing readers to further resources on the 100 research rules of the game.

The paper documents 100 research rules of the game.

Research limitations/implications

There are many other rules of the game not included in the author’s list of 100 research rules of the game.

Originality/value

This paper is a one-stop-shop brief introduction to the author’s 100 research rules of the game.

• Rules of the game
• Publishing research
• Writing research

Brennan, N.M. (2019), "100 research rules of the game: How to make your research world class; how to successfully publish in top international refereed journals", Accounting, Auditing & Accountability Journal , Vol. 32 No. 2, pp. 691-706. https://doi.org/10.1108/AAAJ-02-2019-032

Emerald Publishing Limited

Copyright © 2019, Niamh M. Brennan

Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial & non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode

Introduction to, and philosophy behind, the 100 research rules of the game

It is important for doctoral students, early career researchers and even more senior colleagues to know the basic, and at times minor, pedantic principles required to successfully publish research – what I call the “rules of the game [1] , [2] ”. These basic principles are ones I have learned and am still learning from colleagues and through trial and (a lot of) error. I share these rules with the intention of helping those at an earlier stage in their career to learn the principles which are often so taken for granted that they remain unspoken, tacit knowledge.

As a supervisor, I make sure all my masters and doctoral students know the rules of the game, but not all supervisors appear to do the same (if they know the rules of the game themselves, which is an assumption not always merited). These 100 research rules of the game complement Brennan’s (2019) “100 PhD rules of the game”, written for doctoral students.

These notes are a one-stop-shop resource for researchers. They also reflect some of my pet bugbears. Brennan (1998) is a precursor to this short article, written for my Master of Accounting students, providing guidance on how to write a masters dissertation.

Each rule is accompanied by a brief description/some advice and citations to support the rule. Most of the papers cited are short articles on a research rule-of-the-game topic.

I have organised the 100 rules into six sections. They start with some high-level general advice on conducting research and being a researcher (11 rules). Then I deal with some principles in designing the research (20 rules). It is not enough to conduct excellent research. Authors must sell their work by writing in a convincing and compelling manner. The quality of the writing is more than half the battle in successful publishing. Section 3 therefore has the largest number of rules (39 rules). Section 4 covers conferencing research (six rules), as a precursor to getting research published (Section 5, 17 rules). The challenging issue of co-authoring is covered in Section 6 (six rules). The rules conclude with some repetition, the most important rule of all (one rule) ( Table I ).

It is not enough to read and know these rules. They must be applied in practice, which is a lifelong learning process. As Kavanagh and Scally (2018 , pp. 8-9) observe, “games are epistemologically beyond the compass of lists and definitions and can only be properly known through playing. Games are phenomenological practices as they emotionally engage players, giving them a meaningful experience and opportunity to express themselves”. A checklist ( Appendix ) helps the authors self-assess their own work for application of the rules. Make sure you can answer “yes” to (almost) all the rules (as appropriate) before you submit your manuscript for review.

This is a high-level introduction/summary. For each rule, there are swathes of additional resources available to obtain greater depth of understanding of each rule.

My list of 100 rules is not exhaustive. For example, research integrity is critical, but is only touched on in these rules. These rules do not guarantee success in the world of academic international peer-review publishing. If properly used, they should ensure the research meets some basic requirements for top-quality publishing.

Bamber (2016) , ter Bogt (2014) , Buckby (2013) , Cortese (2009) , L’Huillier (2012, 2014) and Parker (2012, 2015) are amusing takes on the rules of the game in academic life.

I have written this paper in the hope that some or all of it may prove to be a game changer for readers.

100 research rules of the game

Notes: a I have adapted this phrase, thanks to Gardiner and Kearns (2011) ; b I thank Yiannis Gabriel for this rule; c I thank Steve Evans for this rule; d I thank Elizabeth Morton for this rule; e I thank Gus de Franco for this phrase

I use the phrase “rules of the game” tongue-in-cheek, capturing theoretical physicist Edward Teller’s sentiment that (pure) research “is a game, is play, led by curiosity, by taste, style, judgment, intangibles” (cited in Reagan, 1967 , p. 1383). Kalfa et al . (2018) have a darker take on playing the game in academia.

Further resources complementing this paper are available at: www.niamhbrennan.ie and @100RulesoftheGame

Appendix. Self-assessment checklist

Agarwal , R. , Echambadi , R. , Franco , A.M. and Sarkar , M.B. ( 2006 ), “ Reap rewards: maximizing benefits from reviewer comments ”, The Academy of Management Journal , Vol. 49 No. 2 , pp. 191 - 196 .

Alvesson , M. and Sandberg , J. ( 2011 ), “ Generating research questions through problematization ”, Academy of Management Review , Vol. 36 No. 2 , pp. 247 - 271 .

Anglim , J. ( 2013 ), “ Writing a concise introduction to a psychology journal article: an article deconstruction ”, available at: http://jeromyanglim.blogspot.com/2013/11/writing-principles-forconcise.html (accessed 11 July 2018 ).

Ball , R. and Brown , P. ( 1968 ), “ An empirical evaluation of accounting income numbers ”, Journal of Accounting Research , Vol. 6 No. 2 , pp. 159 - 178 .

Ball , R. and Brown , P.R. ( 2014 ), “ Ball and Brown (1968): a retrospective ”, The Accounting Review , Vol. 89 No. 1 , pp. 1 - 26 .

Bamber , M. ( 2016 ), “ Would you kindly reject me? ”, Accounting, Auditing & Accountability Journal , Vol. 29 No. 1 , available at: https://doi.org/10.1108/AAAJ-01-2015-1949

Bansal , T. and Corley , K. ( 2011 ), “ From the editors: the coming of age for qualitative research: embracing the diversity of qualitative methods ”, The Academy of Management Journal , Vol. 54 No. 2 , pp. 233 - 237 .

Bartunek , J.M. , Rynes , S.L. and Ireland , R.D. ( 2006 ), “ What makes management research interesting, and why does it matter? ”, Academy of Management Journal , Vol. 49 No. 1 , pp. 9 - 15 .

Bealls , J. ( 2018 ), “ Bealls’ list of predatory journals and publishers ”, available at: https://beallslist.weebly.com/ (accessed 17 July 2018 ).

Belcher , W.L. ( 2014 ), Writing Your Journal Article in 12 Weeks , Sage , Los Angeles, CA .

Bem , D.J. ( 1995 ), “ Writing a review article for psychological bulletin ”, Psychological Bulletin , Vol. 118 No. 2 , pp. 172 - 177 .

Bem , D.J. ( 2003 ), “ Writing the empirical journal article ”, in Darley , J.M. , Zanna , M.P. and Roediger , H.L. III (Eds), The Compleat Academic: A Career Guide , American Psychological Association , Washington, DC , pp. 171 - 201 .

Bergh , D. ( 2002 ), “ From the editors: deriving greater benefit from the reviewing process ”, Academy of Management Journal , Vol. 45 No. 4 , pp. 633 - 636 .

Bergh , D.D. ( 2003 ), “ Thinking strategically about contribution ”, Academy of Management Journal , Vol. 46 No. 2 , pp. 135 - 136 .

Brennan , N. ( 1998 ), Accounting Research: A Practical Guide , Oak Tree Press , Dublin , available at: https://researchrepository.ucd.ie/bitstream/handle/10197/2924/01_07%20Brennan%20Accounting%20Research%20A%20Practical%20Guide.pdf (accessed 6 June 2018 ).

Brennan , N.M. ( 2019 ), “ 100 PhD rules of the game to successfully complete a doctoral dissertation ”, Accounting, Auditing & Accountability Journal , Vol. 32 No. 1 , pp. 364 - 376 .

Buckby , S. ( 2013 ), “ ‘The accounting conference’ ”, Accounting, Auditing & Accountability Journal , Vol. 26 No. 3 , pp. 498 - 499 .

Carpenter , M.A. ( 2009 ), “ Editor’s comments: mentoring colleagues in the craft and spirit of peer review ”, The Academy of Management Review , Vol. 34 No. 2 , pp. 191 - 195 .

Colquitt , J.A. ( 2013 ), “ Crafting references in AMJ submissions ”, The Academy of Management Review , Vol. 56 No. 5 , pp. 1221 - 1224 .

Colquitt , J.A. and George , G. ( 2011 ), “ Publishing in AMJ – part 1: topic choice ”, Vol. 54 No. 3 , pp. 432 - 435 .

Colquitt , J.A. and Ireland , R.D. ( 2009 ), “ Taking the mystery out of AMJ’s reviewer evaluation form ”, Academy of Management Journal , Vol. 52 No. 2 , pp. 224 - 228 .

Colquitt , J.A. and Zapata-Phelan , C.P. ( 2007 ), “ Trends in theory building and theory testing: a five-decade study of the Academy of Management Journal ”, Academy of Management Journal , Vol. 50 No. 6 , pp. 1281 - 1303 .

Corley , K.G. and Gioia , D.A. ( 2011 ), “ Building theory about theory building: what constitutes a theoretical contribution? ”, Academy of Management Review , Vol. 36 No. 1 , pp. 12 - 32 .

Cortese , C. ( 2009 ), “ Reflections of a new academic ”, Accounting Auditing & Accountability Journal , Vol. 22 No. 2 , pp. 300 - 301 .

Craig , J.B. ( 2010 ), “ Desk rejection: how to avoid being hit by a returning boomerang ”, Family Business Review , Vol. 23 No. 4 , pp. 306 - 309 .

Daft , R.L. ( 1985 ), “ Why I recommended that your manuscript be rejected and what you can do about it ”, in Cummings , L.L. and Frost , P. (Eds), Publishing in the Organizational Sciences , Irwin , Homewood, Ill. , pp. 200 - 209 .

Davis , D. ( 2001 ), “ PhD thesis research: where do I start? ”, working paper, Columbia University, available at: www.columbia.edu/~drd28/Thesis%20Research.pdf (accessed 14 June 2018 ).

Davis , M.S. ( 1971 ), “ That’s interesting! Towards a phenomenology of sociology and a sociology of phenomenology ”, Philosophy of the Social Sciences , Vol. 1 No. 2 , pp. 309 - 344 .

Echambadi , R. , Campbell , B. and Agarwal , R. ( 2006 ), “ Encouraging best practice in quantitative management research: an incomplete list of opportunities ”, Journal of Management Studies , Vol. 43 No. 8 , pp. 1801 - 1826 .

Eden , D. and Rynes , S. ( 2003 ), “ From the editors: publishing across borders: furthering the internationalization of AMJ ”, Academy of Management Journal , Vol. 46 No. 6 , pp. 679 - 683 .

Edmondson , A.C. and McManus , S.E. ( 2007 ), “ Methodological fit in management field research ”, Academy of Management Review , Vol. 32 No. 4 , pp. 1246 - 1264 .

Evans , J.H. , Feng , M. , Hoffman , V.B. , Moser , D.V. and Stede , W.A. ( 2015 ), “ Points to consider when self-assessing your empirical accounting research ”, Contemporary Accounting Research , Vol. 32 No. 3 , pp. 1162 - 1192 .

Faff , R.W. ( 2015 ), “ A simple template for pitching research ”, Accounting & Finance , Vol. 55 No. 2 , pp. 311 - 336 .

Feldman , D.C. ( 2004a ), “ The devil is in the details: converting good research into publishable articles ”, Journal of Management , Vol. 30 No. 1 , pp. 1 - 6 .

Feldman , D.C. ( 2004b ), “ What are we talking about when we talk about theory? ”, Journal of Management , Vol. 30 No. 5 , pp. 565 - 567 .

Gardiner , M. and Kearns , H. ( 2010 ), Turbocharge Your Writing. How to Become a Prolific Academic Writer , Thinkwell , Glenelg North .

Gardiner , M. and Kearns , H. ( 2011 ), “ Turbocharge your writing today ”, Nature , Vol. 475 No. 7354 , pp. 129 - 130 .

Gardiner , M. and Kearns , H. ( 2018 ), “ Free guides and planners ”, available at: www.ithinkwell.com.au/resources (accessed 22 June 2018 ).

George , G. ( 2012 ), “ Publishing in AMJ for non-US authors ”, Academy of Management Journal , Vol. 55 No. 5 , pp. 1023 - 1026 .

Graff , G. , Birkenstein , C. and Durst , R. ( 2006 ), “They Say, I Say”: The Moves that Matter in Academic Writing , 2nd ed. , W.W. Norton & Company , New York, NY .

Grant , A.M. and Pollock , T.G. ( 2011 ), “ Publishing in AMJ : setting the hook ”, Academy of Management Journal , Vol. 54 No. 5 , pp. 873 - 879 .

Grey , C. and Sinclair , A. ( 2006 ), “ Writing differently ”, Organization , Vol. 13 No. 3 , pp. 443 - 453 .

Harrison , D. ( 2002 ), “ From the editors: obligations and obfuscations in the review process ”, Academy of Management Journal , Vol. 45 No. 6 , pp. 1079 - 1084 .

Harzing , A.-W. ( 2018 ), “ Publish or perish ”, available at: https://harzing.com/resources/publish-or-perish (accessed 10 July 2018 ).

Healy , K. ( 2018 ), “ The plain person’s guide to plain text social science ”, available at: https://kieranhealy.org/publications/plain-person-text/ (accessed 5 August 2018 ).

Hempel , P.S. ( 2014 ), “ The developmental reviewer ”, Management and Organization Review , Vol. 10 No. 2 , pp. 175 - 181 .

Ireland , R.D. ( 2009 ), “ From the editors: when is a ‘new’ paper really new? ”, Academy of Management Journal , Vol. 52 No. 1 , pp. 9 - 10 .

Kalfa , S. , Wilkinson , A. and Golan , P.J. ( 2018 ), “ The academic game: compliance and resistance in universities ”, Work, Employment and Society , in press .

Kavanagh , D. and Scally , K. ( 2018 ), “ What is theory if theorizing is a game? ”, working paper, University College Dublin .

Kinney , W.R. Jr ( 1986 ), “ Empirical accounting research design for PhD students ”, Accounting Review , Vol. 61 No. 2 , pp. 338 - 350 .

Klag , M. and Langley , A. ( 2013 ), “ Approaching the conceptual leap in qualitative research ”, International Journal of Management Reviews , Vol. 15 No. 2 , pp. 149 - 166 .

Köhler , T. ( 2016 ), “ From the editors: on writing up qualitative research in management learning and education ”, Academy of Management Learning and Education , Vol. 15 No. 3 , pp. 400 - 418 .

Koopman , P. ( 1997 ), How to Write an Abstract , Carnegie Mellon University , available at: https://users.ece.cmu.edu/~koopman/essays/abstract.html (accessed 6 July 2018 ).

Lebo , M.J. ( 2016 ), “ Managing your research pipeline ”, PS: Political Science & Politics , Vol. 49 No. 2 , pp. 259 - 264 .

L’Huillier , B. ( 2012 ), “ Publish or perish ”, Accounting Auditing & Accountability Journal , Vol. 25 No. 6 , pp. 1071 - 1071 .

L’Huillier , B.M. ( 2014 ), “ The conference ”, Accounting, Auditing & Accountability Journal , Vol. 27 No. 4 , available at: https://doi.org/10.1108/AAAJ-12-2013-1556

Locke , K. and Golden-Biddle , K. ( 1997 ), “ Constructing opportunities for contribution: structuring intertextual coherence and ‘problematizing’ in organizational studies ”, Academy of Management Journal , Vol. 40 No. 5 , pp. 1023 - 1062 .

Mayer , K.J. and Sparrowe , R.T. ( 2013 ), “ Integrating theories in AMJ articles ”, Academy of Management Journal , Vol. 56 No. 3 , pp. 629 - 634 .

Mensh , B. and Kording , K. ( 2017 ), “ Ten simple rules for structuring papers ”, PLoS Computational Biology , Vol. 13 No. 9 .

Michelon , G. ( 2018 ), “ What reviewers (don’t) like ”, available at: www.arc-eaa.com/blog/what-reviewers-don’t (accessed 10 July 218 ).

Morley , J. ( 2018 ), Academic Phrasebank , University of Manchester , Manchester , available at: www.phrasebank.manchester.ac.uk/ (accessed 24 July 2018 ).

Mudrack , B. ( 2015 ), “ Writing a good cover letter. The goal of a good cover letter ”, available at: www.aje.com/en/arc/dist/docs/Writing-a-cover-letter-AJE-2015.pdf (accessed 11 July 2018 ).

Oxford Research Encyclopedias ( 2013 ), Author Instructions , Oxford University Press , New York, NY , available at: http://politics.oxfordre.com/fileasset/images/OREPOL/Author%20Instructions%20-%20Preparing%20Your%20Manuscript%20and%20Quick%20Guide.pdf (accessed 17 July 2018 ).

Parker , L.D. ( 2012 ), “ The conference paper ”, Accounting, Auditing & Accountability Journal , Vol. 25 No. 1 , p. 204 .

Parker , L.D. ( 2015 ), “ Conference session poet ”, Accounting, Auditing & Accountability Journal , Vol. 28 No. 7 , available at: https://doi.org/10.1108/AAAJ-06-2014-1723

Parker , L.D. and Northcott , D. ( 2016 ), “ Qualitative generalising in accounting research: concepts and strategies ”, Accounting, Auditing & Accountability Journal , Vol. 29 No. 6 , pp. 1100 - 1131 .

Pollock , T.G. and Bono , J.E. ( 2013 ), “ Being scheherazade: the importance of storytelling in academic writing ”, Academy of Management Journal , Vol. 56 No. 3 , pp. 629 - 634 .

Pratt , M.G. ( 2009 ), “ From the editors: for the lack of a boilerplate: tips on writing up (and reviewing) qualitative research ”, Academy of Management Journal , Vol. 52 No. 5 , pp. 856 - 862 .

Ragins , B. ( 2012 ), “ Reflections on the craft of clear writing ”, Academy of Management Review , Vol. 37 No. 4 , pp. 493 - 501 .

Reagan , M.D. ( 1967 ), “ Basic and applied research: a meaningful distinction? ”, Science , Vol. 155 No. 3768 , pp. 1383 - 1386 .

Reuber , A.R. and Sharma , P. ( 2013 ), “ The anatomy of a paper ”, Family Business Review , Vol. 26 No. 2 , pp. 113 - 120 .

Rougier , N.P. , Droettboom , M. and Bourne , P.E. ( 2014 ), “ Ten simple rules for better figures ”, PLoS Computational Biology , Vol. 10 No. 9 .

Rynes , S. ( 2002 ), “ Some reflections on contribution ”, Academy of Management Journal , Vol. 45 No. 2 , pp. 311 - 313 .

Rynes , S. and Gephart , R.P.Jr ( 2004 ), “ From the editors: qualitative research and the ‘ Academy of Management Journal ’ ”, The Academy of Management Journal , Vol. 47 No. 4 , pp. 454 - 462 .

Rynes , S.L. ( 2006a ), “ Academy of Management Journal editors’ forum on the review process: making the most of the review process – lessons learned from award-winning authors ”, Academy of Management Journal , Vol. 49 No. 2 , pp. 189 - 190 .

Rynes , S.L. ( 2006b ), “ Observations on ‘anatomy of an R&R’ and other reflections ”, Academy of Management Journal , Vol. 49 No. 2 , pp. 208 - 214 .

Sandberg , J. and Alvesson , M. ( 2011 ), “ Ways of constructing research questions: gap-spotting or problematization? ”, Organization , Vol. 18 No. 1 , pp. 23 - 44 .

Seibert , S.E. ( 2006 ), “ Anatomy of an R&R (or, reviewers are an author’s best friends…) ”, The Academy of Management Journal , Vol. 49 No. 2 , pp. 203 - 207 .

Shaw , J.D. ( 2012 ), “ Responding to reviewers ”, The Academy of Management Journal , Vol. 55 No. 6 , pp. 1261 - 1263 .

Stolowy , H. ( 2017 ), “ Letter from the editor: why are papers desk rejected at European Accounting Review ? ”, European Accounting Review , Vol. 26 No. 3 , pp. 411 - 418 .

Stolowy , H. ( 2018 ), “ The importance of the cover letter ”, European Accounting Association Newsletter, March, available at: https://gallery.mailchimp.com/ffab1869b934ebf881c40649e/files/9f14def8-9912-4e66-ad7f-75c456e9d80c/2018_03_06_EAR_letter_of_editor_v2.pdf (accessed 10 July 2018 ).

Strunk , W. Jr and White , E.B. ( 2000 ), The Elements of Style , 4th ed. , Allyn and Bacon , Boston, MA .

Sutton , R.I. and Staw , B.M. ( 1995 ), “ What theory is not ”, Administrative Science Quarterly , Vol. 40 No. 3 , pp. 371 - 384 .

Sword , H. ( 2012 ), Stylish Academic Writing , Harvard University Press , Boston, MA .

ter Bogt , H.J. ( 2014 ), “ Academic accounting work ”, Accounting, Auditing & Accountability Journal , Vol. 27 No. 5 , available at: https://doi.org/10.1108/AAAJ-03-2014-1631

The British Council ( 2010 ), “ Poem: eye halve a spelling checker ”, available at: https://learnenglish.britishcouncil.org/sites/podcasts/files/Poem-eye-halve-a-spelling-checker.pdf (accessed 11 July 2018 ).

Tucker , B.P. , Parker , L.D. and Merchant , K.A. ( 2016 ), “ With a little help from our friends: an empirical investigation of co-authoring in accounting research ”, The British Accounting Review , Vol. 48 No. 2 , pp. 185 - 205 .

Weick , K.E. ( 1995 ), “ What theory is not, theorizing is ”, Administrative Science Quarterly , Vol. 40 No. 3 , pp. 385 - 390 .

Whetten , D.A. ( 1989 ), “ What constitutes a theoretical contribution? ”, Academy of Management Review , Vol. 14 No. 4 , pp. 490 - 495 .

Wiens , K. ( 2012 ), “ I won’t hire people who use poor grammar. Here’s why ”, available at: http://blogs.hbr.org/2012/07/i–-wont–-hire–-people–-who–-use–-poo/ (accessed 1 June 2018 ).

Acknowledgements

Publisher’s note: The Publisher would like to acknowledge the amendment to the copyright license for the article “100 research rules of the game: How to make your research world class; how to successfully publish in top international refereed journals” by Niamh Brennan published in the Accounting, Auditing & Accountability Journal .

Corresponding author

Related articles, all feedback is valuable.

Please share your general feedback

Report an issue or find answers to frequently asked questions

Contact Customer Support

Designing and integrating purposeful learning in game play: a systematic review

• Research Article
• Published: 29 December 2015
• Volume 64 , pages 219–244, ( 2016 )

Cite this article

• Fengfeng Ke 1  

7802 Accesses

166 Citations

4 Altmetric

Explore all metrics

Via a systematic review of the literature on learning games, this article presents a systematic discussion on the design of intrinsic integration of domain-specific learning in game mechanics and game world design. A total of 69 articles ultimately met the inclusion criteria and were coded for the literature synthesis. Exemplary learning games cited in the articles reviewed and developed by credible institutions were also analyzed. The cumulative findings and propositions of the game-based learning-play integration have been extracted and synthesized into five salient themes to clarify what, how, where, and when learning and content are embedded in and activated by gameplay. These themes highlight: (a) the types of game-based learning action—prior-knowledge activation and novel-knowledge acquisition, (b) the modes in which learning actions are integrated in game actions—representation, simulation, and contextualization, (c) the blended learning spaces contrived by game mechanics and the game world, (d) the occurrence of meta-reflective and iterative learning moments during game play, and (e) the multifaceted in-game learning support (or scaffolding). Future directions for the design and research of learning integration in digital games are then proposed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or Ebook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

The Gamification of Learning: a Meta-analysis

Virtual reality and gamification in education: a systematic review

Gamification Applications in E-learning: A Literature Review

Aarseth, E. (2003). Playing research: Methodological approaches to game analysis. Paper presented at the Melbourne, Australia DAC conference, May 2003.

Adamo-Villani, N., & Wright, K. (2007). SMILE: An immersive learning game for deaf and hearing children. In ACM SIGGRAPH 2007 educators program (p. 17). New York: ACM.

Adams, D., & Clark, D. B. (2014). Integrating self-explanation functionality into a complex game environment: Keeping gaming in motion. Computers and Education, 73 , 149–159.

Article   Google Scholar  

Adams, D. M., Mayer, R. E., MacNamara, A., Koenig, A., & Wainess, R. (2012). Narrative games for learning: Testing the discovery and narrative hypotheses. Journal of Educational Psychology, 104 (1), 235–249. doi: 10.1037/a0025595 .

Aldrich, C. (2009). The complete guide to simulations and serious games: How the most valuable content will be created in the age beyond Gutenberg to Google . John Wiley & Sons.

Amory, A. (2007). Game object model version II: A theoretical framework for educational game development. Educational Technology Research and Development, 55 (1), 51–77.

Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., Raths, J., & Wittrock, M. C. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s Taxonomy of Educational Objectives . New York: Longman.

Andersen, E., Liu, Y. E., Snider, R., Szeto, R., & Popović, Z. (2011). Placing a value on aesthetics in online casual games. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1275–1278). New York: ACM.

Ang, C. S. (2006). Rules, gameplay, and narratives in video games. Simulation and Gaming, 37 (3), 306–325.

Barab, S. A., Gresalfi, M., & Ingram-Goble, A. (2010). Transformational play: Using games to position person, content, and context. Educational Researcher, 39 (7), 525–536.

Barab, S. A., Scott, B., Siyahhan, S., Goldstone, R., Ingram-Goble, A., Zuiker, S. J., et al. (2009). Transformational play as a curricular scaffold: Using videogames to support science education. Journal of Science Education and Technology, 18 (4, Special Issue: Emerging Technologies for Learning Science), 305–320.

Barab, S., Thomas, M., Dodge, T., Carteaux, R., & Tuzun, H. (2005). Making learning fun: Quest Atlantis, a game without guns. Educational Technology Research and Development, 53 (1), 86–108.

Blakesley, C. C. (2013). The role of narrative in the design of an educational game. Dissertation abstracts international section A: Humanities and social sciences.

Boud, D., Keogh, R., & Walker, D. (2013). Reflection: Turning experience into learning . New York: Routledge.

Google Scholar  

Boyan, A., & Sherry, J. L. (2011). The challenge in creating games for education: Aligning mental models with game models. Child Development Perspectives, 5 (2), 82–87.

Bruner, J. (1990). Acts of meaning . Cambridge: Harvard University Press.

Buchanan, K. (2007). Beyond attention-getters: Designing for deep engagement. Dissertation abstracts international section A: Humanities and social sciences.

Champion, E. (2008). Otherness of place: Game-based interaction and learning in virtual heritage projects. International Journal of Heritage Studies, 14 (3), 210–228.

Charsky, D., & Ressler, W. (2011). “Games are made for fun”: Lessons on the effects of concept maps in the classroom use of computer games. Computers & Education, 56 (3), 604–615.

Clark, D. B., Nelson, B. C., Chang, H. Y., Martinez-Garza, M., Slack, K., & D’Angelo, C. M. (2011). Exploring Newtonian mechanics in a conceptually-integrated digital game: Comparison of learning and affective outcomes for students in Taiwan and the United States. Computers and Education, 57 (3), 2178–2195.

Clark, D., Tanner-Smith, E., & Killingsworth, S. (2014). Digital games, design and learning: A systematic review and meta-analysis (executive summary) . Menlo Park: SRI International.

Cooper, S. (2014). A framework for scientific discovery through video games . New York: Morgan & Claypool Publishers.

Book   Google Scholar  

Cooper, S., Khatib, F., Treuille, A., Barbero, J., Lee, J., Beenen, M., et al. (2010). Predicting protein structures with a multiplayer online game. Nature, 466 (7307), 756–760.

Creswell, J. W. (1998). Qualitative inquiry and research design: Choosing among five traditions . Thousand Oaks: Sage.

Csikszentmihalyi, M. (1991). Flow: The psychology of optimal experience (Vol. 41). New York: HarperPerennial.

d Baker, R. S., Corbett, A. T., Roll, I., Koedinger, K. R., Aleven, V., & Cocea, M. (2013). Modeling and studying gaming the system with educational data mining. International handbook of metacognition and learning technologies (pp. 97–115). New York: Springer.

Chapter   Google Scholar  

D’Angelo Cynthia, N. N. (2010). Scaffolding vector representations for student learning inside a physics game. Dissertation abstracts international section A: Humanities and social sciences.

Deci, E. L., & Ryan, R. M. (2010). Self-determination . New York: Wiley.

Dede, C. (2005). Why design-based research in both important and difficult. Educational Technology, 45 (1), 5–8.

Dempsey, J. V., Haynes, L. L., Lucassen, B. A., & Casey, M. S. (2002). Forty simple computer games and what they could mean to educators. Simulation and Gaming, 33 (2), 157.

Denner, J., Werner, L., & Ortiz, E. (2012). Computer games created by middle school girls: Can they be used to measure understanding of computer science concepts? Computers and Education, 58 (1), 240–249.

DeShazo, J., Harris, L., Turner, A., & Pratt, W. (2010). Designing and remotely testing mobile diabetes video games. Journal of Telemedicine and Telecare, 16 (7), 378–382.

DeVries, D. L., & Edwards, K. J. (1973). Learning games and student teams: Their effects on classroom process. American Educational Research Journal, 10 (4), 307–318.

Dickey, M. D. (2005). Engaging by design: How engagement strategies in popular computer and video games can inform instructional design. Educational Technology Research and Development, 53 (2), 67–83.

Dickey, M. D. (2006). Game design narrative for learning: Appropriating adventure game design narrative devices and techniques for the design of interactive learning environments. Educational Technology Research and Development, 54 (3), 245–263.

Dickey, M. (2007). Game design and learning: a conjectural analysis of how massively multiple online role-playing games (MMORPGs) foster intrinsic motivation. Educational Technology Research and Development, 55 (3), 253–273.

Echeverria, A., Garcia-Campo, C., Nussbaum, M., Gil, F., Villalta, M., Amestica, M., et al. (2011). A framework for the design and integration of collaborative classroom games. Computers and Education, 57 (1), 1127–1136.

Frasca, G. (1999). Ludology meets narratology: Similitude and differences between (video) games and narrative. Retrieved March 10, 2013 from http://www.ludology.org/articles/ludology.htm .

Garris, R., Ahlers, R., & Driskell, J. E. (2002). Games, motivation, and learning: A research and practice model. Simulation and Gaming, 33 (4), 441.

Gee, J. P. (2004). Learning by design: Games as learning machines. Interactive Educational Multimedia, 8 , 15–23.

Granic, I., Lobel, A., & Engels, R. C. (2014). The benefits of playing video games. American Psychologist, 69 (1), 66–78.

Green, C. S., & Bavelier, D. (2012). Learning, attentional control, and action video games. Current Biology, 22 (6), R197–R206.

Habgood, M. P. J., & Ainsworth, S. E. (2011). Motivating children to learn effectively: Exploring the value of intrinsic integration in educational games. The Journal of the Learning Sciences, 20 (2), 169–206.

Habgood, M. P. J., Ainsworth, S. E., & Benford, S. (2005). Endogenous fantasy and learning in digital games. Simulation and Gaming, 36 (4), 483–498.

Haugom, R., Solbjørg, O. K., Pettersen, K. Y., & Eikaas, T. I. (2007). A simulation game for nonlinear control theory education. Modeling, Identification and Control, 28 (2), 45–51.

Hays, R. T. (2005). The effectiveness of instructional games: A literature review and discussion. Retrieved May 10, 2006, from http://adlcommunity.net/file.php/23/GrooveFiles/Instr_Game_Review_Tr_2005.pdf .

Howard-Jones, P. A., & Demetriou, S. (2009). Uncertainty and engagement with learning games. Instructional Science, 37 (6), 519–536.

Hsu, C., & Tsai, C. (2013). Examining the effects of combining self-explanation principles with an educational game on learning science concepts. Interactive Learning Environments, 21 (2), 104–115.

Hwang, G., Sung, H., Hung, C., Huang, I., & Tsai, C. (2012). Development of a personalized educational computer game based on students’ learning styles. Educational Technology Research and Development, 60 (4, Special Issue on Personalized Learning), 623–638.

Hwang, G., Yang, L., & Wang, S. (2013). A concept map-embedded educational computer game for improving students’ learning performance in natural science courses. Computers and Education, 69 , 121–130.

Infante, C., Weitz, J., Reyes, T., Nussbaum, M., Gómez, F., & Radovic, D. (2010). Co-located collaborative learning video game with single display groupware. Interactive Learning Environments, 18 (2), 177–195.

Ito, M. (2008). Education vs. entertainment: A cultural history of children’s software. The ecology of games: Connecting youth, games, and learning (pp. 89–116). Cambridge: MIT Press.

Järvinen, A. (2008). Games without frontiers: Theories and methods for game studies and design . Tampere: Tampere University Press.

Jenkins, H. (2002). Game design as narrative architecture. In: Harrington P., & Frup-Waldrop, N. (Eds.). First person. Cambridge: MIT Press. Retrieved from http://www.anabiosispress.org/VM606/1stPerson_hjenkins.pdf .

Kafai, Y. B. (1995). Minds in play: Computer game design as a context for children’s learning . London: Routledge.

Ke, F. (2008). A case study of computer gaming for math: Engaged learning from gameplay? Computers and Education, 51 (4), 1609–1620.

Ke, F. (2014). An implementation of design-based learning through creating educational computer games: A case study on mathematics learning during design and computing. Computers and Education, 73 , 26–39.

Ke, F., & Abras, T. (2013). Games for engaged learning of middle school children with special learning needs. British Journal of Educational Technology, 44 (2), 225–242.

Kickmeier-Rust, M. D., & Albert, D. D. (2010). Micro-adaptivity: Protecting immersion in didactically adaptive digital educational games. Journal of Computer Assisted Learning, 26 (2), 95–105.

Kiili, K. (2005). Digital game-based learning: Towards an experiential gaming model. Internet and Higher Education, 8 (1), 13–24. doi: 10.1016/j.iheduc.2004.12.001 .

Kinzie, M. B., & Joseph, D. R. D. (2008). Gender differences in game activity preferences of middle school children: Implications for educational game design. Educational Technology Research and Development, 56 (5/6, Special Thematic Issue on Game-Based Learning), 643–663.

Klopfer, E., Osterweil, S., & Salen, K. (2009). Moving learning games forward: Obstacles, opportunities, & openness. Boston: The Education Arcade. Retrieved April 28, 2011, from http://education.mit.edu/papers/MovingLearningGamesForward_EdArcade.pdf .

Klopfer, E., & Squire, K. (2008). Environmental detectives—the development of an augmented reality platform for environmental simulations. Educational Technology Research and Development, 56 (2), 203–228.

Koenig, A. D. (2008). Exploring effective educational video game design: The interplay between narrative and game-schema construction. Dissertation Abstracts International Section A: Humanities and Social Sciences .

Kolb, D. A., Boyatzis, R. E., & Mainemelis, C. (2001). Experiential learning theory: Previous research and new directions. In R. J. Sternberg & L. F. Zhang (Eds.), Perspectives on thinking, learning, and cognitive styles (Vol. 1, pp. 227–247). Mahwah, NJ: Lawrence Erlbaum.

Lazarou, D. (2011). Using cultural-historical activity theory to design and evaluate an educational game in science education. Journal of Computer Assisted Learning, 27 (5), 424–439.

Leemkuil, H., & de Jong, T. (2012). Adaptive advice in learning with a computer-based knowledge management simulation game. Academy of Management Learning and Education, 11 (4), 653–665.

Lenhart, A., Kahne, J., Middaugh, E., Macgill, A. R., Evans, C., & Vitak, J. (2008). Teens, video games, and civics: Teens’ gaming experiences are diverse and include significant social interaction and civic engagement. Pew Internet and American Life Project. Retrieved from http://www.pewinternet.org/2008/09/16/teens-video-games-and-civics/ .

Lester, J. C., Spires, H. A., Nietfeld, J. L., Minogue, J., Mott, B. W., & Lobene, E. V. (2014). Designing game-based learning environments for elementary science education: A narrative-centered learning perspective. Information Sciences, 264 , 4–18.

Li, Z. Z., Cheng, Y. B., & Liu, C. C. (2013). A constructionism framework for designing game-like learning systems: Its effect on different learners. British Journal of Educational Technology, 44 (2), 208–224.

Lindley, C. A., & Sennersten, C. C. (2008). Game play schemas: From player analysis to adaptive game mechanics. International Journal of Computer Games Technology . Article ID 216784. doi: 10.1155/2008/216784 .

Liu, M., Horton, L., Kang, J., Kimmons, R., & Lee, J. (2013). Using a Ludic simulation to make learning of middle school space science fun. International Journal of Gaming and Computer-Mediated Simulations (IJGCMS), 5 (1), 66–86.

MacCallum-Stewart, E. (2011). Stealth learning in online games. Digital games and learning (pp. 107–128). New York: Continuum.

Malone, T. W., & Lepper, M. R. (1987). Making learning fun: A taxonomy of intrinsic motivations for learning. Aptitude, Learning, and Instruction, 3 (1987), 223–253.

Martinez-Hernandez, K. J. (2012). Development and assessment of a chemistry-based computer video game as a learning tool. Dissertation abstracts international section A: Humanities and social sciences.

Miller, L. M., Chang, C. I., Wang, S., Beier, M. E., & Klisch, Y. (2011). Learning and motivational impacts of a multimedia science game. Computers and Education, 57 (1), 1425–1433.

Miller, C. S., Lehman, J. F., & Koedinger, K. R. (1999). Goals and learning in microworlds. Cognitive Science, 23 (3), 305–336.

Münz, U., Schumm, P., Wiesebrock, A., & Allgöwer, F. (2007). Motivation and learning progress through educational games. IEEE Transactions on Industrial Electronics, 54 (6), 3141–3144.

O’Neil, H. F., Chung, G. K., Kerr, D., Vendlinski, T. P., Buschang, R. E., & Mayer, R. E. (2014). Adding self-explanation prompts to an educational computer game. Computers in Human Behavior, . doi: 10.1016/j.chb.2013.07.025 .

O’Rourke, E., Haimovitz, K., Ballweber, C., Dweck, C., & Popović, Z. (2014). Brain points: A growth mindset incentive structure boosts persistence in an educational game. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 3339–3348). ACM.

Ota, K. R., & DuPaul, G. J. (2002). Task engagement and mathematics performance in children with attention-deficit hyperactivity disorder: Effects of supplemental computer instruction. School Psychology Quarterly, 17 (3), 242–257.

Pareto, L., Haake, M., Lindström, P., Sjödén, B., & Gulz, A. (2012). A teachable-agent-based game affording collaboration and competition: Evaluating math comprehension and motivation. Educational Technology Research and Development, 60 (5), 723–751.

Perry, D., Robinson, J., Cruz, S., Aragon, C., Chowning, J. T., & Peters, M. (2014). Game design for bioinformatics and cyberinfrastructure learning: a parallel computing case study. Concurrency and Computation: Practice and Experience, 26 (13), 2303–2315.

Piaget, J. (1962). Play, dreams, and imitation in childhood . New York: W.W. Norton & Co.

Plass, J. L., Homer, B. D., Hayward, E. O., Frye, J., Huang, T. T., Biles, M., et al. (2012). The effect of learning mechanics design on learning outcomes in a computer-based geometry game. E-learning and games for training, education, health and sports (pp. 65–71). Berlin: Springer.

Prensky, M. (2005). Computer games and learning: Digital game-based learning. Handbook of computer game studies, 18 , 97–122.

Ravenscroft, A. (2007). Promoting thinking and conceptual change with digital dialogue games. Journal of Computer Assisted learning, 23 (6), 453–465.

Richards, J., Stebbins, L., & Moellering, K. (2013). Games for a digital age: K-12 market map and investment analysis. In Joan Ganz Cooney Center at Sesame Workshop , New York.

Robertson, J., & Howells, C. (2008). Computer game design: Opportunities for successful learning. Computers and Education, 50 (2), 559–578.

Ronimus, M., Kujala, J., Tolvanen, A., & Lyytinen, H. (2014). Children’s engagement during digital game-based learning of reading: The effects of time, rewards, and challenge. Computers and Education, 71 , 237–246.

Ryan, R. M., Rigby, C. S., & Przybylski, A. (2006). The motivational pull of video games: A self-determination theory approach. Motivation and emotion, 30 (4), 344–360.

Salen, K., & Zimmerman, E. (2004). Rules of play: Game design fundamentals . Cambridge: MIT Press.

Sarbin, T. R. (1986). Narrative psychology: The storied nature of human conduct . Westport: Praeger Publishers/Greenwood Publishing Group.

Schell, J. (2014). The art of game design: A book of lenses . Boca Raton: CRC Press.

Schrader, C., & Bastiaens, T. (2012a). Learning in educational computer games for novices: The impact of support provision types on virtual presence, cognitive load, and learning outcomes. International Review of Research in Open and Distance Learning, 13 (3), 206–227.

Schrader, C., & Bastiaens, T. J. (2012b). The influence of virtual presence: Effects on experienced cognitive load and learning outcomes in educational computer games. Computers in Human Behavior, 28 (2), 648–658.

Schwartz, R., & Perkins, D. (1989). Teaching thinking—issues and approaches . Pacific Grove: Midwest Publications.

Sedig, K. (2008). From play to thoughtful learning: A design strategy to engage children with mathematical representations. Journal of Computers in Mathematics and Science Teaching, 27 (1), 65–101.

Shaffer, D. W. (2006). Epistemic frames for epistemic games. Computers and Education, 46 (3), 223–234.

Shelton, B. E., & Scoresby, J. (2011). Aligning game activity with educational goals: Following a constrained design approach to instructional computer games. Educational Technology Research and Development, 59 (1), 113–138.

Shute, V. J., & Ke, F. (2012). Games, learning, and assessment. In Assessment in Game-Based Learning (pp. 43–58). New York: Springer.

Shute, V. J., Ventura, M., & Kim, Y. J. (2013). Assessment and learning of qualitative physics in Newton’s playground. The Journal of Educational Research, 106 (6), 423–430.

Sicart, M. (2008). Defining game mechanics. Game Studies, 8 (2), 1–14.

Sotamaa, O. (2010). When the game is not enough: Motivations and practices among computer game modding culture. Games and Culture, 5 (3), 239–255.

Squire, K.D. (2003). Gameplay in context: Learning through participation in communities of civilization III players . Unpublished PhD thesis, Instructional Systems Technology Department, Indiana University.

Steinkuehler, C. A. (2008). Cognition and literacy in massively multiplayer online games. Handbook of research on new literacies (pp. 1–38). Mahwah: Erlbaum.

Strauss, A., & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques . Newbury Park: Sage.

Suits, B. H. (1978). The grasshopper: Games, life and utopia . Toronto: University of Toronto Press.

Tanes, Z., & Cho, H. (2013). Goal setting outcomes: Examining the role of goal interaction in influencing the experience and learning outcomes of video game play for earthquake preparedness. Computers in Human Behavior, 29 (3), 858–869.

Toprac, P. K. (2009). The effects of a problem-based learning digital game on continuing motivation to learn science. Dissertation abstracts international section A: Humanities and social sciences. ProQuest.

Tricot, A., & Sweller, J. (2014). Domain-specific knowledge and why teaching generic skills does not work. Educational Psychology Review, 26 (2), 265–283.

Villalta, M., Gajardo, I., Nussbaum, M., Andreu, J. J., Echeverría, A., & Plass, J. L. (2011). Design guidelines for classroom multiplayer presential games (CMPG). Computers and Education, 57 (3), 2039–2053.

Vogel, J. F., Vogel, D. S., Cannon-Bowers, J., Bowers, C. A., Muse, K., & Wright, M. (2006). Computer gaming and interactive simulations for learning: A meta-analysis. Journal of Educational Computing Research, 34 (3), 229–243.

Von Wright, J. (1992). Reflections on reflection. Learning and Instruction, 2 (1), 59–68.

Vygotsky, L. S. (1978). The role of play in development. Mind in society (pp. 92–104). Harvard: Harvard University Press.

Wainess, R., Koenig, A., & Kerr, D. (2011). Aligning instruction and assessment with game and simulation design. CRESST report 780. National Center for Research on Evaluation, Standards, and Student Testing (CRESST), Los Angeles.

Watson, W. R. (2007). Formative research on an instructional design theory for educational video games. Dissertation abstracts international section A: Humanities and social sciences, ProQuest.

Webster, J., Trevino, L. K., & Ryan, L. (1993). The dimensionality and correlates of flow in human-computer interaction. Computers in Human Behavior, 9 , 411–426.

Weppel, S., Bishop, M., & Munoz-Avila, H. (2012). The design of scaffolding in game-based learning: A formative evaluation. Journal of Interactive Learning Research, 23 (4), 361–392.

Whittington, J. L. (2011). Serious games: How instructional design and game experts design multimodal learning environments. Dissertation abstracts international section A: Humanities and social sciences.

Williams, D., Yuxin, M., Feist, S., Richard, C. E., & Prejean, L. (2007). The design of an analogical encoding tool for game-based virtual learning environments. British Journal of Educational Technology, 38 (3), 429–437. doi: 10.1111/j.1467-8535.2007.00707 .

Wilson, K. A., Bedwell, W. L., Lazzara, E. H., Salas, E., Burke, C. S., Estock, J. L., et al. (2009). Relationships between game attributes and learning outcomes review and research proposals. Simulation and Gaming, 40 (2), 217–266.

Wouters, P., & van Oostendorp, H. (2013). A meta-analytic review of the role of instructional support in game-based learning. Computers and Education, 60 (1), 412–425. doi: 10.1016/j.compedu.2012.07.018 .

Wu, S. (2009). Reducing unproductive learning activities in serious games for second language acquisition. Dissertation Abstracts international section B: The sciences and engineering.

Yee, N. (2006). Motivations for play in online games. CyberPsychology and Behavior, 9 (6), 772–775.

Young, M. F., Slota, S., Cutter, A. B., Jalette, G., Mullin, G., Lai, B., et al. (2012). Our princess is in another castle a review of trends in serious gaming for education. Review of Educational Research, 82 (1), 61–89.

Zapata-Rivera, D., VanWinkle, W., Doyle, B., Buteux, A., & Bauer, M. (2009). Combining learning and assessment in assessment-based gaming environments: A case study from a New York city school. Interactive Technology and Smart Education, 6 (3), 173–188.

Download references

Author information

Authors and affiliations.

Educational Psychology and Learning Systems, College of Education, Florida State University, Tallahassee, FL, 32306-4453, USA

Fengfeng Ke

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Fengfeng Ke .

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (XLSX 40 kb)

Rights and permissions.

Reprints and permissions

About this article

Ke, F. Designing and integrating purposeful learning in game play: a systematic review. Education Tech Research Dev 64 , 219–244 (2016). https://doi.org/10.1007/s11423-015-9418-1

Download citation

Published : 29 December 2015

Issue Date : April 2016

DOI : https://doi.org/10.1007/s11423-015-9418-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Game-based learning
• Educational game design
• Intrinsic fantasy
• Learning-play integration
• Find a journal
• Publish with us
• Track your research

• What is Project TEAM?
• Research Evidence
• Discussion Questions

The Game Plan

• Research Opportunities
• For Professionals

In the Project TEAM training, trainees learn a “Game Plan” that they can use to help do activities they want and need to do. The Game Plan has four bases, just like a baseball field. The names of the bases are Goal, Plan, Do, and Check. These bases are each named for a step in the Game Plan.

The bases can help you to remember the steps of the Game Plan and get closer to doing an activity you want to do. Each base has a name, a symbol, a hand motion, and a question to ask. All of these things can help you remember the Game Plan in different ways!

To learn more about the steps of the Game Plan, click on its name or symbol below. Each page will teach you about one step of the Game Plan and show you how to fill out the Game Plan Worksheet based on your own activity goal. You can download the full Game Plan Worksheet now.

• About Project TEAM
• For Parents
• First Base: Goal
• Second Base: Plan Step 1
• Second Base: Plan Step 2
• Second Base: Plan Step 3
• Third Base: Do
• Home: Check

Project TEAM is funded by the National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR grant number 90IF0032-01-00). (PI- Kramer).

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Perspective
• Published: 21 June 2024

Using games to understand the mind

• Kelsey Allen   ORCID: orcid.org/0000-0002-8461-0652 1   na1 ,
• Franziska Brändle   ORCID: orcid.org/0000-0003-1031-337X 2   na1 ,
• Matthew Botvinick   ORCID: orcid.org/0000-0001-7758-6896 1 ,
• Judith E. Fan   ORCID: orcid.org/0000-0002-0097-3254 3 ,
• Samuel J. Gershman   ORCID: orcid.org/0000-0002-6546-3298 4 ,
• Alison Gopnik   ORCID: orcid.org/0000-0002-7183-0867 5 ,
• Thomas L. Griffiths   ORCID: orcid.org/0000-0002-5138-7255 6 ,
• Joshua K. Hartshorne   ORCID: orcid.org/0000-0003-1240-3598 7 ,
• Tobias U. Hauser   ORCID: orcid.org/0000-0002-7997-8137 8 , 9 , 10 ,
• Mark K. Ho   ORCID: orcid.org/0000-0002-1454-4768 6 ,
• Joshua R. de Leeuw   ORCID: orcid.org/0000-0003-4815-2364 11 ,
• Wei Ji Ma   ORCID: orcid.org/0000-0002-9835-9083 12 ,
• Kou Murayama   ORCID: orcid.org/0000-0003-2902-9600 10 ,
• Jonathan D. Nelson   ORCID: orcid.org/0000-0002-1956-6691 13 ,
• Bas van Opheusden 6 ,
• Thomas Pouncy 4 ,
• Janet Rafner 14 ,
• Iyad Rahwan   ORCID: orcid.org/0000-0002-1796-4303 15 ,
• Robb B. Rutledge   ORCID: orcid.org/0000-0001-7337-5039 16 ,
• Jacob Sherson   ORCID: orcid.org/0000-0001-6048-587X 14 ,
• Özgür Şimşek   ORCID: orcid.org/0000-0001-5449-0437 17 ,
• Hugo Spiers   ORCID: orcid.org/0000-0002-6792-6356 8 ,
• Christopher Summerfield   ORCID: orcid.org/0000-0002-2941-2653 18 ,
• Mirko Thalmann 2 ,
• Natalia Vélez 4 ,
• Andrew J. Watrous 19 ,
• Joshua B. Tenenbaum   ORCID: orcid.org/0000-0002-1925-2035 20 &
• Eric Schulz   ORCID: orcid.org/0000-0003-3088-0371 2  

Nature Human Behaviour volume  8 ,  pages 1035–1043 ( 2024 ) Cite this article

2372 Accesses

70 Altmetric

Metrics details

• Computational science
• Human behaviour

Board, card or video games have been played by virtually every individual in the world. Games are popular because they are intuitive and fun. These distinctive qualities of games also make them ideal for studying the mind. By being intuitive, games provide a unique vantage point for understanding the inductive biases that support behaviour in more complex, ecological settings than traditional laboratory experiments. By being fun, games allow researchers to study new questions in cognition such as the meaning of ‘play’ and intrinsic motivation, while also supporting more extensive and diverse data collection by attracting many more participants. We describe the advantages and drawbacks of using games relative to standard laboratory-based experiments and lay out a set of recommendations on how to gain the most from using games to study cognition. We hope this Perspective will lead to a wider use of games as experimental paradigms, elevating the ecological validity, scale and robustness of research on the mind.

This is a preview of subscription content, access via your institution

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 12 digital issues and online access to articles

111,21 € per year

only 9,27 € per issue

Buy this article

• Purchase on Springer Link
• Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

The theory of social games: outline of a general theory for the social sciences

Novel 3-D action video game mechanics reveal differentiable cognitive constructs in young players, but not in old

The Austronesian Game Taxonomy: A cross-cultural dataset of historical games

Newell, A. et al. Human Problem Solving Vol. 104 (Prentice-Hall, 1972).

Gobet, F., de Voogt, A. & Retschitzki, J. Moves in Mind: The Psychology of Board Games (Psychology Press. 2004).

Gough, C. Number of gamers worldwide 2021. Statista 13 , 2020 (2019).

Google Scholar  

Suchow, J. W., Griffiths, T. & Hartshorne, J. K. Workshop on scaling cognitive science. Cognitive Science Society , https://cognitivesciencesociety.org/wp-content/uploads/2020/06/cogsci20a-W6-doc.pdf (2020).

Brändle, F., Allen, K. R., Tenenbaum, J. & Schulz, E. Using games to understand intelligence. Proc. Ann. Meeting Cogn. Sci. Soc. 43 , https://escholarship.org/uc/item/17z2q92d (2021).

DePaulis, T. Board games before Ur? Board Game Stud. J. 14 , 127–144 (2020).

Greitemeyer, T. & Mügge, D. O. Video games do affect social outcomes: a meta-analytic review of the effects of violent and prosocial video game play. Pers. Soc. Psychol. Bull. 40 , 578–589 (2014).

PubMed   Google Scholar  

Bertram, L. Digital learning games for mathematics and computer science education: the need for preregistered RCTs, standardized methodology, and advanced technology. Front. Psychol. 11 , 2127 (2020).

Manzano-León, A. et al. Between level up and game over: a systematic literature review of gamification in education. Sustainability https://doi.org/10.3390/su13042247 (2021).

Fleming, T. M. et al. Serious games and gamification for mental health: current status and promising directions. Front. Psychiatry https://doi.org/10.3389/fpsyt.2016.00215 (2017).

Arjoranta, J. How to define games and why we need to. Comput. Games J. 8 , 109–120 (2019).

Stenros, J. The game definition game: a review. Games Cult. 12 , 499–520 (2017).

Wittgenstein, L. Philosophical Investigations: The German Text, with an English Translation by G.E.M. Anscombe, P.M.S. Hacker and Joachim Schulte 4th edn (Blackwell, 2009).

Aarseth, E. in From Literature to Cultural Literacy (eds Segal, N. & Koleva, D.) 180–188 (Palgrave Macmillan UK, 2014).

Gentner, D. Structure-mapping: a theoretical framework for analogy. Cogn. Sci. 7 , 155–170 (1983).

Spelke, E. S., Breinlinger, K., Macomber, J. & Jacobson, K. Origins of knowledge. Psychol. Rev. 99 , 605–632 (1992).

CAS   PubMed   Google Scholar  

Allen, K. R., Smith, K. A. & Tenenbaum, J. B. Rapid trial-and-error learning with simulation supports flexible tool use and physical reasoning. Proc. Natl Acad. Sci. USA 117 , 29302–29310 (2020).

CAS   PubMed   PubMed Central   Google Scholar  

Dubey, R., Agrawal, P., Pathak, D., Griffiths, T. L. & Efros, A. A. Investigating human priors for playing video games. PMLR 80 , 1349–1357 (2018).

Pouncy, T. & Gershman, S. J. Inductive biases in theory-based reinforcement learning. Cogn. Psychol. 138 , 101509 (2022).

Tsividis, P. A. et al. Human-level reinforcement learning through theory-based modeling, exploration, and planning. Preprint at https://doi.org/10.48550/arXiv.2107.12544 (2021).

van Opheusden, B. et al. Expertise increases planning depth in human gameplay. Nature 618 , 1000–1005 (2023).

Coutrot, A. et al. Global determinants of navigation ability. Curr. Biol. 28 , 2861–2866 (2018).

Wu, S. A. et al. Too many cooks: Bayesian inference for coordinating multi-agent collaboration. Top. Cogn. Sci. 13 , 414–432 (2021).

Carroll, M. et al. On the utility of learning about humans for human–AI coordination. Proc. 33rd International Conf. Neural Inf. Process. Syst. (eds Wallach, H. M. et al.) 5174–5185 (Curran Associates, 2019).

Kumar, A. A., Steyvers, M. & Balota, D. A. Semantic memory search and retrieval in a novel cooperative word game: a comparison of associative and distributional semantic models. Cogn. Sci. 45 , e13053 (2021).

McCarthy, W. P., Hawkins, R. D., Wang, H., Holdaway, C. & Fan, J. E. Learning to communicate about shared procedural abstractions. Preprint at https://doi.org/10.48550/arXiv.2107.00077 (2021).

Jun, E., Hsieh, G. & Reinecke, K. Types of motivation affect study selection, attention, and dropouts in online experiments. Proc. ACM Hum. Comput. Interact. 1 , 1–15 (2017).

Brändle, F., Binz, M. & Schulz, E. in The Drive for Knowledge: The Science of Human Information Seeking (eds. Cogliati Dezzi, I. et al.) 147–168 (Cambridge Univ. Press, 2022).

Leonard, J. A., Garcia, A. & Schulz, L. E. How adults’ actions, outcomes, and testimony affect preschoolers’ persistence. Child Dev. 91.4 , 1254–1271 (2020).

Chu, J. & Schulz, L. E. Play, curiosity, and cognition. Annu. Rev. Dev. Psychol. 2 , 317–343 (2020).

Murayama, K. A reward-learning framework of knowledge acquisition: an integrated account of curiosity, interest, and intrinsic–extrinsic rewards. Psychol. Rev. 129 , 175–198 (2022).

Kosoy, E. et al. Exploring exploration: comparing children with RL agents in unified environments. In Proceedings of "Bridging AI and Cognitive Science Workshop", (2020).

Brändle, F., Stocks, L., Tenenbaum, J., Gershman, S. & Schulz, E. Empowerment contributes to exploration behaviour in a creative video game. Nat. Hum. Behav. 7 , 1481–1489 (2023).

Pedersen, M. K. et al. Measuring cognitive abilities in the wild: validating a population-scale game-based cognitive assessment. Cogn. Sci. 47 , e13308 (2023).

Hartshorne, J. K., de Leeuw, J. R., Goodman, N. D., Jennings, M. & O’Donnell, T. J. A thousand studies for the price of one: accelerating psychological science with Pushkin. Behav. Res. Methods 51 , 1782–1803 (2019).

Thorne, H. T., Smith, J. J., Morgan, P. J., Babic, M. J. & Lubans, D. R. Video game genre preference, physical activity and screen-time in adolescent boys from low-income communities. J. Adolesc. 37 , 1345–1352 (2014).

Jansz, J., Avis, C. & Vosmeer, M. Playing the Sims2: an exploration of gender differences in players’ motivations and patterns of play. N. Media Soc. 12 , 235–251 (2010).

Hartshorne, J. K., Tenenbaum, J. B. & Pinker, S. A critical period for second language acquisition: evidence from 2/3 million English speakers. Cognition 177 , 263–277 (2018).

PubMed   PubMed Central   Google Scholar  

Awad, E. et al. The moral machine experiment. Nature 563 , 59–64 (2018).

Allen, K. R. et al. Lifelong learning of cognitive styles for physical problem-solving: the effect of embodied experience. Psychonomic Bull. Rev. , https://doi.org/10.3758/s13423-023-02400-4 (2023).

Ashmaig, O. et al. A platform for cognitive monitoring of neurosurgical patients during hospitalization. Front. Hum. Neurosci. 15 , 702 (2021).

Andrews, G. Gameplay, gender, and socioeconomic status in two American high schools. E-Learn. Digit. Media 5 , 199–213 (2008).

Duggan, M. Gaming and Gamers (Pew Research Center, 2015).

Chassy, P. & Gobet, F. Risk taking in adversarial situations: civilization differences in chess experts. Cognition 141 , 36–40 (2015).

Morsch, F. & Feist, M. Fritz v.12 (Chessbase, 2009).

Holdaway, C. & Vul, E. Risk-taking in adversarial games: what can 1 billion online chess games tell us? Proc. Ann. Meeting Cogn. Sci. Soc. 43 , https://escholarship.org/uc/item/403764rd (2021).

Coutrot, A. et al. Entropy of city street networks linked to future spatial navigation ability. Nature 604 , 104–110 (2022).

Coughlan, G. et al. Toward personalized cognitive diagnostics of at-genetic-risk Alzheimer’s disease. Proc. Natl Acad. Sci. USA 116 , 9285–9292 (2019).

Chu, Y. & MacGregor, J. N. Human performance on insight problem solving: a review. J. Probl. Solving 3 , 6 (2011).

Gick, M. L. & Holyoak, K. J. Analogical problem solving. Cogn. Psychol. 12 , 306–355 (1980).

de Groot, A. D. Het Denken van den Schaker: Een Experimenteel-Psychologische Studie (Noord-Hollandsche Uitgevers Maatschappij,1946).

Chase, W. G. & Simon, H. A. Perception in chess. Cogn. Psychol. 4 , 55–81 (1973).

Gobet, F. et al. Chunking mechanisms in human learning. Trends Cogn. Sci. 5 , 236–243 (2001).

Stafford, T. & Dewar, M. Tracing the trajectory of skill learning with a very large sample of online game players. Psychol. Sci. 25 , 511–518 (2014).

Eisenberg, I. W. et al. Uncovering the structure of self-regulation through data-driven ontology discovery. Nat. Commun. 10 , 2319 (2019).

Rafner, J. et al. Digital games for creativity assessment: strengths, weaknesses and opportunities. Creat. Res. J. 34 , 28–54 (2022).

Schulz, E., Bertram, L., Hofer, M. & Nelson, J. D. Exploring the space of human exploration using Entropy Mastermind. Preprint at bioRxiv , https://doi.org/10.1101/540666 (2019).

Brown, H. R. et al. Crowdsourcing for cognitive science—the utility of smartphones. PLoS ONE 9 , e100662 (2014).

Griffiths, T. L. Manifesto for a new (computational) cognitive revolution. Cognition 135 , 21–23 (2015).

Almaatouq, A. et al. Scaling Up Experimental Social, Behavioral, and Economic Science (Computational Social Science Lab at the University of Pennsylvania, 2021).

Cross, L., Cockburn, J., Yue, Y. & O’Doherty, J. P. Using deep reinforcement learning to reveal how the brain encodes abstract state-space representations in high-dimensional environments. Neuron 109 , 724–738 (2021).

Peterson, J. C., Bourgin, D. D., Agrawal, M., Reichman, D. & Griffiths, T. L. Using large-scale experiments and machine learning to discover theories of human decision-making. Science 372 , 1209–1214 (2021).

Bongiorno, C. et al. Vector-based pedestrian navigation in cities. Nat. Comput. Sci. 1 , 678–685 (2021).

Sullivan, J., Mei, M., Perfors, A., Wojcik, E. & Frank, M. C. SAYCam: a large, longitudinal audiovisual dataset recorded from the infant’s perspective. Open Mind 5 , 20–29 (2021).

Rafferty, A. N., Zaharia, M. & Griffiths, T. L. Optimally designing games for behavioural research. Proc. R. Soc. A 470 , 20130828 (2014).

Pymunk v.6.6.0, https://pymunk.org (Victor Blomqvist, 2023).

Juliani, A. et al. Unity: a general platform for intelligent agents. Preprint at https://doi.org/10.48550/arXiv.1809.02627 (2020).

Mazarakis, A. & Bräuer, P. Gamification is working, but which one exactly? Results from an experiment with four game design elements. Int. J. Hum. Comput. Interact. 39 , 612–627 (2023).

Hodges, N. How to gamify cognitive tasks to increase participant engagement. YouTube , https://www.youtube.com/watch?v=2TBllI4t3v4&t=2s (Sage, 2023).

Silvertown, J. A new dawn for citizen science. Trends Ecol. Evol. 24 , 467–471 (2009).

van Opheusden, B., Acerbi, L. & Ma, W. J. Unbiased and efficient log-likelihood estimation with inverse binomial sampling. PLoS Comput. Biol. 16 , e1008483 (2020).

Acerbi, L. & Ma, W. J. Practical Bayesian optimization for model fitting with Bayesian adaptive direct search. Adv. Neural Inf. Process. Syst. 30 (2017).

Gravetter, F. J. & Forzano, L.-A. B. Research Methods for the Behavioral Sciences 4th edn (Wadsworth, 2012).

Lewkowicz, D. J. The concept of ecological validity: what are its limitations and is it bad to be invalid? Infancy 2 , 437–450 (2001).

Lickliter, R. & Bahrick, L. E. The salience of multimodal sensory stimulation in early development: implications for the issue of ecological validity. Infancy 2 , 451–467 (2001).

Yang, Q. et al. Monkey plays Pac-Man with compositional strategies and hierarchical decision-making. eLife 11 , e74500 (2022).

Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P. & Dolan, R. J. Model-based influences on humans’ choices and striatal prediction errors. Neuron 69 , 1204–1215 (2011).

Kirchner, W. K. Age differences in short-term retention of rapidly changing information. J. Exp. Psychol. 55 , 352–358 (1958).

Gershman, S. J. Uncertainty and exploration. Decision 6 , 277–286 (2019).

Shallice, T. Specific impairments of planning. Phil. Trans. R. Soc. Lond. B 298 , 199–209 (1982).

CAS   Google Scholar  

Costa-Gomes, M., Crawford, V. P. & Broseta, B. Cognition and behavior in normal-form games: an experimental study. Econometrica 69 , 1193–1235 (2001).

Schaul, T. A video game description language for model-based or interactive learning. In 2013 IEEE Conference on Computational Intelligence in Games (CIG) 1–8 (IEEE, 2013).

Download references

Acknowledgements

We thank A. A. Kumar and Y. Harel for helpful discussions.

Author information

These authors contributed equally: Kelsey Allen, Franziska Brändle.

Authors and Affiliations

DeepMind, London, UK

Kelsey Allen & Matthew Botvinick

Max Planck Institute for Biological Cybernetics, Tübingen, Germany

Franziska Brändle, Mirko Thalmann & Eric Schulz

Stanford University, Stanford, CA, USA

Judith E. Fan

Harvard University, Cambridge, MA, USA

Samuel J. Gershman, Thomas Pouncy & Natalia Vélez

University of California, Berkeley, Berkeley, CA, USA

Alison Gopnik

Princeton University, Princeton, NJ, USA

Thomas L. Griffiths, Mark K. Ho & Bas van Opheusden

Boston College, Boston, MA, USA

Joshua K. Hartshorne

University College London, London, UK

Tobias U. Hauser & Hugo Spiers

Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK

Tobias U. Hauser

University of Tübingen, Tübingen, Germany

Tobias U. Hauser & Kou Murayama

Vassar College, Poughkeepsie, NY, USA

Joshua R. de Leeuw

New York University, New York, NY, USA

University of Surrey, Guildford, UK

Jonathan D. Nelson

Aarhus University, Aarhus, Denmark

Janet Rafner & Jacob Sherson

Center for Humans and Machines, Max Planck Institute for Human Development, Berlin, Germany

Iyad Rahwan

Yale University, New Haven, CT, USA

Robb B. Rutledge

University of Bath, Bath, UK

Özgür Şimşek

University of Oxford, Oxford, UK

Christopher Summerfield

Baylor College of Medicine, Houston, TX, USA

Andrew J. Watrous

Massachusetts Institute of Technology, Cambridge, MA, USA

Joshua B. Tenenbaum

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Eric Schulz .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Peer review

Peer review information.

Nature Human Behaviour thanks the anonymous reviewers for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Cite this article.

Allen, K., Brändle, F., Botvinick, M. et al. Using games to understand the mind. Nat Hum Behav 8 , 1035–1043 (2024). https://doi.org/10.1038/s41562-024-01878-9

Download citation

Received : 15 June 2022

Accepted : 03 April 2024

Published : 21 June 2024

Issue Date : June 2024

DOI : https://doi.org/10.1038/s41562-024-01878-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Game Planner Software: How to Plan a Game Development Project

Transform your game idea into a structured project plan.

Planning is perhaps one of the least "sexy" aspects of game development , but it's usually what makes the difference between a successful project and a failed one.

Whether you’ve just finished your last major game development project, or you are venturing out for the very first time, it's easy to get carried away when you get a great new idea for a game. You are excited and itching to start building right away. If you are a developer, you probably want to break open Unreal or Unity and start putting together a prototype. If you are a creative designer, you may already be thinking about concept art.

Some would also argue that meticulous planning is for AAA studios and that small indie projects can be managed using a simple to-do list. In practice, this is rarely the case.

Let's dive deeper into what it takes to plan a successful game development project and what tools can help you in that endeavor.

How to plan a game development project

Selecting the right game planner software.

Whether you're working as part of a large interdisciplinary team or a solo game designer, you need a way to track your progress and organize your assets. Without an organized process, things will inevitably fall between the cracks, potentially derailing the entire development project.

There are many project management apps and documentation tools that can help you keep things on track. If your game development project is simple and small in scope, a combination of Google Docs and Sheets may suffice. More complex projects, however, require a more sophisticated set of tools.

Artwork credit

A great solution for managing your game development project is Nuclino . It's a unified workspace for collaborative game design documentation, worldbuilding , and project planning .

Nuclino allows you to create long-form documents and organize them in a variety of visual ways. The nested list view is handy for organizing and collaborating on your game design documentation in real time. The Kanban board view is great for prioritizing features and planning sprints . The table view can be used to easily sort and filter long lists of game design assets. The mindmap-style graph allows you to visualize the links between different topics, levels, characters, and game assets like in a wiki .

Nuclino items can contain a variety of content, including text, videos, images, files, tasks with due dates and reminders, tables, code blocks, interactive embeds, and more. This allows you to document, share, and collaborate on anything, from game proposals and storyboards to character profiles and concept art. Internal links can be used to easily link related documents and topics together.

All that content can be collaborated on in real time, with every change automatically saved in the version history. Comments and mentions can be used to communicate and exchange feedback asynchronously , preserving the context of every decision.

Visual collaboration is seamlessly built into Nuclino. You can add an infinite collaborative canvas anywhere and create diagrams and whiteboards directly within your design document, without switching tools.

You can use it to visualize your game's core gameplay loop, capture different mechanics and interaction flows, brainstrom ideas using sticky notes, organize concept art, and more.

Nuclino also comes with an AI-powered assistant called Sidekick that can help game designers with various aspects of the writing process. With Sidekick, you can generate ideas for characters and plot points, instantly generate descriptions and dialogue, get suggestions for more concise or engaging language, and much more.

Sidekick also allows you to instantly generate unique concept art, storyboards, and other images in a variety of styles – 2D and 3D, abstract and photorealistic, detailed and simple.

You can connect Nuclino to a wide range of other tools, including Discord , Google Drive , Miro , and more, seamlessly integrating it into your game planning process.

What users say about Nuclino:

"Designing a game requires a huge number of complex, inter-related documents. Game engines, code, tools, processes, character designs, market research, background research, customers, business models... Nuclino is saving us hours when it comes to ‘finding that one thing’ that you didn't need until now, be it a process, design sketch, or meeting notes."

— Matt Bond , Lead Game Designer at Psyon Games

Initial planning

Once you have picked your game planner software, it's time to dive into the actual planning.

Keep in mind, that whatever plan you come up with shouldn't be immediately set in stone. Game design is a highly fluid process, and your plan needs to reflect that. A game development plan tends to be a living document that changes as you identify new requirements or tasks, finish things early or late, and learn more about what you're building.

The first step of planning any game is to create a game design document , which will serve as a blueprint for your game throughout the development process.

Begin with the core game concept , answering the following questions:

What is the core idea behind the game? How can it be summarized in a compelling game pitch ?

What type of video game is it? Will it be 2D or 3D?

What are some of the key features it must have?

Who are its characters? When and where does the story take place?

Who is our target audience?

Which platforms will be supported?

It may not always seem like it, but ideating and answering these basic questions is one of the hardest parts of the game development process . This information will serve as the backbone of your entire project.

Proofing your game concept

The next step is the proof of concept . At this stage, you need to determine whether the ideas you've generated are viable. In order to do that, answering the following questions can be helpful:

What is our budget? How much do we expect it to cost to develop this game?

Do we have the technological capabilities to build it?

Which gaming engine and other game development software will we use?

Who will be on our team? Who will be responsible for what?

What is our estimated development timeframe?

How will we monetize the game?

If you are a part of a big game development studio and are planning to build your game under the umbrella of a publisher, then thoroughly proofing your game concept becomes a vital step that needs to be completed early on. You will need to give your publisher a clear overview of what they can expect and get their approval.

On the other hand, if you are an indie game developer working without publisher oversight, you can usually afford more flexibility during this stage. A detailed proof of concept may still be required if you are relying on crowdfunding websites like Kickstarter.

Creating a task backlog

The next step is to determine the requirements of your game. Then, each requirement will need to be split into a list of supporting features. In turn, each feature will need to be further broken down into tasks for each department, including programming, art, animation, level design , sounds, and so on.

A good technique for capturing requirements from the perspective of the player is to write user stories . A user story usually takes the form of a short sentence, written in simple, informal language, for example:

As a player, I want to launch the main menu so that I can start a new game.

As a player, I want to change options so that I can tweak/update my play experience.

As a player, I want to save my progress so that I can pick up where I left off later.

Put together a production schedule

After you have listed all the tasks that need to be completed, you need to prioritize and assign them. The best place to get started is to create a high-level overview of the entire production schedule.

There are many great diagramming tools you can use to visualize your development timeline, such as Miro , LucidChart , Diagrams.net , Gliffy , and more. If you are using Nuclino as your game planner, you can easily embed an interactive preview of your roadmap directly into a page.

How you handle low-level task prioritization and scheduling depends on the size of your team as well as the specific methodology you prefer (Kanban, Agile, Waterfall, and so on). Whatever approach you choose, make sure to pay attention to task dependencies to avoid production bottlenecks.

Keep in mind that no matter how thorough your initial planning was, the scope and requirements of your game project can change many times over the course of the development process. Stay flexible and be prepared that the final product may look quite different from what you originally envisioned. Make sure your game development plan and game design document stay up-to-date and evolve together with your project.

Ready to get started?

• Why Nuclino?
• Apps & Integrations
• Sidekick (AI)

• THE STRATEGY JOURNEY Book
• Videos & Tutorials
• Strategy Journey Analyzer [QUIZ + WORKBOOK]
• COMMUNITY FORUMS
• Transforming Operating Models with Service Design (TOMS) Program
• ABOUT STRATABILITY ACADEMY

4 steps to create a Winning Game Plan

By   Julie Choo

Published: April 30, 2020

Last Update: January 9, 2024

TOPICS:   Culture & Careers , Data & AI , Gameplans & Roadmaps , Operating Model , Service Design , Transformation

In all walks of life, in business and in sport…

In all things that drive people… the 4 pillars of a good life: health, wealth, knowledge and connectivity… there are many problems to overcome.

And they just keep coming from the disruption caused by our fast-changing and increasingly digital economy as well as times of uncertainty and instability (such as a coronavirus pandemic or a global financial crisis), that inevitably lead to economic downturns.

Your ability to problem-solve, and define appropriate strategies and tactics to overcome those things that challenge you, is what’s going to make that difference to winning or losing … in the ‘game of life’ and the ‘game of business’.

This is why you need to be good at developing and executing ‘game plans’ , that will help you to overcome your problems and achieve the best possible outcomes.

When you go to battle in anything, against any opposition, including invisible enemies such as a virus, it’s best to have a good or even a great game plan, with as much data as possible to help you make the best decisions, and take the best actions.

Your ‘game plan’ is your best chance of winning, and if you don’t have one, then you have definitely lost the fight because you will waste both time and resources… your money on the wrong activities and actions during your battle… that can leave you in danger of even more losses in the future, when you have drained those resources with little left in the tank.

Why you need a ‘game plan’ should be obvious, even if many people rush to building solutions, without one, which result in poor results. And they do this because they don’t know how to develop a good game plan.

In THE STRATEGY JOURNEY Framework , we have developed a process and tools, to help you develop a winning game plan, including templates canvases such as the Game Plan Designer and Game Play Map . We also have a newly upcoming webinar called the Accelerator Masterclass that shows you how companies, projects, governments and people, who have applied the 4 steps that we recommend, have managed to achieve better outcomes, faster in their problem-solving and business transformation efforts. The training will teach you how to do the same with your business, project and career, to solve whatever problems you might have, step-by-step.

Here’s an introduction to this training…

What is a game plan?

Strategy without tactics is the slowest route to victory. Tactics without strategy is the noise before defeat. – SUN TZU

A ‘game plan’ is a set of strategies and tactics with actionable steps to help you solve a specific problem based on the context and consequences of the problem, that give you the best possible set of outcomes in your solution proposal based on the inputs and resources that you have at your disposal to fight the battle in the ‘game’.

The best possible input and the more your resources, then the better your outcomes and your ability to achieve them faster too. This is ‘science’ of gamification.

And how good your game plan is, and how you might execute it, so the ‘game play’ that you chose to take, is the ‘art’ of gamification.

The best analogy for this is in sport, and a sports game plan could be applied to different sports.

Take the example of a football match or basketball game or motor race such as a Formula One (F1) Grandprix. I like these examples because they explicitly have coaches and strategists, that need to make decisions on how to start the game, play the game and change the game, during the game in order to win.

In basketball, you see coaches draw on clipboards during timeouts to direct their players into a specific set of actions in their gameplay, to score a goal. In football you have the same, when coaches substitute players during the game with instructions to change the style of play, to score more goals as well as prevent more goals from being scored by the opposition. And during any F1 Grand Prix, you have different teams and their drivers, using different tire strategies, settings in their cars, which are all built different subject to regulations of course, based on the data they collect and analyze, to perform at their best during qualifying as well as during the race in order to outwit and outpace their opposition. This is science.

In all of these sports, you also need the brilliance of the players and drivers too during the game or race, as that brilliant pass or goal, or overtake, is what actually wins the entire team a game or a race. Of course, these moves take practice and skills, from dedicated professionals operating at the pinnacle of their professions. If you read Sir Alex Ferguson’s book, ‘Leader’ , which is a Harvard Business School textbook, he will tell you how many times David Beckham or Cristiano Ronaldo practiced their shots, and also how he built ‘Manchester United’ into a winning team, for nearly two decades. This is art.

A winning ‘game plan’ has both ‘science’ and ‘art’ to make it perform at its best, to deliver the best possible outcomes.

So to be clear, a game plan is not a ‘solution’, as we have illustrated this using our sporting examples. A game plan comprises the solution or solutions in ‘what’ outcomes it achieves and ‘how’ it achieves them.

And this is true in business as well as in sport, and in the pursuit of finding and executing the best possible solutions to any problem, in all walks of life.

You won’t find it difficult to prove that battles, campaigns, and even wars have been won or lost primarily because of logistics. – General Dwight. D. Eisenhower

Just look at those countries like South Korea and Taiwan who were prepared with game plans for a viral pandemic, versus countries like the United Kingdom who weren’t ready having spent their resources on other things and cut infrastructure spending on their health care system, and even reacted late, because they were making it up as they went, reacting to only new data, while trying to hide old data that reflected their poor decision making. It’s a case of poor inputs as well as poor resources, that led the UK such a high number of deaths.

Game plan on what ?

The concept of a game plan is equally relevant in diverse arenas, be it in the world of business, the realm of sports, the path of career growth, or the pursuit of education. So, how could a game plan be used?

Business Game Plan :

In the dynamic world of business, a game plan acts as a blueprint for success. Just like in a game of chess, where each move is strategically designed, a business game plan outlines steps to achieve objectives. It starts with identifying the problem reality – analyzing market trends, competition, and customer needs. Prioritizing based on consequences helps focus on critical tasks that will yield the most significant impact. Defining the ideal scenario sets clear goals, and then developing a solution proposal, complete with actionable steps and a roadmap, ensures a well-structured execution. Much like Amazon’s AI strategic approach , a robust game plan maximizes resources, minimizes wastage, and propels businesses towards their objectives.

Career Game Plan:

Much like a roadmap to success, a career game plan navigates the professional journey. Identifying the problem reality might involve assessing current skills and job satisfaction. Prioritizing could mean focusing on skill development or seeking new opportunities. The ideal scenario could be achieving a leadership role or excelling in a chosen field. Developing a solution proposal involves crafting a step-by-step approach, such as pursuing higher education, networking, or taking on challenging projects. This structured path helps individuals move from entry-level positions to fulfilling, impactful careers.

Education Game Plan:

For students and learners, an education game plan acts as a compass for academic success. Identifying the problem reality involves recognizing academic strengths and weaknesses. Prioritizing could involve setting goals for subjects that require more attention. The ideal scenario might include achieving a certain GPA or mastering specific skills. Developing a solution proposal entails planning study schedules, seeking additional resources, and leveraging support networks. This approach ensures that learners excel in their studies and achieve their educational aspirations.

In each of these spheres, a winning game plan is the secret to turning aspirations into accomplishments. Whether in business, sports, career, or education, the art and science of gamification provide a structured approach to achieving success and overcoming challenges. Just as Sun Tzu’s wisdom reminds us, “Strategy without tactics is the slowest route to victory. Tactics without strategy is the noise before defeat.”

Future Proofing Game Plan (Trends and adapting to digital transformations) :

In the era of relentless digital transformations, a well-crafted game plan becomes a strategic shield against obsolescence and a roadmap to agile adaptation. Take, for instance, a traditional brick-and-mortar retailer recognizing the shift to e-commerce. Acknowledging the problem reality (declining foot traffic), they prioritize pivoting to online sales and define an ideal scenario (robust online presence) such as with the example of Netflix vs Blockbuster . Netflix had realized the potential of online streaming platforms and utilized researched data to provide original and better experience for their audience and ultimately, became one of the top streaming platforms in the world. Having the insight and an agile game plan allowed Netflix to not only future-proofs against digital disruption but also facilitates a seamless transition into the digital marketplace, securing the retailer’s relevance and competitiveness.

An individual can leverage a game plan to navigate the evolving digital landscape. Identifying the problem reality (limited digital skills), they prioritize acquiring in-demand digital competencies, defining an ideal scenario (a versatile digital skill set), and developing a solution proposal (enrolling in online courses or seeking mentorship). This proactive approach ensures adaptability to shifting job requirements, future-proofing their career against digital advancements, and positioning them as invaluable contributors in the ever-changing professional realm.

The science of gamification

In the science of gamification, there are 4 steps in the process to create a winning game plan, that give you the best possible outcomes based on your inputs and resources.

The 4 steps to create a winning game plan are:

1. Identify the PROBLEM REALITY

Clarity on the context of the problem you need solving, who its for, and what makes it problem is the first step, because this sets the direction of what actions you will take to solve that problem.

In business and in your career, there are many problems to solve. There are problems in society too, from climate change, to social inequality, to health pandemics.

There is a lot of data to gather that will inform you of what this problem is, who is impacts, why its important, that will help set you up on your problem solving journey to define your game plan.

If you don’t have enough data, then you are likely to go in the wrong direction and build solutions that are inadequate, that do not solve the problem, and that address the wrong problem, or worst you are wasting your time and resources on a problem that doesn’t really exist.

So what makes up the problem or problems?

In the sporting examples that we have alluded to so far, the problems that clubs like the Manchester United Football Club, or the Ferrari Formula One Team have are multifaceted. These sporting businesses are big businesses too that make yearly revenue in the many 10s to 100s of millions. The big problems that they need to deal with in their businesses include:

• Winning games and races
• Scoring a certain number of goals or points in order to win a championship
• Building loyal fans and interest among spectators who would pay for their services, by purchasing tickets or tv subscriptions to watch games and races, buying merchandise
• Finding the funds to run their business activities, from sales as well as sponsorships, or investment backers and owners

… the list goes on…

Different types of businesses or organizations big and small will have their own sets of problems that they need to deal with in order to run and operate their enterprises, from getting traffic to their website, to making sales, to cutting costs, to training staff, to packaging and delivering goods to customers, to maintaining quality, health and safety or speed of delivery …

There a lot of problems to solve, and lots of ‘logistics’ to sort out, and this is why a grand game plan and many smaller ones too, are required. And they need to be prioritized, in order to ensure resources, in the people, processes, data and systems that the business and teams have, are not wasted on the wrong things.

Your ‘game plan’ is only as good as the inputs in data that you have on ‘what is the problem reality?’ – to help you clarify its purpose and set direction as you start to define its proposed solutions.

2. Prioritize based on CONSEQUENCES

Once you have all the data that gives a clear indication of the problem reality, where there is a big problem with lots of smaller ones too, then the second step is to prioritize these different smaller problems that manifest. It is important to prioritize so that you can work out the order by which to address them in your solution, based on the resources that you have. If you don’t then you will run out of resources and not achieve anything. This is what leads the whole game plan to fail, or the cause of failure when you don’t have a game plan. You are taking the wrong actions based on making the wrong decisions, and wasting time and resources, including precious funds in your business.

It is important that you realize you can’t fix everything, and sometimes, somethings have to give way, to more important things. And there are often tough decisions to make, and depending on your role in an enterprise, you may be that decision maker or you are the data scientist or analyst that is putting the scenarios together to help the boss make that decision, or possibly someone that contributes to the data or needs to be informed by the data. There are actually four roles here, of people who are:

• R esponsible,
• A ccountable,
• C ontribute, or

The objective of this step is to work out what happens in different scenarios, if different problems are not fixed and what could happen if it was, and even play around with the order or sequencing of these activities.

This analysis is what give the enterprise and its decision maker different options, by which to act on.

So in the example of the COVID-19 pandemic, we see all these different data scientists along with input from their Chief Medical and Chief Scientific Offers working for their respective governments, analyze the data that they have from STEP 1, to determine how the virus could spread, depending on how they manage their lockdowns, from timing how different services are closed or restrictions lifted.

These scenarios backed by the data that they have of different quality and accuracy, are the options that they present to the decision makers, like a Secretary of Health, a Governor, a Prime Minister or President to make or take the right decisions.

And again in business, the best example comes from Amazon, where analysts and AI systems in a company, would be mining and then analyzing all the data in the company to show different options in how Amazon can increase sales, manage costs and perform different business activities, such as adding a new system, or buying another business in an M&A acquisition, and so on…

The goal is to prioritize, including ruling things in and ruling things out, as well as sequencing activities to form multiple scenario options for consideration.

3. Define the IDEAL SCENARIO (Outcomes)

With the different scenario options available from STEP 2, the next step is to agree what is the IDEAL scenario and hence the recommendation to the decision maker.

In this step, the decision makers will evaluation the options presented by the analyst and scientists, and make the decision on what to proceed with.

Recalculations may be needed, as the decision-makers will ask questions related to the pros and cons of each option, what data was used, how good is the data, and what makes the recommended IDEAL scenario the best one.

The goal is to dive into the outcomes and both qualify them and quantify them – another step in the data science process.

The outcomes which form the objectives of the ‘game plan’, need to be S.M.A.R.T in order to enable the quantification and qualification process.

S.M.A.R.T objectives or outcomes need to be:

• M easurable
• A ctionable
• T ime-bound

It is when you are performing the data science through quantifying and qualifying of your smart objectives with their outcomes that you consider what resources you have to make the best IDEAL scenario possible too. This is what makes it ‘realistic’. You can’t move into STEP 4, when you are starting to action the game plan, if you don’t have the resources to make your gameplay.

4. Develop the SOLUTION PROPOSAL (Gameplay & Roadmap)

The fourth and final step of creating a winning game plan is to take the final IDEAL scenario from STEP 3 and break it down into stages in that form the ‘gameplay’.

The ‘gameplay’ is the sequence of events that the enterprise and its players will execute, to take the game plan ‘live’, and that gives it that chance to win the game.

And in each game play stage, there is a ‘priority play’, that first step or first move. When you play a game of chess or participate in any strategy game, this first move can often be the decisive one that set the direction for what comes after. They don’t call it first mover advantage for nothing 😉

So, you have to determine what is that first move.

And within each gameplay stage, you can breakdown the stages into activities, those actionable steps, that are also specific, measurable, realistic and time-bound that can then be sequenced out in a ‘roadmap’.

In Tesla’s strategy, which is a major case study we discuss in THE STRATEGY JOURNEY Book and part of our training course The Accelerator Masterclass , you will see how Elon Musk came up with this strategies and tactics, and this first mover gameplay that Tesla used to set itself up for success to achieve its mission and vision.

Tesla’s first move was to create an electric sports car that was relatively expensive for car and tech enthusiasts who would move likely be willing to pay a premium to experience a new innovation, our innovators and early adopters in the Innovation Adoption Lifecycle. This move was set to increase brand awareness and help the company to build out its infrastructure to support the logistics of not just building more electric cars better and faster, but also batteries and solar cells, which would support the company’s long-term strategy in its mission and vision to become a major energy player specializing in re-newable energy and storage in the energy market.

In the science of gamification, you have there are three key secrets to that the create winning game plans:

1. Play to win

2. Take your shot

3. Enjoy the ride

Step 4 is is where you have moved from ‘Play to win’ to ‘Take your shot’, as its where you begin to put your game plan into action through your gameplay.

And when you have the roadmap of activities and those next steps to execute the best strategy and tactics that you can play based on the resources that you have, then

And let’s use the game of monopoly to illustrate the Telsa Case Study…

Tesla is a ‘player’. It has chosen a ‘car’ to represent it on the board.

The game is the energy market

Tesla is trying buying up lots of land and infrastructure, represented by all the properties on the board, to support its mission to be a major player in the game.

Tesla is also building lots cars and batteries, all backed by the latest technology, which are the little houses and hotels that it is can build on each of its properties.

And Elon Musk has been pretty transparent on his game plan with Tesla’s gameplay stages. He even published them in 2016, in his article ‘PART DEUX’ for everyone to see in on Tesla’s website . Supporting Tesla grand game plan is a pretty extensive roadmap of activities that the company is trying to execute which requires a lot of resources and funding – hence investors spent much of 2019 speculating on Tesla’s cashflow issues.

Through executing the 4 steps to create a winning game plan, you will start your journey to to learn and practise the ‘art’ and ‘science’ of gamification, and play the game where you get to see how your strategies and tactics will help you win the game.

Gaming should be fun, and for many people including nearly 900million online gamers around the world , it is and they are addicted to it.

Using gamification, with the process we have set out in the 4 steps to creating a winning game plan, you too can have fun as you solve different problems in your business, projects and career, and in all walks of life, to support those 4 pillars of a good life that drive us all – health, wealth, knowledge and connectivity.

About the author

Julie Choo is lead author of THE STRATEGY JOURNEY book and the founder of STRATABILITY ACADEMY. She speaks regularly at numerous tech, careers and entrepreneur events globally. Julie continues to consult at large Fortune 500 companies, Global Banks and tech start-ups. As a lover of all things strategic, she is a keen Formula One fan who named her dog, Kimi (after Raikkonnen), and follows football - favourite club changes based on where she calls home.

You might also like

Culture & Careers , Data & AI , Gameplans & Roadmaps , Operating Model , Service Design , Strategy Journey Fundamentals , Transformation

The Impact of Co Creation in Modern Business

Service Design

9 Steps to your Winning Customer Journey Strategy

Business level strategies: what are they, how to use it.

You're reading a free article with opinions that may differ from The Motley Fool's Premium Investing Services. Become a Motley Fool member today to get instant access to our top analyst recommendations, in-depth research, investing resources, and more. Learn More

Surprise! Your Child's Summer Job Can Turn Into a Million-Dollar Roth IRA

• Anyone who meets the income requirements, even a child under 18, can contribute to a Roth IRA.
• If your child is working a paid summer job this year, it's a great time to consider opening a Roth IRA.
• You can open a custodial IRA account to help your child start building towards their million-dollar retirement.
• Motley Fool Issues Rare “All In” Buy Alert

It won't happen overnight, but with consistency and a game plan, it will be easier to reach the finish line.

If you're aiming to set your child up for a comfortable future, opening a Roth IRA might be a great place to start. A Roth IRA is an individual retirement account that allows savers to contribute after-tax dollars in exchange for tax-free growth and withdrawals in retirement. This account type is particularly appealing for kids because it doesn't come with age restrictions, and children are often in a much lower tax bracket when they first start earning money than they will be later.

So if your child landed a summer job this year, here's how that paycheck can put them on a path toward a million-dollar Roth IRA.

Image source: Getty Images.

How to start your child's Roth IRA journey

Let's say your child is 16 years old and landed a paid summer internship. You, a family member, or another adult can research the best custodial Roth IRAs , open an account at a financial institution, and manage the account until the child is eligible to take control (age 18 in most states). However, here are a few things to consider before stashing money in a Roth IRA :

• Your child can only contribute to the account during the years when they have earned income from a job or self-employment activities.
• If your child's income exceeds the limits , they won't be able to make direct contributions to a Roth IRA for that year.
• Keep good records so that you can prove your child's income if needed.
• Work with a Certified Public Accountant or other professional to ensure you're following the rules.

Start contributing money to your child's Roth IRA

You can contribute up to 100% of your child's earned income to a Roth IRA, with a maximum of $7,000 for 2024. If your child only earned $5,000 from their summer job and no other income during the year, the total contribution to their account is capped at $5,000 for the year. Keep in mind that allowance, dividends, and interest income do not count as earned income.

You can incentivize your child to contribute some of their earnings into a Roth IRA by offering to match some of their contributions. For instance, using the example above, they could contribute $2,500, and you could contribute the remaining $2,500 to help them max out their Roth IRA for the year. This might be a steep savings rate for a child, so start them off with budgeting. Show them how to track their monthly expenses so they can determine if the way they are using their money will get them closer to the life they want.

Put your child's paycheck to work

Teaching your child good savings habits will help them meet their annual contribution goals, but teaching them how to invest will take their goals to the next level. Depending on where you open their Roth IRA, they can invest in various assets, including individual stocks and exchange-traded funds . Educate your child on the importance of diversification and long-term investing. Show them how different assets work and how to research performance, and instill the discipline to stay committed to their investment plan through market fluctuations. It's also important to let them know about risk and the possibility of their portfolio fluctuating in value.

One practical way to get your child excited about investing is to show them the power of compounding. Teach them how to calculate how their portfolio can grow over time if they invest a certain amount of money and receive a certain average return. Below is an example of how annual contributions of $7,000 could grow over time at 7%, 8%, 9%, and 10% returns, which are in line with historical averages .

Data source: Author calculations.

The key to helping your child turn their paycheck into a million-dollar Roth IRA is consistency. Get them involved in the Roth IRA contribution process from the start so they understand the power of saving and investing. Although you and your child are not required to make contributions to the account every year, show them the difference between consistent contributions and sporadic ones. The more your child understands the value of time and how the Roth IRA works, the more excited they will be to allocate a portion of their paycheck toward building their million-dollar retirement portfolio.

The Motley Fool has a disclosure policy .

Related Articles

Premium Investing Services

Invest better with The Motley Fool. Get stock recommendations, portfolio guidance, and more from The Motley Fool's premium services.

Pharma’s digital Rx: Quantum computing in drug research and development

The development of molecular formulations that become drugs to treat or cure diseases is at the heart of the pharmaceutical industry. Development is so fundamental that pharma spends a full 15 percent of its sales on R&D—a huge sum that accounts for more than 20 percent of total R&D spending across all industries in the global economy. This investment goes hand in hand with innovation: constantly seeking to improve the R&D process, pharma companies have for decades been early adopters of computational chemistry’s digital tools, such as molecular dynamics (MD) simulations and density functional theory (DFT). More recently, pharma R&D has taken advantage of artificial intelligence (AI). The next digital frontier is quantum computing  (QC).

In a recent article , we analyzed the impact of QC on the chemical industry, which, similarly to pharma, relies on the development and manufacture of molecules, and concluded that it will be one of the first industries to benefit. In this article, we explain the profound impact that QC could have on the pharma industry and present use cases for its application. We also provide a set of strategic questions to get clarity on the path forward for industry players.

Pharma’s focus on molecular formations makes it well suited for QC

Identifying and developing small molecules and macromolecules that might help cure illnesses and diseases is the core activity of pharmaceutical companies. Given its focus on molecular formations, pharma as an industry is a natural candidate for quantum computing. The molecules (including those that might be used for drugs) are actually quantum systems; that is, systems that are based on quantum physics. QC is expected to be able to predict and simulate the structure, properties, and behavior (or reactivity) of these molecules more effectively than conventional computing can. Exact methods are computationally intractable for standard computers, and approximate methods are often not sufficiently accurate when interactions on the atomic level are critical, as is the case for many compounds. Theoretically, quantum computers have the capacity to efficiently simulate the complete problem, including interactions on the atomic level. As these quantum computers become more powerful, tremendous value will be at stake.

The basics of quantum computing

A conventional computer, built on transistor-based classical bits operated by voltages, can be in only one of two states: 0 or 1. A quantum computer, instead, uses systems based on quantum physics, such as superconducting loops or ions hovering in electromagnetic fields (ion traps), which are operated by microwave radiation or lasers, respectively. As a result of the laws of quantum mechanics, such systems can be held in a special physical state, called a quantum superposition, in which quantum bits (qubits) exist in a probabilistic combination of the two states—0 and 1—simultaneously.

The implications of these effects for QC are dramatic. Qubits can process far more information than conventional computers can. Qubits use the characteristics of quantum-mechanical systems to solve complex equations in a probabilistic manner, so a computation solved with a quantum algorithm enables sampling from the probabilistic distribution of being correct. The combination of greater speed with probabilistic solutions means that quantum computing fits well with a certain subset of computing needs and applications, such as optimization, the simulation of chemicals, and AI.

While the technology behind quantum computing is rather difficult to understand intuitively (see sidebar, “The basics of quantum computing”), its impact is much easier to grasp: it will handle certain kinds of computational tasks exponentially faster than today’s conventional computers do. Thus, once fully developed, QC could add value across the entire drug value chain—from discovery through development to registration and postmarketing.

QC’s biggest impact on pharma will be in the discovery phases

While QC may benefit the entire pharma value chain—from research across production through commercial and medical—its primary value lies in R&D (Exhibit 1).

Currently, pharma players process molecules with non-QC tools, such as MD and DFT, in a methodology called computer-assisted drug discovery (CADD). But the classical computers they rely on are sorely limited, and basic calculations predicting the behavior of medium-size drug molecules could take a lifetime to compute accurately. CADD on quantum computers could increase the scope of biological mechanisms amenable to CADD, shorten screening time, and reduce the number of times an empirically based development cycle must be run by eliminating some of the research-related “dead ends,” which add significant time and cost to the discovery phase. Exhibit 2 shows where QC-enhanced CADD would improve the development cycle.

QC could make current CADD tools more effective by helping to predict molecular properties with high accuracy. That can affect the development process in several ways, such as modeling how proteins fold and how drug candidates interact with biologically relevant proteins. Here, QC may allow researchers to screen computational libraries against multiple possible structures of the target in parallel. Current approaches usually restrict the structural flexibility of the target molecule due to a lack of computational power and a limited amount of time. These restrictions may reduce the chances of identifying the best drug candidates.

In the longer term, QC may improve generation and validation of hypotheses by using machine-learning (ML) algorithms to uncover new structure-property relationships. Once it has reached sufficient maturity, QC technology may be able to create new types of drug-candidate libraries that are no longer restricted to small molecules but also include peptides and antibodies. It could also enable a more automated approach to drug discovery, in which a large structural library of biologically relevant targets is automatically screened against drug-like molecules via high-throughput approaches.

One could even envision QC triggering a paradigm shift in pharmaceutical R&D, moving beyond today’s digitally enabled R&D toward simulation-based or in silico drug discoveries—a trend that has been seen in other industries as well.

The following QC use cases apply to different aspects of drug discovery and will emerge at different points over an extended timeline. All of them, however, may enable more accurate and efficient development of targeted compounds.

Target identification and validation

During target identification, QC can be leveraged to reliably predict the 3-D structures of proteins. Obtaining high-quality structural data is a lengthy process often leading to low-quality results. Despite all efforts, researchers have yet to crystallize many biologically important proteins—be it due to their size, solubility (for example, membrane proteins), or inability to express and purify in sufficient amount. Pharma companies sometimes develop drugs without even knowing the structure of a protein—accepting the risk of a trial-and-error approach in subsequent steps of drug development—because the business case for a given drug is potentially so strong.

AlphaFold, developed by Google’s DeepMind, was a breakthrough in AI-driven protein folding but has not resolved all of the challenges of classical computing-based simulation, including, for example, formation of protein complexes, protein-protein interactions, and protein-ligand interactions. It’s the interactions that are most difficult to classically solve and, thus, may benefit from QC, which allows for the explicit treatment of electrons. Additionally, QC may allow for strong computational efficiencies here given that Google’s AI model—which is trained on around 170,000 different structures of protein data—requires more than 120 high-end computers for several weeks.

Hit generation and validation

QC’s ability to parallel process complex phenomena would be particularly valuable during hit generation and validation. With existing computers, pharma companies can only use CADD on small to medium-size drug candidates and largely in a sequential manner. Computing power is the bottleneck. With powerful enough QC, pharma companies would be able to expand all use cases to selected biologics as well, for instance, semi-synthesized biologics or fusion proteins, and perform in silico search and validation experiments in a more high-throughput fashion. This use case would go beyond the identification of the protein and eventually encompass almost the entire known biological world. With a robust enough hit-generation and validation approach, this step would already deliver potential lead molecules that are much easier and quicker to optimize.

Lead optimization

During lead optimization, which is a top-three parameter to improve R&D productivity, 1 Steven M. Paul et al., “How to improve R&D productivity: The pharmaceutical industry’s grand challenge,” Nature Reviews Drug Discovery , March 2010, Volume 9, pp. 203–214, nature.com. QC may allow for enhanced absorption, distribution, metabolism, and excretion (ADME); more accurate activity and toxicity predictions for organ systems; dose and solubility optimization; and other safety issues.

Data linkage and generation

The metalevel of R&D very much consists of linking appropriate data together—for instance, creating sensible connections between data points through effective (semantic) management. The more complex the biological information that can be processed, the more extensive the graphs that inform the drug discovery research process become. There is currently research on “topological data analysis” under way that aims to identify “holes” and “connections” across large data sets. 2 Silvano Garnerone, Seth Lloyd, and Paolo Zanardi, “Quantum algorithms for topological and geometric analysis of data,” Nature Communications , January 2016, Volume 7, Article 10138, nature.com. This may at some point enable R&D specialists to identify concrete cases and “industry verticals” where such algorithms are applicable, for example, in identifying connections across brain cells in response to a drug.

Moreover, QC could be used to “deepfake” missing data points throughout the research process, that is, generate a type of fake data by using ML algorithms. This could be particularly useful wherever there is a scarcity of data, such as in rare diseases, that can then be mitigated through artificial data sets. QC will set a new bar here regarding speed in training ML models, amount of initial data needed, and level of accuracy.

Clinical trials

Clinical trials could be optimized through patient identification and stratification and population pharmacogenetic modeling. 3 Paul et al., 2010. In trial planning and execution, QC could optimize the selection of the trial sites. QC could also augment causality analyses for side effects to improve active safety surveillance.

Beyond research and development

While the potential value of QC in pharma R&D is immense, it will also likely play a role further down the value chain. In the production of active ingredients, QC may aid in the calculation of reaction rates, optimize catalytic processes, and, ultimately, create significant efficiencies in the development of new product formulations. In the business-related value pools, QC in pharma could include the optimization of logistics (for instance, the optimization of on-site flows of materials, heat, and waste in production facilities) and improvements in the supply chain. Finally, toward market access and commercial, QC may even enable automatic drug recommendations.

Rollout of QC in pharma R&D will occur over two clear time horizons, characterized by a gradual tech transition

The development of quantum computers began nearly four decades ago, but it is the gains in QC technology realized over the past few years that paved the way for practical applications in pharma. We see the key, value-adding QC activities in pharma unfolding over two distinct eras as the technology further matures (Exhibit 3):

• From 2020–30: Not fully error-corrected QC. Early commercial activities related to quantum computing are already under way as we leave the first horizon—which focused on quantum-inspired algorithms over the past 40 years—and enter the horizon of not fully error-corrected QC. Often referred to as “noisy intermediate-scale quantum” (NISQ), this phase describes the not-error-corrected characteristics of near-term devices that are based on an initially considerable number of quantum bits (qubits) to solve problems classic computers can’t solve yet and do not provide fault tolerance. The timeline for the development and implementation of QC technology, and its adoption across companies, is very much under debate. NISQ, as a class of probabilistic computers that still (mostly) produce error-prone results, may potentially provide a near-term solution for a limited set of use cases. Companies eyeing QC’s potential should take this uncertainty into consideration.
• Beyond 2030: Fully error-corrected QC. Beyond 2030, fully error-corrected QC is expected, in which full value through QC will be captured. In this horizon, QC gets implemented at scale, and later adopters also implement the technology. In other words, chemicals players may start creating value with QC by the mid-2020s ; pharma companies are expected to move more solidly into the space shortly thereafter. Compared with the chemical industry, pharma researchers primarily target more complex and larger molecular systems, which can’t be replicated with either high-performance computers or today’s limited quantum computers.

Exactly when a particular company begins to capture QC’s benefits will depend on its tech starting point (that is, its current level of R&D digitization) and its business focus: the number of small active pharmaceutical ingredients (APIs) in its portfolio. Pharma companies that have a strong footprint in CADD and focus their R&D on smaller molecules will be among the first to take advantage of emergent QC. Exhibit 4 maps key CADD methods along the drug-discovery continuum and offers an indication of the applicability of QC. It’s expected that QC will be mostly applicable in the discovery phase of hit generation, hit-to-lead, and also in lead optimization.

In the next five to ten years, we expect that the first QC tools pharma players deploy will rely on hybrid methodologies that use classical algorithms alongside QC subroutines when they can create additional value. The prominent examples are the imaginary time evolution (an algorithm to find the ground-state and excited-state energy of many-particle systems) and the variational quantum eigen-solver, or VQE (an algorithm to calculate the binding affinity between an API and a target receptor). The value that algorithms such as VQE will add depends on the size of the quantum hardware. Describing small-molecule drugs generally requires less-mature quantum computers, while biologicals will be tackled only as QC matures.

Taking steps now can position pharma players for QC success later

The pharma sector is well positioned to take full advantage of this opportunity. Its tech-ready culture already embraces a wide array of digital tools: CADD, AI, ML, and non-QC DFT- and MD-simulation tools already play a big role in the sector’s R&D. On top of this, pharma players are already working with quantum-chemical simulations, so the barrier to entry is quite low. Scientists will not have to change the way they develop drugs in any fundamental way—they will just be working with more capable tools.

That said, companies will make their own decisions regarding whether and how to move toward a QC-enabled business. Some pharma players may take a pass on deploying QC, others may wait and observe, while still others are going “all in,” ginning up early in-house development. Most pharma players, however, will likely undertake joint-development strategies with upstream players. No matter what, answering some key strategic questions will help companies make more informed decisions on their stance for QC.

Assess the opportunity

Pharmaceutical companies should assess QC now and potentially lay the groundwork to reap the benefits of the technology later. QC may give many of them a huge opportunity, yet each pharma player needs to figure out how much exposure it has and the size of its QC opportunity in the context of its current pace of development. Thus, pharma players should consider three key strategic questions to determine their optimal QC strategy (Exhibit 5):

• Will QC demonstrate promise to disrupt my area of play and reorganize the competitive landscape?
• Have I identified opportunities in my value chain where QC’s potential may translate into value and in which time horizon?
• Can I dedicate resources to investigate QC opportunities, and can I scale up capabilities?

Subject to the above answers, moving early can help secure valuable intellectual property for the algorithms that drive QC and can also address a key issue: pharma won’t be the first industry sector to benefit from QC, so late-moving players could face a lack of suitable talent.

Establish partnerships

Some pharmaceutical players have already realized the need to join forces on the topic of QC and have started to collaborate and/or form partnerships. For example, QuPharm formed in late 2019 by major pharmaceutical players to pool ideas and expertise around QC use cases. QuPharm also collaborates with the Quantum Economic Development Consortium (QED-C), which was created in 2018 by the US government as part of the National Quantum Initiative Act and aims to enable commercial QC use-case efforts. Additionally, the Pistoia Alliance is a life sciences membership organization, which was organized to facilitate precompetitive collaboration and foster R&D innovation.

Partnering with pure quantum players taps into their existing expertise to test early use cases and facilitate development. At the moment, there are more than 100 QC-focused companies—both start-ups and established firms—around the world, focusing on software, hardware, or enabling services. Approximately 25 companies are targeting applications in the pharma industry. Less than 15 focus on algorithms or solutions for pharma players, and very few are focusing exclusively on the needs of pharma players.

Develop capabilities

Digital talent gaps are already a reality, and QC may only exacerbate them. Unlike other important digital tools, such as AI, quantum computing depends on niche know-how. Pharma companies already struggle to attract people with capabilities in the less specialized digital technologies, and hiring quantum-computing experts may prove to be even more of a challenge.

Ensure organizational collaboration

A pharma company’s “way of working” will also be central to its success in QC. The traditional walls that separate the work of the organization’s various functions and units—for example, research, tech, business—will have to fall away. Cross-functional collaboration in both spirit and action will characterize the pharma companies that are able to take full advantage of QC.

Quantum computing could be the key to exponentially more efficient discovery of pharmaceutical cures and therapeutics as well as to hundreds of billions of dollars in value for the pharma industry. Experts predict, for example, that today’s $200 billion market for protein-based drugs could grow by 50 to 100 percent  in the medium term if better tools to develop them became available. Given QC’s vast potential, we expect global pharma spending on QC in R&D to be in the billions by 2030. Pharma companies would be well advised to assess the QC opportunity for themselves and begin laying the groundwork in securing their place in this new competitive and technological landscape.

Matthias Evers is a senior partner in McKinsey’s Hamburg office, and Anna Heid is a consultant in the Zurich office, where Ivan Ostojic is a partner.

The authors wish to thank Nicole Bellonzi, Matteo Biondi, Thomas Lehmann, Lorenzo Pautasso, Katarzyna Smietana, Matija Zesko, and the many industry/academia experts for their contributions to this article.

Explore a career with us

Related articles.

A game plan for quantum computing

Quantum computing is coming: How can your company prepare?

• How To Run A Games User Research Playtest
• All Articles
• Book A Study
• The Playtest Kit
• The 2023 Playtest Survey
• Find an Games UX Agency or Consultant
• Games UX Jobs
• Talks, Articles & Books
• Free Student Support

Preparing a games user research study

Preparing a games user research study requires a lot of admin to make sure the right people turn up, the right version gets tested and that the study runs smoothly. Learn how to recruit participants, and prepare a great study.

Last updated: January 1, 2021

This is AN EXTRACT FROM THE gAMES USER RESEARCH BOOK

The ultimate resource for aspiring or junior researchers who want to start a career in games. Learn how to run professional quality playtests, improve the UX of games and make games players love.

Start running playtests, getting job interviews, and making games better today.

Learn more about the book

Preparation before running a study will ensure the sessions run smoothly and successfully answer the research objectives. The preparation needed to run a games user research study is very similar to most types of user research study – with perhaps more complexity in the technical setup and code screening required, as we will see. 

Having  earlier covered designing a study,  in this section we’ll cover all of the main stages required to prepare for running the study, and highlight some of the challenges specific to researching video games. This includes:

• Finding research participants
• Understanding the game and the build
• The technology
• The paperwork
• Preparing the game team

Recruiting games research participants

Most research methods need users to take part in the study. This requires finding the type of people who would play the game once it’s released, inviting them to take part in the study, and then making sure they turn up.

This takes a lot of coordination and time, and it is sensible to start recruitment as soon as the study plan is complete. At this point, it will be clear how many users are needed and the session length. The amount of time recruitment takes varies based on the method used but can be between a few days and a few weeks.

Finding the right players

We  previously covered how to run kick-off meetings . In this meeting agree who the participants of the study should be, which will prevent arguments later about the participants not being representative or suitable. Recruitment should be aligned with who the game is aimed at, and can be informed by data from other teams, such as the marketing department. Some sensible criteria to recruit on include:

• Experience with other games (e.g. This is a high budget first person shooter, so we’re looking for people who have played other big FPS releases from last year).
• Familiarity with previous games in the series (e.g. This is a sequel to a popular sports franchise, so we’re looking for people who played the last edition)
• Demographic information, to some extent – for example if it’s aimed at children. Recruiting based on age and gender amongst adults is not recommended because it risks reinforcing inaccurate stereotypes about who games are for, and furthers representation issues.

It’s important to spend time making sure that the study is looking for the right kind of players. Getting the wrong kind of players means that the study’s findings won’t be relevant to the real game design decisions being made. If a participant has never used dual-stick controls before they will have a bad time in a study looking at a first person shooter which assumes prior experience with those controls. This may be useful when designing tutorials, but isn’t relevant when testing other parts of the game, since the audience is meant to be experienced first person shooter players. 

Screening participants

To make sure that the participants are appropriate, it’s necessary to screen them before inviting them to take part in the study. Screening is the process of checking that they match the correct criteria for participation in that study. It’s reasonably common for people to sign up to take part in studies they are not suitable for, because of the money offered to take part, so checking they are suitable is essential to avoid mis-recruits. This can often take multiple stages:

• A questionnaire asking for information about what games they play, or checking other screening criteria, written so it’s not obvious which answer is the correct one to take part in the study. This questionnaire can also check their availability for the intended dates and times of the study.
• Some teams also phone participants prior to confirming their session. This conversation covers more in-depth questions to check they really have played the games they say they have. 

Having checked they are the right kind of person to take part in a study, people can then be booked in to the time and dates needed for the study.

Making sure people turn up

No-shows, when a participant fails to turn up, are expensive and embarrassing. Expensive because it wastes the time of the researcher and the designers or producers observing. Embarrassing, because everyone then has nothing to do, and are sat in an observation room twiddling their thumbs until the next session is scheduled to begin. To reduce the risk of people not turning up, there are three tactics to consider using.

When participants fail to turn up, this is an embarrassing and expensive error!

Firstly participants should be incentivised – given money, or something close to money such as vouchers, to pay them for taking part. This makes people more likely to bother to turn up, rather than deciding they can’t be bothered on the day. It also helps find more ‘normal’ users – people who will do it for free are likely to be either much bigger fans, or much more vocal critics, than an average player. Recruiting only unpaid participants will introduce a sampling bias in the kind of users taking part in your research sessions.

Secondly, consider recruiting one or more spare participants. This means booking one more participant than the study requires – either to wait around all day, or as an extra session at the end of the day. If people do fail to show up, the spare can be used to make up for the missing participant.

Send confirmed participants a calendar invite and phone them the day before to remind them the study is taking place, and the time and place it will occur. This will help avoid the session being accidentally forgotten, or misunderstandings about when and where they are meant to go.

Finding the right participants takes a lot of work and is a specialist skill. Many research teams either have a dedicated team member hired to handle this or outsource it to an external participant recruitment company. Recruiting participants is complex, and perhaps the most suitable thing to outsource if budget is available to do so.

Understand the game and the build

Identifying usability and experiential issues within a game requires understanding the game very well. This not only covers understanding the intention of the game – how it’s meant to work – but also the state of the build when it will be tested. This is sometimes called code screening.

Running studies prior to the game being finished will mean that the version being tested is incomplete. Also, because games are complex systems, it is very likely that there will be bugs within the version being tested. Often game teams will want to create a custom release (build) for the test. When preparing the study, researchers need to play this test version of the game enough to understand what it’s contents are, what the bugs are in it, and how to overcome any bugs or incomplete sections so that this can be handled while moderating a session. 

It’s also sensible to check the test build against the study’s objectives – is the required content in there that allows the objectives to be answered? If not it’ll require a negotiation with colleagues to decide whether to change the test objectives or provide an updated build. 

Understand the design intent

As well as understanding the version being tested, researchers also need to understand the goal of the game and the design intent behind it (hopefully the designers have thought about it!). The study design will require some observations to be made during the sessions (for example, where players get lost, or how many times they die). For each of these, understanding the design intent is necessary to help decide whether an issue exists. This can require a lot of conversations with the people who designed these features or scenarios to help uncover how they want the player to experience them, such as how many  times they want players to fail before solving a puzzle. It also might require designers to articulate and quantify their design intent, which they might not be particularly experienced with. Building close relationships with colleagues is once again particularly important to running successful studies.

Researchers need to understand the design intent, to earn trust

Time spent understanding the game, and the design decisions that have gone into it, will help a researcher run a successful analysis of the data that comes out of a study. This allows them to reliably identify the most important issues for the team. This preparation will also help researchers lead more useful discussions around fixing issues, informed by an understanding of what has occurred before, which will be explained later in this book.

Preparing the technology

Testing video games will inevitably require being comfortable with technology – perhaps more so than in other domains of software development. In addition to preparing the version of the game being tested, there is also technical setup required to record or stream what happens in the study.

Getting the build ready

How to prepare the build will depend on the hardware being used for testing – such as whether it’s a console, mobile or PC game. Regardless of the specific process, a researcher will need to install the build and test that it is working appropriately. Because the software is still in development and hasn’t gone through QA, there is a reasonable chance that technical problems will occur when installing the game into the test environment. Leave plenty of time to trial and troubleshoot this. 

There is a tradeoff that will need to be negotiated with the game team – often they will want to provide the final build as late as possible so they can continue to make changes, but the researcher will want enough time to install it and check that it works as intended. Giving a  deadline for the final build to be provided two days before testing starts can be a good compromise, giving the researcher enough time to react to problems. 

Setting up the research lab technology

A research study also requires some custom technology to allow the session to be observed or recorded. Many research teams have permanent spaces (research labs) that allow them to do this without having to set up the recording technology each time. Smaller research teams might have to put up with using a meeting room for running their studies. A second room will also be useful to set up an observation room – a dedicated space to encourage teams to view live. Offering live viewing increases colleagues’ engagement and understanding of what’s occurring in the sessions, and amplifies the impact of the studies. If a dedicated room isn’t possible, many tech setups allow the sessions to be streamed live to people’s desks.

Technical setups for recording video from the sessions that combine what happens in the game, with video and audio recorded in the room can be done reasonably cheaply, using a combination of screen sharing software, HDMI splitters and recording software. There are many guides online about how to build a user research lab that supports recording and streaming video for testing one participant at a time, and I covered some potential setups in my previous book  Building User Research Teams . 

A difference between games user research and other industries is the need to host many players playing simultaneously, to support some of the methods explained previously. This greatly increases the technical complexity, and some advanced lab setups that support these studies have been covered by Seb Long in the  Games User Research  book, and at the  #GamesUR Summit , the videos from which are available on YouTube on the GRUX SIG channel.

Preparing the space

As well as setting up the room to handle the technical requirements for running the study, it’s also important to think about the impact of the room on the participant’s experience, and the areas the participant moves through to reach that room. The room itself should be neutral and avoid intimidating or biasing the participant. Avoid an overly clinical aesthetic that makes people feel they are being watched in a laboratory, and avoid heavy brand marketing that may change people’s opinions about what they are playing. 

Ensuring that the participant’s experience is considered and curated throughout their interactions with the study will help create a more comfortable and natural environment. This might include giving them specific information about what to do on arrival, thinking about where they get taken when they arrive, and briefing any reception staff so that they handle the participants appropriately.

In addition to preparing the room that the participants will be in during a study, the space for observation should also be prepared. If the technical setup supports live streaming, it’s often sensible to book a space where observers can view it communally. This encourages discussion between members of the development team and allows members of the research team to sit in and help guide that discussion. Preparing this space can involve ensuring that the video stream can be seen by all using a projector or large screen, creating space where post-its can be captured, and providing refreshments to encourage attendance. Booking meeting rooms can often be difficult in busy offices, so try and reserve them with plenty of time in advance.

Preparing the paperwork

Running user research studies generates a lot of paperwork, and it’s easy to forget to make and print these in the lead up to a study. These documents can include:

• Pre-study information to be given to participants
• Information to be given to the building’s reception desk.
• Non-disclosure agreements for participants
• Consent forms for participants
• Any surveys or questionnaires to be distributed to participants
• A method to capture the notes during the study

Templates for many of these are available for free on the website for my previous book,  Building User Research Teams  but we’ll explain each in brief now.

Prepare participants before they arrive

Pre-study information for participants should be emailed in advance, and tell them when and where the study will occur, as well as other logistical information such as travel options, parking or what to do when they arrive. As discussed previously, no-shows can be both expensive and embarrassing for a researcher, and by making sure participants have the information to hand, and know what to expect, it will help reduce the chance of this happening. 

Creating a positive first impression

When participants arrive at the front-desk of the building, ensuring they have a positive experience will help reduce their anxiety and encourage more natural behaviour. It’s reasonably intimidating to go to a big corporate office, and making sure that any reception staff understand who they are and how to handle their arrival will make the experience less scary. To help with this, create a document for reception staff that describes who will be arriving and what actions to take on their arrival – who to call, and where to ask the participant to wait.

Consent and non disclosure

As covered in the first part of this book, secrecy is considered very important for many games with extensive marketing strategies. Create a non-disclosure agreement in collaboration with a legal team, and ask participants to sign it. This will help discourage leaks, and increase the studio’s confidence that running user research studies is safe. 

Running ethical user research requires the participants to understand what the study is about, what information will be gathered, and how their data will be stored and used. This is often handled by combining an in-person briefing from the moderator with a document that the participant can read, sign, and keep a copy of. Giving this information on a document with a verbal briefing helps ensure that the participant has understood and is giving informed consent – an essential ethical requirement. Prepare a document that explains the high-level goal of the study (without revealing too many details that may impact their behaviour), for example “ We are interested in learning about your experience with the game to help improve it” . The document should also list the data that will be captured – e.g. audio recordings, video recordings, their survey responses. The document should also give instructions for how they can request a copy of their data or remove consent at a later point. 

Preparing questionnaires and surveys

There are two types of questionnaire that might be useful to prepare for a study, and these can be done on paper or on a computer using survey tools such as Qualtrics. Although the participants should have been screened before they were invited to participate, it can be useful to reconfirm their habits around what games they play at the start of a session. This can help identify mis-recruits where the wrong person has been invited, participants have lied, or someone other than the invited person has turned up. It can also make it simpler to use the data about their playing habits as part of the analysis, since they will have consented to that data being collected and used on the consent form. Questionnaires will also need to be prepared and distributed when the study design requires surveys either during gameplay (for example after every level), or at the end of the session.

How will notes be taken?

The last thing to prepare is the method of note-taking that will be used by any researchers working on the study. For structured studies, where the things being observed are all identifiable in advance, some teams like to use a spreadsheet for note-taking. For more unstructured studies, where the player has greater autonomy over what they do it’s not possible to anticipate in advance what feedback will be collected. In these situations mind maps can greatly speed up data collection and analysis. Some details on how to do this are also covered in my previous book,  Building User Research Teams . 

Also don’t forget to print out all of the above paperwork, and the discussion guide created in the previous section, before the study starts!

Preparing the game team 

When running user research, it is important to ensure that the game team feels involved with the study. Their active participation increases the likelihood that the research study will impact their decision making, and justify the investment in running studies. As covered earlier, active engagement with the team to decide the research objectives, and working together to make sure the researcher understands the design intent is essential to running a useful study and interpreting the data correctly. 

One of the easiest ways to fail to get buy-in is by not involving them in the study being run. At the minimum, user researchers should be inviting them to view research sessions, and invite them to a debrief where the results are discussed. Sending these invites can be easily forgotten, and people have very full calendars, so invites should be sent early in study preparation. Immediately before the study, remind the game team that the study is occurring, what the objectives are, and how they can observe the sessions will help increase engagement.

With a more mature team and an experienced researcher, once a relationship has been established and their understanding of research has improved, more exciting collaborations can be explored – such as collaboratively analysing the data from a study to come up with the results together. 

Run a pilot test

Plenty of things can and do go wrong when running a study. These could include:

• The build crashes and players lose their saved game
• Bugs occur in the game that make progress impossible
• The recording device fails to record
• The microphone wasn’t on, and the interview doesn’t get picked up
• The consent forms weren’t printed out
• The survey has the wrong scale on it
• The task set to participants was misunderstood, and people played the wrong part of the game.

In order to reduce the chance that these issues disrupt the real study, it’s very important to run a pilot study. This is a practise run of the study, using a pretend participant (usually a colleague), pretending they are a real participant and running through the complete study. It’s tempting to skip bits during a pilot – e.g. not asking the fake participant to fill out the consent form, or playing less of the game than a real participant would. That can be necessary when time is short, but each skipped part increases the risk of not noticing a problem with the study until it’s too late.

Running the pilot the day before the study is due to begin means that the real test build can be used, avoiding the risk of bugs emerging between the pilot and the real test. The day before still gives enough time to react to most technical or study design issues that might emerge. 

More on Preparing a Games User Research Study.

In this section, we’ve touched on a lot of the tasks that a researcher will be doing to prepare to run a successful study. We’ve only scratched the surface, and there is a lot more work that can be done to describe and optimise these processes, as well as other considerations such as the secure handling of personal data.  The processes required for running user research studies are covered in more depth in my previous book  Building User Research Teams , which might be a helpful resource when establishing research at a games studio that hasn’t done it before.

Integrate player insight throughout development

Every month, get sent the latest article on how to plan and run efficient high quality playtests to de-risk game development. And get Steve Bromley’s free course today on how to get your first 100 playtesters for teams without much budget or time.

Plus y our free early-access copy of ‘Playtest Plus’ – the essential guide to the most impactful playtests to run throughout development of your game

Meet the author

Steve Bromley is an expert user researcher, who works with studios of all sizes to run playtests , and integrate user research into the game development process .

Keep Exploring

How much playtesting is enough?

When to start (and stop) playtesting throughout game development.

Scope your studies to increase research impact

How to scope games user research studies for impact – uncovering the context for decision making, and setting sensible objectives.

How to budget for games user research

How much does user research cost, what to budget for user research, and how best to spend the budget you have to de-risk game development.

Master Games User Research

Free monthly new articles teaching playtesting & how to be a games user researcher. Join the most interesting conversations about games user research, discover job opportunities, and be introduced to new ways to think about game development.

Human Subjects Office

Medical terms in lay language.

Please use these descriptions in place of medical jargon in consent documents, recruitment materials and other study documents. Note: These terms are not the only acceptable plain language alternatives for these vocabulary words.

This glossary of terms is derived from a list copyrighted by the University of Kentucky, Office of Research Integrity (1990).

For clinical research-specific definitions, see also the Clinical Research Glossary developed by the Multi-Regional Clinical Trials (MRCT) Center of Brigham and Women’s Hospital and Harvard  and the Clinical Data Interchange Standards Consortium (CDISC) .

Alternative Lay Language for Medical Terms for use in Informed Consent Documents

A   B   C   D   E   F   G   H   I  J  K   L   M   N   O   P   Q   R   S   T   U   V   W  X  Y  Z

ABDOMEN/ABDOMINAL body cavity below diaphragm that contains stomach, intestines, liver and other organs ABSORB take up fluids, take in ACIDOSIS condition when blood contains more acid than normal ACUITY clearness, keenness, esp. of vision and airways ACUTE new, recent, sudden, urgent ADENOPATHY swollen lymph nodes (glands) ADJUVANT helpful, assisting, aiding, supportive ADJUVANT TREATMENT added treatment (usually to a standard treatment) ANTIBIOTIC drug that kills bacteria and other germs ANTIMICROBIAL drug that kills bacteria and other germs ANTIRETROVIRAL drug that works against the growth of certain viruses ADVERSE EFFECT side effect, bad reaction, unwanted response ALLERGIC REACTION rash, hives, swelling, trouble breathing AMBULATE/AMBULATION/AMBULATORY walk, able to walk ANAPHYLAXIS serious, potentially life-threatening allergic reaction ANEMIA decreased red blood cells; low red cell blood count ANESTHETIC a drug or agent used to decrease the feeling of pain, or eliminate the feeling of pain by putting you to sleep ANGINA pain resulting from not enough blood flowing to the heart ANGINA PECTORIS pain resulting from not enough blood flowing to the heart ANOREXIA disorder in which person will not eat; lack of appetite ANTECUBITAL related to the inner side of the forearm ANTIBODY protein made in the body in response to foreign substance ANTICONVULSANT drug used to prevent seizures ANTILIPEMIC a drug that lowers fat levels in the blood ANTITUSSIVE a drug used to relieve coughing ARRHYTHMIA abnormal heartbeat; any change from the normal heartbeat ASPIRATION fluid entering the lungs, such as after vomiting ASSAY lab test ASSESS to learn about, measure, evaluate, look at ASTHMA lung disease associated with tightening of air passages, making breathing difficult ASYMPTOMATIC without symptoms AXILLA armpit

BENIGN not malignant, without serious consequences BID twice a day BINDING/BOUND carried by, to make stick together, transported BIOAVAILABILITY the extent to which a drug or other substance becomes available to the body BLOOD PROFILE series of blood tests BOLUS a large amount given all at once BONE MASS the amount of calcium and other minerals in a given amount of bone BRADYARRHYTHMIAS slow, irregular heartbeats BRADYCARDIA slow heartbeat BRONCHOSPASM breathing distress caused by narrowing of the airways

CARCINOGENIC cancer-causing CARCINOMA type of cancer CARDIAC related to the heart CARDIOVERSION return to normal heartbeat by electric shock CATHETER a tube for withdrawing or giving fluids CATHETER a tube placed near the spinal cord and used for anesthesia (indwelling epidural) during surgery CENTRAL NERVOUS SYSTEM (CNS) brain and spinal cord CEREBRAL TRAUMA damage to the brain CESSATION stopping CHD coronary heart disease CHEMOTHERAPY treatment of disease, usually cancer, by chemical agents CHRONIC continuing for a long time, ongoing CLINICAL pertaining to medical care CLINICAL TRIAL an experiment involving human subjects COMA unconscious state COMPLETE RESPONSE total disappearance of disease CONGENITAL present before birth CONJUNCTIVITIS redness and irritation of the thin membrane that covers the eye CONSOLIDATION PHASE treatment phase intended to make a remission permanent (follows induction phase) CONTROLLED TRIAL research study in which the experimental treatment or procedure is compared to a standard (control) treatment or procedure COOPERATIVE GROUP association of multiple institutions to perform clinical trials CORONARY related to the blood vessels that supply the heart, or to the heart itself CT SCAN (CAT) computerized series of x-rays (computerized tomography) CULTURE test for infection, or for organisms that could cause infection CUMULATIVE added together from the beginning CUTANEOUS relating to the skin CVA stroke (cerebrovascular accident)

DERMATOLOGIC pertaining to the skin DIASTOLIC lower number in a blood pressure reading DISTAL toward the end, away from the center of the body DIURETIC "water pill" or drug that causes increase in urination DOPPLER device using sound waves to diagnose or test DOUBLE BLIND study in which neither investigators nor subjects know what drug or treatment the subject is receiving DYSFUNCTION state of improper function DYSPLASIA abnormal cells

ECHOCARDIOGRAM sound wave test of the heart EDEMA excess fluid collecting in tissue EEG electric brain wave tracing (electroencephalogram) EFFICACY effectiveness ELECTROCARDIOGRAM electrical tracing of the heartbeat (ECG or EKG) ELECTROLYTE IMBALANCE an imbalance of minerals in the blood EMESIS vomiting EMPIRIC based on experience ENDOSCOPIC EXAMINATION viewing an  internal part of the body with a lighted tube  ENTERAL by way of the intestines EPIDURAL outside the spinal cord ERADICATE get rid of (such as disease) Page 2 of 7 EVALUATED, ASSESSED examined for a medical condition EXPEDITED REVIEW rapid review of a protocol by the IRB Chair without full committee approval, permitted with certain low-risk research studies EXTERNAL outside the body EXTRAVASATE to leak outside of a planned area, such as out of a blood vessel

FDA U.S. Food and Drug Administration, the branch of federal government that approves new drugs FIBROUS having many fibers, such as scar tissue FIBRILLATION irregular beat of the heart or other muscle

GENERAL ANESTHESIA pain prevention by giving drugs to cause loss of consciousness, as during surgery GESTATIONAL pertaining to pregnancy

HEMATOCRIT amount of red blood cells in the blood HEMATOMA a bruise, a black and blue mark HEMODYNAMIC MEASURING blood flow HEMOLYSIS breakdown in red blood cells HEPARIN LOCK needle placed in the arm with blood thinner to keep the blood from clotting HEPATOMA cancer or tumor of the liver HERITABLE DISEASE can be transmitted to one’s offspring, resulting in damage to future children HISTOPATHOLOGIC pertaining to the disease status of body tissues or cells HOLTER MONITOR a portable machine for recording heart beats HYPERCALCEMIA high blood calcium level HYPERKALEMIA high blood potassium level HYPERNATREMIA high blood sodium level HYPERTENSION high blood pressure HYPOCALCEMIA low blood calcium level HYPOKALEMIA low blood potassium level HYPONATREMIA low blood sodium level HYPOTENSION low blood pressure HYPOXEMIA a decrease of oxygen in the blood HYPOXIA a decrease of oxygen reaching body tissues HYSTERECTOMY surgical removal of the uterus, ovaries (female sex glands), or both uterus and ovaries

IATROGENIC caused by a physician or by treatment IDE investigational device exemption, the license to test an unapproved new medical device IDIOPATHIC of unknown cause IMMUNITY defense against, protection from IMMUNOGLOBIN a protein that makes antibodies IMMUNOSUPPRESSIVE drug which works against the body's immune (protective) response, often used in transplantation and diseases caused by immune system malfunction IMMUNOTHERAPY giving of drugs to help the body's immune (protective) system; usually used to destroy cancer cells IMPAIRED FUNCTION abnormal function IMPLANTED placed in the body IND investigational new drug, the license to test an unapproved new drug INDUCTION PHASE beginning phase or stage of a treatment INDURATION hardening INDWELLING remaining in a given location, such as a catheter INFARCT death of tissue due to lack of blood supply INFECTIOUS DISEASE transmitted from one person to the next INFLAMMATION swelling that is generally painful, red, and warm INFUSION slow injection of a substance into the body, usually into the blood by means of a catheter INGESTION eating; taking by mouth INTERFERON drug which acts against viruses; antiviral agent INTERMITTENT occurring (regularly or irregularly) between two time points; repeatedly stopping, then starting again INTERNAL within the body INTERIOR inside of the body INTRAMUSCULAR into the muscle; within the muscle INTRAPERITONEAL into the abdominal cavity INTRATHECAL into the spinal fluid INTRAVENOUS (IV) through the vein INTRAVESICAL in the bladder INTUBATE the placement of a tube into the airway INVASIVE PROCEDURE puncturing, opening, or cutting the skin INVESTIGATIONAL NEW DRUG (IND) a new drug that has not been approved by the FDA INVESTIGATIONAL METHOD a treatment method which has not been proven to be beneficial or has not been accepted as standard care ISCHEMIA decreased oxygen in a tissue (usually because of decreased blood flow)

LAPAROTOMY surgical procedure in which an incision is made in the abdominal wall to enable a doctor to look at the organs inside LESION wound or injury; a diseased patch of skin LETHARGY sleepiness, tiredness LEUKOPENIA low white blood cell count LIPID fat LIPID CONTENT fat content in the blood LIPID PROFILE (PANEL) fat and cholesterol levels in the blood LOCAL ANESTHESIA creation of insensitivity to pain in a small, local area of the body, usually by injection of numbing drugs LOCALIZED restricted to one area, limited to one area LUMEN the cavity of an organ or tube (e.g., blood vessel) LYMPHANGIOGRAPHY an x-ray of the lymph nodes or tissues after injecting dye into lymph vessels (e.g., in feet) LYMPHOCYTE a type of white blood cell important in immunity (protection) against infection LYMPHOMA a cancer of the lymph nodes (or tissues)

MALAISE a vague feeling of bodily discomfort, feeling badly MALFUNCTION condition in which something is not functioning properly MALIGNANCY cancer or other progressively enlarging and spreading tumor, usually fatal if not successfully treated MEDULLABLASTOMA a type of brain tumor MEGALOBLASTOSIS change in red blood cells METABOLIZE process of breaking down substances in the cells to obtain energy METASTASIS spread of cancer cells from one part of the body to another METRONIDAZOLE drug used to treat infections caused by parasites (invading organisms that take up living in the body) or other causes of anaerobic infection (not requiring oxygen to survive) MI myocardial infarction, heart attack MINIMAL slight MINIMIZE reduce as much as possible Page 4 of 7 MONITOR check on; keep track of; watch carefully MOBILITY ease of movement MORBIDITY undesired result or complication MORTALITY death MOTILITY the ability to move MRI magnetic resonance imaging, diagnostic pictures of the inside of the body, created using magnetic rather than x-ray energy MUCOSA, MUCOUS MEMBRANE moist lining of digestive, respiratory, reproductive, and urinary tracts MYALGIA muscle aches MYOCARDIAL pertaining to the heart muscle MYOCARDIAL INFARCTION heart attack

NASOGASTRIC TUBE placed in the nose, reaching to the stomach NCI the National Cancer Institute NECROSIS death of tissue NEOPLASIA/NEOPLASM tumor, may be benign or malignant NEUROBLASTOMA a cancer of nerve tissue NEUROLOGICAL pertaining to the nervous system NEUTROPENIA decrease in the main part of the white blood cells NIH the National Institutes of Health NONINVASIVE not breaking, cutting, or entering the skin NOSOCOMIAL acquired in the hospital

OCCLUSION closing; blockage; obstruction ONCOLOGY the study of tumors or cancer OPHTHALMIC pertaining to the eye OPTIMAL best, most favorable or desirable ORAL ADMINISTRATION by mouth ORTHOPEDIC pertaining to the bones OSTEOPETROSIS rare bone disorder characterized by dense bone OSTEOPOROSIS softening of the bones OVARIES female sex glands

PARENTERAL given by injection PATENCY condition of being open PATHOGENESIS development of a disease or unhealthy condition PERCUTANEOUS through the skin PERIPHERAL not central PER OS (PO) by mouth PHARMACOKINETICS the study of the way the body absorbs, distributes, and gets rid of a drug PHASE I first phase of study of a new drug in humans to determine action, safety, and proper dosing PHASE II second phase of study of a new drug in humans, intended to gather information about safety and effectiveness of the drug for certain uses PHASE III large-scale studies to confirm and expand information on safety and effectiveness of new drug for certain uses, and to study common side effects PHASE IV studies done after the drug is approved by the FDA, especially to compare it to standard care or to try it for new uses PHLEBITIS irritation or inflammation of the vein PLACEBO an inactive substance; a pill/liquid that contains no medicine PLACEBO EFFECT improvement seen with giving subjects a placebo, though it contains no active drug/treatment PLATELETS small particles in the blood that help with clotting POTENTIAL possible POTENTIATE increase or multiply the effect of a drug or toxin (poison) by giving another drug or toxin at the same time (sometimes an unintentional result) POTENTIATOR an agent that helps another agent work better PRENATAL before birth PROPHYLAXIS a drug given to prevent disease or infection PER OS (PO) by mouth PRN as needed PROGNOSIS outlook, probable outcomes PRONE lying on the stomach PROSPECTIVE STUDY following patients forward in time PROSTHESIS artificial part, most often limbs, such as arms or legs PROTOCOL plan of study PROXIMAL closer to the center of the body, away from the end PULMONARY pertaining to the lungs

QD every day; daily QID four times a day

RADIATION THERAPY x-ray or cobalt treatment RANDOM by chance (like the flip of a coin) RANDOMIZATION chance selection RBC red blood cell RECOMBINANT formation of new combinations of genes RECONSTITUTION putting back together the original parts or elements RECUR happen again REFRACTORY not responding to treatment REGENERATION re-growth of a structure or of lost tissue REGIMEN pattern of giving treatment RELAPSE the return of a disease REMISSION disappearance of evidence of cancer or other disease RENAL pertaining to the kidneys REPLICABLE possible to duplicate RESECT remove or cut out surgically RETROSPECTIVE STUDY looking back over past experience

SARCOMA a type of cancer SEDATIVE a drug to calm or make less anxious SEMINOMA a type of testicular cancer (found in the male sex glands) SEQUENTIALLY in a row, in order SOMNOLENCE sleepiness SPIROMETER an instrument to measure the amount of air taken into and exhaled from the lungs STAGING an evaluation of the extent of the disease STANDARD OF CARE a treatment plan that the majority of the medical community would accept as appropriate STENOSIS narrowing of a duct, tube, or one of the blood vessels in the heart STOMATITIS mouth sores, inflammation of the mouth STRATIFY arrange in groups for analysis of results (e.g., stratify by age, sex, etc.) STUPOR stunned state in which it is difficult to get a response or the attention of the subject SUBCLAVIAN under the collarbone SUBCUTANEOUS under the skin SUPINE lying on the back SUPPORTIVE CARE general medical care aimed at symptoms, not intended to improve or cure underlying disease SYMPTOMATIC having symptoms SYNDROME a condition characterized by a set of symptoms SYSTOLIC top number in blood pressure; pressure during active contraction of the heart

TERATOGENIC capable of causing malformations in a fetus (developing baby still inside the mother’s body) TESTES/TESTICLES male sex glands THROMBOSIS clotting THROMBUS blood clot TID three times a day TITRATION a method for deciding on the strength of a drug or solution; gradually increasing the dose T-LYMPHOCYTES type of white blood cells TOPICAL on the surface TOPICAL ANESTHETIC applied to a certain area of the skin and reducing pain only in the area to which applied TOXICITY side effects or undesirable effects of a drug or treatment TRANSDERMAL through the skin TRANSIENTLY temporarily TRAUMA injury; wound TREADMILL walking machine used to test heart function

UPTAKE absorbing and taking in of a substance by living tissue

VALVULOPLASTY plastic repair of a valve, especially a heart valve VARICES enlarged veins VASOSPASM narrowing of the blood vessels VECTOR a carrier that can transmit disease-causing microorganisms (germs and viruses) VENIPUNCTURE needle stick, blood draw, entering the skin with a needle VERTICAL TRANSMISSION spread of disease

WBC white blood cell

How America Saves 2024

The current state of defined contribution plans in america, how does your plan stack up, industry participation rates , industry benchmark reports, key stats you should know .

• All investing is subject to risk, including the possible loss of the money you invest.

Things you buy through our links may earn Vox Media a commission.

All the Gossip on the Biden Family’s Post-Debate Blame Game

Democrats have been in full panic mode since Joe Biden ’s disastrous debate performance against Donald Trump with pundits, donors, and average voters questioning whether the president should be replaced at the top of the 2024 ticket. But it seems one crucial constituency is not engaging in the national conversation about whether Joe should drop out: the Biden family.

As President Biden huddled with his wife, children, and grandchildren on a preplanned trip to Camp David over the weekend, multiple outlets published dishy behind-the-scenes reports on how the family members think he should proceed. One thing is clear: They want him to stay in the race. But there was some conflicting gossip about who the Bidens blame for Joe’s bad performance and what they plan to do about it (which is pretty remarkable, as this may be the least leak-prone White House in recent memory). Here, a roundup of what we’ve learned.

The Bidens are not talking about Joe dropping out.

You might assume that the president called a family meeting to help him decide whether he should heed calls for him to drop out of the race. But the family had already planned to gather at Camp David this past weekend to participate in a photo shoot with celebrity photographer Annie Leibovitz. And multiple reports emphasized that this was not a formal family meeting and the Bidens weren’t actively debating whether the president should drop out. As the New York Times reported :

One of the people informed about the situation said “the entire family is united” and added flatly that the president was not getting out of the race and had not discussed doing so. “You get up and keep fighting,” the person said.

Two sources told the paper that if Biden was thinking about quitting, he wouldn’t have that discussion at Camp David, “where too many people outside the family might overhear.”

Jill Biden tells Vogue , “We will continue to fight.”

First Lady Jill Biden is the driving force behind the president’s decision to continue his reelection campaign, according to a “person familiar with the dynamics.”

“The only person who has ultimate influence with him is the First Lady,” the source told NBC News. “If she decides there should be a change of course, there will be a change of course.”

During a postdebate stop at Waffle House on Thursday night, Joe Biden told reporters, “I think we did well.” But as the president and First Lady appeared at various rallies and fundraisers over the weekend, the spin evolved. Yes, Joe had a bad night — but it was no more than that. Jill articulated this message at a fundraiser on Friday.

“After last night’s debate, he said, ‘You know, Jill, I don’t know what happened. I didn’t feel that great,’” she said. “And I said, ‘Look, Joe, we are not going to let 90 minutes define the four years that you’ve been president.’”

Jill Biden reiterated this — and declared that her husband will “continue to fight” — in an editor’s note attached to the top of her Vogue profile , which was published on Monday morning:

Editor’s Note:  The debate on June 27 spurred a discussion about whether President Joe Biden should remain the Democratic nominee. Dr. Jill Biden, the first lady and  Vogue’ s August cover subject, has fiercely defended her husband and stood by him. Reached by phone on June 30 at Camp David, where the Biden family had gathered for the weekend, she told  Vogue  that they “will not let those 90 minutes define the four years he’s been president. We will continue to fight.” President Biden, she added, “will always do what’s best for the country.” Whatever happens in the weeks and months between now and November, it is Dr. Biden who will remain the president’s closest confidant and advocate.

Hunter Biden is advising his dad to keep running.

Jill Biden has always been one of her husband’s fiercest defenders, so he should probably be seeking advice from more objective sources, too. But instead, the other main voice in his ear right now is his son Hunter, who we can all agree has not made the best life choices . Per the Times :

One of the strongest voices imploring Mr. Biden to resist pressure to drop out was his son Hunter Biden, whom the president has long leaned on for advice, said one of the people informed about the discussions, who, like others, spoke on condition of anonymity to share internal deliberations. Hunter Biden wants Americans to see the version of his father that he knows — scrappy and in command of the facts — rather than the stumbling, aging president Americans saw on Thursday night.

Hunter has even been joining meetings with his father and top White House aides in recent days. Per NBC News:

While he is regularly at the White House residence and events, it is unusual for Hunter Biden to be in and around meetings that his father is having with his team, these people said. They said the president’s aides were struck by his presence during their discussions. … One of the people familiar with the matter said Hunter Biden has been closely advising his father since the family gathered this past weekend at Camp David after Thursday’s debate. This person said Hunter Biden has “popped into” a couple of meetings and phone calls the president has had with some of his advisers. Another person familiar with the matter said the reaction from some senior White House staff has been, “What the hell is happening?”

The Biden grandkids think TikTok is the answer.

The younger Bidens’ big idea is exactly what you’d expect from well-meaning grandkids with no political experience. Per the Times :

Other family members were trying to figure out how they could be helpful. At least one of the president’s grandchildren has expressed interest in getting more involved with the campaign, perhaps by talking with influencers on social media, according to the informed person.

The Biden family blames campaign staffers.

Members of the Biden family privately “trashed his top campaign advisers” over the weekend and even urged the president to “fire or demote people in his political high command,” according to Politico. Complaints reportedly range from Biden being poorly prepared to overly prepared:

Among the family’s complaints about the debate practice: that Biden was not prepared to pivot more to go on the attack; that he was bogged down too much on defending his record rather than outlining a vision for a second term; and that he was over-worked and not well-rested. The blame was cast widely on staffers, including: Anita Dunn, the senior adviser who frequently has the president’s ear; her husband, Bob Bauer, the president’s attorney who played Trump in rehearsals at Camp David; and Ron Klain, the former chief of staff who ran point on the debate prep and previous cycles’ sessions.

But a senior Biden aide said it is “not true,” and other sources said there is no expectation that anyone will actually be fired. And Axios reported that Biden personally assured Ron Klain that no one blames his top staffers:

But the president smoothed it over: He called former chief of staff Ron Klain, who led the team, and one of the things they talked about was that neither he nor the family blames the prep.

Some Biden staffers blame Biden.

Eight anonymous people “involved in or briefed on the president’s debate preparation” hit back at efforts to blame the Biden team, telling the Washington Post that he was adequately prepared and they were as shocked as anyone by his terrible night:

So aides were bewildered by his performance. Many felt they had never seen him collapse so dramatically. After all, Biden was a veteran of numerous debates — as a senator, vice-presidential nominee and presidential candidate. And they did not understand why he gave an entirely different answer on the age question than the one they spent more than a week perfecting.

Other Biden campaign staffers blame CNN.

Mid-debate, Biden officials started telling reporters that the president had a cold in an attempt to play down his raspy voice. When Biden’s raspiness disappeared the following day, they looked around for something else to blame and settled on the debate host, CNN. According to Politico, they criticized the network for everything from the lack of live fact-checking to Biden’s pale makeup:

… Biden’s campaign staff only grew angrier at CNN as to how the debate was run, according to several people familiar with the conversations. Their complaints were lengthy, including that the moderators should have fact-checked Trump more often, that Biden was not told which camera he’d be on when not speaking and that the makeup staff made him appear too pale, according to the three people. Biden did, however, agree to the terms of the debate before it was held.

The Biden campaign blames the entire media.

Jen O’Malley Dillon, the president’s top campaign strategist, said that if there’s a dip in the president’s polling, pundits will be to blame. Per the Times :

By Saturday evening, Ms. O’Malley Dillon wrote a memo accusing “the beltway class” of counting out Mr. Biden prematurely. “If we do see changes in polling in the coming weeks, it will not be the first time that overblown media narratives have driven temporary dips in the polls,” she wrote.

As Bloomberg reported , another top Biden campaign staffer blamed the Pod Save America guys specifically:

In another memo, deputy campaign manager Rob Flaherty argued that even if the president’s polls did decline, it was merely a temporary reflection of “reactionary” coverage by the chattering class. Flaherty went on to swipe at “self-important” podcasters — a clear reference to the popular “Pod Save America” show, hosted by former Obama administration officials who expressed alarm in the aftermath of the debate. “Breaking news: People think Joe Biden’s old. They did coming into the debate, they do coming out of the debate,” he wrote.

Many blame Biden’s inner circle for keeping him isolated and hiding his condition.

On Tuesday, Politico reported that many Democrats are now looking back at Biden’s entire presidency and blaming his top aides, along with the First Lady and his sister Valerie Biden Owens, for forming “an increasingly protective circle around him, limiting his exposure to the media and outside advice.” Per Politico:

Following the debate, the pervasive view throughout much of the party is of Biden’s inner circle as an impenetrable group of enablers who deluded themselves about his ability to run again even as they’ve assiduously worked to accommodate his limitations and shield them from view. … When aides to the president suggested he was the best and only candidate who could beat Trump, few pushed back. “The fact is, there wasn’t an open dialogue about whether he should run except for the people who would benefit from him running,” said a Democratic operative close to the campaign. They described the inner circle, Donilon especially, as convinced “that this was going to be about Trump, not about Biden, and at the end of the day, people just wouldn’t vote for Trump. But here we are, we’re sitting in July, and the race is about Biden, and it’s about a trait you can’t fix.”

Senior deputy press secretary Andrew Bates pushed back after the piece was published, telling Politico that complaints about Biden’s tight inner circle are “unfair distortions of processes that exist in every administration,” and saying the president “actively seeks input from a wide range of individuals inside and outside the administration.”

This piece has been updated.

More on politics

• How Can Biden Be Replaced? A Guide to Democrats’ Next Steps.
• What Kamala World Is Thinking
• How to Pick a New Democratic Presidential Candidate Fast
• early and often
• biden family
• trump-biden debate

Most Viewed Stories

• The Conspiracy of Silence to Protect Joe Biden
• What the Jeffrey Epstein Documents Reveal About Donald Trump
• Sixteen Kids and a Hit Man
• Tom Scocca: Unraveling My Medical Mystery  

Editor’s Picks

Most Popular

• The Conspiracy of Silence to Protect Joe Biden By Olivia Nuzzi
• What the Jeffrey Epstein Documents Reveal About Donald Trump By Margaret Hartmann
• What Kamala World Is Thinking By Gabriel Debenedetti
• How to Pick a New Democratic Presidential Candidate Fast By Jonathan Chait
• Sixteen Kids and a Hit Man By James D. Walsh
• Tom Scocca: Unraveling My Medical Mystery   By Tom Scocca

What is your email?

This email will be used to sign into all New York sites. By submitting your email, you agree to our Terms and Privacy Policy and to receive email correspondence from us.

Sign In To Continue Reading

Create your free account.

Password must be at least 8 characters and contain:

• Lower case letters (a-z)
• Upper case letters (A-Z)
• Numbers (0-9)
• Special Characters (!@#$%^&*)

As part of your account, you’ll receive occasional updates and offers from New York , which you can opt out of anytime.

The Daily Show Fan Page

Explore the latest interviews, correspondent coverage, best-of moments and more from The Daily Show.

The Daily Show

S29 E68 • July 8, 2024

Host Jon Stewart returns to his place behind the desk for an unvarnished look at the 2024 election, with expert analysis from the Daily Show news team.

Extended Interviews

The Daily Show Tickets

Attend a Live Taping

Find out how you can see The Daily Show live and in-person as a member of the studio audience.

Best of Jon Stewart

The Weekly Show with Jon Stewart

New Episodes Thursdays

Jon Stewart and special guests tackle complex issues.

Powerful Politicos

The Daily Show Shop

Great Things Are in Store

Become the proud owner of exclusive gear, including clothing, drinkware and must-have accessories.

About The Daily Show