Focus Group Insights on Game Design Elements

Table of Links

Abstract and 1 Introduction

2. Background and Related Work and 2.1. Gamification

2.2. Game Design Elements and 2.3. Gamification Effects

2.4. Related Work on Gamification Negative Effects

3. Systematic Mapping and 3.1. The Research Questions

3.2. Search Strategy and 3.3. Inclusion and Exclusion Criteria

3.4. Applying the Search Strategy

3.5. Data Extraction

4. Systematic Mapping Results

5. Focus Group: Developer Perception on the Negative Effects of Game Design Elements

5.1. Context and Participant Characterization

5.2. Focus Group Design

5.3. The Developers’ Perception on The Negative Effects

5.4. On the Perceived Usefulness, Ease of use and Intent of Adoption of Mapped Negative Effects

5.5. Participant Feedback

6. Limitations

7. Concluding Remarks

7.1. Future Research Directions

Acknowledgements and References

5.2. Focus Group Design

We carefully designed and performed our focus group by following the guideline proposed by Kontio et al. [107]. Figure 6 depicts the steps adopted throughout the focus group. We organized these steps in three major phases: (1) Preparation for the focus group session; (2) Discussion by group of game design elements; and (3) Discussion on the perception of the mapped negative effects. We describe each phase and step hereafter.

Phase 1: Preparation for the focus group session – This phase consists of the collection of preliminary resources for supporting the execution of the focus group session. For this purpose, we follow two steps. Step 1 consisted of recruiting developers with experience in gamification to engage in discussions. As mentioned in Section 5.1, we contacted developers of the VazaZika development team for participating in our study. We obtained the acceptance of four developers via Consent Form in which we explain our research goals, information about the waiver statements, and that the information provided by each participant will be treated confidentially and used for study purposes only. Step 2 aimed to collecting basic information to characterize the participants via the Participant Characterization Form. Our major goal was profiling each developer, so we could better interpret our study results. We opted for asking short and simple questions in order to prevent participants from being tired or discouraged to participate in discussions right after filling out the form. More specifically, we collected data on the education level, years of experience with software development (academy and industry), the number of software projects they participated, and the level of knowledge about gamification and game design elements (see Table 16).

Due to the geographic distribution of our participants, we used an online environment to promote discussions on the negative effects of game design elements. Figure 7 illustrates the virtual template that we designed using the MURAL online tool[2]. In practice, by using this tool, we were able to build an interactive mural to facilitate the conduction of the focus group session. Our mural has eight well-defined sections. Sections A to G aimed at driving the discussion regarding the negative effects of game design elements.

In order to facilitate discussions, we have semantically grouped the game design elements by section as follow: (A) Badges and Rewards; (B) Competitions, Challenges, and Goals; (C) Leaderboards and Rankings; (D) Points, XPs, and Virtual Currencies; Feedback and Achievements; (E) Avatars; and (F) Quizzes. Finally, Section G aimed at driving the discussion and capturing the level of agreement on the usefulness, ease to use, and intention to use the information on potential negative effects of game design elements.

Figure 8 illustrates the template that we have defined for each aforementioned section. A section is composed of four parts (1, 2, 3, and 4). Part 1 contains a short description of the involved game design elements and the identified negative effects based on the literature. Part 2 is designed for capturing votes on the awareness degree concern the negative effects based on a four-point scale: completely unaware, mostly unaware, mostly aware, and completely aware. Finally, parts 3 and 4 were designed for participants to add notes on the Pros and Cons of using the game design elements, respectively.

Phase 2: Discussion by game design elements – This phase consists in collecting data regarding the developer’s perception of the negative effects identified in the literature. As aforementioned, we have divide the game design elements in groups to facilitate the discussions and used a Mural discussion canvas. Thus, each group of game design elements is discussed in isolation. We

defined three steps as follows. Step 1 aimed at introducing each group of game design elements, and the negative effects. For this purpose, the moderator of the session read this information out loud (part 1 of Figure 8). Next, in Step 2, we asked each participant to assign one vote to the awareness degree of the group of game design elements according to the four-point scale depicted in part 2 of Figure 8. Each participant voted individually, without knowing the votes of his colleagues. At the end, the moderator showed the number of votes per awareness agree. Finally, in Step 3, we asked the participants to discuss and elicit Pros and Cons of using the game design elements. In this step, each comment should be documented as a note in the appropriate part: part 3 for positive comments and part 4 for negative ones, as shown in Figure 8. The note was just a brief summary of a Pro or a Con, typically taken within seconds, and we constantly asked participants to share knowledge and experiences surrounding the use of each game design element, to enrich the discussions and understandings. Additionally, whenever the moderator felt that a note is poorly written, he asked the participants to provide further considerations on the note.

Phase 3: Discussion about the perception of the mapped negative effects – After discussing all groups of game design elements, the focus group session ended with a final discussion about the perceptions regarding the mapped negative effects. This phase has two steps described as follows. In Step 1, based on the main constructs of the Technology Acceptance Model (TAM) [113], we asked each participant to assign one vote to their agreement with each of the following statements (as illustrated in Section G of Figure 7): (1) I find the information on potential negative effects of game design elements useful to support game design element adoption decisions; (2) I find the information on potential negative effects of game design element easy to use to support game design element adoption decisions; and (3) I intend to use the information on potential negative effects of game design element to support game design element adoption decisions. We have used the five-point scale depicted in Figure 7 (G). Finally, Step 2 aimed at collecting data about the participant’s experience with the focus group session. Thus, by the end of the focus group session, we asked participants to fill out a Follow-up Form. We aimed at assuring that each developer felt confident and comfortable to discuss the negative effect of game design elements.

We emphasize that the focus group session was conducted online via a Zoom Meeting. Additionally, we kept video and audio records of the session to support the data analysis. We often accessed the video and audio records for understanding what developers meant with each note. The focus group session was conducted in January 24, 2022 lasting one and a half hours.

[2] https://www.mural.co/