Violent video games study highlights the benefits of Registered Reports [Top 100 journal articles of 2019]

This article is part 5 of a series reviewing selected papers from Altmetric’s list of the top 100 most-discussed scholarly works of 2019.

Registered Report format adds rigour to violent video games study

Registered Reports is a publishing format used by over 200 journals that emphasizes the importance of the research question and the quality of methodology by conducting peer review prior to data collection. High quality protocols are then provisionally accepted for publication if the authors follow through with the registered methodology.

This format is designed to reward best practices in adhering to the hypothetico-deductive model of the scientific method. It eliminates a variety of questionable research practices, including low statistical power, selective reporting of results, and publication bias, while allowing complete flexibility to report serendipitous findings. [Centre for Open Science]

As we’ve previously discussed in RealKM Magazine, questionable research practices are undermining public confidence in science. However, the #24 most discussed article¹ in Altmetric’s top 100 list for 2019 highlights the benefits of Registered Reports in eliminating these practices. The article looks at the evidence linking violent video game engagement with adolescents’ aggressive behaviour.

A large proportion of young people play video games, which has given rise to concerns in regard to the potential impact of these games on behaviour. Research that has investigated this relationship has had widely varied findings, with some researchers concluding that there are social and cognitive benefits, while others have concluded that violent video games cause aggressive behaviour, with some even arguing that they contribute to mass shootings. Policy-makers and professional organisations have expressed similarly varied positions.

However, the article authors caution that “Recently, a series of open letters published by scholars have cautioned the public and policy-makers that both the positive and negative effects of time spent gaming, their addictive potential, cognitive benefits and aggressive effects may have been overstated.”

Looking at the research that has been previously conducted, the authors identified a number of problem issues:

• the presence of publication bias, including that studies originally thought to have been following best practices showed particularly strong evidence of publication bias

• the problem of survey measurement flexibility, which has increased the chance of false-positive results

• a reliance on self-reported data entirely provided by young people, which makes responses susceptible to ‘mischievous responding’, a phenomenon in which research participants exaggerate their responses

• methods for computing self-reported measures of aggressive emotions that alternate between approaches that use all of the available scale items and those that use a subset of items.

In consideration of these issues, the article authors sought to use the Registered Reports format to rigorously test the hypothesis that time spent playing violent video games is positively associated with adolescents’ everyday behavioural aggression. To address the limitations of previous studies, the authors:

• pre-specified exactly how they would operationalise key variables before conducting the study

• relied on carers, not adolescents, to judge the presence or the absence of aggressive behaviour

• used a power analysis to test the hypothesis that there was a statistically and practically significant effect relating violent video game play to aggressive behaviour.

As shown in the abstract below, the article authors have concluded from the findings of their study that “the results derived from our hypothesis testing did not support the position that violent gaming relates to aggressive behaviour.”

What does this mean for knowledge management?

The use of research evidence is a critical aspect of evidence-based KM. For effective evidence-based practice, knowledge managers need to be able to draw on research that has been rigorously conducted and is not compromised by questionable research practices. This means that the Registered Reports format is in the interests of better evidence-based KM.

Author abstract

In this study, we investigated the extent to which adolescents who spend time playing violent video games exhibit higher levels of aggressive behaviour when compared with those who do not. A large sample of British adolescent participants (n = 1004) aged 14 and 15 years and an equal number of their carers were interviewed. Young people provided reports of their recent gaming experiences. Further, the violent contents of these games were coded using official EU and US ratings, and carers provided evaluations of their adolescents’ aggressive behaviours in the past month. Following a preregistered analysis plan, multiple regression analyses tested the hypothesis that recent violent game play is linearly and positively related to carer assessments of aggressive behaviour. Results did not support this prediction, nor did they support the idea that the relationship between these factors follows a nonlinear parabolic function. There was no evidence for a critical tipping point relating violent game engagement to aggressive behaviour. Sensitivity and exploratory analyses indicated these null effects extended across multiple operationalizations of violent game engagement and when the focus was on another behavioural outcome, namely, prosocial behaviour. The discussion presents an interpretation of this pattern of effects in terms of both the ongoing scientific and policy debates around violent video games, and emerging standards for robust evidence-based policy concerning young people’s technology use.

Header image source: Magnus Fröderberg/norden.org on Wikimedia Commons, CC BY 2.5 DK.

References:

Also published on Medium.