A New Way to Measure Media Violence

Editor's Note: The following article was originally published at Psychology Today. This version has been lightly edited.

Does it matter what we watch, or what sort of media we consume? This question has been debated in various forms. One particularly heated issue, especially in the psychology, has been whether or not violent video games lead to greater levels of aggressive behaviors. Some studies have indicated yes; others have suggested no. The existing meta-analyses, which attempt to aggregate all of the available data and studies, are themselves divided. What is one to think?

Trying to Resolve the Disputes

At the Human Flourishing Program at Harvard, we aim to make use of sophisticated statistical methods to help address these sorts of controversies. In the case of the meta-analyses on video game violence, some of the problem has been exclusive focus on just the “statistical significance” of the estimates; one meta-analysis indicates “yes, statistical significance”, and another not. The problem may be that there is not a single answer; video game violence may matter in some circumstances, but not in others. The effects may be heterogeneous. In fact, earlier this year, we published new metrics for assessing meta-analyses that better account for such heterogeneity.

The reporting of meta-analyses typically focuses on the average effect size across the various studies and settings considered. Many of the statistical techniques accommodate the fact that the effects may be larger in some settings than others. But for purposes of reporting, they still focus on the average effect size across all studies. Our new metrics instead characterize—across different studies and contexts—the estimated proportion of studies with effect sizes above or below certain thresholds (for example, the proportion with positive effects, or with negative effects, or the proportion of settings where the outcomes are increased by more than 20%).¹ This can sometimes give a better sense as to policy importance when the effects are heterogeneous and may be of different sizes in different settings.

When these new approaches were applied to the question of video game violence, we were able to show that, although there are indeed some disagreements across the competing meta-analyses, there is also a lot of agreement. (See our paper published last month in Perspectives on Psychological Science for the technical details). Specifically, when using our metrics, the meta-analyses, including those that claim “statistical significance” and those that do not, indicate that in the vast majority of circumstances, video game violence does increase aggressive behavior, but in almost all such contexts, the average effects are relatively modest. 

Video game violence, thus, does seem to increase aggressive behavior, by at least a little. But the various meta-analyses also agree that violent video games rarely increase the likelihood of high levels of aggressive behavior by 40 percent. Some disagreements remain—such as in how many contexts (e.g. different types of games) playing these games increases the likelihood of high levels of aggressive behavior by 20% to 40% for instance. But the data from all these studies seem to agree that there are usually at least small detrimental effects.

Practical Implications

So, does this matter in practice? Sometimes, these video game studies are criticized for the fact that they really tell us nothing about longer-term outcomes, and this is mostly true. Most of the experimental studies carried out in laboratory settings only assess very short-term effects of playing a violent video game on one particular occasion versus not playing a video game. However, small effects across millions of people can add up. 

Moreover, if one is to take seriously theories of character development, then character and behaviors are shaped by the formation of habits and repeated actions across many settings. This would suggest that if violent video games are played frequently, as indeed they often are, this might have at least some effects on one’s character. The effects on an individual occasion are almost certainly small, but over many occasions, they might be non-negligible.

A focus on what is good, admirable, and noble may be more conducive to flourishing than playing violent video games or watching media depictions of suicide.

Mass Shootings?

Recently, there have been discussions about whether violent video games might play a role in bringing about mass shootings, with many of the media reports claiming that scientific research had disproved this. Our research was even cited in a recent New York Times piece about this very question. But as indicated above, one limitation of the data is that the most rigorous studies are done in lab settings and long-term outcomes are not measured. Although the lab data suggests that effects on average are small, small effects on average are compatible with effects that are, for particular individuals, large or determinative. 

However, it is unlikely that being randomized to play a violent video game, on one specific occasion, is going to lead to a mass shooting. Thus, laboratory experiments do not really provide the right type of data to sort out how important video game violence may, or may not, be in giving rise to mass shootings. The media reports that claim that video game violence plays no role in mass shootings are not justified; but nor are the media reports that blame the shootings on video games. The truth is that we simply do not know. The right studies have not been done. Even our more nuanced analyses are compatible with video game violence playing an important role in mass shooting, and compatible with their being entirely irrelevant. A different type of study (e.g. which assessed violent video game practices retrospectively for those who carried out mass shootings and how that compared with the general population, controlling for other variables) would be needed.

Suicide

Some of these considerations are also relevant to other forms of media. Consider the recent debate over whether the portrayal of suicide on television shows and movies might increase the likelihood of viewers attempting suicide themselves. Once again, our new metrics can be helpful here. Our research on the topic of suicide was recently published in JAMA Psychiatry and indicated that in about two- thirds of the settings (different movies, television shows, etc.), the media portrayal of suicide increased subsequent suicide rates, but that in about one-in-five settings, the portrayal may have even had moderate protective effects, though considerably less often than they were detrimental. 

Again, the effects are relatively small, but small effects across many people can add up. For example, another recent analysis in JAMA Psychiatry indicated that the release of the Netflix series 13 Reasons Why resulted in an excess of 103 adolescent suicides, even after controlling for prior temporal and seasonal trends. Here, it would seem not a good idea to say that because study effects are “small,” these 103 adolescents who took their own lives do not matter.

A Better Way Forward for Flourishing

The results of these various studies certainly do not imply we should resort to some form of extreme censorship or impose restrictions on freedom of speech. However, we do think the evidence concerning the effects of various forms of media should be made clear, and producers should at least be aware of it when making decisions. Moreover, from the consumer’s perspective, while the effects are mostly small, there are of course numerous ways to make use of one’s leisure time. A focus on what is good, admirable, and noble may be more conducive to flourishing than playing violent video games or watching media depictions of suicide.

Tyler J. VanderWeele, Ph.D., is Professor of Epidemiology in the Departments of Epidemiology and Biostatistics at the Harvard School of Public Health, a faculty affiliate of the Institute for Quantitative Social Science, and Director of the Program on Integrative Knowledge and Human Flourishing at Harvard University.

1. Our new metrics are different than traditional "vote-counting" meta-analysis methods because they acknowledge the heterogeneity of effect sizes and also the varying sample sizes and standard errors across studies which "vote-counting" methods for "statistical significance" ignore.