(This is part of an ongoing discussion of the 2018 OSR Survey results. See the table of contents at the bottom of this post for links to the other parts.)
Over the past few years, I have been continually surprised by the degree of disagreement, confusion, and, occasionally, vitriol regarding the meaning of OSR. I think it is uncontroversial to state that the meaning is contested. It is unclear, however, whether there is any consensus and, more generally, what is the structure of underlying beliefs. Summarizing, the results of the survey support at least three main aspects of OSR meaning. Agreement about these three aspects varies by respondent age, with younger respondents seeing greater conflict between the aspects compared to older respondents. There are other variables to consider, but I will focus on age and self-declared participation in this post.
Age and participation might influence beliefs about OSR for many reasons. Considering rules, older participants might have played older games when they were originally released and so have a different frame of reference compared to younger players. Self-declared participants in the OSR may have different priorities, or more information, compared to self-declared non-participants. We asked respondents directly about participation. (Do you participate in the OSR? Yes/No.) We also asked five questions about identification with the OSR using a seven-point agreement scale (for example, I feel a bond with the OSR). Respondents who participate in the OSR also identify more with the OSR (by approximately 1.75 points of agreement, or the difference between Somewhat agree and Strongly agree).
Before I describe the associations in the survey between meaning beliefs and age or participation, it is worth first looking at the beliefs in aggregate, over all responses. Following are the means of the OSR is… questions, which I think of as the basic “meaning” questions. We also asked each of these questions in terms of respondent desires, so we will be able to look at the discrepancy between what people believe OSR is and what they want it to be.
OSR meaning across all responses
These averages tell a simple story, which is that respondents agree most that OSR refers to a play style and set of design principles. Respondents agree somewhat that OSR is a scene or movement and that OSR is a collection of aesthetics. Respondents are close to neutral, on average, about the commercial aspect of OSR. Looking at the text of each question, and the averages, three pairs seem to hang together conceptually, tapping into game design, social, and commercial aspects, with aesthetics and specific rules off in their own space. Formal statistical analysis1 leads to the same conclusion, so I averaged each of those three pairs into new scores for the game design, social, and commercial aspects. In the following figures, I color code aspects where possible, using black for rules, green for commerce, and blue for social interaction. I created similar scores for aspects as desired (that is, what respondents say they would like OSR to mean).
The following figure shows a box plot of the three aspects. Quartiles determine the look of box plots. The central line in each box shows the median. The diamonds represent outside values, which are values more than 1.5 quartiles away from the median.
OSR meaning and desired meaning based on quartile
One way to interpret the the greater number of outside values for the rules aspect compared to the other aspects is that there is more consensus around the rules aspect. The quartiles are tighter. Looking at a similar figure that compares OSR participants to non-participants yields similar conclusions, though participants seem to both perceive and want a greater social component. (The right pane below is OSR participants.) It is perhaps unsurprising that the discrepancy between perception and desire is greater for non-participants than participants on average, as that mismatch may be part why non-participants avoid participating to begin with.
Box plots comparing meaning for self-declared OSR participants with non-participants
Age provides a potential explanation, at least partially, for broader misunderstanding and disagreement, shown in the following figure. Younger respondents seem to have stronger beliefs about OSR meaning compared to older respondents (as in, positions more strongly held). Further, there is a stark divide between commerce and social/game design aspects for younger players. Older players seem to agree, relatively speaking, that OSR is more about rules than social interaction or commerce, but are basically neutral, on average, regarding OSR as commerce or social interaction. (The pattern looks similar if I plot a figure using years playing tabletop roleplaying games as the independent variable, rather than age.) I suggest only interpreting the linear predictions (the lines) for age < 60, as the number of responses decreases markedly for age > 60.
OSR meaning aspect trends over respondent age
This pattern is similar for non-participants compared to participants, though older non-participants have much stronger aversion to the social component of the OSR (see figure below). Though the left pane may look sparse compared to the right pane, 372 responses is substantial.
OSR meaning aspect trends over respondent age, separating non-participants (left) and participants (right)
1. A three-factor model fits the data best, with the three item pairs loading heavily on factors consistent with the three aspects explained above. For stats nerds, I used exploratory factor analysis with maximum likelihood estimation and oblique promax rotation. I chose oblique rotation because the latent variables are correlated above a conventional threshold, making orthogonal rotation less appropriate. I chose to score latent variables using averages rather than factor scores so that interpretation of the results in terms of the seven-point response scale is possible. ↩
Somehow the first box plot doesn’t look like an agregate of the second box plot. Most notable in the social aspects. The lower bar for the combined sets is lower than the lower bar of either set.Though my experience with statistics tells me that this doesn’t necessarily mean there’s an error.
I find it very interesting that older respondents see the commercial aspect more strongly.Though of course the trend line only goes from 3.5 for 20 year olds to 4 for 60 year olds. Not sure if that means much.
One of my suspicions is that older people tend to have different experiences and approaches to social media interactions than the younger ones, which leads to a lower emphasis on the social aspects. Whether younger people have a stronger tendency to be interested in design innovation and older people are more set in their ways, I don’t know. But staticians should know whether that’s a general trend throughout the whole population.
One guess about the social aspect is that older participants may react more negatively to the fractious social media component of the OSR (“drama”) than younger people who experience all social media spaces as subject to these dynamics.
@Ben L.
I will be curious to see how social media participation in talking about tabletop RPGs varies by platform or frequency for participants of different ages. Also, the frequency of mentioning drama in any of the free-response questions.
To be honest, I think the initial survey may have missed an important possibility. The answer to the question of “What is OSR?” might have rightly included “A reactionary political movement within the RPG Community aimed against the Story-Game Indie Revolution”. My impression has been that this is the original source of the OSR as a Movement. And it may be, by missing this option, one might not be able to draw entirely accurate conclusions about its true meaning, relevance and trajectory. I’m not sure about this, of course, as I’m a bit of a late-comer to the scene (having poked my head up out of my hole in 2006, more or less), but that’s been my impression all along. Please feel free to ignore this point if I’m wrong, or better yet, correct me. I’d be curious to hear other views on this.
Honestly, vbwyrde I think most people in the OSR had never even heard of story games until they were well into the scene. This is certainly true of me. Also true of many people who later got into a lot of fruitless fights with story game folks.
@Ben L.
I have met a number of people who discovered OSR through the indie/story games scene, several of which I have gamed with in person. There are several natural channels that people might follow, such as Apocalypse World ⟶ Dungeon World ⟶ OSR or Burning Wheel ⟶ Torchbearer ⟶ OSR. Ron Edwards published several articles in Fight On! And so forth.
Here are a couple of articles perhaps relevant to someone doing archaeology:
(It is worth noting that this survey is not investigating history directly, nor is it using historical methods.)
This seems to be pretty much entirely incorrect. The start of the OSR as a movement can basically be pinpointed as a negative reaction to 3rd edition D&D (and it’s subsequent developments in 3.5, 4e, 5e). If the OSR is ‘against’ anything, it’s WotC’s take on traditional D&D and the style of games strongly influenced by that.
I’d also suggest that phrasing it as ‘a reactionary political movement’ is unnecessarily confrontational. It sounds like you have an axe to grind.
@Emmy
My experience has also been that reaction was primarily against 3E and 4E D&D.
Emmy, an informal (130 respondent) survey I did back in June 2009 timeframe bore that out to some degree, at least within a small audience within the OSR blogosphere.
The conclusion I came to at the time:
“4E did not beget the OSR, it was already alive and well thanks to 3E and a previous interest/use of older editions (previous to 3E).”
Unfortunately, when I purged my blog in shutting it down back in 2011, that was one of the posts I deleted. I did save an archive of the relevant posts.
@vbwyrde
That seems like historical question. The history of the term might be different from what meaning people attach to OSR now.
I am even more of a latecomer (you can see my discovery and trajectory almost entirely in the chronology of this blog, which I started shortly after finding OSR conversations online in 2011).
Interesting work. I’d be interested in seeing a more complete methodology, e.g. distribution of surveys, total sample size, etc. (Mostly because I’m always interested in seeing interesting survey work.)
@Victor Raymond
You can see some of that info in the participation post, including total sample size and where the participant heard about the survey. Ben and I shared links to the survey on the big social media platforms with which we interact and several forums that seemed relevant. Let me know if there are other particular details you think would be informative and I’ll try to include them in a future post.
As I recall, the OSR began as a “back to the roots” thing in reaction to first and foremost D&D 3e/3.5e. It was also about going back to using older versions of the game as well as the original gaming style. The clones/simulacra and finally what is now known as the OSR came after that.
Also, interesting post and nice to see someone using science to investigate gaming.
Eyeballing it, I really don’t think you can justify fitting trends through those high variability point clouds. What are the r-squared and p values for those lines? (I’d also be concerned about heteroskedasticity and non-normal distribution of residuals thanks to edge effects at 1 and 7.)
@Anonymous
All relationships in this post are significant (p < .001). This is not terribly informative, however, given the large sample size. The effect size conveys more information. In this context, the effect size is the regression coefficient, which you can see graphically as the slope of the lines. Horizontal, flat lines would mean zero effect. Lines with steeper slope represent stronger associations. The r-squared values are indeed relatively low, as age only explains a small amount of the variance in the aspect scores (I think the values are around 3% though I will have to go back to the analysis to check that). You can also see this visually in the slopes. Taking the commercial aspect as an example, the prediction for age = 20 is around 3.5 while the prediction for age = 60 is around 4 (across all responses). So 40 years shifts the prediction by only half a scale point. The distributions approach normal, though the rules-related aspect has some skew. I will post histograms when I get a chance.
@Anonymous
As a side note, most researchers avoid plotting the actual data along with model results. They just plot summary statistics (like means by group) or some visual approximation of their model predictions. This is due to both limited space and to serve a rhetorical end. Happily, on my blog I can be as complete as I want.
I think of the relationship between age and endorsement presented above more as an argument for the coherence of the three underlying aspects than as a complete explanation for the variation in the aspects. The factor analysis itself is entirely based on covariation within the meaning questions, but that is difficult to show graphically without some other relationship. I should maybe add some of the numbers for goodness of fit. A multidimensional scaling plot is perhaps a better bet, but I suspect most readers would be less familiar with interpreting such a figure and tbh the number of questions is really too small for a good visualization, probably. I might play around with MDS here when I get a chance, but I suspect a post presenting such a figure would need to spend most of the words explaining how to think about a multidimensional scaling plot generally speaking.
Pingback: CAR-PGa NEWSLETTER Vol. 28, No. 1 January 2019 – CAR-PGA