AO3 Ship Stats: Year In Bad Data
You may have seen this AO3 Year In Review.
It hasn’t crossed my tumblr dash but it sure is circulating on twitter with 3.5M views, 10K likes, 17K retweets and counting. Normally this would be great! I love data and charts and comparisons!
Except this data is GARBAGE and belongs in the TRASH.
I first noticed something fishy when I realized that Steve/Bucky – the 5th largest ship on AO3 by total fic count – wasn’t on this Top 100 list anywhere. I know Marvel’s popularity has fallen in recent years, but not that much. Especially considering some of the other ships that made it on the list. You mean to tell me a femslash HP ship (Mary MacDonald/Lily Potter) in which one half of the pairing was so minor I had to look up her name because she was only mentioned once in a single flashback scene beat fandom juggernaut Stucky? I call bullshit.
Now obviously jumping to conclusions based on gut instinct alone is horrible practice... but it is a good place to start. So let’s look at the actual numbers and discover why this entire dataset sits on a throne of lies.
Here are the results of filtering the Steve/Bucky tag for all works created between Jan 1, 2023 and Dec 31, 2023:
Not only would that place Steve/Bucky at #23 on this list, if the other counts are correct (hint: they're not), it’s also well above the 1520-new-work cutoff of the #100 spot. So how the fuck is it not on the list? Let’s check out the author’s FAQ to see if there’s some important factor we’re missing.
The first thing you’ll probably notice in the FAQ is that the data is being scraped from publicly available works. That means anything privated and only accessible to logged-in users isn’t counted. This is Sin #1. Already the data is inaccurate because we’re not actually counting all of the published fics, but the bots needed to do data collection on this scale can't easily scrape privated fics so I kinda get it. We’ll roll with this for now and see if it at least makes the numbers make more sense:
Nope. Logging out only reduced the total by a couple hundred. Even if one were to choose the most restrictive possible definition of "new works" and filter out all crossovers and incomplete fics, Steve/Bucky would still have a yearly total of 2,305. Yet the list claims their total is somewhere below 1,500? What the fuck is going on here?
Let’s look at another ship for comparison. This time one that’s very recent and popular enough to make it on the list so we have an actual reference value for comparison: Nick/Charlie (Heartstopper). According to the list, this ship sits at #34 this year with a total of 2630 new works. But what’s AO3 say?
Off by a hundred or so but the values are much closer at least!
If we dig further into the FAQ though we discover Sin #2 (and the most egregious): the counting method. The yearly fic counts are NOT determined by filtering for a certain time period, they’re determined by simply taking a snapshot of the total number of fics in a ship tag at the end of the year and subtracting the previous end-of-year total. For example, if you check a ship tag on Jan 1, 2023 and it has 10,000 fics and check it again on Jan 1, 2024 and it now has 12,000 fics, the difference (2,000) would be the number of "new works" on this chart.
At first glance this subtraction method might seem like a perfectly valid way to count fics, and it’s certainly the easiest way, but it can and did have major consequences to the point of making the entire dataset functionally meaningless. Why? If any older works are deleted or privated, every single one of those will be subtracted from the current year fic count. And to make the problem even worse, beginning at the end of last year there was a big scare about AI scraping fics from AO3, which caused hundreds, if not thousands, of users to lock down their fics or delete them.
The magnitude of this fuck up may not be immediately obvious so let’s look at an example to see how this works in practice.
Say we have two ships. Ship A is more than a decade old with a large fanbase. Ship B is only a couple years old but gaining traction. On Jan 1, 2023, Ship A had a catalog of 50,000 fics and ship B had 5,000. Both ships have 3,000 new works published in 2023. However, 4% of the older works in each fandom were either privated or deleted during that same time (this percentage is was just chosen to make the math easy but it’s close to reality).
Ship A: 50,000 x 4% = 2,000 removed works
Ship B: 5,000 x 4% = 200 removed works
Ship A: 3,000 - 2,000 = 1,000 "new" works
Ship B: 3,000 - 200 = 2,800 "new" works
This gives Ship A a net gain of 1,000 and Ship B a net gain of 2,800 despite both fandoms producing the exact same number of new works that year. And neither one of these reported counts are the actual new works count (3,000). THIS explains the drastic difference in ranking between a ship like Steve/Bucky and Nick/Charlie.
How is this a useful measure of anything? You can't draw any conclusions about the current size and popularity of a fandom based on this data.
With this system, not only is the reported "new works" count incorrect, the older, larger fandom will always be punished and it’s count disproportionately reduced simply for the sin of being an older, larger fandom. This example doesn’t even take into account that people are going to be way more likely to delete an old fic they're no longer proud of in a fandom they no longer care about than a fic that was just written, so the deletion percentage for the older fandom should theoretically be even larger in comparison.
And if that wasn't bad enough, the author of this "study" KNEW the data was tainted and chose to present it as meaningful anyway. You will only find this if you click through to the FAQ and read about the author’s methodology, something 99.99% of people will NOT do (and even those who do may not understand the true significance of this problem):
The author may try to argue their post states that the tags "which had the greatest gain in total public fanworks” are shown on the chart, which makes it not a lie, but a error on the viewer’s part in not interpreting their data correctly. This is bullshit. Their chart CLEARLY titles the fic count column “New Works” which it explicitly is NOT, by their own admission! It should be titled “Net Gain in Works” or something similar.
Even if it were correctly titled though, the general public would not understand the difference, would interpret the numbers as new works anyway (because net gain is functionally meaningless as we've just discovered), and would base conclusions on their incorrect assumptions. There’s no getting around that… other than doing the counts correctly in the first place. This would be a much larger task but I strongly believe you shouldn’t take on a project like this if you can’t do it right.
To sum up, just because someone put a lot of work into gathering data and making a nice color-coded chart, doesn’t mean the data is GOOD or VALUABLE.
2K notes
·
View notes
I was rewatching s1e3 and something finally clicked for me..
Please forgive me if this seems obvious to you. It helps me to type out my thoughts, but I'm sure I'm just an idiot and no one else needs this explained to them, lol. That said - I was always slightly confused by the emotional weight of the holy water arc during the flashback sequence. Particularly I was confused by how angry Crowley got when Aziraphale referred to their relationship as fraternizing in the 1862 fight. I mean, "to associate or form a friendship with someone, especially when one is not supposed to" is exactly what they are doing, right? So why the 80 year breakup?
Crowley says he wants the holy water for if "it" all goes pear shaped. The phrasing is necessarily vague, and could mean lots of things. Since I know what he eventually uses it for, I was thinking about it in the context of Armageddon, or maybe more generally and vaguely about Crowley not always choosing to go along with Hell, and associating with Aziraphale. But there was not much reason for Crowley to already be thinking about Armageddon back then.
As we know from the full diary entry Neil posted, the timeline of the Edinburgh entry, and the cut bookshop opening scene, it seems like Crowley and Aziraphale were spending A LOT of time together by the 1800's. When Crowley is pulled back down to Hell in 1827, he learns that Hell is paying more attention to him than he'd previously thought. Crowley realizes at this point that spending so much time with Aziraphale is actively putting him in real danger. He recognizes that, and instead of breaking things off, or seeing Aziraphale less, he doubles down. If this relationship is dangerous, then he wants the tools to fight for it.
That's what I think I didn't get about the holy water request. It's not just general insurance, it's specifically insurance for if Hell finds out about him and Aziraphale. It's also a super vulnerable request because in making it, Crowley is openly acknowledging how important their relationship is to him. Aziraphale casually brings up the arrangement at the beginning of the conversation, and that's part of it, right? Because the whole basis of their relationship is the arrangement. It continues to be the pretense under which they meet, despite the relationship clearly having developed beyond that. And the arrangement, as Crowley proposed it in 537, is born out of convenience, and the assumption that Heaven and Hell would never notice anyway.
Crowley's request for insurance breaks that facade. He's acknowledging that it's not convenient, or safe, but he wants to do it anyway, despite the risk.
Aziraphale, on the other hand, is not ready for the screen to be taken away so abruptly. To make it worse, he assumes Crowley wants the holy water as an escape, rather than a weapon. Suddenly he is confronted with both the danger their association poses, and the idea that Crowley might choose to take his own life. He can't imagine the guilt of being directly responsible for the latter.
I also think the strength of his own emotional response to the thought of losing Crowley catches Aziraphale off guard. He hasn't admitted to himself how much he actually cares, and it scares him. Worrying about Heaven is more comfortable and familiar, so he falls back on that and switches to "If they knew I'd been... fraternizing!"
But bringing up the threat of Heaven reads to Crowley as Aziraphale saying "You may be willing to put yourself at risk for the sake of our relationship, but I am not." The word choice of "fraternizing" comes off as a dismissive and demeaning way to describe a relationship that Crowley just admitted he would risk his life for.
It's an unintentionally deep cut when Crowley is already at his most vulnerable, and so he lashes out. As far as we've seen, this is possibly the first time Crowley has truly lashed out at Aziraphale. So yeah, 80 year breakup makes sense!
And what makes this so much worse is what happens next. Crowley reaches out again in 1941 with a dramatic gesture (rescuing Aziraphale from the Nazis, saving his books). It's clear they've missed each other. They don't discuss the fight, but it's there subtextually. Aziraphale, tentatively and thrillingly, refers to them as friends, for the first time ever. He tells Crowley that he trusts him.
And then, that very same night their worst fears are confirmed. Just when they've finally reconciled a fight over the dangers of their relationship, and just when Aziraphale has finally admitted that it is not a relationship of convenience, but genuine friendship, they are exposed. Crowley is going to face punishment from Hell, explicitly for being Aziraphale's "trusted confident", and he doesn't have insurance. If Aziraphale's trick hadn't succeeded, Crowley would have had no way to protect himself.
idk it just makes me feel things ok
2K notes
·
View notes
I've seen people remark on how awkward the 1967 scene is and that is so frustrating because, for me, it is one of the most emotionally resonant flashbacks in the entire series. It is so multifaceted and ripe with implication and that assertion is baffling. As though just because this conversation appears to be hard for them, it must mean that there has to be some sense of weirdness or awkwardness between them?
This scene feeds heavily into my theory that 1941 ended in some sort of aborted romantic moment between the two, most likely initiated by Crowley. Aziraphale can barely stand to look at Crowley because the very first moment he looks him in the face, he can't stop himself from giving him this hooded eyes, barely contained look of longing.
The next thing we see is Aziraphale immediately launching into a statement about his fear for Crowley's existence that is as brutally sincere as it is heartrending. His eyes are wide, his voice is heavy with emotion, and it's clear that he is terrified beyond belief to lose Crowley. Even as he acquiesces and gives him the holy water, you can see that he wants to take it back and deny him it all over again.
Then, of course, Crowley asks if he can give him a lift, which is definitely something that they both know is a totally different question than what lies on the surface, given that they're mere feet from the bookshop and at first Crowley frowns so deeply that it's almost cartoonish but a moment after Aziraphale turns him down you get this glimpse of very real sadness:
Aziraphale sees it for what it is and in an attempt to comfort him, without being able to do what currently seems impossible to him, shares a fanciful but resigned fantasy about spending time together unbothered and unrestrained, all to the tune of these tight little, loving smiles:
When he asks again, you can just see Crowley's desperation for Aziraphale not to go. It's hard to say how long they'd been apart, but it's safe to say that for them, that previous interaction likely is very fresh in their minds.
Aziraphale has always been more fearful than Crowley when it comes to their feelings for each other. You could even potentially look at the holy water as a metaphor for their relationship. In his expressions of concern about The Arrangement, Aziraphale has always been remarking on how Crowley could be destroyed, similarly to his words here. So when he's telling him, "You go too fast for me, Crowley," what he's really saying is, "I'm terribly afraid and I'm not ready to take that step if it means that I could lose you." And it's plain to see by the wistful look on his face that it pains him greatly to say it:
The scene so quickly cuts to Crowley looking intensely at the holy water after Aziraphale has left the car (as if trying to convince you that that was the real point of the scene) that it's easy to miss this devastated expression on Crowley's face:
There's no look of perceived rejection on his face. Just a somber look of resignation. There are so many barriers in front of them, and I think that Crowley was willing to risk it but understood that Aziraphale wasn't ready to.
This is the most honest and laid bare we ever see these two be when it comes to their emotions. There's so much being said without being said and even their actual words (i.e. Crowley remembering exactly the amount of time when the 'fraternizing' conversation happened) are so full of emotion that it might even be a bit hard for some people to watch.
It's not awkward. It's just that the scene is just so incredibly earnest and heavy with coded language that it's easy to be swept up by the fact that the two aren't engaged in their typical banter and bickering. What we truly have here is an incredibly difficult and loving conversation between two people who are stuck in a seemingly impossible situation.
1K notes
·
View notes