Quick summary of results:
- Xbox Game Pass games on average have a review score of 77.6, compared to PlayStation Now’s average score of 72.1
- A greater proportion of Xbox Game Pass games are higher quality, but PlayStation Now has a greater number of higher quality games
- Xbox Game Pass has 15 “top” games whereas PlayStation Now has 27
- High quality Xbox Game Pass games are on average 10% longer (hours per game) than high quality PlayStation Now games
- Xbox Game Pass is cheaper from a per high quality game perspective, but PlayStation Now has more high quality games
Over the last few years, Microsoft and Sony have competed in a service that could determine the future of gaming. Rather than have players buy individual games, both gaming giants have created Netflix-like subscription gaming services that allow individuals to pay a monthly premium for access to hundreds of games. While I’m aware that there are other services in this market (such as EA Access), my main focus will be Xbox Game Pass and PlayStation Now, as these are the two largest subscription gaming services currently available.
Why am I doing this analysis, you ask? Although I already have a sizable backlog of games that’s growing over time, I often found myself wondering whether these subscription gaming services are worth it. More specifically, out of the libraries offered by Xbox Game Pass and PlayStation Now, how many games are actually worth playing? Answering this question could help me determine whether (and how long) I should subscribe to these services in the future.
But first, an introduction to the services. Xbox Game Pass was introduced in mid-2017 and currently costs $9.99/month. The subscription service allows you to download nearly 200 games (including both Xbox One and Xbox 360 games) onto your Xbox One to play online or offline. A big draw to Xbox Game Pass is that for Xbox exclusives, subscribers can play these games on the day that they launch. Unfortunately, Xbox exclusives have been less impressive in this generation compared to PlayStation’s, but recent additions such as Forza Horizon 4 and Sea of Thieves could turn that around. With the subscription, Xbox Game Pass also provides discounts to purchase the games in the catalogue. Another interesting recent announcement is that Microsoft is planning to expand PC compatibility with the Xbox Game Pass library.
On the other side is Sony’s Playstation now, which was launched in 2014 with an hourly rental model, but switched to a subscription model in 2016 at $19.99/month (or at a heavily discounted $99.99/year). This higher monthly price is justified by Sony’s service having over 500 PS4, PS3, and PS2 games in its collection. One of the previous drawbacks of PS Now was that you couldn’t download the games and a streaming only service depended heavily on an excellent internet collection. Fortunately, Sony has started allowing downloading of certain games in their library and will expand that further in the future. In addition, because it started out as a streaming-only service, PS Now is fully compatible with PC, although reliable internet is still a prerequisite.
Now, onto the good stuff. First I will describe my methodology for my analysis and then present results as well as a recommendation. If you’re interested in my code, it’s on Github here.
For my source of review scores, I chose Game Rankings as it has the most comprehensive dataset and is actually focused on games rather than something more general like Metacritic. To get this data, I scraped the list of all games with 5 or more reviews (throwing out games with sparse data) and used that as my master list of games paired with review scores. For games that had multiple entries on Game Rankings due to the website’s separation by platform (for example, GTA V had multiple entries for PS4, XONE, etc.), I combined the data and took a weighted average review score.
Next, I manually compiled the game collections for Xbox Game Pass and PlayStation Now. I then took these collections and used a string matching package to do the vast majority of my work in matching the games to their review scores. This script did over 80% of the work, and I was able to fill in the gaps manually as well as correct a few false positive errors. Overall, close to 90% of the games in both collections combined were matched to a review score on Game Rankings. Presumably, the majority of the non-matches were because the game did not have enough reviews. Also, note that some of Xbox Game Pass’ games are unreleased, which obviously result in a “no match.”
The full matched data set is available here and I will do my best to keep it updated as new games are added to each list.
|Reviewed||No Review||Total||Reviewed %|
In terms of the average quality of games, Xbox Game Pass beats out PlayStation Now substantially, with an average review score of 77.6 over 72.1. Proportionally, Xbox Game Pass also has more higher quality games than PlayStation Now, but looking at the actual count of high quality games, PlayStation Now has the edge. This shouldn’t be surprising as Sony’s service has nearly three times the number of games available.
|Avg Score||Games >90||% >90||Games >85||% >85||Games >80||% >80|
You can clearly see how the distribution of game quality favors Xbox Game Pass in the following histograms:
I then focused on the small subset of games that have review scores of greater than 90. These are what I would call industry/genre-defining games that are must plays for not only hardcore gamers but also casual fans. Xbox Game Pass offers the following 15 games that meet this threshold (note that Metro 2033 Redux is a questionable addition to this list given that it only has the minimum 5 reviews required for this analysis):
|Title||# Reviews||Review Score|
|Ninja Gaiden Black||38||94.76|
|Batman: Arkham City||112||94.06|
|Elder Scrolls IV: Oblivion||216||93.47|
|Gears of War 2||83||93.32|
|Gears of War||142||92.32|
|Batman: Arkham Asylum||125||92.18|
|Forza Horizon 4||48||91.70|
|The Walking Dead: The Complete First Season||24||91.63|
|Gears of War 3||70||91.49|
|Metro 2033 Redux||5||90.20|
|What Remains of Edith Finch||58||90.17|
Due to its larger library, PlayStation Now boasts 27 games with a review score higher than 90:
|Title||# Reviews||Review Score|
|UNCHARTED 2: AMONG THIEVES||82||96.43|
|THE LAST OF US||68||95.09|
|Metal Gear Solid 2: Sons of Liberty||69||95.02|
|RED DEAD REDEMPTION||123||94.34|
|BATMAN: ARKHAM CITY||112||94.06|
|GOD OF WAR HD||154||93.79|
|Metal Gear Solid 4: Guns of the Patriot||85||93.53|
|THE ELDER SCROLLS IV: OBLIVION||216||93.47|
|JETPACK JOYRIDE DELUXE||10||93.00|
|GOD OF WAR II HD||76||92.68|
|BATMAN: ARKHAM ASYLUM GAME OF THE YEAR||125||92.18|
|GOD OF WAR III||70||92.07|
|Metal Gear Solid: Snake Eater||87||91.86|
|UNCHARTED 3: DRAKE’S DECEPTION GAME OF THE YEAR EDITION||72||91.78|
|SHADOW OF THE COLOSSUS CLASSICS HD||145||91.73|
|THE WALKING DEAD GAME OF THE YEAR EDITION||24||91.63|
|Metal Gear Solid V: The Phantom Pain||75||91.52|
|GOD OF WAR: CHAINS OF OLYMPUS HD||80||91.44|
|ICO CLASSICS HD||66||90.29|
|Metro 2033 Redux||5||90.20|
Another interesting note is that there are 52 games that overlap between the two services. Unsurprisingly, these have a relatively high average review score of 78.7. This makes sense, because while Microsoft and Sony ultimately want to offer differentiated games to compete in the market, the higher quality games that are available on both platforms should be available to consumers regardless of which service they choose.
As a point of reference, on Game Rankings, there were (as of October 2018) 58 unique games with a review score of greater than 90 for Xbox One and Xbox 360. If we remove Ninja Gaiden Black and Star Wars Knights of the Old Republic from Xbox Game Pass’ list since they were on the original Xbox, we see that Xbox Game Pass covers about 22% of the top games available to its consoles. On the other hand, there were 63 unique games with a review score of greater than 90 for PS4 and PS3. If we remove the PS2, PSP, and older games (God of War, Grim Fandango, God of War II, Shadow of the Colossus, God of War: Chains of Olympus, Ico) from PlayStation Now’s list, we see that PlayStation Now covers about 33% of the top games available to its consoles. Both sets of top games have average ages of about 10 years (released in 2008-2010 on average), although Xbox GP has the slight edge here. This is expected given that these subscription services tend to provide content that has been around for a while, rather than the best new releases. The fact that Xbox Game Pass doesn’t have a that much lower average age of top games than PlayStation Now, even though it includes day-one Xbox exclusives speaks to the fact that the majority of Xbox exclusives (Forza Horizon 4 being the only recent exception on that list) are not industry-defining high-quality games.
Lastly, using my gaming backlog service of choice, HowLongToBeat, I was able to put together the following stats on how long it would take to play through each collection’s games that had review scores higher than 90 and 85. What’s neat is that PlayStation Now’s higher quality games tend to be much shorter, and looking through the list of games, I see a greater prevalence of indie games in Sony’s collection. Whether this is good or bad can be left to subjective opinion.
|>90 Hours||> 90 Average||>85 Hours||> 85 Average|
So what do we do with all of this information? First off, it’s possible to make some kind of educated opinion on whether Xbox Game Pass and PlayStation Now are priced fairly. An interesting note is that although Sony’s service has over three times the library size, it only contains about twice as many “good” (scored higher than 80) games than Xbox Game Pass. This is somewhat consistent with its price being nearly double that of Microsoft’s service.
Next, by focusing on the quantity of the top-quality games in each subscription service, we can decide for ourselves whether it’s worth it to invest in subscribing to these services. For example, for Xbox Game Pass, assuming you’ve never played any of the top 15 games with review scores above 90, we can assume that it’ll take you just over 5 months at 2 hours of gaming per day to finish those games. This timeline of course shortens if you’ve played some of the games already and lengthens if you want to lower your threshold to a review score of 85 (or lower).
Another way to look at this is by trying to estimate how much each game costs. At about 21 hours per game, Xbox Game Pass essentially gets you 2.8 games a month at 2 hours per day, so roughly $3.52 per game. At about 19 hours per game, PlayStation Now provides about 3.3 games per month, so roughly $6.23 per game ($3.12 per game if you buy the annual subscription). Both are great deals compared to buying games upfront, and PlayStation Now will last longer before you run out of good games to play.
In terms of which service is best, I don’t think there is a clear winner. As I mentioned above, the two services are priced fairly relative to each other, and both are great values if you sign-up for short-term subscriptions to play through the best games in each catalogue before cancelling. Ultimately, it might come down to which console you have (until PC streaming becomes more reliable) as well as how important Xbox Game Pass’ launch releases are to you. Personally, I will probably do both at some point to catch up on the highly rated games in each service.