The Pathfinder RPG, published by Paizo Publishing, knocked D&D off the top of the heap. And that is an interesting story in itself.
The Open Game LicenseWhen Dungeons & Dragon's original publisher, TSR, was alive and kicking, they pursued an aggressive policy when it came to people publishing anything labeled as being compatible with D&D. That is to say, they usually threatened people with lawsuits, which got people to stop quickly given the price of lawyers and the time required to fight legal battles. These hobbyist writers/publishers would rather spend their time and money on the games they loved, and that's hardly a fault. The side effect was a broadening of the RPG market place in the form of things that looked like D&D but were not quite D&D. Or if you were Games Designer's Workshop, you just published a game called Traveller that side-stepped any notion of being like D&D. Thus for a good long while, D&D was a walled garden, to borrow a phrase from the information technology industry.
Flash-forward to 2000 and you find a very different situation. Wizards of the Coast had released Dungeons & Dragons 3rd Edition under the Open Game License, which is patterned after the GNU Public License. Simply put, anything published for 3rd Edition was either Open Game Content or Product Identity (with some very specific exceptions). Open Game Content could be republished and reused by anyone, whereas Product Identity was the stuff that was still protected by copyright laws and could not be reused or republished without permission. This was a very clever idea and I'm personally not sure of the motives behind such a move. From a sheer business standpoint, the independent publishers take on all the risk of developing new Open Game Content and the D&D team can just plunder the best of the lot for their own work. The terms of the d20 System License (the legal terms of actually utilizing the 3rd Edition D&D ruleset under the OGL) was also crafted to sell the big three core rulebooks (Player's Handbook, Dungeon Master's Guide, and Monster Manual), since the terms forbid actually describing character generation and a few other key rule components.
But, Wizards of the Coast had let one seriously bad-ass genie out of the bottle with the Open Game License. Because what do you think a business is going to do when the competition they themselves created starts to outshine them? Shut them down, of course. This is what happened with the OGL and Wizards; the smaller publishers were starting to flourish after the flood of OGL products died down and only the strong and interesting ones remained. But the OGL is non-revokable, so a different tactic was required.
Wizards of the Coast announced 4th Edition D&D in 2007, and there was much rejoicing at the time by the D&D players. At the same time, they released the Game System License, the terms under which third parties could publish things for 4th Edition. The terms of the GSL are restrictive and severely limit what can and cannot be published, and third party support, which was a critical component of the 3rd Edition juggernaut, was not forthcoming. Wizards of the Coast had taken a third step towards D&D's fall from grace.
Paizo PublishingI hear somebody asking "What do you mean, a third step? What were the first and second step?" Well, the first step was actually the Open Game License and the d20 System License. The second step happened when Paizo Publishing was formed in 2002. Or perhaps in 2007 with the announcement that Dragon magazine and its companion Dungeon were being canceled. It's a bit muddled on which was more instrumental in D&D's fall. Maybe there were four steps to doomsday.
Wizards of the Coast used to have a periodicals department, which took on the task of publishing both Dragon and Dungeon from TSR, magazines for D&D players and dungeon masters, respectively. In 2002, the periodicals department was spun off into its own company, Paizo Publishing, which continued publishing the magazines that used to belong to Wizards of the Coast.
Paizo trundled along publishing things and experimenting a bit with the format of Dungeon, cooking up the very first adventure path in the pages of that magazine, called The Shackled City. This was huge, since even somewhat linked adventure modules were as rare as hen's teeth by 2003. To publish a multi-part cohesive campaign was unheard of. It was followed by Age of Worms, Savage Tide, and Scales of War, before Wizards of the Coast canceled their licensing agreement with Paizo in 2007.
Paizo continued to produce adventure modules for D&D 3.5, however, making adventure modules a cornerstone of their business which continues to this day. With the looming release of D&D 4th and the implications of the GSL, Paizo announced they would publish their own fantasy RPG, called Pathfinder RPG in March of 2008. They opened with a world-wide beta test and even released a hefty copy of the beta rules for people to actually hold in their hands. The beta test seriously got people excited, and since it was a modification of the D&D 3.5 rules, all those hardback books people had from the previous edition were still applicable. Which was one of the big complaints about 4th Edition. Because of that backwards compatibility, all the players who were disgruntled at the direction of 4th Edition swung straight over to Pathfinder. Add in a bustling and healthy organized play program to rope in new players and one realizes why Pathfinder simply exploded in popularity.
Paizo also did something that the game industry as a whole thought was not possible anymore. They managed to make money with adventure modules, a business model the industry thought was a dead end since before the demise of TSR. The last time a game system was written in support of selling adventure modules that I can recall is the BECMI version of D&D (which I'll cover tomorrow). Paizo manages to successfully ride the supplement treadmill, something that destroyed White Wolf Publishing.
All this, and managing to grab the top spot from D&D as well. No small feat for anyone. Wizards of the Coast's other big competitor was White Wolf Publishing, and they've been gone for some time now. You have long-time companies like Palladium Books and Flying Buffalo, but they're very quiet these days and publish rarely compared to younger companies. Well, truthfully, Palladium Books is never quiet, but their publishing schedule is pretty quiet.
What's Next?In 2012, Wizards of the Coast announced it was going to have a world-wide playtest of what it called 'D&D Next', which everybody correctly assumed was the next edition of D&D. What was interesting is the surveys that accompanied the playtest material (various iterations of which are still sitting on my hard drive here). The questions, when coupled with the fact that Wizards of the Coast had released premium versions of 1st, 2nd, and 3.5 edition D&D, seemed to indicate that Wizards was fishing around to figure out which editions of D&D were the most popular. They also decided to start selling PDFs of older D&D material on DriveThruStuff as well. Again, likely to determine which editions of D&D were the most popular.
Looking at the top titles as of this writing for the Wizards of the Coast section of DriveThruStuff, the breakdown looks like this for the top 40: 15 BECMI D&D products, 19 AD&D 2e products, 4 D&D Next products, 1 D&D 3.5 product, 1 AD&D 1e product. So clearly BECMI and 2nd Edition are fan favorites, at least by the sales metrics DTS uses. But that doesn't really account for things being skewed due to the PDF selection largely being products released before 3rd Edition, so there's definitely some bias at work here. If I had to guess which editions were actually the most popular without having to poke WotC for their closely guarded survey data, it'd be (in no particular order) AD&D 2e, BECMI D&D, and D&D 3e/3.5.
That isn't some wild-ass guess either. It's based on the fact that D&D 5e shows very clear influences from those three editions. People want things to be fairly streamlined, which is where BECMI and 2nd Edition influence it. At the same time, the players want enough options to make their characters feel unique, which is where 3e's influence factors in. We see the fruits of the OGL experiment as well, because the ability scores as saving throws is something Castles & Crusades did, and the class levels never have empty levels, showing influence from the Pathfinder RPG.
But D&D has the unique problem now of having to claw its way back to the top. It never had to do this before, so it will be an interesting road ahead for the new edition.