The future of character rankings

mischiefmaker

Hey everyone, hope you enjoyed the 10/14 edition of character rankings! The people who helped out -- Junebugkiller, Leugensmiff, HawkeyeSucks, Budge Player Cadet, and locked -- put a lot of time and thought into their contributions, so if you liked the rankings, please do tell them how much you appreciated their work as the rankings would not have been the same without them.

Unfortunately, at this point it's become pretty clear that the current format of rankings (full writeups for every character every 3 months) is unsustainable, even with help -- the 6/14 edition of rankings almost burned me out on the game completely, so I decided to put less pressure on myself to get this edition out, and it took nearly a month, with lots of people agitating about it in the meantime. Moving forward, this will only get worse, since the number of characters (and therefore the number of character interactions) will only be growing.

The commonly proposed solution is to get more help. This edition taught me that even when I have high-quality writers helping out, it doesn't actually reduce the time taken by that much! That probably makes no sense, so I'll explain:

• Communication isn't free. The more people helping out, the more you need to coordinate to make sure all the characters are covered, that there's not disproportionately more being said about one character than another, and that everyone's contributions are being used. Someone has to spend time making sure people are working on the right things, and because my collaborators are awesome people, sometimes this just turns into us chatting about the game, which is a lot of fun but not exactly conducive to getting rankings out.
  
• Editing is expensive. As nice as it was to have different perspectives, sometimes I disagreed with what other people wrote, and I often wanted to change up some wording or ordering to make the writeups have a consistent voice. Reading through everything they wrote (which was a lot!), processing what they were getting at, deciding whether I agree, and then taking their words and fitting them into a consistent narrative all takes a lot more time than I had thought.

NERD ALERT: if you're at all interested in why adding more people to a knowledge-based project doesn't make it go any faster, I highly recommend The Mythical Man-Month by Frederick P. Brooks Jr., the classic work on the subject.

Making the problem worse is that it's getting harder and harder to write new and interesting things about each character, and I get the sense that while people enjoy the rankings, by the time everything is published there are new characters we haven't covered and potentially changes in the game that make things obsolete. I think most people would prefer these to be more frequent, and ideally would get updated whenever a major change happens or a new character is introduced.

How do we reconcile the problem of the rankings taking so long and yet wanting to do them even more frequently? I have a few ideas:

• I can stop editing. Basically, I'll do the data collection and contribute a little here and there, but essentially I'll just publish the rankings in a private google doc and give access to anyone who wants to write something. Once we have something for everyone, I'll publish what they wrote verbatim.
  
  This will take way less time and effort on my part, but there may be a wide disparity of opinions, potential factual errors, and possibly contradictions between writers in the same piece. Also, there's no guarantee that writeups will actually get done -- people are busy, after all, and most of us would rather write 2 pages on Sentry than a paragraph on 2* Wolverine.
  
• We can publish rankings with no commentary. This is really easy -- we could do this easily every month and probably even more often as long as people don't get tired of voting. Would it be interesting, though?
  
• I can publish my personal rankings, and only comment on new characters and characters with significant change. This eliminates the work/wait of voting and data collection, and is probably a manageable amount of work for me to do once every six weeks, but there's no protection against my personal biases, since we'll lose the diversity of perspectives we would get with more people contributing.

Any other ideas? Post them here and I'll put up a poll after a couple of days.

Isay_Isay

Perhaps limit the extensive writeups for new characters and existing characters who saw a major change in their ranking (nerfs/buffs/metashift/WAT?!s)?

These are an excellent tool for older and newer players and your efforts on each edition is truly appreciated.

Nonce Equitaur 2

I thought about this myself when I tried to fold in the six new characters. I thought about tossing up the rankings block, and then asking everyone to pick the characters they thought are ranked too low, and the ones they thought are ranked too high. That's considerably easier than ranking all the characters. Here's my list.

Ranked too low: Lady Thor, Sentry, Magneto (Classic), Deadpool, Captain Marvel (3), She-Hulk, Storm (Classic), Daredevil
Ranked too high: Wolverine (Patch), Punisher, Psylocke, Doctor Doom, Spider-man, Beast

This is also easier to iterate. Another method would have people pick the rankings for only the 6 new characters (Lady Thor, Blade, Devil Dinosaur, Colossus, Doctor Octopus, Ms. Marvel), based on the existing 1-48 ranking.

When the characters are ranked 1-54, then you could do the too low/too high poll.

Writing up all the characters is too much -- it's best to focus only on the new ones. Though it's certainly fun to read.

Here's the special edition of the character block from the Character Compendium.

rank Max Tile Damage .. Crit .... Power cost ....... Max HP .. Namelink to stats (link to discussion)
01 64/09/11/10/82/73/4.0 - _9/__/__/__/_8/11 -10960- [goto=wolverine4]Wolverine[/goto] (X-Force)
02 70/79/12/11/61/13/3.5 - 12/_8/__/__/14/__ -10200- [goto=thor3]Thor[/goto] (Modern)
03 61/79/13/11/70/12/3.0 - _8/11/__/__/_7/__ -10200- [goto=sentry3]Sentry[/goto] (Dark Avengers)
03 73/82/64/09/10/11/3.5 - 12/10/_9/__/__/__ -16440 - [goto=thor4]Thor[/goto] (Goddess of Thunder)
04 82/10/73/64/11/09/3.5 - 12/__/10/12/__/__ - 9590 - [goto=nickfury4]Nick Fury[/goto] (Director of SHIELD)
05 11/12/61/70/13/79/4.0 - __/__/_5/PA/__/PA - 6800 - [goto=daken3]Daken[/goto] (Classic)
06 79/13/70/12/11/61/3.5 - _9/__/_8/__/__/12 - 8500 - [goto=blackpanther3]Black Panther[/goto] (Man Without Fear)
07 61/70/13/12/79/11/4.0 - PA/14/__/__/_9/__ - 6800 - [goto=wolverine3]Wolverine[/goto] (Patch)
08 79/13/70/12/11/61/4.5 - 15/__/PA/__/__/_9 - 5100 - [goto=thehood3]The Hood[/goto] (Classic)
09 12/13/11/70/61/79/4.5 - __/__/__/10/PA/_8 - 6800 - [goto=blade3]Blade[/goto] (Classic)
09 79/70/61/11/12/13/3.0 - 19/11/12/__/__/__ - 8500 - [goto=captainamerica3]Captain America[/goto] (Steve Rogers)
10 12/70/13/11/79/61/3.0 - __/14/__/__/10/PA -11475- [goto=thehulk3]The Hulk[/goto] (Indestructible)
11 11/61/12/13/70/79/4.0 - __/_8/__/__/_8/_7 - 6800 - [goto=thepunisher3]The Punisher[/goto] (Dark Reign)
11 11/73/09/64/82/10/3.0 - __/_7/__/_3/_7/__ -19180- [goto=devildino4]Devil Dinosaur[/goto] (Gigantic Reptile)
12 61/70/79/11/12/13/3.2 - _9/_8/10/__/__/__ - 6800 - [goto=magneto3]Magneto[/goto] (Classic)
13 13/79/12/11/61/70/3.0 - __/_8/__/__/_5/10 - 5950 - [goto=humantorch3]Human Torch[/goto] (Classic)
14 13/61/11/79/12/70/4.5 - __/_6/__/14/__/PA - 6800 - [goto=deadpool3]Deadpool[/goto] (It's Me, Deadpool)
15 11/79/70/13/12/61/3.8 - __/_8/10/__/__/_6 - 6800 - [goto=psylocke3]Psylocke[/goto] (Classic)
16 79/13/70/61/12/11/3.0 - PA/__/PA/12/__/__ - 5950 - [goto=falcon3]Falcon[/goto] (Mighty Avengers)
17 79/70/13/11/12/61/3.0 - _8/11/__/__/__/13 - 10200- [goto=colossus3]Colossus[/goto] (Classic)
17 70/79/12/11/13/61/4.5 - PA/_7/__/__/__/_9 - 8500 - [goto=captainmarvel3]Captain Marvel[/goto] (Modern)
18 11/61/12/79/70/13/4.0 - __/14/__/11/19/__ - 5950 - [goto=blackwidow3]Black Widow[/goto] (Grey Suit)
19 10/11/49/55/09/43/4.0 - __/__/_9/_8/__/PA - 3430 - [goto=blackwidow2]Black Widow[/goto] (Original)
20 10/11/69/54/12/61/3.0 - __/__/_9/__/__/12 - 7176 - [goto=doctordoom3]Doctor Doom[/goto] (Classic)
21 43/49/09/11/55/10/3.0 - 10/10/__/__/_5/__ - 5880 - [goto=ares2]Ares[/goto] (Dark Avengers)
22 70/79/61/13/12/11/3.5 - 14/13/20/__/__/__ - 8500 - [goto=ironman3]Iron Man[/goto] (Model 40)
23 70/12/11/13/79/61/3.5 - _9/__/__/__/10/_9 - 5100 - [goto=storm3]Storm[/goto] (Mohawk)
24 79/13/61/70/12/11/3.0 - 12/__/_5/PA/__/__ - 6800 - [goto=spiderman3]Spider-Man[/goto] (Classic)
24 11/12/79/13/61/70/3.0 - __/__/11/__/12/PA - 6800 - [goto=doctoroctopus3]Doctor Octopus[/goto] (Otto Octavius)
25 13/70/79/12/61/11/3.0 - __/_9/_9/__/12/__ -10200- [goto=shehulk3]She-Hulk[/goto] (Modern)
26 70/13/79/12/61/11/3.5 - 11/__/_8/__/_9/__ - 6800 - [goto=beast3]Beast[/goto] (Modern)
27 49/55/10/09/43/11/3.5 - 12/_8/__/__/14/__ - 5880 - [goto=thor2]Thor[/goto] (Marvel NOW!)
28 11/61/11/12/61/12/3.0 - __/_6/__/__/_6/__ - 8611 - [goto=ragnarok3]Ragnarok[/goto] (Dark Avengers)
29 43/11/49/09/55/10/3.5 - PA/__/11/__/12/__ - 2940 - [goto=storm2]Storm[/goto] (Classic)
30 82/11/73/09/64/10/3.0 - 14/__/13/__/_9/__ - 8220 - [goto=invisible4]Invisible Woman[/goto] (Classic)
31 09/10/43/49/11/55/4.0 - __/__/_5/PA/__/PA - 3920 - [goto=daken2]Daken[/goto] (Dark Avengers)
32 11/10/12/69/54/61/3.0 - __/__/__/_5/__/11 - 5023 - [goto=loki3]Loki[/goto] (Dark Reign)
33 49/55/10/09/11/43/4.5 - _8/_7/__/__/__/_9 - 4900 - [goto=captainmarvel2]Captain Marvel[/goto] (Ms. Marvel)
33 11/61/70/79/12/13/3.5 - __/_7/_7/11/__/__ - 6800 - [goto=daredevil3]Daredevil[/goto] (Classic)
34 11/43/55/49/09/10/3.2 - __/14/13/_9/__/__ - 3920 - [goto=magneto2]Magneto[/goto] (Marvel NOW!)
35 43/49/09/11/55/10/4.0 - PA/12/__/__/_6/__ - 3920 - [goto=wolverine2]Wolverine[/goto] (Astonishing X-Men)
36 11/55/10/09/43/49/3.0 - __/_8/__/__/_5/10 - 3430 - [goto=humantorch2]Human Torch[/goto] (Johnny Storm)
37 11/43/49/55/09/10/4.5 - __/_9/_9/PA/__/__ - 3430 - [goto=hawkeye2]Hawkeye[/goto] (Modern)
38 55/49/43/09/10/11/3.0 - 19/11/12/__/__/__ - 4900 - [goto=captainamerica2]Captain America[/goto] (Modern)
39 07/22/06/07/25/09/3.0 - __/_6/__/__/_6/__ - 3520 - [goto=juggernaut1]Juggernaut[/goto] (Classic)
40 08/09/10/46/36/41/4.5 - __/__/__/PA/__/16 - 2460 - [goto=bullseye2]Bullseye[/goto] (Dark Avengers)
41 09/49/11/55/10/43/3.0 - __/11/__/_5/__/17 - 4900 - [goto=moonstone2]Moonstone[/goto] (Dark Avengers)
42 27/07/07/08/31/24/3.5 - _9/__/__/__/10/_9 - 1620 - [goto=storm1]Storm[/goto] (Modern)
43 19/07/22/25/06/07/4.0 - __/__/_9/10/__/__ - 1540 - [goto=blackwidow1]Black Widow[/goto] (Modern)
44 27/31/24/08/07/07/3.5 - _8/10/19/__/__/__ - 2700 - [goto=ironman1]Iron Man[/goto] (Model 35)
45 19/22/07/25/06/07/4.5 - __/_7/__/_8/__/__ - 1320 - [goto=hawkeye1]Hawkeye[/goto] (Classic)
46 06/07/07/19/25/22/3.5 - __/__/__/_7/__/12 - 2640 - [goto=venom1]Venom[/goto] (Dark Avengers)
47 43/11/49/55/10/09/3.0 - _7/__/_9/13/__/__ - 3920 - [goto=spiderman2]Spider-Man[/goto] (Bag-Man)
48 06/07/19/25/07/22/4.0 - __/__/__/20/__/13 - 1540 - [goto=yelena1]Yelena Belova[/goto] (Dark Avengers)

loroku

mischiefmaker wrote:

[*] I can publish my personal rankings, and only comment on new characters and characters with significant change. This eliminates the work/wait of voting and data collection, and is probably a manageable amount of work for me to do once every six weeks, but there's no protection against my personal biases, since we'll lose the diversity of perspectives we would get with more people contributing.

This, this, this! The rankings are way too helpful to stop - and if you stopped, surely someone else would start - but limit the comments to ONLY new characters and characters with significant change (I can see this as either significantly changed ranks, or characters that were overhauled - but if you mean only one of those, I'd pick overhauls). There's no need to write something clever about how Venom and Yelena suck over and over. Link to the previous ranking and get on with it.

I personally don't even care if the opinions come from one person: the comments will sort it out. Heck: NorthernPolarity wrote what is basically the BIBLE for new players and it's just his (?) opinion and no one cares about that, and it's been up for months and months. Don't sweat being perfect! All we want is some rankings.

Leave out the memes, leave out the songs, leave out 90% of the comment, and just show us that beautiful heat map and you will still be the 2nd most popular / reputable person on this board after NorthernPolarity.

vudu3

I love the rankings but completely understand that it's a huge under-taking for you. If you need to scale back for you sanity my vote would be to limit the amount of writing you do, especially for older characters that haven't changed at all since the last version. There's no reason why you need to write something new about GSBW every time you do this. This would mean you'd be writing and/or editing the entries for six to ten characters assuming you stick with the quarterly updates (which I think is the right frequency). If this is still too much then my vote would be to publish the rankings as-is with a single summary post noting any big move and your personal thoughts on a few key characters or meta-game changes.

I'm not a fan of simply posting what everyone else has written without any editing. This would ruin the flow, limit the interactions/references between character write-ups and potentially mean conflicting information not only between individual entries but possibly within the same entry if multiple people contributed. I also have a feeling that every single character entry would contain the phrase "I personally believe this character is over-rated/under-rated". I don't think the write-ups would contain anything that's not already available in the individual character threads.

Unknown

Please don't stop the rankings - these posts are the best I've seen for a video game anywhere. They're fun to read and extremely (extremely!) useful to both beginner and advanced characters for planning out where to invest and how to play. I'd vote for simply re-collecting the data, re-sorting the rankings and re-using past commentary for characters where nothing major has changed. You could just link to the past write-up for characters, so you can just say "Look, Bag-man is stil terrible, see here" and be done with it. Don't quit!

vudu3

loroku wrote:

Leave out the memes, leave out the songs, leave out 90% of the comment, and just show us that beautiful heat map and you will still be the 2nd most popular / reputable person on this board after NorthernPolarity.

How are you going to get a heat map from one person's personal ranking?

Adam12

I think the easiest and most obvious solution is to continue on the 3-month ranking cycle with voting, collect the data, and only do writeups for new characters, characters with significant changes, and maybe the top 5. Probably talking about 15 or writeups each time. As others have said, wasting your time writing more about Yelena is just that, a waste of time.

Nonce Equitaur 2

You may as well just do the next poll now. Pre-rank everyone with the existing order to make things a *lot* easier on the voters, and add in the new characters at the levels you think appropriate.

Results could be back in a week, and there would be no point in doing a write-up, except for the new guys, and any big movements from the previous rankings.

Unknown

First of all thanks again so much for this to you and The people who helped out -- Junebugkiller, Leugensmiff, HawkeyeSucks, Budge Player Cadet, and locked. I agree with basically everything that has been said above.
1) The Rankings are awesome, useful, insightful....
2) No need to write about two thirds of the charachters, only the new ones and overhauls or ones that have improved so much... because of the meta.
3) Every three months seems about right.
4) A collaberative effort makes for better data, but personal rankings by strong players can be just as good, and comments after can take care of the rest.

Unknown

How about making a thread only with rankings, without the write-ups and maybe optimal builds that you can update every month with people voting? People can contribute to that thread with their write-ups or their ideas on best builds/balance/agreement/disagreement and so on.

wymtime

First thank you to everyone who contributed to the list and to mischiefmaker. You do a great job on these rankings.

I would say do the vote once every 3 months post the rankings based on the community and then post your own personal rankings. You can update write ups on new and changed characters, and then use previous wirte ups or meme for older characters. This way you can still have a write up on each character, but don't have to do as much work. I would actually enjoy seeing your rankings compared to the community and then you can still have the fun of releasing the rankings.

Nonce Equitaur 2

We're 6 characters behind -- waiting 3 months would be too long. Doing a new poll now would mostly remove the need for a poll write-up for the 48 characters therein.

Unknown

Deep deep thanks for everyone that has a hand in putting these rankings together. I'm 361 days in, and I still await new editions like a kid at Christmas. They're informative and thought-provoking in the best way possible. Between this and Nonce's cataloguing, players need to look no further to play the game right. So...while I'd hate to see any change, if at all possible I'd like to see the crowd-sourced voting continue, with write-ups that are "worth" writing (meta-changers, nerfs/buffs, new characters). In lieu of that, second best would be to drop the voting, since the most interesting thing is the commentary and the discussion that it prompts.

loroku

vudu3 wrote:

loroku wrote:

Leave out the memes, leave out the songs, leave out 90% of the comment, and just show us that beautiful heat map and you will still be the 2nd most popular / reputable person on this board after NorthernPolarity.

How are you going to get a heat map from one person's personal ranking?

What? No, the rankings would still be done by everyone, just the comments are from one person.

rkd80

I echo the previous sentiments and express my gratitude towards these write ups. This combined with NorthernPolarity's guides were instrumental in contributing to my enjoyment of the game. Well, getting into DjangoUnbuffed did not hurt.

So for me personally, the two most important things I look(ed) for are:

1) The absolute ranking. The averaging of so many votes really means something.
2) Recommended build.

While the descriptions themselves are entertaining, amusing and enjoyable, if I were to distill everything then all I am interested in: Who is the best and which covers do I need?
For everything else dedicated forum threads already exist for characters.

At the end of the day, I cannot imagine how much time was put into this. And to solve problems of people slaving away for no reward, usually, an incentive is provided. So two options:
1) If you want access to these reviews, then access must be purchased. This has some drawbacks, like people leaking out the information or creating an "elite"/"exclusive" MPQ crowd.
2) Create a PayPal account. As soon as the votes come in, the account needs to reach a certain number from all of us contributing. Once the number is reached, MM & Company begin writing. They split the donations how they see fit.

Unknown

Yes, thank you to everyone that helps on these, especially the people who do the write ups and everything that entails. Also whoever paid for the survey site, I remember it being mentioned that we couldn't do a free one last time. Is this something we could get D3 to pay for to be a monthly poll? We already have a community manager, so 5 bucks or whatever for a survey they could put the results on their facebook, I'm sure that would at least get some people to buy covers for whoever is at the top.

I think once every other month is frequent enough for now, every 3-4 months and there are just too many changes. It seems like we are 2 new characters every month, then a new 4* every few seasons (the anniversary was a once a year thing), and 1 big change in the game every month (True Heal, TUs, Charged tiles, etc). The memes, songs, and ratings for skills is nice, but I think it's only need for new or revamped characters.

If it's not going to cost that much, even just a bare bones survey results and heat map every month would work, I think we are smart enough to find the individual character page to see why a characters' ranking changed so much. Sure this time Sentry lost to LThor, but it was only 4.16 to 4.36, so it's more like they are 2a and 2b and then we have 4th place.

Another idea, again depends on survey costs, would be to split the rankings into PvE and PvP. Sure Sentry is top drawer in PvP, but I'd argue he's below cStorm and mHawk in PvE. I'd argue those 2 are among the best for PvE, since they are boosted so often. I've had plenty of times where I got 2 criticals and had Hawkeye dish out 12k damage. The new 2* Ms. Marvel is kind of dud for PvP, but throwing all that TU AP out there either makes a crit, or lets you get a CD tile with a match 4 that might have gone off, or at the very least makes those 14+ AP skills usable. Some like LThor and XForce shine everywhere.

vudu3

loroku wrote:

vudu3 wrote:

loroku wrote:

Leave out the memes, leave out the songs, leave out 90% of the comment, and just show us that beautiful heat map and you will still be the 2nd most popular / reputable person on this board after NorthernPolarity.

How are you going to get a heat map from one person's personal ranking?

What? No, the rankings would still be done by everyone, just the comments are from one person.

You voted for mischiefmaker posting his own personal rankings and nothing else.

loroku wrote:

mischiefmaker wrote:

[*] I can publish my personal rankings, and only comment on new characters and characters with significant change. This eliminates the work/wait of voting and data collection, and is probably a manageable amount of work for me to do once every six weeks, but there's no protection against my personal biases, since we'll lose the diversity of perspectives we would get with more people contributing.

This, this, this!

Unknown

I was surprised at the size of the write-ups considering we've already had a few rankings, but there were quite a few changes, particularly in r60. There probably aren't going to be as many changes in the next 4 months, eg I doubt we'll get a Sentry nerf by next Valentines. Most of the new characters no one cares about. We probably don't need as extensive a write-up for characters for whom nothing has changed.

The PVE survey sounds like a neat idea to mix things up. I doubt the results will be as accurate considering people just don't have the same passion for PVE as they do PVP (eg, all the PVE hate threads lately), but would be interesting nonetheless.

There have got to be at least a few coders who could write a webapp for a survey. You could even include character icons or pics for quicker sorting, as opposed to the generic form used for this one.

Unknown

I have greatly benefited from the commentary certainly, but not as much as the overall rankings along with the basic best build recommendations. I can't imagine all the time and work these have taken for you and like all these folks it's really appreciated.

My preference would be for you to publish data along with just a minimum commentary. No memes, theme songs, etc even though they're funny. Just for example: "X-Force is # 1, best build is 5/5/3 because the jump in power from 4-5 on Green/Black is just too good to pass up." If you happen to have extra time to throw out additional insights sometimes, awesome. You don't have to for every single character every single time.

Pwuz_

mischiefmaker wrote:

[*] I can stop editing. Basically, I'll do the data collection and contribute a little here and there, but essentially I'll just publish the rankings in a private google doc and give access to anyone who wants to write something. Once we have something for everyone, I'll publish what they wrote verbatim.

This will take way less time and effort on my part, but there may be a wide disparity of opinions, potential factual errors, and possibly contradictions between writers in the same piece. Also, there's no guarantee that writeups will actually get done -- people are busy, after all, and most of us would rather write 2 pages on Sentry than a paragraph on 2* Wolverine.

A few points towards this idea.

#1 Exam style rules. Yes, we can all write 2000 words about Sentry, but how much can one of us say in 200 words or less? You have to be more concise and get to your point. I'm not sure what precise amount of characters is appropriate, but there should be a VERY strict limit on how much can be said.

#2 Add a counter point or new point section next to the actual comments. Give people a way to counter what is written without removing what was already written. To that same point, add in 1 or 2 separate spots for memes. I don't think every character NEEDS a meme, but those involved have an idea for one, go for it. If two memes show up, have a quick 24 hour between the editors over which to release into the wild.

#3 Use what we have currently as a base. Who wants to write the same thing over again for 2* Wolverine? A better question is what else needs to be said about him that hasn't been said already? I don't think we should leave it blank and say "Refer to an older Ranking," because new players will (or should) be using these too!

#4 DON'T MAKE THE EDITING OPEN TO ANYONE WHO ASKS!!! Besides those who are already involved, I would ask that any new editors are vetted by some method. View their posts on here, a sample of some of their writing, and a screen shot of their current roster. I know, this sounds like the most work, and I'm not suggesting that you personally do all that for each additional editor, but I just don't want some jerk to sneak in and make this a bigger head ache than it sounds like it has been already.

I would like to thank you so much for all your hard work on these previous rankings. I don't expect any future rankings to be quite as well polished as the previous ones, but that's ok.