How Demi does playtesting

Vhailorx

This is a recurring issue, but how is Demi playtesting new designs?

Did Demi think marvel was really good? (This would explain the various resource drains that they have implemented over the past 6 weeks, but would also strongly suggest that Demi doesn't understand their own game)

Did Demi know that marvel was mediocre at best? (This might be Demi's continued work to fleshbout a slow, defensive style of play, but then why work so hard to bleed cp out of the game in advance of her release?)

Or, to be fair, are we players all just wrong, and Carol will be great?

Obviously, I am using Carol as an example, but there are  plenty of others.  Did Demi know that Gambit was stupidly powerful? Did Demi know just how bad kingpin was?  And if they are so incapable of anticipating character strength, how are they playtesting?  Do they just play 100 matches and make sure that every new release can win 90% or more of matches?  That would be a terrible test (since speed and damage taken are at least as important as reliability). 

Have we ever gotten any insight into the playtesting process?  Can we think of ideal paremeters for testing strength that are both accurate and administratively easy?

Every character isn't going to be great and that's ok.  But if Demi doesn't actually know how strong their designs are, then it will continue causing big problems.

fmftint

The most likely answer you'll get is the same canned answer they give for most questions,
Policy does not allow us to discuss specifics of ______

Tony_Foot

Hopefully not giving away too many insider secrets:

bbigler

They probably playtest against Banner, Wasp and Kingpin because they are some real competition. 

tiomono

Or maybe they just design characters without thinking that they have to make each one better than the last?

I think Carol's damage is probably lower because it seems they very highly value boardshake.

In general they rely more on metrics of usage which would make it seem as though the dartboard above me is pretty accurate.

HoundofShadow

The metrics that competitive players use are predictable and boring(?):

1) damage/ap: high damage/low ap

2) power cost not more than 2 matches.

3) can the character help to end the match as fast as the current fastest team or can that character help to do it even faster?

4) ability with repeater must be a fortified 1-turn repeater, the effect(s) must activate immediately because activating after 2 turns is too long. 

5) abilities must not be complicated or requires certain setup to activate. When they use an ability, the ability must be fired successfully every time. No pop-up telling them they can't fire the power due to not meeting x condition(s) is allowed.

No more Wasp, Banner, Dr Ock, Kingpin type of power are allowed in 5* land -> too slow and too troublesome to activate.

I think competitive players are pretty much numb to the game, that's why they are always expecting the next 5* to help them win matches even faster.

Continue to use speed as a metric to determine whether the devs know how to play the game and you will continue to be disappointed by most of the 5*. There's another part of the world in MPQ where speed is not the priority, and it's a big world out there.
 
I'm sure they know what kind of broken mechanics competitive players want in the game to help them win matches and collect rewards faster so that they have more time to spend on their real life activities, but meta 5* characters are going to happen probably once or twice per year. March is approaching in a couple of days. That's another 32 weeks of disappointing 5* or slow 5* to handle.

Vhailorx

tiomono said:

Or maybe they just design characters without thinking that they have to make each one better than the last?

I think Carol's damage is probably lower because it seems they very highly value boardshake.

In general they rely more on metrics of usage which would make it seem as though the dartboard above me is pretty accurate.

Who said every character had to be better than the last (other than d3's accounting division)?

Demi has overvalued boardshake heavily for a long time now (since at least 2015), so that part of Carol's design makes sense.  It's a blindspots that Demi refuses to acknowledge or change.

But they seem to be overvaluing repeaters a lot now too.  (5loki, Emma, 5kp, now 5marvel).  Having one lingering blindspot for a long time is bad enough.  But adding a new one despite significant commentary from the playerbase suggests a bigger problem to me.

And regardless of whatever ability strength budgeting system they employ behind the scenes, don't they at least try the new characters pre-release?  Brigby seems to have access to them for answering player questions in announcement threads.  Why does does their own internal playtesting settup not highlight these design deficiencies?

And, of course, what I really want to know is why there is such an enormous disconnect.  Players knew within a few minutes or hours that Gambit was crazy powerful, and 5* kingpin was trash (I think I posted a comment in gambitc.  Did it really take the devs weeks or months to learn the same things?   If so, shouldn't that bother demi more than it seems to?

Straycat

Assuming there is supposed to be balance, then they actively don't want every character to be better than the last. Which is why they like boardshake and repeaters. Its not that they are overvalued, necessarily, its that they are random i.e. "balanced".

With 5Marvel in particular tho, I can't imagine any playtesting where the red repeater survived 3 rounds since it endangers itself with the random row destruction. It should at least not destroy itself, or start fortified.

Throw in all the signs of them wanting us to deplete our hoards before her tho, and I do question why thought this would be any good.

Vhailorx

Stray,

boardshake is not balanced because it is random.  Or at least that is not what I mean when I say it's overvalued.

What I mean is that Demi assigns more value to boardshake abilities that is justified by its utility in game. 

Imagine a character that was pure boardshake, literally could not do anything else. And then imagine a character that was pure direct damage output.  if Demi were valuing those two abilities in a balanced fashion than these two characters should be evenly matched 1:1, which is to say that Demi should calibrate the amount/cost of the boardshake to roughly equate to the cost/amount for damage.  For perfect balance you would want to see a 50:50 win rate over a large sample of matches.  (Obviously, this is a thought expirement that is handwaving away a lot of variables for the sake of simplicitly.)

But Demi pretty consistently overvalues boardshake, meaning that a pure boardshake character designed by Demi will do much worse than a 50:50 win rate against a pure damage character designed by Demi.

It's not that boardshake by itself is a problem, it's that Demi thinks boardshake is better than it really is relative to other abilities in the game.  And the net result is that Demi keeps pumping out these boardshake-based (and repeater based) characters that are consistently ignored by the players.  And this has been happening for a long time.

(Sorry if this is redundant or misses the point of your post.)

bluewolf

There are unanswerable questions from the players' perspective re:  recent stores which appeared to be an attempt to drain resources.  Making an assumption like "The devs want us to use up our CP so that we need to spend to champ Super-Awesome Carol immediately!" seems a bit of a leap.  I think anything they do has several factors at play - how it benefits players, how it benefits the devs, and how many ways each one might benefit.

For one thing, there was no digital day.  So some of the recent stores could be the devs thinking they might get some people spending with more interesting CP stores.

Second, with all the mediocre 5's in general (Kitty excluded), CP hoards have got to be up above the average.  So pushing some stores that provide a place to spend that CP makes sense.  Reasons are:  1.  The less resources you have, the more you feel motivated to play to rebuild/keep your resources up.  2.  If you use up resources, you might spend some money to chase an essential 5 (even if CM is not good, she will be essential and therefore all competitive players want at least one cover).  3.  The two most recent stores had some more interesting 5's in them (BSSM and all the meta 5's) which is pretty nice for players (and whales) who otherwise have no hope in significantly making any progress on them from Classics.

All of this is to say I'm not sure they think Carol is necessarily Super-Awesome.  I still think their driving motivation for design lately is "something different" while trying to capture the essence of the character.  There's a video posted recently which I think explains a lot.

https://www.youtube.com/watch?v=h_bDkMRTAjw

Vhailorx

I don't think I ever said:

"The devs want us to use up our CP so that we need to spend to champ Super-Awesome Carol immediately!"

I think it's fair to assume that the favorites store etc were intended to bleed cp out of players.

I think the second question of why Demi is trying to do that is much more of an open question (as you go into in some detail bluewolf).
 

Straycat

Vhailorx said:

Stray,

boardshake is not balanced because it is random.  Or at least that is not what I mean when I say it's overvalued.

I think we just misunderstand what we mean by overvalued and balanced? I'm not saying they overvalue it as in they think it is as good as damage. I'm also not saying that its balanced as in there would be a 50:50 win split between damage and boardshake characters. I'm saying balance as not in too good or bad. All powers should have some condition or downside that keeps them balanced, and that's already built in with those random powers.

If anything they like them so much because they are uninspired or out of ideas.

Just a theory, and not that important because whether they think these powers are good or not they keep making them.

tiomono

Vhailorx said:

tiomono said:

Or maybe they just design characters without thinking that they have to make each one better than the last?

I think Carol's damage is probably lower because it seems they very highly value boardshake.

In general they rely more on metrics of usage which would make it seem as though the dartboard above me is pretty accurate.

Who said every character had to be better than the last (other than d3's accounting division)?

Demi has overvalued boardshake heavily for a long time now (since at least 2015), so that part of Carol's design makes sense.  It's a blindspots that Demi refuses to acknowledge or change.

But they seem to be overvaluing repeaters a lot now too.  (5loki, Emma, 5kp, now 5marvel).  Having one lingering blindspot for a long time is bad enough.  But adding a new one despite significant commentary from the playerbase suggests a bigger problem to me.

And regardless of whatever ability strength budgeting system they employ behind the scenes, don't they at least try the new characters pre-release?  Brigby seems to have access to them for answering player questions in announcement threads.  Why does does their own internal playtesting settup not highlight these design deficiencies?

And, of course, what I really want to know is why there is such an enormous disconnect.  Players knew within a few minutes or hours that Gambit was crazy powerful, and 5* kingpin was trash (I think I posted a comment in gambitc.  Did it really take the devs weeks or months to learn the same things?   If so, shouldn't that bother demi more than it seems to?

I'm not sure anyone has claimed that each character needs to be better than the last. It just seems with new releases we are only comparing them to the best in their tier. So when a character like carol rolls up and is obviously slow or weak (compared to the meta) that's what we as players focus on. The devs seem to have their thinking somewhere very different than that. We as players have no real clear insight into that though.

Long answer short (from my perspective): The players will never have a clear answer from the devs on this until they decide to do any communication beyond the limited info they give brigby.

Vhailorx

tiomono said:

Vhailorx said:

tiomono said:

Or maybe they just design characters without thinking that they have to make each one better than the last?

I think Carol's damage is probably lower because it seems they very highly value boardshake.

In general they rely more on metrics of usage which would make it seem as though the dartboard above me is pretty accurate.

Who said every character had to be better than the last (other than d3's accounting division)?

Demi has overvalued boardshake heavily for a long time now (since at least 2015), so that part of Carol's design makes sense.  It's a blindspots that Demi refuses to acknowledge or change.

But they seem to be overvaluing repeaters a lot now too.  (5loki, Emma, 5kp, now 5marvel).  Having one lingering blindspot for a long time is bad enough.  But adding a new one despite significant commentary from the playerbase suggests a bigger problem to me.

And regardless of whatever ability strength budgeting system they employ behind the scenes, don't they at least try the new characters pre-release?  Brigby seems to have access to them for answering player questions in announcement threads.  Why does does their own internal playtesting settup not highlight these design deficiencies?

And, of course, what I really want to know is why there is such an enormous disconnect.  Players knew within a few minutes or hours that Gambit was crazy powerful, and 5* kingpin was trash (I think I posted a comment in gambitc.  Did it really take the devs weeks or months to learn the same things?   If so, shouldn't that bother demi more than it seems to?

I'm not sure anyone has claimed that each character needs to be better than the last. It just seems with new releases we are only comparing them to the best in their tier. So when a character like carol rolls up and is obviously slow or weak (compared to the meta) that's what we as players focus on. The devs seem to have their thinking somewhere very different than that. We as players have no real clear insight into that though.

Long answer short (from my perspective): The players will never have a clear answer from the devs on this until they decide to do any communication beyond the limited info they give brigby.

I can only speak for myself of course, but I know that I have been saying things like "I think the community as a whole is too low on GED. He seems upper mid tier to me." I was similarly borderline-positive on cable. So I don't have a problem with the idea that not every character should be evaluated against the meta.  it's just that Carol seems a lot worse than that, so criticism seems warranted.

Additionally, the current meta is getting fairly old by mpq standards, (about 8 months for thorkoye, since ethr Gambit nerf, and about 6 months for kitty and her semi-meta position). So people are starting to look for the *next thing*  and Demi did some atypical things that might suggest  that they are anticipating a meta-shift, all of which may contribute to frustrations over carol's mediocrity.

ZootSax

Whenever I think of developer playtesting, I think back to the contest to create your best team and what odd results the winners were in the opinion of most of us on the forums (https://forums.d3go.com/discussion/67949/mpq-infographic-best-team-contest-winners-10-2-17).  I'm guessing their playtesting is largely simulated game, similar to the ones implied by the contest infographic, which would invariably lead to drastically different results than live people playing in a highly-competitive environment.

bbigler

I see 2 conflicting ideologies here: should they make new characters appeal to casual players or competitive players? (Of course, there's a spectrum of players between casual and competitive).

Competitive players surely spend the most money, so pleasing them is good for business, but that can also lead to power creep, which could ultimately destroy a game. It's been said that achieving 0 turn wins is "jumping the shark", and I agree.  I want a game that requires strategy and skill (but I also have a competitive nature).  So, they need to slow down the power creep by releasing average characters that hopefully appeal to more casual players. I think one of their goals is to convert casual FTP players into more competitive spending players. 

Hindsight is always 20-20. I think it was too early for them to release another meta breaking 5* character, so instead they made the Favorite Legends store for more people to get the meta and have fun with it. When that gets tiresome, I would expect another meta breaking character. Bishop certainly shook up the 4* meta (along with Kitty). Honestly, it will be hard to make a combo better than Thorkoye....quick AP production, huge AOE and true healing. Gritty is close with its purely passive power death cascades. 

So, I guess I'm trying to say that every new character shouldn't redefine the meta, but they also shouldn't be duds either.  Power creep happens in nearly all games to keep players interested, so old characters in this game should be updated to keep them relevant, (which might challenge the meta and increase variety). The power creep is naturally kept in check in PVP since the top meta characters fight each other, but PVE needs periodic updates to avoid 0 turn wins (like new SCLs). 

Thanos

https://www.youtube.com/watch?v=h_bDkMRTAjw

Good video, hopefully the folks at Demiurge watch it a couple of times and learn something from it.

Vhailorx

I love that we got the same video posted twice in this thread!

wymtime

I think when designing characters they are trying too hard to make interesting and new mechanics that they feel will make the game fresh and interesting.  Here are some of the powers they have made recently.  

Loki purple with basically infinate black CD and then destroying those CD doing damage.  It is actually a really strong power.

Loki green repeater AOE repeater damage good in theory but way too long of a repeater tile to be good.

kitty buffing tiles.  Amazing power with grocket.  New top tier power.  

Kitty red gives special tiles when when you get hit.  Horrible power most people never use.

king pin blue give a special tile possible big damage and steal AP.  Problem is it is too conditional.

king pin green can give attack tiles and repeater dial doing direct damage.  Attack tiles are weak ,and repeater does not fire often enough.

cable blue stun plus green charge to ecxelerate healing 

god emperor doom true healing when an enemy makes a yellow or black match and steals AP.  Actually a really good power true heal and AP steal

Doom burst healing  and board shuffle just weak.

mow 5* carol heavy repeater and CD dependent.  Red, green and yellow are all interesting mechanics, but too often will get matched away because of random placement and multiple turns for CD to go off.

overall I think they are just overthinking what we want and are error on the side of caution rather than creating another Gambit.  The problem is they don’t go back and rebalance characters enough after release.

The rockett

Over the years I have been playing many people have been trying to bring to the attention of Demi and D3 some on going issue like play testing.  This goes back some time to the Boss Rush  disaster.  There have been other  scenarios where we, the player base, have questioned them many time on what and why they have done things.  For the most part, it is a black hole.   It feels like they do not want to engage us.  

During Gambit’s prerelease, so many people were screaming about this meant and how OP he was.  It took them 2 months to finally listen to us and then they BUFFED him to an insane level.  Months and month later they kill him.  How does a character like that get released into the game?  How did they miss THAT big.......Twice!!  This is why this keeps coming up time and time again over the years. 

The Allaince Rush event comes out, awesome concept, and falls flat because of what each person had to do for the ally to get full rewards.  Then the 2nd one comes out and makes it even more head scratching.  

With all the ideas that the player base has posted here or Discord it just seems like it doesn’t matter to them. This is why I care less and less about what they do and what happens next in game.  The drive and passion for the game is dead.  It will be the same asks from the player base and we will get the same response from the Dev team.......nothing. So why care anymore what they do and how they do it?  I hope something good comes along that they have tested for End Game, cause God knows they really dropped the ball (and they admitted it), because this is all getting boring and old. Nothing else to do expect wait and fade away like the rest of the vets. 

Zeofar

Vhailorx said:

Did Demi think marvel was really good? (This would explain the various resource drains that they have implemented over the past 6 weeks, but would also strongly suggest that Demi doesn't understand their own game)

Did Demi know that marvel was mediocre at best? (This might be Demi's continued work to fleshbout a slow, defensive style of play, but then why work so hard to bleed cp out of the game in advance of her release?)

This is twisted logic. What you're trying to do confirm speculation that Carol is intended to be a meta-defining character with nothing to support that conclusion. The recent stores/sales don't necessarily need a special explanation relating to Carol's power level.

So what if a lot of players were certain that Carol was going to be a super splashy design on the level of Kitty, Thor, and Okoye? They were wrong, she isn't, and it's fine to admit that. It doesn't need to prove that "Demi can't do balance" just because it doesn't line up with expectations founded on pure speculation. As HoundofShadow pointed out, it's pretty obvious that this character doesn't tick the usual boxes for competitive interest, and it's probably safe to assume that Demi understands this.

If you're really committed to the premise that there is some kind of resource-drain manipulation going on, at least be prepared to think a little more broadly. Perhaps they think they can get a better read on players' interest in these stores when they precede a highly anticipated release (I.e. whether they're found good enough to justify losing investment something else that is expected to be valuable). Maybe it's an attempt to condition players into believing that stores like this before a release are a signal that the new character will be underwhelming; and that they can trigger a whale spending spree at some point by subverting this expectation.

Alternatively, there's the possibility that the coordination you're certain exists is more coincidence than anything else. Support Circuit was introduced recently. Can you explain how that plays into the Captain Marvel release as well?