Face it D3 this was very poor planning

Tarouza

*Don't worry this will be picture free*

After seeing the q&a with Kabir and watching the YouTube video I can't help but be flabbergasted by the absolutely horrendous planning by D3 in regards to the Ultron event.

In the Q&A someone asked about this not running in the season and Kabir laughed it off in answer saying he asked marvel to change the movie date and got stared at like a crazy person

In the video, one of the Devs said they were working on it for months and they have been doing a lot of testing (all good things in my opinion)


Here is the million dollar question, if you knew about it for months and knew when the dates for the event would be, then why in the bleeping tinykitty did you still run everything on the same schedule and have events running with ending times in the middle of this event. How hard would it have been to say at the beginning of the season to say hey we have the great age of ultron event coming out to coincide with the movie, we are going to cut the season a couple events and lower the season progressions by like 15-20 percent?

Why would you run a 5 day simulator event to end right after ultron launches when you could have run a deadpool PVE instead that's only 3 days?

Hey we have the major new mechanic that ties in to this movie that is one of the biggest releases of the year, maybe we should try and stress test our servers?

You guys have been rolling out new character after new character nerf after nerf and a bunch of other stuff that has been met with pretty much overwhelmingly negative feedback by your playerbase. Is it really that much of a surprise when people are deciding bleep this tinykitty and then throw it in your faces?

You guys knew these dates for a long time, there is no reason why there should be competitive events running at a time you had to at least suspect that you would have server issues. Your planning has been absolute tinykitty and there is no getting around it. It just reeks of gross incompetance

scottee

"Stress test the servers" doesn't help when a bajillion people are trying to log in at the same time.

tanis3303

Yea, I'll forgive them for the server issues, that's something only Neo can fix. But OP is 100% correct, they had full control over when this event ran and what was running along side of it. And as far as that aspect goes, they tinykittying blew it.

Punisher5784

Tarouza wrote:

*Don't worry this will be picture free*

After seeing the q&a with Kabir and watching the YouTube video I can't help but be flabbergasted by the absolutely horrendous planning by D3 in regards to the Ultron event.

In the Q&A someone asked about this not running in the season and Kabir laughed it off in answer saying he asked marvel to change the movie date and got stared at like a crazy person

Yupp that was me asking. I was not trying to be a jerk when I brought that up. They could of ran a short PVE that ended when PVP ends before the huge new Ultron event would start, but instead the developer jokes (which was funny) and everyone was laughing at me. Now look what happened! I knew this would happen, there's way too much going on at the end of a PVE and a PVP season. I hate asking for compensation for lost time and effort but it's a lot of covers we're losing out on.

Hopefully this gets resolved soon. I hate when they run things on Friday and then disappear for the weekend.

Tarouza

I'm pretty sure that there is a way to test the affects of a large number of people connecting at the same time.

Like people have said, every big online game has server issues, that's a fair statement which bring me to why have competitive multi day events ending when you know you are going to have server issues?

Even from a basic marketing standpoint why have other events running to pull attention away from you huge event

Vhailorx

I think that demiurge has tacitly admitted that they had basically no control over when this event ran (ie ultron). But it does seem likely that they had some control over what other events were running against it.

I would guess that is why these last several PVP events have been relatively undesirable/older characters. That only leaves the season rewards as an outstanding problem. I can understand why Demiurge would want to avoid announcing the Ultron event more than 4 weeks in advance (ie back before the beginning of Season 13) since that sets players expectations and creates all sorts of other problems.

But I do think that demiurge should be prepared to make good on the top 100 alliance rewards, even if that means giving out more than the typical 2000 covers. A lot of people will have worked quite hard over the past several weeks to get that professor X cover (dealing with many character, boost and mmr changes), and it would be very unfortunate to lose that (very desirable) reward due to circumstances entirely beyond the players' control.

Also scottee, isn't "stress testing servers" the precise test that one should do on servers ahead of time when one anticipates a billion people trying to log in all at the same time?

Spoit

But guys, events are always running, clearly they don't have any control about when all the other stuff is scheduled!

scottee

Vhailorx wrote:

Also scottee, isn't "stress testing servers" the precise test that one should do on servers ahead of time when one anticipates a billion people trying to log in all at the same time?

Oh, it is. But stress testing doesn't give you more capacity. You could argue that they should have upped their server capacity, but my guess is that solution doesn't make sense for a temporary event. Not sure if you can just rent servers for two weeks to handle a higher load and then go back to normal.

slidecage

i doubt its just the servers that are causing the problem,. One little line of code just has to be a tiny bit off and BOOM everything crashes. It was not poor planning. If you think its easy then go out yourself and try to write a PC program and then have 10,000+ people on at once and then see how freaking hard it is to program.

Though the event should of been held in a test with an old superstar or something


How do you check internet connection. ALL i know its it goes into this box how do i put a check on it

Vhailorx

scottee wrote:

Vhailorx wrote:

Also scottee, isn't "stress testing servers" the precise test that one should do on servers ahead of time when one anticipates a billion people trying to log in all at the same time?

Oh, it is. But stress testing doesn't give you more capacity. You could argue that they should have upped their server capacity, but my guess is that solution doesn't make sense for a temporary event. Not sure if you can just rent servers for two weeks to handle a higher load and then go back to normal.

You're right that if they only expect a short term spike it doesn't make sense to spend a ton of cash upgrading server capacity that won't be used in a month or two. But it's also not a particularly good strategy to just accept big server problems if they are a known/expected risk.

Maybe demiurge/sega should ask marvel to pony up some cash for infrastructure build-out whenever they are forced to do synergistic releases.

I bet you can rent server capacity in the short term, but integrating it with an existing infrastructure might be a hassle. Still, everyone knows that age of ultron will be HUGE (would you bet against that movie making 1.5 billion globally?), so planning for some extra server capacity around this time wouldn't have been unreasonable.

Tarouza

The poor planning goes more into why have competitive events ending today. With such a hugely advertised event theoretically bringing in a larger playerbase then normal, common sense and history of too many online games to count says that there will be server issues. The mystique event could have been skipped and then people have the option to start the captain America event tomorrow. Instead of sim running ending today, they could have done deadpool vs mpq and had it end yesterday so people fighting vid placement don't lose out because some people can get in and others can't.

Unknown

It's not easy - but it isn't as difficult as Demiurge seem to make it at every turn.

If they stress test it and it fails - then why the hell release the game if they know it will fail? That makes less sense than not testing it at all.

The problem is that no-one at Demiurge seems to have the least bit of lateral thought. The previous PVE could have been shorter for no overlap. They could actually have *gasp* done a mid-season break while Ultron was launched. I get that there may well have been commercial pressure to run Ultron at that exact time, but where is the commercial pressure to run a PVP with Squirrel Girl as the prize?

Honestly, it's like watching a safety video on how not to do things, where the inevitable happens no matter how many glaring signs are there. Let's hope the developers don't get a sudden urge to run with scissors or play with matches.

Doesn't Demiurge have any management in charge of these poor kids?

scottee

But it's not completely failing. Some people are getting in. Shoot, my alliance is on Round 4. So the entire thing didn't crash. It's just that a lot of people can't get in.

Kaazz

scottee wrote:

"Stress test the servers" doesn't help when a bajillion people are trying to log in at the same time.

Dude, that's what stress testing means - you run a simulation on the QA environment that simulates massive players logging on and attempting to play. As a professional software engineer, I'm aghast at the continual issues D3 has with big events like this. It's not like the event snuck up on them without warning!

scottee

Kaazz wrote:

scottee wrote:

"Stress test the servers" doesn't help when a bajillion people are trying to log in at the same time.

Dude, that's what stress testing means - you run a simulation on the QA environment that simulates massive players logging on and attempting to play. As a professional software engineer, I'm aghast at the continual issues D3 has with big events like this. It's not like the event snuck up on them without warning!

Then what were their options? If you stress test and know you can handle a million server requests a minute, and you end up with 2 million, what can you do? I ask because I really don't know. My guess is that all the coding in the world can't help that. Is it possible to rent server capacity for two weeks?

I think this was part of the reasoning behind time slices as well. This is the first event without slices in a long time, but before the slices, there were occassionally servers getting overloaded at the end of events because everyone kept refreshing their leaderboards.

Unknown

scottee wrote:

Kaazz wrote:

scottee wrote:

"Stress test the servers" doesn't help when a bajillion people are trying to log in at the same time.

Dude, that's what stress testing means - you run a simulation on the QA environment that simulates massive players logging on and attempting to play. As a professional software engineer, I'm aghast at the continual issues D3 has with big events like this. It's not like the event snuck up on them without warning!

Then what were their options? If you stress test and know you can handle a million server requests a minute, and you end up with 2 million, what can you do? I ask because I really don't know. My guess is that all the coding in the world can't help that. Is it possible to rent server capacity for two weeks?

I think this was part of the reasoning behind time slices as well. This is the first event without slices in a long time, but before the slices, there were occassionally servers getting overloaded at the end of events because everyone kept refreshing their leaderboards.

Questions that should be asked in the run up - we have no idea if this happened as they never communicate fully.

1. Did they run a stress test and did it pass?
2. Did they run the correct tests to accurately predict likely outcomes?

If the stress tests came back with any concerns then the obvious questios that are raised are:
1. Can they increase infrastructure capacity?
2. Can they delay the release of the software?
3. Can they do anything to lessen the load on the infrastructure elsewhere in the system?

Now we don't know about the testing, but if it was adequate then it should have come up with concerns. I get that delaying the game might not have been an option, and increasing capacity might not have been cost effective or realistic, but they totally missed the 3rd point.

This stuff really isn't complicated, but you have to examine worst case scenarios if you can ever hope to plan for something like this. It is really amateur.

Spoit

scottee wrote:

But it's not completely failing. Some people are getting in. Shoot, my alliance is on Round 4. So the entire thing didn't crash. It's just that a lot of people can't get in.

That's the entire problem! If some people can get in and hit matches in pvp or sim, and others have been locked out for hours, do you not see how that could cause a few problems. Especially w.r.t. shield cooldowns and refreshes

Unknown

Kaazz wrote:

scottee wrote:

"Stress test the servers" doesn't help when a bajillion people are trying to log in at the same time.

Dude, that's what stress testing means - you run a simulation on the QA environment that simulates massive players logging on and attempting to play. As a professional software engineer, I'm aghast at the continual issues D3 has with big events like this. It's not like the event snuck up on them without warning!

Even big online presences such as Blizzard have this issue when they release a big event/expansion. And you can bet THEY stress test pretty hardcore, in the realm of 8 million+.

fmftint

Stress test? They had 10 employees play test in house. Is that not enough?

Buret0

Tarouza wrote:

Even from a basic marketing standpoint why have other events running to pull attention away from you huge event

Because if your Alliance isn't hammering the event, there's going to be eight hour windows where you can't do anything.

The nodes don't appear to refresh, so you need to wait for the next Ultron round before you can hammer the event again. If your alliance is casual, you might only have three or four people even doing the event.

scottee

ephyzephy wrote:

Questions that should be asked in the run up - we have no idea if this happened as they never communicate fully.

1. Did they run a stress test and did it pass?
2. Did they run the correct tests to accurately predict likely outcomes?

If the stress tests came back with any concerns then the obvious questios that are raised are:
1. Can they increase infrastructure capacity?
2. Can they delay the release of the software?
3. Can they do anything to lessen the load on the infrastructure elsewhere in the system?

Now we don't know about the testing, but if it was adequate then it should have come up with concerns. I get that delaying the game might not have been an option, and increasing capacity might not have been cost effective or realistic, but they totally missed the 3rd point.

This stuff really isn't complicated, but you have to examine worst case scenarios if you can ever hope to plan for something like this. It is really amateur.

That's exactly my point. And why as a consumer, I'm not upset at server issues in a video game at the release of a big event.

I assumed there's be more traffic than they've ever seen before. Because it's the biggest event they've done, and everyone wants to get in right away. (and time slices are out of the question because of the alliance nature of the event)

It doesn't make sense to me that they delay the event. And my guess is that it's unrealistic to get additional server capacity online for two weeks.

That only leaves #3. And the co-running events would realistically not have much effect on this. Everyone was trying to get in at 12pm Eastern to play the Ultron event. Maybe they could shut off alliance chat or shut off the leaderboards to decrease the load. (both of which people have complained about in the past)

If you knew ahead of time that there'd be server overload at the beginning of the event, but that it'd eventually die down, would you run the event anyways? I would.