Server is Down (8/5); Update: Service Restored!

LakeStone

Hi Everyone -

We are currently looking into the server issue that is currently happening. We apologize for the inconvenience that this may have caused and we will get you an update as soon as we can.

-Thank you

*** UPDATE ***

As most of you know already the server went down again approximately at 9:01PM PST and back up at 3:49AM PST. All players were affected by this outage. We were able to identify the issue and will be working on this today and we do not anticipate another outage. We appreciate everyone's patience as the recent outages are inconvenient for all players.

Mod Edit: Merged server down and server up threads, pasting the "server is back up" post into the OP. - Riggy

whitecat31

LakeStone wrote:

Hi Everyone -

We are currently looking into the server issue that is currently happening. We apologize for the inconvenience that this may have caused and we will get you an update as soon as we can.

-Thank you

Thank you for the official confirmation.

Unknown

wow. way too be on top of it only 3 hours later! thanks for finally letting us know lol

Thanos

You D3 guys must be tired of getting that late night call to get up and go post in the forums.

Nellobee

I bet it is super fun to get up in the middle out the night and deal with us sarcastic tinykitties.

Unknown

***insert obligatory "about as fun as EU players having to get up in the middle of the night" comment here*** /drunpleasant

daveomite

It's been down 5+ hours so far.

ThatOneGuyjp189512

aaaaand we're back!

Unknown

Yes.. but for how long??! *insert spooky music*

Unknown

heyhellowhatsnew wrote:

Yes.. but for how long??! *insert spooky music*

Don't jinx us.

vudu3

Don't think of it as a server issue. Think of it as a free six hour shield for everyone!

wymtime

It is Deadpool who is crashing the server. He is upset that his PVE only lasted 2.5 days and he won't be in the spotlight again until Friday. His feeling is if he is not in the spotlight we should not be able to play!!!

LakeStone

Hi Everyone -

As most of you know already the server went down again approximately at 9:01PM PST and back up at 3:49AM PST. All players were affected by this outage. We were able to identify the issue and will be working on this today and we do not anticipate another outage. We appreciate everyone's patience as the recent outages are inconvenient for all players.

-Thank you

Nonce Equitaur 2

At this point, it would be good to have

1. A message explaining the timeline of server issues, and an explanation of the issue that caused the problem.

2. Merging the other 20 server-issue threads.

3. Stickying the timeline / solution thread.

EDIT: Just did a count ... there are 28 different threads about the servers in the first two pages. Thanks for the upvotes -- yeah, I would clean all this up if I still had the power to do so.

acescracked

Invest in a freaking Failover system. There is a **** of vendors and professional services companies that can help you with that.

Unknown

acescracked wrote:

Invest in a freaking Failover system. There is a tinykitty of vendors and professional services companies that can help you with that.

How do you failover a software failure? There's no vendor that offers that.

acescracked

fidsah wrote:

acescracked wrote:

Invest in a freaking Failover system. There is a tinykitty of vendors and professional services companies that can help you with that.

How do you failover a software failure? There's no vendor that offers that.

Huh?? That's CA's main source of revenue. All the major vendors offer a solution. In a quick nutshell you constantly test for application response. You can mirror databases, everything.
http://www.ca.com/us/products/infrastructure-management.aspx
HP:
http://www8.hp.com/us/en/software-solutions/it-automation-orchestration/index.html
Microsoft's systems center even can do it;
http://www.microsoft.com/en-us/server-cloud/products/system-center-2012-r2/default.aspx
http://www.infoworld.com/d/microsoft-windows/tour-of-microsoft-system-center-2012-185442

Services Vendor:
http://tiag.net/infrastructure-optimization-and-it-service-management-itsm/

The above data center mgmt products are probably overkill (except system center). Though all the vendors offer ala carte purchasing. D3 would most likely have to get two weeks of on-site professional services for implementation as well. True, the transactions that occur in this game between client and server are worldwide and super fast. However there are ways to have five 9's availability..all comes down to money

Unknown

Nonce Equitaur 2 wrote:

At this point, it would be good to have

1. A message explaining the timeline of server issues, and an explanation of the issue that caused the problem.

2. Merging the other 20 server-issue threads.

3. Stickying the timeline / solution thread.

EDIT: Just did a count ... there are 28 different threads about the servers in the first two pages. Thanks for the upvotes -- yeah, I would clean all this up if I still had the power to do so.

Office politics sucks. Even worse when you're not actually employed there, right?

Unknown

acescracked wrote:

fidsah wrote:

acescracked wrote:

Invest in a freaking Failover system. There is a tinykitty of vendors and professional services companies that can help you with that.

How do you failover a software failure? There's no vendor that offers that.

Huh?? That's CA's main source of revenue. All the major vendors offer a solution. In a quick nutshell you constantly test for application response. You can mirror databases, everything.
http://www.ca.com/us/products/infrastructure-management.aspx
HP:
http://www8.hp.com/us/en/software-solutions/it-automation-orchestration/index.html
Microsoft's systems center even can do it;
http://www.microsoft.com/en-us/server-cloud/products/system-center-2012-r2/default.aspx
http://www.infoworld.com/d/microsoft-windows/tour-of-microsoft-system-center-2012-185442

Services Vendor:
http://tiag.net/infrastructure-optimization-and-it-service-management-itsm/

The above data center mgmt products are probably overkill (except system center). Though all the vendors offer ala carte purchasing. D3 would most likely have to get two weeks of on-site professional services for implementation as well. True, the transactions that occur in this game between client and server are worldwide and super fast. However there are ways to have five 9's availability..all comes down to money

And none of those systems would have done anything in the event of this outage! Amazing! If the software can't run, you can't have service, and that was the case with these outages. If any of those systems could have prevented or mitigated this outage, you'd have never seen posts like

IceIX wrote:

We were hoping we could bring them back up quickly remotely, in which case a 5 minute outage wouldn't have warranted much. Unfortunately, that wasn't the case this time.

The software was incapable of running. All these systems would have done was failed over to a node that was also incapable of running, and the downtime would have remained exactly the same.