On Bracketing and SHIELD Training

IceIX

To give a brief postmortem and a recent change, since they’re linked:

With our growing user base (yay!), our servers have been hit harder lately, especially at peak hours and Event ends. We’re investigating ways to bring the load down so that users can continue to play without experiencing any large scale interruptions. We made a change recently on the back end that allows us to “shard” our servers for Tournaments and Events so that any single server won’t get hit as hard during peak hours.

On Monday (5/12), our servers hit a snag with one of the remaining portions of the system that isn’t yet in this newer style of sharding, the S.H.I.E.L.D. Training leaderboards. With a larger than usual load from our Event combined with the load from our first Season coming to an end, users in S.H.I.E.L.D. Training broke the camel’s back. We’ve taken steps to prevent this in the future, but it demonstrates the reasons for bracketing and sharding our server structure.

The changes that we made to help with this with S.H.I.E.L.D. Training are as follows:
• New players enter into smaller brackets
• Older players are divided into new, smaller brackets
• Older players see their S.H.I.E.L.D. Training placement rank go up since they will be in a smaller bracket
• Since S.H.I.E.L.D. Training does not have placement rewards, this has no real effect on player progress

Our matchmaking, scaling, and rubber banding systems are always changing and will very likely continue to do so for the life of the game. We may make changes that require some user readjustment such as this one, but please understand that we always make these alterations with the full intention to make the game a better one for everyone.

Unknown

So why did you guys feel the need to change the PVP brackets from a very fair time based entry system to planned mmr / season points brackets? Some of us are having scores of 900 and being out of the top 50 with this new change. It's pretty ridiculous and really makes no sense to have it this way. can we please go back to the time based entries?

Unknown

Thanks for the post Ice.

Can you give us any information about how this shard change impacts PvP events and lightning round MMR?

IceIX

Mechanically what happened there is that with multiple shards for entry, you will not necessarily get put into only a single first come first served bracket, but one of the currently available ones on one of the shards. All shards talk to one another, so it's not a subset of players you're facing but the whole player base. This has the upshot that when players join en masse (start and ends of tourneys) that they don't hit pockets of lag or server issues. It has the downside in that if two friends join at the same time that they won't necessarily join the same bracket so it's less visibly deterministic.

Unknown

IceIX wrote:

you will not necessarily get put into only a single first come first served bracket, but one of the currently available ones on one of the shards.

Is this just simple loadbalancing at say the F5 level or is there an algorithm based on your MMR that gets looked at as well (as many on the forum are speculating)?

Unknown

Riggy wrote:

IceIX wrote:

you will not necessarily get put into only a single first come first served bracket, but one of the currently available ones on one of the shards.

Is this just simple loadbalancing at say the F5 level or is there an algorithm based on your MMR that gets looked at as well (as many on the forum are speculating)?

Heh, F5 is a vendor, not a networking layer

But from what IceIX said, it sounds like it's a random allocation to whichever shards are running at the time. They could have implemented something that would look at MMR, but that would require switching users between shards, and I don't think they would have taken on the additional complexity.

KaioShinDE

And those random brackets just "happen" to be randomly filled with a ton of good players who are all roughly as good as you.

Sure...

As usual they are only giving us half the story and expect people to swallow it.

Clintman

Riggy wrote:

IceIX wrote:

you will not necessarily get put into only a single first come first served bracket, but one of the currently available ones on one of the shards.

Is this just simple loadbalancing at say the F5 level or is there an algorithm based on your MMR that gets looked at as well (as many on the forum are speculating)?

I would like an official response on this question as well. It would go a long towards helping us understand whether the spike in points in these brackets is due to players suddenly getting hardcore for the seasons or if its due to a design change in how brackets are filled.

Unknown

ZenBrillig wrote:

Riggy wrote:

IceIX wrote:

you will not necessarily get put into only a single first come first served bracket, but one of the currently available ones on one of the shards.

Is this just simple loadbalancing at say the F5 level or is there an algorithm based on your MMR that gets looked at as well (as many on the forum are speculating)?

Heh, F5 is a vendor, not a networking layer

Heh, we refer to F5's as a step in our networking setup. Incoming requests get load-balanced by the F5's and then shipped to the actual servers / VMs. So in some ways, it is a networking layer, just not something in say the TCP stack.

IceIX

At this very second and with the understanding that we continue to noodle with it, it's random with some slight weighting so that newer users will tend towards brackets with other newer users and vice versa. It doesn't by any means preclude a Day 200 user from dropping into a bracket with 999/1000 Day 5s, just that weighting gives a gentle push to the RAND function when the event server looks at which shard to give a player to. It's FAR more likely that you got "unlucky" than anything nefarious from the sharding tech shoving you into a bracket full of hardcores.

Unknown

Riggy wrote:

ZenBrillig wrote:

Riggy wrote:

IceIX wrote:

you will not necessarily get put into only a single first come first served bracket, but one of the currently available ones on one of the shards.

Is this just simple loadbalancing at say the F5 level or is there an algorithm based on your MMR that gets looked at as well (as many on the forum are speculating)?

Heh, F5 is a vendor, not a networking layer

Heh, we refer to F5's as a step in our networking setup. Incoming requests get load-balanced by the F5's and then shipped to the actual servers / VMs. So in some ways, it is a networking layer, just not something in say the TCP stack.

Sloppy terminology

DecoyDuck

These are spectacularly reasonable explanations for what people are experiencing. Why would you let us go on for days and days with all the wild-eye conjectures and hypotheticals? Let's begin theorizing for an explanation... no wait, let's not.

Unknown

IceIX wrote:

At this very second and with the understanding that we continue to noodle with it, it's random with some slight weighting so that newer users will tend towards brackets with other newer users and vice versa. It doesn't by any means preclude a Day 200 user from dropping into a bracket with 999/1000 Day 5s, just that weighting gives a gentle push to the RAND function when the event server looks at which shard to give a player to. It's FAR more likely that you got "unlucky" than anything nefarious from the sharding tech shoving you into a bracket full of hardcores.

A suggestion, you guys could track the point thresholds for top 5/10/25/50 across all brackets and try to tweak the gentle push in the bracketing to keep the threshold score somewhat similar globally, instead of some brackets where 900 points will get you top 5 and other brackets where 900 gets you out of top 50.

LordWill

So would I be correct in assuming that due to the change it doesn't really matter WHEN I decide to join a PVP event and I will be randomly placed in an open bracket whenever I decide to join?

Unknown

IceIX wrote:

At this very second and with the understanding that we continue to noodle with it, it's random with some slight weighting so that newer users will tend towards brackets with other newer users and vice versa. It doesn't by any means preclude a Day 200 user from dropping into a bracket with 999/1000 Day 5s, just that weighting gives a gentle push to the RAND function when the event server looks at which shard to give a player to. It's FAR more likely that you got "unlucky" than anything nefarious from the sharding tech shoving you into a bracket full of hardcores.

If by gentle you mean a 300 pound defensive linemen pushing a skinny Steve Rodgers out of the way to get to the quarterback then I can see that

Unknown

Ice:

My son started playing exactly two days after me. My roster is somewhat better because I got into the awesome Djangos! Yet since this change, he has in every PVP been placed in a significantly easier bracket, where his scores allow him to easily place top 10 with 600-700, while in my brackets I'm falling outside top 50 with 800s, since my Top 10 is always over 1000.

Are you saying this is merely coincidence? Or is there some other parameter happening as well?

Unknown

sup123 wrote:

IceIX wrote:

At this very second and with the understanding that we continue to noodle with it, it's random with some slight weighting so that newer users will tend towards brackets with other newer users and vice versa. It doesn't by any means preclude a Day 200 user from dropping into a bracket with 999/1000 Day 5s, just that weighting gives a gentle push to the RAND function when the event server looks at which shard to give a player to. It's FAR more likely that you got "unlucky" than anything nefarious from the sharding tech shoving you into a bracket full of hardcores.

A suggestion, you guys could track the point thresholds for top 5/10/25/50 across all brackets and try to tweak the gentle push in the bracketing to keep the threshold score somewhat similar globally, instead of some brackets where 900 points will get you top 5 and other brackets where 900 gets you out of top 50.

I don't think you could do that effectively, since you can't predict what the thresholds will be until close to the end of the event, and the brackets fill up quite quickly. You could perhaps use historical scores to try to estimate how hard individual players are likely to push, but that would just lead to more meta-gaming the system, so I can't say that'd be recommended.

Unknown

IceIX wrote:

At this very second and with the understanding that we continue to noodle with it, it's random with some slight weighting so that newer users will tend towards brackets with other newer users and vice versa. It doesn't by any means preclude a Day 200 user from dropping into a bracket with 999/1000 Day 5s, just that weighting gives a gentle push to the RAND function when the event server looks at which shard to give a player to. It's FAR more likely that you got "unlucky" than anything nefarious from the sharding tech shoving you into a bracket full of hardcores.

I've gotten unlucky three times in a row now and wouldn't mind some variety, although the second two times were slightly less hardcore brackets. Could be your gentle push is working just a little bit too well?

I feel like, by putting people of the same level in the same bracket, you are artificially inflating the bracket difficulties and making competition stiffer than it needs to be - or should be for an enjoyable game experience. 660/1000 players of the same level competing for the same two reward tiers is a much more intense competition than whatever a truly random bracket would give you. Sure, you could have bad luck and end up in a bracket with 800 people of the same level. But more likely it would be 300. Okay, the numbers are stupid, but what I'm getting at - you're basically deliberately putting more big fishes in one pond, without giving them more food, so they tear each other apart in an attempt to get at what little is there.

I have no issues with bracketing by player strength, but then the whole structure of the events would need to reflect that, with different reward tiers for different difficulty level brackets.

And I realize you said 'slight weighing' but when I joined my last bracket it was very obviously still new and empty, I could see 20 of the ~40 people in it, and they were all the same strength level as me or higher, not one weaker player among them. Of course one example is no where near representative, but over the course of the last few PvPs my playing experience has been a repeating trend of the same and by the sound of it, so has the majority of the forums. It certainly feels like your weighing algorithm is more than slight.

Unknown

ZenBrillig wrote:

sup123 wrote:

IceIX wrote:

At this very second and with the understanding that we continue to noodle with it, it's random with some slight weighting so that newer users will tend towards brackets with other newer users and vice versa. It doesn't by any means preclude a Day 200 user from dropping into a bracket with 999/1000 Day 5s, just that weighting gives a gentle push to the RAND function when the event server looks at which shard to give a player to. It's FAR more likely that you got "unlucky" than anything nefarious from the sharding tech shoving you into a bracket full of hardcores.

A suggestion, you guys could track the point thresholds for top 5/10/25/50 across all brackets and try to tweak the gentle push in the bracketing to keep the threshold score somewhat similar globally, instead of some brackets where 900 points will get you top 5 and other brackets where 900 gets you out of top 50.

I don't think you could do that effectively, since you can't predict what the thresholds will be until close to the end of the event, and the brackets fill up quite quickly. You could perhaps use historical scores to try to estimate how hard individual players are likely to push, but that would just lead to more meta-gaming the system, so I can't say that'd be recommended.

Yeah, it would be something they would keep track of and tweak as events go but I think it would be better than the current day of the user, which according to IceIX's post seems like the metric they are using to give the gentle push.

Of course my opinion is biased, because I've been playing the game for some time but only recently have been very active in events. So I'm at Day 127 and barely started the 3* transition, and the only difference between me and a Day 60 player is a handful of single 3* covers. If days played is really the metric they are going to use, I'm pretty much screwed.

Unknown

IceIX wrote:

At this very second and with the understanding that we continue to noodle with it, it's random with some slight weighting so that newer users will tend towards brackets with other newer users and vice versa. It doesn't by any means preclude a Day 200 user from dropping into a bracket with 999/1000 Day 5s, just that weighting gives a gentle push to the RAND function when the event server looks at which shard to give a player to. It's FAR more likely that you got "unlucky" than anything nefarious from the sharding tech shoving you into a bracket full of hardcores.

Wow, this is a horrible way to handle a PVP game. You shouldn't be playing favorites to anyone, at all. New players need to build up their rosters normally and then sink or swim like the rest of us.

IceIX

Rajjeq wrote:

Ice:

My son started playing exactly two days after me. My roster is somewhat better because I got into the awesome Djangos! Yet since this change, he has in every PVP been placed in a significantly easier bracket, where his scores allow him to easily place top 10 with 600-700, while in my brackets I'm falling outside top 50 with 800s, since my Top 10 is always over 1000.

Are you saying this is merely coincidence? Or is there some other parameter happening as well?

More likely coincidence than not. I'd have to look at precise data in each case to make 100% sure. But overall within statistical variances (because random is indeed random), it's working out how we want.