Card draw stats and an opinion of why

Houdin

So just as an interest moment. Even though with booster crafting coming in now this becomes a bit of a moot point, but it interests me and I know many others, So I thought I'd get some opinions. 
 I know there is much debate whether card draw is weighted in some way. 

I firmly believe the code is written to weight for cards you already have. 

Which is why some of us have multiples of cards that others have been chasing forever.
 

Just as a moment of statistics. I went back to take alook at the mythic order I pulled from amonkhet that I tracked.

At one point I pulled the same four mythics 12 times in a row.

So help me if I'm wrong here.

That is an individual event of 4 out of 24. 

So 1/6.

For each time one of those same cards was pulled consecutively the math runs as 1/6 × 1/6 × etc to 1/6 choose twelve.

Which gives me a probability of occurrence for the set of....

One in two billion, one hundred and 76 million seven hundred and eighty two thousand, three hundred and thirty six.

Which seems a bit not well...random. lol

Now having looked at that kind of math. This is what I believe occurs after thinking about it for the last two years.

. 

New set comes out.

You have no mythics.

The set contains 25 different mythics.

We will call them 

Group A - 12 cards 

Group B - 8 cards

Group C - 5 cards.

The rng gods smile on you and a mythic appears.

The code checks to see if you have any cards. 

You do not so it weights the group's as follows.

A = 60% 

B = 40%

C = 10%

You receive your card based on those odds.

Time passes and you get another mythic.

The code checks to see you do have one.

Instantly that card receives a 50% probability.

The remaining group's percentages are now calculated from the remaining 50% probability.

I'm sure it's more complicated but something like that explains the groupings of cards to the individual player.

Volrak

Houdin said:

At one point I pulled the same four mythics 12 times in a row.

So help me if I'm wrong here.

That is an individual event of 4 out of 24. 

So 1/6.

For each time one of those same cards was pulled consecutively the math runs as 1/6 × 1/6 × etc to 1/6 choose twelve.

Which gives me a probability of occurrence for the set of....

One in two billion, one hundred and 76 million seven hundred and eighty two thousand, three hundred and thirty six.

A useful analogy might be 1,000 people each making 1,000 die rolls.  Let's say that somewhere in all those rolls, someone got 10 rolls in a row which were the same number.

You could look at the odds of that specific person getting that specific result for the 10 specific rolls on which it happened, and say it's one in 60 million, and conclude that the dice must absolutely be rigged.  But if you frame the scenario correctly, including how many rolls were made in total, and, given that you're choosing how to measure randomness after already seeing the data, considering how many different "interesting" events that could occur that would result in a similar conclusion (e.g. 10 of one of the other 5 numbers in a row; 10 in a row of rolls of only two numbers, n in a row instead of 10 in a row, n in a row without getting two consecutive rolls the same, etc, etc), then suddenly seeing at least one of all those things happen somewhere might actually be odds-on.

In your case, framing the scenario correctly might consider the full set of players it could happen to, the entire set of all card drops over their playing history, and the entire set of "interesting" events (e.g. max runs of m from a pool of n for all different m and n, x dupes of one card with 0 of another, number of PPs without a mythic, without MP, with only guaranteed rare, etc, etc, etc).  For your specific example you'd also need to account for the fact that the first 4 cards could have been literally anything; if there were dupes amongst those, the 5th could also be literally anything, etc, until an initial group of 4 was formed, and only the remaining ~7 drops actually mattered to get the observation you had.

None of the above is to give any evidence that mtgpq drops aren't rigged towards giving dupes.  I'm just hoping to explain why even apparently improbable individual scenarios like this don't carry anywhere near as much weight of evidence as it might seem at first.

Volrak

So if we're serious about understanding dupe rates, here's something we can do right now to test it out.  I've set up a sheet at https://docs.google.com/spreadsheets/d/1JUeuf0BcOlNHpt3dA7SQ838ws7AOlWMEz-4AAG2dYrE/edit#gid=157340072 (it's the ORI mythics tab) where anyone can enter data about the number of origins mythics they own.

• Why Origins?  It's the set least affected by biases such as elite packs, event rewards, exclusive cards, and staggered set releases.
• Why mythics?  Despite having less data due to less drops, it's mythics that people want to know about.
• Why now?  When booster crafting hits, we'll lose all our dupe data.  Now's the last time we'll be able to capture it this easily.
• What about privacy?  If you really don't want others to know your collection, you can enter anonymously.  (But you're going to orb into a whole bunch more ORI mythics soon anyway, right?)
  

Once we have all the data we're going to get (presumably when booster crafting arrives), I'll take the data entered and compare it with data generated using uniform random drops.  Anyone else will also be welcome to perform their own analysis, as the entered data will remain public.

The more data we get, the more likely we can reach some kind of conclusion.  So please contribute!

ElimGarak

Done. Good work @Volrak

majincob

Thanks @Volrak ! 

Tilwin90

@Volrak nice effort
I will take some time today or tomorrow to input this information from my other spreadsheet where I already calculated my dupes and conversion crystals (I don't trust manual scripts so in the case of a bug I am prepared to give full data for proper conversion)

Theros

Houdin said:

So just as an interest moment. Even though with booster crafting coming in now this becomes a bit of a moot point, but it interests me and I know many others, So I thought I'd get some opinions. 
 I know there is much debate whether card draw is weighted in some way. 

I firmly believe the code is written to weight for cards you already have. 

Which is why some of us have multiples of cards that others have been chasing forever.
 

Just as a moment of statistics. I went back to take alook at the mythic order I pulled from amonkhet that I tracked.

At one point I pulled the same four mythics 12 times in a row.

So help me if I'm wrong here.

That is an individual event of 4 out of 24. 

So 1/6.

For each time one of those same cards was pulled consecutively the math runs as 1/6 × 1/6 × etc to 1/6 choose twelve.

Which gives me a probability of occurrence for the set of....

One in two billion, one hundred and 76 million seven hundred and eighty two thousand, three hundred and thirty six.

Which seems a bit not well...random. lol

Now having looked at that kind of math. This is what I believe occurs after thinking about it for the last two years.

. 

New set comes out.

You have no mythics.

The set contains 25 different mythics.

We will call them 

Group A - 12 cards 

Group B - 8 cards

Group C - 5 cards.

The rng gods smile on you and a mythic appears.

The code checks to see if you have any cards. 

You do not so it weights the group's as follows.

A = 60% 

B = 40%

C = 10%

You receive your card based on those odds.

Time passes and you get another mythic.

The code checks to see you do have one.

Instantly that card receives a 50% probability.

The remaining group's percentages are now calculated from the remaining 50% probability.

I'm sure it's more complicated but something like that explains the groupings of cards to the individual player.

you forgot to add weigh pull not based on what you already own. good example is elite packs. what people are more likely to get is the average or the least useful card in an elite pack. Torment of Hailfire and wormcoil engine are good examples of what many have reported within the coalition. kozilek and emarakul were perfect examples when the Olivia mythic pack were in vault.
you get the least useful card from an elite probably to make you spend and try another shot

HypnoticSpecter

Bumping for folks to enter data in Volrak's chart above before tomorrow. 

Volrak

Thank you to the 23 people who shared their ORI mythics dupe data before booster crafting.  A total of 282 mythics were entered, which is enough for an interesting analysis.

Mythic totals from all collections ranged from 18 Nissa's Revelations and 17 Day's Undoings, down to 7 each of Skaab Goliath and Thopter Support Network.  This large range might appear to be evidence of non-uniform randomness, but if you actually take the time to simulate 282 uniform random drops from 24 possibilities, you would learn that a large range similar to 18-7 is completely expected.  (If you're not convinced, and you're handy with Excel or coding, you should try it and see for yourself.)

In the graph below, the red line shows the entered mythic totals, sorted by number of drops for easy comparison.  The blue line shows simulation results of 282 uniformly random (non-tiered) mythic drops, with error bars of 1 standard deviation from the simulated averages.  The entered data tracks well with this simulation.



The light grey line shows what happens if "dupe stacking mode" is turned on in the simulation, which aims to do something like this:

The code checks to see you do have one [mythic].

Instantly that card receives a 50% probability.

The remaining group's percentages are now calculated from the remaining 50% probability.

I've implemented it as every drop beyond the first within each of the 23 simulated collections gets a 50% chance of being a guaranteed dupe.  You can see that turning "dupe stacking mode" on no longer matches the entered data.

The next two graphs show results for simulated tiered drop rates, rather than uniform random.  Again, the red lines are the entered data, and the blue lines show the simulations.  The grey lines show the same tiered simulations with dupe stacking turned on as well.

With mild tiers (17%, 22%, 28%, 33%):


With more weighted tiers: (10%, 20%, 30%, 40%):


None of the tiered results track well with the entered data, with or without "dupe stacking mode" enabled.  The line most similar to the hypothesis of the first post in this thread is the grey line of the third graph.  What actually happened (red line) markedly different.  Out of all simulations, the only one which explains what actually happened is plain old uniform random drops.

Thanks again to everyone who shared their data, and also to OP who proposed an actually testable hypothesis - that goes beyond what most armchair theorists on these forums have been willing to put forward.

Caveats

• Back in the day, QB was set up to award a mythic of a particular colour as a prize, with the colour rotating between each QB.  ORI mythics aren't uniformly distributed between colours.  So mythics coming from QB prizes would make the recorded drops less uniform than what we can expect from actual drops.  Only one entered collection listed QB prizes as a significant factor.
• Some collections converted dupes at some stage then collected more mythics.  This would make recorded drops more uniform than what we can expect from actual drops.  Nobody reported this as a significant factor.

Gilesclone

Thanks again @Volrak for more excellent analysis!

span_argoman

Thanks @Volrak for putting this to rest.

Volrak

span_argoman said:

Thanks @Volrak for putting this to rest.

No worries - but to be clear, the lack of anything strange in this analysis doesn't prove that nothing strange can ever possibly occur.  I'm always interested in things that look odd (even if I expect them to typically be false alarms).

span_argoman

Volrak said:

span_argoman said:

Thanks @Volrak for putting this to rest.

No worries - but to be clear, the lack of anything strange in this analysis doesn't prove that nothing strange can ever possibly occur.  I'm always interested in things that look odd (even if I expect them to typically be false alarms).

Yup I know what you mean. Testing a null hypothesis and proving that it's wrong doesn't mean the alternative hypothesis is correct. Merely that it is more likely.

Houdin

I'm going to have to disagree. Just to make an example. In two years I never once pulled. Pull from tomorrow. In all the hundreds of origins packs. Since crafting the card I have gotten it 3 times. The same holds true for other crafted cards. Having never received them before, now that I have them I am pulling duplicates of the cards. 
I do realize that statistically this is a possible event although on the outskirts of improbable. However, when you must constantly use the extreme edges of a probability bell curve to allow for events. It becomes more likely that events are weighted towards that extreme as a norm.

Bil

      It's been a while now since people started to feel there was something wrong about the random machine around here and we've seen tons of examples.
       The data we got access to thanks to volrak post clearly shows there's another rarity factor beside the basic rarity of cards. We got endless examples of chain duplicate pulls that seem beyond any statistical prediction.
      I'm almost sure those duplicate factors do apply to elite packs too which is even more indecent when we talk about 400 hard to get pink and a no exchange policy.
      It's been a while since the player base asks for transparency about the random machine and as we can see with this data,  it would be nice and smart from the devs to finally explain why the random is broken around here.

Volrak

Houdin said:

Just to make an example. In two years I never once pulled. Pull from tomorrow. In all the hundreds of origins packs. Since crafting the card I have gotten it 3 times. The same holds true for other crafted cards. Having never received them before, now that I have them I am pulling duplicates of the cards.

I do realize that statistically this is a possible event although on the outskirts of improbable.

But is it really close to impossible?  This is kind of my point.. without being a bit scientific about it, we're stuck making blind guesses about how probable something is or is not.  (For starters, you'd have to know how many AKH packs (not ORI) you've opened before and after crafting.)  It's well known that probability is something that human intuition really sucks at, not to mention that relying on memory instead of hard data just isn't rigorous in any sense. 

If your pull data is truly non-random in a way which hasn't been shown before, then you could prove it by recording it carefully (here is a top-notch example of someone who's decided to do just that; other people are recording their pulls in the drop rates spreadsheet).

Houdin

Just to give some concrete numbers. We will take my duplicates when crafting dropped as I had counted them using the booster spreadsheet.
Keep in mind two things.
1. I am a whale. Sad and I hate myself for it everyone I look in the mirror lol.
2. I only stopped converting dupes after kaledesh.

I had 163 origin rare duplicates at that point.
That's just in a portion of the two years I have been playing. 

Again. I'm sure the probability curve taken overall for all players would allow for this in my case alone. The problem is that this happens to a plethora of players. The majority of players cannot by definition sit on the edge of a curve.

DBJones

As volrak said, Pull from Tomorrow is from Amonkhet not Origins, so the number of Origins duplicates you've gotten is irrelevant.

julianus

And another factor you'd need to incorporate for accurate estimation is when certain cards were released. If I remember correctly, Pull from Tomorrow was not part of the initial set of Amonkhet cards, it was part of a secondary release. So, depending on when you opened each of your various packs, it may not be as improbable that you didn't get it earlier. And more broadly, this demonstrates the challenges in really trying to accurately estimate the probabilities.

Of course, I've long thought that the drop rates (at least for rares and above) were weighted, but that's more from personal disappointment than hard data.