Tiered rarities

Volrak

Brace yourselves.  Today I received the first evidence of tiered drop rates within a rarity which stands up to scrutiny.  (Some recent discussion on this topic can be found here.)

What I received was a list of 138 cards which are proposed as the set of cards which are less likely to drop than their rarity would suggest.  The list came to me fourth-hand.  The original source is unknown, but is reportedly someone who looked at the source code of the game.  According to them, each card is assigned a hidden parameter of value 0, 1, or 2, and:

"if the value is 1, card drop rate will be halved comparing to drop rate of a cards with the same rarity
if it is 2 - the drop rate for this card will be 25% from normal drop rate for others cards with the same rarity"

The proposed list of cards and tier parameters is below.  I've added the set and rarity to each card and sorted the list in a way that's more useful (at least to me).

 Tier  Set  Rarity Name
 2     BFZ  m       Drowner of Hope
 2     BFZ  m       Felidar Sovereign
 2     BFZ  m       Hero of Goma Fada
 2     BFZ  m       Nissa's Renewal
 2     BFZ  m       Omnath, Locus of Rage
 2     BFZ  m       Serpentine Spike
 2     BFZ  m       Ulamog, the Ceaseless Hunger
 2     BFZ  m       Wasteland Strangler
 2     BFZ  r       Akoum Hellkite
 2     BFZ  r       Desolation Twin
 2     BFZ  r       From Beyond
 2     BFZ  r       Guardian of Tazeem
 2     BFZ  r       Ruinous Path
 2     BFZ  r       Titan's Presence
 2     BFZ  r       Veteran Warleader
 1     BFZ  m       Akoum Firebird
 1     BFZ  m       Bring to Light
 1     BFZ  m       Drana, Liberator of Malakir
 1     BFZ  m       Emeria Shepherd
 1     BFZ  m       Exert Influence
 1     BFZ  m       Greenwarden of Murasa
 1     BFZ  m       Oblivion Sower
 1     BFZ  m       Void Winnower
 1     BFZ  r       Beastcaller Savant
 1     BFZ  r       Blight Herder
 1     BFZ  r       Brutal Expulsion
 1     BFZ  r       Conduit of Ruin
 1     BFZ  r       Defiant Bloodlord
 1     BFZ  r       Endless One
 1     BFZ  r       Gruesome Slaughter
 1     BFZ  r       Lumbering Falls
 1     BFZ  r       March from the Tomb
 1     BFZ  r       Noyan Dar, Roil Shaper
 1     BFZ  r       Radiant Flames
 1     BFZ  r       Sanctum of Ugin
 1     BFZ  r       Scatter to the Winds
 1     BFZ  r       Shambling Vent
 1     BFZ  u       Breaker of Armies
 1     BFZ  u       Coastal Discovery
 1     BFZ  u       Drana's Emissary
 1     BFZ  u       Grip of Desolation
 1     BFZ  u       Grovetender Druids
 1     BFZ  u       Kor Bladewhirl
 1     BFZ  u       Malakir Familiar
 1     BFZ  u       Ondu Rising
 1     BFZ  u       Pathway Arrows
 1     BFZ  u       Plated Crusher
 1     BFZ  u       Processor Assault
 1     BFZ  u       Resolute Blademaster
 1     BFZ  u       Slab Hammer
 1     BFZ  u       Tajuru Warcaller
 1     BFZ  u       Ulamog's Nullifier
 1     BFZ  u       Ulamog's Reclaimer
 1     BFZ  u       Valakut Invoker
 1     SOI  m       Angel of Deliverance
 1     SOI  m       Engulf the Shore
 1     SOI  m       Geier Reach Bandit / Vildin-Pack Alpha
 1     SOI  m       The Gitrog Monster
 1     SOI  m       Ulvenwald Hydra
 1     SOI  r       Altered Ego
 1     SOI  r       Anguished Unmaking
 1     SOI  r       Cryptolith Rite
 1     SOI  r       Drogskol Cavalry
 1     SOI  r       Fevered Visions
 1     SOI  r       From Under the Floorboards
 1     SOI  r       Nephalia Moondrakes
 1     SOI  r       Scourge Wolf
 1     SOI  u       Aberrant Researcher /Perfected Form
 1     SOI  u       Call the Bloodline
 1     SOI  u       Daring Sleuth / Bearer of Overwhelming Truths
 1     SOI  u       Gisa's Bidding
 1     SOI  u       Hope Against Hope
 1     SOI  u       Howlpack Resurgence
 1     SOI  u       Inner Struggle
 1     SOI  u       Morkrut Necropod
 1     SOI  u       Neglected Heirloom
 1     SOI  u       Paranoid Parish-Blade
 1     SOI  u       Silverstrike
 1     SOI  u       Stensia Masquerade
 1     SOI  u       Uninvited Geist / Unimpeded Trespasser
 1     SOI  u       Veteran Cathar
 1     SOI  u       Kessig Forgemaster / Flameheart Werewolf
 1     KLD  m       Cataclysmic Gearhulk
 1     KLD  m       Combustible Gearhulk
 1     KLD  m       Noxious Gearhulk
 1     KLD  m       Torrential Gearhulk
 1     KLD  m       Verdurous Gearhulk
 1     KLD  r       Aetherflux Reservoir
 1     KLD  r       Authority of the Consuls
 1     KLD  r       Bomat Courier
 1     KLD  r       Eliminate the Competition
 1     KLD  r       Fateful Showdown
 1     KLD  r       Insidious Will
 1     KLD  r       Wildest Dreams
 1     KLD  u       Cloudblazer
 1     KLD  u       Contraband Kingpin
 1     KLD  u       Empyreal Voyager
 1     KLD  u       Hazardous Conditions
 1     KLD  u       Restoration Gearsmith
 1     KLD  u       Unlicensed Disintegration
 1     KLD  u       Veteran Motorist
 1     KLD  u       Voltaic Brawler
 1     KLD  u       Whirler Virtuoso
 1     AKH  m       Bontu the Glorified
 1     AKH  m       Hazoret the Fervent
 1     AKH  m       Kefnet the Mindful
 1     AKH  m       Oketra the True
 1     AKH  r       Archfiend of Ifnir
 1     AKH  r       Gideon's Intervention
 1     AKH  r       Honored Hydra
 1     AKH  r       Trial of Knowledge
 1     AKH  r       Trial of Solidarity
 1     AKH  r       Trial of Strength
 1     AKH  r       Trial of Zeal
 1     AKH  u       Bontu's Monument
 1     AKH  u       Enigma Drake
 1     AKH  u       Hazoret's Monument
 1     AKH  u       Kefnet's Monument
 1     AKH  u       Lord of the Accursed
 1     AKH  u       Oketra's Monument
 1     AKH  u       Rhonas's Monument
 1     AKH  u       Trial of Ambition
 1     AKH  u       Wayward Servant
 1     HOU  m       Angel of Condemnation
 1     HOU  m       Wildfire Eternal
 1     HOU  r       Doomfall
 1     HOU  r       Imaginary Threats
 1     HOU  r       Inferno Jet
 1     HOU  r       Overcome
 1     HOU  u       Appeal -> Authority
 1     HOU  u       Bloodwater Entity
 1     HOU  u       Claim -> Fame
 1     HOU  u       Consign -> Oblivion
 1     HOU  u       Farm -> Market
 1     HOU  u       Resolute Survivors
 1     HOU  u       River Hoopoe
 1     HOU  u       Struggle -> Survive
 1     HOU  u       Unraveling Mummy

Unfortunately, this proposed list was shown to me after booster crafting destroyed the easiest ways to test it (via dupe data).  But it's still testable by other means.  In the "bulk open" tabs of the drop rates spreadsheet, individual card drops have been recorded for various sets.  For example, the cards opened for 387 KLD rare drops (out of 58 possible KLD rares) have been recorded.  The proposed list says 7 of those 58 cards are "tier 1", and drop less often than the rest.  So what do the stats say?

Assuming uniform drop rates, it's common sense that we can choose any group of 7 KLD rares, and the chance of a KLD rare dropping as one of those 7 will be on average 7/58 (12.1%).  On top of that, given 387 drops, we can pick a confidence level we'd like to have (say 99.9%), and do some stats to get a confidence range around that average (such as 6.6% to 17.6%).  Another way to put it: If we run the experiment 1000 times, each time recording 387 rare drops, we can expect our chosen group of 7 KLD rares to get between 6.6% and 17.6% of the drops in 999 of those experiments (and probably only just outside that range in the 1000th run).

On the KLD bulk open tab, 15 out of 387 rare drops (3.9%) belong to the group of 7 in the proposed list.  3.9% is outside of our 99.9% confidence range (6.6% to 17.6%)!  So, assuming the proposed list is independent of the data, we can say with 99.9% confidence that the 7 KLD rares in the proposed list do indeed drop with lower chance than the rest of the KLD rares.  For other sets and rarities the trends look like they go in the right direction, but in most cases there isn't enough data to prove anything statistically.  Also note that we haven't proven the theory that the chance is "halved", just shown that the chance is lower.

"Assuming the list is independent of the data" is important - someone could look at a dataset of 387 recorded drops, pick the least-dropped 7 cards, and claim they are less likely to drop than the rest.  In that case, the analysis would be invalid, since we'd effectively be trying to use some data to prove itself.  The presence of two non-mtgpq cards in the list ("The Gitrog Monster" and "Gisa's Bidding") makes a genuine source more plausible to me, but the source of the list is still an open question.

Dropspot

Volrak you could try requesting link to booster crafting sheet some players did. I am sure at least a dozen made, probably the ones with larger collection of dupes. Not sure if this info would help, but it's the best source to info prior to BC.

andrewvanmarle

Its anecdotal of course but the list did coincide with the cards i crafted and never got.... 

Mainloop25

Same here, I don't have any of the tier 2 BFZ mythics and almost never see them outside of PvE. I only have one of the tier 2 rares, even. 

Gideon

As shocking as this gotcha moment is we have a new developer now. I hope moving forward they don’t assign different weights to cards in the same category. This is one of the reasons the game started to feel like a rotting banana. If they want to continue moving in the right direction then they should ditch this practice immediately.

majincob

Indeed, as I've said before, if they will be required to disclose drop rates they will probably clean up the rates to be uniform.

Corn_Noodles

majincob said:

Indeed, as I've said before, if they will be required to disclose drop rates they will probably clean up the rates to be uniform.

You're so silly. Why would D3 do that? What have they done that shows they would do something so helpful?

octal9

"if the value is 1, card drop rate will be halved comparing to drop rate of a cards with the same rarity
if it is 2 - the drop rate for this card will be 25% from normal drop rate for others cards with the same rarity"

I refute this claim of theirs, specifically. 

The card list matches what I've seen, but I think they're using publicized drop data to create the above claim, meaning (as you've mentioned) we can't use the claim to validate the data.

edit: MTGPQ's database contains cards that aren't in the game itself, such as Glassblower's Puzzleknot, or Ice Over, or Hyena Pack. The existence of them in this list (and yes, the list itself) just means that they've seen the same files anybody can see by literally looking at an APK without decompilation.

edit 2: It also leaves out the list of cards with SubRarity value 4.

JopeX37

Anecdotally, having opened a lot of BFZ (at least 15 old school 600 crystal PP's) plus tons of extra fat packs and boosters, I didn't get a single one of the "2" BFZ mythics until booster crafting dropped last week.  Additionally, 2's comprise of the only mythics I'm still missing from BFZ.  Similarly, several of the 2 rares were on my want list until BC last week as well.  Fascinating.  I'm curious about where this list came from and what cards may have potentially been on it for the sets not represented.

octal9

@JopeX37 as I mentioned the list comes from the game itself; the values are accurate. But I highly doubt the translation of those values (the bit I quoted above) are accurate. There's nothing I've seen that compares against this value in the game.

Again, I propose the originator (i.e., not Volrak) took already existing, publicized data to craft a narrative.

Volrak

octal9 said:

@JopeX37 as I mentioned the list comes from the game itself; the values are accurate.

This confirms data independence.  So as per my first post, the probability that KLD rare drops are uniform, and our observation of significantly fewer drops for the same 7 KLD rares mentioned in the proposed list occurs just by chance, is (far) less than 1 in 1000.

octal9 said:

"if the value is 1, card drop rate will be halved comparing to drop rate of a cards with the same rarity
if it is 2 - the drop rate for this card will be 25% from normal drop rate for others cards with the same rarity"

I refute this claim of theirs, specifically.

I agree with you on this - the claim about "2" cards is completely untested, and like I said about the KLD rares subset of the "1" cards:

we haven't proven the theory that the chance is "halved", just shown that the chance is lower.

I also tried simulating KLD rare drops and comparing the results to the entered data from the spreadsheet; there's no strong support here for the "halved" theory either.



Looking at the tail region (right side of graph), you can see that halving the odds of a group of 7 rares (simulated results in purple line) doesn't match the actual drop data (red line) all that much better than uniform odds (blue line).

The green line simulates something different: rerolling once if a card in the group of 7 is rolled.  But that puts the simulated tail even lower than the entered data.

A fairly good match (not shown) can be gotten by simulating the following: If one of the specified group of 7 is rolled, then there's a 25% chance of keeping the roll, and a 75% chance of rerolling once.  But this is getting a little elaborate, and we don't really have enough data to support such a specific theory.  So all I'm comfortable saying for now is the chance is "less" in some way.

octal9 said:

It also leaves out the list of cards with SubRarity value 4.

I'd be very interested to see such a list.

Gilesclone

Interesting.  The only part of my data not corrupted by crafting are the uncommons from my alt.  I have not yet and do not intend to craft uncommons for this account, rares are a better value in my mind.  There are 58 listed cards out of 376.  If they were equally likely, that’s 15.4%.  If they’re only 75% as likely, that’s 11.6%.  Of my uncommons from those sets, 13 out of 112 total uncommons were on the list: 11.6%.

It’s too small a sample to mean much, but it is an interesting result.

Marvaddin

Well, I don't know anything, but just to add an idea, the reroll thing might occur just when you don't have the card, being canceled if you own it. It just came to my mind as I read the list, because I had 2 copies of supposedly rare cards, like Doomfall and Threats from HOU. Unraveling Mummy also came to me a lot.

So BFZ has the rarest cards, huh? Now I know why even events give OGW rares...

Phillmoore

Not sure if this helps but with only 16 rares left to collect 7 of them appear in that list. Go figure 

wereotter

Corn Noodles said:

majincob said:

Indeed, as I've said before, if they will be required to disclose drop rates they will probably clean up the rates to be uniform.

You're so silly. Why would D3 do that? What have they done that shows they would do something so helpful?

Apple has made it part of their requirements to exist on iPhone. So they can leave it as is and people will know drop rates are tiered within rarities, or they can clean it up to save face. 

Volrak

Here's another way to visualise some of the "tiered rarity" cards.  The graphs below use individual drop data entered for rares and mythics in KLD, AKH, and HOU sets.  Each card is shown as a dot according to how many times it dropped in the recorded data.  For example, the first graph shows that one of the KLD rares dropped 12 times, two of them dropped 11 times, and so on.  The red dots are the cards from these sets marked "1" in the list of tiered rarities.


Certainly in the KLD and AKH sets, the "tier 1" cards (red circles) are dropping significantly less often than other cards.  It's a bit more inconclusive for HOU.

Some boring details:

• KLD cards here are excluding AER.
• AKH and HOU sets have cards which were released to the game after the initial release of the set; those cards are excluded here.
• No consideration is given to paid/event exclusive cards.  Excluding those could give a better analysis.

Zzyzzx

I know that the evidence doesn't exactly support what I'm about to express, but the gut feeling I get surely corroborates it.  And I know Volrak will say that it's the mercy of randomness, and my brain agrees whether or not my gut agrees. 
I've opened a lot of Origins packs, most of them the old-school Fat Packs & Big Boxes, and regrettably did not record most of those early pulls on the spreadsheet because most were pre-SOI.  Since the 2nd month this game released, I have chased a scant handful of Origins cards.
1) War Oracle - The only Uncommon card in Origins that I've had to craft.
2) Languish - One of the many Rare cards in Origins that I had to craft.
3) Helm of the Gods - Another Rare Origins card I had to craft.

Since crafting, I've drawn War Oracle twice in freebie packs.
I drew Helm of the Gods the same week after crafting in the monthly Origins pack reward just last week.
Today I just drew Languish and had to come and find this thread.

And now I realize that this thread discussed rarities inside rarities, not situational dupe probability (higher chance of pulling a rare I already have vs. one I don't).

Volrak

Zzyzzx said:

I know that the evidence doesn't exactly support what I'm about to express, but the gut feeling I get surely corroborates it.  And I know Volrak will say that it's the mercy of randomness, and my brain agrees whether or not my gut agrees. 

Eh... I'd go with *probably* at the mercy of randomness, because: "The explanation requiring the fewest assumptions is most likely to be correct." – William of Ockham

However: "Absence of proof is not proof of absence." – William Cowper

In any case: “In God we trust. All others must bring data.” – W. Edwards Deming

Krishna

So far, I have 8 Ixalan rares, and received 3 dupe rares of those. two of those just happened to be the rare thats in the PW bundle for sale.

Also, I received a dupe Ornithopter from Elite Pack and another poster famously received 2 dupe Ornithopters. you can calculate the odds on that and see that some cards within a rarity have a higher drop rate, or we're just extremely unlucky.