The LL algorithm

bbigler

Humans have a bad habit of assuming that all good or bad things happen on purpose, due to the actions of others.  Sometimes, things happen by random chance, but we don’t like dealing with randomness because it’s out of our control.  And unfortunately, we have a hard time accepting this and want to control everything, which is a human flaw.  

Aweberman

atomzed said:

Aweberman said:

These pulls have simply been recorded in a note on my phone, rather than by video, if that matters.  But if anybody is willing to see this raw data, I'm happy to share it.

I think it is awesome you track it to such an extent! Can you share it? I am curious to see. 

Some of the interesting things that we can see include 
(1) what are the dry spells and hot spells of pulls? 

(2) assuming someone pull as they go along, whether they can cover who they want
(3) how much LL pulls the person would need to keep up. 

Sure thing, @atomzed.

Here's the link: http://checkwolf.com/mpq.txt

It's about 4500 pulls.  Because of the informal and quick way it's formatted, it may also be helpful to consult this for a general timeframe:
https://forums.d3go.com/discussion/comment/928558/#Comment_928558

Some other notes about my tracking methods: 
- The list is sorted in reverse chronological order, so the most recent week is first.
- Each week is comprised of two lines: 1) the characters in tokens, and 2) the pulls themselves
- For the purposes of this list, a week begins on Monday morning.  I open all of my tokens from rewards for events that ended Sunday night/Monday morning, then start a new week.
- The characters in tokens are listed in alphabetical order and represented in the pull records as "a", "b", and "c", based on that order. Sometimes the names are shortened strangely; this is to keep the formatting on my phone consistent.
- Numerals and capital letters represent 4* pulls; I always reset the count at the start of the week.  These are designed to facilitate quick addition when figuring statistics in my head.
- The numbers at the end of the first line are 5* pull rate and current status of each character.

Here's an example, from a few weeks ago:

Abigail 5/Big Wheel 9/Elektra 1: 2/21 7/6/1
12345678a1234567890Ab

Here's an explanation of this coding:

Line 1: It is Abigail's 5th week in tokens, Big Wheel's 9th, and Elektra's 1st. So, this is March 7-13, 2022.  The second piece of information tells me that I had 21 LTs for the week, from which I pulled 2 5*.  The third bit tells me that at the end of the week (technically, on the morning of March 14), Abigail had 7 covers, Big Wheel had 6, and Elektra had 1.  (I had more trouble covering Big Wheel than any other character since Onslaught; I actually pulled the 13th cover for Big Wheel on my 306th LT with him in tokens, the Saturday before he exited.)

Line 2: The week started out with 8 consecutive 4* pulls, followed by one for Abigail.  There were then 11 consecutive 4* pulls, capped with one for Big Wheel.  (When tracking 4* pulls, a "0" actually means "10", at which point I switch to capital letters.)

Here's another part of an example:
My worst-ever stretch was when I opened a mini-hoard after Adam Warlock entered tokens.  The first part looked like this:
a1234567a1bc1b1ab1234b123c1234567b123456789a1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ123c1234567890ABCDa

So after going 11-for-44 (including a best-ever stretch of 6-for-9), I followed up with an 0-for-39; this was marked by the numbers 1 through 0, then A through Z, then back to 1-3.

I'm happy to answer any other questions.

Bad

I think I supplied my lack of maths knowledge thanks to Google. 
The possible combinations of 9 powers between 36(numbers of times I got a 5*) would be m=9 and n=36 and the result would be a combination where it doesn't matter the order of the elements and it can be selected one element more than one time and it would be:

n! / ((n - m)! * m!) = 708930508.


So inside 708 millions of combinations on real statistics draws certainly a result as optimal as mine would be a miracle(and repeated throughout 3 hoards more, 3 miracles).
And if this math operation is correct(made on a online combinations calculator) this topic should be as clear as day: for pulling on a game like this it's needed an algorithm like this one.

Yepyep

Bad said:

Yepyep said:

Haha that's an amusing but inapt analogy. As far as we know, there is no invisible hand of the market overseeing cancer outcomes with an eye to profit. Even less is there an owner of the entire cancer game who created the game for fun (us, and cancer isn't fun; my partner is in chemo and I am NOT making light of this) and profit (them, though cancer is massively profitable.

Was I making light of this? Because I don't remember. Like you said it was just an analogy.
Altought not related to this game and fyi there is already an innovative process called car-t and it's already saving lives and less hurting than chimio but still is expensive and there are logistic issues.

No, you weren't. I had referred to it as "the entire cancer game," not you. I just wanted to be clear I wasn't making light of it. 

And I wasn't saying that you are wrong. I was just saying that you can't do what you are proposing to do because it is an impossibility. You are trying to infer an algorithm using a minuscule dataset. It's like taking a drop of seawater and attempting to infer the pattern of distribution of salinity in the sea it came from. Or maybe more like a can of Coke and trying to infer the recipe for all Cokes in the world because that is a manipulated system like our game is.

Yepyep

Vhailorx said:

bbigler said:

Clearly, there’s a lot of disagreement to Bad’s theory.  So, I’ll state the facts and then my opinions:  

Facts:
1. Your personal 5* results could fit into multiple theories, so just because a theory fits your results doesn’t make it true.

2. The best theory fits all results from all players, avoids confirmation bias and can be proven by others.  No data is excluded in order to “save” a theory and it also makes logical sense.  

3. It’s been discussed and agreed upon before that your personal string of 5* pulls are determined in advance (to some unknown number). 

4. The results from all players that track a large number of pulls show that it is always near 15%, but I have not seen any consensus that the RNG tries to give out “missing covers”. 

My Opinion:
1. I do not consider 300 pulls to be a large enough sample size to determine statistical averages.  3,000 pulls would be far more accurate, but still doable.  

2. I’m very confident as a software engineer that their algorithm hasn’t been touched in years.  (If it works, don’t change it). I’m also confident that they implemented this in the simplest way possible, unless player complaints had them change it.  Most likely, it’s a simple RNG system with each pull being independent.

As I said, we should remember the law of parsimony.  If the two options we are considering are (1) there is one algorithm and it's 15% (5+5+5) and all the anecdotes about unusual hoard cracking results are noise, or (2) there is a baseline algorithm + multiple hidden algorithms that adjust outcomes based on real $ spending or which 5* will leave the LT pool next or some other factor, then we should always rely on (1) just because it is so much simpler and explains all the observed data just as well.

Ah, Occam's razor. Of course your point is correct. We are trying to divine information about a system. If the only two possibilities available to describe the system are your (1) and (2), then the principle of parsimony suggests that (1) is more likely than (2) to describe it. As you said.

But that doesn't mean that it does describe it. It's an artificial set of conditions, like taking a multiple choice exam with those as the only options. Using this approach is how some theists with a modicum of science-learning convince themselves of the existence of god: the two options they're considering are (1) god created everything or (2) complicated physics + gravity + inconceivably vast stretches of (space/time) inevitably resulted in everything. They apply Occam's razor and conclude that (1) makes more sense. 

 Bad said:

I think I supplied my lack of maths knowledge thanks to Google. 
The possible combinations of 9 powers between 36(numbers of times I got a 5*) would be m=9 and n=36 and the result would be a combination where it doesn't matter the order of the elements and it can be selected one element more than one time and it would be:

n! / ((n - m)! * m!) = 708930508.


So inside 708 millions of combinations on real statistics draws certainly a result as optimal as mine would be a miracle(and repeated throughout 3 hoards more, 3 miracles).
And if this math operation is correct(made on a online combinations calculator) this topic should be as clear as day: for pulling on a game like this it's needed an algorithm like this one.

The problem with this thinking is that if your specific instance was 1 in 708930508 then there were 708930507 other instances of equal likelihood, i.e. every other possible result you could come up with. So the oddity would be that someone else would get the exact result you got, not that you got it once. It's the repeat of that result that you could learn from, not that you got it once. In straight theory, absent application, the assumption would be that every instance of the 708930508 possibilities would occur in that many draws.

EDITED multiple times to try to be more careful with my words... lol

KGB

Bad said:

I think I supplied my lack of maths knowledge thanks to Google. 
The possible combinations of 9 powers between 36(numbers of times I got a 5*) would be m=9 and n=36 and the result would be a combination where it doesn't matter the order of the elements and it can be selected one element more than one time and it would be:

n! / ((n - m)! * m!) = 708930508.


So inside 708 millions of combinations on real statistics draws certainly a result as optimal as mine would be a miracle(and repeated throughout 3 hoards more, 3 miracles).
And if this math operation is correct(made on a online combinations calculator) this topic should be as clear as day: for pulling on a game like this it's needed an algorithm like this one.

Are you sure you used the right math formula or that you interpreted the results correctly? What google search did you use and what website to come up with this calculation.

KGB

Bad

KGB said:

Are you sure you used the right math formula or that you interpreted the results correctly? What google search did you use and what website to come up with this calculation.

KGB

Well obviously the maths level would be different across countries but it should be advanced high studies prior to enter university(and I studied the opposite branch of science a life ago).
On english, the combination calculator should be this
one https://www.omnicalculator.com/statistics/combination
There are results of 177 millions and 708 millions of combinations but I don't know which one is the right in this case
Of course, each time you get a 5* the millions of combinations increase dramatically.
Of those hundreds of millions combinations, which part of results would be optimal for to champ 2 character is something totally exceeding me.
I would bet a small fraction.

KGB

Bad said:

KGB said:

Are you sure you used the right math formula or that you interpreted the results correctly? What google search did you use and what website to come up with this calculation.

KGB

Well obviously the maths level would be different across countries but it should be advanced high studies prior to enter university(and I studied the opposite branch of science a life ago).
On english, the combination calculator should be this
one https://www.omnicalculator.com/statistics/combination
There are results of 177 millions and 708 millions of combinations but I don't know which one is the right in this case
Of course, each time you get a 5* the millions of combinations increase dramatically.
Of those hundreds of millions combinations, which part of results would be optimal for to champ 2 character is something totally exceeding me.
I would bet a small fraction.

Thanks for the link. You've used the wrong formula / website.

The total number of objects (36) and sample size (9) is not what you want. That says there are 36 unique covers and you are drawing 9 of them. That's not what you did.

Instead there are 9 unique covers (3 per character) and you are drawing 36 times with repetition to get the cover distribution for 3 characters. This site won't get you the answer you want since it's not the formula for doing that.

KGB

Bad

Now I'm pretty sure the correct answer is CR9,36 = 177232627.00000003
(704 millions should be if the order matters, something close to lottery).

On 177 millions of combinations there would be those valid combinations differents of mine for to champ the 2 on those number of pulls.

I got A 3/6/5, E 5/4/4 and C 3/3/3.
Different results could be A 4/5/5, E 6/4/4 and C 3/3/3 or A 5/5/3 E 4/3/3 and C 5/4/4, etc... Those results champing Crystal and Elektra, or Abigail and Crystal, and all the variations in there. How many combinations would be? 1 million? ( I think that's too much but it doesn't matter).
Well, there would be 176 millions of combinations not optimal for to champ 2 characters like:
    A 0/34/0 E 0/1/0 C1/0/0 
or A 15/0/1 E 9/3/0 C 5/0/5
or A 0/0/0 E 5/4/5 C 0/2/10
or A 1/1/1 E 20/3/7 C 0/0/3
These type of results I was talking about on the beginning perfectly valid on statistics combinations but however there is no one pulling them because we pull through an algorithm which helps us to champ characters.

bbigler

@Bad For the sake of the argument, let’s say that you’re right (even though I don’t believe so) and that their algorithm helps you to fully cover 5* characters.  How does that information help anyone?  People are still going to pull until the character is fully covered.  

bbigler

Vhailorx said:

bbigler said:

Clearly, there’s a lot of disagreement to Bad’s theory.  So, I’ll state the facts and then my opinions:  

Facts:
1. Your personal 5* results could fit into multiple theories, so just because a theory fits your results doesn’t make it true.

2. The best theory fits all results from all players, avoids confirmation bias and can be proven by others.  No data is excluded in order to “save” a theory and it also makes logical sense.  

3. It’s been discussed and agreed upon before that your personal string of 5* pulls are determined in advance (to some unknown number). 

4. The results from all players that track a large number of pulls show that it is always near 15%, but I have not seen any consensus that the RNG tries to give out “missing covers”. 

My Opinion:
1. I do not consider 300 pulls to be a large enough sample size to determine statistical averages.  3,000 pulls would be far more accurate, but still doable.  

2. I’m very confident as a software engineer that their algorithm hasn’t been touched in years.  (If it works, don’t change it). I’m also confident that they implemented this in the simplest way possible, unless player complaints had them change it.  Most likely, it’s a simple RNG system with each pull being independent.

As I said, we should remember the law of parsimony.  If the two options we are considering are (1) there is one algorithm and it's 15% (5+5+5) and all the anecdotes about unusual hoard cracking results are noise, or (2) there is a baseline algorithm + multiple hidden algorithms that adjust outcomes based on real $ spending or which 5* will leave the LT pool next or some other factor, then we should always rely on (1) just because it is so much simpler and explains all the observed data just as well.

I agree that the simpler algorithm is what they implemented because a more complex algorithm is more prone to bugs and harder to maintain.  In addition, a more complex algorithm that isn’t so random would be more easily discernible by the players.  And they don’t want players to exploit the system.  

I think their main goal is to keep the players playing the game, and having a random gambling like reward system keeps people coming back for more.  The pretty flashing colors are the little rewards to keep you happy.  They want to keep you on the edge of satisfaction, always wanting more.  

Bad

bbigler said:

@Bad For the sake of the argument, let’s say that you’re right (even though I don’t believe so) and that their algorithm helps you to fully cover 5* characters.  How does that information help anyone?  People are still going to pull until the character is fully covered.  

Now that's the million dollar question.
And my answer is: because if you pull when the algorithm refreshes, the previous pulls very well could not help you anymore. 
Which brings us to the next million dollar question. When the algorithm refreshes? And my answer is: I'm not sure but pulling in the way I did certainly it's inside it.

atomzed

Aweberman said:

atomzed said:

Aweberman said:

These pulls have simply been recorded in a note on my phone, rather than by video, if that matters.  But if anybody is willing to see this raw data, I'm happy to share it.

I think it is awesome you track it to such an extent! Can you share it? I am curious to see. 

Some of the interesting things that we can see include 
(1) what are the dry spells and hot spells of pulls? 

(2) assuming someone pull as they go along, whether they can cover who they want
(3) how much LL pulls the person would need to keep up. 

Sure thing, @atomzed.

Here's the link: http://checkwolf.com/mpq.txt

It's about 4500 pulls.  Because of the informal and quick way it's formatted, it may also be helpful to consult this for a general timeframe:
https://forums.d3go.com/discussion/comment/928558/#Comment_928558

Some other notes about my tracking methods: 
- The list is sorted in reverse chronological order, so the most recent week is first.
- Each week is comprised of two lines: 1) the characters in tokens, and 2) the pulls themselves
- For the purposes of this list, a week begins on Monday morning.  I open all of my tokens from rewards for events that ended Sunday night/Monday morning, then start a new week.
- The characters in tokens are listed in alphabetical order and represented in the pull records as "a", "b", and "c", based on that order. Sometimes the names are shortened strangely; this is to keep the formatting on my phone consistent.
- Numerals and capital letters represent 4* pulls; I always reset the count at the start of the week.  These are designed to facilitate quick addition when figuring statistics in my head.
- The numbers at the end of the first line are 5* pull rate and current status of each character.

Here's an example, from a few weeks ago:

Abigail 5/Big Wheel 9/Elektra 1: 2/21 7/6/1
12345678a1234567890Ab

Here's an explanation of this coding:

Line 1: It is Abigail's 5th week in tokens, Big Wheel's 9th, and Elektra's 1st. So, this is March 7-13, 2022.  The second piece of information tells me that I had 21 LTs for the week, from which I pulled 2 5*.  The third bit tells me that at the end of the week (technically, on the morning of March 14), Abigail had 7 covers, Big Wheel had 6, and Elektra had 1.  (I had more trouble covering Big Wheel than any other character since Onslaught; I actually pulled the 13th cover for Big Wheel on my 306th LT with him in tokens, the Saturday before he exited.)

Line 2: The week started out with 8 consecutive 4* pulls, followed by one for Abigail.  There were then 11 consecutive 4* pulls, capped with one for Big Wheel.  (When tracking 4* pulls, a "0" actually means "10", at which point I switch to capital letters.)

Here's another part of an example:
My worst-ever stretch was when I opened a mini-hoard after Adam Warlock entered tokens.  The first part looked like this:
a1234567a1bc1b1ab1234b123c1234567b123456789a1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ123c1234567890ABCDa

So after going 11-for-44 (including a best-ever stretch of 6-for-9), I followed up with an 0-for-39; this was marked by the numbers 1 through 0, then A through Z, then back to 1-3.

I'm happy to answer any other questions.

Thanks! Let me get to a computer to extract this and make sense of the data.

Bad

Today I had some spare time and I went so far as to try to develop and write down optimal combinations. 
I chose a easy example, how many optimal combinations getting 39 succesful 5*s and champing the 3. I lose enough time and I got tired lol, so finally I rounded numbers for possible missing links.
This post could effective finish and complete this thread. 

All 3 being 5/4/4: 

A 544 E544 C544. A454 E544 C544. A445 E 544 C544

A 544 E 454 C544. A544 E445 C544. 

A 544 E544 C454. A544 E544 C445

A 454 E454 C544. A445 E445 C544.

A 454 E544 C454. A445 E544 C445.

A 544 E454 C454 .A544 E445 C445.

A 454 E544 C454. A445 E544 C445 

A 454 E454 C454. A445 E445 C445

17 combinations 

All 3 being 5/5/3: exactly the same. 17 combinations 

Mixed:

-A 544 E544 C553. A544 E544 C535. A544 E544 C355.( *A545 E544 C553. A445 E544 C553. A545 E445 C553. A544 E553 C544. A544 E535 C544. A544 E355 C544. A553 E544 C544. A535 E544 C544. A355 E544 C544. A553 B454 C544.A553 B454 C355. A553 B445 C544) (+12 combinations)

-A553 E544 C544  A535 E544 C544. A355 E544 C544.(* same as above +12)

-A544 E553 C553. A544 E535 C535. A544 E355 C355 (*)

-A553 E544 C553. A535 E544 C535. A355 E544 C355 (*)

-A553 E553 C544. A535 E535 C544. A355

E355 C544 (+12)

87 combinations 

Total 124 combinations. 

Rounding for possible missing branches I didn't calculate: + - 300 optimal combinations for to champ 3 characters. 

Now, the total possible combinations for 9 powers getting 39 succesful 5*s is:

CR 9,39= 314457495.00000006

So the combinations not optimal for to champ 3 characters getting 39 succesful 5*s is: 314457195.00000006.

314 millions not optimal versus 300 (more or less) optimal combinations. 

KGB

There are a lot more combinations of mixed cases that you guessed (300).

That said it's still unlikely (~35%) to get a perfect distribution from just 39 covers. That's why the suggestion is 300 tokens (~45 covers) so you get 6 extra AND you set your shards targets to 2 of the 3 so that you get another cover for 2 of them via shards to potentially fix a problem where 45 still isn't enough.

A couple of years back, someone wrote a simple program to do all this. It just divided up 39 5* tokens into 9 covers and calculated whether it had a perfect distribution for champing and they ran it something like 1 million times to get the ~odds of it happening. Then they did the same for 40 tokens, 41 tokens and so on until they got a 90% confidence number of getting enough to champ. It was 410 pulls you needed to reach 90% confidence in champing (this was before shards).

It was Hadronic. Here's a plot of his results.  If you search for his name you'll find a lot more related posts around this time since then bonus heroes were introduced instead of the shard system we have now.

https://forums.d3go.com/discussion/comment/851105#Comment_851105

https://forums.d3go.com/discussion/73395/new-system-means-new-plots/p1

KGB

bbigler

Are you purposely ignoring the probability between optimal and non-optimal combinations?

Because it’s far more probable to get this:
5/4/4, 5/4/4, 5/4/4
than to get this:
0/0/0, 0/0/0, 13/13/13
or to get this:
13/0/0, 13/0/0, 13/0/0
or to get this:
2/2/2, 3/3/3, 8/8/8

Bad

bbigler said:

Are you purposely ignoring the probability between optimal and non-optimal combinations?

Because it’s far more probable to get this:
5/4/4, 5/4/4, 5/4/4
than to get this:
0/0/0, 0/0/0, 13/13/13
or to get this:
13/0/0, 13/0/0, 13/0/0
or to get this:
2/2/2, 3/3/3, 8/8/8

And why? It's exactly the same probability for all those combinations. All the balls are inside the bag.
The fact that you always got a similar result like the first thanks to the algorithm doesn't mean that combination has more probabilities.

KGB said:

There are a lot more combinations of mixed cases that you guessed (300).

That said it's still unlikely (~35%) to get a perfect distribution from just 39 covers. That's why the suggestion is 300 tokens (~45 covers) so you get 6 extra AND you set your shards targets to 2 of the 3 so that you get another cover for 2 of them via shards to potentially fix a problem where 45 still isn't enough.

Thanks for the link.
I tried to calculate 3 champed characters with 39 pulls because it's more easy thanks to the 544 and 553 patterns. I'm aware that there would be several millions of results champing 2 and dozens of millions champing one. But also there would be dozens of millions not champing anyone.
The "golden number 300" increases slightly the chances by 6 5*s more. And at the same time increases the million of combinations doubling them, so increases the amount of not optimal combinations along with the optimals, as well as any added 5* entering the mix.

bbigler

Bad said:

bbigler said:

Are you purposely ignoring the probability between optimal and non-optimal combinations?

Because it’s far more probable to get this:
5/4/4, 5/4/4, 5/4/4
than to get this:
0/0/0, 0/0/0, 13/13/13
or to get this:
13/0/0, 13/0/0, 13/0/0
or to get this:
2/2/2, 3/3/3, 8/8/8

And why? It's exactly the same probability for all those combinations. All the balls are inside the bag.
The fact that you always got a similar result like the first thanks to the algorithm doesn't mean that combination has more probabilities.

This is where your logic is faulty.  They don’t all have the same probability!  It’s NOT like a single pull from an enormous bag of balls.  It’s 45 pulls from 45 small bags.  

I think you can understand if you consider this:

You pull your first 5* cover, and let’s say it’s #1 out of the possible 9.  On your next 5* pull, are you more likely to get cover #1 again or the other 8?

Obviously, your odds are better to get the other 8.  The probability of getting the same 5* cover twice is 1/9 = 11%.  So, let’s say your 2nd pull is cover #4.  Is your 3rd pull more likely to be cover #4 again or the other 8? 

On your next pull, your probability of getting a different cover than your last pull is 89%. In this example, is your 3rd pull more likely to be covers #1 or #4 again or the other 7?

Of course, you’re more likely to get the other 7 because there are more of them.  Follow this pattern and you’ll see why you get natural even distribution over time.  

That’s why we don’t need a special algorithm to help us champ characters 

Bad

If that reasoning was realistic no lotto ticket on any lottery should be different than 23456

bbigler

Bad said:

If that reasoning was realistic no lotto ticket on any lottery should be different than 23456

First, that’s not the same thing.  

Second, do you seriously disagree with known truths that are universally accepted and provable?
Third, I’m giving up trying to explain this to you