Page 3 of 3 FirstFirst 123
Results 41 to 50 of 50

Thread: Randomness of loot tables.

  1. #41
    Quote Originally Posted by Lheann View Post
    That RNG runs at a pace of 1000's of numbers per second. When a subsystem needs a random number it calls the RNG subsystem and the current number in the stream is bounded into the requested range and returned.

    What this does is prevent streaking within a single subsystem because no single subsystem can dominate the RNG number stream.
    no, it does not. as each event is independent of every other event, it cant prevent streaks.

    Quote Originally Posted by Lheann View Post
    And anyone that has played with C based simple RNG's will know that if you use every number generated in place the streaks and patterns will be obvious.
    no, your prng's implementation is obviously broken. it does not fit the criteria "every event is independent of every other event". because then there would be no pattern. and its very unlikely that patterns occurred during "playing around" are result of the prng implementation.

    in general you will have a high entropy/ low bitrate (pseudo) random source that is used to feed entropy to your prng whichs bitrate is only dependent on processor speed. ****ing around with this system wont help. in this case your assumptions were even exploited by you. these assumtions were backed up by means or sillirion who said that they have a special kind of randomness in oa. so everyone with little math knowledge knew there is something to exploit.


    Quote Originally Posted by Lheann View Post
    But what if 100% actually equaled a value of 100000?
    nothing, you program will just run slower

    Quote Originally Posted by Lheann View Post
    Does this smooth the RNG effects some (ie a type of filter)? The answer is yes. When A RNG runs the wider the range of validity for the return value the more likely that streaking will not be as apparent (ok this is more masking than it is filtering). Think of it like this.
    dropping lower bits doesnt do anything (today). your reasoning is based on wrong assumtions.

    Quote Originally Posted by Lheann View Post
    int dropRate = lookUpDropRateForItem( itemID ) ;
    int percentage = ( rand() % 100 ) + 1 ; // 1-100 range percentage
    if ( dropRate >= percentage )
    dropTheFats() ;
    code is wrong! hint: assume RAND_MAX to be 150

    Quote Originally Posted by Lheann View Post
    Would suggest that a game with such other complex systems went cheap on one of the most critical subsystems is laughable. I think like AAD and AAO and AR and DR that the overall loot drop system and math is very complex.
    thats not what we call "complex".

    Quote Originally Posted by Lheann View Post
    I design large scale distributed systems for the military as my day job and these type of discussions get my blood flowing and I can go on for some time about them. As always, feel free to rip, flame, compliment or disregard my post as you see fit.
    when you cant run a business yourself and you cant find an employment you go and kill people for money. you should have said that first.

  2. #42
    So let us try to address these comments.
    Quote Originally Posted by Eiffel View Post
    no, it does not. as each event is independent of every other event, it cant prevent streaks.
    Actually yes it does. I was referencing a working model that I have seen all the source for. Since you have not had the chance to review the source code that I did for the specific implementation I am discussing and referencing in the example I submit you have insufficient knowledge to make such a bold statement.

    Further you are implying a rule to a model you have no detailed knowledge of.

    Quote Originally Posted by Eiffel View Post
    no, your prng's implementation is obviously broken. it does not fit the criteria "every event is independent of every other event". because then there would be no pattern. and its very unlikely that patterns occurred during "playing around" are result of the prng implementation.
    Have you looked at any of the standard RNG calls and their source code? Do you even understand how the simple basic ones that are built into most programming languages work?

    Have you read the vast references on RNG's and all the discussion on how to make them work in a manner that eliminates streaking? Streaking is by far the worse problem followed by patterns that occur in RNG's of all type. There are government funded projects to create a true RNG that is free of these problems. While there are a few very nice ones that do meet this criteria they all have one problem and that is the need for external triggering inputs. So I will assume that FC does use one of these and instead uses one that is based on more standard approach. Which more than likely suffers some form of streaking issues. Again this is all assumption on all our parts but AO is almost 11 years of release time plus 4 years of development time so I am betting they are using a 1999-2000 time frame RNG at best. And those RNG models are pretty well understood and they clearly had streaking issues with the pure software based solutions.

    Quote Originally Posted by Eiffel View Post
    in general you will have a high entropy/ low bitrate (pseudo) random source that is used to feed entropy to your prng whichs bitrate is only dependent on processor speed. ****ing around with this system wont help. in this case your assumptions were even exploited by you. these assumtions were backed up by means or sillirion who said that they have a special kind of randomness in oa. so everyone with little math knowledge knew there is something to exploit.
    You do realize what you described is just an implementation of the theory I laid out? But then neither of us know what FC uses. Your description is a bit over the top for what I would suspect any game company uses. I bet FC's RNG is based on a software solution that uses on server time data to seed/reseed itself.

    And so what if I used math and logical deduction to identify a significant advantage for myself? Does this mean you would not do that same?

    Quote Originally Posted by Eiffel View Post
    code is wrong! hint: assume RAND_MAX to be 150
    If RAND_MAX was 150 (which it is not) then that code would not work correctly. But since RAND_MAX is not 150 which I knew when I wrote it then it works fine. Again if you have ever looked at the source code for rand() you would know. But for clarity here is one very popular implementation of srand and rand
    Code:
    void __cdecl srand (unsigned int seed)
    {
    #ifdef _MT
      _getptd()->_holdrand = (unsigned long)seed;
    #else /* _MT */
      holdrand = (long)seed;
    #endif /* _MT */
    }
    
    int __cdecl rand (void)
    {
    #ifdef _MT
      _ptiddata ptd = _getptd();
      return( ((ptd->_holdrand = ptd->_holdrand * 214013L + 2531011L) >> 16) & 0x7fff);
    #else /* _MT */
      return(((holdrand = holdrand * 214013L + 2531011L) >> 16) & 0x7fff);
    #endif /* _MT */
    }
    You will notice the 16 bit shifts in the code. that makes MAX_RAND 32768. So in my example the code is just fine. Maybe you should consider looking at how the actual call I referenced works before you say it does not work that way. Also verified that RAND_MAX is 32768 in all version of GCC 3.x and 4.x as well for the standard rand() call. The above is the published source for the Visual Studio compiler version.

    While we are here, you will notice the value holdrand in the functions above. You do see where it stores the full result of the rand call and is used the next time rand is called. It is because of this design that I said using the standard C based RNG's you will see streaking and that each event does in fact rely on the previous event. Only by reseeding prior to every call can you say that each event does not rely on the previous. This is something that the arc4random() call does. But because the reseeding mechanism is based on a value that rolls over it is possible to implement arc4random in a manner that does actually streak as much and as often as the standard rand() call. Of course if you follow the recommended method of use then it does what it is supposed to fairly well.

    Then on to the masking effect I was discussing. Because of the design of the above source code you will notice the obvious that any reduction in the range of the RNG is effectively compressing the results. This increases streaking. So my previous pseudo code that used a modulo of 100 to get a percentage results in 327 RNG values that equate to each of my percentage points. because of this compression effect you induce streaking in this type of RNG. That is why using a larger value say 10000 with this specific RNG implementation will mask this effect by changing the compression from 327:1 to 3.27:1.

    That brings us back to the one FC uses. It is most likely a seeded math formula that stores the results of one processing pass for use in the next processing pass and has some automatic reseeding mechanic. Again we are talking about a design from the 1999-2000 time frame most likely. We are sure that it has survived software migration of the servers from one host company to the next. That tells me it does not rely on any special hardware and further supports the an advanced version of the basic rand() or arc4random() style RNG. Of course this is a guess until FC just tells us.

    Quote Originally Posted by Eiffel View Post
    thats not what we call "complex".
    Well I don't know what you call complex but the referenced systems in AO are much more complex math than a straight up simple random call and compare for loot generation. Which is what I was getting at, why would AO cheap out on how loot generation works if there are examples of much more complex systems that we can all point to.

    I grant you it is not the level of complexity as say the math used in guide missile flight control systems, but relative to other math systems in AO the reference systems are complex.

    And anyway the whole post was about creating control randomness to allow for the "special ao randomness" as you called it.

    Quote Originally Posted by Eiffel View Post
    when you cant run a business yourself and you cant find an employment you go and kill people for money. you should have said that first.
    Actually I do run a business myself and if the systems I design are used to track down and kill people that are intent on harming my country then I say it was a damn good day at work. Using my skills to design and develop system that defend my country is a privilege I am proud of.

    Now your remark was hostile and uncalled for and I don't even understand why you chose to make it. Here we had moved this thread to a nice discussion and you come along and are hostile and personally attack me for my choice of work profession. Bad panda.
    Lheann
    President of When I Grow Up

    Lhisa - MA - RK1
    MaxKillz - Enf - RK1
    Namaru - Enf - RK1

    "If you find yourself loosing a fight, your tatics suck."

  3. #43
    Quote Originally Posted by Lheann View Post
    Actually yes it does.
    interleaving consumption does not change the fact that every event is independent of every other event. no, it does not work.

    Quote Originally Posted by Lheann View Post
    You do realize what you described is just an implementation of the theory I laid out? But then neither of us know what FC uses. Your description is a bit over the top for what I would suspect any game company uses.
    no, its not your theory. its the basic layout of a modern os's rng. and if you claim rand() to be broken badly, how come you exspect professionals to not look into that issue? and for me, on my implementation of the standard, it just works like described.

    Quote Originally Posted by Lheann View Post
    And so what if I used math and logical deduction to identify a significant advantage for myself? Does this mean you would not do that same?
    nothing ofc, thats part of the game. its bad coding, thats all.

    Quote Originally Posted by Lheann View Post
    If RAND_MAX was 150 (which it is not) then that code would not work correctly.
    [...] that makes MAX_RAND 32768. [...]
    for RAND_MAX= 100 the probability to roll a 5 is:
    p_5 = 1/100
    as for 55:
    p_55=1/100

    now assume RAND_MAX is not a multiple of 100, for example 150:
    p_5=2/150
    p_55= 1/150

    because 105 will get mapped to 5 by the modulo operation. got it? for 32768 you have the numbers 0-67 show up 1/327th more often than the numbers up to 99.

    Quote Originally Posted by Lheann View Post
    The above is the published source for the Visual Studio compiler version.
    ya, thats looking evil. but the point remains valid: your one line of code is simply broken. and if you use that super-fast prng you should have read the docs. and i bet they warn about that.

    Quote Originally Posted by Lheann View Post
    Well I don't know what you call complex but the referenced systems in AO are much more complex math than a straight up simple random call and compare for loot generation.
    thats a constant time operation. ok, its still complex, but thats the lowest class... its not even a loop.

    Quote Originally Posted by Lheann View Post
    And anyway the whole post was about creating control randomness to allow for the "special ao randomness" as you called it.
    your ideas simply do not work properly.

  4. #44
    Quote Originally Posted by Eiffel View Post

    a bunch of stuff
    I agree to disagree. My experience demonstrates clearly different results than you are predicting.
    Lheann
    President of When I Grow Up

    Lhisa - MA - RK1
    MaxKillz - Enf - RK1
    Namaru - Enf - RK1

    "If you find yourself loosing a fight, your tatics suck."

  5. #45
    Quote Originally Posted by Lheann View Post
    [...]
    As I said the data is old and the easy to present excel sheets are lost as they went with the hard drive failure that led me to upgrade to a new computer. And really recovering family photo's, bank data, professional items and email ranked much higher in importance than any AO related data. Though I do make regular backups of my pref folder now as you know that is a PITA to redo on many toons. I know, backup backup backup.
    [...]
    Some of what you've been posting is an interesting read, but your assumptions are constructed in such a way that they need the data which is no longer available in order to be confirmed; The reasoning in them is not sound by itself.

    In order to prove or even be reasonably certain your hypothesis regarding the broken drop rates is correct, we'd need some hard numbers. Saying you've been farming the area X times isn't enough; Someone might've killed some of the monsters there, making each run different.

    In other words: You'd need a representative(*) number of kills per NPC which drops the item(s) in question. You'd have to take note of how many NPCs you killed, and how many drops you got.
    Counting drops per hour just doesn't work for this kind of sampling, as circumstances could make you kill fewer (or more) NPCs in any given time frame.

    Further, if the number of kills is too low, you could be chalking up normal variety as abnormal variety. With an item which has 1% drop rate, it is statistically speaking completely possible to have 200 kills without any drops for then to have two drops in a row. A more detailed sample (more kills) will account for these variances.

    (*):To put it on the edge: It's not enough to kill a monster 100 times, without a single drop, to determine that an item which drops in 1% of the kills doesn't drop for a given time slot.
    ::: My Tools & Stuff :::
    ::: Cratine Savagedheals Enfine Zoewrangle Demoder :: Solitron Demotionform :: IRC Demoder Savagedlight :::
    ::: AOItems :: Blog :: CIDB :: HelpBot :: ItemsBot :: PlanetMap Viewer :: Tower Wars :: Twitter :::

  6. #46
    Quote Originally Posted by Demoder View Post
    Some of what you've been posting is an interesting read, but your assumptions are constructed in such a way that they need the data which is no longer available in order to be confirmed; The reasoning in them is not sound by itself.
    Yes, and I thought I acknowledge that the lost of the data was a blow to the theory. Hand written notes tend to loose meaning and important facts associated with them are forgotten. That is why I posted what I did and built on it. Turning all the notes back into data at this point feels like a waste of time since we are talking about it not being a representative set of data and that the data collected does not provide the insight needed to confirm the theory.

    Quote Originally Posted by Demoder View Post
    In order to prove or even be reasonably certain your hypothesis regarding the broken drop rates is correct, we'd need some hard numbers. Saying you've been farming the area X times isn't enough; Someone might've killed some of the monsters there, making each run different.
    First I have not ever said the drop rate was broken. I simply said it is effected by external events and postulated why that may be so.

    Ah here is a real stick in the mud for the desired method of data collection. This is a MMO. There is no way to actually ever achieve what you are asking for the mobs in questions in the OP and from my test data. So, then how do we proceed? You seems to be asking for a data set that is impossible to achieve given the system we are trying to collect data in. Yet in real life which is vastly more complex and full of interference we manage to collect data and build theories. So the acceptance of interference or noise in data set has to be made.

    Also you suggest that this data is to provide the drop rate of and item from a mob. Where as I think and hope I have been saying that I observed a difference in drop rate over time of an item that is a shared drop across many mob types that seems to correlate to an external event.

    Also note in my earlier post that I indicated my theory that each item type has counters vice each mobs loot table having counters. That in fact my theory is more about controlling the drop rate of an actual item type regardless of the mob type or loot tables of the mobs being killed.

    Quote Originally Posted by Demoder View Post
    In other words: You'd need a representative(*) number of kills per NPC which drops the item(s) in question. You'd have to take note of how many NPCs you killed, and how many drops you got.
    Counting drops per hour just doesn't work for this kind of sampling, as circumstances could make you kill fewer (or more) NPCs in any given time frame.
    Drops per hour is the data I collected and if that is not a usable data set then I guess I am done.

    There is no way to verify you are alone on a play field and stay that way the entire time, so there is no method of ensuring you are the only one killing aliens. Also we have no proof in support of or against that number of players in a play field affect certain kinds of drop rates. So any attempt to collect the data with only you on the play field could impact the results. Collector chest drops would support numbers don't matter but team missions would indicate that numbers do matter due to number of awards generated. So FC has actually used both mechanics and the safe bet is that they continue to do so in more systems that we realize.

    Quote Originally Posted by Demoder View Post
    Further, if the number of kills is too low, you could be chalking up normal variety as abnormal variety. With an item which has 1% drop rate, it is statistically speaking completely possible to have 200 kills without any drops for then to have two drops in a row. A more detailed sample (more kills) will account for these variances.
    And here is the biggest got you. No data set any one person collects can be large enough to be of any value. S10 has been around how long? How many aliens have been killed in S10? During that entire time and during data collection periods did FC leave the loot tables unmodified? If I say that Cha Grade 2's have a 2% drop rate from testing over two months for 2 hours a day as you like, FC could say well there are 65 months of 24 hours a day of kill data and that data says the drop rate is 8%. As each day goes by, the ability of the player to collect data that has any meaningful sample size goes away. Any theory based on player collected data is going to have assumptions in it.

    That is why I said many times now that there are assumptions in my write up. There are assumptions because of a simple truth. If we assume nothing, then the only statement we can postulate is "There is insignificant amounts of data and knowledge about the system in question to offer any viable theory."

    That is the equivalent of giving up and moving on to something else. So instead I offer a theory based on the data and testing I have done, coupled with some knowledge of software engineering and how problems are generally solved in systems like loot generation. I then submitted the theory to the community via these forums for a peer review. Last time I checked, the next step in the scientific process is that the community attempts to duplicated my results with the same test, similar test and evaluate the data that was presented. From that holes are identified and questions are raised.

    In this case your peer review is finding the data set lacking in the method it was collected and the type of data that was collected. That is fine and accepted and actually what I was hoping for. Your review findings are probably along the lines of theory based on a weak statistical small data set using collection protocol that does result in the type of data needed. Just saying that is my take away of your post.

    But does the data demonstrate anything? Yes I think it does. There was a pattern, that I used to make a crap ton of credits easily. It repeated day after day. Given that the play time was during the week at strange hours, it is reasonable to assume that the server population was within a standard variation and that S10 population was probably inside of a standard deviation as well. But these last two are assumptions that I can not prove or disprove.

    Now I will couple your post with Eiffel's about RNG's and offer this:
    If we are not able to take a controlled sample set of sufficient size on one side and that AO's is using a truly random RNG that adhere's to the "every event is independent of every other event" as the whole of its loot generation then there is no way to prove this.

    But if we accept that AO has to be fun to retain paying customers and that truly random drops would not benefit the fun in any meaningful manner. Then you can see where a system that controls drop rate % on an over time or kill structure could be a positive for the game. No company is going to make a MMO and have content at any range be completely susceptible to the Infinite Kills = Zero Drops. Likewise the Kills = Drops possibility is also unacceptable. So a system that relies on a pure RNG will never be the final solution.

    To date I have not used or seen any software system that provides the limiting and control structures needed to provide an over time randomness that is not susceptible to counter resets caused by external events.

    I now once again ask the community to offer up their own test and result and postulate their own theories on how the loot randomness works. It has been a great discussion with some very smart people. A few of you I would love to sit at the bar and have a drink with while talking software engineering in general. As always I am impressed by the level of knowledge and understand that AO players have. One of the best reasons to keep playing this game is the people that play it with me.

    TLDR: Have enjoyed this but it is cutting into my game now. Gotta go kill stuff.
    Last edited by Lheann; May 2nd, 2012 at 00:11:30.
    Lheann
    President of When I Grow Up

    Lhisa - MA - RK1
    MaxKillz - Enf - RK1
    Namaru - Enf - RK1

    "If you find yourself loosing a fight, your tatics suck."

  7. #47
    Quote Originally Posted by Lheann View Post
    But if we accept that AO has to be fun to retain paying customers and that truly random drops would not benefit the fun in any meaningful manner. Then you can see where a system that controls drop rate % on an over time or kill structure could be a positive for the game. No company is going to make a MMO and have content at any range be completely susceptible to the Infinite Kills = Zero Drops. Likewise the Kills = Drops possibility is also unacceptable. So a system that relies on a pure RNG will never be the final solution.
    Truly random drops only negatively impact fun if the drops are irreplaceable and sufficiently rare to the point that there is a significant chance of them not dropping past whatever the acceptable ceiling of kills happens to be. If an item that important has that low of a drop rate then the problem is not how the RNG is set up.
    I am wiser than any god or scientist, for I have squared the circle and cubed Earth's sphere, thus I have created 4 simultaneous separate 24 hour days within a 4-corner (as in a 4-corner classroom) rotation of Earth. See for yourself the absolute proof.

  8. #48
    Quote Originally Posted by Lheann View Post
    There was a pattern, that I used to make a crap ton of credits easily. It repeated day after day.
    Quote Originally Posted by Lheann View Post
    Then you can see where a system that controls drop rate % on an over time or kill structure could be a positive for the game.
    i can see why you want that
    so your assumption is that there was some kind of rate limiter in place for s10. none in s10 for an hour means maximum drop rate for some time until the drop rate gets adjusted again. this happens at times of low online population. its per se unfair. you not only have the advantage of undisturbed farming, you are part of a minority that could exploit that.

    Quote Originally Posted by Lheann View Post
    No data set any one person collects can be large enough to be of any value.
    no, you misunderstood that. what they actually want is small sentence, an advice, that helps them. if you cant give it to them they send you to gather more samples. if you have enough data and give them the math, they wont (be able to) read it. hey, i couldnt even convince you about you modulo op breaking randomness.

    Quote Originally Posted by Lheann View Post
    To date I have not used or seen any software system that provides the limiting and control structures needed to provide an over time randomness that is not susceptible to counter resets caused by external events.
    actually the internal state of my systems prng is saved at shutdown and restored at boot. and as long thats all you need you are good. if you need an internal state you just have to do the same with your own state. if you use the clock you need an internal clock you can save and restore. no magic involved. just coffee...

    i would not do that. theres nothing to gain but only to lose (i.e. a glitch with better drop rates at server start). for stuff like low symbs you can farm even rk mobs and by them with the creds earned. ao not being a single player games solves this fully. for stuff like spirit shroud - you dont really need that and theres a reason items are rare. where is the fun if you are "born" with a burden in your bank? and for everything one needs to progress like quest items - simply up the drop rates.
    one could put stuff like complete patterns, pred armor, muzzles into the boss loot at the end of an team encounter in sl (aka mishes). and maybe even sl-tokens - tradeable for a little money or a shroud, a wep... most important thing is that the game is playable and enjoyable for newcomers. if a vet needs that 100m symb he can farm that creds in a day anyway.

  9. #49
    Quote Originally Posted by Eiffel View Post
    i couldnt even convince you about you modulo op breaking randomness.

    no magic involved. just coffee...
    No I see and understand where any RNG that is not an even division of the modulo you intend to use does create a weighting effect to a subset of the possible returns. And that was the point of the example. I just cannot believe AO uses something so simple for loot generation. That was not ever meant to be an example of how to do it correctly. It was always meant as an example of what I could not except as being the solution.

    In one system our RNG was actually part of the database server. And as you describe maintained state through reboots and power events. Was a feature of that database server. It just had to be accessed via the API. It worked in the manner I described the first time for the most part. With API calls for specific types of returns with one being a straight up call for a percentage return. Not sure about the back end side as that was not something we had source for. I think the important part is that not every one writes their own and a lot of development shops will use a third party version. And not all third party versions are as robust as they could be.

    And in my case the last RNG I wrote actually had a different set of conditions to meet. It had to reproduce the same events if needed as it was part of a training system. And repeatably was required. I know defies the logic of RNG but customers always have stupid requirements about at least one item. So we made the seeding data part of the training scenario and used an interleaving math formula that was a variation of the rand() type math as we needed a much larger RAND_MAX. I guess you can call it a sequence generator if you wanted too. That way each scenario was repeatable but there was no repeating between different scenarios. Further I wrote it so that there was no weighting issue. Which was actually the hard part. Took some work but as you said it is just coffee or in my case tea.

    I have taken your post as a challenge and have actually read about 100-150 more pages of specifications and design documents of various RNG's. I am not the final word expert on RNG's and all I can ever do is express my experience using them and writing a couple for specific requirements sets. It sounds like you are far more involved in the field and have greater experience which is just cool. It continues to be an evolving field of study and that is awesome for those of us that use them. I actually found the one that uses atmospheric noise and quantum noise to be very interesting.
    Lheann
    President of When I Grow Up

    Lhisa - MA - RK1
    MaxKillz - Enf - RK1
    Namaru - Enf - RK1

    "If you find yourself loosing a fight, your tatics suck."

  10. #50
    The walls of text are making my eyes bleed, so I'll dump these here.

    1: Computer RNGs do not produce RNs, but rather imitate the behaviour of RNs. Regular interleaving should reduce the randomness, since the RNG is ultimately cyclic. Depending on your timescale, you could use the clock to modify your seed if you wanted, maybe.

    2: The algorithms require a seed. The same algorithm with the same seed produces the same pRN.

    3: It's really easy to map your RAND_MAX onto whatever bounds you want, preserving randomness and keeping uniform weighting. I lost the thread of the discussion on that.

    4: In testing randomness, any data set is useful. It becomes more statistically accurate as it grows, though.

    <Edit> 5: Computer RNGs all streak. It's a question of how fast and how ugly they are when they streak.

    Math Troll:
    A truly random number generator creating a droprate would choose a time between 0 and infinity, so it would statistically never gief phatz. Amirite?
    Last edited by Sromp; May 3rd, 2012 at 08:15:45.

Page 3 of 3 FirstFirst 123

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •