Page 1 of 5 12345 LastLast
Results 1 to 20 of 93

Thread: Friday with Means - June 26th, 2009 - The dead awaken and consume coffee.

  1. #1

    Funcom employee Friday with Means - June 26th, 2009 - The dead awaken and consume coffee.

    This update was the perfect time to release a /facepalm animation. Most of the team is slowly recovering from being awake for almost 48 hours straight.

    There were a great many things in the update that we thought might destroy the universe.

    A new client RDB for everyone that allows us to add more files and process faster.
    Instanced cities for everyone.
    Instancing Pandemonium.
    Instanced playfields finally being displayed with the correct lighting and effects they were always meant to have.
    Messing with the City Advantages code.
    New apartments accessed in a new way.
    New Hostile Target nano window....

    Any of these could have done it...as easily in the past it could have been the social window...the mechs...the battlestations...or the raid system...or the new instancing system itself. So many opportunities to break the entire game...and force the rollback we have managed to avoid for about 6 years..since 12.6.

    What actually did it?

    The new system for cool-downs for nanos. In the past if we wanted an action to be unavailable for use (beyond the recharge/global cooldown of everything) we were forced to depend on silly blocker nanos..."You can not cast this while XXXXX is running." The new cooldowns meant we could dispense with the silly blocker nanos and properly display (on the shortcut bar even!) the amount of time until these actions were available again. This is where things went "bad".

    "Crashing" or exiting the game very rapidly would break these cooldowns and corrupt your character information. Anything contained in your character after this cooldown information was corrupted...this resulted in your research and perks effectively vanishing. Importantly your "Save" position also became corrupt...anyone who died in this state would then crash the entire server...not surprisingly breaking other characters on the same server who were also in this cooldown state. One broken character would then create more crashes...then more broken characters as a result...and from there it began to spread across all dimensions like a virus. Doctors had the longest cooldown of this kind in common use...so they were immediately the first players who were susceptible to this issue...and in the end what allowed us to locate the problem and eliminate it in the update we released at 5am on Thursday. The sheer number of characters affected by this issue left us no choice but to roll back the entire day...with no readily available way of detecting this kind of corruption in the database it was far preferable to months of crashes as these characters appeared on the servers and hurt them again....and other characters by crash issues that will always persist (ie: lost items due to server crashes).

    I am very sorry it took as long as it did to locate the issue...all the posts from the community on this issue were instrumental in helping us locate it as quickly as we did...the sheer horrible random nature of this issue (in terms of the symptoms of vanishing perks and research) made it very hard to identify. We are very lucky to have the talented and dedicated coders, designers and operational support teams that we have otherwise this issue could have been plaguing us for a much longer time.

    The population on Test was also incredibly helpfull in helping us put an end to this. It is also not fair to say that anything done on test or not done on test could have prevented this issue. Only a server crash during a precise 10 second window would have produced this result...and that is not the fault of the test population...they were extremely helpfull in making sure the other significantly risky improvements to AO in this update ran as smoothly as they did and we are very gratefull for their help. Thank-You Nirvelle for lending us your account! Nothing short of a huge live environment was going to expose this weakness...While I am sorry it happened at all I do think that it could not have been repaired as quickly or as well as it was in any other conditions.

    Stepping into the AO universe and making the kinds of changes we have been making for the past year are always horribly risky. I also like to think we are really beginning to make progress...and despite the loss of 11 hours this week I feel like we are starting to make good progress. Again I am sorry that any of the pain and suffering we normally deal with when combating the old issues of AO spilled out onto our players. I hope we can enjoy another 6 years before this kind of issue gets us again.

    I am going to choose to see the positive aspects of what we are all achieving together in making AO better rather than dwelling on this incident and letting it scare us away from making the dangerous changes that are necessary to make AO the game it always should have been. I love AO now...but it can be so much better as long as we never give up. We'll get there.

    This week despite disasters:

    Macrosun still found time this week to start work on an in-game mail system.
    Genele continued work on the 150-200 new instance.
    More balancing work and meetings abounded...we are just getting started.

    Thank you to all of you who provide the support and inspiration to the whole team to make the super-human effort required to run AO worth every grey hair. Making us laugh at 4am wasn't easy...but you managed it. Thanks again.

    Have a great Anarchy Online Birthday! Thank you all for helping us make it to an amazing 8 years. I'm looking forward to the next 8!

    /salute
    Last edited by Means; Jun 26th, 2009 at 16:54:55. Reason: I hope this makes sense...my brain is still not working as it should.
    Colin Cragg
    This "Means" Cake
    Past Game Director (2008-2011)

  2. #2
    1st

    w00t first time ever!

    "Macrosun still found time this week to start work on an in-game mail system."
    Nice!

    PS: I really think AO has loaded faster and zones faster now
    220 Doc ~ "Dr." Ittledoc As of 1 DEC 07 setup || 218 Trader ~ Ittletrad (TL7 gimpage on Trader profession) parked... TS monkey no reason to go any higher
    220 Enf ~ Ittlejohn (1HB FTW!!!) [E] || 146 Fix ~ Johnittle
    216 NT ~ Nanoblender || 11 Crat "WTB" Spellbot
    ~~~Pheonix Rising is Recruiting~~~
    YMMV; Guaranteed 5 min or 5 miles... whichever comes first

    Quote Originally Posted by BIGMANFORCE View Post
    and i umm also think that we should have a special nano that u can only cast on froobs,that turns them into lil monkeys and forces their char to work the APF elavator!

  3. #3
    nth

    Atleast I got references of new apartment stuff, without wasting 989M not that I would have any use of that cash ;D

    You made lot more sense than I did recently ;D
    Last edited by Ekarona; Jun 26th, 2009 at 16:59:44.
    Ekarona 220/30 Female Solitus Engineer, long term member of Northern Star and proper "poor" gimp.
    Ekaslave 220/low Female Solitus Trader, FLAT(TM) pricing TS, almost all can do!
    Ekaros almost there/almost there too Male Solitus Martial-Artist.
    Ekadv gimp/gimp Female Opifex Adventurer

  4. #4

  5. #5
    I love the new database.

  6. #6
    oh u want a laugh ey. how about this
    Former Assistant Director of Pack of Noobs

    Fake Friday with Means, and it worked!

  7. #7
    /drool

    Is there any chance the new global cooldowns can be modified to work with our tools? There's a rich vein of possibility for fixing MPs casting issues with this...
    bai2u!
    -::l2pvp!1::-
    Electronite: FFA also destroyed Clan hegemony when it comes to tower wars. Ironically the downfall was started by the most active pvpers. Another ironic thing is that the downfall happened due to pvm conflict. Silirrion: (We have pretty good anti-troll filters by now though) Means: Thong-wearing troxes will always be a part of this game and a point of AO pride. Keldros: Obviously reall trolls don't use conditioner Marlark: If this forum was Swedish in it's language .. id pawn you any day. 220 NT: tl7 is a joke most of the time. 90% of the people are double double dead. some are worth debuffing tho. Mastablasta: you guys are right and I'm wrong. Ebag: No. You alpha me'd due to the stat bug. More Ebag: I don't have any twinks currently, nor do I participate much in mass TL7 PvP (though I do go occasionally, usually just to watch). Questra: an MP in sneak eNSDed me and did about 20k damage in 10-12 seconds

  8. #8
    Quote Originally Posted by Chrys View Post
    Is there any chance the new global cooldowns can be modified to work with our tools? There's a rich vein of possibility for fixing MPs casting issues with this...
    SS is meant to block all casting
    220 Shade | 220 NT | 220 Crat | 220 Fixer | 220 Agent | 165 Adv

  9. #9

  10. #10
    gief more instances plx!
    Make that new instance interesting, with some tactics, not tank n' spank... a few different mobs would be fun also, sick of killing always the same faces :P
    PvP n' Stuff (more soon™)

    Proud President of Mercenaries

  11. #11
    Quote Originally Posted by Means View Post
    Thank-You Nirvelle for lending us your account!
    OMG ! Account sharing !!! Ban Means ! Ban devs ! Ban them all !!!!


    Oh well, great work on this patch, not everything is perfect (i'd even say the opposite) but it's good to see the A-team.. er... AO team still invest a lot of efforts to make our game better.

    This message is approved by Krumpf.

  12. #12
    Happy B-Day AO Big Thank You to Means and Crew getting it all worked out for us, and putting in all the overtime... My God, did you guys really stay awake 48 hours to work all these problems out? 0.o All of you deserve a big fat bonus for all you do. Looking forward to seeing all the new comming changes, new PF, mail system, ect... These really will help out alot.

    Take it easy over the weekend, you all deserve a much needed break.

    Cdogg, nice vid there made me laugh

  13. #13
    Since you replied to the one thread regarding revamping the noob experience I'd like to see more on your take on that. It's a good thing you put up that you'd seen it, a lot of us are worried about player retention and how that's starting to affect the game population.

  14. #14
    Quote Originally Posted by Graftmage View Post
    SS is meant to block all casting
    I know you NTs love that in order for your nemesis to survive, they have to disable all possibility of making your life miserable <3 But we'd like a second opinion plx xD
    bai2u!
    -::l2pvp!1::-
    Electronite: FFA also destroyed Clan hegemony when it comes to tower wars. Ironically the downfall was started by the most active pvpers. Another ironic thing is that the downfall happened due to pvm conflict. Silirrion: (We have pretty good anti-troll filters by now though) Means: Thong-wearing troxes will always be a part of this game and a point of AO pride. Keldros: Obviously reall trolls don't use conditioner Marlark: If this forum was Swedish in it's language .. id pawn you any day. 220 NT: tl7 is a joke most of the time. 90% of the people are double double dead. some are worth debuffing tho. Mastablasta: you guys are right and I'm wrong. Ebag: No. You alpha me'd due to the stat bug. More Ebag: I don't have any twinks currently, nor do I participate much in mass TL7 PvP (though I do go occasionally, usually just to watch). Questra: an MP in sneak eNSDed me and did about 20k damage in 10-12 seconds

  15. #15
    Thank you for completing the angel outfit (Veteran halo) and the new white outfit.(and light robe, working in social)

    The apartments are a nice addition, but where is teh orbital bar?
    Another birthday surprise on the way?

    More documentation on the GUI issue though, please

    Kind Regards
    Ariensky
    Humankind can not gain anything, without first giving something in return.
    To obtain; something of equal value must be lost.
    That is the 1st law of equivalent exchange


    Rubi-Ka needs: a nickel statue of an astronaut pointing at the sky
    With the description / plate saying:
    When the stars burn out and I find I lack the strength to continue...one of YOU wil pick up the flag and carry it forward.
    This really isn't a corporate product anymore...it belongs to all of us. Where it goes it up to us.

  16. #16
    Quote Originally Posted by Means View Post
    Any of these could have done it...as easily in the past it could have been the social window...the mechs...the battlestations...or the raid system...or the new instancing system itself.
    It was the BSs, they were bs to begin with and look at what they just did now, toss em.

    Quote Originally Posted by Means View Post
    What actually did it?

    The new system for cool-downs for nanos.
    No, it was the BSs darn it.



    Quote Originally Posted by Means View Post
    Macrosun still found time this week to start work on an in-game mail system.
    But you said you didn't like those. Be more useful if they applied it to player shops so GMSs allow us to search and receive things. Would even make shops in instanced cities possible I image.

    All the same that sounds handy and is appreciated.

  17. #17
    Quote Originally Posted by Means View Post

    ...

    The new cooldowns meant we could dispense with the silly blocker nanos and properly display (on the shortcut bar even!) the amount of time until these actions were available again. This is where things went "bad".

    ...
    A note on the cooldown.
    I wrote already in another thread but, I used to have my AO configureed so that the recharge time was only displayed on the actual nano I casted and not on every nano icon in my shortcut bar, the others just greyed out.

    Now I have the number on every single nano icon.

    The only possible configurations now are:
    1) all grey out with or without fade + timer on all
    2) all grey out with or without fade and no timer at all
    3) etc. ^^

    BTW
    Good work all(Testserver included)!

    PS
    On the road to a more WoW like AO, please don't remove the character skillings

  18. #18
    I remember 12.6 (Oh my God... am I that old?!) and honestly you managed well today.

    Your current team is progressively fixing things that we didn't hope to be fixed or improved anymore (like timers on nanos this time). We have been often told in the past that touching the legacy code for anything else than minor tweaks was not something that could be seriously considered... but you are doing it. You already did it. You want to continue... and even rebalance professions (something else we were told long ago that could never happen in-depth). OK, the least we can do to thank you is to grant you the right to cause Apocalypse once a year! (I said once!)

    It's good to see some old aspects of the game finally polished and working as they should have always done, like, once again this time, the cool-down timers on nanos.

    I hope that, the same way, when the new engine is about to be released and massively advertised, you'll be able to improve the game for newcomers, make it more entertaining and also make the learning curve more newcomers-friendly. (And get rid of so many misleading and confusing features, weapons, items that make no sense now, and made little anyhow in the past.) But that's another topic.

    On that day, though, make a social T-shirt and sign for us "Newbies welcome here" that we can wear or hold to tell new players we are ready to help them and they can talk to us.

    By the way, off topic again, do you think you'll have something to show or tell us soon about the new engine? I need badly to update the info page we made about the old new engine...
    Chrisax
    MP 220, AL 30, Research 70, active, Born on Rimor
    In AO since 2001
    Founder and elected President of Athen Paladins
    www.athenpaladins.org
    Member of the Clerical Staff of the Council of Truth

  19. #19
    Quote Originally Posted by Chrisax View Post
    On that day, though, make a social T-shirt and sign for us "Newbies welcome here" that we can wear or hold to tell new players we are ready to help them and they can talk to us.
    This please Nice idea Chrisax!

  20. #20
    Quote Originally Posted by Means View Post
    ...snip...
    There were a great many things in the update that we thought might destroy the universe.
    ...snip...
    Hmmmm....



    Thanks for the update... Lotsa exciting stuff.
    Mekhdoc 220/27/70 Equip | Mekh 220/28/67 Equip | Shadesch 220/21/70 Equip
    Mekhkeeper 220/22/70 Equip | Roflmao 220/15/50 | Fixyaself 200/23/64 Equip

Page 1 of 5 12345 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •