Page 1 of 2 12 LastLast
Results 1 to 10 of 13

Thread: Better pre-release testing

  1. #1
    Join Date
    Mar 2012
    Posts
    158

    Default Better pre-release testing

    The Steam launch seems to include at least two fairly common fatal crash bugs (thrown items and window focus, both now in the issue list). This is starting to generate some negative responses on Steam.

    I love the game and I'm totally here to support - I funded the kickstarter, I bought the Steam game, I left a positive review... but frankly I'm a little embarrassed to be recommending a game with fatal crash bugs at launch. To the extent that I felt the need to include release stability as a caveat in my review.

    I feel a little bad that I only got an hour of play in on the last release before Steam went public... but then as far as I can tell, we only had that version for two days before launch.

    With Steam opening up the game to a demanding, paying audience, perhaps there should now be a better pre-release test system. Can we get a Steam test client like Dota 2 has? And honestly, I think any version should get at least a few weeks of play with a wide pre-release testing group before it's put onto Steam.

    That's my 2c. I'm down to help.
    Last edited by shockeroo; 11-17-2015 at 11:33 PM.

  2. #2
    Join Date
    Sep 2012
    Location
    San Francisco, CA
    Posts
    675

    Default

    Well...

    if it makes you feel any better, in my day job, we just launched an app update that crashes in some Locales due to date parsing mistakes.
    And other folks unable to log in because of other problems with the Authentication server update.
    And other folks with random crashes because who knows why.

    It happens, and that's computers in the 21'st century. We will fix it in version 4.0.2, I promise!

  3. #3
    Join Date
    Dec 2012
    Location
    Wales, UK
    Posts
    27

    Default

    The negative responses on Steam are for the most part an attempt to get the team bugfixing - disappointing, because they're already doing so!
    I suggest a dedicated Bug Reports sub-forum on the Steam page to discourage that avenue of feedback.

    Regarding the release, we've all been crying for it for months, so understandably the dev team were probably under quite some pressure to schedule as soon as possible.
    The crashes really -are- a serious issue and people will recommend the game far better when they're sorted, but because of that, I expect they'll be sorted VERY soon.

  4. #4
    Join Date
    Mar 2012
    Posts
    158

    Default

    I don't accept that the release system can be left as-is. This isn't an isolated occurrence; we had another release with a fatal crash in R58 less than six months ago. Under the current system this is very likely to happen again - something that I do not think is acceptable with a paying audience.

    Ultimately this is of course up to the creator, but were I managing this project (which is my day job), I'd be revising the release system as a priority as soon as the Steam version is stable.

  5. #5
    Join Date
    Dec 2008
    Posts
    1,467

    Default

    I fully agree with the OP. You all know I'm a fan of ADOM and its dev team, but I was rather puzzled when I saw that they were going to make the grand Steam release with only one day and a half for testing by the community. Honestly that is asking for trouble, in fact I would have been surprised if there were no serious bugs.

    Note that I'm not speaking in hindsight - this is something I already said several times on Facebook and probably the forums: "no matter if you release earlier or later, with less or more features, but *please* let the community test the Steam release well, as bugs can leave a bad impression".

  6. #6
    Join Date
    Dec 2012
    Location
    Wales, UK
    Posts
    27

    Default

    Sorry if I was unclear - I DO agree that more stringent beta testing might be an idea, and I'm not trying to shout that down. Reviewing the system is a good idea.

    I just think that in the Steam instance it was something that a lot of people were putting pressure on to be brought forward, plus I don't know how much notice Steam required for release. This is all just supposition but it may well be that the game couldn't have been released to crowdfunders any earlier because it took longer to complete than expected, but the Steam release date wasn't alterable by that point. Maybe two days wasn't intentional?

  7. #7
    Join Date
    Jul 2012
    Posts
    489

    Default

    Quote Originally Posted by Al-Khwarizmi View Post
    I fully agree with the OP. You all know I'm a fan of ADOM and its dev team, but I was rather puzzled when I saw that they were going to make the grand Steam release with only one day and a half for testing by the community. Honestly that is asking for trouble, in fact I would have been surprised if there were no serious bugs.
    yeah the testing was a little quick.

    these guys aren't professional though, you kind of expect a few guys.

    hopefully the sales go well and we can have someone professional doing the fixes etc full time for awhile.
    Wins:
    Gnome Assassin / Dwarven Paladin (ULE) / Ratling Duelist (UNE) / High Elf Archer / Gnomish Wizard x ~5 / Gray Elven Wizard (UCG) - Archmage / Gnomish Weaponsmith / Grey Elf Elementalist / Dwarven Priest / Trollish Barbarian / Drakling Farmer / Mist Elf Wizard / Human Beastfighter / Ratling Archer (UNE) / Gnomish Mindcrafter

    Classes Left
    Many!

  8. #8
    Join Date
    Mar 2008
    Location
    Germany, Witten
    Posts
    994

    Default

    Quote Originally Posted by Kelibath View Post
    I suggest a dedicated Bug Reports sub-forum on the Steam page to discourage that avenue of feedback.
    It's already there: http://steamcommunity.com/app/333300/discussions/2/
    Thomas Biskup
    ADOM & Ultimate ADOM Maintainer
    https://www.adom.de - https://www.ultimate-adom.com

  9. #9
    Join Date
    Mar 2008
    Location
    Germany, Witten
    Posts
    994

    Default

    Quote Originally Posted by shockeroo View Post
    The Steam launch seems to include at least two fairly common fatal crash bugs (thrown items and window focus, both now in the issue list). This is starting to generate some negative responses on Steam.

    I love the game and I'm totally here to support - I funded the kickstarter, I bought the Steam game, I left a positive review... but frankly I'm a little embarrassed to be recommending a game with fatal crash bugs at launch. To the extent that I felt the need to include release stability as a caveat in my review.

    I feel a little bad that I only got an hour of play in on the last release before Steam went public... but then as far as I can tell, we only had that version for two days before launch.

    With Steam opening up the game to a demanding, paying audience, perhaps there should now be a better pre-release test system. Can we get a Steam test client like Dota 2 has? And honestly, I think any version should get at least a few weeks of play with a wide pre-release testing group before it's put onto Steam.

    That's my 2c. I'm down to help.
    Thanks for the feedback.

    I should give an official statement on this:
    1. Nobody is more embarrassed about the current problems than all of us at Team ADOM. For weeks before the release we had a very stable version and due to some last minute changes (yeah, never do that, we know) we broke something in the game. We are still investigating and also considering to roll back to an earlier version of the component if we can't find the problems quickly.
    2. I am solely responsible for the decision to do such a "sudden" release. There are many reasons behind it and the whole decision process actually was pretty complex. To give you some ideas:


    • 2015 went differently than expect and I am the one to blame. My real life job required a lot more work than ever expected (I am one of three CEOs of a German IT company called QuinScape). 2015 has been an absolutely amazing year for the company and we grew big time but that ate up so much energy and time that I often was too exhausted to do meaningful extensive work on ADOM. If I ever was close to a burn-out it probably was this year. This was even more frustrating as ADOM already was "close" to a Steam release at the end of last year. So we built quite a bit of frustration for ourselves.
    • In the end the pieces came together in such a way that by mid October there was a small time window visible for a release within the next eight weeks. Sadly the timing proved very difficult: We were sure that it made no sense to release ADOM during a Steam sale. So we had to pass the Halloween sale. But autumn/Christmas sales are another regular event so we had to time that, too. Christmas itself also was off-limits. Directly after Christmas might have been an option.
    • But besides pressuring ourselves supporters from the crowdfunding campaign naturally inquired about the very slow progress. Rightfully so. So we decided to not let another 2-3 months go by without much changes.
    • We could have developed more on ADOM during that time but we didn't want to do it to have a very stable Steam release.
    • Which in the end we botched by exchanging one central component in the last minute (expecting many benefits) and in the end causing certain stability problems.

    So it's a combination of bad timing, unexpected real life events, growing pressure and stupid last minute decisions that led to the current result: a somewhat unstable Steam release (and I say "somewhat" as I also got lots of feedback saying 'I'm not having any problems').

    So what are we going to do? Easy:
    1. Not dwell on the past but look towards the future. Meaning: get our job done quickly and release a bug fixed version ASAP.
    2. And then resume delivering content with determination and energy. Lots of great ideas already have sprung up from three days of discussions with fresh ADOMites and I'm looking forward to the many positive effects this will have on the evolution of ADOM.


    So sorry for any temporary trouble you might experience right now - blame me. We do not do this intentionally or out of lack of interest and these troubles for sure hurt us more than you... at least in the mid and long term.

    So I'm now back to trying to do my best supporting the required bug fixes.

    See ya soon!
    Thomas
    Thomas Biskup
    ADOM & Ultimate ADOM Maintainer
    https://www.adom.de - https://www.ultimate-adom.com

  10. #10

    Default

    ADOM is a labor of love produced by some dudes in their sparetime, not the work of a gaming studio (e.g. Darkest Dungeon) or some indie dev doing it full time (e.g. cogmind). Yeah it's a bumpy ride sometimes, but compare what we have now to what we had pre-resurrection. I'm very excited for ADOM's transition and everyone I've convinced to buy the game in the last couple of days has been pleased.

    Admittedly, I would be more pleased if I hadn't precrowned with cat's claw yesterday but what can do you.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •