Return to “Suggestions”

Post

Handling of alpha and beta

#1
(I think the technical thread was probably the wrong place for this, hope this is now the right place for this discussion)

I hope this is the right place for this, but I think as alpha is early next year, some thought on how alpha testing then later beta testing will be handled?

for alpha, then I would suggest using the forum, but for beta, maybe a online bug tracker should be thought of been used? there is an old game called warzone 2100 (now freeware and opensource) using the trac bug tracker

http://developer.wz2100.net/

its not a bad system, I used it a bit to help report bugs about the game, but I certainly think for a larger player base of beta testers, then a online tracker would be far more useful then a forum based one, especially when dealing with new builds and such

I think this should be discussed fairly early, before the alpha test which is planned early next year, so getting ready for it, and planning on what kind of system is going to be used is well worth doing IMO

there are a few other online bug trackers, but I think Trac is possibly a decent one to consider, not sure what it takes to set the thing up mind, looks like you install it yourself, but worth a thought, as I don't think a forum is very good for beta testing, not if there are a lot of bugs and users, then its very likely (consdering we are talking about one person dev team here) bugs might get missed, using an online tracker like Trac bugs won't get missed, and can be sorted by milestone and versions

again worth a consider, before the events take place, unless others got better ideas
Post

Re: Handling of alpha and beta

#3
I would agree for the alpha yes, as we might not really be testing that much, and the number of testers is roughly going to be in their 70's so far, but this may change, but still a small number, and a limited amount ton test

but in beta, I don't think this will work I beta tested for Egosoft X3 reunion, and they did use the forum as a bug tracker, and they used a code system to handle the bugs reports, but I don't think it was really an effective system, especially when there was build after build (NDA expired) as it did start to get pretty confusing after a while and some bugs kept on returning, (weren't fully fixed) including an infamous one called Bob (named by a fellow SP tester) but then of course there were a lot of staff to to keep an eye on the forums, and delete / close any unneeded posts, but it did get confusing after a while when we had to test one beta after the next ( the game was very rushed) so I personally don't think a forum for beta testing and one guy doing all the work is such a good idea) bugs will get swamped by new bugs, previous version game bugs still active for whatever reason

so I still strongly suggest and online tracker for beta testing, if you take a look at Trac, you can sort bugs by milestone, version, and very easy to see what bugs are for what build of the game, and can easy close those that are dealt with, but can easy reopen the bug report if need be, I think for a single person, this will be a far better systemm then a forum, which is why I suggest it, alpha fine in forum, beta, I would strongly be against, it will or may get too much for everyone, especially if people keep posting the same bugs, or similar, with a bug tracker like trac this shouldn't be too much of an issue (but will no doubt will still happen) but sorting by version / milestone will be very easy
Post

Re: Handling of alpha and beta

#4
Shame the aplha is quite expensive, I'm about to go with black friday special offer of beta but I don't think I'm going to spend 75$ on LT right now ... chrismas are behind the corner :D
I read somewhere here about Josh considering lowering the cost of alpha/beta test access ... is that true and if so is there some more info on that ?

Thx, Vl.
Post

Re: Handling of alpha and beta

#5
I think alpha is pretty decently priced, if there was a deal to lower it, it would I think it will make the lower beta access tiers a bit worthless, as $60 for the beta and full works (inside information etc) then the black friday deal at $45,

so really if alpha comes down anymore, it would I think make the other tiers a bit worthless really, but I think its fine as it is, (granted its good value for a UK backer that is)
Post

Re: Handling of alpha and beta

#6
matthewfarmery wrote:(granted its good value for a UK backer that is)
Well, that's the matter as well. I live in Czech Rep which is in the middle of the Europe. And the sallaries here are quite ... hm, let's say ... different :lol:

edit: ok, so I went with the $45 pledge allowing beta participation :) ... can't wait for that
Post

Re: Handling of alpha and beta

#7
vlasticz wrote:Shame the aplha is quite expensive, I'm about to go with black friday special offer of beta but I don't think I'm going to spend 75$ on LT right now ... chrismas are behind the corner :D
I read somewhere here about Josh considering lowering the cost of alpha/beta test access ... is that true and if so is there some more info on that ?

Thx, Vl.
He did consider it and ended up creating the Black Friday beta level. He said he will not lower the cost of the alpha level because he wants it to be rather exclusive.
Post

Re: Handling of alpha and beta

#10
matthewfarmery wrote:its not a bad system, I used it a bit to help report bugs about the game, but I certainly think for a larger player base of beta testers, then a online tracker would be far more useful then a forum based one, especially when dealing with new builds and such
If you're dealing with professional testers who classify bugs correctly, yes.
If the only qualification for a tester is having some extra money lying around then a bug tracker is better used internally - out of reach of your testers. =)
There is no "I" in Tea. That would be gross.
Post

Re: Handling of alpha and beta

#12
There is no alpha access pledge level at this time. The $75 pledge is to obtain a combat prototype which will be released prior to the alpha stage of game testing, but does not include participation in testing. This is a very important distinction that Josh had made several times.
I am 42.
Post

Re: Handling of alpha and beta

#14
but alphas are basic anyway, that is the point of them, the prototype model can be classed as an alpha really, because there isn't that much there, beta will have access to a great deal more, so a forum for alpha / prototype is good in that sense, as there might not be much to test, or find bugs, but later, a forum based tracker I don't think works, and I know how they did it at egsoft, for a one man dev team, he would have to spend a lot of time managing the forums, at least with a online tracker, it should be a lot easier, due to the nature of having different versions too, if we do hit the 100k but to be honest with you, I think Josh will burn out when we hit beta, and having to deal with 3 ports, this is my greatest fear, and a forum based tracker will end up making this harder and tougher for all of us, most of all Josh himself, that is why I brought this suggestion up,
Post

Re: Handling of alpha and beta

#15
matthewfarmery wrote:but alphas are basic anyway, that is the point of them, the prototype model can be classed as an alpha really, because there isn't that much there
From a game developer prospective, no, the combat prototype is in no way an alpha. Alpha tests are generally internal tests of the completed game structure, not targeted testing of a single aspect of the game to iron out wrinkles in behaviors. Alpha tests are far more involved than what the combat prototype covers.

Literally, a game in alpha testing is in its first completed design stage - there may be some very small features still missing, but in general the game is a whole product which has been assembled and functions as a whole rather than as individual modules. The alpha stage allows the designer to ensure everything connects together as intended, there are no readily apparent bugs, and the game performs as expected when everything is 'turned on', as it were.

The beta testing stage is designed to crowd-source bug-finding and identify balance issues, which would not be readily apparent if only a single person/small team were working on testing.

Now, I agree that a forum-based bug reporting structure is not ideal. I doubt Josh is planning to go that route for anything beyond the prototype, though I would expect there to be a bug discussion thread so that tech-savvy players can help others with addressing issues that can be resolved outside of code changes. It also can help players to see what the progress of a bug is toward resolution if players can post what they have reported and what responses they have received.
I am 42.

Online Now

Users browsing this forum: No registered users and 2 guests

cron