( search forums )
Beta Testing System
Soldat Forums - Soldat Talk - General Discussions
YoZiZzO
March 1, 2006, 2:28 am
I'd really like to know the justification for having a closed beta test, as opposed to one open to the public. The closed system seems selfish and pointless, but feel free to prove me wrong.

Open testing leads to:

-Greater chance of discovering bugs
-The opinions of a much greater body of players (A lot of useless and biased opinions will exist this way, obviously, but can be neglected. Still, it is preferable to a small group who may consider themselves objective. Think of it like something that becomes open source; a handful of bad but with many great additions.)

I believe one of MM's reasons for a closed test was the fear that people would whine at him before the patch is actually released. Well, that logic seems to be flawed considering the experimental weapon balance has already been posted for the next version.

rabidhamster
March 1, 2006, 2:55 am
it is also possible that the beta didn't want to be mistaken for an actual version. after a while, i'm sure some 11 year old would come here and download the beta thinking, "soldat beta? is that the new name for it? hey guys! there's a new sodlat! it's called soldat: beta edition!"

edit: oh yeah, welcome to the forums :D

YoZiZzO
March 1, 2006, 3:17 am
 Quote:Originally posted by rabidhamsterit is also possible that the beta didn't want to be mistaken for an actual version. after a while, i'm sure some 11 year old would come here and download the beta thinking, "soldat beta? is that the new name for it? hey guys! there's a new sodlat! it's called soldat: beta edition!"

edit: oh yeah, welcome to the forums :D


Heh. I suppose, but noobs do all sorts of more annoying things, so that shouldn't be a concern.

Thanks for the welcome.

-VzX- Silverflame
March 1, 2006, 4:00 am
i dont care, im glad that i can help MM by testing it!!!!!11

FliesLikeABrick
March 1, 2006, 4:43 am
we've had many discussions about this and I don't remember the full list of reasons, but the ones I always advocated were:
1) It would be very difficult to manage server versions, client versions, and keep everything running smoothly in the public beta test. The extra time and effort required to organize this outweigh all the benefits.
2) The current beta testing schema accomplishes the task fine
3) There would be a lot of false bug reports to sort through, or repeated reports.
4) --Many other things brought up by other beta testers--

peemonkey
March 1, 2006, 4:43 am
It should stay this way in my opinion. The beta testers are the ones who know their [CENSORED], and even though everyone likes to blame them for when they dont like a version, they continue on and soldat's still aroond.

Deleted User
March 1, 2006, 4:54 am
It'd be bad to have a bunch of nabs like us beta testing a game. NOT A GOOD IDEA. I like the beta testers better, even if they did make m79 nabby.

Deleted User
March 1, 2006, 4:56 am
offtopic:whats a good private server i only play publics

ontopic:then there will be to much majors usen bugs

person
March 1, 2006, 6:50 am
 Quote:
2) The current beta testing schema accomplishes the task fine


That is very, very highly debatable.

Due to the other reasons you suggested, I can see why it would be preferable to not have the public beta test; but I think it should really be expanded.

I think I've mentioned before that maybe each region (ie: Asia Pacific, Oceania, East/West Europe and North/South America? - Those seem to be the areas which play the game most) should have a beta testing team or something, just to get a different perspective.


Some of the bugs in 1.3.1 make the beta testing team look as though they had done barely any testing - like the team change bug for instance. It happens SO OFTEN. And there are so many more little things...



One thing I'd like to know though, is how some of those maps make it to become default maps. Some are just awful, and never fail to clear a server of activity... ie: Dropdown/2 (both... and lets not forget Ash and Lanubia or whatever its called).
Again, I reckon any maps being considered for "default" status should go through some kind of testing, perhaps witht the same teams for beta testing.

FliesLikeABrick
March 1, 2006, 7:05 am
cita: please don't post offtopic stuff in threads, even if you have something ontopic to say too.

Person, maybe that one reason isn't that great, but like you said... the other ones make sense and justify it.

Ok
March 1, 2006, 8:47 am
Yea I was about to say that.
^_^
Even though I think they do a good job, its still debatable.

Nevertheless, a beta team that is chosen wisely is much better than just letting the mass public be a factor.
Espacialy when its the SOLDAT community

rabidhamster
March 1, 2006, 11:46 am
 Quote:Originally posted by citAthen there will be to much majors usen bugs

how bout, whenever a major joins, they explode, and the game removes itself, with a nice little message like, OMG YOU HAXXORED SOLDAT!

Hercule Poirot
March 1, 2006, 11:52 am
 Quote:Originally posted by person

One thing I'd like to know though, is how some of those maps make it to become default maps. Some are just awful, and never fail to clear a server of activity... ie: Dropdown/2 (both... and lets not forget Ash and Lanubia or whatever its called).
Again, I reckon any maps being considered for "default" status should go through some kind of testing, perhaps witht the same teams for beta testing.


Yes some of the maps aint really good but in the end it's Michals decision to take em or not we just recommend few maps that's all

Ash & Lanubia are like the best maps there was added imho i don't like dropdown2 or snakebite and some of the HTF game mode maps are pretty wierd o.o

And yes all new maps ARE tested in our beta testing sessions...

Michal Marcinkowski
March 1, 2006, 2:48 pm
A public beta test would look like a normal release. It's nearly impossible to get some constructive criticism from so many people. Closed betas have their disadvantages too but they fit Soldat at this stage.

ThaD
March 1, 2006, 5:08 pm
yup, Michal can always release a new version if something's wrong in the previous one :P

Deleted User
March 1, 2006, 6:14 pm
How many games publically test?

The beta testers were chosen for a reason, we should trust their descisions..

dropdown2 is better than Voland...

I actually sorta like ctf_crashed


ctf_run ftw.

But we will see more nice maps now that the mappers are armed with polyworks.

Swarmer
March 1, 2006, 7:01 pm
You could do a compromise. Make an invite-only system or a sign-up system. You could limit the testers to 50 or something. Then you would have a wider range of opinions and no random newbies providing useless comments.

YoZiZzO
March 1, 2006, 8:31 pm
 Quote:we've had many discussions about this and I don't remember the full list of reasons, but the ones I always advocated were:
1) It would be very difficult to manage server versions, client versions, and keep everything running smoothly in the public beta test. The extra time and effort required to organize this outweigh all the benefits.
2) The current beta testing schema accomplishes the task fine
3) There would be a lot of false bug reports to sort through, or repeated reports.
4) --Many other things brought up by other beta testers--
1. I wasn't implying the beta should be spoonfed to the public; instead, just host the download for the new beta and possibly a download for the new dedicated server. Those who wish to participate can. There don't have to be any official beta servers or anything.

2. I disagree. In 1.3.1, I checked all of the apparent fixes, and found that I could get onto the roof of b2b despite the "fix" about 10 seconds after I installed the version. It's just an example of one embarrassing bug that could've been removed for release.

3. People post plenty of false/repeated bug reports on these forums consistently. Plus, if the system were implemented, for every person who made a false report there would be another tester to verify it.

 Quote:It should stay this way in my opinion. The beta testers are the ones who know their , and even though everyone likes to blame them for when they dont like a version, they continue on and soldat's still aroond.
Who else is to blame for a version? The people who didn't test it? People seem to have this idea that the current beta testers are better at testing than others merely because they have been testing for a while. Testing isn't a practiced skill; I could name plenty of unbiased people who would make good testers (most of the fracturedsoldat members).

 Quote:It'd be bad to have a bunch of nabs like us beta testing a game. NOT A GOOD IDEA. I like the beta testers better, even if they did make m79 nabby.
That is one of the worst justifications ever. In fact, the testers are often accused of being "nabs" who are testers merely because they are vets. Skill has nothing to do with it; an objective viewpoint should be the ideal requirement for being a tester.

 Quote: A public beta test would look like a normal release. It's nearly impossible to get some constructive criticism from so many people. Closed betas have their disadvantages too but they fit Soldat at this stage.
Again, I'd like to make the parallel with forums. What's the point of having a public forum with sections such as suggestions and testing if it is impossible to get any constructive criticism from them? Beta criticism should be compared to posts in these forums: consider the well-written and objective ones, disregard subjective and poorly-founded ones.

 Quote:Nevertheless, a beta team that is chosen wisely is much better than just letting the mass public be a factor.
Espacialy when its the SOLDAT community
You essentially stated what Jelly and peemonkey said, but with zero evidence to support it.

Pero
March 1, 2006, 10:17 pm
 Quote:Originally posted by ZamorakHow many games publically test?

The beta testers were chosen for a reason, we should trust their descisions..

dropdown2 is better than Voland...

I actually sorta like ctf_crashed


ctf_run ftw.

But we will see more nice maps now that the mappers are armed with polyworks.

dropdown2 is better than Voland...
Riiight

Deleted User
March 1, 2006, 10:52 pm
They are similar, but it is too easy to use m79 on voland, and I hate the spawn points.

Pulp
March 2, 2006, 12:00 am
Well, YoZiZzO got a point, there isn't any openness in the recruiting process of how to become a betatester. It's rather logic, keeping in mind the quite few mistakes made by the betatesters and MM in the past, you start to ask yourself questions about the decency of the betatesters. It's a closed community, recruiting is based on nepotism. They have the best intentions with dolldat though, but somehow always manage to make absurd feeble-minded decisions, which just makes every version more tragically humorous...

A loudly hurray for MM though, to give a bunch of scurfy bull-buggering gnomes ( or 'vertically challenged persons', as you wish ) an almost harmless occupation...





Deleted User
March 2, 2006, 4:39 am
Beta Testing works on Representation. Beta Testers Represent the good players in the community, therefore, the current Beta Testing system is pretty damn good.

Ok
March 2, 2006, 12:24 pm
 Quote:Originally posted by JellyBeta Testing works on Representation. Beta Testers Represent the good players in the community, therefore, the current Beta Testing system is pretty damn good.


Here comes the part that is debatable.
No matter who is in there, there will always be those who doubt the beta team.
or any other group that makes decisions for the majority, even goverements that actualy are chosen by the community, are being doubted all the time.

Since there isn't any system that allows us to choose them, we can only do our best to make them see things better, if they miss something make them see it, and if we think they are wrong, use the freedom of speech granted to us to speak out and if that doesnt work just punch Chakra in the face, that always sorts him our right