Afghanistan First

Elvis

Member
Joined
Mar 6, 2009
Messages
2,918
Reaction score
14
Location
Pennsylvania
Country
llUnited States
SO Elvis, how long have you been testing Afghanistan? Or is the date NDA applicable? Have you been testing NATO and if so before or after Afghanistan? And finally is CMN at the Campaign testing stage yet?

My best guess for CMN dates is from CMSF module dates. If CMN hasn't started the Campaign testing stage yet then I dont think we will be seeing it soon. Campaigns seem to take about 3-5 months to work out. Unless CMN doesn't follow the pattern.
NDA is probably applicable for the CMN campaign testing question but since Steve already mentioned it (in the quote I posted on this very page) I see no problem with confirming what he says to be true. Campaign testing has not begun with CMN because TO&E still has some finishing up. Until that happens there can be no firm battles built. It has been said many times and many ways and I'm not sure if people believe it or not but....TO&E seems to be the single largest time consumer in the creation of new games and modules. I have no idea how it is done because "employees" are the ones that do it. I just know that the recurring statements regarding hold ups almost always revolve around that. Further, Steve has said publicly and privately that because it was so cumbersome for CMSF it was getting an overhaul for CMN to help streamline the process for furture CM games and modules. Time will tell if the new process is succesful. Also, there is a reason to believe that the campaign testing will take less time with Normandy but I should probably not comment on that until after release.

The other questions are probably less informative than they may seem on the surface. I guess I got Afghan about 2 weeks before I got CMN and that was toward the tailend of 2009. The reason I say it may not be informative is because I was a little surprised (and hurt.......just kidding) when the forums for both were opened to me and some of the guys had already had both titles for a while. Not real idea how long before me but certainly long enough for me to feel like I was playing catchup. It seems that most CMSF testers were left out until it was more ready for hard testing. As I type this I suppose I could go back into each forum and see when the first posts were timestamped but even that wouldn't provide you much info (and may be closer to me walking the edge of NDAs).

NATO I got a decent amount of time after the other 2.

I hope I have answered you clearly and directly.

One further thing on testing an dthe number of testers. Please forgive me if I have said this before. There are many different personality types testing just as there are many different types of players. For instance, I always refer to myself as a "taker" because I almost never touch the editor and have never made a real scenario with any CM title....ever. But some guys enjoy making scenarios more than anything else. I believ Andreas when testing CMx1 got far more enjoyment out of map making and scenario design than he did as a player. And then there are guys who love to make mods and work on textures. And then there are the type that I imagine thewood and Adam to be (either one please forgive me if I am wrong), these guys really enjoy testing what the game system does. What makes it tick. And I believe they hardly play a battle just for the fun of fighting a battle. Nothing wrong with any of these types but each will experience the game in a different way.
 

Geordie

CM Moderator
Joined
Jan 27, 2005
Messages
2,111
Reaction score
13
Location
Scotland
Country
llUnited Kingdom
Agree with KG Jag there, thanks for the insight. It seems that CMN is still a fair way off then but CM-A may, just may be round some corner. The NATO module then seems to be behind the others. We seem to be plodding along though, which can only be positive but I imagine a little strained for the company. So it might be CMA for the Summer (late), CM-N for the Winter and CMAF NATO sometime in the future.
 

Michael Dorosh

der Spieß des Forums
Joined
Feb 6, 2004
Messages
15,733
Reaction score
2,765
Location
Calgary, AB
First name
Michael
Country
llCanada
One further thing on testing an dthe number of testers. Please forgive me if I have said this before. There are many different personality types testing just as there are many different types of players. For instance, I always refer to myself as a "taker" because I almost never touch the editor and have never made a real scenario with any CM title....ever. But some guys enjoy making scenarios more than anything else. I believ Andreas when testing CMx1 got far more enjoyment out of map making and scenario design than he did as a player. And then there are guys who love to make mods and work on textures. And then there are the type that I imagine thewood and Adam to be (either one please forgive me if I am wrong), these guys really enjoy testing what the game system does. What makes it tick. And I believe they hardly play a battle just for the fun of fighting a battle. Nothing wrong with any of these types but each will experience the game in a different way.
Tell us what the difference between "testing" and "playing a bunch of times and hoping to randomly find bugs" is. I'd be dead curious to know how your beta group differentiates between the two.
 

Elvis

Member
Joined
Mar 6, 2009
Messages
2,918
Reaction score
14
Location
Pennsylvania
Country
llUnited States
Tell us what the difference between "testing" and "playing a bunch of times and hoping to randomly find bugs" is. I'd be dead curious to know how your beta group differentiates between the two.
Easy. One is playing and testing scenarios and if you see something odd you report it and it gets looked into. The other is "we have altered the way XXX works. Can some of you check it out and see what you think" or "I think I saw a problem with YYY, can some of you set up scenarios to make sure it is working as expected"
 

thewood

Member
Joined
Feb 19, 2004
Messages
2,594
Reaction score
12
Location
Boston
Country
llUnited States
What a real tester does in a real software company is set up specific scenarios designed to highlight either specific changes or features and then run it multiple times and with different variables changed. Each run is documented and multiple testers' results are compiled and analyzed for trends, exceptions, errors, etc.

I have a sneaking feeling that you don't see much of that at bfc.
 

Elvis

Member
Joined
Mar 6, 2009
Messages
2,918
Reaction score
14
Location
Pennsylvania
Country
llUnited States
What a real tester does in a real software company is set up specific scenarios designed to highlight either specific changes or features and then run it multiple times and with different variables changed. Each run is documented and multiple testers' results are compiled and analyzed for trends, exceptions, errors, etc.

I have a sneaking feeling that you don't see much of that at bfc.
That kind of stuff is what falls into this category:

The other is "we have altered the way XXX works. Can some of you check it out and see what you think" or "I think I saw a problem with YYY, can some of you set up scenarios to make sure it is working as expected"
EDIT AT ADD:

That type of testing is more to determine if something works as intended or desired. True bugs are a different animal.
 
Last edited:

vulture

Member
Joined
Jan 20, 2010
Messages
192
Reaction score
4
Location
Ossett
Country
ll
There are places for both kinds of testing. Yeah, you have to systematically test things, which is laborious and amazingly dull work. But it is also true that you are never going to cover every eventuality no matter how thorough you are. There is a lot to be said for playing a game that puts you under pressure, which forces you to do things in a different way to the way you normally do things. Or to put it another way, playing the game throws scenarios at you that force you to explore a different part of the space of possible actions.

The aim of testing is to explore that space as thoroughly as possible and find the places where things don't work. Systemic testing and creating code test units can do a lot of the work, but there is still a need to have a large number of testers simply playing the game in their own deranged ways to cover weird corners of game space. That's not an excuse for skipping the systemic testing of course. (and you can 100% guarantee that every game will ship with at least one bug that you can't believe no-one noticed).
 

dalem

Member
Joined
Jan 4, 2009
Messages
2,298
Reaction score
62
Location
Columbia Heights, MN
Country
llUnited States
There is a place for bug-free releases too. They did it with their first few, some people expect them to do it with the rest.

-dale
 

Elvis

Member
Joined
Mar 6, 2009
Messages
2,918
Reaction score
14
Location
Pennsylvania
Country
llUnited States
There is a place for bug-free releases too. They did it with their first few, some people expect them to do it with the rest.

-dale
Not only have they never released a bug free title but they have never left one "finished" in a bug free condition.
 

dalem

Member
Joined
Jan 4, 2009
Messages
2,298
Reaction score
62
Location
Columbia Heights, MN
Country
llUnited States
Not only have they never released a bug free title but they have never left one "finished" in a bug free condition.
Oh, the CMx1 stuff was as close to bug-free as you get. Nothing affecting ability to play, nothing largely wrong except tweaks. One of the reasons I stopped buying a game a month on spec was because I got so frustrated with the rest of the industry's offerings in that regard after my CMBO experience.

-dale
 

thewood

Member
Joined
Feb 19, 2004
Messages
2,594
Reaction score
12
Location
Boston
Country
llUnited States
My point isn't that any piece of SW ships bug free, but how well its tested before it gets to a beta test. Looking at some of the issues, I would guess there is little seperation between feature testing and beta testing. The term beta testing gets thrown around a lot, but I find game companies have almost no internal game testing left and have outsourced it all to what they call beta testers. In effect, they use beta testers as QA and alpha.

And when you have volunteers doing QA and alpha, you get crappy releases and an extra year of development spent fixing BIG bugs. But at least you get people to pay $60 to be beta testers. That is the real revolution in SW that I would like to find a way to replicate.
 
Last edited:

thewood

Member
Joined
Feb 19, 2004
Messages
2,594
Reaction score
12
Location
Boston
Country
llUnited States
I'm with dale on CMBO. CMBO spoiled for other game releases. It had some bugs, but nothing huge. Not only that, but BFC was pretty quick on getting fixes out. Unfortunately, I hold companies to that yardstick still.
 

Elvis

Member
Joined
Mar 6, 2009
Messages
2,918
Reaction score
14
Location
Pennsylvania
Country
llUnited States
Oh, the CMx1 stuff was as close to bug-free as you get. Nothing affecting ability to play, nothing largely wrong except tweaks. One of the reasons I stopped buying a game a month on spec was because I got so frustrated with the rest of the industry's offerings in that regard after my CMBO experience.

-dale
All almost true. And, as you point out,a release close to bug free was/is the exception not the norm. I think time has either clouded your memory of CMBOs release or you weren't around for it (I honestly don't recall). At the end of this post I am posting the fixes that were in the first of the 12 patches that CMBO had.

The trouble with speaking in absolutes is that few things are absolute.

* * * * * * * * * * * * * * * * * * * *
v1.01 6/14/2000
* * * * * * * * * * * * * * * * * * * *

* Play-by-email (PBEM) fixes:

- Both players MUST upgrade to v1.01. v1.01 cannot play PBEM with
the original v1.0. This is due to bug and security fixes which
changed the format of PBEM files.

- Barbed wire does not lose its ability to slow down infantry.

- Doing a normal save-game no longer causes the occasional skipping
of one player's orders phase.

- Units no longer occasionally target smoke toward the map corner.

- There is no longer the occasional problem of being unable to give
orders to a vehicle, seemingly for no reason.

- Reinforcements don't arrive differently upon subsequent reloads of
a PBEM file.

- Vehicles no longer sometimes switch from 'destroyed' to
'abandoned'.

- Captured vehicle crew are counted properly when one side
surrenders.


* TacAI fixes:

- Less fickle with targeting changes, i.e. targeting 'stickiness'
increased slightly.

- Buttoned-up tanks are less likely to target low-visibility targets
(like hiding infantry) at longer ranges, even when the target *is*
spotted by another friendly unit.

- Less likely to fire smoke at less-important targets, especially
ones that are already pinned down or panicked.

- Paths made by TacAI no longer occasionally have visually
'floating' waypoints.

- Judges 'cover' behind stone walls and bocage properly, not
prematurely forcing units to abandon these positions as if they're too
exposed.


* New explosion graphics (when using "high quality smoke" option)

* Camera can rotate on a 'point' rather than around a circle (SHIFT-J
option).

* Units can hide during setup.

* Handicapper unit choice is more logical. Can also go to +200%.

* Player can give the AI an "experience bonus" that raises all AI
units from 1 to 3 experience levels.

* Infantry "close combat" versus armored vehicles has been reworked:
- Demo charges and grenades thrown at armored vehicles now have a
visible blast wave.
- Tossed demo charges vs. armored vehicles are more accurate.
- Various bug fixes

* Some computer-player tweaks, including no longer moving gun-damaged
tanks into the map corner.

* Nonmoving infantry is slightly harder to spot (i.e. max range
decreased a bit).

* Pillboxes are slightly harder to spot (if in cover), and can 'hide'
(which helps save ammo). Also, they are now far more resistant to
artillery.

* In hotseat play, uncontacted enemy vehicles' engine sounds are not
audible.

* Flamethrowers stop firing once their AREA target is burning.

* Map elevation algorithm is improved, making roads run more smoothly
up/down hill faces, not stair-stepped.

* Gun crews won't abandon their (functioning) guns until they're fully
ROUTED (rather than BROKEN) provided they have less than 50%
casualties.

* Sneaking units, when fired upon in cover, are more likely to stop
and return fire rather than keep moving.

* In the action phase the rewind button would sometimes cause a
building-destruction animation to play at the incorrect time. This
has been fixed.

* Fanatic units are subject to panic (like normal units) when attacked
by flamethrower.

* Captured (or escaped) infantry is allowed to move at run speed,
regardless of initial weapon type. However, this can only happen
as a result of running for cover when under attack; no captured
unit may be given orders to "run".

* Smoke shells leave a small impact crater.

* When a unit kills an enemy armored vehicle, it now (usually) plays
the appropriate voice sound effect.

* "Detailed armor hit text" is not shown to the enemy when a vehicle
is hit by a mine or artillery outside of enemy observation.

* When repositioning a unit during setup, the colored line always
turns red to indicate that a position is not allowed.

* Sound contacts are no longer indicated (to the enemy) as knocked out
or destroyed even if they actually are. And if destroyed, they are
not shown on the map at all. And they are never shown as being
located off the map edge.

* Armored vehicles are slightly less likely to be immobilized or
receive gun damage from nearby blasts (e.g. artillery).

* You can give movement orders to bogged vehicles, which they'll
execute if they're able to un-bog.

* Split-squads' casualties are tallied correctly in the after-action
report.

* 'Target Next' order works for vehicles with machineguns but no main
gun.

* 0 vs. 0 final score is reported as a 'draw' not an Axis victory.

* Units 'notice' more quickly that an enemy unit has surrendered, and
don't shoot at them just seconds after surrendering.

* Bailed vehicle crews who are less than 'fully' spotted are drawn
with generic infantry uniforms.

* Infantry moves more slowly on slopes.

* Automatic force picker for Quick Battles picks fewer defensive
fortifications.

* Captured guns and mortars are tallied correctly in the after action
report.

* Interior graphic problem with the SPW 251/1 halftrack fixed.

* If a tank became immobilized while rotating its turret, sometimes
the turret would keep spinning. This has been fixed.

* If you move a vehicle carrying passengers during setup, its
passengers' command and control links are updated immediately and
correctly.

* No 'reinforcements have arrived' message is shown if all of them are
airplanes.

* Vehicles in 'hunt' mode are a little more likely to stop before
shooting.

* Sometimes the 'hide roofs' and 'transparent buildings' options would
interfere with one another. This has been fixed.

* When loading or saving a file in the editor, the dialog
'default' directory is the Saved Games directory.

* Slight problem of diagonal bocage and walls not quite matching 3D
placement on map is fixed.

* No prisoner will ever be shown as "reloading".

* Some open-top armored vehicles were suffering crew casualties from
nearby explosions, even when buttoned-up. This has been fixed.

* Unit editor allows U.S. Paratroops to carry up to 3 rifle grenades
(was 2).

* Bazookas, panzerschrecks, and PIATs display caliber and muzzle
velocity in data window.

* Sometimes reloading a 1-player game would have the wrong fog of war
setting (it wasn't being reloaded properly). This is fixed.

* Sometimes shells were passing through buildings when firing at the
top story. Fixed.
 

Elvis

Member
Joined
Mar 6, 2009
Messages
2,918
Reaction score
14
Location
Pennsylvania
Country
llUnited States
I'm with dale on CMBO. CMBO spoiled for other game releases. It had some bugs, but nothing huge. Not only that, but BFC was pretty quick on getting fixes out. Unfortunately, I hold companies to that yardstick still.
I remenber many annoying bugs for the first 6 months but people we MUCH more forgiving back then. Part of the difference, I believe, was from CMSFs initial release condition being much worse than CMBOs.

Which reminds me...I meant to add to my post above that in no way do I mean to imply that CMBO was released in the same condition as CMSF. Only that it was a far cry from bug-free. Not only that but if my memory os correct the first CMBO patch was available before some people had received their copy in the mail so a lot of people never really played the 1.00 version. I did.

And for kicks in my brief scan I just saw this in patch 1.12:

Quick Battles have a new "unrestricted" force type.
Ten years is a long time folks. I barely remember what I had for breakfast......................
 

thewood

Member
Joined
Feb 19, 2004
Messages
2,594
Reaction score
12
Location
Boston
Country
llUnited States
They weren't annoying to most people because most of it hadn't been done before. The main sticker for CMSF is you had an idea of how something was supposed to work because they had it working in CM1. Call it CM and people will expect to start from that point. Then completely screw up the release and you have doubled down on credibility loss.

btw, IIRC Elvis, you stated at one time that you agreed with Steve that CMSF's release was no worse than CMBO's. I can't remember if it was here or another forum. This is going back a couple years probebly. I remember it so well because it took a while after you said for me to five you any semblence of credibility back in my book.
 

dalem

Member
Joined
Jan 4, 2009
Messages
2,298
Reaction score
62
Location
Columbia Heights, MN
Country
llUnited States
All almost true. And, as you point out,a release close to bug free was/is the exception not the norm. I think time has either clouded your memory of CMBOs release or you weren't around for it (I honestly don't recall). At the end of this post I am posting the fixes that were in the first of the 12 patches that CMBO had.

The trouble with speaking in absolutes is that few things are absolute.
I have repeated a dozen times that I glommed onto CM when it was still being billed as Computer Squad Leader. I downloaded and played the earliest demos available a gajillion times. I don't know if I pre-ordered but it wouldn't surprise me if I did. I followed the forums and even participated a bit from their earliest incarnation.

I remember it all a helluva lot better than you do, apparently. And the fix list you posted contains nothing that made the game, or parts of it, unplayable. As was my original point.

-dale
 

Elvis

Member
Joined
Mar 6, 2009
Messages
2,918
Reaction score
14
Location
Pennsylvania
Country
llUnited States
btw, IIRC Elvis, you stated at one time that you agreed with Steve that CMSF's release was no worse than CMBO's. I can't remember if it was here or another forum. This is going back a couple years probebly. I remember it so well because it took a while after you said for me to five you any semblence of credibility back in my book.
No. If you dig back to those you'll find I said essentially what I am saying now. CMBO and CMBB were releaseds with problems. Not saying they were as bad as CMSFs problems but to remember CMBO and CMBB as released close to problem free is just not remembering correctly. Back then I spent a TON of time on the general forum reading almost all of every thread.


Before hitting "post" I searched myself on Dosomefink to see if that is what you meant and here are a couple of my very early posts there. I suspect this is what you are referring to because one was a direct repponse to a post my you.

Next, my point was because there was a beta demo many things that may not have been caught with the CMBO release were caught and fixed before release otherwise I suspect there would have been similar issues with the CMBO final product. And even in the final release there were very big issues with the game that were fixed in subsequent patches (a couple of them are mentioned above by Kingfish). Sound like simialr complaints to me...pathfinding...TACAI...To ignore that these problems exisited in CMBOs release is really rewriting history. It is remembering these things that I have done. I didn't feel like I was dealt a beta when I purchased CMBO and saw those things and waited for patches and future games to fix them. We all look back on those heady days with rose colored glasses. When I look back on those days the glass was half full and that is the way I feel now. At the risk of putting words into your mouth back with CMBO the glass was half full and CMSF the glass is half empty.

Another point I was making, that I probably didn't make clear, about the beta demo aren't the problems with it. It is that it gave people a chance to see the engine. Test drive the car and see if you like long before the games release so that when it was released people were already familiar with the UI and the way the game worked. Any problems people had with those things they had already come to terms with. Whether it was embracing the way it worked or walking away from the game never to be heard from again. And if you remember my previously mentioned war with the CC crowd you will will remember that there was a large number of people who left and never came back because it wasn't thier bag. And they bashed the game. That is the exact same case we are seeing with CMSF. It does not appear to be your bag. You didn't get that preview with CMSF. The first time you saw the interface was when the final release was in your hands. If you weren't around for the beta demo time of CMBO then that means you had a the regular demo to play before deciding to buy CMBO. Either way you had a chance to see if you liked it before your purcahse and you knew what you were getting.
We are talking about semantics here but you are putting words into my mouth I didn't say. That is where were are finding ourselves in disagreement. I never said "CMSF's launch was well done" however you did say " poorly implemented game". There is a very large area between "well done" and "poorly implemented". This is not, in my opinion, a black or white issue as in "if it isn't perfect on release than it is a disaster". I understand you didn't say that either but are tending to speak in extremes. Similar to the "never tested WeGo". "Never" is pretty definitive. I'm not picking nits here I am only trying to point out that there is a distinction and room for something other than these types of chisled in stone kind of statements.

CMBO if you recall had a very different release than every other CM title yet still holds up to my point. CMBB is the shitstorm everyone whos been around for a while remembers. CMBO was different for a couple of reason. The beta demo was released in October or November and the final game was the following June. When the beta demo was released there were loads and loads of people who hated it because it wasn't RT like the CC series or top down or thought that the 3D model took away from the strategy. People from other forums attacked the CM forum with bile and people from the CM forum attacked the CC forum etc...I am talking about the beta demo release now (although those wars went on for years). Steve has always said that he wishes they hadn't released the beta demo because he felt it distracted from them making the CMBO and unltimately resulted in a delay of it's release. I never agreed with that. I think that the beta demo created thousands of beta testers that shook that thing hard and were vocal about it (in a good, constructive way) and BFC (probably still BigTime Software still) had to address these issues in the final release. So although it delayed the release it created a cleaner final product. Another thing the beta demo did was create realistic expectations. When the game was fianlly released everyone who had been playing the beta demo knew how the game looked and worked. Its strenghts and weaknesses and ended up with a polished version of the game they had been playing for 8 or 9 months. No surprises with UI or MG reloading values or anything.

Why the history? To illustrate a few points. To remind (or inform) that there was a time when CMBO was ganged up on and things were heated. Nearly all of my real world friends (expect Peng) HATED the game and it's interface and did not buy the game and found it too confusing (they are all morons, of course). To point out why CMBO came out of the box better than all the other titles. Also, it might help to show the "through my eyes" thing I've been trying to say. When CMSF was released I knew what to expect. So nobody is rewriting history just some people don't remember parts of it or weren't there when it was going down.
 
Top