Combat Mission on a high-end machine

Redwolf

Member # 3665
Joined
Sep 2, 2002
Messages
5,113
Reaction score
43
Location
MA, USA
Country
llUnited States
I am using VMMAP, which is made by the same guys that made Process Explorer, I believe. Which counter are you talking about? The 'Commit Charge' or the 'Virtual Size'?
I'm not familiar with that program. You need whatever gives you the current virtual memory mappings.

If you want to see what the heck is going on you'll need to hunt down a complete map such as Unix like OSes provide in /proc/<pid>/maps or ideally an equivalent to Linux' smaps file which also tells you specifically which of the virtual mappings are backed by physical pages.
 

Fleischer

Member
Joined
Jun 20, 2011
Messages
156
Reaction score
0
Location
Oslo
Country
llNorway
I'm not familiar with that program. You need whatever gives you the current virtual memory mappings.
Both these programs use the same counters and definitions, as they were made by the same people(Sysinternals). Which counter is it that you see multiply? You need to be more specific, because the definition of these terms vary from program to program.

If you want to see what the heck is going on you'll need to hunt down a complete map such as Unix like OSes provide in /proc/<pid>/maps or ideally an equivalent to Linux' smaps file which also tells you specifically which of the virtual mappings are backed by physical pages.
VMMAP can do that, but it's still not clear to me which type of memory you see multiplying. Windows distinguishes between reserved and committed memory, and if the reserved memory keeps growing without the commit charge following, then it's a memory leak, because the application is reserving memory that is never used. If, on the other hand, the commit charge follows the amount of reserved memory with the working set being stable, then something else in the system must be memory intensive, causing the balance manager to trim the working set of CM. In any case, it must be a memory leak when the application runs out of address space without even utilizing a quarter of available RAM! That is just common sense.
 

Redwolf

Member # 3665
Joined
Sep 2, 2002
Messages
5,113
Reaction score
43
Location
MA, USA
Country
llUnited States
In any case, it must be a memory leak when the application runs out of address space without even utilizing a quarter of available RAM! That is just common sense.
No, not in the narrow sense of defining a memory leak. Just because you use a lot of memory, even if you are wasteful, doesn't mean you lost track of allocated memory.

I still think your impression of the relationship between virtual mappings and physical pages actually in use is very fuzzy. It is perfectly normal for the sum of virtual mappings to be many times physical memory used. And it is perfectly harmless, except if you try to live in a 32 bit address space in 2013 (duh).
 

Fleischer

Member
Joined
Jun 20, 2011
Messages
156
Reaction score
0
Location
Oslo
Country
llNorway
No, not in the narrow sense of defining a memory leak. Just because you use a lot of memory, even if you are wasteful, doesn't mean you lost track of allocated memory.

I still think your impression of the relationship between virtual mappings and physical pages actually in use is very fuzzy. It is perfectly normal for the sum of virtual mappings to be many times physical memory used. And it is perfectly harmless, except if you try to live in a 32 bit address space in 2013 (duh).
Well, you refuse to tell me which counter you are referring to, which means it is quite impossible to know what you are talking about, or indeed that you *know* what you are talking about :)
 

Redwolf

Member # 3665
Joined
Sep 2, 2002
Messages
5,113
Reaction score
43
Location
MA, USA
Country
llUnited States
Well, you refuse to tell me which counter you are referring to, which means it is quite impossible to know what you are talking about, or indeed that you *know* what you are talking about :)
It's not my fault that Microsoft and related companies make a mess out of designations for standard kinds of memory. The virtual address space is a hardware feature from the MMU so it isn't different OS concepts that apply here. It's just people trying to be cute and making up their own names for an audience who they assume doesn't know the hardware.

I think I posted in that long BFC that that Phil didn't like where exactly it is in process explorer.
 

NUTTERNAME

Member
Joined
Jan 3, 2010
Messages
1,943
Reaction score
37
Location
N
Country
llVietnam
So I built a new rig with an Intel Core i5 3570k CPU overclocked to 4.4 GHz and a Gigabyte GTX 670 graphics card. Wanting to get *something* out of my old, beloved CM, I thought maybe it would be nice to just see STUGs and Shermans roll around smoothly at 60 FPS for a change, so I fired it up:

14 fps min, around 20 fps average at ground level. 20 fps min and about 28 fps average higher up. I knew CM was poorly optimized, but this was horrible and totally unplayable for anyone who cares about performance. For the sake of it, I ran a benchmark with a GTX 660 Ti card also, and the results were the same. No antialiasing turned on, btw., which makes it even more bizarre.

I get about the same with my 'junk-yard-dog' computer. Q6600 o/c to 3.1 GHz...4GB DDR3@1333...Sata2HD and low buck Zotac GT520 vid card.

But, would 60 FPS look that much better?
 

Fleischer

Member
Joined
Jun 20, 2011
Messages
156
Reaction score
0
Location
Oslo
Country
llNorway
It's not my fault that Microsoft and related companies make a mess out of designations for standard kinds of memory. The virtual address space is a hardware feature from the MMU so it isn't different OS concepts that apply here. It's just people trying to be cute and making up their own names for an audience who they assume doesn't know the hardware.

I think I posted in that long BFC that that Phil didn't like where exactly it is in process explorer.
I think it would be wrong to blame Microsoft for not having the "right" definition for something that it is not at all clear how one should define. Memory management is a lot more complicated than physical versus virtual memory. And the people who made PE are also Microsoft, might I remind you.
 

Redwolf

Member # 3665
Joined
Sep 2, 2002
Messages
5,113
Reaction score
43
Location
MA, USA
Country
llUnited States
I think it would be wrong to blame Microsoft for not having the "right" definition for something that it is not at all clear how one should define. Memory management is a lot more complicated than physical versus virtual memory. And the people who made PE are also Microsoft, might I remind you.
Sure it is. Everybody uses the same terms except the microsofts. I mean they even call swapspace "virtual memory" in some consumer-facing programs for crying out loud. (luckily the developer docs don't repeat that mistake)

In any case the concept of what is virtual address space should go by without messing up terms, and does in all real operating systems, because it is directly coupled to the hardware (the MMU), so there can be no implementation detail difference behind the terminology mess.
 

Fleischer

Member
Joined
Jun 20, 2011
Messages
156
Reaction score
0
Location
Oslo
Country
llNorway
I get about the same with my 'junk-yard-dog' computer. Q6600 o/c to 3.1 GHz...4GB DDR3@1333...Sata2HD and low buck Zotac GT520 vid card.
Yes, my impression is that it's the "newer" cards that are problematic. I remember having perfomance issues the first time around 2008; after a graphics card upgrade, I would get 10 fps with the 'Al Huqf' scenario. Before that, things were really smooth. The game is still living in 2007, I think.

But, would 60 FPS look that much better?
Once you've gotten used to 60 fps, your eyes will bleed when going below.
 

Fleischer

Member
Joined
Jun 20, 2011
Messages
156
Reaction score
0
Location
Oslo
Country
llNorway
Sure it is. Everybody uses the same terms except the microsofts. I mean they even call swapspace "virtual memory" in some consumer-facing programs for crying out loud. (luckily the developer docs don't repeat that mistake)
Which program is this?

In any case the concept of what is virtual address space should go by without messing up terms, and does in all real operating systems, because it is directly coupled to the hardware (the MMU), so there can be no implementation detail difference behind the terminology mess.
Except that there is. There is shared memory, private memory, locked memory, guarded memory, reserved memory, committed memory, paged memory, non-paged memory, kernel memory, user memory ... the list goes on. Sorry, but it's clear that you don't know much what you are talking about when you oversimplify things into virtual vs. physical memory. It's not at all clear what should be included in a general 'virtual memory' performance counter, and it's not at all clear what you are talking about when you're trying to make a point about CM running out of "virtual memory" and not "physical memory".
 

NUTTERNAME

Member
Joined
Jan 3, 2010
Messages
1,943
Reaction score
37
Location
N
Country
llVietnam
Yes, my impression is that it's the "newer" cards that are problematic. I remember having perfomance issues the first time around 2008; after a graphics card upgrade, I would get 10 fps with the 'Al Huqf' scenario. Before that, things were really smooth. The game is still living in 2007, I think.



Once you've gotten used to 60 fps, your eyes will bleed when going below.
I suppose I should have mentioned it's Win8 32bit.

But, I find that I don't need to 'movie-theatre' every aspect of the game. It's like speed-reading where I just need to digest what is going on and then act accordingly. I might follow some 'golden-boys' and their adventures, but it isn't that much a visually driven experience. At least for me. I seem to focus on the reality aspect more than the buttons on uniforms and neato-explosions.
 

Redwolf

Member # 3665
Joined
Sep 2, 2002
Messages
5,113
Reaction score
43
Location
MA, USA
Country
llUnited States
Which program is this?
Uh? The default task manager uses the term "virtual memory" for swapspace. That's complete BS and obviously has been decided by somebody at Microsoft who doesn't understand virtual memory and wanted to make up a cute term for the average user. Luckily most of the programmer-facing microsoft documentation isn't affected by this.

Except that there is. There is shared memory, private memory, locked memory, guarded memory, reserved memory, committed memory, paged memory, non-paged memory, kernel memory, user memory ... the list goes on. Sorry, but it's clear that you don't know much what you are talking about when you oversimplify things into virtual vs. physical memory. It's not at all clear what should be included in a general 'virtual memory' performance counter, and it's not at all clear what you are talking about when you're trying to make a point about CM running out of "virtual memory" and not "physical memory".
So I just fired up process explorer and the value you need is simply called "Virtual size". I really don't know where your confusion comes from. We are at the point now where I have to question whether we have a serious discussion here.

When CMx2 runs out of memory it is that virtual size. There is nothing more complicated about it.

And it has very little to do with physical memory so you can just toss the rest of those fields if you want to find out why CMx2 crashes.
 
Last edited:

Fleischer

Member
Joined
Jun 20, 2011
Messages
156
Reaction score
0
Location
Oslo
Country
llNorway
Uh? The default task manager uses the term "virtual memory" for swapspace. That's complete BS and obviously has been decided by somebody at Microsoft who doesn't understand virtual memory and wanted to make up a cute term for the average user. Luckily most of the programmer-facing microsoft documentation isn't affected by this.
Sorry, but this is just foolish anti-Microsoft attitude. No, the task mananger shows the private part of the commit charge afaics, which is something entirely different than "swapspace". This is a quite natural number, and it is not given that they should have picked the number you want.

So I just fired up process explorer and the value you need is simply called "Virtual size". I really don't know where your confusion comes from. We are at the point now where I have to question whether we have a serious discussion here.

When CMx2 runs out of memory it is that virtual size. There is nothing more complicated about it.

And it has very little to do with physical memory so you can just toss the rest of those fields if you want to find out why CMx2 crashes.
Redwolf, this discussion started with you making a point about CM not running out of physical but Virtual Memory. You then pointed me to Process Explorer because it allegedly showed the "right" number. Then you wouldn't tell me which number it was. I suspect this is because you have NO idea what the 'Virtual Size' counter really indicates. :)

So let me tell you: The 'Virtual Size' counter is an indicator of virtual memory usage, but unlike the task manager 'Virtual memory', it includes reserved and shared memory also. That means you also have memory that is not mapped(used) and memory that is shared between other processes. The concept of 'reserved memory' is something Windows uses to allow you to reserve large portions of contiguous memory for future use, usually for performance reasons, without it being mapped to a backing store("swapspace"). Very fast and efficient. And like I explained earlier: If the process keeps increasing its 'Virtual size' counter out of proportion to the working set(physical memory), that means you have a lot of reserved memory allocated through the VirtualAlloc system call that never gets committed. A classic indication of a memory leak, indeed. Thus the amount of physical memory(working set) used is VERY important for what we are talking about, and so is the commit charge. This is why I asked you about the counters and why your statements are too general.
 

Redwolf

Member # 3665
Joined
Sep 2, 2002
Messages
5,113
Reaction score
43
Location
MA, USA
Country
llUnited States
Sorry, but this is just foolish anti-Microsoft attitude. No, the task mananger shows the private part of the commit charge afaics, which is something entirely different than "swapspace". This is a quite natural number, and it is not given that they should have picked the number you want.



Redwolf, this discussion started with you making a point about CM not running out of physical but Virtual Memory. You then pointed me to Process Explorer because it allegedly showed the "right" number. Then you wouldn't tell me which number it was. I suspect this is because you have NO idea what the 'Virtual Size' counter really indicates. :)

So let me tell you: The 'Virtual Size' counter is an indicator of virtual memory usage, but unlike the task manager 'Virtual memory', it includes reserved and shared memory also. That means you also have memory that is not mapped(used) and memory that is shared between other processes. The concept of 'reserved memory' is something Windows uses to allow you to reserve large portions of contiguous memory for future use, usually for performance reasons, without it being mapped to a backing store("swapspace"). Very fast and efficient. And like I explained earlier: If the process keeps increasing its 'Virtual size' counter out of proportion to the working set(physical memory), that means you have a lot of reserved memory allocated through the VirtualAlloc system call that never gets committed. A classic indication of a memory leak, indeed. Thus the amount of physical memory(working set) used is VERY important for what we are talking about, and so is the commit charge. This is why I asked you about the counters and why your statements are too general.
Very sorry but all that matters for the crash is the sum of memory mapping, which process explorer gives as "virtual size". It matters not whether any of it is shared and it matters not how much of it is currently backed by physical pages.

Your description of symptoms of a memory leak are just external symptoms that you can observe without source code, but you have no way to know whether the program actually loses track of any memory or not.

Defending MS over using the term "virtual memory" for swapspace in some toy utilities is just foolish. Even process explorer, download also from MS, gets it right.
 

Fleischer

Member
Joined
Jun 20, 2011
Messages
156
Reaction score
0
Location
Oslo
Country
llNorway
Very sorry but all that matters for the crash is the sum of memory mapping, which process explorer gives as "virtual size". It matters not whether any of it is shared and it matters not how much of it is currently backed by physical pages.

Your description of symptoms of a memory leak are just external symptoms that you can observe without source code, but you have no way to know whether the program actually loses track of any memory or not.

Defending MS over using the term "virtual memory" for swapspace in some toy utilities is just foolish. Even process explorer, download also from MS, gets it right.
Oh, boy - why do you keep insisting on your own flawed conceptions of this? The 'Virtual Size' is not mapped memory - that is precisely what reserved memory is not. It's not mapped to any backing store, RAM or swapspace, at all. If you want the 'the sum of memory mapping' you would want to look at the commit charge or 'Private Bytes' counter.
 

Redwolf

Member # 3665
Joined
Sep 2, 2002
Messages
5,113
Reaction score
43
Location
MA, USA
Country
llUnited States
I think the crazy MS terminology brainwashed you. What you run out of in the CMx2 crashes is those memory mapping regardless of whether they are backed by physical pages or not.

"To memory map" only means to establish a region of virtual memory that might or might not be part of some action later. But what you run out of is the maximum space for such regions before they do anything and before they are backed by anything. That is why you get so confused about the only 700 MB RAM being taken and then going KABOOM.

And this isn't any different between windows and real operating systems, because the basic working mechanism is dictated by the hardware, the MMU.
 

Fleischer

Member
Joined
Jun 20, 2011
Messages
156
Reaction score
0
Location
Oslo
Country
llNorway
I think the crazy MS terminology brainwashed you.
HAHAHA. *Laughing hard*! I think you've made it clear now that you DO in fact believe in Satan(Microsoft).

"To memory map" only means to establish a region of virtual memory that might or might not be part of some action later. But what you run out of is the maximum space for such regions before they do anything and before they are backed by anything. That is why you get so confused about the only 700 MB RAM being taken and then going KABOOM.
Hahaha, no. The only thing that really made me confused was you giving the impression you were tech savvy. The 700MB and then going KABOOM is what you claim - I haven't observed that at all. And given that you don't know the difference between reserved and committed memory, and indeed that you believe memory management implementation is the same everywhere because "the MMU is the same", means I'm certainly not going to take your word for it.
 

Redwolf

Member # 3665
Joined
Sep 2, 2002
Messages
5,113
Reaction score
43
Location
MA, USA
Country
llUnited States
To be a bit more constructive, I suggest you do this. Assuming you have a reproducible test case of CMx2 crashing.

Using process explorer or whatever other tool displays that value and observe or log virtual size. Watch where it goes before CMx2 crashes. It should go right up to your limit of virtual address space right before the crash (which normally is 4 GB for 32 bit processes on 64 bit OSes, 2 GB default for 32on32 and you can switch it up to 3 GB for 32on32). It really is that simple, you do not have to mess with any of the other values and you don't have to take into account how a given virtual page is backed, if it is backed at all.

If you would post such a log we can debate whether it is to be classified as a leak much better.

I apologize for not following microsoft terminology and not going my own measurement, but I really can't spend my time learning things for OSes that are (a) needlessly different from everybody else (b) make up their own terms instead of using already established terms, or worse use existing term for something else, and that (c) are not used in professional computing. My windows computer is purely for gaming and I haven't had CMx2 crash with a 64 bit kernel.
 

Fleischer

Member
Joined
Jun 20, 2011
Messages
156
Reaction score
0
Location
Oslo
Country
llNorway
So how would "such a log" (log of what?) help you determine whether it is a memory leak or not? What would you look for?
 

Redwolf

Member # 3665
Joined
Sep 2, 2002
Messages
5,113
Reaction score
43
Location
MA, USA
Country
llUnited States
So how would "such a log" (log of what?) help you determine whether it is a memory leak or not? What would you look for?
You can't. But we would have a much better discussion about what's going on than what we had in the last days :)

It might look interesting, such as sudden jumps which might or might not correlate with events in the game. Or it might look boring.

"log" = graph of virtual size over time until crash.
 
Top