[Search for users] [Overall Top Noters] [List of all Conferences] [Download this site]

Conference 7.286::visualc

Title:Microsoft Visual C/C++
Moderator:PLUGH::needle
Created:Tue Mar 16 1993
Last Modified:Wed Jun 04 1997
Last Successful Update:Fri Jun 06 1997
Number of topics:1121
Total number of notes:4385

1096.0. "NT v4.0 <-> threads <-> mem usage ?" by EDSCLU::JAYAKUMAR () Sat Mar 22 1997 14:35

[Cross posted in ::NT_DEVELOPERS. Hopefully someone here would have solved
this easily]

Folks,

I need some help in tuning my application which creates a large number of
threads (~4000), and as a result uses a large amount of memory (~150 MB). I 
guess most of this memory is used up for stack space for each thread. I am 
trying to see if I can reduce the default value of the stack size for a thread,
and therby reduce the memory utilization of my application.

1. Whats the default stack size for a thread in NT?  The doc. is not very clear
   (see for the extract of the doc at the end).

         A sample test program which creates 500 threads (with no local/global 
         variables) had 12248K as the VM size. So is the default stack size 
         for a thread ~ 24K(in Alpha)?

2. My next, more important question is how do I reduce this stack size, so that 
   I can reduce the memory size (VM) of my application which creates a large
   number of threads? 

   These following options:

             - linker /STACK,
             - dwStack parameter in CreateThread(),
             - using Editbin

   supposidely exists to change the default stack size. But unfortunately 
   they only help me in increasing it but not reducing it.

3. There is a function shrnkstk() provided in the MSDN CD, which is supposed
   to shrink the stack to the bare minimum. Using this function didn't make
   any difference.

Thanks
-Jay
============================================================================

		Under linker options:

>>>>
Stack Allocations

This option sets the size of the stack in bytes. Command-line equivalent: 
/STACK:reserve[,commit]

The Reserve text box (or the reserve argument on the command line) specifies 
the total stack allocation in virtual memory. The default stack size is 1 MB. 
>>>>


		Under CreateThread:

>>>>
dwStackSize

Specifies the size, in bytes, of the stack for the new thread. If 0 is
specified, the stack size defaults to the same size as that of the primary
thread of the process. The stack is allocated automatically in the memory space
of the process and it is freed when the thread terminates. Note that the stack
size grows, if necessary.
>>>>>                               


Thanks
-Jay


T.RTitleUserPersonal
Name
DateLines
1096.1BHAJEE::JAERVINENOra, the Old Rural AmateurMon Mar 24 1997 13:0421
    I'm not too familiar with the intricacies of NT stacks (and memory
    allocation in general)... maybe someone else can shed some more light
    on this.
    
    There was some discussion earlier about this, either here or in the
    developers' conference.
    
    Yes, I believe the default stack _reserve_ is 1 MB, i.e. 1 MB virtual
    memory is reserved (not necessarily committed). I'm not sure how much
    the default commit is. As you've noted, you can set both using various
    techniques.
    
    I don't quite understand why you say the /STACK option and EDITBIN only
    help to increase the stack size - you can decrease it also. Try
    something like /STACK=1024,1024 and the memory usage should be much
    lower than with the default.
    
    Also note that the space reserved will be at least one page, i.e. 8k on
    Alpha, 4k on Intel. Looks like in your example, each thread creates 3
    pages. Don't know what the other two other are for, though...
    
1096.2EDSCLU::JAYAKUMARMon Mar 24 1997 13:5792
Thanks for your response,

>>    I don't quite understand why you say the /STACK option and EDITBIN only
>>    help to increase the stack size - you can decrease it also. Try
>>    something like /STACK=1024,1024 and the memory usage should be much
>>    lower than with the default.

There was no difference from /STACK=0,0 to /STACK=8192,8192. The VM size 
increases 24K per thread. When I specify /STACK=8193,8193 each thread takes up
32K (ie. 1 more page)
 
Given below is a simple test program.

-Jayakumar
---------------------------------------------------------------------------
#include <windows.h>
#include "stdio.h"

void main(int argc, char **argv)
{
   HANDLE        nThr;
   DWORD         nId;
   int           i;

   for (i=0; i<500; i++)
   {
      nThr = CreateThread(
        NULL,                 
        0,                   
        (void *)threadFunc,            
        (void *)(i+1),                 
        0,                                  
        &nId );       
   }

   Sleep (100000);
}

void threadFunc (int i)
{
   ShrinkStack ();/* presense or absence of this call makes no difference */

   Sleep (100000);
}

/**
 ** function to shrink the stack to a bare minimum (from MSDN CD)
 **/
void ShrinkStack (void) 
{
    MEMORY_BASIC_INFORMATION mbi;
    PBYTE pbStackCrnt, pbStackBase, pb1stCommitPage, pbGuardPage;
    SYSTEM_INFO si;
    DWORD dwPageSize, dwProtectOld;

    // Get the size of a page for this machine.
    GetSystemInfo(&si);
    dwPageSize = si.dwPageSize;

    // Use _alloca() to get the current stack pointer
    pbStackCrnt = (PBYTE) _alloca(1);


    // Find the base of the stack's reserved region.
    VirtualQuery(pbStackCrnt, &mbi, sizeof(mbi));
    pbStackBase = (PBYTE) mbi.AllocationBase;


    // Find the address of the first page of
    // committed storage for the stack.
    pb1stCommitPage = (PBYTE) mbi.BaseAddress;

    // Find the address for the page beneath the
    // current stack page.  This page should have the 
    // PAGE_GUARD protection attribute flag.
    pbGuardPage = pb1stCommitPage - dwPageSize;

    // If the guard page overlaps the bottom-most page
    // of the stack's region, we cannot shrink the stack.
    if (pbStackBase == pbGuardPage)
        return;

    // Free the committed pages from the base
    // of the stack region to the guard page.
    VirtualFree(pbStackBase, 
        pbGuardPage - pbStackBase, MEM_DECOMMIT);

    // Set the PAGE_GUARD flag on for the page
    // beneath the current bottom of stack.
    VirtualProtect(pbGuardPage, dwPageSize, 
        PAGE_READWRITE | PAGE_GUARD, &dwProtectOld);
}
1096.3end of the road!EDSCLU::JAYAKUMARMon Mar 24 1997 19:1118
Looks like the stack size can't be lowered than 24K on Alpha and 12K on Intel
-----------------------------------------------------------------------------

microsoft.public.vc.language news group
From: felixk@mvps.org (Felix Kasza [MVP])

Jay

You won't ever get below this limit. An NT stack, in its smallest
form, has a data page plus two guard pages; with 4K pages (Intel) or
8K pages (Alpha) this gives the results you noticed.

2000+ threads are not the way to handle large client counts anyway;
have you looked into thread pooling and I/O completion ports?

Cheers,
Felix.
1096.4BHAJEE::JAERVINENOra, the Old Rural AmateurTue Mar 25 1997 06:039