How to avoid memory troubles?

Discussion in 'General troubleshooting' started by paulhuizer, Oct 12, 2009.

  1. Hello everyone,
    After reading on this forum that each Application Pool is allocated 200MB on a IIS7 machine, I decided to try and figure out how to measure the memory usage of my app.

    Well .. it seems that I have the following values:

    Process proc = Process.GetCurrentProcess();
    proc.StartTime = 10/11/2009 7:55:09 AM
    proc.WorkingSet64 = 195579904
    proc.PeakWorkingSet64 = 211046400
    proc.VirtualMemorySize64 = 670838784
    proc.PeakVirtualMemorySize64 = 711757824
    proc.PagedMemorySize64 = 192761856
    proc.PeakPagedMemorySize64 = 207810560

    A workingSet64 value of 195MB !!! Is that scary? With a peak of 211MB???

    Then I looked at which modules are running, and how much memory the various modules were using. I found this:

    proc.Modules, foreach module...
    The application pool has 176 dlls loaded (25 of this dlls are actually written by me), the rest is OS and/or ASP.NET I assume.

    Total memory usage, required to startup these modules, = 157863936
    Total memory usage, required to startup 'my' modules= 6692864
    Total memory usage, required to startup OS and ASP.NET modules = 151171072

    So, the max limit is 200MB.
    Windows and ASP.NET already consume 151 MB
    And my app requires 7MB to start.
    In total: 157.8MB

    Any ideas how I can find out whether I can expect to have problems soon?
    Am I already in trouble?


    Anyone have an idea on how I can determine which module uses the memory space between WorkingSet64=195MB and the Total required memory of 157.8MB?


    Thanx,
    Paul
     
  2. Additionally ...

    the 200MB allocated .. is that as private memory or virtual memory?

    Regards,
    Paul
     
  3. Bruce

    Bruce DiscountASP.NET Staff

    An easier way to measure memory is to pop up the task manager.

    Find the w3wp.exe process and look at the Memory (Private Working Set)

    >the 200MB allocated .. is that as private memory or virtual memory?

    Private Memory.
     
  4. Hello bruce,
    thanks for your response.

    When I use taskmanager, I can only take a look on my local PC. I don't think it is possible to have a look at the application pool on the webserver.

    If I run my own applicaiton on my local PC, then I can see using TaskManager it only takes 58mb (WebDev.Webserver.Exe).
    If I run my code to check the memory usage, I indeed get a comparable figure if I look at WorkingSet64 which is varying as well around the 60mb.

    I think it is safe to conclude that the WorkingSet64 values obtained through taskmanager or through c# code, are similar indicators. The advantage for me to use the c# coding option is, that I can run it on the webserver as well.


    Attached I have enclosed a text file containing the results of three part test.
    The first part contains simple memory usage queries in c#.

    The second part includes a list of all loaded modules (more than 150!!), including the 26 modules of my own application.

    The third test part contains some results I've done using MemoryFailExceptions ... a gracefull way of detecting memory limits.

    It is interesting to have a look at. Here's a summary:
    Memory in use by my application: 6.7MB + 24MB (GC.GetTotalMemory) = 31MB

    Memory in use by all modules together is: WorkingSet64 = 216MB.
    My App is using: 31MB
    The OS and ASP.NET together are using: 216MB - 31 = 185MB.

    Does the above calculation makes sence?

    If it does, it also means that, to stay within the limits of 200MB, I need to profile and optimise my app such, that it can not use more than say 20MB.
    Is that correct?

    Paul




    ps. A live version of the text file can temporarily be obtained from: http://www.reddingssloep.nl/memusage.aspx
     

    Attached Files:

  5. Bruce

    Bruce DiscountASP.NET Staff

    i hooped on the server and the memory usage on your site is around 85MB.

    i do, however, see some events indicating that the site has recycled due to high memory usage.
     
  6. Thanx bruce for that info,

    What's the best way for me to be able to see and log that my website reached the memory limits and needed to be recycled?

    I do rather extensive exception handling, and log those events in a table. Since I did not see this one, probably I still have some unhandled exception. The way to monitor this is probably Health Monitor right? Or do you have other suggestions?

    Looking forward to your response....
    Paul

    ps. Could you do me a favor please?
    Check my memory usage again, using your tools, and compare the value you see with the memory values you see on the page you can find here: http://www.reddingssloep.nl/memusage.aspx

    I think the 85MB memory you found is the same as the value I get when I request for workingset64 ...

    Difficulty is ... in the course of time this value varies... (going up and down)

    So, what I'm asking is wheter you could:
    step 1. Check my memory usage
    step 2. Check on the memuasage.aspx page to which indicator the found value corresponds.


    Thank you very much....
     
  7. Bruce,
    I've taken the following steps:
    - I now log each 10 minutes some memory usage parms, so I can monitor.
    (If you could be so kind, and take look again at my memory and let me know the value and the time (UTC) you looked at it ... I can use that to validate the idicators I'm using.)

    - I've implemented HealthMonitoring ... a custom provider in order to catch any unhandled exception and log about it. I've done a small test, and it turns out that it works. Next hours/days we'll see how it goes in the realworld ...

    Hopefully with this extra logging stuff, we'll see better when and where we go wrong...

    Thanx sofar for your coop,
    Paul
     
  8. Health monitoring "All events" does not seem to detect ApplicationRecyle event.
    Strange...
     
  9. mjp

    mjp

    No it doesn't. You seem to be making assumptions about that usage because you can't track it in your application. I can assure you though, the O/S and .NET are not using 90% of the memory available to you.

    Think about it. If that was the case, with hundreds of users on each server, the O/S would be using all the memory than the server has available (and more).
     
  10. Sorry I made a mistake in my code.
    Health Monitor is picking up application pool recycle event as well.

    mjp: Humm, you probably are right. It doesn't sound logical indeed...
    wisemx: thanx for the pointer ... I got it working now!

    Paul
     
  11. to mjp:
    I agree that I was trying to put together some numbers. It was an attempt to get a better understanding on the memory limit the app has to take into account.


    For now, I think I'll focus simply on a max limit of 200MB for the WorkingSet64 value. Meanwhile keep an eye on the app recycle's ...

    If you have better suggestions, I'm open for it.
    Regards,
    Paul
     
  12. I've worked a bit more on the HealthMonitor Provider.
    It now also logs the memory usage values, in the WebEvent eventhandler.
    This way I'm able to read the memory usage, the moment the app pool is shutting down.

    I'll leave the topic a couple of days ... I'll monitor the logs and if I see things that need attention, I'll pick it up again.

    Thanx for your help sofar,
    Paul
     
  13. Paul...I really appreciate your feedback and updates.
    I've been testing the HM services here for a future Webcast.
    All the best,
    Mark
     
  14. Memory Limit Issue

    Hi Paul,

    Have you managed to resolve the memory issue?

    I have been trying to resolve this issues for 2 weeks now. Our application pool seems to be restarting at random times, sometimes just logging into our ASP.NET application will restart the pool, other times the application is blistering fast, then out of nowhere, you are taken back to the login page.

    The reason we are taken back to the login page is because the pool is restarted due to the application exceeding its memory limit and we lose our sessions.

    Any help would be appreciated.

    Regards,
    Vishal
     
  15. mjp

    mjp

    You may want to start here.
     
  16. Logging Memory Usage and Setting Health Monitoring for 'All Events'

    Hi Paul,

    Can you tell me how you get your live memory stats?

    I had a look at http://www.reddingssloep.nl/memusage.aspx and I was quite amazed you could get this info from the DASP server.

    I am going through a slightly paranoid phase re memory usage and leakage.

    Also regarding Health Monitor settings could you confirm that the following will result in "All events" and everything being logged using the settings at this link... http://geekswithblogs.net/azamsharp/archive/2006/09/10/90827.aspx
     

Share This Page