web704: Error 503. The service is unavailable.

Discussion in 'General troubleshooting' started by team929, Jun 21, 2009.

  1. Is anyone else having trouble with this?

    My sites been running since it's start 2 months ago without a problem thenthe server went down around 11:50pm (PST, 6/20/2009) all of a sudden and is still down with the same message at the time of this writing 11am(PST, 6/21/2009)

    Can a staff or someone else confirm if this is just me?

    I've emailed support but being a Sunday, not sure if I'll get a response.

    Thanks in advance.
     
  2. Bruce

    Bruce DiscountASP.NET Staff

    503 is a very generic error and i can't really tell what's going on unless I check on the server.

    You'll have to wait for support to check the site and respond to you.
     
  3. Interesting, as I have started seeing the same problem on web704 too, first time was about 4 weeks ago. Now it happens at least every week, but recently less than daily. I have only started looking through the forums today for answers/help. Just wondered if you got to the bottom of it?

    PS I have replied to 2 threads so far on this same subject, I am not trying to spam my problem all over the place, just seeking some help and adding my experiences into the mix....
     
  4. Bruce

    Bruce DiscountASP.NET Staff

    open a support ticket. we'll be able to look at the event log and tell you why your site is getting that error.
     
  5. Hi Bruce, I did actually do this and I was kindly supplied with an event log entry. (below). I have optimised all the code I can, removed any routines I thought might be hungry (although they are code snippets taken from sources which claim no problems with them) and run profilers against my staging server. I also enabled raw logs and have reviewed them up to the point when it hangs, but all I see is normal activity leading up to the stop. (Web crawler bots mainly). However I am still at a loss :confused:

    Is there any more detailed analysis I can do or that dasp can provide to help? I am not looking for a code review, that is clearly outwith your remit, but anything that would help me home in on 10000's lines of code would help :) Does memory usage climb constantly until it hangs? Is a peak in CPU use that causes it? It's difficult to track this one down on the staging server, which although is publicly online, does not receive the hits that the live one does (obviously).

    Thanks

    A process serving application pool 'customerPool_xxxxxx' exceeded time limits during shut down. The process id was '10160'.

    ******
    [ Name] Microsoft-Windows-WAS
    [ Guid] {524B5D04-133C-4A62-8362-64E8EDB9CE40}
    [ EventSourceName] WAS

    - EventID 5013
    [ Qualifiers] 32768
    Version 0
    Level 3
    Task 0
    Opcode 0
    Keywords 0x80000000000000

    - TimeCreated
    [ SystemTime] 2009-09-06T07:56:29.000Z
    EventRecordID 1592811
    Correlation

    - Execution
    [ ProcessID] 0
    [ ThreadID] 0
    Channel System
    Computer web704.dotnetplayground.com
    Security

    - EventData
    AppPoolID customerPool_xxxxxx
    ProcessID 10160
     
  6. Bruce

    Bruce DiscountASP.NET Staff

    this error indicate that the process has gone into a hung state. It is very difficult to troubleshoot on the server because we have to catch it while it is hanging.

    I recommend you running the same requests in your dev environment and see if it hangs.
     
  7. It doesn't hang at all, or even show high memory use or any leaks on the staging server ... although as I said, it's hard to replicate live traffic...
     
  8. Bruce

    Bruce DiscountASP.NET Staff

    hmmm.. sorry.. unfortunately, there's limited thing we can do as a host.

    is your dev server running iis 7?
     
  9. Windows Server 2008, IIS7 however it's 64 bit. That's the only difference.

    (Went again, while I was on the site this morning (07:15 UTC)) Was viewing a table using LINQ from a listview. Nothing unusual.
     
  10. Bruce

    Bruce DiscountASP.NET Staff

    we also run w/ 64 bit OS but the application pool is set to run in 32 bits mode. May be that's the problem.
     
  11. I think it's unlikely as I have to 'enable 32 bit' in my staging app pool too so that the 32 bit components I use work on staging and live. However I do have 64bit versions of the components. Just wondered 2 things. Can I ask for my app pool to be run in 64bit mode and is it possible to increase the app pool memory allocation even if it's for a bit more money a month? (I realise this may be masking the issue, but just interested to know).

    Thanks for continuing to comment on my problem, it's appreciated. A clear indication of dasps quality service.
     
  12. mjp

    mjp

    The application pool cannot be increased on a per-site basis, nor can we change the mode it runs in, I'm afraid.
     
  13. Bruce

    Bruce DiscountASP.NET Staff

    we do not support 64 bit app pool because of many potential problem.

    200 MB is actually quite a bit. Most sites on the server use about 50MB.
     
  14. Good to know, I suppose like others on here I could use a scheduled task and the api to recycle the app pool each day at a low usage time. Is there a way to query the app pool for memory usage using the api? If not is there a way to request it as a future enhancement?
     
  15. Bruce

    Bruce DiscountASP.NET Staff

    We may be able to set a daily recycle for your application pool. create a ticket.

    Not currently.
     
  16. With no changes to the site, this has just stopped happening, I am nervous when something appears to be resolved without knowing why. Any ideas?
     
  17. Bruce

    Bruce DiscountASP.NET Staff

    i wish i have an answer for you. it could be a specific page in your site that's causing the high CPU usage.
     
  18. Im sorry, but I think "resolved it self" isnt really the right way to think about your issue. Its not a bug to begin with. Its just that your application didnt exceed memory or CPU limitations to cause the issue again.

    I want to point out a couple things that I think would be helpful to everybody with such issues. They are just some general comments that I think arent really understood very well.

    These issues are almost always related to the application itself. To explain, we have customers serving nearly a million hits a day to 60,000 unique visitors and never exceed resources or encounter 503 errors. At the same time, I have seen customers that run certain well known (I wont name any names) CMS packages that cant surf their own website without exceeding memory allocation.

    The point being while its natural to assume the issue is related to something on the server, ESPECIALLY when the application works locally, just because the application compiles, doesn't mean its ready to put into production.

    An application running locally with an entire computer to itself and only one user should not be used as the litmus test. Just as a web designer needs to think about how long it will take a mobile or modem user to download his images, as developers, we need to think about the impact of actual usage in our target production enviornment.

    Ok we dont HAVE TO, we could just pay a couple hundred a month and have dedicated servers to make up for it, but I think most people dont really see that as a solution.

    So where does that leave us? We should spend our time reviewing the http logs and see what actions occurred durring the time of the 503. Going through the logs will tell us things like what pages were being hit, how many users were on etc. With these clues we can then start to look at the application and decide what we can change to make it more efficient.

    We run windows here - there is no special sauce added. Its windows. Its a pretty safe bet to assume its not some weird variation in configurations that caused the errors most customers see. Remember, .Net is taking care of the environmental issues. Not to say it cant be that one undiscovered bug thats haunting you, but without investigating, were not going to know what it is anyway.

    At the end of the day - it all goes back to the application. That is the one and only place answers can be located.
     
  19. I do understand all of that. I just wish I knew what changed, be it visitor behaviour, some sort of attack that stopped, who knows. The only idea I have is that back in May/June I started using the asp.net membership system much more extensively, so many more users were logging in, however it wasn't till some months later that it started having issues. I was quite well down the line with various analysis tools and monitors and made many, many improvements to the code design. However none of those improvements correlate with the issue going away, it happened every day even after the improvements were made. I had given up and was looking for a dedicated host, then it just stopped. It hasn't happened once since I last posted despite the fact the site usage has risen slightly. However the episode has made me consider my code more carefully. Fingers crossed it doesn't come back. I am really just posting this in case others benefit in some way.
     

Share This Page