Detecting Crawlers - Request.Browser.Crawler

Discussion in 'Windows / IIS' started by Will1968, Dec 12, 2017.

Tags:
  1. Hi,

    From what I have been reading for Request.Browser.Crawler to detect if a page request is from a crawler IIS needs to have been set up for this.

    Do you know if the DASP IIS servers have been?

    Thanks,

    Will
     
  2. mjp

    mjp

  3. Hi,

    I set up some email alerts to see if crawlers were being detected by Request.Browser.Crawler and they were.

    So all good.

    Because I have a large Google Merchant feed my site was getting swamped by Google bots checking landing pages.

    So I needed to redesign the behaviour of the website depending on the origin of the page request.

    Finally don't have my website restarting every 15 mins!!

    Will
     
    mjp likes this.

Share This Page