4 Replies Latest reply on Jul 1, 2014 9:01 PM by Pankaj.Rajankar

    Web Crawler Proxy settings

    Karthik V - 981672

      Hi,

       

      Am trying to do a web crawl for some internet sites and am providing the required proxy properties as well as the crawl.scope.mode as ANY in the site.xml.

       

      When i start the crawl it tries to fetch the seed URL's and it is not able to fetch any of the seed URL's and it gets completed after retiring the URL's based on the depth of the crawl. This happens the same in my development box(UNIX) as well as the local(Windows). Please help me out asap. I am stuck with a deadline and am not able to find any information with respect to this in any other forums or even in the archives here.

       

       

      Thanks & Regards,

      Karthik V