Got ASP 2.0? GoogleBot Can Trip a Bug in the Code (Search Engine Roundtable)
Got ASP 2.0? GoogleBot Can Trip a Bug in the Code (Search Engine Roundtable)
Brendan Kowitz wrote a blog post about an interesting anomaly he noticed as Googlebot was crawling his site. Apparently, in March of 2006, Googlebot’s User-Agent string was changed which triggered a bug in ASP.NET webservers. What this would mean is that Googlebot would not crawl the pages properly; instead, it would get 500 Internal Server Errors, thereby assuming that the pages were …