I'm looking to add some simple tools to my website to help people with their SEO. Simple things such as PR checker.

 

One I'm looking to add is a simple tool to check when Google last crawled a specific website. I've seen them all over but nothing about how they work.

 

Anyone have any idea?

At a guess, they sniff the UA for all visitors, discarding everything except Googlebot.

Edited May 11, 201214 yr by Renaissance-Design

Ah okay. I'll have a look into that then. Just thought it would be a nice addition to my website. I have found a great plugin though for a list of tools. Still needs some styling work done but:

 

http://www.trand.co.uk/seo-tools/

It always strikes me as being odd that Google analytics does not have a subsection for showing detailed stats of their crawler activity because what little info one does get from the webmaster tools section might as well be written in crayon.

 

<?php
$u_agent=$_SERVER['HTTP_USER_AGENT'];

if (eregi("Googlebot", $u_agent)) {
  //// do something.. append a text file or Insert a row to a datbase
  //// IP logging also useful if site scrapers masquerading as search bots
}

?>

 

Googlebot also comes in several flavours there is the basic one

Mozilla/5.0 (compatible; Googlebot/2.1; but has variations such

as Mobile, News, Video and Images http://support.google.com/webmasters/bin/answer.py?hl=en&answer=1061943

Edited May 12, 201214 yr by Sogo7