Search Engine Web Crawler - Site Authentication

AdSense Web Crawler visits the publishers' sites, and letting the AdSense system back at Google know what the pages are all about. But the problem come when the site need to be login and asking for password. Hence, Adsense Web Crawler unable to crawl behind that login page. The result is that users end up with poorly targeted ads and the AdSense Crawler ends up with sleepless nights, wondering what could have been if only it had crawled these pages.

Due to such problems, Google Adsense have launched a new feature called Site Authentication to take care of the problem. Using Site Authentication, you can give the crawler access to your login-protected pages by passing it information to log into your site. For example, let's say your news site has a premium content area, with articles that only paying members can access. To get ads on those pages, you can use Site Authentication to provide the crawler with a test username and password. It's an easy process that starts just by logging into your AdSense account and finding the 'Site Authentication' link under the 'AdSense Setup' tab. Once you've supplied Adsense Crawler with a username, password, and a few other details, all you have to do is verify that you own the site through Google Sitemaps.

To check the authentications, login to your adsense account, go to Site Authentication and just follow the instructions on the page. Please note that you will only publisher with Google Account can access to this feature.

Once you've set up your authentication rule and verified ownership, it may take 1-2 weeks for Adsense Crawler to visit your blog or websites in the future.