Get content from the website only open to Search Bots


Ever experienced this? You ask Google to look something up; the engine returns with a number of finds, but if you try to open the ones with the most promising content, you are confronted with a registration page instead, and the stuff you were looking for will not be revealed to you unless you agree to a credit card transaction first….

hiden eyes
The lesson you should have learned here is: Obviously Google can go where you can’t.

In fact, many sites that force users to register or even pay in order to search and use their content, leave a backdoor open for the Googlebot, because a prominent presence in Google searches is known to generate sales leads, site hits and exposure.

So, disguise yourself as a Googlebot !!!

Quite simple: by changing your browser’s User Agent. Copy the following code segment and paste it into a fresh notepad file. Save it as Useragent.reg and merge it into your registry.

Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE’SOFTWARE’Microsoft’Windows’CurrentVersion’Internet Settings’5.0’User Agent]
@=”Googlebot/2.1?
“Compatible”=”+http://www.googlebot.com/bot.html”

and ……… Voila! You’re done!

You may always change it back again. I know only one site that uses you User Agent to establish your eligability to use its services, and that’s the Windows Update site.

To restore the IE6 User Agent, save the following code to NormalAgent.reg and merge with your registry:

Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE’SOFTWARE’Microsoft’Windows’CurrentVersion’Internet Settings’5.0’User Agent]
@=”Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)”

NOTE:
Opera allows for on-the-fly switching of User Agents through its “Browser Identification” function,

while for Mozilla/FireFox browsers a switching utility is available as an installable extension from this url:
help://chrispederick.myacen.com/work/firefox/useragentswitcher/download/

Popularity: 34% [?]

Listen to this article Listen to this post
Bookmark and Share: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • Digg
  • Technorati
  • del.icio.us
  • YahooMyWeb
  • blinkbits
  • Furl
  • IndianPad
  • feedmelinks
  • blogmarks
  • Internetmedia
  • Linkter
  • MyShare
  • PlugIM
  • Simpy
  • Spurl

Leave a Comment

You must be logged in to post a comment.


Close
E-mail It