PDA

View Full Version : a lot of recurssion


blade700
07-20-2007, 09:53 AM
Is anyone using robots.txt to stop Google reading submit.php? There's no point in it trying to follow that link on every page is there?

I'm thinking that there's a lot fo recurssion at the moment and looking for ways to chop it down a bit.

vegabond
07-21-2007, 01:58 PM
No, I did not! If you don't want to be crawled by SE then you may set NO Follow attribute for this link :)

blade700
07-21-2007, 02:30 PM
thought the "nofollow" meant that it got traversed but no PR leak

droolingmnky
08-02-2007, 07:13 AM
The nofollow won't stop it from getting crawled, it'll just stop the passing of link juice. If you don't want it crawled then add it to robots.txt