June 27, 201214 yr Hello all, I have created robots.txt for my site.earlier it was working properly.Robot was followed the rules written on robots.txt. But since last few days suddenly i faced issue regarding web robots visits pages that i have disallowed earlier in robots.txt and send me spam mail. Please help me, why this happens?
June 27, 201214 yr Hello all, I have created robots.txt for my site.earlier it was working properly.Robot was followed the rules written on robots.txt. But since last few days suddenly i faced issue regarding web robots visits pages that i have disallowed earlier in robots.txt and send me spam mail. Please help me, why this happens? Because robots.txt is an optional guideline that only well-behaved robots (like Googlebot) follow. Spam bots don't care about your rules, they're out to harvest as many email addresses as they can.
Create an account or sign in to comment