August 6, 201412 yr Good evening, I have discovered a plugin I am using on my Wordpress website is causing a duplicate content problem which I am trying to resolve via the robot file (the author doesn't have a fix annoyingly!). So basically, the listings are featured on the homepage, and each have a seperate page with tabbed sections of info, you can see the website here. It is causing duplicate page titles and meta descriptions :- Original page: www.website.com/directory/ Duplicate: http://www.website.com/directory/categories/clothing?tab=directory Original: http://www.website.com/directory/listing/companyname Duplicate: http://www.website.com/directory/listing/companyname?tab=reviews So what I want is for the robots to crawl the directory and it's listings, but not all those duplicate pages. I have put something in my robot file like this disallow: /directory? which stops any duplicates being picked on anything that has /directory?=example , but I don't know how to apply it to the top two examples as they have changing categories and business names. Is it possible to just stop anything after /companyname and /category? I am quite new to this so would appreciate any help or comments on this. Thanks in advance
August 14, 201411 yr I would use parameters exclusion in Google Webmaster Tools. You can exclude pages with ?tab there.
September 6, 201411 yr Adding Robot.text is really veryu important for nay website as if we want ot hide our site from the google robots than robots,text is really very importtant.
Create an account or sign in to comment