Skip to content
View in the app

A better way to browse. Learn more.

Web Designer Forum

A full-screen app on your home screen with push notifications, badges and more.

To install this app on iOS and iPadOS
  1. Tap the Share icon in Safari
  2. Scroll the menu and tap Add to Home Screen.
  3. Tap Add in the top-right corner.
To install this app on Android
  1. Tap the 3-dot menu (⋮) in the top-right corner of the browser.
  2. Tap Add to Home screen or Install app.
  3. Confirm by tapping Install.

Need help with robot file please

Featured Replies

Good evening,

 

I have discovered a plugin I am using on my Wordpress website is causing a duplicate content problem which I am trying to resolve via the robot file (the author doesn't have a fix annoyingly!).

 

So basically, the listings are featured on the homepage, and each have a seperate page with tabbed sections of info, you can see the website here. It is causing duplicate page titles and meta descriptions :-

 

Original page: www.website.com/directory/

Duplicate: http://www.website.com/directory/categories/clothing?tab=directory

So what I want is for the robots to crawl the directory and it's listings, but not all those duplicate pages.
I have put something in my robot file like this disallow: /directory? which stops any duplicates being picked on anything that has /directory?=example , but I don't know how to apply it to the top two examples as they have changing categories and business names. Is it possible to just stop anything after /companyname and /category?
I am quite new to this so would appreciate any help or comments on this.
Thanks in advance

 

  • 2 weeks later...
  • 2 weeks later...

Adding Robot.text is really veryu important for nay website as if we want ot hide our site from the google robots than robots,text is really very importtant.

Create an account or sign in to comment

Account

Navigation

Search

Search

Configure browser push notifications

Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.