Tell me more ×
WordPress Answers is a question and answer site for WordPress developers and administrators. It's 100% free, no registration required.
up vote 0 down vote favorite

Trying to find the answer here but can't seem to locate this specific scenario...

We have products that include multiple, defined resources - for example, each has 3 distinct user guides for different situations, each has a few videos, etc.

Well, we used to simply have a page for each product, and all of this information in accessible tabs. This worked well for SEO. But we have over 800 products, and started getting request to, for example, show all of a certain type of manual, or all of a certain type of video.

So, we decided to make a resource section using posts, each post in a specific resource category (i.e. Manuals) and representing a single resource (such as, a manual) - and then tag them each for a specific product. When viewing each of these resources directly, they include a link to the actual product page. This allows the resources to be polled independently, and shown in user requested archives.

On the product page side, we are then 'INCLUDING' these individual resources into the tabs for each product, shown in their entirety, using a plugin that lets us query our resources. Each resource is shown within these tabs in their entirety, with only the product links stripped, making the content look native to the page - and in fact our pages look exactly like they used to.

But suddently, a question has come up - does this now create a duplicate content situation? We really want to leave our resources indexed but in the eyes of a search engine I could see where this might be a problem.

I also should add, when a site search is performed (using standard WP search), only the individual resource posts come up - so it does not see that information as existing on the actual product pages, as one would expect - but I presume for visiting bots that is not the case, they are seeing each product page and all of it's included tab content, as well as the same content in their individual resource posts?

Would really appreciate any thoughts on how to best deal with this from an SEO perspective.

Thanks

CLARIFICATION

I am adding this to clarify what I'm asking as it relates to Wordpress:

1) Am I correct that under this scenario and how wordpress works, the SE's will see the full product posts with all of the included content?

2) Since the Wordpress 'search' doesn't see it this way (presumably because it is searching the database, not the actual generated pages), then if we were to noindex the 'Individual Resource' posts, will Wordpress' inability to show the main product posts in a search have a negative effect on SE's for those pages? Would a better search plugin be required in this case?

3) Finally, if we don't have any menu items pointing to the individual resource pages, and the only time they would ever appear is via a search (which is the case currently), will the SE's still find these pages due to how Wordpress works, even if we don't include them in the site map?

Thanks

share|improve this question
1  
This is more of an SEO question rather than an actual WordPress one (in my opinion) and Im not sure if SEO questions are relevant or not. Double check the FAQ..wordpress.stackexchange.com/faq#questions and look at the "RELATED" questions to the right for more related to yours. – Aliyah Feb 16 at 23:27
Well, we're trying to understand how Wordpress is delivering this information, and how we need to adjust Wordpress to control it - after all, the content is managed and delivered by Wordpress? It's a gray area, but the answer needs to be related specifically to Wordpress and it's inherent controls for delivering content. – Soyo Feb 16 at 23:32
Until someone answers, you can check the related q's off to the right. – Aliyah Feb 16 at 23:34
maybe this helps? wordpress.stackexchange.com/questions/46854/… – Aliyah Feb 16 at 23:35
Hi thanks - I did see that already, it is discussing sub-pages and URL structure - but not including that content in its entirety directly into a front page. In fact, what we're doing seems a bit unique unfortunately... So the way wordpress handles canonicalization, site maps, etc. all plays a role in our dilemma that I can't seem to find addressed anywhere else... though, I too am continuing to look ;-) I appreciate your feedback and help... – Soyo Feb 16 at 23:42
show 3 more comments

1 Answer

up vote 0 down vote

If the resources and the products are accessible on different unique URLs, but those pages shown the same content in certain areas, it could definitely be flagged as duplicate content.

Sometimes you can't avoid duplicate content. An example, to stay on the WordPress topic, is when you have blog posts, and category pages that list entire posts. The posts or category page would be seen as duplicate content.

You can control somewhat how Google and other search engines treat this content. I'll list a couple of ways. I will treat the question as if it was purely WordPress related, and you can translate it to your own implementation. (I believe that will be the best way for WP SE to benefit).

Canonical Pages

A subtle solution is using canonical Pages. On the post's page you would insert the follow meta tag in the <head> section:

<link rel="canonical" href="http://www.example.com/post-title/" />

Remember to self-close /> this tag (or any <meta> tag). Also, here's an article from Google about using Canonical Pages.

This new option lets site owners suggest the version of a page that Google should treat as canonical. Google will take this into account, in conjunction with other signals, when determining which URL sets contain identical content, and calculating the most relevant of these pages to display in search results.

Using the Robots <meta> tag

You can instruct search engines to not index a page. Also, using the same tag, you can instruct them to ignore all the links found on that page (and not crawl through them).

<meta name='robots' content='noindex, nofollow' />

noindex will, as the name implies, instruct the search engine to not index this page. nofollow will instruct the search engine to not 'click through' on any links found on that page.

Do not confuse <meta name='robots' content='nofollow' /> with <a rel='nofollow>'.

Using robots.txt

You can instruct search engines to ignore entire sections of your website using a robots.txt file. Place this file in the root directory of your website, and make sure it can be reached through http://www.example.com/robots.txt.

The contents of this text file should be the following:

User-agent: *
Disallow: /cgi-bin/
Disallow: /tmp/
Disallow: /~joe/

Sitemap: http://www.example.com/sitemap.xml.gz

A good idea is to include a sitemap (there are plugins for WordPress).

Please know that there is no such thing as 'wildcards' (like *) in robots.txt, even if Google says different. The asterisk on the User-agent line is the only allowed wildcard. It will not work on Disallow directives!

There is also no such thing as the Allow: directive. While Google may follow these improvements on the robots.txt concept, they are certainly not obeyed by all search engines. Unless you are specifically catering for Google, only use the directives as described on the official robots.txt website.

Good to know is that even without an explicit wildcard, you can still target multiple things.

Disallow: / will prevent search engines from indexing your entire website (root directory and everything in it).

Disallow: /joe/ will prevent search engines from indexing everything inside the joe folder, which is located inside the root directory.

Disallow: /joe will prevent search engines from indexing everything inside the root directory starting with joe. So joe.html and joey.html will not be indexed, but hank.html will.

Last remarks

Remember that even if you do all three of these things (which I encourage), search engines do not explicitly have to obey these instructions. They are just that: instructions. Especially malware crawlers will ignore anything you instruct simply because they want to find out all they can about your website.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.