Google recently have been working hard on developing tools and new methods to combat duplicate content. 301’s remain the best fix and prevention is best done with robots.txt blocks and nofollow, but the new tools are great if you can’t get access to redirect or block.

duplicate-cat

Duplicate content causes a split of page rank, can cause some pages to be filtered from rankings but big websites seem to not care about the issue. If the big site’s don’t care about it, why should a site for a small business concentrate on often making lengthy changes or spend time on re-directs that some sites don’t even bat an eyelid at the issue?

Let’s have a look at some examples.

BBC

On the whole the SEO on the BBC is good but they do have a duplicate content issue on the site. I first pointed this out in a post back in February. The problem seen was two URL’s for each page.

http://news.bbc.co.uk/sport1/hi/football/teams/a/arsenal/7831046.stm

You also have a second URL, the difference it’s in the folder sport2 and not sport1

http://news.bbc.co.uk/sport2/hi/football/teams/a/arsenal/7831046.stm

On top of that there is also the low graphic version of the page.

http://news.bbc.co.uk/sport1/low/football/teams/b/blackpool/7831046.stm

And under the sport2 folder

http://news.bbc.co.uk/sport2/low/football/teams/b/blackpool/7831046.stm

Facebook

Another post I did a while ago where your profile can be loaded up on two URL’s

http://www.facebook.com/johnpcampbell

and

http://en-gb.facebook.com/johnpcampbell

Also some profiles now appearing with ?_fb_noscript=1 after the URL’s. That example above isn’t indexed but these two are http://www.facebook.com/wgardner69 and http://en-gb.facebook.com/wgardner69?_fb_noscript=1 some random person!

LinkedIn

Spotted by a work colleague of mine Neil Walker (follow him on twitter @theukseo) he noticed LinkedIn had a duplication problem with two URL’s for his profile.

http://www.linkedin.com/pub/neil-walker/4/41a/793

http://www.linkedin.com/in/internetmarketingoptimisation

Travel Supermarket & Virgin Media

Another spot form Neil was a very strange duplication on Travel Supermarket & Virgin Media. This time it looked like they have duplicated content on a sub domain rather than having two URL’s for one page of content.

Twitter

Can’t remember who spotted this (please comment and I’ll link) but twitter has a https duplication problem and a mobile sub domain duplicating.

m.twitter.com/johnpcampbell

twitter.com/johnpcampbell

https://twitter.com/johnpcampbell

Looking today there is also explore.twitter.com/johnpcampbell indexed but they have a fix in place in the form of a 301 re-direct to twitter.com/johnpcampbell

Should you still care about duplicate content?

In all these example due to the size and the power of the sites it’s not really having an adverse effect on their overall performance (like throwing a dart at godzilla! he’s not going to feel a thing). Google seems to be able to work out which is the correct URL to display. It would be nice to know the effects of correcting this as these sites have so many pages.

Just a little fix to stop duplicate content on twitter would cut the crawling time of Google allowing the search engine to spider more pages. Unfortunately we’ll never know but I’ll keep on fixing site-wide duplicate content issues.

Do you thing big companies need to sort out duplicate content issues? Add a comment

A huge part of being a good SEO is digesting news, blogs, views, reports, whitepapers and filtering out the crap. He’s a list of 25 companies in the UK offering SEO services who are publishing some content that isn’t crap. They range from the small to the large but all tend to push out good content on RSS.

Links to Site in alphabetical order

Amaze
Base One
Big Mouth Media
Branded3
Bronco
Data Dial
Distilled
Fresh Egg
Further
Green Light
I Crossing
Just Search Ltd
Lake Star Media
Latitude
NorthSouthMedia
Push On
Receptional
redsauce
SEO Consult
SEO Optimise
Site Visibility
Stuck On
The SEO Company
Verve Search

RSS Feeds in alphabetical order

Amaze
Base One
Big Mouth Media
Branded3
Bronco
Data Dial
Distilled
Fresh Egg
Further
Green Light
I Crossing
Just Search Ltd
Lake Star Media
Latitude
NorthSouthMedia
Push On
Receptional
redsauce
SEO Consult
SEO Optimise
Site Visibility
Stuck On
The SEO Company
Verve Search

Twitter Accounts – 18 account 6 missing (Branded3, Data Dial, Further, Push On, redsauce, The SEO Company)

amazeplc
baseonegroup
bigmouthmedia
bronco
distilled
freshegg
Warren_Cowan
icrossing_uk
justsearching
lakestarmedia
Latitude_Group
northsouthmedia
Receptional
seoconsult
seoptimise
SiteVisibility
StuckonSEO
LisaDMyers

Follow all in one click with http://tweepml.org/25-UK-SEO-Companies/

Missed any off ? Then please add to the comments and I’ll add them on the post if they are any good!

EDIT A few to add from comments from people

Verticle LeapRSS @verticleleap

MediaEdge – No RSS – @MECmanchester

MediaVest – No RSS – @mediavest_leeds

Ayima – No RSS – No Twitter

Multiple XML sitemaps can be used to help detect how the indexing of the site is progressing or if there are any problems. For example if you have 20,000 pages in the site and only 15,000 are indexed it’s hard to find out which 5,000 pages are not indexed by the search engine.

Google Webmaster Tools feeds back on this information saying how many pages have been indexed from each XML sitemap, best of all you can submit multiple sitemaps to the search engines.

multiple-sitemaps

Breaking the sitemap down can pinpoint which sections of your site are not getting indexed. So lets break that 20,000 pages down.

pages-indexed

So we can see that the main problem is with the products section not getting indexed with only 44% of pages indexed. This now means you can start to find out the reasons why your product pages are not being indexed.

That then opens a new can of worms as it might be due to duplicate content, poor coding, internal linking, server issues etc. The good point is that you now know what area is the problem.

To link the XML sitemaps together you should have one sitemap called sitemapindex.xml that links to the other sitemaps using the xml protocol. Submit all the sitemaps to Google Webmaster Tools and list all the sitemaps in the robots.txt. E.g.

Sitemap: http://www.domain.com/sitemapindex.xml
Sitemap: http://www.domain.com/categories.xml
Sitemap: http://www.domain.com/sub-categories.xml
Sitemap: http://www.domain.com/prodcuts.xml
Sitemap: http://www.domain.com/blog.xml

So if you think your site has an indexing problem then split your site down logically into different XML sitemaps. It’s not going to solve your problem but will point you in the right direction.

Posted in SEO.

As we know ever the biggest sites have duplicate content and SEO problems (See the BBC duplicate content post). I’ve come across another site site with a duplicate content problem today, Facebook.

two-facebooks

With the recent change to get a URL with your name in it also seems that my profile can also appear on another domain.

http://www.facebook.com/johnpcampbell

and

http://en-gb.facebook.com/johnpcampbell

At the moment the en-gb.facebook URL isn’t indexed in Google but there are other people’s profiles indexed. Twitter also have a duplicate content issue with indexing on the secure https server.

Wonder if they’ll start to 301 to the other? I’ve seen a number of Google results with both URL’s listed.

Posted in SEO.

If you’re an avid reader of any UK SEO blogs you should have noticed posts about the recent shift with foreign (predominantly American) sites, outranking many UK sites on certain search terms. The problem has been seen under a wide selection of search phrases from many SEO companies / individuals in the UK. There have also been some indicated of the problem reserved with UK sites ranking well in Australia search results.

So what have we seen?

– 1st / 2nd page can seem unaffected with no movement at all.
– Around 10-30 sites rank from the 2nd / 3rd page thus pushing rankings 2 to 3 pages down.
– Domains are .com and hosting in foreign country or have a country specific TLD e.g. ww.domain.be

What should you do?

– Continue to add relevant unique content to your site be that news or a blog
– Continue to build links to the site, don’t increase this rate rapidly.
– Read SEO blogs to track the changes.
– Clean up any errors showing on Google Webmaster Tools, doesn’t have any effect just something to take your mind of the problem.
Don’t panic and sit this problem out, Google should* reverse this problem.
* Nothing is guaranteed but with so much talk about this problem in the UK Google has to rectify this problem.

The issue seen by many UK SEO companies is that when Google makes a change such as the one seen you can’t really do much to rectify it. Clients will be annoyed about this stance but you can only sit it out and wait for the Search Engine to update, explain the issue to clients and keep them up to date. Most importantly you can’t really complain about Google, for all we know they could shut down tomorrow and there is nothing we can do about it!

An Idea

I do have one idea that could help thanks to Bing. Collective pressure from UK SEO companies on Google to either publically acknowledge the problem or get a fix in place. The only way this could succeed is by getting the issue in the public domain, on the BBC, ITV, Sky News etc. This then forces Google into doing something to prevent losing users to Bing, which is actually succeeding in being decent alternative to Google for UK public.

If you want to read up more on this or need proof to see that this isn’t an isolated case then here are 11 articles on the problem.

Google American Results Conspiracy – Fresh Egg

US websites ranking in Google.co.uk an example commercial laundry equipment – A UK SEO

Google uk mixing in us based queries – SEOptimise

Big Geo Problems Still Exist in Google UK Serps – PPC Blog

Google UK Serps Com On Down – North South Media

Have you noticed Googles shift from UK to US? – Argo IBC

The mystery of the UK SERPS – updated – Search Cow Boys

Will the Real Google Search Results Please Stand Up – SEO Design Solutions

Is Google’s Geo Targeting Off In Web Search? – Search Engine Round Table

UK Ranking Problems – UK Business Forums

Poor State of UK search today – Andrew Girdwood – Big Mouth Media

If you have any more, add them into a comment below and I’ll add them to the above list. Do you think collective pressure would get a responce from Google?

A little off SEO topic but I’ve come across a presentation style called Pecha Kucha. The idea is very simple.

20 slides – 20 seconds on each slides.

The slides should be pretty simple, maybe just a full screen picture without any text. It’s seems a great way to get over a topic quickly while keeping your audience’s attention. Often watching presentations online can be really boring (never mind in a stuffy office) but if you know that every twenty seconds a slide is going to change, users should stay to view all 20 slides.

I came across Pecha Kucha watching this video about building Links from the team at www.sitevisibility.co.uk

19 Ways To Build Links to Your Website – Pecha Kucha Kelvin Newman from SiteVisibility on Vimeo.

Posted in SEO.

There has been lots of chatter about a recent shift in ranking by Google with US based sites ranking in Google.co.uk, seoptimise and Search Round Table reported in articles and there was plenty of tweets about the movement. It’s caused a few problems with plenty of UK sites moving down the rankings due to the influx of US based sites.

It’s hard to quantify sometimes what has shifted in the rankings has done as you don’t have before and after data of the top 20, 30, 40 rankings.

Fortunately I have a report from January the 18th 2009 for the top 40 sites in google.co.uk on a search for “commercial laundry equipment”.

I’ve then taken the top 40 websites for the same search query today and the difference is concerning with US sites ranking in the place of many good UK sites. There is also sites from Belgium, India, Spain and China ranking in Google.co.uk.

The list is big so I have added it into a Google Docs Spreadsheet which can be seen by clicking the image.

commercial-laundry-cleaning-search-engine-results

It would be great to get someone from Google to have a look over this. If anyone has seen something similar with their clients or SERP’s post the details below in a comment. Also link to your own blog with an example of the movement of US sites in the UK rankings.

UPDATE 17/06 : Fresh Egg have an article on a similar problem with the key phrase Pet Surgery. Check out the article which has a great conspiracy theory behind the US sites ranking.

Rather than spending time on the odd blog post here and there I’ve added a SEO guide section to the site. I’ll be adding a page in there every so often about a specific subject. To start SEO Friendly URL’s which concentrates on the common problem with SEOFURL’s

Head over to “A Simple guide to SEO Friendly URL’s” to hopefully learn something new and useful.