Imagine the nausea and anxiety of having your site disappear without trace from multiple major search engines in the space of a few days… no longer able to be located on Yahoo’s new search engine, and consequently MSN, plus other Yahoo search partners! And this a niche tourism site which, by design, had for the past few years enjoyed No.1 placement on all the major search engines for an array of relevant keyword phrases, and was generating 10,000+ “qualified” visitors per month.
We are not talking about a slip in the rankings here either – we are talking about a total purge from Yahoo’s web results, despite the site being a long-standing, high ranking site within the Yahoo directory. We are also talking about a site which continued to rank No.1 on Google throughout this little crisis. A site which had not had any significant recent modification, had no “dodgy” optimisation trickery, and which has been a stable, reliable repository of 300+ pages of useful, regularly updated content. How could a site like this disappear, you ask? Could it happen to me, you ask? Let me explain how it evolved…
After checking search results intently for a week, hoping the site would bounce back into the limelight, and wracking our brains to figure out what had happened, we decided to subscribe to Yahoo’s Site Match program to get the site re-indexed. And re-indexed within a couple of days it was, according to the Site Match stats, but still it did not reappear in the index. An urgent message to Site Match support received a very prompt response, as follows;
“Hello Ben,
Yahoo does not like custom NOT Found pages used to advertise other links to your site. You may want to revisit this approach.”
Best Regards, Overture Site Match Support, Overture@PositionTech.com
Gulp… the site had always had a custom 404 page, and since a major site revamp 3 years ago the 404 page has had a mini site-map providing links to content relocated onto other sites. We very quickly blew all that stuff away, but the anticipated reappearance in the Yahoo search results did not occur. Again, a mayday to Site Match support received another prompt response, as follows:
“Hello Ben,
It is apparent that you had fixed your 404 problem. However, we ran your URL through an HTML validator and found that it did have errors in the code. Even though this is not critical, every little bit helps in getting good placement to drive traffic to your site. Here is that URL to the validator. We hope this helps. http://validator.positiontech.com/anthr.cfm”
Best Regards, Overture Site Match Support, Overture@PositionTech.com”
Well, under stress, its always good to have a goal…. So after a visit to the validator URL above, and verifying that there were indeed some issues, we downloaded a copy of CME’s HTML Validator Lite 3.5 from www.htmlvalidator.com and systematically eliminated every issue. None were major – some extra and tags, which translates as incorrect nesting. The home page title had an ampersand done as “&” instead of “&” So now, there is not a single HTML error in the site’s index.htm page…
Now, I have to say we had serious reservations about something as simple as this being the key to reinstatement in the Yahoo index. I mean, since when have search engines been pedantic to the point where they will completely banish a site for a few minor code errors? Google, the epitome of search engines, has obviously accommodated or ignored the minor errors on the site.
White text on white background, ok… but an ampersand? Please!
Well, to my complete amazement, within 12 hours the site was back in the Yahoo index, sitting at No.1 for the expected range of keywords within Yahoo web results.
All the best stories have a moral, and this one could even achieve parable status.
- First thing is, go check your custom 404 pages.
- Second, tidy up your HTML code before they drop your site.
This exercise has got to be good for you (just like organic muesli, bran flakes etc). After all, how often do find a legitimate excuse to mess with raw HTML code these days? Honestly, either option is about as appealing as a bout of self-flagellation!
Speaking for myself, I have an intense dislike of anything that raw – muesli and/or html. I must also make the point that I did not make the HTML code mistakes myself, you understand? (Well, maybe I did the & ampersand thingy) J
Complacency is the issue – the site was error free on creation, but has been online for several years, generating business. The regular maintenance, rebuilds, rearrangements etc had obviously allowed accumulation of some minor HTML code errors over time, and we had foolishly not kept a close eye on that particular issue, relying on the HTML editing software too much.Like most webmasters, since their invention I’ve cheerfully and unashamedly used a “WYSIWYG” HTML Editor - which of course inserts the dodgy code all by itself. In my case, the culprit is Microsoft’s FrontPage, which I am very fond of. I love it because it does lots of really, really clever stuff at a very sensible price. There is a touch of irony to this story, because MSN Search is currently getting its results via Yahoo… Microsoft’s own software could therefore be responsible for eliminating some very good sites from its own search index.
So now, we’re back in the Yahoo index at No.1 but not yet out of the woods. We are out a hundred US bucks though! To sign up for the Site Match program, first there was the US$50 sign-up fee for the indexing for a year, and then the US$50 in advance for the 15c per click-thru from the search result listing…. However, we’re very grateful for the rapid and accurate advice from the Site Match support team!
I wonder what happened to those guys?
