Online Marketing Newsletter - March 2004

The New Search World – Two Goliaths
A month ago Yahoo began rolling out its new search engine, dropping Google as it's main source of search results.

With the ‘New Yahoo” there are only two major search engines now (for the moment, see below about MSN); Google and Yahoo.

Now that Yahoo no longer displays results from the Google search engine, Google's share of the search market (in North America) has dropped down to about 50% (They're still number 1). Most of these searches are performed at the Google and AOL search sites.

Yahoo which owns Inktomi, AltaVista, and All-the-web as well as Overture, now accounts for about 44% of searches (About 15% of this are searches on MSN search. Most of these results come from the Inktomi search engine whihc Yahoo owns).

So together Google and Yahoo account for about 94% of searches in North America. All the other search engines make up the remaining approximately 6% of searches. Needless to say, improving your results on Yahoo and Google is the name of the game.

Both Yahoo and Google spoke at the search engine conference I attended in Orlando last month. It was interesting to hear them both talk about improving relevancy as a major goal and to let us know that SPAM detection will be a key focus over the coming months.

Both have been and will continue to implement more “signal” detectors; filters to detect SPAM or indicate the possibility of SPAM. In many cases penalties will be automatically applied. In other cases a signal detector will trigger a flag that will be examined by a human.

One major area of focus that was talked about is employing link analysis to detect the rampant growth in various linking tricks many companies have developed to try to improve their rankings such as setting up numerous domain names and cross-linking the sites to inflate a site's apparent popularity.

Google explained how relatively easy it is to detect this kind of link SPAM. It's very obvious when they see numerous links between sites, but very few links from the rest of the “community of sites” for a topic pointing to those sites. Google acknowledged that they will go so far as looking up the ownership of the domain names. There's even been a rise in a practice know as “domain name registration by proxy” as companies try to hide their domain name ownership.

I can't help thinking how much better these companies would do in the long run if they focused there efforts on building the best sites they can for their visitors, why employing legitimate optimization methods that are not considered SPAM, instead of spending all their energy trying to artificially affect there rankings.

Many of these companies are in for a real shock when Google or Yahoo detects their SPAM and the site is penalized or removed from the search engine. I've worked for clients who've had this happen. It takes months, sometimes more than a year, to recover!

First as I mentioned in the last newsletter, in our testing just after the announcement of the new Yahoo Search Engine, we found that most of our clients already had some number of pages indexed (The new Yahoo search engine database likely began life as a copy of the Inktomi database. Plus Yahoo had been crawling the web for some time before the annoucement.)

Yahoo will continue its ‘free crawl', following links on web pages looking for new pages. So your site and the pages on it may be found as long as we do a good job of Link Building. Then it's largely a matter of having a ‘search friendly' site design for Yahoo, and all search engines, to be able to crawl the site and index many of the pages).

Yahoo also has a free “site submit” that we can use to submit a site that hasn't been found, but as is the case with all free submission tools it can take weeks to months for the crawler to visit your site and there are no guarantees (It's much better to let a search engine find a site by following links to it from other sites).

Yahoo also has a new paid inclusion program that we can use when it makes sense. This program will be handled by Overture, which is owned by Yahoo. The new program is called Overture Site Match.

The new inclusion program works very much as the former Inktomi Site Submit program did. There's an annual submission fee, starting at $49 for the first URL down to as low as $10 per additional URL in quantity (For up to sites with 1000 or less URLs to submit. There's another program for larger sites).

Yahoo also charges a cost per click fee of 15 cents or 30 cents depending on the category. I think this is going to be very confusing and even controversial. People have gotten somewhat used to idea that cost per click (also call Pay per click, or PPC) programs mean you are paying for position, the more you are willing to pay per click the higher up in the results your listing will appear. These listings are usually displayed in a separate area of the search results and are referred to as Sponsored Results, etc.

However, this is not the case with the Yahoo's new program. It works as most Paid Inclusion programs have always worked; you are paying to have your page included in the database. Where the page will appear in the results of a search will still depend on all the criteria the search engine evaluates to determine the ranking of all the pages it finds related to a specific search. (PS – You should be aware that there have been similar inclusion programs for some time for very large sites, such as an ebay or Amazon.com might use to submit pages. These programs are called Trusted Feed or XML feeds and they usually include a cost per click fee. Yahoo's new program is a hybrid of theses past programs)

Now that there is a cost per click fee we have to be concerned with controlling those costs. Overture offers the same three options for managing costs as it does for it's Pay-per-click search engine ad programs:

• Non-Stop Traffic Plan: You authorize Overture to charge your credit card for a specified amount when you have 3-5 days of funds left in your account. This ensures uninterrupted service with minimum account management.
• Fixed Budget Plan: You set a monthly budget that you want to spend on Overture Site Match. When your balance reaches $0.00, your URLs will be deactivated until the next month (Unless you add more money to the account)
• Pre-Payment Plan: You choose when to add money to your account, giving you flexibility over the amount and timing of the deposits into your account. When your account is depleted, your URL's will be deactivated until you add more money to your account.

When should you use Yahoo's (Overture) Site Match to submit pages from your site? We suggest only paying to submit pages if the pages have not been found within a reasonable time or if the contents of the page(s) change frequently and you want these changes picked up quickly. Pages submitted via Site Match are refreshed (re-crawled) every 48 hours (this a great way to test the results of optimization done on a page; make a change, see it's effect within 3 days).

AltaVista and All-the-web (FAST), both owned by Yahoo will begin using the new Yahoo index and no longer keep separate databases. In fact, All-the web has already started the switch over. The two search sites will remain open, at least for the foreseeable future, but the results will come for the Yahoo search index.

MSN to roll out its search engine soon?
As we've mentioned, before MSN Search has been developing its own search engine for some time now. Currently most of the search results come from the Inktomi search engine database that is owned by Yahoo.

Recently MSN announced that they would be releasing a redesign of MSN Search on July 1st which many people assumed meant they would be bringing out the new search engine at that time.

However, according to MSN, the changes to be implemented in July are mainly cosmetic changes needed to comply with FTC recommendations regarding the labeled of Pay-per-click ads.

Previously MSN has stated that it may not release the new MSN search engine until 2005.

George Aspland
eVision, LLC
Branford, CT