Lesson #14 - Where do visitors come from? Search engines.

We have talked about several different sources of visitors in previous lessons: Resonance, Links and Advertising. In this lesson we will talk about the most powerful form of free traffic on a long-term basis: Search engines.

If you are creating a typical content web site, perhaps half of your daily traffic will be coming from search engines. In other words, if you have a content web site, about half of your visitors will arrive from search engines each day. They will go to a search engine, type in some sort of query, and a page from your web site will come up in the results.

[For completeness, let's look at the whole traffic picture for a second. If half of a typical content web site's traffic comes from search engines, then perhaps a quarter of its traffic will come from links. The final quarter will come from "other" -- people who type in the URL by hand, people who see the URL in a magazine and type it in, people who get an email from a friend about your site and click on your URL in the email, people who habitually return to your site daily, and so on. On any given day, these ratios might change a bit. For example, if you get linked by Fark.com of Digg.com one day, a disproportionately large amount of your traffic will come from links that day. But over time, on average, the 50/25/25 rule or something close to it will hold for "normal" content web sites. If you are running a blog, your ratios will be different -- you may get a tremendous percentage of your visitors from daily returning traffic. If you are running a retail web site or a business brochure site, a lot of your traffic may come from advertising. See Lesson #2 - The different types of web sites for a discussion of the different types of sites.]


[Half or more of the traffic that a "typical" content web site gets will come from search engines. Three-quarters of this search engine traffic will come from Google. The rest will come from all other search engines. Other sources of traffic include links from other sites, miscellaneous traffic and daily returning traffic.]

When we talk about "search engines" on today's web, we can simplify things by talking strictly about Google. This may change in the future, but at least for now, Google is king. If half of your site's traffic is coming from search engines, then that normally means that half of your search engine traffic (25% of your total traffic) comes from google.com. And then there are all of the Google international sites like Google.co.uk, Google.it, etc. -- they make up a quarter of your search engine traffic. In other words, three-quarters of your search engine traffic will come from Google. Yahoo search, MSN search, AOL search, Ask.com and all the other little search engines like Dogpile, etc. together make up only 25% of your search engine traffic.

Given that this is the way the world works, you can immediately see three things:

1. If you have a content web site of any kind, you want to make sure that your site is in Google.
   
   
2. You would like to have as many pages as possible in Google. The more pages you have, the more likely you are to come up in Google's search results.
   
   
3. You would like to come up as high as possible in Google's search results. The higher you are in the results, the better chance you have of getting clicked on by a visitor.

Point #1 is easy. You would probably have to go out of your way these days to NOT be in Google. Google's spiders are very thorough, and if ANYONE links to your site, your site will be discovered and listed in Google's index. As mentioned in the lesson on links, It usually takes a week or two for Google to discover your site once it gets linked by another site, and then might take a week or two after that for Google to plow into the site and find all of its pages.

Point #2 talks about what I call the "Google Footprint" of your site. The bigger your Google Footprint, the more visitors you will have coming to your site. You can see the effect of footprint size by looking at some of the largest sites on the Web. For example, look at Wikipedia.org. It has 23 million pages indexed by Google. The way you find that is by doing this:

• Go to Google.com
  
• Select the "Advanced search" option
  
• Type wikipedia.org into the "Domain" field
  
• Click the "Google Search" button

Google reveals that it has indexed about 23,600,000 pages from wikipedia.org. Now go look up the Alexa rank of Wikipedia.org and today you find that Wikipedia's rank is 64. Here are some other examples of this phenomenon:

• About.com has an Alexa rank of 67 today and about 7,530,000 pages in its Google footprint.
  
• CNN.com has an Alexa rank of 28 and about 1.74 million pages in its Google footprint.
  
• Microsoft.com has an Alexa rank of 11 and 13.9 million pages in its footprint.
  
• And so on...

There are exceptions, and the exceptions are educational. Drudgereport.com, for example, has only 439 pages in its footprint yet its Alexa rank is 262 today. That's because almost all of Drudge's traffic comes from daily returning traffic, not from its Footprint. Wikipedia, on the other hand, gets a huge portion of its traffic from its Footprint. If there are 23,000,000 Wikipedia pages in Google and each one yields just one visitor a day, that's 23 million visitors per day for Wikipedia. CNN's traffic is a combination of its large footprint and its daily returning traffic.

From your perspective, anything that you can do to increase the number of pages in your site will increase the size of your Google footprint. In general, the larger your footprint, the more visitors you can get.

Point #3 talks about your site's placement in the Google search results for a given search term. This is also known as the "rank" of your pages. If your pages are ranked highly by Google, then your pages will come up higher in a page of Google search results than a page with lower rank.

For example, let's say that you type "robotic" as a search term into Google. Google has 7,100,000 pages to choose from. In other words, the Google spiders have gone out and looked at billions of pages, and 7,100,000 contain the word "robotic". When Google shows a page of search results, it has to pick which of those 7,100,000 pages it is going to place first, second, third and so on. How does Google pick?


[Click for larger image]
How does Google pick which page will go first, second, third, etc. in a page of search results like this? Read below for details.

No one really knows for sure how Google picks because it is a closely guarded Google secret, but it is believed that:

• Google will pick a page that has the word "robotic" in the title.
• Google will pick a page that has the word "robotic" placed throughout the body of the article.
• Google will choose a page that links out to other robotic pages.
• Most important of all, Google will pick a page that lots of other sites have linked to using the word "robotic" in the link.

That last one really is important. The more outside pages that link to your site, the more "important" your site is perceived to be by Google. Part of this "importance" is crystallized in a number called the PageRank for the page. You can see the PageRank if you download the Google Toolbar -- it will show you the PageRank for every page that you visit. For example, the PageRank for WebKEW right now is 6 as shown in the screen shot below:


[Click for larger image]

PageRank ranges from zero to 10. When you first create your page, its PageRank will be zero. If a lot of sites link to your page, then over the course of several months your PageRank will rise. If sites with a high PageRank link to your site, then your PageRank will be higher. If a group of sites with a PageRank of zero link to your site, or if no sites link to your site, then your PageRank will be low.

As a general rule, the page with the most external links wins when it comes to the order of a Google search result page.

PageRank, and external links from high-PageRank sites, are important. There is an entire industry that has built up around helping pages to raise their position in the Google search result pages. This industry is called the Search Engine Optimization (SEO) industry. If you search in Google for search terms like PageRank and search engine optimization you will find thousands of sites offering you information, tutorials, forums and services. In general, however, it is my belief that there are no shortcuts -- if you want your pages show up at the top of the search results for a certain keyword, having lots of external sites linking to your site is key. External links are the only thing that matters significantly to Google's algorithms.

What does this mean? Here are some observations:

• It means that if you are creating a new Web site, you need to contact every friend and relative you can think of who has a Web site and ask them to link to your site. Every little bit helps.
  
  
• It means that once you have created one Web site that has a decent PageRank, you have something of value. You will be able to link to other sites that you create to give them PageRank.
  
  
• It means that you will often see sites offering to sell PageRank, and now you will understand why they do it.
  
  
• It means that you will look at popular blogs like dailykos.com, and in a sidebar you will see something like "Blogroll" or "Friends", and there will be 50 links to other sites. This is a way of sharing PageRank. These sites link back and forth to each other in order to increase the PageRank of the group (among other benefits).
  
  
• It means that if you see a web site that you like, and you want to help the person who created that web site to succeed, you should link to that web site.

Obviously there is a lot to learn about search engines as a traffic driver. We will continue this discussion in the weeks and months to come. If you have specific questions, please email them to me or insert them in the comments below.