Expectations for indexing on search engines and Search Engine Optimisation (SEO) progress

Download a pdf of this article.

Creating a new website or a new online business has parallels with opening a traditional bricks and mortar business.  Large amounts of time and energy are required to run a successful business and a disproportionate amount of time and energy is required in the opening stages.  Nothing ever seems to happen quickly enough; and it is only in time that people and customers seem to know about your business and value its worth.

It is worth considering how the search engines are working and 'thinking' to understand how long it takes to be indexed in a search engine listing and then to see results from your efforts at search engine optimisation (SEO).

A standard website on a new domain name might easily take up to two or three weeks to begin to be initially indexed by Google and the other search engines, even if it is properly structured and sensibly organised.  If there are a few links from other websites or blogs, this process can be quicker and it certainly helps to have installed Google Webmaster Tools to let Google know some things, for example where your target market lies.

After that, it is likely that Google will begin to "crawl" your website with its "spider" a few pages at a time each day.  Google doesn't want to hog your bandwidth and there are a few billion other web pages to worry about, so it doesn't do this all at once.  There are things a good SEO and designer can do to help, such as ensuring your pages load as fast as possible but patience is necessary.  If Google crawls 3-4 new pages every time it visits your new site, it is going to take a month to cover even a small new online shop of, say, 100 products.  Even then, it has probably only read a little bit of each page the first time round.

Once Google has read the information from your website, it sends that data back to its servers to be processed and indexed.   Then it needs to share that information with its other data centres around the World and update what it knows all the time.  Google probably has 300-400 data centres in the world, none of which has exactly the same information at any one time. 

Quite apart from the sensible SEO work you can do to build and promote your website properly, it is only in time that your web pages will begin to appear in the index (let alone the top rankings) for what you perceive to be the correct search results.

For example, after at least several weeks, or more likely months (according to the competition), a web page focusing on the keywords "wedding fascinators" may just rank well for people typing that exact expression only.  Later in the progress, the web page may appear in ranking results if people are typing in a phrase like "red and blue wedding fascinators".  Only much later is your web page likely to succeed in what is known as a broad match search, for example "fascinators for a wedding or bride".  Exact, phrase and broad are specific concepts used by Google for search terms.  It does takes time to achieve broad match results, which have more traffic, even if you are doing everything right. 

This is natural. As with a bricks and mortar business people need time to know and trust a new business.  To a certain extent this is exactly what Google is trying to mimic and so the age of a domain can have an impact on rankings where all other things are equal.

The nub of this is that you must expect a new web business to build slowly over time. Getting indexed and being ranked, especially ranked well, takes a while to achieve and should be measured in months and years with careful monitoring by an expert, not measured in days or weeks.  This doesn't mean you should delay your SEO work.  Quite the reverse, start early and do a little bit every day or week.

One way to consider getting more traffic to a web site in early days is to use well managed Pay Per Click (PPC) advertising.  This can also be very useful to establish conversion rates and how useable your website really is when organic search engine traffic is low.  It can also help establish whether your chosen keywords are truly commercial and so focus your SEO efforts better.  After time PPC can be reduced as your SEO efforts begin to kick in effectively.

Steven Loveridge 
www.stevenloveridge.net