Once your site has built up a reasonable body of content, it is usually a good idea to add a search function to the site, to make it easier for your visitors to find the information they need. This is, after all, the primary purpose of putting the information on the site in the first place.

We do not usually add search to a new site. Many sites will never need a search function – their body of content will remain small enough that, assuming the information is organised clearly, a search function is not required. When a site launches it almost never has enough content to warrant a search, but if the site is grown and developed, the search will become necessary quite soon.

At this point you might be expecting me to explain the difficulties and complexities of building a search engine for your site, but I’m afraid I’m about to disappoint you. Adding search to a site is terribly easy – we use Google.

Google have spent hundreds of millions of dollars over decades on some of the brightest minds in the world to develop a search engine more powerful and relevant than anything else available. Nothing else even comes close. Certainly no search engine I could write with my limited budget and time could even begin to rival the power of Google.

Google’s Custom Search will add the full power of the Google search engine to a search of just your site. Both the search bar and the results will be hosted on your site itself. And, it’s all free. Free, of course, never comes without a cost, and in this case the search bar and the search results will be Google branded, and the search results will contain Google adverts (for which you do not get paid). This product is recommended for much smaller sites, with limited budgets, and whose primary goal is not brand promotion.

Google’s Site Search product, however, is not free. For a nominal fee, you get optional Google branding and no adverts. You also get the option to completely control the display of your results. The fee starts at $100 per year for up to 20000 searches, which should easily cover most search needs. It is certainly a lot less than I would charge to develop and implement a search engine for you.

There are some issues you should be aware of when using Google search for your site. The most obvious is the delay between when you update the site, and when those updates are available in the search. Google doesn’t search your site directly. It searches its index of your site, which it updates regularly. It calculates how often to index your site based on how often the content of your site changes – if you update your site often, Google will re-index your site every couple of days. This means that it may take a couple of days before your new content can be searched, and that the old information is still available for those few days.

In most cases this is not a problem, but in some cases, it may be critical to ensure that the information in the index is accurate. Google provides On-Demand indexing to cater for this. Each search account is allocated a limited number of pages per year that can be indexed immediately. For the $100 account, this number is 500. So, if you update fewer than 2 pages a day, you should be able to make sure that your site is always 100% updated almost immediately. We haven’t actually done this yet, but it should be possible to extend the Black Square CMS to do these notifications automatically.

Still, if you update your site often, and it is critical that the changes are searchable immediately, you may need a custom built search engine for your site.

The other major problem is that Google will only search the pages of your site, and then only the pages it can get to. If content on your site is hidden behind a login, or you use clever AJAX loading tricks to populate pages with content according to the users’ actions, Google cannot index that content, and it cannot be made searchable. Alternatively, if the information you want searched is stored in a database, and there isn’t a correlation between the pieces of information and pages (ie: each piece of information has its own page), the Google’s search model will probably not work for you.

In these cases, you’ll need a custom search engine built specifically to cater to your individual needs. Custom built search engines will never be able to perform searches with anywhere near the accuracy or relevance that Google can, but they will usually do the job well enough to provide a useful service to your visitors. We’ll discuss how a custom search engine is put together in another post.