Analyzing Search Engines
Uploaded by Ethan on Oct 20, 2000
1. Formulate five criteria for the evaluation of search engines
To effectively evaluate three different search engines from the perspective of an advanced web user, the following criteria were established:
1. Relevance and accuracy of search results
2. Search speed
3. Advanced search options
4. Other services
5. Site design and layout
More information about these criteria and how they were used is available below.
2. Test three search engines against your criteria
For this evaluation, the following three search engines were tested:
1. AltaVista - http://www.altavista.com/
2. Google - http://www.google.com/
3. DirectHit - http://www.directhit.com/
3. Explain your methodology
The criteria outlined in Part 1 were reached by considering the things that would be important to an advanced user. Of course, factors like Relevance and Speed are critical to such a user (particularly in a work environment). The search options criteria is particularly important to advanced users who are after a specific page and do not want to waste time 'wading' through results from a 'standard' search. The other criteria are important to most users and mean that the search engines are more usable and useful.
The search results from the three test search engines (Part 2) were evaluate according the criteria, explained it more detail below.
Criteria 1 - Relevance and Accuracy of ResultsThis criterion was to evaluate the usefulness of a particular search engine according to the accuracy of its results. I defined the relevance, according to personal standards, after browsing the pages in the results, for three separate search terms. The Relevance Percentage is calculated by counting the number of relevant pages that appear in the first 20 search 'hits' and then multiplying by 20 (for a percentage).
Criteria 2 - Search SpeedThis criterion was to evaluate which of the test search engines returns results quicker.
However, it is important to note that the speed test is both a test of the Internet connection to the remote machine, as well as the time taken by the search engine to perform the search. Furthermore, tests of Internet speeds are very variable based on time of day, other internet traffic etc and are therefore of only limited accuracy. For this reason, no 'hard numbers' were calculated (ie timed searches). Instead, I rated each search engine based on 5 searches per engine at the same time, same day. The possible Speed Ratings are as follows: Excellent, Good, Poor
Criteria 3 - Search OptionsThis criteria was to evaluate the search engines advanced search options. This is...