Saturday, May 1, 2004

Lookout, there’s incredibly fast mail searching ahead

OUTLOOKPOWER PRODUCT OF THE MONTH

By David Gewirtz

It's rare that we see 5-star products, especially when it comes to email-related products. Email is such a study in compromises that even when developers pull out all the stops and make superhuman efforts, the resulting product is rarely 5-star material.

That's why we were so pleasantly surprised to come upon Lookout, a product that substantially exceeded our expectations. Lookout is OutlookPower Magazine's Product of the Month and our first 5-star winner.

If you've spent any time using Outlook (whether it's Outlook XP or 2003), you know the find function is decidely sub-par. My main Outlook PST file is nearing the 2 gigabyte limit, and I have a number of ancillary PST files for archives and such. Whenever I need to search for a message or old correspondence, I shudder.

In fact, I recently had to pull together all the correspondence on a deal we were working on, and because I needed to find all the historical correspondence dating back a year or so, I started up the find process before going to bed, knowing that I wouldn't see a list of all the matches for hours.

Lookout ends all that. The Lookout toolbar, shown in Figure A, is a replacement for the Outlook find subsystem that makes performing searches on your Outlook email a breeze.

FIGURE A

Just type your search into the toolbar. (click for larger image)

Blazingly fast index creation

It always baffled me why Outlook didn't index email messages in such as way as to make doing finds efficient. I was sure that this oversight would be fixed in Outlook 2003. Of course, it wasn't. Lookout, however, does just that. It indexes all the messages (actually all the data) in your Outlook PST files.

Indexing is important. Without an index, if you wanted to find the string "Product of the Month" in your message archive, Outlook would have to open every message and message-by-message, paragraph-by-paragraph, character-by-character, compare every single byte in every message to the search string. This, as you well know, takes a very long time.

Indexing is simple, at least in concept. It's often tremendously hard to implement efficiently. When data is indexed, instead of iteratively searching every single message, a single lookup is made into the index database, and the matching messages are returned, nearly immediately. This is how nearly all modern database systems work, and Lookout applies this mechanism to Outlook.

What shocked me about Lookout was how fast the original index was to create. I've seen databases and other large text stores index before, and a two gigabyte data store, even on a machine as fast as mine, normally would take hours and hours to generate. Lookout indexed my main 2GB PST file, as well as my supporting archive files (totally another 4GB), in just about 20 minutes.