In earlier postings here and here, I’ve written about SETI@home and the D2OL avian-flu project, which take advantage of the idle CPU cycles on desktop and laptop computers around the world, for interesting and potentially important research. Here’s another interesting application: what if we could use distributed computing to pool our idle CPU cycles to create a free, grass-roots alternative to an emerging monopoly in the business world — a monopoly whose recent behavior suggests that it may someday succumb to the Dark Side of the Force, a la Annakin Skywalker’s transformation into Darth Vader?

The “emerging monopoly” I have in mind is Google, whose dominance of the search-engine industry dwarfs that of Yahoo and Microsoft. Obviously, Yahoo and Microsoft haven’t given up, and there are other search-engine competitors out there, too. But I doubt if an of us would be surprised to wake up five or ten years from now, and find that Google’s dominance of the search industry is roughly equivalent to Microsoft’s dominance of the desktop operating system industry. And just as many people are firmly convinced that Microsoft is the Evil Empire, there are more and more people beginning to worry that Google’s original motto of “do no evil” is slowly being pushed aside as it moves into new markets like China. If Google does fall prey to the Dark Side, millions of us may find that our ability to freely discover information on the Internet has been severely curtailed.

So, what’s the alternative? Well, what if a bunch of people pooled their available computing resources and created a “grass roots” search engine? It turns out that such a project is already underway; it’s based in England, and it’s known as “Majestic-12“. (The name, as Wikipedia informs us here, is taken from the code name of an alleged secret committee of high-level scientists, military leaders, and government officials, supposedly formed in 1947 at the direction of Harry Truman, to investigate UFO activity in the aftermath of the Roswell UFO incident.) According to a March 2006 article in the Guardian (“If distributed processing can search for aliens, why not web pages?“), the project currently consists of about 60 volunteers, whose computers are “crawling” through roughly 50 million Web pages a day; when I checked the Majestic-12 home page on April 23, 2006, the group had grown to 83 volunteers. The project has now indexed roughly 2.8 billion Web pages, or roughly a third of the 8 billion pages Google is thought to have indexed.

It remains to be seen whether Majestic-12 will achieve the “critical mass” needed to really compete with Google. The project’s creator, Alex Chudnovsky, estimates that “fewer than 10,000 people taking part in this project can recrawl the whole of Google’s database every single day.” Keep in mind that SETI@home was launched in 1999 and now has 440,000 volunteers; and the more recent D2OL avian-flu project has 80,000 volunteers in 93 countries. So expanding the Majestic-12 volunteer community to 10,000 people is not as crazy as it sounds.

Whether it actually happens remains to be seen; at this point, I’m only mildly concerned about Google’s tolerance of censorship in China, and I don’t feel sufficiently outraged to volunteer my idle CPU cycles to Majestic-12 (though one of the more practical reasons is that Majestic-12’s software only runs on Windows, Linux, and freeBSD; if a MacOS version is released, I’ll reconsider). In the meantime, I’m more interested in the concept than the specific manifestation; as an open, grass-roots project, Majestic-12 is philosophically compatible with Wikipedia — and I think that represents a fundamental shift in the way society collects, organizes, and disseminates information.