WebAnts(tm) Progress Report -- March 1995


The main work done during March was on the prototype indexers, following the master-slave interaction model described in the February report. The following results have been achieved so far:

Protocol Extension

An additional request type has been added to the protocol: UNCLAIM. After a correspond-ant has successfully claimed a document, it may still be unable to explore it for some reason (i.e. document no longer exists, the site is down, etc.). Under these circumstances, the correspond-ants should sent an UNCLAIM request to their consult-ant:

An unclaim request takes the form: "UNCLAIM url", where url is fully specified. This request says "I could not explore this document" A master may only response with:

Test Run

A preliminary test run has been made, running several correspond-ants (between 2 and 5 at different times) on different machines here at CMT, and I am happy to report that the cooperation model works! Unfortunately, the consult-ant does get bogged down at times when fetching robots.txt files (although this diminishes with time as more of them are cached locally). More on this next month when I've been able to tune them more carefully and follow their progress better.


I will be attending the Third International WWW Conference in Darmstadt, Germany in April to participate in the workshop on indexing the web.

I have received interest from Johnny Irons at the University of Aukland about running an indexer for New Zealand. I am exploring ideas now for having territories to facilitate this sort of arrangement.

I have corresponded briefly with Darrell Woelk, head of the Infosleuth project at MCC, about ways in which out projects could collaborate. I am hopeful that an overlap of interest will exist, as such collaboration would be very beneficial.

I corresponded briefly with Jeremy D. Zawodny, a CS student at Bowling Green State University in Ohio. Jeremy was interested in the possibility of doing some independent study work relating to WebAnts. I am looking forward to hearing back from him.

Last updated 07-Apr-95 by John Leavitt (jrrl@cmu.edu)