SPARTAN is a lightweight (“spartan”) HTTP client written entirely in the Java programming language. It is neither a web crawler, a bot, nor a web spider.

If you are visiting this page, you likely noticed the SPARTAN User-Agent string in your web server logs:

Mozilla/5.0 (compatible; SPARTAN/0.1; +http://prokaryote.info/spartan)

SPARTAN was developed in 2008 as a small but necessary software component for a larger joint research project with Michigan State University. It provides persistent access to content that is linked over GUIDs and supports the Guide browser-based knowledge service and the Scribe Microsoft Word document annotation service, as well as several knowledge base search services.

It is used by prokaryote.INFO to retrieve remote resources from public databases on the web and to provide persistent resource linking.

This client is designed to respect the content retrieval policies of each web site or FTP repository it accesses. For large data sets, the client is rate-limited to a maximum of one download per day, and caches resources locally. Where MD5 hashes are available, this client checks the cached copy and the remote hash in order to determine whether the remote resource has changed.

For smaller downloads on individual resources or API requests, SPARTAN uses an intelligent caching and throttling mechanism to prevent redundant requests and repetitive downloads.

Please note that the new User-Agent for SPARTAN is:

Mozilla/5.0 (compatible; SPARTAN/0.1; +http://spartan-project.info)