Condor - High Throughput Computing

DiskRouter

DiskRouter started as a project to look at network buffering to aid in wide-area data movement. In its present form, it is a piece of software that can be used to dynamically construct an application-level overlay network. It uses hierarchical buffering using memory and disk at the overlay nodes to help in wide-area data movement. DiskRouter provides a rich functionality. Application-level multicast, running computation on data streams and using higher-level knowledge for data-movement are some of the currently supported features.

DiskRouter has been deployed in a number of sites and has aided in tens of terabytes of wide-area transfers.

DiskRouter is a research project by George Kola , Miron Livny and other members of the Condor Team at University of Wisconsin-Madison.

Publications/Talks

  • George Kola and Miron Livny, "DiskRouter: A Flexible Infrastructure for High Performance Large Scale Data Transfers", Submitted for publication, Apr 2004. [PDF]

  • Talk - Condor Week, April 2004 - "Reliable and Efficient Grid Data Placement using Stork and DiskRouter" [Powerpoint]

  • Talk - Condor Week, April 2004 - "Data Pipelines: Real Life Fully Automated Fault-tolerant Data Movement and Processing" [Powerpoint]
  • Talk - Condor Week, May 2003 - "DiskRouter: A mechanism for high performance large scale data transfers" [PowerPoint] [PDF] [PS]



    condor-admin@cs.wisc.edu