Orchestrating Data-Centric Workflows

Adam Barker, Jon B. Weissman, Jano van Hemert

Research output: Chapter in Book/Report/Conference proceedingConference contribution

29 Citations (Scopus)

Abstract

When orchestrating data-centric workflows as are commonly found in the sciences, centralised servers can become a bottleneck to the performance of a workflow; output from service invocations are normally transferred via a centralised orchestration engine, when they should be passed directly to where they are needed at the next service in the workflow. To address this performance bottleneck, this paper presents a lightweight hybrid workflow architecture and concrete API, based on a centralised control flow, distributed data flow model. Our architecture maintains the robustness and simplicity of centralised orchestration, but facilitates choreography by allowing services to exchange data directly with one another, reducing data that needs to be transferred through a centralised server. Furthermore our architecture is standards compliment, flexible and is a non-disruptive solution; service definitions do not have to be altered prior to enactment.

Original languageEnglish
Title of host publicationCCGRID 2008: EIGHTH IEEE INTERNATIONAL SYMPOSIUM ON CLUSTER COMPUTING AND THE GRID, VOLS 1 AND 2, PROCEEDINGS
Place of PublicationLOS ALAMITOS
PublisherIEEE COMPUTER SOC
Pages210-217
Number of pages8
ISBN (Print)978-0-7695-3156-4
DOIs
Publication statusPublished - 2008

Keywords

  • Systems architecture
  • workflow optimisation
  • Web services
  • decentralised orchestration

Fingerprint

Dive into the research topics of 'Orchestrating Data-Centric Workflows'. Together they form a unique fingerprint.

Cite this