ICWS Talk 2: Application of Management Frameworks to Manage Workflow-Based Systems: A Case Study on a Large Scale E-science Project

This is the second talk, and it discusses integration of Hasthi with LEAD Project, a Large Scale E-Science cyber-infrastructure.

Citation: Srinath Perera, Suresh Marru, Thilina Gunarathne, Dennis Gannon, Beth Plale, “Application of Management Frameworks to Manage Workflow-Based Systems: A Case Study on a Large Scale E-science Project,” icws, pp.519-526, 2009 IEEE International Conference on Web Services, 2009

DOI: http://doi.ieeecomputersociety.org/10.1109/ICWS.2009.52

Abstract: Management architectures are well discussed in the literature, but their application in real life settings has not been as well covered. Automatic management of a system involves many more complexities than closing the control-loop by reacting to sensor data and executing corrective actions. In this paper, we discuss those complexities and propose solutions to those problems on top of Hasthi management framework, where Hasthi is a robust, scalable, and distributed management framework that enables users to manage a system by enforcing management logic authored by users themselves. Furthermore, we present in detail a real life case study, which uses Hasthi to manage a large, SOA based, E-Science Cyberinfrastructure.

Slides:

ICWS Talk 1: Enforcing User-defined Management Logic in Large Scale Systems

Early July I presented two papers at ICWS, here is the first one. This is the base of my thesis.

Citation: Srinath Perera, Dennis Gannon, “Enforcing User-Defined Management Logic in Large Scale Systems,” services, pp.243-250, 2009 Congress on Services – I, 2009

DOI: http://doi.ieeecomputersociety.org/10.1109/ICWS.2009.52

Abstract: The ubiquity of information technology, technological advances, and utility computing trends have motivated largescale systems, but managing and sustaining these systems is far from trivial. Automatic or semi-automatic monitoring and control are a potential solution to this problem. However, since management scenarios differ from system to system, a generic management framework that can manage a wide variety of systems should support user-defined management logic. This paper proposes a novel architecture that can manage large-scale systems according to user specified management logic that depends on both global and local assertions of the managed system. Furthermore, the paper demonstrates that despite having a global view of the managed system, a management framework can scale to manage most real world usecases.

Slides:

Dissertation: Enforcing User-Defined Management Logic in Large Scale Systems

As I promised before, following is the information related to my Dissertation and Defence (At Indiana University).

Title: Enforcing User-Defined Management Logic in Large Scale Systems
Year: 2009
Primary Advisor: Prof. Dennis B. Gannon
Thesis Committee: Prof. Dennis B. Gannon (Chair), Prof. Geoffrey C. Fox, , Prof. Beth A. Plale, Prof. David B. Leake, Dr. Sanjiva Weerawarana

Thesis can be found through ProQuest here (I paid and enabled open access, and therefore, you do not need an account to download it). Also there are two papers.

  1. Srinath Perera, Dennis Gannon, “Enforcing User-Defined Management Logic in Large Scale Systems,” services, pp.243-250, 2009 Congress on Services – I, 2009
  2. Srinath Perera, Suresh Marru, Thilina Gunarathne, Dennis Gannon, Beth Plale, “Application of Management Frameworks to Manage Workflow-Based Systems: A Case Study on a Large Scale E-science Project,” icws, pp.519-526, 2009 IEEE International Conference on Web Services, 2009

Slides used for the presentation can be found below.

Abstract:
Due to advances in distributed systems, social motivations, and economic motivations, scales of systems are on the rise. In large-scale systems, changes–caused by failures, maintenance, and additions–are a norm rather than an exception, and therefore, manually keeping these systems running is difficult, if not impossible. System management, which monitors and controls systems, is a prominent solution to this problem.
However, management usecases differ from system to system, yet developing a specific management framework for each system defeats the purpose of building system management frameworks in the first place. Management frameworks that enforce management logic authored by users provide a solution for this problem. These frameworks enable users to change framework’s decision logic to cater for user’s specific requirements, and after deployed, they monitor and control target systems in accordance to the user-defined management logic. If these logic assert only a single component of the system, we call them local logic, and if these logic assert multiple components in the system, we call them global logic. The global logic depend on a global view about a system, which is non-trivial to support in large-scale systems. However, they enable users to reason about the target system explicitly and, therefore, provide a natural way to express management usecases.
This dissertation presents a new, dynamic, and robust management architecture that manages large-scale systems by enforcing user-defined management logic that depend on a global view of the managed system. Using empirical analysis, we have shown that it scales to manage 100,000 resources, which demonstrates that the architecture can manage most practical systems. This is a testament that despite its dependency on a global view of the managed system, a system management framework can manage systems in accordance to user-defined management logic and can still scale to manage most real world systems. Furthermore, we have demonstrated that the architecture is robust in the face of failures and stable with respect to different operational conditions.