PIPEL: modelo de gerência da elasticidade para aplicações organizadas em pipeline

Abstract

In the computing environment workflows has become a standard for many scientific experiments. Scientific workflows consist of several applications structured in an activity flow, where the output of one becomes the input of another. A pipeline application is a type of workflow that receives a set of tasks, which must pass through all stages of this application in a sequential manner, which can lead to a prohibitive execution time. Considering this problem, pipeline applications can benefit from the use of different resources for each stage, or performed in a distributed way. However, specific dependencies and distributed computing problems arise due to the interaction between the processing stages and the mass of data that must be processed. The input stream for applications that use pipeline standards can be intense, erratic or irregular. According to the task flow behavior some stages may have degraded performance, delaying subsequent stages and ultimately interfering in the application’s performance. An alternative to solve this is to allocate the maximum available resources (over-provisioning) in each application stage. However, this technique can generate a high infrastructure costs and the possibility that some resources remain idle in certain moments. Thus, the elasticity in cloud computing environments appears as an alternative, exploring the pay-as-you-go concept. In this context, we propose an elastic model based on the PaaS layer (Platform as a Service) cloud, named Pipel. This model allows pipeline applications to take advantage of the dynamic resource provisioning capabilities of cloud computing infrastructure. Pipel uses a reactive approach, using thresholds for elasticity decisions based on the CPU load of virtual machines in each application stage. Each stage has a load balancer (called stage controller) and a number of operating resources. The stage controller receives the tasks that the stage should run, allocates in a queue and then distribute the tasks in virtual machines available at each stage. According to established rules Pipel performs elasticity actions on the cloud environment. To validate this proposal we developed a prototype that has been tested in two different scenarios: (i) without elasticity and (ii) with elasticity. In each scenario we used four different processing loads: (i) Increasing; (ii) Decreasing; (iii) Constant and (iv) Oscillating. The results showed a reduction of 38% of the application’s execution time using the elasticity provided by Pipel.