Research On Parallel Application System For Triplet-based Architecture

The age of Big Data open the door to a new approach in data exploration and utilization.Stream processing is a promising alternative candidate for programming in multi-core systems for exposing distributed parallelization.Scheduling task and data parallelism properly in stream programming is a challenging problem in the face of varying application characteristics.Meanwhile,the stream data coming from real-time and high-throughput applications are not only continuous but also dynamic.It motivates the need for a more intelligent approach to establish a parallel application system.We present an asynchronous concurrency stream programming framework which offers a novel parallel programming model for scheduling task and data parallelism in the message driven execution paradigm.A key property of this framework is exposing controlled-grained parallelism,which allow us to controll the granularity of task and data parallelism in stream graph.By incorporating self-awareness loop into message-driven based stream programming model,the framework provides dynamic and reconfigurable stream graph in adapting to the data flow changes.It redistributes the data streams and computing resources to implement adaptive load balancing in stream programs.The main contributions of this paper are as follows:(1)We present a stream programming model for scheduling task and data parallelism based on the message passing paradigm.The programming model decomposes the stream computation procedure into two main processes: the divergence process and the convergence process.To control the hazards associated with communication and synchronization in stream programs,Effect Deterministic Communication(EDC)technique is proposed for dynamicly adjusting the stream graph and reconnecting the corresponding channels during the execution.To reduce the costs of buffering and latency associated with excessive data parallelism,the framework offers an effective approach to schedule the granularity of data parallelism.It uses a data parallelism paradigm to parse the integrated message and allocates data blocks through message-based virtual data channels.Controlled-Grained Parallelism is proposed for exploiting parallelism in an appropriate granularity suitable for multicore architectures.The approach bridges the gap between the parallel scale and the architecture of stream programs.(2)Self-Aware Parallelism is proposed to automatically trace data stream changes and take appropriate actions in stream programs.Contrast to the synchronous stream programming model,the message-driven self-aware stream programming mechanism(StreamAware)allow applications to adjust itself for changes during its operation.It implements a self-awareness loop based on Asynchronous Finite State Machine(AFSM)to respond to the changes of data stream.The AFSM represents the entire state space of the self-awareness loop and describes the state transition rule.It enables the stream programs to dynamicly schedule task and data parallelism in the message driven execution paradigm.(3)We propose a self-aware load balancing strategy(SALB)which enables stream programs to trace data stream changes and balance load by adjusting the stream graph and reconnecting the corresponding message channels during the execution.To reduce the cost of latency associated with unbalanced subtree,Adaptive Bandwidth Oriented Message Channel is proposed to adjust the virtual bandwidth of data flow branch.To balance load in limited computing resources,SALB offers an effective approach for reusing the node resources by changing the functions and links of associated nodes to update the stream graph.