Distributed memory machines consisting of multiple autonomous processors connected by a network are becoming commonplace. Unlike specialized machines like systolic arrays, such systems of autonomous processors provide virtual parallelism through standard message passing libraries {PVM or MPI). In the area of parallelizing existing numerical algorithms, two main approaches have been proposed: automatic parallelization techniques and explicit parallelization. In the present work, we focus our studies on the second approach. The parallelization paradigm found to be most effective for numerical algorithms on distributed memory machine was to provide the user with a clientjserver architecture. The most difficult part to design is the SPMD code which is initiated by a client process to speedup the computing time. To do this, our methodology aims: at reusing the systolic model principles for the display of the potential parallelism inside nested loops, and justifying the the aggregation of iteration of loops so as to reduce communication overheads while exploiting coarse-grained parallelism. Each aggregation is a bloc of fine-grained computations not located in the same hyperplan of a given space. It also defines an atomic unit of computation i.e no synchronization or communication is necessary during the execution of the fine-grained computations inside a bloc. Thus all necessary data must be available before such atomic executions. This imposes the constraint that splitting the set of fine-grained computations does not result in deadlocks.
