MPI support in the DIRAC Pilot Job Workload Management System

Parallel job execution in the grid environment using MPI technology presents a number of challenges for the sites providing this support. Multiple flavors of the MPI libraries, shared working directories required by certain applications, special settings for the batch systems make the MPI support difficult for the site managers. On the other hand the workload management systems with pilot jobs became ubiquitous although the support for the MPI applications in the pilot frameworks was not available. This support was recently added in the DIRAC Project in the context of the GISELA Latin American Grid. Special services for dynamic allocation of virtual computer pools on the grid sites were developed in order to deploy MPI rings corresponding to the requirements of the jobs in the central task queue of the DIRAC Workload Management systems. The required MPI software is installed automatically by the pilot agents using user space file system techniques. The same technique is used to emulate shared working directories for the parallel MPI processes. This makes it possible to execute MPI jobs even on the sites not supporting them officially. Reusing so constructed MPI rings for execution of a series of parallel jobs increases dramatically their efficiency and turnaround.

Data and Resources

Additional Info

Field Value
Source MPI Support in the DIRAC Pilot Job Workload Management System
Author Tsaregorodtsev, A., Hamar, V.
Maintainer CCSD
Last Updated May 16, 2026, 09:04 (UTC)
Created May 16, 2026, 09:04 (UTC)
Identifier in2p3-00703341
Language en
contributor Centre de Physique des Particules de Marseille (CPPM) ; Aix Marseille Université (AMU)-Institut National de Physique Nucléaire et de Physique des Particules du CNRS (IN2P3)-Centre National de la Recherche Scientifique (CNRS)
coverage New-York, United States
creator Tsaregorodtsev, A.
date 2012-05-21T00:00:00
harvest_object_id a17e6926-5094-4d04-82d6-0b0266a743c9
harvest_source_id 3374d638-d20b-4672-ba96-a23232d55657
harvest_source_title test moissonnage SELUNE
metadata_modified 2024-04-11T00:00:00
set_spec type:POSTER