DIRAC Site Director: Analysis and Performance Evaluation
Description
To take advantage of the computing power offered by grid and opportunistic resources, developers of the DIRAC Workload Management System have adopted the Pilot-Job paradigm and deliver tools to deploy Pilot-Jobs on available resources. In this work, we study the DIRAC Site Director, the mechanism designed to supply grid resources with Pilot-Jobs, making use of the so-called grid push model that prevents to fully exploit the grid resources. Data focus on the analysis of the capabilities and limitations of the DIRAC Site Director in the LHCb production environment. Data mainly include CSV and JSON files related to the jobs and pilot-jobs submitted and processed, as well as a Jupyter Notebook to present plots based on data. Data show the evolution of the Site Directors through time highlighting the benefits of the different updates that seem to increase the number of pilot-jobs submitted as well as the number of jobs processed.
Files
Steps to reproduce
- Raw data and transformed data come from the LHCbDIRAC production server, web interface and command-line interface: more details about data collection are available in `/resources/<component>/README.md`. - `DIRAC Site Director - Toward an efficient Pilot-Jobprovisioning on Grid Resources.ipynb` is the main file: it gathers data from `resources` and contains the source code to generate plots in `results/plots`. You need to have installed Python3 along with jupyter notebook, matplotlib, seaborn, numpy and pandas to run it properly: ``` jupyter notebook 'DIRAC Site Director - Toward an efficient Pilot-Jobprovisioning on Grid Resources.ipynb' ```