This data repository contains input and output files for large-scale experiments conducted on the three German national supercomputers HAWK, SuperMUC-NG, and JUWELS.
More structure can be seen when switching to "Tree" view below. Then, the files are structured into three folders (io-benchmark, two-systems,three-systems`), and the outputs were generated with the following software versions:
IO-Benchmark: DisCoTec version e8fad8ae, inputs and outputs both compressed into the same file. Includes file preparation scripts and darshan outputs. The darshan files are also uploaded to https://hpcioanalysis.zdv.uni-mainz.de/traces (search DisCoTec in "Application" or HAWK in "Cluster), where they can be explored interactively.
Two-system experiment: DisCoTec version e81a10589, UFTP copy scripts (in the DisCoTec repository) version 125ce10a5.
Three-system experiment: DisCoTec version 46127fe94c1, UFTP copy scripts (in the DisCoTec repository) version 58d1b9ba9.
The input files were generated with https://github.com/SGpp/DisCoTec-combischeme-utilities version 88d11fb0, exact commands can be seen from the scrips in this repository.
After the combination scheme is assigned to systems, the conjoint file sizes need to be generated as detailed in https://github.com/SGpp/DisCoTec/blob/main/tools/subspace_writer/README.md .
The resulting scheme*json and *conjoint.sizes files are part of the input generator folders uploaded here.
The measurements for the I/O benchmark were taken on October 6 2023.
The measurements for the two-system setup were taken on September 4 2023 between 14:30 and 15:30.
The measurements for the three-system setup were taken on November 8 2023 between 13:30 and 15:00.
Raw data are uploaded as .tar.gz files in the respective folder.
DisCoTec, e81a10589 (first case study) and 46127fe94c1 (second case study)
; Van Craen, Alexander