Under copyright Constraint(s) on Use: This work is protected by the U.S. Copyright Law (Title 17, U.S.C.). Use of this work beyond that allowed by "fair use" or any license applied to this work requires written permission of the copyright holder(s). Responsibility for obtaining permissions and any use and distribution of this work rests exclusively with the user and not the UC San Diego Library. Inquiries can be made to the UC San Diego Library program having custody of the work. Use: This work is available from the UC San Diego Library. This digital copy of the work is intended to support research, teaching, and private study.
Rights Holder and Contact
Lawrence Livermore National Laboratory and University of Illinois at Urbana-Champaign
Description
The dataset comprises trace files from high performance computing (HPC) simulations. The trace files contain records of every I/O operation executed by a simulation application run, including I/O operations from HDF5, MPI-IO, and POSIX and all of the parameters supplied to those operations, e.g. file name, offset, and flags. The traces are generated by executing a simulation application that is linked with the Recorder tracing tool (https://github.com/uiuc-hpc/Recorder). The Recorder trace tool intercepts the I/O calls made by the application, records the I/O trace record, and then calls the intended I/O call so that the operation executes. This material is based upon work supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research under the DOE Early Career Research Program. This work was partially supported by NSF SHF grant 1763540 . This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344 (LLNL-MI-811381). Research Data Curation Program, UC San Diego, La Jolla, 92093-0175 (https://lib.ucsd.edu/rdcp) Wang, Chen; Snir, Marc; Mohror, Kathryn (2020). High Performance Computing Application I/O Traces. In Lawrence Livermore National Laboratory (LLNL) Open Data Initiative. UC San Diego Library Digital Collections. https://doi.org/10.6075/J0Z899X4 For each application execution, Recorder generates a set of trace files. The files are in a binary format (in this collection, .ift containing trace records and .mt containing trace metadata) that is optimized to reduce the overhead of trace collection on the application. The tool recorder2text.c in Recorder/tools can be used to parse and generate a human readable text file from the binary traces. There are also scripts in the Recorder/tools/reporter directory that can be used to perform different kinds of analysis on the traces, or can serve as examples of how to parse and analyze the binary trace data. The detailed documentation on how to read and visualize Recorder traces can be found at: https://github.com/uiuc-hpc/Recorder For each application execution, Recorder generates a set of trace files. The files are in a binary format (in this collection, .ift containing trace records and .mt containing trace metadata) that is optimized to reduce the overhead of trace collection on the application. The tool recorder2text.c in Recorder/tools can be used to parse and generate a human readable text file from the binary traces. There are also scripts in the Recorder/tools/reporter directory that can be used to perform different kinds of analysis on the traces, or can serve as examples of how to parse and analyze the binary trace data. The detailed documentation on how to read and visualize Recorder traces can be found at: https://github.com/uiuc-hpc/Recorder Version update. On 2021-02-02, trace files from 4 applications (HACC-IO, MILC-QCD, pF3D-IO, and VPIC-IO) were added to this dataset. In addition, runs from 1024 scale processes were added to the data for all 17 applications.
Type
dataset
Identifier
ark:/20775/bb95276921
Language
No linguistic content
Subject
I/O performance High performance computing (HPC) I/O library performance High Performance Computing (HPC) application I/O
If you're wondering about permissions and what you can do with this item, a good starting point is the "rights information" on this page. See our terms of use for more tips.
Share your story
Has Calisphere helped you advance your research, complete a project, or find something meaningful? We'd love to hear about it; please send us a message.
