| Hilfe-Text | Hierarchical Data Format 5 (HDF5) library and tools version 1.10.5-openmpi-4.0-intel-18.0. The
corresponding dependency mpi/openmpi/4.0-intel-18.0 (aka compiler suite and MPI)
module should (and automatically will) be loaded first.
The HDF5 technology suite includes:
* A versatile data model that can represent very complex data objects and
a wide variety of metadata.
* A completely portable file format with no limit on the number or size of
data objects in the collection.
* A software library that runs on a range of computational platforms, from
laptops to massively parallel systems, and implements a high-level API
with C, C++, Fortran 90, and Java interfaces.
* A rich set of integrated performance features that allow for access time
and storage space optimizations.
* Tools and applications for managing, manipulating, viewing, and
analyzing the data in the collection.
When using HDF5 on a Lustre file system, the internal data
structures should be aligned with the striping of the file
system. You can query and set the stripe size for a directory
with the commands lfs getstripe and lfs setstripe.
About striping have a look at
http://wiki.lustre.org/index.php/Configuring_Lustre_File_Striping
Next, the hdf parameters should be tuned as follows:
H5AC_cache_config_t mdc_config;
hid_t file_id;
file_id = H5Fopen("file.h5", H5ACC_RDWR, H5P_DEFAULT);
mdc_config.version = H5AC__CURR_CACHE_CONFIG_VERSION;
H5Pget_mdc_config(file_id, &mdc_config)
mdc_config.evictions_enabled = 0 /* FALSE */;
mdc_config.incr_mode = H5C_incr__off;
mdc_config.decr_mode = H5C_decr__off;
mdc_config.flash_incr_mode = H5C_flash_incr__off;
H5Pset_mdc_config(file_id, &mdc_config);
There is an interesting thread about this in the hdf-forum:
http://hdf-forum.184993.n3.nabble.com/Question-re-Howison-et-al-Lustre-mdc-config-tuning-recommendations-td2469878.html
Documentation: /opt/bwhpc/common/lib/hdf5/1.10.5-openmpi-4.0-intel-18.0/doc
Examples: /opt/bwhpc/common/lib/hdf5/1.10.5-openmpi-4.0-intel-18.0/share/hdf5_examples
Web: http://www.hdfgroup.org/
In case of problems, please contact 'bwunicluster-hotline (at) lists.kit.edu'
or submit a trouble ticket at http://www.support.bwhpc-c5.de.
The full version is: 1.10.5-openmpi-4.0-intel-18.0 |