TY - JOUR
T1 - AdiosStMan: Parallelizing Casacore Table Data System using Adaptive IO System
AU - Wang, Ruonan
AU - Harris, C.
AU - Wicenec, Andreas
PY - 2016/7/1
Y1 - 2016/7/1
N2 - © 2016 Elsevier B.V.In this paper, we investigate the Casacore Table Data System (CTDS) used in the casacore and CASA libraries, and methods to parallelize it. CTDS provides a storage manager plugin mechanism for third-party developers to design and implement their own CTDS storage managers. Having this in mind, we looked into various storage backend techniques that can possibly enable parallel I/O for CTDS by implementing new storage managers. After carrying on benchmarks showing the excellent parallel I/O throughput of the Adaptive IO System (ADIOS), we implemented an ADIOS based parallel CTDS storage manager. We then applied the CASA MSTransform frequency split task to verify the ADIOS Storage Manager. We also ran a series of performance tests to examine the I/O throughput in a massively parallel scenario.
AB - © 2016 Elsevier B.V.In this paper, we investigate the Casacore Table Data System (CTDS) used in the casacore and CASA libraries, and methods to parallelize it. CTDS provides a storage manager plugin mechanism for third-party developers to design and implement their own CTDS storage managers. Having this in mind, we looked into various storage backend techniques that can possibly enable parallel I/O for CTDS by implementing new storage managers. After carrying on benchmarks showing the excellent parallel I/O throughput of the Adaptive IO System (ADIOS), we implemented an ADIOS based parallel CTDS storage manager. We then applied the CASA MSTransform frequency split task to verify the ADIOS Storage Manager. We also ran a series of performance tests to examine the I/O throughput in a massively parallel scenario.
U2 - 10.1016/j.ascom.2016.05.003
DO - 10.1016/j.ascom.2016.05.003
M3 - Article
SN - 2213-1337
VL - 16
SP - 146
EP - 154
JO - Astronomy and Computing
JF - Astronomy and Computing
ER -