Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Close LH5 file stream if I/O error is thrown #105

Open
gipert opened this issue Sep 18, 2024 · 0 comments
Open

Close LH5 file stream if I/O error is thrown #105

gipert opened this issue Sep 18, 2024 · 0 comments
Labels
bug Something isn't working lh5 HDF5 I/O

Comments

@gipert
Copy link
Member

gipert commented Sep 18, 2024

If (de)serialization fails for whatever reason, any following operation on the same file fails with something like

File ~/sw/src/legend/legend-pydataobj/src/lgdo/lh5/core.py:299, in write(obj, name, lh5_file, group, start_row, n_rows, wo_mode, write_start, page_buffer, **h5py_kwargs)
    292 if wo_mode in ("w", "write", "of", "overwrite_file"):
    293     h5py_kwargs.update(
    294         {
    295             "fs_strategy": "page",
    296             "fs_page_size": page_buffer,
    297         }
    298     )
--> 299 return _serializers._h5_write_lgdo(
    300     obj,
    301     name,
    302     lh5_file,
    303     group=group,
    304     start_row=start_row,
    305     n_rows=n_rows,
    306     wo_mode=wo_mode,
    307     write_start=write_start,
    308     **h5py_kwargs,
    309 )

File ~/sw/src/legend/legend-pydataobj/src/lgdo/lh5/_serializers/write/composite.py:57, in _h5_write_lgdo(obj, name, lh5_file, group, start_row, n_rows, wo_mode, write_start, **h5py_kwargs)
     55 if not isinstance(lh5_file, h5py.File):
     56     mode = "w" if wo_mode == "of" or not os.path.exists(lh5_file) else "a"
---> 57     lh5_file = h5py.File(lh5_file, mode=mode, **file_kwargs)
     59 log.debug(
     60     f"writing {obj!r}[{start_row}:{n_rows}] as "
     61     f"{lh5_file.filename}:{group}/{name}[{write_start}:], "
     62     f"mode = {wo_mode}, h5py_kwargs = {h5py_kwargs}"
     63 )
     65 group = utils.get_h5_group(group, lh5_file)

File ~/.virtualenvs/legend-dev/lib/python3.12/site-packages/h5py/_hl/files.py:562, in File.__init__(self, name, mode, driver, libver, userblock_size, swmr, rdcc_nslots, rdcc_nbytes, rdcc_w0, track_order, fs_strategy, fs_persist, fs_threshold, fs_page_size, page_buf_size, min_meta_keep, min_raw_keep, locking, alignment_threshold, alignment_interval, meta_block_size, **kwds)
    553     fapl = make_fapl(driver, libver, rdcc_nslots, rdcc_nbytes, rdcc_w0,
    554                      locking, page_buf_size, min_meta_keep, min_raw_keep,
    555                      alignment_threshold=alignment_threshold,
    556                      alignment_interval=alignment_interval,
    557                      meta_block_size=meta_block_size,
    558                      **kwds)
    559     fcpl = make_fcpl(track_order=track_order, fs_strategy=fs_strategy,
    560                      fs_persist=fs_persist, fs_threshold=fs_threshold,
    561                      fs_page_size=fs_page_size)
--> 562     fid = make_fid(name, mode, userblock_size, fapl, fcpl, swmr=swmr)
    564 if isinstance(libver, tuple):
    565     self._libver = libver

File ~/.virtualenvs/legend-dev/lib/python3.12/site-packages/h5py/_hl/files.py:241, in make_fid(name, mode, userblock_size, fapl, fcpl, swmr)
    239     fid = h5f.create(name, h5f.ACC_EXCL, fapl=fapl, fcpl=fcpl)
    240 elif mode == 'w':
--> 241     fid = h5f.create(name, h5f.ACC_TRUNC, fapl=fapl, fcpl=fcpl)
    242 elif mode == 'a':
    243     # Open in append mode (read/write).
    244     # If that fails, create a new file only if it won't clobber an
    245     # existing one (ACC_EXCL)
    246     try:

File h5py/_objects.pyx:54, in h5py._objects.with_phil.wrapper()

File h5py/_objects.pyx:55, in h5py._objects.with_phil.wrapper()

File h5py/h5f.pyx:122, in h5py.h5f.create()

OSError: Unable to synchronously create file (unable to truncate a file which is already open)

I haven't investigated in detail but I guess this should be resolved by closing the HDF5 before throwing the exception.

@gipert gipert added bug Something isn't working lh5 HDF5 I/O labels Sep 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working lh5 HDF5 I/O
Projects
None yet
Development

No branches or pull requests

1 participant