DAS Convert

Convert and downsample distribute acoustic sensing (DAS) data acquired by Silixa iDAS to seismological data formats. Main purpose is to quickly convert and downsample massive amounts of high-resolution TDMS data to MiniSeed.

To handle the massive amount of data generated by DAS interrogators, the conversion tool is leveraging parallel IO and signal-processing processing. On production systems a throughput of 200 MB/s can be archived while converting and downsampling (from 1 kHz to 200 Hz).

The signal processing routines are based on the Pyrocko, mature and well tested seismological framework.

Installation

python3 setup.py install

Documentation

Find the online documentation at https://pyrocko.org/idas_convert/docs/.

Usage

Dump a config file

das_convert dump_config

Example Config

--- !idas.iDASConvertConfig
# Loading TDMS in parallel and process
nthreads_loading: 1
nthreads_processing: 8
queue_size: 32
processing_batch_size: 8

# Threads used for downsampling the data
nthreads: 8

# Input paths
paths:
- /home/isken/src/idas-convert

# Out path, see pyrocko.io for details
outpath: '%(tmin_year)s%(tmin_month)s%(tmin_day)s/%(network)s.%(station)s_%(tmin_year)s%(tmin_month)s%(tmin_day)s.mseed'

# Overwrite mseed meta information
new_network_code: ID
new_channel_code: HSF

downsample_to: 200.0

# MiniSeed record length
record_length: 4096
# MiniSeed STEIM compression
steim: 2

plugins:

# A plugin handling the communication with the GFZ tage file system
- !das_convert.gfz_tapes.GFZTapesConfig
  enabled: false
  bytes_stage: 1T
  waterlevel: 0.6
  wait_warning_interval: 600.0
  release_files: true
  path_tapes_mount: /projects/ether/
  path_tapes_prefix: /archive_FO1/RAW/

# A Telegram bot to keep up-to-date with the process
- !das_convert.telegram_bot.TelegramBotConfig
  enabled: false
  # Telegram API Token
  token: 9e98b8c0567149eb861838a1d770be7d
  # Telegram Chat ID
  chat_id: -1237123123
  # A status message will be dispatched every 3600 s
  status_interval: 3600.0

Start Conversion

das_convert my_config.yml

See das_convert -h for more options.

Citation

DAS Convert - Convert distributed acoustic sensing data, Isken, Marius Paul; Christopher, Wollin; Heimann, Sebastian; Javier, Quinteros; Karl-Heinz, Jäckel; Philippe, Jousset (2021): DAS Convert - Convert distributed acoustic sensing data. V. 1.0. GFZ Data Services. https://doi.org/10.5880/GFZ.2.1.2021.005

Legal Stuff

Licence: GNU General Public License, Version 3, 29 June 2007

Copyright © 2021 Helmholtz Centre Potsdam GFZ German Research Centre for Geosciences, Potsdam, Germany

DAS Convert is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. DAS Convert is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see <http://www.gnu.org/licenses/>.