• Antelope Release 5.2-64 SunOS 5.10 2012-04-24

 

NAME

orb2wf - archive waveform data from an ORB into a database

SYNOPSIS

orb2wf [-c chanmatch] [-m srcmatch] [-r srcreject]
       [-p pf-file] [-S state-file] [{-v|-vv|-vvv}]
       [-dbm dbmaster] orb db [start-time [window]]

DESCRIPTION

orb2wf writes waveform data from an ORB ring buffer to a continuous database. The output waveform data is written into fixed duration sample grids of waveform sample values. As new waveform sample values are read from ORB input packets, each sample value is projected into its appropriate waveform file and sample position. When a new sample value projects into a non-existing waveform file, then the sample rate corresponding to the new sample value is used to determine a regular grid of sample values over the fixed duration of the new waveform file. Also created is a new wfdisc row. The new waveform file is pre-filled with special sample values to indicate missing data, referred to as gap sample values. As new sample values are read for that channel, they are projected into the appropriate sample value slots in the waveform files. When new waveform files are created, the corresponding start time of the first sample is always exactly on a second boundary and aligned with the day-hour-minute boundaries according to the fixed waveform durations. When new waveform files are created, the sample rate used to determine the sample grid is rounded off to the nearest 0.1 sps for sample rates of 0.1 or higher. Note that this method of waveform archiving insures that the time accuracy of each sample value is within +-0.5 of the output waveform sample interval. This method of waveform archiving produces one wfdisc row and waveform file per channel per output waveform duration interval regardless of the input waveform time fragmentation or data time order.

This program is a replacement for the old cdorb2db(1) program originally written to archive CD1.0 data produced by CTBTO systems. Although this program can be used to archive CD1.0 data, it has been developed as a general alternative to orb2db(1). In most situations this program should be used instead of orb2db(1) to archive waveform data. The reason for this is that this program will produce well healed waveform archive databases regardless of the time fragmentation of the input ORB data. When archiving data that is out of time order or displays lots of time tears, orb2db(1) will produce heavily fragmented archive databases with many wfdisc rows and potentially many short archive waveform fragments. The down side to using orb2wf is that it cannot compress the output waveform archive on-the-fly. The output waveform format will be a fixed-size-per-sample format, such as 4-byte integer or floating point formats. Post processing compression of waveforms can be done with db2msd(1) once a particular time range of data is no longer being updated with new data.

A big difference between orb2wf and cdorb2db(1) is the way in which data channels with changing sample rate or changing metadata are handled. In cdorb2db(1), the sample rate of incoming ORB packets are only used once when new waveform channels are processed. The sample rate of the new waveform channel is fixed forever according to the sample rate of the first ORB packet processed. Subsequent ORB packets for each channel are projected to the nearest sample grid according to the start time for each ORB packet and the sample rate of the original ORB packet that was used when the new channel was first created. The rest of the samples after the start sample for each ORB packet are then laid down into the fixed sample grid sequentially ignoring the samplerate of the input ORB packet. This can result in unexpected behavior when the sample rate for a channel of data is changed substantially. orb2wf will always treat each waveform sample value individually by applying the ORB packet sample rate to determine the real sample time and then project into the fixed output sample grid. Also in orb2wf is a sample rate threshold tolerance which is applied to each ORB packet sample rate against the fixed sample rate of the output waveform sample grid. If a new ORB packet for a given channel has a sample rate that exceeds this tolerance, then a new version of the output waveform sample grid is created to accomodate the changed channel sample rate. The same thing will happen if calib, calper or segtype change for a given data channel. See orbwfproc_export(3O) for a more detailed explanation of how this works.

OPTIONS

PARAMETER FILE

Following is an example of a parameter file.


#   orb2wf parameter file

preferred_waveform_file_range 86400  # waveform files should cover this range of seconds:
                                     #      86400 = 1 day
                                     #       3600 = 1 hour

pause_timeout                 3600   # maximum time (in seconds) to wait after pause for
                                     # the matching continue.  This is a failsafe, in case the
                                     # program issuing the original pause fails to deliver
                                     # a continue for some reason.  The period should
                                     # probably be less than the ringbuffer size, and
                                     # definitely greater than the worst case performance
                                     # of any backup or cleanup script.

too_old	31556925 # ~ 1 year          # fill in with a time x to discard packets
                                     #  with time less than system time - x
too_new	604800   # ~ 1 week          # fill in with a time to discard packets
                                     #  with times greater than system time + x
decent_interval               60     # interval at which to save state file in seconds
max_open_files                500    # force opening and closing files after specified number

calibfromdb                   no     # marry calib, calper, segtype from a master db?
heap_throttle                 10M    # heap memory based throttle on input
maxduration                   300.0  # maximum time duration for buffering data before flushing to disk
realtime_thresh	              60.0   # data latency threshold for reducing buffering
samprate_thresh	              10.0   # percent change in sample rate that triggers new wfdisc rows
calib_thresh	              1.0    # percent change in calib value that triggers new wfdisc rows
calper_thresh	              1.0    # percent change in calper value that triggers new wfdisc rows
nullcalib	              no     # Should output calib, calper and segtype all be set to NULL values?
translations &Tbl{                   # SEED code translations for output
     .*     ONETSTA_OCHANLOC           # noop translation
}
datatype                      input  # output waveform data type
add_wfdisc_headers            yes    # should a copy of the wfdisc row be added as a waveform header?
gain                          1.0    # gain value for converting floating data to integer data

pf_revision_time 1319822905

The orb2wf parameters are as follows.

BUGS AND CAVEATS

orb2wf is a skin for orbwfproc and is exactly the same executable as orbwfproc. The program name is used to determine which skin is being run and then parses arguments differently depending on whether the database or ORB version is being run.

SEE ALSO

orbwfproc(1)
orbwfproc_import(3o)
orbwfproc_export(3o)

AUTHOR

Danny Harvey
Boulder Real Time Technologies, Inc.
Printer icon