NAME
rtbackup - real-time disk archive program - second generation
SYNOPSIS
rtbackup [-v] [-V] [-n] [-A] [-E] [-R] [-g]
[-p parameter_file]
[-m mail_to]
[-r reject_stas]
[-s sta_match]
[-t start_time]
[-e end_time]
[-w wfname]
rt_db
DESCRIPTION
rtbackup replicates and heals database named
rt_db containing waveforms
to a database on a disk directory. The output disk directory is preferably on a separate disk
from the operational real time data acquisition system.
rtbackup is in its second generation,
the original version archived day volumes to a tar tape. With the advent of large cheap disk
drives which have a much longer shelf life than tape storage, is it now advantageous to archive
to external network attached storage. The original version of
rtbackup is now called
rtbackup2tape.
The other development which necessitated a rewrite of the original
rtbackup is the unfolding
capability of long-term (days to months) local storage at seismic stations. This can cause a
large asynchronicity between stations with good communications and those with marginal, intermittent,
or down communications.
rtbackup now makes liberal use of
trexcerpt build databases which have a duration
of a month or a year depending on the parameter file specification. Each day of data is organized
into a station-channel-day of data split precisely on day boundaries. The use of
trexcerpt(1)
has the advantage of removing any overlaps in the data which might have occurred.
Data are not archived unless the lddate on the
wfdisc row is older than a number of days specified in the parameter file.
The question of what to do with gapped data from a station which reconnects later is a bit tricky.
The most conservative approach was taken with the default action of the program which is to not
remove data from the archive. This has the problem in some pathological cases to allow for
duplicate or overlapped waveform data in a database. If you are feeling trusting and bold, you can
use the
-E and the
-R options which will attempt to remove overlapping waveforms in
the archive database and produce the cleanest (
dbverify(1) ) database.
OPTIONS
-
-v
verbose
-
-V
Extremely verbose. Used for debugging purposes only.
-
-x
Skip system resource check.
-
-n
No operation. Shows list of actions, but does not archive any data.
-
-A
Archive all. Skip lddate checks.
-
-E
Expert mode. Supresses some warnings. Required to remove archive waveforms.
-
-g
Eliminate marked data gaps in calling trexcerpt, i.e. data gaps where the waveform has been filled in
with a specific unlikely or impossible value .
-
-R
Remove archive waveforms if they can be replaced from original database. This is useful
when data overlaps occur in station data where a long communcation outage has occured.
Can only be used with the -E option.
-
-p parameter_file
Name of parameter file to use. $PFPATH is searched to find parameter file.
The default parameter file is rtbackup.
-
-m mail_to
Email address(es) for error notification, should be quoted if using multiple addresses.
-
-r reject_stas
Regular expression for stations to be rejected. Default is "-".
-
-s sta_match
Regular expression for stations to be matched. Default is ".*".
-
-t start_time
Start time for processing data.
The default start time is 1 January 1970.
-
-e end_time
End time for processing data.
The default end time is now().
-
-w wfname
specify the naming convention for the output waveforms, according to
trwfname(3).
The default pattern %Y/%j/%{sta}.%{chan}.%Y.%j
results in names of the form:
2013/296/PFO.BHZ.2013.296
ENVIRONMENT
needs to be called from rtexec or have sourced $ANTELOPE/setup.csh. Need
environment variable $PFPATH to be set.
PARAMETER FILE
rtbackup parameter file elements are:
-
dirbase
directory base name for building waveform databases
-
dbbase
base name for building database names
-
period
Period of time for database segmentation. Can be "year", "month", or "day".
-
lag
Number of complete days before present which lddate has to have
in wfdisc before processing.
-
dbidserver
Name of idserver to be written into descriptor file for output database to use.
-
dbpath
dbpath to be written into descriptor file for output database to use.
-
dblocks
dblocking type for database
-
backup_dirs
Table list of directories which are backed up to a tar file
-
backup_ev_tables
Table list of database tables which are backed up to a tar file
EXAMPLE PARAMETER FILE
dirbase /anf/TA/build/clean/rt # should be on a different disk than the rtdb
dbbase usarray #
period year # allowed values are "year" or "month"
#period month # allowed values are "year" or "month"
lag 2 # number of days of lag for lddate before processing row
dbpath /anf/TA/dbs/dbmaster/{usarray} # default dbpath
dbidserver anfops:2498 # default dbidserver
dblocks nfs # default dblocks
backup_dirs &Tbl{ # directories to backup to tar file
logs
pf
ttgrid
}
backup_ev_tables &Tbl{ # directories to backup to tar file
arrival
assoc
emodel
event
origerr
origin
predarr
netmag
stamag
}
DIAGNOSTICS
rtbackup is normally run as a cron job under rtexec, so the
errors will be written to the log directory or to the email log
if the
-m option is used.
SEE ALSO
pf(3)
pfecho(1)
rtexec(1)
dbverify(1)
trexcerpt(1)
BUGS AND CAVEATS
Each day of data is backed up individually. Arrivals are subsetted on
day boundaries, and are then joined to the assoc, origin, event, and
origerr tables.
If the arrivals for an event span a day boundary, then the origin and
event rows will appear once in each days database. This could cause a
corrupt database if two days of data are extracted from the archive and
the tables from each day are merged improperly.
rtbackup will check to see if files named .rtdbclean or
.rtbackup exist and are locked. If either of these files are
locked then rtbackup will not execute. This is to avoid potential
situations which can either corrupt the rt_db database and/or
corrupt the execution of rtbackup.
Other programs which can lock .rtdbclean are rtdbclean,
mk_dmc_seed, dmc_nrtwf, rtbackup, event_archive.
AUTHOR
Frank Vernon
Boulder Real Time Technologies, Inc.