NAME
rtdbclean - real-time database and disk cleaner
SYNOPSIS
rtdbclean [-n] [-t] [-w] [-x] [-C] [-c channel_subset] [-v] [-V] [-m mail_to] [-p parameter_file] [orb] rt_db
DESCRIPTION
rtdbclean removes old waveforms and their respective wfdisc rows
from an operational real time database
rt_db.
rtdbclean also removes bogus data which has time more than two days
in the future.
The original
rtdbclean program required that the
orb be specified in the command line. A change in the
orb2db and
orb2dbt processes through the use of the
orb2db_msg have eliminated the requirement for the
orb
to be specified. It is still allowed as an optional argument to maintain backward use compatability.
The
orb argument is ignored in the program.
Initially removes only waveforms and then runs the command orb2db_msg rt_db pause, directing processes writing
to the database to pause, then performs the following operations:
-
marks wfdisc rows whose waveform files were removed
-
cleans up tables listed in pf file in clean_tables. Removes old rows from tables (normally non-core
css 3.0 tables like detection, gaps, retransmission, trigger). Can disable using -t.
-
removed deleted arrivals (where iphase is del) if arrival table exists
-
removes nojoin rows from tables listed in pf file in nojoins if both tables exist
-
crunch rt_db
-
sets arrival.deltim to NULL if arrival.deltim equals 0.0 if arrival table exists
-
remove NULL ndef nass if origin table exists
-
sets calib, calper, segtype in wfdisc if sitechan table exists
-
fix chanids in wfdisc and arrival if sitechan and arrival tables exist
When the complete, rtdbclean runs the command orb2db_msg $rt_db continue,
signaling processes which write to the database to continue,
and sends an email to the specified mail list
in the parameter file notifying if there was an abnormal
completion.
OPTIONS
-
-C
continue processing with no waveforms
-
-n
simulates execution, but does not change the database or remove waveforms
-
-t
skips table checks
-
-w
cleans database tables but does NOT remove waveforms
-
-v
verbose
-
-V
very verbose
-
-x
skip system resource check
-
-c channel_subset
channel selection option
-
-m mail_to
Email address(es) for error notification, should be quoted if using multiple addresses.
-
-p parameter_file
name of parameter file to use. $PFPATH will be searched to find parameter file.
-
orb
name of orbserver
-
rt_db
name of real-time database
containing site, network and accumulated wfdisc/arrival/... tables
ENVIRONMENT
needs to be called from rtexec or have sourced /opt/antelope/setup.csh. Need
environment variable $PFPATH to be set.
PARAMETER FILE
rtdbclean parameter file elements are:
-
min_disk_free
minimum number of free megabytes on disk
-
max_days_db
maximum number of days of complete data in wfdisc waveforms
-
max_days_clean_tables
maximum number of days of data left in cleaned tables
-
min_days_clean_wfdisc
minimum number of days to wait before removing wfdisc rows with
nsamp==0 or abs(endtime-time-(nsamp-1)/samprate)>.0005
-
orb2db_msg_sleep_secs
how long orb2db_msg must pause before starting cleanup. Default is 10 secs.
-
orb2db_msg_timeout_secs
how long orb2db_msg should wait for the message file to be cleared. Default is 900 secs (15 minutes).
-
clean_tables
list of database tables limited to max_days_db.
Only tables allowed to be cleaned are changed detection gap ratechange retransmit trigger wfmeas
-
check_tables
list of database tables to verify existance
-
unique_primary
list of database tables to guarantee unique primary keys
-
nojoins
set of database tables to remove rows which do not join to other specified tables
EXAMPLE PARAMETER FILE
#
# Remove wfdisc records and data according to either free disk space or
# maximum days in database (whichever is more restrictive)
min_disk_free 1000 # minimum free megabytes on disk
max_days_db 30 # maximum days of wfdisc waveforms
max_days_clean_tables 7 # maximum number of days of data
# left in additional cleaned tables
# Remove any anomalously empty wfdisc records after some period
min_days_clean_wfdisc 1 # minimum number of days to wait before
# removing wfdisc rows with nsamp==0
# or abs(endtime-time-(nsamp-1)/samprate)>.0005
# orb2db_msg wait for the message file to be cleared
orb2db_msg_sleep_secs 10 # how long (in seconds)
orb2db_msg_timeout_secs 900 # how long (in seconds)
#
# Remove older records from certain other database tables
clean_tables &Tbl{ # clean up these tables in addition
detection # to wfdisc
gap
retransmit
trigger
}
#
# Check to see if tables exist
check_tables &Tbl{
wfdisc
arrival
assoc
origin
event
origerr
netmag
stamag
calibration
sitechan
sensor
}
#
# Unique rows
unique_primary &Tbl{
assoc
changed
dlevent
emodel
netmag
origerr
origin
predarr
stamag
}
#
# Nojoin table cleanup
nojoins &Arr{ # remove rows from first table which do not join with second table
assoc_arrival &Tbl{
assoc
arrival
}
assoc_origin &Tbl{
assoc
origin
}
predarr_arrival &Tbl{
predarr
arrival
}
origin_event &Tbl{
origin
event
}
# Beware! origin nojoin with assoc will remove "lost" origins that may have
# been created by the chop (keep/delete/reassoc/chop) button in dbloc2. If
# you wish to keep these events, please comment out the next 4 lines.
origin_assoc &Tbl{
origin
assoc
}
emodel_origin &Tbl{
emodel
origin
}
event_origin &Tbl{
event
origin
}
origerr_origin &Tbl{
origerr
origin
}
netmag_origin &Tbl{
netmag
origin
}
stamag_origin &Tbl{
stamag
origin
}
stamag_netmag &Tbl{
stamag
netmag
}
}
DIAGNOSTICS
rtdbclean is normally run as a cron job under rtexec, so the
errors will be written to the log directory. All messages are sent to
STDERR.
SEE ALSO
dbfixchanids(1), dbjoin(1), dbselect(1), dbsort(1), dbsubset(1), orb2db_msg(1), pf(3), pfecho(1), rtexec(1)
BUGS AND CAVEATS
rtdbclean will check to see if a file named
.rtdbclean exists
and is locked. If
.rtdbclean is locked then rtdbclean will not
execute. This is to avoid protential situations which can
either corrupt the rt_db database or corrupt a running process.
Other programs which can lock
.rtdbclean are
mk_dmc_seed,
dmc_nrtwf,
rtbackup,
event_archive.
AUTHOR
Frank Vernon
Boulder Real Time Technologies, Inc.