• Antelope Release 5.10 Linux CentOS release 7.6.1810 (Core) 3.10.0 2020-05-12

 

NAME

notes_best_practices - advice about configuring antelope to work well

DESCRIPTION

This attempts to collect together some advice on how to configure your computer and Antelope system to gain the best performance.

Waveform Acquisition

There are two main programs to acquire waveform data to a database, orb2db(1) and orb2wf(1) (The old cdorb2db program has been decommissioned, essentially replaced by orb2wf(1)). Be aware of the nature of your data when choosing which program to run. The orb2db(1) program saves individual segments of data, which is appropriate if you are saving continuous data that are not heavily fragmented or if you are saving event-triggered data. The orb2wf(1) program pre-allocates continuous grids of sample values on disk, filling them in with data as the packets are acquired. This is appropriate for heavily fragmented data sets and is generally the safest option to avoid disk bloat from data problems if you are acquiring continuous data, however has the down side that you lose a bit of the time resolution of the individual packets and lose information on individual segments. Again, ultimately you must determine how to acquire waveform data based on the properties of the incoming data and the goals of your system.

Memory

On any real time system, the goal is to avoid paging altogether. This means arranging so that (at least most of the time), every active program fits into physical memory. Since memory is so cheap today, it's not difficult to fit all the program code into memory, but the programs allocate space for their operation and often memory map files. It's this space which you need to consider.

orbserver memory maps its buffers into memory. While the ring buffer portion is generally accessed in a pretty sequential manner -- all near the leading edge of the ring buffer -- the orbserver may more randomly access old packets, or start from the trailing edge of the ring buffer and read the entire ring buffer. To be safest, you should count the orbserver buffer files as part of your total memory allocation.

Datascope memory maps database tables. The operating system will share this memory between multiple processes, but very large database tables may use a significant part of your memory budget.

Waveforms typically have to be uncompressed from disk files into 4 byte floats when they are used.

orbassoc memory maps the ttgrid files.

Partitioning

Particularly on a mac, it's advantageous to keep the orbserver ring buffers on a separate partition: this should give you better performance.

Keeping waveforms on a separate partition has the advantage that if the partition is inadvertantly filled up, the real time system will continue to function, and cleaning up the mess will be a bit easier. (Data are likely to be lost, unless the ring buffer still has the data when the waveform acquisition comes back up).

orbserver parameters

A rule of thumb for the size of ring buffers is to have the ring buffer contain about as much data as you would expect for any down time in your network, whether for database cleanup, power outages, or transmission delays. When the system recovers, there will be sudden inrush of data, potentially exceeding the maximum speed at which orb2wf(1) or orb2db(1) can write data out to disk.

Look carefully at the default minimum packet size in orbserver.pf. It should be smaller -- perhaps 30% smaller -- than the average packet size you see in your orbserver, looking at orbstat -s:


% orbstat -s anfexport.ucsd.edu:usarrayTA | tail
TA_Z40A/MGENC/M40             8265     124703    44628.0 25226871  108  5:18:02  3642607  109 15:56:24          2.9    2.855 seconds
TA_Z40A/MGENC/MEP             8265      12503     6380.0 25228540  108  5:17:27  3638999  109 15:55:39          0.4   47.714 seconds
TA_Z41A/MGENC/EP1             8265       5658     1105.2 25313252  107 20:33:32  3639013  109 15:55:51          0.1   35.409 seconds
TA_Z41A/MGENC/EP40            8265      52320    16201.4 25291224  107 20:33:47  3642264  109 15:56:22          0.8    4.884 seconds
TA_Z41A/MGENC/M1              8265       5661     1475.1 25313257  107 20:33:34  3639015  109 15:55:53          0.1   34.380 seconds
TA_Z41A/MGENC/M40             8265      52322    19382.6 25291225  107 20:33:48  3642265  109 15:56:23          1.0    3.855 seconds
TA_Z41A/MGENC/MEP             8265       5670     2872.0 25313259  107 20:33:20  3639014  109 15:55:39          0.1   47.714 seconds
ringbuffer has  1 days 19.3 hours    of data
  23999.999 Mbytes data in 67305032 packets
  average packet size = 357 bytes

This ensures that the leading edge pktid and the trailing edge pktid have a significant gap between them. If this isn't true, then an orbserver client which is delayed while reading packets near the end of the ring buffer could discover the ring buffer has wrapped around while it's been waiting, and it's now reading a packet nearer the leading edge. (Typically, clients are reading packets near the leading edge of the orbserver, so this doesn't happen).

AUTHOR

Daniel Quinlan
Printer icon