following a decrease in gain due to an increased noise floor.
This is controlled by --adaptive-range-scan-delay and defaults
to 5 minutes (vs. the default rescan delay of 1 hour)
The legacy -10 value is still supported for compatbility with old
configs, but new configs should just select an appropriate gain e.g.
60dB.
rtlsdr default gain continues to be the highest regular gain step (~ 49.6dB)
This makes the adaptive gain code only process some fraction of all samples,
rather than every sample. This helps a lot with CPU on slower machines.
We default to a 50% duty cycle to save some CPU without really affecting
the behavior much.
We split the one-second adaptive gain blocks into 20* 50ms "subblocks"
Each subblock is either entirely processed or not processed at all,
based on the configured duty cycle. (This means that the duty cycle
can only be specified in approx 5% increments, but that should be
fine)
For adaptive dynamic range, there's no further complications - we
just look at fewer samples to compute the noise floor.
For adaptive burst, we need to scale up the measured burst rate by
1/duty cycle to account for the samples that we didn't process. It's
also more likely to cause functional changes as it's quite possible
for a relatively small number of loud messages being missed to
affect the computed rate substantially. Not many options there other
than increasing the duty cycle when adaptive burst handling is wanted.
Update adaptive gain control loop to handle interactions between burst detection and dynamic range control.
Internal doc updates & naming cleanups.
Stats tweaks.
This PR adds basic adaptive gain support, which adjusts SDR gain on the fly based on the noise & signal levels seen.
There are two control mechanisms:
Dynamic range control is enabled by the --adaptive-range option. This adjusts SDR gain to try to achieve a minimum dynamic range, regardless of the exact hardware in the RF path.
Burst (loud message) control is enabled by the --adaptive-burst option. This decreases SDR gain when undecodable loud messages are heard, allowing for better reception of nearby aircraft at the expense of range.
This is only the basic implementation - see the PR for remaining work to do.
Give up and exit after 30 seconds of no sample data, rather than just warning and continuing.
background & discussion: https://discussions.flightaware.com/t/cpu-hikes-crash-dump1090-fa/74759
The scenario this addresses is:
* Hardware wedges, USB bulk endpoint stops providing data
* librtlsdr remains in rtlsdr_read_async() waiting for either USB data which never arrives,
or a cancellation via _a different thread_ calling rtlsdr_cancel_async().
* main thread notices the lack of SDR data and complains
* something external e.g. piaware tries a restart and sends SIGTERM
* the signal handler sets Modes.exit = 1; the main thread starts waiting for receive thread termination
* because we're never getting callbacks from rtlsdr_read_async(), we never call rtlsdr_cancel_async
* dump1090 hangs waiting on receive thread termination
To fix this, add a sdrStop() handler function where the general contract is "make the receive thread terminate".
In the rtlsdr case, this calls rtlsdr_cancel_async directly, which will make rtlsdr_read_async() return even
if the hardware is stuck.
The main thread then calls sdrStop() before waiting for receive thread termination.
Also, as discussed in the thread above, there's not really much point in continuing to run if the SDR
has wedged, so bail out after 30 seconds of no sample data.
Also, if pthread_timedjoin_np is available, use it in preference to pthread_join so that we do not wait
indefinitely for the receive thread on shutdown. If the join times out, give up and abort() as we can't
safely continue a clean shutdown while the receive thread is running.
This reinstates the fastpath in the 2.4MHz demodulator that
aborts message demodulation early if the DF bits don't match a message
type that we know how to decode, or stops early if the DF bits can
only correspond to a short 56-bit message.
Do this in a more general form where we test against a set of valid
DF values, rather than a switch/case. Then populate the sets based on
the current error-correction settings (e.g. if we are allowed to repair
DF damage with 1 bit error correction, then a DF17 message could appear
with any of DF=17, DF=16, DF=19, DF=21, DF=25, or DF=1 and still be
correctable to a valid DF17 message)
In the default case this produces a small CPU improvement as short
messages might be able to stop earlier and a few DF values can be ignored
early. With --no-fix-df it produces a larger improvement as relatively few
DF values are valid in that mode.
To further reduce CPU, the default behaviour has changed to _not_
decode DF24 messages (Comm-D ELM messages). These are rarely seen in
the wild and dump1090 can't do anything useful with them. Disabling
them allows us to further reduce the set of valid DF values as they
effectively use all DF values from 24..31, 25% of all possible values.
To re-enable DF24 decoding, use the new `--enable-df24` option.
Finally, avoid doing some CRC calculations twice. This was only happening
once per valid decoded message, not per demodulation attempt, so it's not
such a large win.
Usually, all message candidates are speculatively corrected as if
they were DF11/17/18, even if the DF field has a different value. This
allows correction of messages when there is correctable damage to the
DF field.
However, running every message candidate through a couple of CRC checks
is expensive, CPU-wise, when decoding messages off the air, as there are
a large number of message candidates that are actually just junk data, and
computing CRCs for all of those adds up. The --no-fix-df option allows
disabling this sort of correction. The tradeoff is that messages with
damage to the DF field will not be corrected.
Previously, json stats updates were fixed to 60 seconds, and using --stats--every
with an interval less than 60 seconds produced confusing results.
This commit fixes --stats-every with short intervals, and adds --json-stats-every
to control the json stats update interval. Note that if --json-stats-every is not
a multiple of 60 seconds, then the 1-/5-/15-minute data may not include all data
up to the current time (in that case, the most recent data is reflected in
the "latest" stats)
This implements the alternative approach discussed in PR #89
* Switch all conversion routines to use starch.
main user-visible changes:
* ensure you check out submodules ('git clone --recurse-submodules")
* --version shows the CPU features and DSP implementations in use
* --wisdom allows overriding of the built-in architecture wisdom
* --dcfilter no longer supported
* "starch-benchmark" binary will benchmark all options on the
current machine and can produce a wisdom file to feed to
the --wisdom option
If you have a usecase for --dcfilter, please get in touch and
let me know - it's an edge case and for now there's no starch/DSP
support for it, but support can be written if needed.
In almost all cases the new conversion routines are slightly or
substantially faster than the old conversion routines. The only case
that is slower is SC16/SC16Q11 on a Pi 0, which is around 10% slower
due to changing from heavily approximated lookup tables to higher
quality results (but SC16 is probably already out of reach of a Pi 0)
* No need to build with SC16Q11_TABLE_BITS any more
* Add oneoff/uc8_capture_stats
(reads a UC8 capture; measures min/max/mean I and Q)
* Switch UC8 conversion to 127.4 center, 128 range.
Looking at actual UC8 captures from a RTL2832, the mean I and Q
are actually at 127.4, so use that as the zero point.
This means that the resulting I/Q maximum values could be as large as
127.6. Switch to 128 for simplicity.
* Switch to the new UC8 zero offset in benchmarks, fix some bugs
* Fix some bugs in SC16/SC16Q11 validation, tighten the max error requirements
* Ditch UC8 approximation path, add a NEON VRQSQRTE path.
* Tweak the SC16 exact path, add a new impl that uses a mix of
u32 & floats.
* SC16Q11 impl tweaks:
* add a u32->float exact path
* ditch the approximation path
* add a NEON VRSQRTE path
* add a 12-bit table path (using the full signed I/Q value, not absolute value)
* Ditch SC16 approximation path, add NEON vrsqrte path
* Add oneoff/dsp_error_measurement
This runs sample input through the DSP functions that are
allowed to be inexact and dumps the results as a TSV suitable for
feeding to gnuplot to look at the actual errors.
* Update make clean, make wisdom targets
* Update wisdom based on benchmarking
* Preserve the raw wisdom benchmark data
* Update to latest starch
* Update .gitignore for new wisdom files
* Update starch generated code
* Build starch-benchmark as part of the 'all' target
* Use wisdom from /etc/dump1090-fa/wisdom.local if present
* Package starch-benchmark and a helper script to generate local wisdom data
* Remove submodules in preparation for importing them directly
* Import cpu_features v0.6.0 from https://github.com/google/cpu_features/releases/tag/v0.6.0
* Import starch at commit a725c8491dc33a321565d451b385131e589d8490
from https://github.com/flightaware/starch
These updates were designed to assist those using interactive
mode to tune antennas and SDR gain.
* Add options to display distance and bearing in interactive mode
Distance and bearing instead of latitude and longitude can now be
displayed in interactive mode using the following options to
dump1090-fa and view1090-fa.
--interactive-show-distance Show aircraft distance and bearing
instead of aircraft lat/lon
(requires --lat and --lon)
--interactive-distance-units Distance units ('km', 'sm', 'nm')
(default: 'nm')"
You have to specify a reference --lat and --lon for this to
work of course.
* A new line now shows at the top of the interactive display that
has for the current sample:
Total valid aircraft count
Vidible aircraft count
Will be less than total if the screen hasn't enough lines to show
them all.
Max RSSI
Min RSSI
Mean RSSI
Max Distance
Tot: 47 Vis: 47 RSSI: Max 25.4+ Mean -29.5 Min -36.9- MaxD: 197.3nm+
* Add max distance and min/max RSSI indicators
A '+' after the distance in a row indicates it's the row with the
maximum distance.
A '+' after the RSSI in a row indicates it's the row with the highest
RSSI.
A '-' after the RSSI in a row indicates it's the row with the lowest
RSSI.
The summary line at the top of the screen always shows the values for
ALL aircraft, even those not visible. The row indicators only mark
visible rows though.
In this example, the first aircraft is both the farthest away and has
the weakest RSSI. The second aircraft has the strongest RSSI.
Tot: 47 Vis: 47 RSSI: Max 25.4+ Mean -29.5 Min -36.9- MaxD: 197.3nm+ -
Hex Mode Sqwk Flight Alt Spd Hdg Dist(nm) Bearing RSSI Msgs Ti
────────────────────────────────────────────────────────────────────────────────
A8D5A4 S2 34000 438 252 197.3+ 85 -36.1- 26 2
A39A13 S2ac 5740 FFT525 30750 439 256 98.8 68 -25.4+ 123 0
A70B23 S2ac 2744 LXJ553 43000 419 258 136.1 39 -33.6 174 0
* Finally, a new option '--interactive-callsign-filter' has been added
to allow filtering interactive by callsign. The value can be a
simple string, in which case aircraft with that string anywhere in its
callsign will be displayed, or a regular expression should you want a
more precise match.
Examples:
--interactive-callsign-filter UAL
will match all aircraft with UAL anywhere in its callsign.
--interactive-callsign-filter "^UAL"
will match only those callsigns that start with UAL.
--interactive-callsign-filter "^(UAL|AAL)"
will match only those callsigns that start with UAL or AAL.
--interactive-callsign-filter "^N[0-9]" or "^N[[:digit:]]"
will match only those callsigns that start with "N" and a number
If you need more info on regular expressions, see this link:
https://en.wikipedia.org/wiki/Regular_expression
This tries to retain the existing behaviour of --net / --net-only
while making it easier to say "listen on only these specific ports".
If --net or --net-only is specified, network mode is enabled and
default port values are assumed for any port not otherwise specified
on the command line. Specifying a port of 0 disables that port. This
is the same as the old behavior.
If --net or --net-only is not specified, but at least one
of the --net-xxx-port options is specified, then network mode is
enabled with no default port value and only those ports explicitly
configured on the command line are used.
This means you can e.g. configure a port-30005-only SDR receiver
by specifying just "--net-bo-port 30005"; or configure a
network-receive-only receiver (e.g. for SkyAware display) by
"--device-type none --net-bi-port 30004,30104". These were possible
previously but required turning off a lot of the default ports.
Reorder --ifile pseudo-sdr after the "none" SDR so that a build with no real
SDR support defaults to "none" not --ifile
If the "none" SDR was selected and network mode is not enabled, tell the
user about the problem rather than just failing to do anything useful.
Update all the SDR implementation to use it.
This was getting pretty ugly with code getting copy&pasted in all the SDR
implementations. Unify it all and give it a simpler API. Linked list works out
much simpler than the circular buffer. Also, simplify copying the overlap region
around by just using a separate buffer (it's only a few hundred bytes long, so
the double copy is not a big deal).
The commit provides a basic implementation of support for LimeSDR USB
receivers based on LMS7002 chip. The solution has several limitations:
- passing parameters for tune LimeSDR receiver via command-line
options is not implemented;
- only hardcoded configuration is used (channel 0 of lower band LNA,
LMS_FMT_I16 format, gain, bandwidth and timeout);
- only one device is supported and it is not possible to select a
desired one in case if several devices are connected to the host.
Test: compare the output of the program for RTL and LimeSDR receivers.
Environment:
- RTL2832SDR dongle;
- LMS7002M based USB LimeSDR board;
- 800MHz-2200MHz omnidirectional antenna with SMA connector.
Procedure:
- connect RTL dongle to the host and start the program with the
following parameters:
$ ./dump1090 --device-type rtlsdr --interactive
- wait until several planes will be detected;
- stop the program, connect the LimeSDR board to the host and restart
the program with the following parameters:
$ ./dump1090 --device-type limesdr --interactive
- ensure that the same planes are detected.
Acceptance criteria: the same planes are detected using both receivers
and track information matches with information from the
FlightRadar24 application.
Signed-off-by: Gluttton <gluttton@ukr.net>
--fix --fix was a poor choice for this, as some existing configs
happened to specify --fix twice but only expected 1-bit correction.
So move 2-bit correction to a separate option.
"--fix" or "--fix --fix" now gives 1-bit correction.
"--fix-2bit" or "--fix --fix-2bit" or "--fix-2bit --fix" gives 2-bit correction.
Version bump to 3.8.2~dev
nb: messages with a correctable 2-bit error are only forwarded to network
clients that have enabled verbatim mode (and they will be forwarded with
the 2-bit error still present, so the downstream client must apply its own
correction if desired)
Oliver Jowett
VirusPilot
George Joseph
Determinant
Gluttton
Rosen Penev
Carlos Pizarro
Oliver Jowett
Michael Tatarinov