These messages duplicate ADS-B messages so it's not clear why
they are being interrogated for, and we don't really trust them
enough to do anything with the position information. But
recognizing/decoding them where possible does let us exclude other
Comm-B message types, avoiding some false positives.
This reinstates the fastpath in the 2.4MHz demodulator that
aborts message demodulation early if the DF bits don't match a message
type that we know how to decode, or stops early if the DF bits can
only correspond to a short 56-bit message.
Do this in a more general form where we test against a set of valid
DF values, rather than a switch/case. Then populate the sets based on
the current error-correction settings (e.g. if we are allowed to repair
DF damage with 1 bit error correction, then a DF17 message could appear
with any of DF=17, DF=16, DF=19, DF=21, DF=25, or DF=1 and still be
correctable to a valid DF17 message)
In the default case this produces a small CPU improvement as short
messages might be able to stop earlier and a few DF values can be ignored
early. With --no-fix-df it produces a larger improvement as relatively few
DF values are valid in that mode.
To further reduce CPU, the default behaviour has changed to _not_
decode DF24 messages (Comm-D ELM messages). These are rarely seen in
the wild and dump1090 can't do anything useful with them. Disabling
them allows us to further reduce the set of valid DF values as they
effectively use all DF values from 24..31, 25% of all possible values.
To re-enable DF24 decoding, use the new `--enable-df24` option.
Finally, avoid doing some CRC calculations twice. This was only happening
once per valid decoded message, not per demodulation attempt, so it's not
such a large win.
Usually, all message candidates are speculatively corrected as if
they were DF11/17/18, even if the DF field has a different value. This
allows correction of messages when there is correctable damage to the
DF field.
However, running every message candidate through a couple of CRC checks
is expensive, CPU-wise, when decoding messages off the air, as there are
a large number of message candidates that are actually just junk data, and
computing CRCs for all of those adds up. The --no-fix-df option allows
disabling this sort of correction. The tradeoff is that messages with
damage to the DF field will not be corrected.
DF18 messages don't have a CA field so we can't set airground from that.
That meant that aircraft tracking wouldn't update airground state at all from a DF18
airborne position message, even if the existing tracked airground state was stale.
It seems reasonable to set at least AG_UNCERTAIN if we're seeing airborne positions
with valid altitude data.
Fixes github issue #113
Major changes:
Try to error-correct all messages that potentially could be DF11/17/18
even if the original DF value is different. This allows messages with
correctable damage in the first 5 bits to be correctable.
Track recently-seen DF18 addresses separately to DF17 addresses.
DF18 does not imply Mode S support, so we don't want to treat it like
a known Mode S emitter, but once we've heard some DF18 for an aircraft
we can be more confident about future DF18 messages for the same
address.
Rework the scoring system so it's just a big enum that lists all the
possible outcomes in the order that we want. This makes the relative
ordering of different messages clearer, and makes it easier to move
messages above/below the accept thresholds as needed.
Don't accept 2-bit-error-corrected messages that are from aircraft we
have not previously seen. This greatly reduces the number of garbage
messages when using 2-bit error correction.
Overall results are:
* more CPU required for decoding (approx 30% increase in my tests)
as we're doing a lot more speculative CRC-checking work
* no significant change to message rates with error correction off
* about 5% more 1-bit-corrected DF17 decodes, with a
disproportionate increase in those messages contributing to
successful position decodes / unique aircraft counts
* _fewer_ decodes with 2-bit correction (versus old code with 2-bit
correction), but the message quality is substantially improved,
the rate of garbage decodes / phantom aircraft is greatly reduced
sample stats:
1-bit correction, old code:
141158 total usable messages
137852 accepted with correct CRC
3306 accepted with 1-bit error repaired
27446 DF17 messages
51 unique aircraft tracks
1-bit correction, new code:
141296 total usable messages
137854 accepted with correct CRC
3442 accepted with 1-bit error repaired
27528 DF17 messages
55 unique aircraft tracks
2-bit correction, old code:
142656 total usable messages
137809 accepted with correct CRC
3283 accepted with 1-bit error repaired
1564 accepted with 2-bit error repaired
28803 DF17 messages
349 unique aircraft tracks (<- note that most of these are garbage)
2-bit correction, new code:
142426 total usable messages
137822 accepted with correct CRC
3420 accepted with 1-bit error repaired
1184 accepted with 2-bit error repaired
28666 DF17 messages
55 unique aircraft tracks
residual, use the full 24-bit syndrome.
Previously this would mask off the low 7 bits, which isn't particularly
great - it doesn't really make sense in that it implies a somewhat
random IID value, and it would actually only correct 20 possible bit
errors out of the 51 possible errors, 16 of which were errors in
the top 16 bits of the CRC itself. It is also risky as it will accept
a larger number of possible garbage messages as the lower 7 bits may take
any value.
Instead, use the full syndrome, assuming that the damaged message
had IID=0 - which seems more likely than assuming a random IID, since
IID=0 is used for acquisition squitters and should be arriving regularly.
The reduced rate of corrections for DF11 messages shouldn't have
much of an impact as DF11s carry very little data themselves - they
are mainly used to acquire the aircraft address. Once one good message
for an aircraft turns up (which would require IID=0 anyway before we'd
accept it) it doesn't really matter if we discard more damaged messages,
as they're not contributing to anything useful in the aircraft state
beyond air/ground status, which is also carried in many other messages.
A "reliable" message is a DF17 or DF11 with good CRC.
A "reliable" aircraft is one that's probably real - we decide it's reliable when
we've seen enough reliable messages, or just enough messages in total, that it's unlikely
to be noise.
DF18 transponders don't necessarily do Mode S, so we shouldn't assume
that Mode S messages from that address are valid merely because we
heard a DF18. If it really is Mode S equipped, we should hear a
DF11 at some point.
The old matching process which tracked mode A values as pseudo-aircraft
got very, very expensive with a large number of mode A/C messages (and
with lots of single-bit errors, which seems common with a Beast doing
the reception)
Instead just count A/C messages directly into a 4096-entry array (which
is very fast) and periodically scan the mode S aircraft list to see if
we can match anything up (which is fixed overhead + cost proportional
to the number of mode S aircraft)
With forced inlining this is about as fast, and it is much less
errorprone than the twisty little maze of handcoded bitshifts that
it was before.
(notably, at least one error - in the ACAS RI field - has been fixed)
Oliver Jowett
Oliver Jowett
henry1952
Eric Tran