MLAT Sync Errors

I'm new to this forum, so first: "Hi, Everyone"

Background

I've repurposed an old RPi Model B (HW version 000d) for my ADS-B setup, and it is apparently running fine with FR24 (which was the only one I knew about initially). With fr24feed-status I'm told MLAT is working OK over UDP and it currently sees 35 AC out of 38 tracked AC, so all flags look ok from there.

I initially had it up and running with an old no-name RTL-SDR stick on the supplied antenna (mounted in the attic) but went through 2 more Franklin antennas (the last one worked well...) until I upgraded to the FA antenna and the Blue FA Pro Stick Plus; this all brought the max range out to some 200 nm with reasonable coverage (and only a few overloads due to my location right between EKRK and EKCH and overhead planes at low altitude sometimes).

Now, I discovered ADSB-Exchange, and thought I'd share data this way too (I'm all for open sharing), so I changed my custom raspbian install and put in dump1090-fa instead of the dump1090-mutability and installed the ADSB-Exchange clients to share, including the stats packages, and enabled MLAT there too.

It all went well, and I can also occasionally see planes are shown with MLAT position in the dump1090/tar1090 overviews. The RPi is running with a load of somewhere in the 55-65% CPU allocated to the dump1090-fa; I think previously dump1090-mutability ran with some 40% CPU load but I don't know if the additional load is due to more planes now being seen or some other differences between the two? The feeders and the MLAT share (FR24 and ADB-S Exchange) use some 5-10% additional CPU time, so it does not appear to overload the little RPi.

My issue

However, when I check my MLAT status on feeder.html?5A&T-EKRK104_53 I see a frighteningly high amount of Sync Errors, which I assume is due to timing inconsistencies between the participating receivers.

I have understood that the timing comes from counting frames from the RTL-SDR receiver and not due to the clock on the RPi (which is ntp synced, btw).

0) What is causing the Sync Errors? I've assume it's an inability to properly get a time sync based on ADS-B aircrafts, so essentially some kind of incorrect time-stamping in my setup.

1) Is this something to worry about (maybe my interpretation of the numbers is incorrect)?

2) Is this caused by the slightly underpowered RPi resulting in e.g. loss of data on the USB connection. I'm using a 4.8m high quality USB cable, which should be OK but maybe it simply cannot keep up with the datastream?

3) Or could the RTL-SDR dongle be faulty?

4) Should I do something to reduce the load caused by the dump1090-fa (or is it just the more data it receives that causes this)?

Any advice or comments would be appreciated, thanks!


-- Per.

 

I've replaced the cable with a shorter USB connection (0.8 m instead, or thereabout) and I've also tried stopping the fr24 feed, and it doesn't make any difference.

Occasionally, it works (in the sense that more peers show green), but it seems to be loosing the sync after a while for most peers when enough mutual sync packages has been exchanged between them.

Is there any way to check the status from dump1090-fa, to see if it looses packages?

is there any way to (at least temporarily) reduce the dump1090-fa load (making the b/w smaller, or use less error correction that in the default setup)?

I guess the alternative is to upgrade the RPi; I may have a RPi 2 lying around somewhere, which I guess should be able to do better?

 

Just to update the status: I dug out the slightly newer RPi (a model 2 B version 1.1, which is a 4-core ARMv7).

Now it works; my status looks like below. The CPU load is still quite high for the dump1090-fa (maybe even a bit more on the RPi 2B vs the RPi 1B, but as there are 4 cores (and maybe a better USB subsystem?), it seems fine still).

However, I don't see any incoming MLAT messages (yet) even if there have been planes without position, and so far I've only noticed on plane being displaying MLAT data in the dump1090-fa or tar1090 web i/f.

In the log file, it says like this:

Sep 07 23:35:55 fr24n adsbexchange-mlat[4616]: Receiver status: connected
Sep 07 23:35:55 fr24n adsbexchange-mlat[4616]: Server status: ready
Sep 07 23:35:55 fr24n adsbexchange-mlat[4616]: Receiver: 106.5 msg/s received 16.5 msg/s processed (15%)
Sep 07 23:35:55 fr24n adsbexchange-mlat[4616]: Server: 0.0 kB/s from server 0.2kB/s TCP to server 0.0kB/s UDP to server
Sep 07 23:35:55 fr24n adsbexchange-mlat[4616]: Aircraft: 2 of 6 Mode S, 5 of 8 ADS-B used​

Do I need to do something special to make sure I get all incoming MLAT data also`

 

Looks good. Nothing special is needed for unfiltered unblocked MLAT. Obviously the greater then range the better your feeder will perform and potentially more MLAT results. Most commercial jets and operators are ADS-B equipped. MLAT will be mostly military and other interesting aircraft depending on your location.

High CPU use is a bug in librtlsdr and should be fixed in the future.

Periodically run,

sudo apt update
sudo apt upgrade

 

Thanks James (and wiedehopf) for the help and comments; I'll keep it running as is then. And I guess if the bug is fixed, I could try again to see if the old RPI1B could be useful as another station then somewhere.

My range is out to some 130-170 nm with a max on 202 nm so far. This is on an indoor antenna, with some taller trees around, so not perfect but likely what I could expect, I think. Next step could be to move it outdoor...

And yes, I'll keep the system updated (been running on Unix and Linux devices since 1982, so I'm kind of used to the drill)

 

Unlikely I've tried on a Pi Zero W, maybe if you rolled back to Stretch but Buster is inefficient it seems.

 

@James: OK, thanks; as the Pi Zero W is based on the same SoC (BCM2835) as the RPi1B, it's then unlikely to have any positive impact on that platform either.

[email protected]: Thanks; I'll try to replace the library and see if improves.

I've noticed that as of today (it's now been running for more than 12 hours or so) that my range seems to have dropped quite a lot (max range is around 130 nm now, where it used to be somewhere in the 180-200 nm previously). I can also see that the ratio of positions vs contacts is significantly down. On the RPi1B with my homegrown antenna on a no-name dongle (Franklin) it was around 35%, and with the FA antenna and the Blue stick it increased to some 40% -- but now it's down to 25%.

I'm suspecting that the noise levels have increased; it looks like I loose contact around -26 dB (running with a fixed gain of 49.6 dB) but I seem to remember that it was around -31 dB on the RPi1B.

This could be due to the RPi now being closer to the antenna, it could be the power supply or the RPi2 itself. I'll probably start by swapping back to the previous power supply first...

But I'm wondering what is the easiest way of measuring the noise floor "in situ"; is there any way to get dump1090 to spit out the noise floor? Otherwise I guess it's going to be something like SpyServer and running some SDR locally on my Mac or Linux box to check what it sees.

The good thing is that MLAT now runs fine, as far as I can tell

 

MLAT got to work with the newer RPi2B vs the older RPi1B (the cable didn't make any difference to MLAT). But I forgot to check the combination of the new, shorter cable on the old RPi1B for range...

But yeah, I've replace several things at the same time, and that's why I wanted to measure the noise floor in different combinations instead of changing one part and waiting 12 h and then the next

 

OK, so just for the record: I swapped the PSUs but that didn't make any difference. I then swapped back to the long cable instead of the short one, and in that process I moved the RPi2B further away from the dongle also (that was the main reason for the longer cable in the first place).

That dropped the noise floor back some 6-7 dB(!) (as judged by CubicSDR running via SoapyRemote; at the particular settings I used, the noise floor went from around -71 dB to -78 dB).

So now my range is where it was before all of this started, and I'm tracking flights out to a little more than 200 nm on my indoor attic antenna in somewhat shaded terrain from foilage...

Whether it's the actual cable or the proximity of the dongle and the RPi2B, I don't know. But I'm more happy now that I got MLAT working and maintained my range

I've since added a ferrite core to the USB extender close to the dongle, for good measure, to kill any stray RF noise that travels along the cable. Not sure if it helps, but it cannot hurt.

So apparently in some cases, it is BETTER to have a longer cable between the RPi and the dongle as it may be part of reducing the RFI and the increased noise floor it may cause.

Thanks for the help!

 

@widehopf: Yes, that's a good idea -- although as it could be the RPi2B itself or the PSU or both, I'd need to place both of them under the antenna.

For now, it seems to work OK, so I'll probably just leave it running for some time to see how stable it is before I start the next round of experiments.

I've just set up the old RPi1B on my Franklin antenna, running in parallel (but placed some m away for logistic reasons); it obviously doesn't feed anything, just running dump1090-fa for comparison. It works surprisingly well, with a lower noise floor at the same gain but also less sensitivity. Increasing the gain brings the sensitivity sort of "on par" but it's not tracking positions as well.

Anyway, I'm awaiting delivery of a RTL-SDR v1.3 dongle soon, and want to see how that compares with the Blue dongle.

Thanks again!

 

Yes, it's obviously designed to be a more sensitive/better suited device (due to the LNA and SAW) for ADS-B in the 1090 MHz band It's just good to verify that it actually is also...

The most surprising thing is maybe that my Franklin antenne seems to perform quite well. The no-name dongle with the quickly made Franklin picks up around 70% of the messages that the blue and the FA stick does, occasionally also out to almost 200 nm, and sees around 75% of the planes (same both w/ and w/o position). On average, however, the number of positions reported is only 55% so of course the blue and the FA stick is better, as it should be, with an effective coverage maybe some 40 nm more, I think.

But in reality, I'm more intrigued to see how much better the RTL-SDR v3 dongle is than the no-name currently installed When I get time, I'll try to swap them out on my second local "feed", but I need to change the connector on the coax first.

 

Just a quick update: In the process of testing a few other things on the old RPi1B, I also installed the librtlsdr that @wiedehopf linked to above. It does in fact reduce the CPU load some 10% on the old machine.

Accidentally, it booted up with the MLAT running -- and MLAT it works and appear to run stable. That machine is no longer feeding FR24 nor FA, but it currently runs feeding the adsbexchange incl. MLAT; I'm using it to test antennas currently (it sees a little better in one direction than my main RPi2B based setup).

Not sure if it's also removing the FR24 feed (which took some 5-10% CPU), but anyone that has an old RPi1B should probably also use the less CPU consuming librtlsdr. In my case, it appears to run just fine with MLAT then.

I'm wondering what the cause of the additional CPU load is; I went through the commits, but didn't find any mention of this. Is it related to the efficiency of the USB transfer, or some such?

 

Ah, OK -- thanks for the explanation!

There are always some weird issues somewhere in the underlying libraries/platform, which are often unexpected and quite difficult to find... :-(