Out-of-order position reports

I use a Perl script to take a feed from adsbexchange.com, parse the received JSON data, and put the individual position reports into a MySQL database to display on a Google map in real-time or later. The data seems to be error-free except for the time (JSON 'PosTime' item).

Some position reports are delayed by a few seconds and this causes them to be displayed later. It's particularly obvious when the points of the position reports are connected by a line that should represent the flight path. The plotted data is "zig-zagged" back and forth -- i.e. not a smooth progression.

Here is an example of this phenomenon. Hover over an aircraft icon or green dot icon or scroll down to view the tabular data). Note that the fourth and fifth position reports' should be reversed in order.

Code:

2018-12-31   00:34:10    44.8654° N    123.1983° W    0°    43    194'
2018-12-31   00:35:13    44.8623° N    123.1983° W    0°    67    219'

The tip off here is the Speed (knots) parameter: the aircraft was on landing roll and the position of the 67-knot speed should precede the 43-knot one. (I should know: that's my airplane <g>.) Also, the 00:35:13 report time of the last position report was approximately 30 seconds after the aircraft cleared the runway (as verified by APRS position reports, a whole different topic BTW).

This raises the question: what is the root cause and how can I fix it? Could this be due to different adsbexchange.com feeders having unsynced clocks? Or network or some other latency? Or is the JSON item 'PosTime' not rigorous enough for my purposes?

My own feeder to adsbexchange.com is less than a mile away from the above example if that is any kind of a factor.

Thanks for any insight anyone can shed on this.

--
Joe

 

Thank you for your reply, Marcin. And thanks for helping me out earlier with posting my original message.

True enough. But that should shield the aircraft's ADS-B transmissions from all except nearby receivers and preclude any MLAT.

This cannot be so. My own feeder receiver is located on the airport, less than 1 NM away from the reports in the URLs I gave. That is the reason (or at least part of the reason) why ADS-B coverage was so good close to the ground in my posting.

This I can easily believe. But what receiver(s) could be in play except for my own nearby one when the aircraft is virtually on the ground as in this case. (However, this same problem frequently occurs at higher altitudes too.)

Certainly, I can accept this. I would gladly remove out-of-area messages if I could identify them. In this case, the message in question was simply timestamped a few seconds late.

Some other information that may or may not be relevant:

1. All of this activity was on 1090 MHz although 987 MHz ADS-B is also in use here in the USA. The aircraft in question does not use 978 MHz except for ADS-B IN (receive only).
   
2. The aircraft was located about 5 NM from an FAA basestation which purportedly retransmits ADS-B on occasion. This aircraft is configured to require no retransmission; however, the basestation may have retransmitted a report for a different aircraft and it was picked up by a feeder receiver.

For many, many years APRS has had a similar problem (delayed packets). However, I've generally been able to identify and remove them. If you or anyone can offer some advice on an algorithm to do this for ADS-B, I would appreciate it.

 

Following up to my own postings . . .

I have changed my Perl script to discard position reports in which "Mlat" is true. Made no difference as far as I could see. Still seriously broken.

Wish there was a field that would specify which feeder provided the report so I could get a clue where to look further for the timing issues. If it was mine I would fix it; if someone else's I would discard all from that particular feeder (that's what I do on APRS with a few "bad actors" polluting the data).

Any good ideas, anyone?

 

What end point are you hitting the REST API or?

 

Some background that might help on this. The 'global' web ui and the aircraftJSON are fed from 30 servers all running a copy of VRS. So they may be slightly out of sync due to network latency and load - blah blah blah.

We 'stick' a web browser using a cookie to tell it what global to request. Since VRS looks for an idv and timestamp in the request, if it doesn't find it then weird things happen.

So if you are trying to hit, https://global.adsbexchange.com/VirtualRadar/AircraftList.json and ignoring the cookie the Proxy server sends in the response ... then that might explain it ... hard to say without really investigating further ...

 

https://flight-data.adsbexchange.com/map?icao=A1FBCE&date=2018-12-21

There's some consistent errors in your lat/lon ... that is interesting. What are you running on your feeder?

 

Yes! And I'm starting to believe that what I thought were timestamp errors are actually position errors.

First of all, there is no good reason for that point located out in the Atlantic. It happens frequently -- for example on 2019-01-12. The longitude is off by about 90 degrees. And that aircraft (ICAO A1FBCE) has always had good ADS-B Performance Reports from the FAA.

The bogus point in the ocean is undoubtedly coming from my own feeder because it's the only one capable of hearing that aircraft on the ground (the receiver is in the hangar).

An RTL-SDR and homebrew colinear antenna on the RF side. The computer hardware is an old Windows-7 PC. The software is DUMP1090 feeding ModeSMixer which feeds VRS (version 2.4.2.29243) and provides a feed to FlightRadar24. VRS provides three feeds, one of which is ADS-B Exchange.

No, not global.adsbexchange.com but http://public-api.adsbexchange.com/...g=-123.1923&EngType=1&WTC=0&fDstL=0&fDstU=500 This is an attempt to show G.A. aircraft within 500 km of here and ignore airliners. It's actually somewhat more complicated than that because I rotate that URL with two others to track two specific aircraft as well as all aircraft within 500 km. This is done in the Perl script I mentioned in the original post.

Thanks for taking an interest in this, James.

--
Joe

 

You'll need the keep track of the cookie. That might explain some of the issue.

The Lat/Lon ... no need to feed from VRS, VRS causes all kinds of issues. If you do, use a rebroadcast passthrough from the dump1090 beast port (default 30005).

You probably aren't running MLAT client on windows. But still need to sync the clock with NTP frequently.

I need to send you a Pi.

 

Notice the cookie.

How often do you ping it? The REST API might give better results.

 

Frequent NTP under Windows as you suggest in your next posting.

But it mostly shouldn't matter because the timestamp of a position reports is contained in its PosTime data field (e.g. "PosTime":1478998683550). My Perl script converts the ADSB reported time (PosTime) from epoch time to GMT in MySQL format for displaying.

 

About every ten seconds.

Regarding the REST API, I'm concerned it would involve a major code re-write. There's a lot I don't understand about this process, VRS, even the cookie you speak of. Many of the data fields don't seem to work as documented (i.e. in filtering out airliners by wake turbulence category). I think I'm in over my head. The truth is, I'm a better pilot than a software engineer Perhaps I'll have to settle for "good enough".

 

Well, VRS is supposed to do a lot of things, it does some well, other things not so much.

We're currently dealing with ACARS sat feeds and VRS Web UI not handling them as SAT feeds, even though we configure them as such and etc etc ... therefore they 'stick' on the map ..

I'll write some code and PM you in the next week or two.