I am running a PI with raspberrypi 4.4.34 image, i.e not using a prebuilt image such as PIAware. I installed the related Flight Aware packages and it has been feeding FA data for a year both ADSB and MLAT.
continued - I don't know what I am posting which generates spam / bad content, very frustrating! I edited adsbexchange netcat maint sh to use a free port 5-1-1-8-4, then restarted script. I see the socat socket connections to dump1090-fa and adsbexchange-com but the process dies after a short while and restarts in the loop. Anyway to debug this further? Is the fact I have MLAT configured causing a problem ?
How about ADSBx MLAT-client? Did you install it from adsbexchange repository or from other source (like adsb-receiver project)?
No I have not installed the adsbexchange MLAT client. My PI is currently running fa-mlat-client, piaware, faup1090, dump1090-fa Will dump1090-fa work? I was going to add the adsbex MLAT client after I had verified it was working.
My bad. I see /usr/bin/mlat-client referenced in the mlat maint.sh script. Does this need to be running ? As of now I killed off the adsbe scripts.
Hum, it appears I don't have /usr/bin/mlat-client. I was sure the script was running when I first installed it.. maybe it was just sitting in the loop.
So, looking back setup script the mlat part failed to build package due to dependency missing. Corrected that, created mlat package, installed package. started mlat-client and netcat scripts Here are related processes piaware 535 1 0 2017 ? 1-00:25:06 /usr/bin/piaware -p /run/piaware/piaware.pid -plainlog -statusfile /run/piaware/status.json piaware 12901 535 0 Aug30 ? 00:18:38 /usr/lib/piaware/helpers/fa-mlat-client --input-connect localhost:30005 --input-type dump1090 --results beast,connect,localhost:30104 --results beast,listen,30105 --results ext_basestation,listen,30106 --udp-transport 70.42.6.225:8358:2702256895 piaware 20711 535 0 Aug17 ? 00:36:31 /usr/lib/piaware/helpers/faup1090 --net-bo-ipaddr localhost --net-bo-port 30005 --stdout --lat 37.895 --lon -122.164 dump1090 20693 1 26 Aug17 ? 3-18:28:25 /usr/bin/dump1090-fa --device-index 0 --gain -10 --ppm 0 --net-bo-port 30005 --max-range 360 --net --net-heartbeat 60 --net-ro-size 1000 --net-ro-interval 1 --net-ri-port 0 --net-ro-port 30002 --net-sbs-port 30003 --net-bi-port 30004,30104 --net-bo-port 30005 --json-location-accuracy 1 --lat 37.89460 --lon -122.16400 --write-json /run/dump1090-fa --quiet root 23854 19268 0 13:02 pts/0 00:00:00 /bin/sh /home/pi/adsb-exchange/adsbexchange-mlat_maint.sh root 23859 23854 1 13:03 pts/0 00:00:26 /usr/bin/python3.4 /usr/bin/mlat-client --input-type dump1090 --input-connect localhost:30005 --lat 37.89460 --lon -122.16400 --alt 670 --user mdebusschere-01 --server feed.adsbexchange.com:31090 --no-udp --results beast,connect,localhost:30104 root 23889 19268 0 13:12 pts/0 00:00:00 /bin/sh ./adsbexchange-netcat_maint.sh root 24009 23889 0 13:29 pts/0 00:00:00 /usr/bin/socat -u TCP:localhost:30005 TCP:feed.adsbexchange.com:51184
Output from socat - I assuming server side is closing is the socket. [email protected]:/home/pi/adsb-exchange# /usr/bin/socat -u TCP:localhost:30005 TCP:feed.adsbexchange.com:51184 2018/08/31 15:38:05 socat[24634] E write(6, 0x8ca270, 1008): Broken pipe Output from mlat-client Fri Aug 31 13:55:04 2018 mlat-client 0.2.6 starting up Fri Aug 31 13:55:04 2018 Connected to multilateration server at feed.adsbexchange.com:31090, handshaking Fri Aug 31 13:55:05 2018 Beast-format results connection with ::1:30104: connection established Fri Aug 31 13:55:14 2018 Server says: In-development v2 server. Expect odd behaviour. The multilateration server source code is available under the terms of the Affero GPL (v3 or later). You may obtain a copy of this server's source code at the following location: https://github.com/mutability/mlat-server Fri Aug 31 13:55:14 2018 Handshake complete. Fri Aug 31 13:55:14 2018 Compression: zlib2 Fri Aug 31 13:55:14 2018 UDP transport: disabled Fri Aug 31 13:55:14 2018 Split sync: disabled Fri Aug 31 13:55:14 2018 Input connected to localhost:30005 Fri Aug 31 13:55:14 2018 Input format changed to BEAST, 12MHz clock Fri Aug 31 14:10:05 2018 Receiver status: connected Fri Aug 31 14:10:05 2018 Server status: ready Fri Aug 31 14:10:05 2018 Receiver: 237.0 msg/s received 108.7 msg/s processed (46%) Fri Aug 31 14:10:05 2018 Server: 0.1 kB/s from server 1.2kB/s TCP to server 0.0kB/s UDP to server Fri Aug 31 14:10:05 2018 Results: 114.8 positions/minute Fri Aug 31 14:10:05 2018 Aircraft: 23 of 23 Mode S, 19 of 23 ADS-B used Fri Aug 31 14:25:05 2018 Receiver status: connected Fri Aug 31 14:25:05 2018 Server status: ready Fri Aug 31 14:25:05 2018 Receiver: 265.6 msg/s received 136.8 msg/s processed (52%) Fri Aug 31 14:25:05 2018 Server: 0.2 kB/s from server 1.6kB/s TCP to server 0.0kB/s UDP to server Fri Aug 31 14:25:05 2018 Results: 167.9 positions/minute Fri Aug 31 14:25:05 2018 Aircraft: 17 of 20 Mode S, 18 of 19 ADS-B used
Your Mlat configuration looks ok. Socat fails because that port (51184) is busy, the process dies and after a short while it is restarted by the loop in the script. Choose another port (if there is any free) or step back to the original port 30005.
Yes please use 30005 for now. I have to add more custom feeds and look through those with no data for release back to pool of unused.
Thanks, it appears the zConv-* are all busy where as a few of the reserved are not reporting. Tried back at port 30005 and it seem to be good for 10+ minutes then socket closed out, is that normal behavior? Is there anyway to verify if its working on port 30005? I looked at global view couldn't find anything that might let me find / filter data from my IP etc.
Global view is not returning receiver name. This is merged feed of all feeders not using custom port. Be patient, wait for new port range.
What do you consider the feeder image dump1090 ? I am using stock FltAware pieces for dump1090 /mlat and used the setup.sh from adsbexch. Wouldn't it be easier if we could send a userID aka receiver name in stream from dump1090 similar to mlat? I haven't looked at the code, just assuming additional meta data could be passed.. Then using global view we could use a filter where one could place limit by name(s).
That would require ADSBx to build a client. Right now ADSBx sends the unmolested raw beast - which is very useful to researchers and those who want the technical data out of the stream. If you want to write a client to interface with dump1090 and send receiver specific data - then have at it. Here is a link to git: https://github.com ... start coding and share the code with the community. Here is a list of what it will likely take. Write a server to ingest and process the client data that is sent. Completely rewrite a large portion of how VRS handles receivers (https://github.com/vradarserver/vrs). Make sure it can accept 'meta data' about your feed. Write a server to handle thousands of feeds, process, and serve them without crashing the global web ui - there's 27 instances of those. I suggest rewriting VRS in C++ or something that can handle a large amount of data and make sure it supports concurrency and shards so the web server can handle all ADSBx traffic. Write a web browser that can process a list of thousands of feeds without crashing. You can start with Chromium project: https://github.com/chromium/chromium Or You can accept that ADSBx is not FlightAware, and we are not making millions of dollars off selling feeder data. And you can accept that ADSBx does not accept money to block any aircraft, nor can we give you a 'commercial account' so you can view your own data on the ADSBx map. You can also accept that ADSBx does not offer you a green flashing light to show your feeding is 'on'. You can accept that sending data to ADSBx, then demanding to send that same data back to you so you can view it on a map is a waste of bandwidth, time, and money. View it on your Pi, every dump1090 has a map. You can install VRS and connect it to your dump1090. You can accept that ADSBx offers all data back to the feeder community for free or at cost of storage. Redshift is 14 billion position reports. JSON is 11-13GB per day compressed. You can feed to 32005 and not worry about having a green flashing light on our site that tells you the feeder is working. It's easy to tell if your feeder is sending data - if the shell script with netcat-maint has socat running then it's working. If you want to see the traffic your feeder sees, you can type the lan IP of your feeder into a web browser and see if the map comes up, if you have traffic then your feeder is working. If it is a remote feeder use zerotier.com. I will have custom feeds ports setup hopefully this week. I have to figure out where I have enough bandwidth to support it.
It was simply a suggestion for another approach to the testing setup, but if it worked and was efficient it could be used for all. It is interesting to see feeder overlap in data! I noticed this by using the "default" which sends the entire test map and I could see feeders using the test ports in my local area.. Looking at the current html desktop, the server and desktop HTML client already know how to send / consume the feed(s) clearly seen in the JSON response. It seems the server side just maps port to a name.. So I think the idea that everything needs to be a major rewrite is not necessary true. Anyways, I don't need a green flashing light. I understand how to see my local feed data. I find the ADSBx desktop HTML client superior to the rest - per the mission you can see all the data and more. I noticed the post on Golang support. I am interested in server side - especially something of this volume. I have considerable C (more Windows side) and Java experience with a lot of html/javascript mixed in, but like most people time is scarce. You mention using parts of dump1090 hence this provides an opportunity to build client & server side which allows for dynamic meta data to be included with raw data - no reason to break downstream. I know it's easier said then done. Appreciate all the work put into the current ADSBx - it is a great service.
"Looking at the current html desktop, the server and desktop HTML client already know how to send / consume the feed(s) clearly seen in the JSON response. It seems the server side just maps port to a name.. So I think the idea that everything needs to be a major rewrite is not necessary true." Sort of. VRS generates a bunch of things that aren't JSON. Images. Config files. Most of the VRS HTML web ui is not static files. You can't pull the HTML and feed it JSON and it will work. I've been trying. Feed processing is as follows: A reverse proxy server routes thousands of incoming feeder connections one of the open ports on 10 ingest servers (soon to be 12-14). Each of those 10 ingest servers merges all the incoming feeds and sends them to a pair of distribution servers that handle the connections from 30 web ui servers that server the front end web ui. There is no way to map 1 feed to all the global web ui servers and have it to select in the drop down. You'd have to map thousands of feeds to 30 separate web ui servers. Not mention a JavaScript UI drop down handling 1000's of entries without making every web browser unstable. Tracking an individual feeder would require a complete rewrite of how VRS handles connections and processes data. You'd have to write an entirely new receiver type. Also each 'receiver' can only receive data from one client and must be configured to specific port. That JSON you are looking at is a very small portion of the shenanigans VRS performs. The HTML web UI is simply VRS (Virtual Radar Server) http://www.virtualradarserver.co.uk/
Custom ports 51301 - 51449 are now open. https://customglobal4.adsbexchange.com/VirtualRadar/desktop.html Midnight here. I must get some sleep. Enjoy! Hopefully it doesn't break by morning.
