Problems feeding after reboot with script

Discussion in 'Feeding' started by TrashPanda69, Feb 14, 2019.

  1. TrashPanda69

    TrashPanda69 New Member

    I have a bare Raspian install to which I installed the PiAware application, along with the ADSBX Script, FR24 Application and OpenSky Application

    I installed the script and ended up with an output of:


    Checking for packages needed to run this script...

    Traceback (most recent call last):
    File "<string>", line 1, in <module>
    IndexError: list index out of range


    I asked around, but then a little while later I showed up on the coverage-2 page with MLAT data (TrashPanda69)

    Today I needed to shutdown my Raspberry Pi, so I shut it down via SSH and powered it back up around an hour later. All of my FlightAware, FR24 and OpenSky feeds are working fine, but I don't see myself on the map anymore

    I looked through what I could, and it seems everything is working as it should. The startup scripts are located in rc.local, and if I do a packet capture, I can see data being sent to ADSBX

    Has anyone got any ideas what might be wrong?
     
  2. MDA

    MDA Administrator Staff Member

    Nothing is wrong.
    This message is stretch related but everything works fine.
    Thanks for feeding.
     
  3. TrashPanda69

    TrashPanda69 New Member

    Can you confirm that my username is actually feeding any data? I just wonder why it no longer shows on the map when it did before
     
  4. MDA

    MDA Administrator Staff Member

    Look at sync-matrix, sometimes it takes long time to connect.
    You can check mlat client status:
    Code:
    nano /var/log/daemon.log
    If running results should be like:
    upload_2019-2-15_0-42-8.png
     
  5. James

    James Guest

    Message me the external IP of the Pi and I can check direct on the feed servers.
     
  6. TrashPanda69

    TrashPanda69 New Member

    Thanks, it looks like its not connecting at all

    I get lost connection to the feeds dot adsbexchange on port 31090 and then it just tries to reconnect over and over again
     
  7. James

    James Guest

    It's working .. [edit] :D


    0x4b69150: [14/Feb/2019:20:50:38.472805] id=327589 proto=tcpv4 source=YOURIP:24835
    flags=0xce, conn_retries=3, srv_conn=0x2c33150, pend_pos=(nil)
    frontend=beast_front (id=2 mode=tcp), listener=? (id=1) addr=167.114.60.74:30005

    --

    0x4c9fc60: [14/Feb/2019:23:46:51.958246] id=496365 proto=tcpv4 source=YOURIP:45108
    flags=0xce, conn_retries=3, srv_conn=0x2bbae50, pend_pos=(nil)
    frontend=MLAT_front (id=12 mode=tcp), listener=? (id=1) addr=167.114.60.74:31090
     
    Last edited by a moderator: Feb 15, 2019
  8. James

    James Guest

    MLAT servers are already on a 5GHz water cooled machines ... I think what might need to happen is we spread them out across more machines and roll of the dice load balance them ... if you get in a group that does good MLAT in local geography .. wooohoo .. if not .. so sorry ...
     
  9. James

    James Guest

    I should mention that if anyone has suggestion to make the feeding process easier. Dan and I are open to talking about it.

    The problem is most want to feed FA and FR24, so we have to work within the constraints of those feeder images.

    Which means, an ADSBx feeder has have to have a config file or setup that is not using FA, FR24, etc ways of doing things. MLAT is a pain in the ass, if it was just 1090 ADSB it would be very easy - one bash script from dump1090 and done.

    Minimum fields we need to make a feeder work.

    Latitude
    Longitude
    Altitude
    MLAT Name of Feeder
    ADSBx Feed Port

    ADSBx Feed Port is 30005 unless you want to see your own data on a custom feed server, there are interesting things we can do with the custom ports.

    Unfortunately only about 50% of the people can follow 4 sentences of basic directions on how to setup a custom port.

    I'm not sure how to explain it easier, pick a port with no data then feed to that port and if you see your data then you are good to go. Send a request and wait for me to take time out of my week to manually tag your port. What most frustrating is people will setup a custom port feed, then if I don't immediately change the text on the feeder to their request - they stop feeding it! Seriously, just keep feeding the port. IF you see your data, keep feeding it. Does it matter if it says zCons or BillBobFeeder?

    One thing ADSBx will NEVER have is a feeder dashboard with stats of your feeds.

    That would require redesigning the entire infrastructure, replacing the VRS map completely as well as the custom feed servers. VRS webserver and memory mangement is trash, so even if I figured out how to pull data from just 'your' feed and place it somewhere. The amount of data generated would cost at least $1000 a month for a server, and probably several thousand to maintain. Is this something we need to spend donations on?
     
    Last edited by a moderator: Feb 17, 2019
  10. TrashPanda69

    TrashPanda69 New Member

    My complains with the entire process are:
    • The error (That turned out to be nothing) at the end of the script, with no success or fail. I had no idea if it worked
    • There is a spelling mistake when it asks for "atitude" and then also doesn't specify the units either. Meters, feet, inches?. While this isn't a big deal, it doesn't fill be with confidence
    • Once the script is done, there is no way to actually verify its working. All of the other feeds give you instant feedback where you can view stats. Being able to view SOMETHING would be nice, even if its just a "yes it works" message
    • The messages filling the log file with the failure to feed, which are also inaccurate. It would be nice if they could go away
    Other than that, I don't have any issues with feeding. Here are some other suggestions off the top of my head. Some may be good, some may be bad. I am just throwing them all out there

    • The forum has too much auto-moderation when you sign up. Even just pasting log files stops you from posting, and it seems if it includes a URL it won't let you post. It was so frustrating making my first post and PM'ing you guys
    • It would be AWESOME if there was some way to make the sites infrastructure possible on some kind of distributed computing. Me, and I bet a ton of people would donate resources this way, it would be cool. I assume it would be hard to set up, but I'm just throwing it out there. I would gladly spin up a VM with a 10 cores and a decent chunk of RAM to support the project
    • It would be really nice if you had a map showing where coverage was really needed. I would be way more included to get my setup running perfectly if you really needed a feed in my area
    • Support would be a lot easier on a Discord or Slack channel. That would be awesome. It doesn't even have to be ADSBX specific, maybe someone should make a general ASDB group that we can all be active on?
     
    James likes this.
  11. James

    James Guest

    I can address a few:
    • Forum moderation I can't change due to Russian/Chinese/India Dick Pill spammers. The destroyed the other forum completely. Then the complaining emails start about the spammers. This is the only solution that works so far. Maybe we start a reddit or something ..

    • Not possible due to the nature of the data and how it must be synced. It's not just the VM with 10 cores, it's the amount of data (bandwidth and messages) that must be processed and merged in a timely manner. I really don't want to deal with 100's of emails bitching and moaning that a plane is 10 seconds behind where they see it in the sky. Because it happens. People don't understand the amount of work that goes into making all this work AND trying to generate money for it all. My job let's me be pretty flexible, but I'm spending no joke 20 hours a week at least, closer to 30 hours just on ADSBx.

    • Coverage Map. This we could do by pulling data from Reshift or the Datastore. Problem is ... I don't have time to figure out how to do it and visualize it.

    • I agree. A channel would be interesting. We have a slack but slack is individual invite only - PITA. I created a Discord Server - https://discord.gg/P6QZ5KA
     
  12. Robert

    Robert New Member

    Howdy, newbie here tagging along. In the last few weeks I've set up my station's hardware and software. I started with a fresh image of Stretch and began with dump1090-mutability 1.15. FR24 was my first feeder, then I added FA, and just today I added ADSBX. I ran into a few issues with getting the adsbexchange script to start at boot, but I think I've got that part sorted. I can see entries in daemon.log that show data being received by the script and sent to the server. However, my script shows a custom port (found one with the basic instructions - 51451), but I don't see any data on that port's VRS map. I have local dump1090 data, and both of my other feeders are showing data connections too.

    Not sure where I should begin to troubleshoot this, but this thread looks like a good place to start. If I should post separately let me know!

    For context, I'm feeding directly from the Pi, not routing it through an instance of VRS. Also this is a wired connection (powering the pi with a PoE uplink), but I don't think that's the issue because the daemon.log shows a successful connection and I can see my station's MLAT data on the sync page. Any tips?
     
  13. TrashPanda69

    TrashPanda69 New Member

    Check out the discord channel in the post above you
     
  14. James

    James Guest

    " I ran into a few issues with getting the adsbexchange script to start at boot"

    What do you mean you ran into a few issues? There should be no issues, script method creates 2 bash scripts and and entry in rc.local.
     
  15. Robert

    Robert New Member

    The issue was that the adsbexchange-socat_maint.sh script wouldn't start with the mlat client at bootup. I don't know why, otherwise I wouldn't have posted the question! All I know is that the mlat sync page was showing my data, but the Pi wasn't connecting to (or even trying to connect) to the adsbexchange server. Doesn't matter... I reformatted and went with a fresh install of the adsbreceiver image. Seems to be working well now, I'm about to change the port no. on the script to feed to a custom page.
     
  16. James

    James Guest

    Socat script has nothing to do with MLAT client script.

    socat script runs the statement below inside a while loop, so it restarts if it fails:

    socat -u TCP:localhost:30005 TCP:feed.adsbexchange.com:30005

    mlat script starts the mlat client and is not dependent on socat or any of the other scripts.


    Interesting .. What image did you start with that wasn't working?
     
  17. Robert

    Robert New Member

    Right, I figured that they were independent based on the code. But they should still both start at startup, no? I started the Pi with a fresh image of Stretch lite maybe 3 or 4 weeks ago when I started feeding to FR24. I mangled those first few installs a bit too (like I said, newbie here), so once the ADSBX scripts didn't work properly it was a good sign to start all over again. Didn't take long, and now all 3 feeders (FR24, FA, and ADSBX) are working as far as I can tell. But like the OP of this thread stated, it's difficult to tell if the data is actually being received by the server. my daemon.log shows connections, processing, and uplinks, but until I can get the custom port feed working, there's no way for me to know for sure.

    I just rebooted my Pi with the TCP:54154 edit in the socat script, but the custom VRS page for that receiver is still showing 0 flights/traffic.
     
  18. James

    James Guest

    I don' think daemon.log is the place to be looking ... it's not difficult .. if planes appear then you are sending data .. there a few thousand feed deployed .. very few have issues 99% are user error ..

    Not unexpected - FR24 breaks every other feeder by turning off the beast output when FR24 starts dump1090. I have no good things to say about FR24.

    What is 54154 ?

    51xxx ports are for beast.


    try running the command manually and see if data comes up.

    socat -u TCP:localhost:30005 TCP:feed.adsbexchange.com:51141

    https://customglobal.adsbexchange.com/VirtualRadar/desktop.html#

    select zCons-51141

    If that doesn't work, error messages will show why.

    If that works then htop and see if the scripts are running. Raspbian emulates rc.local using a system.d service.

    https://www.raspberrypi.org/documentation/linux/usage/rc-local.md



    Message me your external IP and I will see if we have connections from you.
     
    Last edited by a moderator: Feb 24, 2019
  19. Robert

    Robert New Member

    Got it to work, thanks! When I said "54154", that was a typo. In the script I had actually entered "51451", which is on the CustomGlobal4 feeder map. My pi was connecting, sending data, and I could see MLAT data on . the sync page, but no planes were showing up. I switched the port to the one you suggested, and voila. That's the only thing I changed. Is port 51451 not meant for feeds from N.A.?

    Anyway, thanks for the assistance. All up and running now.
     
  20. James

    James Guest

    There's someone connected to 51451, at least according to the Proxy server.