I can look at that ... I think it only gets parsed every 30 seconds and it might be set to 'average' it
Several of the "system" charts seem to have challenges: System load - my current system load shows 0.81, 1.03, 1.07... Grafana says 111.0%, 106.7%, 26.2%. My root disk shows 4% use (59G size, 1.9G used, 56G available) but Grafana says 47.10% free. Now, the next challenge... figuring out what all is being written to the SD card and, wherever possible, moving it to a RAM drive.
It's all in ram. Grafana wont match exact system loads because it's snapshots and intervals of up to 30 secs. Using... https://github.com/prometheus/node_exporter
Not all of the feeders are so enlightened. FR24 writes stats to the drive every 5 minutes unless disabled, and OpenSky dumps stuff to syslog fairly often. Just have to make a few tweaks to fix that. I really despite microSD cards for primary storage.
Hah. FR24 also breaks other feeds with updates. OpenSky has a lot of bugs and issues - but it is what it is. You can tweak prometheus if you want to store a long time series - just don't run it out of ram.
PS it might be writing some logs I missed. Looking at the pi sero there are some 2kb-6kb writes every now and then -- sounds like a system log somewhere But I don't know if that detects the ram drive writes
If you are not interested in the logs you can switch a lot off using a log configuration setting. Edit the file /etc/rsyslog.conf and just after the section starting ############### #### RULES #### ############### erase everything below and add the following line. *.* ~ ... that should work sudo nano /etc/rsyslog.conf etc etc You could also set logs to write to /var/run which should be a ram drive or write a script to write them to the ram drive, zip, and upload to a remote server every x hours.
At any rate even with logs there isn't a meaningful amount of writing being done that would wear SD card out. A power loss would be more dangerous during a write cycle.
I disable the FR24 updater from /etc/cron.d, so it only updates when I update via apt from their repo. That way I can check each destination afterward. The last iteration of this feeder was up for a touch over a year with no issues. I updated monthly during that period, with a reboot, and had no problems. I don't mind a reasonable level of writes... but hitting the card every 5 minutes to write out the number of aircraft being tracked is farking useless.
I actually had to replace the card when I installed the new image! I imaged my old card with Etcher, booted it up, all was good, so I ran an apt update/upgrade. About halfway through, the card went read-only and it's permanently locked (and apt failed spectacularly, since it was right in the middle of installing the latest kernel image). And that was a good Samsung EVO card, too. The card was probably 18 months old, having run PiAware to start, then one of the scripted multi-destination (including ADSBX) installs.
Grafana might be writing to a log on the disk as well. I'll investigate more - everything else should writing to the ram disk.
You can also mount a tmpfs for /var then edit /usr/lib/tmpfiles.d/var.conf Configure it to write to your tmpfs aka ram drive ... but according to this you need to create the locations your services will look for a few solutions here: https://www.raspberrypi.org/forums/viewtopic.php?t=34820
Yeah there is a lot of stuff on my image setup to write to /run .. which should be ram unless I missed something
I believe if you are using the newest Raspbian like ADSBx image does then /run is linked to /var/run .. which is automatically mounted ram drive. I think the SD card issue where Jessie and before. ADSBx writes the .json files to /run/dump109-mutability .. so that should take care of most of the write cycles .. then if you disable logs ... should be near bullet proof
May I suggest three things: Use a WD Pi drive for primary storage instead of SD card. Use a smaller Samsung EVO plus sd card 32Gb Max Use zram
There are a few things we should kill I suppose. lighttps will log on error. comment out the log entry in /etc/lighttpd/lighttpd.conf There are probably a few others to run off if you really want a zero SD write image. (read only). We'll need to create a tmpfs for /var at boot and create the appropriate system directories and configurations.
At the moment 16G microSD card is ca. 8USD. Burn image, test it, make backup. Restore backup if necessary. RAM on pi can not be easily exchanged. TOSHIBA cards are relatively good and not so expesive.
I agree that $8 every few years trumps the trouble of redesigning an operating system to write to RAM. Tho the Pi Wifi (or another component) is likely to fail long before the RAM. "DRAM uses normal capacitors that are directly connected to the circuit and are constantly discharged and recharged. Solid state semiconductor capacitors don't really have any appreciable life cycle limit. Of course being ordinary capacitors, and very tiny ones, they drain and lose their contents in nanoseconds when unpowered. MTBF is roughly 350 years on DRAM. Flash uses a floating gate transistor that permanently stores a charge in a totally electrically insulated floating gate that isn't connected to anything. Hot carrier injection or tunneling is used to force electrons though the insulator and into the floating gate at very high voltages where they are essentially trapped forever (until reprogrammed again) and electrostatically cause the non floating gate to be on or off depending on the content of the charge trapped in the floating gate. Thus representing one or zero. The high voltage and tunneling of electrons through the insulator breaks down the dielectric insulator with each erase/program cycle as electrons tunnel through the material and change the molecular state and electron composition. Basically the high voltage tunneling poisons the insulator with free electrons (?) until the dielectric starts becoming a conductor and progressively leaking so much that the floating gate can no longer hold an isolated charge. It just conducts and drains instantly through the insulator when you try to fill it and can't ever be a "1" again. The thinner the insulator (eg smaller die feature size = higher density) the less time it takes for the insulator to be destroyed, hence the declining write cycles with each process shrink. In higher capacity and second and third generation SSDs it's no longer a concern due to the industry now understanding things like wear leveling, write amplication, etc. A device with 120 GB and 10,000 rewrite cycles would have to write 120 GB of data before each block had a write count of one, rince and repeat 9,999 more times, and you end up with decades of typical use."
James, you don't need to explain it to me. I wanted to be an "devil's advocate" . Anyway backup is always good idea.
It was for everyone else! I would guess that most SD failures come in 2 categories -- manufacturing defect and wrecking the SD during burn or write cycle by powering off. I'm not sure how much wear leveling is done and SD wear would depend on the amount of data written and the write cycle. aircraft.json probably every second probably not smart and will eat an SD in a few years even with wear leveling. Occasional log writes not going to appreciably affect the SD unless we're talking 5 years. Everyone makes a big deal out of it but I usually get 20 or so 8GB SD for $2-3 each on a clearance deal every year. Honestly .. if we get more than 2 years out of an SD or a Pi -- that's great! Read Only Script - anyone brave enough to test? https://github.com/adafruit/Raspberry-Pi-Installer-Scripts/blob/master/read-only-fs.sh FYI it's not going to work out of the box - but gives a good idea on how to do it.
