RPi- keeping it running continuously

SatNOGS Software
1

Hello!

The raspberry pi used to run my ground station is stored in an outdoor enclosure near my building. I noticed that every few days the raspberry pi needs to be unplugged and plugged back in because it automatically shuts off. Does anyone have any recommendations as to how to avoid having to manually unplug the system? I was considering implementing a cron job to automatically reboot the system every night at midnight but I also know that the date and time needs to manually be set because there is no internal clock on the RPi.

I’m just trying to keep my ground station online continuously without having to unlock the enclosure every few days.

Any advice or insight helps! Thanks!

2

This sounds a bit odd. I have an RPi running in a currently cold, damp loft without too much bother. It does fall over every now and again but it is reasonably rare. Is there an external factor, like power, tht is causing this?

You really shouldn’t need to reboot, so something is not quite right there

3

the unit it is in is temperature controlled…

The power supply is one that was designed specifically for use with the RPi so I would hope that it isn’t power related.

My biggest thing is that every time it is rebooted the clock is stopped for eight seconds putting it behind, so the next time it reboots it’s then 16 seconds behind and so on. I’ve been researching ways for it to set the clock each time it reboots via the internet but nothing seems to be working.

4

This doesn’t seem right at all. So we can rule out power, temperature and probably humidty.I’m guessing you used the RPi image as well.

The next thing on my list would be the SD card

5

Doesn’t it automatically adjust it’s time? I’ve never set a clock on any of my 3 pi’s that run continuously. Although every once in a while one WILL “freeze”, upon rebooting the clock updates automatically.

6

It’s worth investigating a few things. Could be a memory leak or something killing the box if this has anything besides the satnogs client running on it. Standard sysadmin tools like top and sar can help narrow this down leading up to the crash. Your syslog in /var/log/messages can also help after the crash to see what happened (sar can too as it keeps a historical log).

Also, it could be a hardware issue. It’s worth trying a different sd card, these fail quite often. They were never intended to support the churn of an operating system running off of them indefinitely, but mostly handle it ok. They are often not rated for the same temperature ranges that the RPi supports. I just recently had to swap one out on my octopi box. Have another RPi which has temperature sensitivity issues, when I get time I plan to swap the sd card on that one too.

As far the clock issue, sounds like your ntp client is not working, plenty of tutorials on fixing that out there. Also, ntp will gradually move your clock back in sync, not just change it suddenly. So its worth giving it a day or two to see what happens after booting.

7

I had the same issue, I need to reboot my rPi every few days, it was totally disconnected and unresponsive but lights are on and pwer was OK…
Then I found because I run the rPi in a air sealed box, there is not enogh air circulation in the box and because of the heat, my rPi is crashed…
Everytime I go to my backyard, open the box, unplug the rPi for a minute or two, and plug it back it back to work again, then I realised, it is a temp issue…
Use heatsinks… If you cannot find the large ones, even the standard heatsinks works better than none…
If possible find a way to create air circulation in the container…
Also i experienced that this heat problem killed my SD Cards 3 times…
Another possibility is the power supply… But I am almost sure you already know this issue and already took proper precautions…
Thats all for the moment…
Please continue to share the developments with us…

1 Like
8

If your pi has network, it should be synchronized using either ntp or timesyncd (at least standard linuxes do this).

You can grep in /var/log/syslog or /var/log/ntpdate*.log for failure about sync or connection with time servers. You can also check if an ntp is installed or if a program systemd-timesyncd is coming with your systemd package and running. This would at least easily fix the time issue.

9

I had a similar issue but it wasn’t heat related. For some unknown reason, the RPi would lose network connectivity yet all other processes ran fine. I now run a srcipt every 10 minutes to check. If it fails, it restarts the network. If after another 10 min it still fails, then the RPi reboots.
This system is now working fine for me.

10

@Zathras could you give me your script? beacuse it happens to my station too

11

Chris Thompson (AC2CZ, main contributor on FoxTelem) says this about creating a headless FoxTelem RPi:

The PI has a habit of becoming disconnected from the Wifi. With a headless machine you then don’t know if it is working. It seems to be because the Wifi hardware gets powered off and then it has a hard time reconnecting. I did this to prevent the power off:
sudo iw dev wlan0 set power_save off

Not sure if it’s of any help.

12

It’s crude but it works.

add this to root cron
*/10 * * * * /home/pi/bin/network-monitor2.sh >> /home/pi/bin/network-monitor.log 2>&1

create this file in /home/pi/bin network-monitor2.sh

#!/bin/sh

LOGFILE=/home/pi/bin/network-monitor.log

# cron script for checking lan connectivity
# change 192.168.1.1 to whatever IP you want to check.
IP_FOR_TEST="192.168.42.1"
PING_COUNT=1

INTERFACE="eth0"
PING="/bin/ping"
FFLAG="/home/pi/bin/stuck.fflg"

# ping test
$PING -c $PING_COUNT $IP_FOR_TEST > /dev/null 2> /dev/null
if [ $? -ge 1 ]
then
    echo "$(date "+%d %m %Y %T") : $INTERFACE seems to be down, trying to bring it up..." >> $LOGFILE
        if [ -e $FFLAG ]
        then
                echo "$(date "+%d %m %Y %T") : $INTERFACE is still down, REBOOT/RESTART to recover ..." >> $LOGFILE
                rm -f $FFLAG 2>/dev/null
#                sudo reboot
				sudo /etc/init.d/networking restart
        else
                touch $FFLAG
                echo "$(date "+%d %m %Y %T") : Taking $INTERFACE down.." >> $LOGFILE
                sudo ifconfig $INTERFACE down
				sleep 10
                echo "$(date "+%d %m %Y %T") : Bringing $INTERFACE up.." >> $LOGFILE
				sudo ifconfig $INTERFACE up
        fi
else
    echo "$(date "+%d %m %Y %T") : $INTERFACE is up" >> $LOGFILE
    rm -f $FFLAG 2>/dev/null
fi
3 Likes