Raspberry Pi 3 - Memory Issues and ZRAM

Good evening all,

I’m running two Pi3’s which are in production and are currently using the provided images. However, I’ve been seeing a lot of performance issues and crashes due to memory consumption. This was experienced quite often when receiving longer passes or the subsequent waterfall generation. These are certainly expected to some degree.

Over the last few days, I’ve been doing some experimentation with some performance tweaks and have had a bit of luck enabling ZRAM on the units. There’s a fair amount of information on ZRAM on this platform. There is an increased CPU load with the compression, but it seems to be negligible.

In the end, I implemented scripts from this repo:

Prior to enabling, I was seeing memory consumption at or close to 100% during just the recording with frequent crashes during waterfall generation. At present, I seem to be averaging about 30-40% during the pass and 50-60% during waterfall generation. I also don’t have any memory errors or failures since enabling.

It obviously doesn’t prevent gnuplot from being a memory hog, but may help to mitigate on lower powered machines.

@pierros @Acinonyx Would this be worth considering including in the image?

@Zathras, you may be interested in this post too.

The below image may be worth sharing as well. This is during NOAA 18 (SatNOGS Network - Observation 20132) with ZRAM enabled.

This was during the waterfall generation at peak consumption:

1 Like

I’ve added zram to stations prod 23 and dev 119. Both VHF.
Will see how it goes.


@Zathras How are your stations behaving with it enabled? I just checked and don’t seem to have any memory related errors, just a loss of connectivity about 24 hours ago [quote=“Zathras, post:3, topic:1380, full:true”]
I’ve added zram to stations prod 23 and dev 119. Both VHF.
Will see how it goes.

I haven’t noticed any issues. All seems to be ok.

1 Like

@Acinonyx can we add those on our ansible scripts?

1 Like

@pierros Just to update on this, I’ve been running zram enabled on both of my pi stations for 7 days now. Since implementing, I haven’t had any memory related failures on either box nor have I seen anywhere near the memory consumption as before.

i will note that I do still have a few passes with no waterfall in the last week. However, I’ve been able to track most of them back to issues such as connectivity, overlapping passes, or I was messing with the configs.

@Zathras may be able to provide more information as he implemented shortly after I did.

I haven’t seen anything so far, though I’ve recently upgraded all stations to RPi image and updated using satnogs-setup.
I did add zram to all stations as well. The production ones are rather busy…

Yes, we can. I’ve opened an issue.