Specifically, it’s happening with the curl setup script/satnogs-setup installation (docker, ansible). It seems to happen after the first few successful packets are demodulated with the flowgraph.
This happens on >50 stations (so is a very common problem with the default/basic/nominal setup).
In my case, it’s on a desktop computer with Debian 12, though I think the issue presents on Raspberry Pi instances too.
Related thread: AX.100 decoder crashing
I think, I have already mentioned this, but for me it seems to be an important point for locating the root cause of the problem. After testing several docker images I am pretty sure, the problem occured with the first commit for the V2.1. So the ax.100 decoder never worked for any 2.1.x. Unfortunately there were many changes, the deltas are very large, so it is difficult to isolate this specific issue. But at least we know the exact time, when it happened. As far as I can see, there were no changes in the decoder itself but in the surrounding environment. This is why a few records are decoded before every crash.
Thank you for the extra info.
For some reason i was unable to replicate since yesterday but today i was able to. I still have no idea what is going on and how to fix it but will get there.
In the meantime a removed our blocks for decoding and added the deframer from gr_satellites and seems to work. The commit is Reenable AX.100 Mode 5 flowgraph (2e569a85) · Commits · librespacefoundation / SatNOGS / satnogs-flowgraphs · GitLab. If someone can test it with a station it would be helpful. Release takes time (and we had and some problems with the pipelines) so when it is ready we will inform you.
Edit: Also if you take a close look the error in this thread and the other is not the same. Both indicate something memory related.
Hi, I would be more than happy to test with my station, with the caveat that I know next to nothing about computers. It took me three days to get satnogs working 
I can enter command line instructions and I know where the sd card goes…
are the docker-images updated with this changes, they all look very new?
The published image has the flowgraph disabled. I am trying to fix the packaging of the satnogs-flowgraph with the gr_satellites block because it fails right now. When it is ready i am gonna post here.
edit: The packaging is fixed so i will start the process to publish the new docker images
librespace/satnogs-client:latest
OBS: 14927662
satnogs-3677 | INFO satnogsclient.radio.grsat Starting gr_satellites for observation 14927662 at 57600 sps
satnogs-3677 | rot_init called
satnogs-3677 | initrots4_dummy: _init called
satnogs-3677 | rot_register (1)
satnogs-3677 | rot_register (2)
satnogs-3677 | dummy_rot_init called
satnogs-3677 | INFO satnogsclient.observer.observer Start gnuradio thread.
satnogs-3677 | ZMQ IQ SUB Connecting to 127.0.0.1:16888
satnogs-3677 | ZMQ audio SUB Connecting to 127.0.0.1:16889
satnogs-3677 | ZMQ FRAME SUB Connecting to 127.0.0.1:16887
satnogs-3677 | rig_check_rig_caps: p1=0x7f9a230640, p2=0x7f9a236bc0, rig_model=0x7f9a230640, macro_name=0x7f9a236bc0
satnogs-3677 | initrigs4_dummy: _init called
satnogs-3677 | rig_init: rig_model=Hamlib NET rigctl
satnogs-3677 | rig_init: rx_range_list1 is empty, using rx_range_list2
satnogs-3677 | rig_init: rig does not have tx_range!!
satnogs-3677 | netrigctl_init version 20230106.0
satnogs-3677 | Found Rafael Micro R828D tuner
satnogs-3677 | RTL-SDR Blog V4 Detected
satnogs-3677 | usb_claim_interface error -6
satnogs-3677 | [INFO] Opening Generic RTL2832U OEM :: 00000007...
satnogs-3677 | Found Rafael Micro R828D tuner
satnogs-3677 | RTL-SDR Blog V4 Detected
satnogs-3677 | correlate_access_code_tag_bb :debug: Access code: 930b51de
satnogs-3677 | correlate_access_code_tag_bb :debug: Mask: ffffffff
satnogs-3677 | [INFO] Using format CF32.
satnogs-3677 | Allocating 15 zero-copy buffers
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 2622
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 110695
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 238341
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 283823
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 569149
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 571093
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 594981
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 598713
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 610779
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 614990
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 619951
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 731060
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 849961
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 888099
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 961484
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 1031652
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 1080965
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 1143596
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 1216489
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 1223723
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 1300254
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 1335501
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 1366158
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 1430363
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 1479338
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 1490971
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 1514662
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 1582783
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 1651175
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 1667682
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 1910223
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 2204893
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 2213304
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 2263108
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 2449303
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 2454520
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 2528179
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 2542194
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 2543057
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 2577252
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 2702966
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 2927807
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 2934206
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 2945794
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 3024469
satnogs-3677 | correlate_access_code_tag_bb :debug: writing tag at sample 3171648
satnogs-3677 | netrigctl_close: done
satnogs-3677 | netrigctl_close: done
satnogs-3677 | INFO satnogsclient.radio.grsat Stopped gr_satellites
satnogs-3677 | INFO satnogsclient.radio.grsat Processing kiss file
satnogs-3677 | INFO satnogsclient.radio.grsat Total frames: 0
satnogs-3677 | INFO satnogsclient.observer.observer Observation 14927662 finished.
satnogs-3677 | INFO satnogsclient.observer.observer Executing post-observation script.
satnogs-3677 | INFO: 14927662, Norad: 69015, Sat: 0 FRONTIERSAT, Baud: 9600.0, TLE: {"tle0": "0 FRONTIERSAT", "tle1": "1 69015U 26100AM 26246.67769612 .00004420 00000-0 19194-3 0 9991", "tle2": "2 69015 97.3882 143.2549 0009579 120.3517 239.8668 15.22789724 18768"}
satnogs-3677 | INFO satnogsclient.observer.observer Artifacts upload for observation 14927662 successful.
satnogs-3677 | /mnt/backup/pisatnogs/IQ/satnogs_14927662_436150000.raw : 53.37% ( 79.8 MiB => 42.6 MiB, /mnt/backup/pisatnogs/IQ/satnogs_14927662_436150000.zst)
satnogs-3677 | INFO satnogsclient.scheduler.tasks Upload of waterfall image for observation 14927662 successful.
satnogs-3677 | INFO satnogsclient.scheduler.tasks Upload of audio recording for observation 14927662 successful.
the snr is too weak. need stronger signal. because the error ussualy happen when got the strong signal and decode
yes, there is planned OBS 14933464 of another satellite, but it had a similar problem. We will see.
Much better now - the decodes obviousely come from gr-satellites, but no more crash.
The logfile here
OE6ISP_log.txt (101.5 KB)
Stefan/OE6ISP
update file manually, or update docker to latest version?
update docker-image to 2.1.2
Image: librespace/satnogs-client:2.1.2
it work on my station. thank you! very simple. only change one line. for decoding data, just put satyaml of ax100 mode 5 satellites on gr-satellites satyaml folder..
observation 14938126 - ion-scv-014
i got many decoded data
full waterfall:
time for push rank 
so no crash and some data also present right? I want to check a bit the frames but this should be enough for a fix for now right?
It works in some way, yes. As far as I understand, the satnogs-flowgraph is bypassed now and gr-satellites does the decode - without being crashed. Maybe the many debug-messages should be turned off.
EDIT:
AX.100 is back with stunning performance!
thank you @sdoukos !
