Wow, strange. From your file I only get this much:
abn@ubuntu:~/satnogs$ ./medet /tmp/data_2020-05-14T18-43-02.s /tmp/out24 -S
Reading /tmp/data_2020-05-14T18-43-02.s…
pos=106828844 ( 99.97%) ( 6,11378,50) sig= -5 rs=( 0, 0, 0, 1) 1DFCCF36
Total: 47.048683
Processing: 2.447309
Correlation: 12.480673
Viterbi: 29.137943
ECC: 2.279354
Remainder: 0.703403
Packets: 3519 / 5603
Elapsed time: 00:06:47.508
abn@ubuntu:~/satnogs$
But with a few tweaks to pll_alpha and the AGC I got this instead:
From Svalbard to the Sahara!
This is what works for me at the moment:
This is from the file I got from you this morning. Pretty heavy NOAA-bombing on the frequency so only a short LRPT-image.
Sorry, real work and real life got in the way. Just taking another look at this.
Constellation plot
@anders, this is a plot from the middle of Observation 2235610 this morning. Looks like mine are similarly askew from 0.5. I assume this doesn’t really matter for a strong signal, but worth fixing nonetheless. I don’t know the theory any more than you do - sounds like you’ve got further with adjusting this than me.
I don’t get the same jerks as you do, so assume that’s down to local interference. (I can’t figure out a quick and easy way to capture a video). Looks like you’re getting similar artefacts through the image that I get on the periphery. I’ve not looked into the image encoding to know whether this is a direct artefact of poor signal or more complicated.
Pi processing
I’m still only getting valid symbol files by running the IQ through GnuRadio Companion on my VM and not on the Pi. This is only reversing the IQ Sink scaling but it must be having some other effect:
$ medet symbol_file_from_iq_via_vm.s wed1 -cd
Reading symbol_file_from_iq_via_vm.s...
pos=133095009 ( 99.98%) ( 1,15208,48) sig= -256 rs=(-1,-1,-1,-1) A379EEB2
Total: 130.029175
Processing: 5.378638
Correlation: 43.732464
Viterbi: 73.015411
ECC: 6.255593
Remainder: 1.647070
Packets: 3168 / 6620
Elapsed time: 00:06:16.728
$ medet symbol_file_from_pi.s wed2 -cd
Reading symbol_file_from_pi.s...
pos=133120000 ( 99.98%) ( 0, 0,64) sig= 96 rs=( 1, 1, 1, 1) FFFFFFFF
Total: 69.802116
Processing: 0.007021
Correlation: 0.044231
Viterbi: 63.398159
ECC: 5.117730
Remainder: 1.234980
Packets: 8126 / 8126
There’s something odd going on there - the former run has 3168/6620 packets. This is commensurate with about half the waterfall image looking like it’s got a reasonable signal. The latter has 8126/8126 but there’s an awful lot of FFFFFFFFs.
I wonder if there’s a way to detect a valid constellation and not output gibberish when there’s not. Sort of a squelch? That would reduce the symbol file size a bit. Then again, I wonder how much of medet could be done in the flowgraph in realtime.
So, I thought I’d copy my flowgraph to the Pi and run it manually to see what happens:
$ ./satnogs_lrpt_demod.py
/home/pi/.gnuradio/prefs/vmcircbuf_default_factory: No such file or directory
vmcircbuf_createfilemapping: createfilemapping is not available
gr::vmcircbuf_sysv_shm: shmat (2): Invalid argument
gr::vmcircbuf_sysv_shm: shmat (2): Invalid argument
gr::vmcircbuf_sysv_shm: shmat (2): Invalid argument
Seems I can ignore the first couple as warnings; not sure about the remainder but they do seem to stop and don’t recur on next run so I assume it’s to do with setting stuff up.
This creates a 128MB file, but still doesn’t process through medet:
$ medet_arm /tmp/data_2020-05-20T12-35-37.s /tmp/wed3 -cd
Reading /tmp/data_2020-05-20T12-35-37.s...
pos=133152768 ( 99.99%) ( 0, 0,64) sig= 99 rs=( 0, 0, 0, 1) FFFFFFFF
Total: 117.661163
Processing: 0.004719
Correlation: 0.049110
Viterbi: 108.473068
ECC: 8.914034
Remainder: 0.220229
Packets: 8128 / 8128
There’s something different happening on the Pi compared to my VM. Question is - what?
With a bit of faff I’ve managed to get the flowgraph to run on the Pi with the GUI rendered onto my dev box. I imagine it’s not at all performant, but I do get a good constellation:
It must be something after that:
I get a fine symbol stream and subsequent image running from the IQ on my dev machine, but nothing using the Pi’s symbol stream.
Pushing the VM stream to the Pi gives the same image so medet_arm runs fine.
You didn’t remove the GUI for the Pi? Random guess but wouldn’t that mess things up?
Also - how do you get it to run as part of an observation? Where to put it and how to tell satnogs-client to use it? (Wedging it in, so to say)
I tried it with the GUI disabled first, then realised that wasn’t going to tell me much, so figured I’d try it with the GUI. It worked surprisingly well. I just connect from my Ubuntu VM to the Pi over ssh with X forwarding (ssh -X I think) and it all appears on-screen like magic. Really surprised it was that easy.
I hard coded everything, including the input filename in the flowgraph. Compiled in GRC then copied the build .py file to the Pi using scp. Run it like any other command. That way I didn’t need to mess with command line arguments, but I guess you could do it that way too.
Thanks for getting this to work! The updated LRPT flowgraph worked out of the box for me.
Have you tried it on a Raspberry Pi station @cgbsat or just on x86-esque? I still can’t figure out why it performs differently on my Pi.
Good point, which I forgot to mention. I’m running it on an x86, so CPU, RAM and tmpfs diskspace are not a problem.
I have not tried this on an RPi, nor did I try the pre-gnuradio 3.8 version of lrpt_demod.py, but I do note that the 180kHz quadrature rate is about 15% higher than the 156.25kHz from the old version. Also note that the currently the Doppler correction block filters the bandpass; it may be that that requires more CPU power.
Would the Raspberry Pi run out of horsepower processing an IQ file rather than a stream? I’ll have to watch top while running the flowgraph next time.
I’m not sure whether I should expect feeding the same IQ file in twice to be deterministic? Although I guess I could try running the same data through twice on the same system then I’ll know. Struggling to work out where to attack next.
I would, perhaps naively, think that processing an IQ file would give better results as it does not require realtime processing that a stream does. However, I think that with the 3.7 version of lrpt_demod.py, LRPT observations could be processed in realtime using an RPi.
Ping @vk5qi…
I was doing realtime processing of LRPT (at least to the QPSK soft-bit stage) on a RPi 3B+ without issues. It wasn’t possible to do the image decoding step however, that ran afterwards in a post-processing script.
Remember that as this flowgraph is going to be kind of separate from the regular satnogs flowgraphs, we can fix various parameters - e.g. we could drop the quadrate rate down to 156.25 kHz if that’s an issue.
Hi all, just chiming in as I did see something like that in the past.
With client version 0.9 I had a working automated LRPT receive chain going. But when watching the output of satnogs-client during an observation I sometimes saw a string of OOOO’s in the log, which is printed by librtlsdr when a buffer is overrun, so apparently sometimes the RPi CPU could not keep up with the data stream from the dongle.
Since with the flowgraph included in version 1.3 the observation bandwidth has only increased, the chance of this happening is even bigger, so perhaps it is better to not do the symbol recovery in real time in the flow graph when using an RPi, but just record an IQ file and then run meteor_demod and medet in a post observation script so the CPU gets some breathing room during the observation. Another advantage is that you don’t need to change anything in satnogs-client, you just need to configure the post observation script in the settings. Of course you will end up with huuuuge IQ files, so they should be stored somewhere else than on the ramdisk or it will choke the Pi. I will see if I can get that to work in the near future.
Given the LRPT flowgraph is going to be a completely separate flowgraph (since it breaks the satnogs convention of doing all processing in realtime), there’s no reason why the observation bandwidth can’t be narrowed back to what we were using in the original LRPT flow-graph.
I was able to do the QPSK demodulation portion in realtime on a RPi 3B+.
Ok, the 288000 sample rate IQ files are a bit big to handle, so maybe not touching the default flowgraph an using th dump iq functionality is not a good idea after all.
Could someone put a copy of the latest working flow-graph up somewhere? I’d like to give it a test on my RPi 4 station and see how the CPU load goes…
Well, it does work using the default flowgraph and running meteor_demod and medet on the dumped IQ file in a post observation script:
I had a go at running your flowgraph on my station. Didn’t spot any obvious overruns in syslog, but perhaps they show up somewhere else.
Observation: https://network.satnogs.org/observations/2346479/
(It looks a bit low in frequency to me? The receiver I’m using is pretty well on frequency…)
Unfortunately I don’t seem to be able to get anything useful out of the observation. medet_arm does produce a .dec file (~7Mb), but i can’t seem to get it to output any images.
The IQ recording is here if you want to have a play: https://satnogs-iq.s3-ap-southeast-2.amazonaws.com/2346479.raw
Are you running an RPi 4 or RPi 3B+? I see you are using an SDRPlay at 1 msps, an rtlsdr is at 2.048 msps in satnogs which takes quite a bit more CPU load to handle.
The setup where I saw occasional overruns was with an RPi 3B+, client 0.9, flowgraph from your repository and an rtlsdr at 2.048 msps. It was really on the edge, because on some observations I saw overruns and on other observations I didn’t. I also tried an airspy mini, but with that it was really impossible since the minimum sample rate of that is 3 msps.
