Meteor MN2 decoder for rpi 3B+


#1

Hi all,

Here is a quick set of instructions on making the pi3b+ work to demod Meteor LRPT and auto upload it. cgbsat[m] and Darkside deserve the most credit as they did the ‘technical’ stuff;

On the pi3, install imagemagick for the ‘convert’ utility by doing sudo apt-get install imagemagick

I used an 8Gb USB flash plugged into the pi and mounted on /datadrive/ easier to wear out a USB than to break the internal SD card with lots of writes. make the /datadrive directory owned by satnogs:satnogs and chmod 666 it so world+dog can write to it. Plug in the USB memory stick, run dmesg get the device like /dev/sda then run mke2fs /dev/sda1

mkdir /datadrive
mount /dev/sda1 /datadrive/

vi /etc/fstab 
add 
/dev/sda1             /datadrive        ext2 defaults,noatime  0       0

as root:

cd /datadrive
mkdir meteor
chown -R satnogs:satnogs meteor
chmod 666 meteor/
mkdir complete/meteor
chown -R satnogs:satnogs complete/meteor 
chmod 666 complete/meteor
cd /datadrive

Get the medat_arm version from http://orbides.org/page.php?id=1023 this will run on the pi3 straight away. Put the medat_arm binary in /datadrive

/home/pi/postob.sh script - make sure satnogs client is configured to run this at the end of the ob. I have added the lines below to my script which also resets my antenna switch.

meteorfile="/datadrive/data_*.s"
 
if [ -f $meteorfile ]
then
        mv /datadrive/data_*.s /datadrive/meteor/
        echo `date +%F" "%T`" Meteor file exists" >> /home/pi/freqset.txt
        /usr/bin/python /datadrive/process_meteor.py &
fi

wget https://github.com/darksidelemm/satnogs-extras/blob/master/scripts/process_meteor.py and put in /datadrive/ directory - this is @darksides script to automate the processing.

modify process_meteor.py as follows:

SOURCE_PATH = “/datadrive/meteor/data_*.s”
DESTINATION_DIR = “/tmp/.satnogs/data/”
RAW_DESTINATION_DIR = “/datadrive/meteor/complete/”
TEMP_DIR = “/datadrive/meteor/”
TEMP_FILENAME = “meteor_image_temp”
MEDET_PATH = “/datadrive/medet_arm”
CONVERT_PATH = “/usr/bin/convert”

You have to tweak some of the satnogs client files, the following was from cgbsat[m] in IRC

add “GNURADIO_LRPT_SCRIPT_FILENAME = ‘satnogs_lrpt_demod.py’” to satnogsclient/settings.py in line ~68 (after the AMSAT_DUV)

add " elif obj[‘mode’] == “LRPT”:
script_name = settings.GNURADIO_LRPT_SCRIPT_FILENAME" to satnogsclient/scheduler/tasks.py in line ~56 (after the APT_SCRIPT)

download https://matrix.org/_matrix/media/v1/download/matrix.org/BlIKHQFRUEvxuiDEcapYRYqs and put in /datadrive/

CD to /datadrive and run grcc -d /usr/bin/ lrpt_demod.grc

Once the above is done, vi /usr/bin/satnogs_lrpt_demod.py and modify the target locations for the bit stream so its dumped in to the /datadrive/ dir

self.bitstream_name = bitstream_name = “/datadrive/” + os.path.basename(decoded_data_file_path) + “ " + datetime.utcnow().strftime("%Y-%m-%dT%H-%M-%S") + “.s”
self.set_bitstream_name("/datadrive/" + os.path.basename(self.decoded_data_file_path) + "
” + datetime.utcnow().strftime("%Y-%m-%dT%H-%M-%S") + “.s”)

Restart the satnogs client with systemctl restart satnogs-client.service

Schedule Meteor MN2 passes and you should be good to go. Just before your 1st ob, run journalctl -f -u satnogs-client.service --lines=400|grep -v apscheduler.executors.default so you can watch for errors. In another window run ‘watch ls -l /datadrive’ so you can see the .s file building like:

-rw-r--r-- 1 satnogs satnogs 121872384 Aug 30 11:09 data_230487_2018-08-30T10-55-04.s
drwxrwxrwx 2 root    root        16384 Aug 25 08:17 lost+found
-rwxrwxrwx 1 pi      pi         158904 Apr 11  2017 medet_arm
drwxrwxrwx 3 satnogs satnogs      4096 Aug 30 09:40 meteor
-rwxr-xr-x 1 root    root         4198 Aug 29 21:52 process_meteor.py
drwxrwxrwx 2 root    root         4096 Aug 28 08:32 satnogs

You can test the decoder part by keeping one of the data_blah.s files and copying it to /datadrive/ during any existing observation, at the end of the ob, postob.sh will be executed, it’ll find the .s file and invoke the decoder script. Output will be uploaded automagically.

If there is something wrong please shout or come to #satnogs in IRC and let me know.


#2

This new feature seems really great!
Just a quick question, in each SatNOGS software update, e.g. like the one asked yesterday do we have to repeat the modification process of the .py files?
If yes, is there any projection of incorporating them into the official build?

Thanks and great work!


#3

For those to be merged upstream we would need to make sure that LRPT demodulation (and decoding?) will be containerized in gnuradio flowgraphs (and not rely on pre- and post- scripts and external programs). I think I caught somewhere an attempt to build a complete (i.e. IQ -> image) LRPT decoder in Gnuradio. If someone knows where we can find it please post it here so we can try to integrate it to gr-satnogs.


#4

The gnuradio LRPT decoder is online here: https://github.com/infostellarinc/starcoder/blob/master/gr-starcoder/lib/meteor_decoder_sink_impl.cc

Note that this is code by infostellar, which has the same aim as SatNOGS.


#5

Hi all,

Are these instructions still current? @cgbsat, I noticed you have created lrpt-demod branches for satnogs-client and gr-satnogs. Do they work? Do you advise on using those, or is it better to use the released packages and follow these instructions?

Thanks!


#6

I wouldn’t pull the full repositories, but just look at and implement the changes to satnogs-client, and put the compiled gnuradio flowchart into the correct gr-satnogs folder or straight into your path.


#7

I did that, but it doesn’t work yet. I don’t see data in the LRPT-observations, but also no errors in the satnogs-client log. It could also be that the SNR is not high enough, do you think that is the case?

Meteor-M 2 LRPT observation

Does the lrpt_demod flowchart have any dependencies I have to install, like grc blocks or something?


#8

I did a lot of tests using the scripts provided (using medet and client on x64) - you can check them here: https://network-dev.satnogs.org/observations/?norad=40069&observer=&station=108&start-time=&end-time=

Meteor needs good SNR to get data…


#9

I configured a test node using the instructions provided. Thank you for providing some initial troubleshooting guidance. This is frequently overlooked and allow people who are just starting to at least fix some of the obvious cases of stupid.

I have corrected a couple of chmod oversights, but could use some additional guidance.

Polite responses are always appreciated by the ignorant.

Bob Segrest

KO2F_Errors_181016.txt (5.5 KB)


#10

Hi Ko2f,

I see the following;

/bin/sh: 1: satnogs_lrpt_demod.py: not found
sh: 1: /home/pi/postob.sh: not found

So please double check these are where they should be, and fix as needed.

postob.sh basically needs to have the following in it:

meteorfile="/datadrive/data_*.s"

if [ -f $meteorfile ]
then
        sleep 20
        mv /datadrive/data_*.s /datadrive/meteor/
        sudo -n -u satnogs chmod 777 /datadrive/meteor/*
        echo `date +%F" "%T`" Meteor file exists" >> /home/pi/freqset.txt
        /usr/bin/python /datadrive/process_meteor.py &
fi

/usr/bin/satnogs_lrpt_demod.py is the path for the demod which your error log shows is missing. You might need to redo the following:
download https://matrix.org/_matrix/media/v1/download/matrix.org/BlIKHQFRUEvxuiDEcapYRYqs and put in /datadrive/

CD to /datadrive and run grcc -d /usr/bin/ lrpt_demod.grc you should end up with

root@satnogs:/datadrive# ls -ltar /usr/bin/satnogs_lrpt_demod.py
-rwxr-xr-x 1 root root 18021 Aug 28 15:43 /usr/bin/satnogs_lrpt_demod.py

Let me know how you get on.

regards,

Paul.


#11

I don’t see an obvious problem with the postob.sh file, but perhaps you can spot something I am overlooking in the attached file.

When I run the grcc command no errors are reported, but I am not finding the satnogs_lrpt_demod.py python file after it finishes…

I have scheduled another test run and will let you know what happens.

Bob Segrest

KO2F_Errors_181016a.txt (3.5 KB)


#12

Ok, the error result is in the attached file.

I don’t understand why it isn’t finding the postob.sh script… and I’m not sure what I’m doing wrong with the grcc command.

Your guidance is appreciated and I’m willing to write the Idiot’s version of the setup instructions if I can get it working.

Bob Segrest
KO2F

KO2F_Errors_181016b.txt (3.9 KB)


#13

Hi Bob,

chown satnogs:satnogs /home/pi/postob.sh

On my system:

-rwxrwxrwx   1 satnogs satnogs   1513 Oct  7 14:42 preob.sh
-rwxrwxrwx   1 satnogs satnogs   1645 Oct  7 14:43 postob.sh

retry grcc -d /usr/bin/ lrpt_demod.grc

then run updatedb and then locate lrpt_demod

root@satnogs:/home/pi# locate lrpt_demod
/datadrive/satnogs/lrpt_demod.grc
/home/pi/lrpt_demod.grc
/usr/bin/satnogs_lrpt_demod.py

Let me know how it goes, it will be something simple!

regards,

Paul.


#14

Thank you Paul,

The chown change make sense and I appreciate you spotting it.

The locate command is a wonderful tool and worth the effort I have expended thus far on its own.

Interestingly enough, after making the ownership change and rerunning the grcc I now have a satnogs_lrpt_demod.py file. I have scheduled a couple passes and will let know what happens.

Thank you for your support!

Bob Segrest
KO2F


#15

Oh well, seems I’m not there yet. Please see the attached file and I look forward to your response.

I removed the comment line that I had at the top of my postob.sh script and I applied chmod 777 to the file. We will see if that makes any difference on the next pass.

Bob Segrest
KO2F

KO2F_Errors_181017a.txt (10.2 KB)


#16

Hi Bob,

/bin/sh: 1: satnogs_lrpt_demod.py: Permission denied

-rwxr-xr-x 1 root root 13556 Aug 2 21:46 satnogs_meteor_decoder.py

chmod 755 satnogs_meteor_decoder.py and chown root:root satnogs_meteor_decoder.py and see what that does, it looks like you are almost there…

Also you probably want to chown satnogs:satnogs /datadrive/process_meteor.py

regards,

Paul.


#17

Thanks Paul,

I have made the changes and the next pass is about an hour away. I’ll let you know what happens.

Bob Segrest
KO2F


#18

Hi Bob,

You can look in /datadrive/meteor/complete for something like data_288151_2018-10-16T20-09-11.s - pick a bigish file corresponding to a pass where the waterfall looked good, and just move the file to /datadrive/

Next time any pass finishes, it will check for the *.s files and start the processing. Keep ‘htop’ running in another terminal to make sure the proper processes start.

regards,

Paul.


#19

Hi Paul,

That is a great idea, but I haven’t managed to get anything stored in my /datadrive/meteor/complete folder yet.

If you can share a pointer to one I would be happy to pull it down and give this a try.

Bob Segrest
KO2F


#20

Ok, I think we are making progress. At least this time is found my postob.sh file and opened it. Then it failed to find the “then” on the line after the If and hung…

The first error file has the log. The second file has a cat of my postob.sh file.

I know python files are particular about how lines are indented. Is this also the case with script (.sh) files?

I will also note that no file starting with data_ is present in the /datadrive folder and it seems to me that something (satnogs_lrpt_demod.py perhaps?) should be putting one there for the postob.sh script to find.

Any ideas would be appreciated…

Bob Segrest
KO2F

KO2F_Errors_181017b.txt (4.9 KB)

KO2F_Errors_181017c.txt (2.0 KB)