Meteor MN2 decoder for rpi 3B+

ko2f · October 18, 2018, 3:34am

Ok, the little light bulb started to glow a bit and it occurred to me that I could test the post observation script by just executing it. When I did, I got the same silly error reported in the log. After trying several things, I ended up just creating a new script file from scratch and the error went away. All I can figure is that I must have somehow inserted some not visible character into the file and it was screwing things up. What I have now is:

meteorfile="/datadrive/data*.s"

if [ -f $meteorfile ]
then
sleep 20
mv /datadrive/data_*.s /datadrive/meteor/
echo ‘date +%F" "%T’" Meteor file exists" >> /home/pi/freqset.txt
/usr/bin/python /datadrive/process_meteor.py &
else
echo “No data file found in /datadrive/ folder.”
fi

Which when executed from the command line, returns “No data file found in /datadrive/ folder.”

The next pass is in the morning and we will see what happens then…

I expect to see my “No data file found” message as I haven’t figured out why the datafile isn’t being created in the datadrive folder yet. There might be something else broken, or there may be a signal reception problem of some sort.

Bob Segrest
KO2F

ko2f · October 18, 2018, 1:51pm

Ok, it looks like I have solved my postob.sh problem.

But I don’t seem to be getting a data file for processing… The node is capturing and uploading a waterfall now [Observation 291122].

And, there is still at least error being reported in the log

KO2F_Errors_181018a.txt (5.2 KB)

I’m open for more ideas…

Bob Segrest
KO2F

pjm · October 18, 2018, 2:15pm

Hi Bob, can you check

/usr/bin/satnogs_lrpt_demod.py

and look at the following

self.bitstream_name = bitstream_name = “/datadrive/” + os.path.basename(decoded_data_file_path) + “" + datetime.utcnow().strftime("%Y-%m-%dT%H-%M-%S") + “.s”
self.set_bitstream_name("/datadrive/" + os.path.basename(self.decoded_data_file_path) + "” + datetime.utcnow().strftime("%Y-%m-%dT%H-%M-%S") + “.s”)

You might have to modify this and restart satnogs client, note the bitsteam name in each line is /datadrive/ - before you tweak it, look into the directory currently specified to see if you have the bitstream output file. If you can find the file, copy it into the /datadrive/meteor/ dir and wait for any next pass.

regards,

Paul.

ko2f · October 21, 2018, 12:55pm

Hi Paul,

Thank you for your guidance! I had to step away for a couple days and just got back around to working on this project again.

I edited the satnogs-lrpt_demod.py file and found that both of the lined you referenced were pointing to the /tmp folder. I changed them to the /datadrive folder, restarted my node and will let you know what happens.

I also looked in my /tmp folder before rebooting and found 1 of the missing data files. I moved it into my root folder and will do some testing with it after the next pass finishes processing.

Bob Segrest
KO2F

ko2f · October 21, 2018, 2:28pm

Ok, the good news is that lrpt demod is now trying to put the data file in the /datadrive folder. Unfortunately, it was denied access to write into this folder… Yet another silly oversight on my part.

I have changed the ownership of /datadrive from root:root to satnogs:satnogs and we will see what happens on the next pass.

Bob

KO2F_Errors_181021a.txt (14.2 KB)

ko2f · October 21, 2018, 5:11pm

changing the ownership of /datadrive to satnogs fixed the last problem. The data file was created in the /datadrive folder and the completed pass displays a waterfall icon on the web page for this observation.

But… It looks like I’m still fighting permission issues because the postob.sh script then failed with another access issue.

Please take a look at the attached log file and let me know what I need to do next…

Bob Segrest
KO2F

KO2F_Errors_181021b.txt (4.8 KB)

pjm · October 21, 2018, 7:27pm

Hi Bob,

/home/pi/postob.sh: 7: /home/pi/postob.sh: cannot create /home/pi/freqset.txt: Permission denied

touch /home/pi/freqset.txt then chown satnogs:satnogs /home/pi/freqset.txt should fix that bit.

for the other error
mv: cannot stat ‘/datadrive/meteor/data_293752_2018-10-21T15-30-30.s’: Permission denied

sudo -u satnogs touch /datadrive/meteor/data_293752_2018-10-21T15-30-30.s and see if that works first.

BTW it appears that meteor MN2 might be having some sort of issue, the VHF + HRPT have been off over the past days.

regards,

Paul.
M0EYT station #91

ko2f · October 21, 2018, 9:48pm

Hi Paul,

I was able to create a freqset.txt file with the touch command without any difficulty.

However the attempt to touch the data file in the /datadrive/meteor folder failed with a Permission Denied error.

Bob

pjm · October 22, 2018, 9:03am

Hi Bob,

OK so you need to resolve the directory ownership issue, I would chown -R satnogs:satnogs /datadrive and chown -R satnogs:satnogs /datadrive/meteor/ then retry as the satnogs user to touch a file. Also whilst you are investigating the issue, chmod -R 777 /datadrive/ and try again as the satnogs user. It sounds like you are almost there with only this minor perms issue to resolve.

regards,

Paul.

ko2f · October 22, 2018, 4:42pm

Hi Paul,

Continuing to make progress! It looks like I’m past the Permissions problem set for the time being.

As there are several datasets in my /datadrive now the system appears to have tried the oldest one first and it failed with the following:

/home/pi/postob.sh: 3: [: /datadrive/data_293752_2018-10-21T15-30-30.s: unexpected operator

I went ahead and moved this file to my home folder thinking that if it got messed up along the way I can at least see what happens with the next data file in the list. The next M2 pass isn’t until 0100 UTC, so we will see…

The log and directory listing are captured in the attached file for your reference.

Bob Segrest

KO2F_Errors_181022a.txt (6.7 KB)

ko2f · October 23, 2018, 1:37pm

Hi Paul,

I’m a bit confused at this point. The data file is being created in the /datadrive and the postob.sh script seems to be moving the data file into the /datadrive/meteor folder. Then things seem to break…

I’m open for next step suggestions.

Bob

KO2F_Errors_181023a.txt (12.9 KB)

pjm · October 23, 2018, 3:19pm

Hi Bob, I’ve put this mornings IQ recording at https://www.dropbox.com/s/301d8lpbt0odecc/data_294544_2018-10-23T09-42-42.s?dl=1 - if you download that file and copy it to /datadrive/ and following the next observation of anything, this should be picked up and processed. You can start by manually cat’ing the postob.sh, and as ‘satnogs’ user, running each line to work out where any errors might be.

regards,

Paul.

ko2f · November 19, 2018, 1:07am

Hello Paul,

Sorry for dropping out for the past few weeks. My work activities ramped up and I had to set things aside for a bit.

I scheduled another pass this morning and actually received a result! While there is nothing in the /datadrive/meteor/complete folder, the was something new in the meteor folder…

pi@raspberrypiKO2F-Test-1:~ $ sudo ls -la /datadrive/meteor
total 178308
drwxrwxrwx 3 satnogs satnogs 4096 Nov 18 19:54 .
drwxrwxrwx 4 satnogs satnogs 4096 Nov 18 09:50 …
drwxrwxrwx 2 satnogs satnogs 4096 Sep 14 18:28 complete
-rw-r–r-- 1 satnogs satnogs 134448580 Nov 18 09:49 data_328142_2018-11-18T14-34-07.s
-rw-r–r-- 1 satnogs satnogs 22177846 Nov 18 10:02 meteor_image_temp.bmp
-rw-r–r-- 1 satnogs satnogs 3562496 Nov 18 10:02 meteor_image_temp.dec
-rw-r–r-- 1 satnogs satnogs 22177846 Nov 18 10:03 meteor_image_temp_thermal.bmp

I have another pass scheduled in a hour or so and I will monitor the log this time to see if it gives me a hint about why the final result didn’t end up in the complete folder.

If you have any other ideas, I would be happy to hear them.

Bob
KO2F

ko2f · November 19, 2018, 2:56am

Ok, I copied the image files from the last pass to my desktop and (other than the dropouts) the image looks good. The thermal image came out as one big black block. Not sure why…

On the pass this evening I captured the following in my log:

Nov 18 21:11:37 raspberrypiKO2F-Test-1 satnogs-client[542]: linux; GNU C++ version 6.2.0 20161010; Boost_106100; UHD_003.009.005-0-unknown
Nov 18 21:24:00 raspberrypiKO2F-Test-1 satnogs-client[542]: Total: 622.659668
Nov 18 21:24:00 raspberrypiKO2F-Test-1 satnogs-client[542]: Processing: 16.908089
Nov 18 21:24:00 raspberrypiKO2F-Test-1 satnogs-client[542]: Correlation: 298.990692
Nov 18 21:24:00 raspberrypiKO2F-Test-1 satnogs-client[542]: Viterbi: 238.274063
Nov 18 21:24:00 raspberrypiKO2F-Test-1 satnogs-client[542]: ECC: 68.413429
Nov 18 21:24:00 raspberrypiKO2F-Test-1 satnogs-client[542]: Remainder: 0.073361
Nov 18 21:24:00 raspberrypiKO2F-Test-1 satnogs-client[542]: Packets: 2829 / 6511
Nov 18 21:24:00 raspberrypiKO2F-Test-1 satnogs-client[542]: Elapsed time: 00:06:06.884
Nov 18 21:24:19 raspberrypiKO2F-Test-1 satnogs-client[542]: Total: 16.785372
Nov 18 21:24:19 raspberrypiKO2F-Test-1 satnogs-client[542]: Processing: 16.779142
Nov 18 21:24:19 raspberrypiKO2F-Test-1 satnogs-client[542]: Correlation: 0.000000
Nov 18 21:24:19 raspberrypiKO2F-Test-1 satnogs-client[542]: Viterbi: 0.000000
Nov 18 21:24:19 raspberrypiKO2F-Test-1 satnogs-client[542]: ECC: 0.000000
Nov 18 21:24:19 raspberrypiKO2F-Test-1 satnogs-client[542]: Remainder: 0.006231
Nov 18 21:24:19 raspberrypiKO2F-Test-1 satnogs-client[542]: Packets: 2829 / 2829
Nov 18 21:24:19 raspberrypiKO2F-Test-1 satnogs-client[542]: Elapsed time: 00:06:06.884
Nov 18 21:24:20 raspberrypiKO2F-Test-1 satnogs-client[542]: Waiting for 120 seconds before processing.
Nov 18 21:24:20 raspberrypiKO2F-Test-1 satnogs-client[542]: Attempting to process: /datadrive/meteor/data_328922_2018-11-19T01-55-20.s
Nov 18 21:24:20 raspberrypiKO2F-Test-1 satnogs-client[542]: Traceback (most recent call last):
Nov 18 21:24:20 raspberrypiKO2F-Test-1 satnogs-client[542]: File “/datadrive/process_meteor.py”, line 115, in
Nov 18 21:24:20 raspberrypiKO2F-Test-1 satnogs-client[542]: result = combine_images()
Nov 18 21:24:20 raspberrypiKO2F-Test-1 satnogs-client[542]: File “/datadrive/process_meteor.py”, line 73, in combine_images
Nov 18 21:24:20 raspberrypiKO2F-Test-1 satnogs-client[542]: subprocess.call([CONVERT_PATH, “-append”, raw_image_path, result_image])
Nov 18 21:24:20 raspberrypiKO2F-Test-1 satnogs-client[542]: File “/usr/lib/python2.7/subprocess.py”, line 168, in call
Nov 18 21:24:20 raspberrypiKO2F-Test-1 satnogs-client[542]: return Popen(*popenargs, **kwargs).wait()
Nov 18 21:24:20 raspberrypiKO2F-Test-1 satnogs-client[542]: File “/usr/lib/python2.7/subprocess.py”, line 390, in init
Nov 18 21:24:20 raspberrypiKO2F-Test-1 satnogs-client[542]: errread, errwrite)
Nov 18 21:24:20 raspberrypiKO2F-Test-1 satnogs-client[542]: File “/usr/lib/python2.7/subprocess.py”, line 1024, in _execute_child
Nov 18 21:24:20 raspberrypiKO2F-Test-1 satnogs-client[542]: raise child_exception
Nov 18 21:24:20 raspberrypiKO2F-Test-1 satnogs-client[542]: OSError: [Errno 2] No such file or directory

It looks like something is still missing or I need to edit a path somewhere. Any suggestions would be appreciated.

This was a night pass and both images looked like big black blocks… Is this normal?

Bob
KO2F

vk5qi · November 19, 2018, 6:28am

Yes. At the moment there is no thermal infrared channel being transmitted, RGB only. As a result, during night you will see… black.

ko2f · November 19, 2018, 12:47pm

Thank you Mark!

One less issue to chase…

runahue · November 21, 2018, 10:23pm

Great tutorial !!!
I am trying it but i get this error after running postob.sh:

convert: length and filesize do not match `/datadrive/meteor/meteor_image_temp.bmp’ @ warning/bmp.c/ReadBMPImage/831.

convert: length and filesize do not match `/datadrive/meteor/meteor_image_temp_thermal.bmp’ @ warning/bmp.c/ReadBMPImage/831.

data*.s is moved to complete directory.
What do you think?
Thank’s 4 your help.

Rico · December 3, 2018, 1:05pm

I also followed the tutorial above, but noticed that the grc file mentioned in the first post no longer compiles with the newest (dec 2018) version of satnogs.

I found an updated one that does compile here:

github.com

darksidelemm/satnogs-extras/blob/0aaba07fa1dab0acc434b87cc8552b174296a4ac/flowgraphs/lrpt_demod.grc

<?xml version='1.0' encoding='utf-8'?>
<?grc format='1' created='3.7.11'?>
<flow_graph>
  <timestamp>Thu May  5 00:22:45 2016</timestamp>
  <block>
    <key>options</key>
    <param>
      <key>author</key>
      <value>Mark Jessop (vk5qi@rfhead.net)</value>
    </param>
    <param>
      <key>window_size</key>
      <value>3000, 3000</value>
    </param>
    <param>
      <key>category</key>
      <value>Custom</value>
    </param>
    <param>
      <key>comment</key>

This file has been truncated. show original

I tested the observation and post observations steps separately, now waiting for a good Meteor pass to see if everything works together.

cgbsat · December 3, 2018, 1:24pm

Several people ran into this problem. @vk5qi added an updated flowgraph to his repository at https://github.com/darksidelemm/satnogs-extras

Rico · December 4, 2018, 7:59am

That is the one I linked to. It works like a charm:

https://network.satnogs.org/observations/347973/

Now let’s wait and see how it looks on a high elevation daylight pass.