PW-Sat2 Telemetry decoder


#7

Thanks for first decoded frames - https://network.satnogs.org/observations/349712/
I’ve downloaded those 3 frames and uploaded to our parser:


#8

Hah! That are live frames to test my decoder struct against :wink:


#9

By the way - do you know why only one observation (here) contains decoded data? Others look also promising but lack of incomplete/not decoded data sets?


#10

I didn’t check all, but: there are a lot of obs with very faint signals. The one that decodes curiously decoded with an old (and very simple compared to the new one) version of the BPSK decoder (basically the one I developed for PICSAT some time ago).
Maybe we should try to capture some IQs to get @surligas into this :wink:


#11

There is perhaps an issue for small baudrates, due to a fixed non-parametric transition width at the low pass filter. An IQ recording will help to fix that!


#12

I have IQ recording of this: https://network.satnogs.org/observations/350783/
Can you use it?


#13

Yes! Upload it if you can.


#14

@csete - I’m also interested! :slight_smile: I’ll put it to our demodulator.


#15

@surligas Done.

@begreen I will send you a PM with the link.


#16

We’ve decided to stay on BPSK 9k6 for most of the time. We’ve finished commissioning of the satellite and now entering experiments phase, so we’ll demand more energy and higher baud rate is one of the energy-saving method.

Find out more:


#17

Hi!
I think that we should remove 1k2, 2k4, 4k8 bps modes from the list - we’re using and will be using in the future 9k6 only.

Another topic that I’d like to discuss is data demodulation - 99,99% observations are useless for now, because the only output is a waterfall. There are no demod data, moreover it’s impossible to demod 9k6 data from ogg file (because of this problem: Ogg artifacts frequency offset).

In our software/webapp dedicated to mission there are right now 377006 frames received and uploaded, so I’m wondering what’s the problem with SatNOGS. So I’d like to solve it and get similar demoded amount of data in the future from satnogs :wink:


#18

Hey,
There is no flowgraph to demodulate in gr-satnogs yet. Maybe you could contribute one? :slight_smile:
I do also have a katai struct decoder laying around I need to do some checks, maybe a refactor with it.

db.satnogs.org is holding something around 30k frames for PW-Sat2…

Give me a ping - I’m always happy to help!


#19

According to:

I must have been wrong! Maybe there is an issue with SNR or something, so still your contribution would be awesome!


#20

Yes, that’s true adn great! But I believe that could be much more - most of the observations lack demod data. I think I can try to adapt our (PW-Sat2) GRC flow graphs that we use to demodulate BPSK and decode AX.25. But it demands some cleaning :smiley: Maybe you’ve already taken a look at this flowgraph and can comment if you think it’s worth to use it?

Concerning your katai struct decoder - I remember that there was an issue with bit endianness (because of lack of kaitai support). Is that resolved now? Another thing is that we’ve updated on-board software introducing a new power saving mode, so there is additional ‘short’ telemetry frame format :smile: However it’s much easier than ‘full telemetry’ frame.

But the first thing I’d like to take up is to figure out and fix (if possible, maybe it’s SNR issue) observations without data.

Thanks for your help @DL4PD!

BTW. Is there any API or sth like this to access data in SatNOGS db in real-time? I know that there is an option to download data in csv but having it in real time would be great!


#21

Sorry for answering that late, but unfortunately I’m a bit busy!
There is a possibility to get telemetry “in realtime” (not really realtime in each case, details will follow).
Just to have it said: a proper implementation for a data warehouse is not yet available but at least it is work in progress :wink:

I’ll give some more hints later…


#22

Hah, I didn’t tell a date, anyway :slight_smile:

To get telemetry frames filtered by satellite you can touch the telemetry API endpoint at

New frames are accessible as soon as they are stored in DB - and that’s why it is not really in realtime! The sync mechanism runs in a periodic task that pushes frames from network observations to db and also the client has a periodic task that pushes observation data every two minutes. As I said: there is a proposal on a new version of the data warehouse API that still needs to be defined. Parts of that might already be WIP.

For the telemetry decoder I could put some effort into it over the next few days, but I would need some data verification - maybe you could help with this?

Best regards,
Patrick


#23

Sure, just tell me how can I help :slight_smile: Should I take the decoder (where can I find the latest version? SatNOGS gitlab?), feed it with some test data and check outputs?


#24

Exactly! You can start like that or we’re doing it both: me for satnogs-decoders and you with your decoder and we can compare results. Starting with some frames from SatNOGS-DB I would suggest!

I’ll sum up some steps how to start with satnogs-decoders - gimme ~ 60’


#25

You can start cloning the satnogs-decoders package at gitlab:

$ git clone git@gitlab.com:librespacefoundation/satnogs/satnogs-decoders.git

If you don’t want to mess up your local python installation you should add a virtualenv using:

$ virtualenv -p python3 env
$ source env/bin/activate

Now enter the satnogs-decoders folder and see if there are missing dependencies:

$ cd satnogs-decoders
$ ./contrib/docker-ksc.sh

At the first run the output should look something like this:

Using default tag: latest
latest: Pulling from librespace/kaitai
c5e155d5a1d1: Pull complete 
221d80d00ae9: Pull complete 
4250b3117dca: Pull complete 
3b7ca19181b2: Pull complete 
1eadaf4c0dff: Pull complete 
4c1388ac89fe: Pull complete 
73713755b9cb: Pull complete 
5784ff1c0372: Pull complete 
752f931db2bc: Pull complete 
5201ac6501dd: Pull complete 
Digest: sha256:6321529d6faad300c61478a575363f8e03354bcc356f19b17286a872ca2ec5af
Status: Downloaded newer image for librespace/kaitai:latest

Once the images are up to date it should look like:

Using default tag: latest
latest: Pulling from librespace/kaitai
Digest: sha256:6321529d6faad300c61478a575363f8e03354bcc356f19b17286a872ca2ec5af
Status: Image is up to date for librespace/kaitai:latest
KSYs compiled successfully!

If this works without any errors you can succeed and try to install the package inside the venv:

$ pip install -e .
Obtaining file:///home/pd/software/satnogs-decoders
Collecting kaitaistruct~=0.8.0 (from satnogsdecoders==0.9.3)
Collecting enum34~=1.1.0 (from satnogsdecoders==0.9.3)
  Using cached https://files.pythonhosted.org/packages/af/42/cb9355df32c69b553e72a2e28daee25d1611d2c0d9c272aa1d34204205b2/enum34-1.1.6-py3-none-any.whl
Installing collected packages: kaitaistruct, enum34, satnogsdecoders
  Running setup.py develop for satnogsdecoders
Successfully installed enum34-1.1.6 kaitaistruct-0.8 satnogsdecoders

If you have any errors please post them here - there are too much possible issues to make a full installation manual. Otherwise you already have a working satnogs-decoders packagae installed. There are some tools in the contrib folder to fetch and decode frames from SatNOGS-DB - you can find a basic description in the README.md file. I would suggest you fetch the snippets (which need to be converted into proper “commandline scripts”) written by me and @azisi from here:

I am holding exported frames from SatNOGS-DB in a folder one up from satnogs-decoders/ called sampledata/db-dumps/, where sampledata/ is used for single frames and db-dumps holds the exported frames in .csv files. To fetch exports from DB you will need an account, but I think you will already have one :wink:

Using the snippets above you can easily try to decode PW-Sat2 frames after downloading the exported frames for, e.g., the last week (down on the DB-Page for PW-Sat2) using the following command:

$ cd satnogs-decoders
$ python ../decode-multiple.py -d Pwsat2 -x ../sampledata/db-dumps/43814-293-20190519T151238Z-week.csv 

This should return a lot of json-like decoded frames.

So yeah, that’s a start :wink:
Let’s see what problems come up…


#26

Some of the values already seem to be right, but there maybe some illegal frames (need to find a possibility to check for valid frames yet), see:

pwsat2_obc_boot_ctr

The 0-values seem incorrect, the other values are 1045, 1088 and 1089.

Edit: