New gr-satnogs architecture

SatNOGS Software
1

Hello there,

I have started a massive refactoring of the gr-satnogs codebase. The ongoing work is at the https://gitlab.com/librespacefoundation/satnogs/gr-satnogs/merge_requests/174

One of the most difficult parts are already done. I managed to create a generic enough API for the available decoders and seems that it works pretty well. The benefits from the system point of view, is drastically less python memory usage (swig interface is simpler) and faster compilation times. From the usage perspective, every decoder now is a variable that can be attached to a generic frame decoder block. To me at least, is much more convenient and easy to use. For every frame decoded, there is also a set of metadata (count of errors or corrections, etc)

As a next step is the metadata together with the decoded data to be serialized and stored or sent to another entity (socket, file, whatever).

My thoughts is also to create an arbiter that can generate a flowgraph on the fly. For this reason, the capabilities of the sat transmitter should be known to gr-satnogs somehow. With this approach there will be no need for a hard dependency of the client and gr-satnogs every time we need to add a new decoder. It is much easier to handle RF related info into the radio code rather than in the client. I believe that hacking and experiment should be also easier.

What we have to decide:

  • Should we continue with the model of storing the generated files? Personally, I would propose to drop it and send them via sockets
  • In case we want to store them in files, shall I proceed with hdf5?
8 Likes
2

Great work @surligas!

Sockets sound much more sensible.

Wouldn’t packaging up artifacts (in hdf5 if we settle for that) be a work for the client?

3

Which generated files do you mean here, the waterfall and the ogg files?

4

Totally fine! I prefer it to be honest in order to keep dependencies minimal.

So full steam ahead for socket communication!

Count also the decoded frames.

2 Likes
5

Can we keep an option to write these to disk as well as send them over a socket? It helps with implementing new features to have these files on disk.

6

Wouldn’t sockets help more? +1 on keeping the “option” to write on files too though.

PSA: I have made this thread public, and would love others to chime in here too. (cc’ing @EA4GPZ for possible de-dupe and co-development with gr-satellites)

2 Likes
7

We could add some scripts that connect to the sockets and write files containing the data transferred.

Just my 2 cents :slight_smile:

2 Likes
8

I do not have any objection about keeping them in files too. However, there is extra information now that can be stored. Either I can drop such information and store only the decoded frames keeping the “compatibility” with the current scheme, or we can use a more structured way that can store them both efficiently.

1 Like
9

Speaking about gr-satellites, I want to re-architecture it during this year (probably before summer). It has grown a lot (over 40 satellites supported) and there is a lot of repetition between the flowgraphs for similar satellites. I want to make things more modular to avoid repetition, and the discussion about Artifacts has already given me a few ideas about how to divide the modules in a natural and useful way.

Regarding integration with SatNOGS, I’m happy to adapt and do things so that the gr-satellites modules (most likely will end up being hierarchical flowgraphs) are easy to drop in into gr-satnogs.I’m also happy to get feedback from the community about what could be useful and what is useless.

5 Likes