RFC: Modulation & Framing Information in SatNOGB DB

SatNOGS Software
1

This post presents a multi-step plan to improve the modulation and framing information in SatNOGS DB.

There have been numerous discussions on this topic in the past, but progress has been stalled for several reasons. Meanwhile there exists an established schema in the form of SatYAML files by gr-satellites. This schema is used by an up-to-date database with modulation and framing information for a large number of satellites/transmitters.

In this top post I present a strategy to adopt the SatYAML schema in SatNOGS to facilitate the development towards a more comprehensive SatNOGS DB and more flexible SatNOGS stations, while also enabling gr-satellites users to fetch desired information directly from SatNOGS DB. In a second post to this thread I’ll discuss some advantages & disadvantages and provide concrete ideas where to start.

:construction: Identified development blockers

On the surface, there is a general lack of time to work on the topic of standardizing modulation and framing information. Nevertheless it’s a common topic of consideration while working on other topics, e.g. when implementing support for the next mission.
A second reason for standardization to be difficult stems from the fact that it is situated at an interface between two sub-projects with different developers.
A third reason is the existing assumption that development on this feature in SatNOGS DB is blocked by a database migration from MariaDB to PostreSQL, which itself has been postponed for many years.

:bulb: Proposal

To eventually reach the original of providing current modulation & framing information in SatNOGS DB, used by SatNOGS stations & others, I would like to propose the following strategy:

  1. (DB) Extend the database schema so that it can store the data provided by SatYAML files in its database. Then set up a new API endpoint that can serve automatically-generated SatYAML files.
  2. (Station) Use hard-coded SatYAML files in satnogs-client to store modulation & framing information as a replacement of the existing ad-hoc “transmitter mode” to flowgraph + parameters mapping.
  3. (Station) Replace hard-coded SatYAML files with data queried from SatNOGS DB API.
  4. (Station & Network) Provide required SatYAML data through the SatNOGS Network jobs API to eliminate the need to satnogs-client to query SatNOGS DB (optional).

What do you think? Does this proposal sound feasible, is it desirable to follow such a strategy? There exist other long time planned architecture change proposals, how does this strategy stand with regards to those (e.g. the “new radio architecture” which is supposed to change the interfaces between satnogs-client and the radio component (satnogs-flowgraphs in the old architecture))?

I look forward to hearing your perspective! Hopefully this discussion can lead us ultimately to progress on this topic.

2 Likes
2

:star2: Advantages

  • The development tasks needed in SatNOGS DB and in SatNOGS Station software become independent of each other (step 1 & 2). This is the key point of this proposal, because it solves the chicken and egg problem at the interface between sub-projects identified in the previous section as a development blocker.
  • Step 1 will allow gr-satellites users to query SatYAML files directly from SatNOGS DB / Step 2 will allow SatNOGS Stations to use SatYAML files directly from gr-satellites. This brings those two user groups closer together and strengthens the “ecosystem” as a whole. It simplifies the exchange of modulation & framing information between the related projects.

For SatNOGS stations with gr-satellites integration using SatYAML files queried from SatNOGS DB there are two (minor) improvements:

  • Selection of the required SatYAML doesn’t need to be done by extracting the (potentially temporary) NORAD ID from the TLE, but instead follows from relations inside SatNOGS DB directly.
  • Updates of gr-satellites is only required to get software updates but not any more to update the SatYAML files.

:no_entry_sign: Disadvantages

  • (A) Some of the modulation & framing information is already available in the SatNOGS DB. Depending on the specific implementation, step 1 can easily lead to data duplication when following the approach of not normalizing the model / storing structured data inside transmitter records.
  • (B) Using SatYAML instead of designing a new format / extending the transmitters API limits the attributes that can be provided to SatNOGS Stations.

Data duplication as in (A) can be eliminated by carefully separating the schema changes (adding new attributes as required) from the actual SatYAML format. Instead of naively extending the transmitter schema with a new complex field to store SatYAML directly, the SatNOGS DB schema should be agnostic to the transport formats used to serve the data. SatYAML will be an additional transport format only, and as such only impacts the design of the respective API endpoint directly.

Potential imitations identified in the future due to using SatYAML instead of a custom transport format in satnogs-client as in (B) can be overcome by two different approaches:
Collaboratively improve the SatYAML schema as needed or introduce a new transport format. By starting with SatYAML, a path towards potential future synergies is prepared.

:checkered_flag: Where do we start?

The motivation to write this post arose from considerations in the context of the AMSAT-DL ERMINAZ mission. To allow larger transfer frames, a change in framing parameters with respect to those used by the LSF Qubik mission were considered.

Such a change requires an update of the SatNOGS Stations Software, specifically the existing flowgraph for Qubik would need to accept a new additional framing parameter (Reed-Solomon interleave depth), and satnogs-client must provide this parameter based on the observed satellite/transmitter.
Historically the ad-hoc dictionary in satnogs-client:radio/flowgraphs.py#L24 would have been extended by this new parameter. Following the new proposed strategy implementing step 2, two hard-coded SatYAML files for Qubik and ERMINAZ would be added instead which would take precedence over the existing ad-hoc dicts. While this change is tiny, it precludes the addition of yet another parameter to the ad-hoc dicts and as such could be considered a first step towards the goal of storing more modulation & framing information in SatNOGS DB.

1 Like