Dashboard for historic DSLWP telemetry?

All raw frames retrieved from the amateur radio payload of the Chinese Lunar satellite DSLWP have been released (raw frames (here](https://github.com/tammojan/dslwp-data/blob/master/raw_frame.csv)) under a public license. There is also a viewer for the telemetry, but as far as I can see it lacks the possibility to go back in time. Which would be useful for viewing historic data. I’m wondering how hard it would be to make a nice grafana dashboard for this data. Would it be possible to (ab)use the Satnogs database for this?

I don’t know the format of the data, but the satellite developers are still around and probably don’t mind sharing it - given that all data is already public domain.

5 Likes

Of course this would be possible!
What we need is:

  • a proper format description of the telemetry
  • a kaitai-struct decoder (for satnogs-decoders
  • equations for the graphs
  • the grafana dashboard

I can offer my help with implementeing the decoder and the dashboard, but someone needs to provide a description for this data. Maybe there is one - I didn’t check the links yet!

Cheers,
Patrick

2 Likes

Would this decoder be enough to describe the telemetry?

1 Like

I think this file together with the previous one should be quite descriptive; maybe a next step could be to convert this to a kaitai struct description?

1 Like

Looks good! I need to find some time during the next days…

Hi, @tammojan has raised my attention towards this thread and I have just published some Python code I had lying around.

Regarding practicality, my opinion:

  1. I wouldn’t bother storing the TM Data Link frames transmitted by the satellite (which is what is stored in the released CSV file with the raw frames) in SatNOGS DB. Recovering the telemetry Space Packets from the TM Data Link frames is not straightforward, so I would use my code for that and store the Space Packets in the SatNOGS DB.

  2. Space Packets are classified according to the transmitter (spacecraft ID) and virtual channel. The virtual channel might be unimportant, but the distinction according to transmitter is important. The DSLWP-B Amateur payload was essentially a duplicated redundant system, so depending on which of the two transmitters sent the packet, some of the variables refer to that transmitter specifically. For example, in the field “PA temperature”, transmitter A transmits the PA temperature of transmitter A and transmitter B transmits the PA temperature of transmitter B. How could this fit into the SatNOGS DB/Grafana dashboard scheme?

  3. I expect that the Kaitai decoder can be easily converted from the construct decoder I’ve written or from the C .h files, but I’ve never done something like that.

  4. Note that we are missing conversion equations for some of the telemetry variables. I’m trying to get the details from Wei Mingchuan BG2BHC.

Merry Christmas,
Dani.

3 Likes

Hey Dani!

I can offer doing the kaitai decoder work.

Just to be clear: these packets you would import into SatNOGS DB are CCSDS Space Packets!? I’m sure this would be the best practice.

On your doubts about virtual channels: as far as I can see that right now is: we can simply add a mapping in the kaitai decoder pointing to each of them.

Merry Christmas!

Totally forgot: PM me if you want to get editor access to play around with dashboards!

Correct. The packets are standard CCSDS Space Packets.

However: the information about Spacecraft ID and Virtual Channel is not in the Space Packets. It is stored in the TM Space Data Link header.

Since we will be using TM Space Data Link for our upcoming missions too (see QUBIK) , this is a nice opportunity for us to pilot the implementation on DB/Dashboards.

I would be in favor of keeping the data in the raw-est of all format. If that’s the layer of TM Space Data Link then so be it. Submission should be done in the format closer to the demodulator (not the decoder) if we are to keep true to our current architecture (and avoid the problems we are now seeing from other soundmodems submitting malformed packages because they are trying to unpackage things).

cc @surligas (for the TM SDL implementation), @fredy @Acinonyx @cshields (for the architectural approach) and @DL4PD you are already here :stuck_out_tongue:

1 Like

Finally someone who understands me! :rofl::+1:

The raw-est of all formats would be the 580GB of IQ data (does not cover all observations) :wink: Otherwise, for reasons that @EA4GPZ mentions in his blog, I think space packets seem like the thing to ingest into the satnogs DB. The code that he wrote contains some heuristics (like PI9CAM packets are more trustworthy than others) that maybe should not be part of the telemetry extraction code in satnogs. Also, since this mission has ended, I doubt that anyone will re-upload the telemetry in an in-between format.

I agree on disabling uploads after this has been imported! The thing is: it would be an additional use-case for the CCSDS specs just prior to our mission. It has to be implemented in any case…

While I was elaborating:

So the point about “raw-ness” should be applied in the context of “data after demodulation”.

Now that we have that clarified, unpacking TM Space Data Link is a form of decoding (even though there are still more to unpack in there), that’s why I believe that we should be keeping primarily this form as our submission and storage baseline.

2 Likes

Wei Mingchuan has shared the conversion equations in the meantime, and Dani has updated his blog and notebook with these equations.

1 Like