Data warehouse POC

,

Update and change to the proposal:

I’ve committed this pipeline to db-dev for testing over the weekend. I’ve made an assumption in my hacky way of abstracting data from the kaitai objects that they will be flat in nature, and that won’t always be the case (as I quickly ran into with CAS-4 decoders).

As much as I would love the plug-and-play nature of an unmodified kaitai decoder working in our pipeline, its going to be too ugly to make it work. I plan to modify the pipeline to expect that the kaitai decoder scripts will all have a function added to them (get_data()?) that will return a dictionary of the telemetry fields and their values. This way the db code knows what to expect from every script and how to handle it, and the backend work that is specific to each satellite can be handled in the decoder script.

I’ll probably whip these changes up and have them in testing on db-dev next weekend.

1 Like