Deletion of data - Usecases


#1

Hello all,

In light of the recent changes in Network (see related commit) and the discussion that spawned from it, we would like to start this thread to get as wide of a view as possible on the issue of deleting observations/artifacts of the past. In all honesty we (the SatNOGS dev team) underestimated the importance of such a feature for a few members of our community and would love to explore in depth the reasons behind the action of deleting and try to address them or even revert the functionality if there is no way to address those.

Some assumptions:

  • Ground station owners are able to delete future scheduled observations on their ground stations. This functionality is not intended to change since it is our principle to allow ultimate control for the usage of the ground station to the owner.
  • Ground station statistics (and their calculation) will change in the near future. We are exploring ways that this information will be readily available to people that need it (owners, admins etc) and when publicly displayed there is sensitivity on how we do it.
  • Vetting will change in the near future, along with the definitions of “Good”, “Bad” and “Failed” (see our lengthy discussions here and here).
  • SatNOGS (Network and DB) is a public open data resource for space activities and should be treated as such, protected from data loss, accidental or not. (e.g. think of Wikimedia projects).
  • SatNOGS is growing fast and the needs for operational management of ground stations and the Network are growing. For example: we need to be able to answer complex questions for the health status of the Network and its stations and be able to automate notifications to owners and mods/admins.

So far we have seen in the previous discussion a few arguments for keeping the “delete past observation” functionality mainly around statistics and “noise cleanup”. I hope that my post here and @fredy’s post are providing some clarifications around those two issues. Another point that was raised was the issue of “control”. With the fear of diving into a complicated cultural-sensitive topic regarding ownership vs. control I would like to point out that once artifacts are produced by a station and then contributed to the Network (and DB), then they become part of the open data resource we are collectively creating and should be treated as such (regarding ownership, alternations, licensing etc). A more specific thread around this and its legal aspects as well as best practices and parallels to other projects will follow soon too.

So, I would like to invite anyone that has a possible use-case for re-introducing the “delete past observations” functionality to bring it forward in detail, keeping in mind the above mentioned assumptions.

Finally, a reminder to all, SatNOGS is an open source collaborative project in its development too. Detailed discussions around roadmap, features, fixes, etc are happening daily on the repos and the #satnogs-dev channel where dozens of people are participating.

ps. pinging people active in the previous thread: @ta2ank, @K3RLD, @cgbsat, @Zathras, @KD9KCK, @DL4PD, @cshields, @vk2byf


#2

My suggestions:

  1. IF stations are allowed to delete completed observations, that the station statistics continue to take this missing data into account.

  2. (and this one depends solely on the admins understanding of the networks data storage availability) if it makes sense to delete the waterfall and audio of “failed” observations in order to conserve disk space, I have no problem with that. I suspect that it is such a small impact that it’s not even worth considering, though.

–Roy
K3RLD