Sudden jump in data count in SatNOGS db

Unfortunately, no… and thanks for bringing this to our attention as there’s a pretty big bug…

Looking in the db of db, there are duplicate and triplicate entries for @n5fxh

My suspicion:

The task that we have running to sync frames from network to db is running every 15 mins (despite being configured to run every minute… hmm). However, due to adding more transmitters and gathering more data, (along with an unrelated error getting thrown by a > in the CMD list from ISS BBS), some of those runs are going longer than 15 mins and another run is causing a sync to be queued before the previous sync is checked to be synced.

My suspected culprit (and fix) is going to be in the calling of demod_to_db as a celery task, which allows it to queue multiple times. This could/should be handled by the parent task, and in hindsight I see no reason the individual task needs to be a celery task of its own (which may help speed things up in the end). That said, I’m going to pull @Acinonyx into this thread as a second set of eyes.

My suggested fix for this:

  • move demod_to_db to utils instead of a celery task
  • the sync_to_db celery task will iterate through each demod instead of queuing them
  • add a check to demod_to_db that looks for the decoded flag before continuing, as an added protection
4 Likes