Automatic scheduling


#1

What, if any, are the plans to allow and implement automatic scheduling of satellite passes?

For my ground stations I’d like to be able to define a list of favourite satellite/transmitter combinations, and have these scheduled automatically, depending on maximum altitude and conflicts with other satellites in this list. The remaining free time can be used for passes scheduled by others. This feature could be implemented as part of the SatNOGS network, but will likely be a significant task, or as a client program, in which case authentication with the network will be required.

How would automatic scheduling fit in with the overall SatNOGS plans?


Auto scheduling
#2

there’s a proposal in the works for this, @pierros is doing to document and put it out there someday soon :slight_smile:


#3

It would be great to have an API for this to implement own scripts - maybe only as an intermediate solution until there is an implementation inside the network code.


#4

Hello all,

During Hamvention 2018 a bunch of SatNOGS contributors met together to brainstorm about the best way forward towards a completely automated scheduling system. Although many directions have been proposed in the past, @Acinonyx spearheaded a new radical direction based on some assumptions and observing the scheduling work done by our regular observers, mainly: @BOCTOK-1, @Saliyath, @gpanag95, @SV1RVP, and others.

Disclaimer

This is a draft proposal and by no means a done-deal direction for SatNOGS Network and Client components. We are inviting as much input and criticism to this proposal as possible to help figure out which is the best way forward for the scheduling Architecture.

Assumptions - Specs

  • The scheduling process needs to be fully automated without a manual intervention
  • Manual observations need to be accommodated it too (observers would still want specific results)
  • The number of stations in the Network will only keep growing
  • Not all observations are equally important
  • Ground stations should be kept busy 100% of the time (unless the owner does not want that)
  • Network and Client resources should be used optimally

Existing model

Today the scheduling is happening as a manual process with the Network constantly predicting “passes” of transmitters for available ground stations. Note how I am not saying “passes of satellites”. It is critical to understand that the basic node of statistics/predictions/passes/observations is the “Transmitter” (or transceiver if the uplink is enabled) as a stored combination of frequency(-ies)/modulation/encoding and information.

Proposed concept

The core concept of the proposed scheduling and observations architecture is the following:

  • Based on a number of prioritization factors (see below) the network maintains a prioritized list of transmitters (thus satellites too) tailored for each station at all times (regular updates are expected in that list).
  • The client constantly queries the Network for an updated version of that list (even during an observation!), and goes through it to calculate if a transmitter (thus satellite) is having a pass. Going through the list in a sequential manner ensures we meet the prioritization criteria.
  • The appropriate Gnuradio flowgraphs are spawned for the duration of this favorable (for the transmitter/satellite) period and results are captured/produced (WF, Audio, Decoded data)
  • All results are posted back to Network, creating a new observation in the process of doing so. (Note how there is no “pending” observation scheduled like we are doing now on the Network, waiting for results)

In practice, the client will be going through the tailored for it list fetched from the Network and one by one checking if the pass is feasible (above the horizon), if not move to the next priority till it is feasible and will spawn an observation.

Prioritization factors

Many different factors have to be taken into account to compile the per-station transmitter list. I will try to list them here not necessarily as a prioritized list :wink: :

  • Frequency capabilities of the station (obviously we will not be serving to a station a list with transmitters it cannot monitor)
  • Preferences of the station owner (self-declared by the owner through Network, or Client?), like a “must monitor” satellite list
  • Statistics of transmitter quality/success rates based on previous observations (but we would also want to monitor some “dead” ones every once in a while)
  • Distribution of observational load, and deduplication among stations (let’s show why GS Networks are really unparalleled as an infrastructure!)
  • Post-launch TLE and Freq determination priorities (when new satellites are deployed or something changes in existing ones)
  • Manual opt-in scheduling by observers, will be dealt as essential a change in priorities for a station

Note: the Network is not agnostic when it comes to what it is expecting from Stations/Clients to do. As we will be compiling the prioritized lists on the Network, it will be able to calculate exactly what it will expect from Clients on Stations. The key difference is that it does not have to calculate it, unless observers want to monitor upcoming passes or schedule manual ones (by essentially changing priorities ). This might seem like a small difference but for the scale of things SatNOGS Network is going for this will be critical.

Transitional way forward

Obviously, there will be a lot of fundamental work and testing that needs to happen in order for us to transition towards any auto-scheduling approach (let alone this radical one!). We are committed though, to a smooth transition without any major hiccups on the Network and overall operations of SatNOGS. We will be starting with some key changes that need to happen around statistics (moving towards transmitters which are rated and not satellites, based on passes) that will need to happen regardless the decided end-model for automated scheduling. Keep an eye for those in the DB and Network in the coming weeks.

Open Questions

Please post here any questions, criticism, suggestions etc you have regarding this proposed approach so we can collectivelly form it to be the best way forward for SatNOGS.


Too easy to schedule observations
Auto scheduling
#5

Another thing to realize if that some data have multiple transponders within the bandwidth of an rTLSDR dongle. This means you could technically have a GNURadio Script handling both of them for a single pass.


#6

We would have to change the way the scripts are designed.
There is e.g. the possibility to have a generic IQ receiver that feeds samples into a UDP sink and connect as much, as there is CPU power and RAM available, demodulators and decoders (maybe split both up in another UDP connection) using this as source.

P.S.: If you would like to see an example of what I mean: the DSLWP lunar orbiter OOT package is designed like this.


#7

Thanks for the detailed plan @pierros! This looks like a good and challenging plan to work on.

The creating a new observation on the network by the client after a pass seems quite a different way of running the network and client, and possibly not compatible with the current way of doing things. Would it be an option to use an intermediate approach where the client schedules a set of observations based on the priority list and reserves these on the network (set them to a pending state)?


#8

@pierros will it be possible to support a truly mobile station in future ie with a gps module within the scheme of auto scheduling which will calculate the observations based on current location instead of settings in network page.

The new proposal makes me think it might be possible as the scheduling will client initiated.


#9

“Prioritization factors”

Without going into details of a proof of concept I’m toying around with, let’s think of automatic scheduling via network.satnogs.org as a small part of the entire SatNOGS ecosystem, and imagine the scenario where the following are true:

  • overriding of priorities and manual observation scheduling is always available, as @pierros already proposed
  • decoded frames from network.satnogs.org are making it to db.satnogs.org
  • frames from db.satnogs.org are decoded into a time-series database warehouse for consumption and visualization
  • said time-series database should be assumed to have irregular time series data because of all of the factors involved in collecting this data (in other words, we can’t assume today that we expect a frame of data from satellite X to arrive once an hour… and this bullet point is the problem I propose we solve given the previous bullet points are already implemented)

Now, let me propose that our end goal be the regular collection of data, as frequently as possible based on our capabilities, telemetry forwarder contributions, and network. Regular, and frequent. Consistency & regularity trumps frequency (1 data point every 4 hours is better than 1 data point at 1 hour, then one at 8, then one at 9, then one at 20). We would take what would today be an irregular time series data and try to make it regular through automation.

In my scenario above we would have the data we need to calculate what transmitters to collect from next - but it would be calculated based on the results that end up in the data warehouse, trying to create a consistent stream of data (based on our capabilities and the satellites up there). This way, the results are what matter.

And, since we focus on the results at the end, our automation would account for the data we are collecting through other means outside of network.satnogs.org. (we have over 18.6 million frames in db.satnogs.org today and over 193,000 observations in network.satnogs.org) Today, we have no control over the satellites that are focused on for users contributing to db.satnogs.org with telemetry forwarders… we may be able to influence them socially but let’s assume they are a fixed value. There are roughly 150 contributors providing decoded data for some 100 satellites. Some of the satellites we would never touch with the network automation because we already get enough data elsewhere (let’s say STRAND-1). Or, maybe it becomes the case that we have seen good data for STRAND-1 consistently every 4 hours and suddenly it has been 5 hours without data, if it is possible for the network to fit that in to fill in the gap then we should do it, and this would be an automated process.

(take “4 hours” and “5 hours” as a very rough example - I haven’t put any real calculations into this and it very well may be that our expectations would be “once a day” per satellite… and this also depends on the network growth meeting or exceeding the growth of transmitters out there)

Anyway - I still think the “prioritized list” method is a good approach to take, and maybe this bigger picture method becomes a v3, I just want us to take a step back, look at the end result (data), and work backward from there to see what works.

In this scenario, a “prioritized list” becomes less of a list for the individual ground stations to pick from, and more of a scheduling change between “1 frame every 24 hours” for one satellite vs “1 frame every 4 hours” for another… Now, this puts the onus of scheduling back on the network, but if we are to think at scale 5-10 years from now the results would be more consistent.

Cheers!


#10

One part of the proposed concept for automatic scheduling involves the local scheduling of observations.

A first step for this is imho to allow local scheduling in the client.
If satnogs-client provides a local interface which offers the already scheduled observations (in form of start and end time) and allows the creation of new scheduled observations, a local service could perform the long-wanted auto-scheduling (e.g. based on cgbsat/auto-scheduler, and implementing priorization ideas from above).
Observations created via network would be added to the local list of scheduled observations as currently by fetching the list of network-scheduled observations.

What changes/implementation steps would be required to allow local scheduling by a local service with the current client architecture?