Auto scheduling


#1

Subject : Auto-scheduling testing.
Reference:


I created a local implementation of auto scheduling on my station. I am using a transmitter priority json file and setting for minimum elevation to calculate and create json file for jobs locally. I have observations created this way with about 2 to 3 months data.

My current testing is on a setup fully autonomous and at remote location. It connects 4 times a day to the satnogs network api, gets the latest tles of all transmitters in my priority list and the priority list json file from a web page I control. All the observations captured this way are stored in a USB drive.

My code kludges involved a modified satnogs-client and other python code. Code is work in progress and embarrassing to share publicly.

Questions:

  1. Is there any satnogs api to push these recorded observations to the network?
  2. What is the time line/priority for creating such an api?
  3. I am using epoch time as observation number for testing. If satnotgs api supports uploading these observations, is there any suggestions on how to handle or convert this to use unique observation number on the network side?
  4. A satnogs API to get TLE’s used by the satnogs network using transmitter or Norad ID 's would be great. Can this be implemented?

#2

Well currently you can’t upload an observation unless one already exists on the network.

I can give you a python API I made for scheduling observations by abusing the UI endpoints.

Because you could schedule things automatically and let them run as normal.

This is the only suggestion I can give. Maybe some people who know whats being worked on can give info.

I have been pushing for an API for making new observations and I hope it happens soon. (Even if its only for your own stations.) I might attempt to write a MR for it.

Here is my old script that no longer works if you think it could be interesting to see how it use to work.


#3

Hi @Jferns,

Thanks for sharing, certainly a really interesting & important topic! I guess you already know the Automatic scheduling proposal formed during Hamvention 2018, or at least your implementation pretty much matches what was proposed. :smiley:

Currently all observations in satnogs-network have to be scheduled via the /observations/new/ in advance. I’ve created the issue satnogs-network#543 in order to track this issue. In the meantime you could use e.g. the code by @KD9KCK to create the observation in advance (there is curently no API, the server-side implementation of the current endpoint is in satnogs-network/…/views.py#L297) and let the satnogs-client upload the observation data as usual.

When you upload the observation, the unique observation number in satnogs-network is returned. So you can continue to use the epoch number as a local identifier (at least as long as no observation was created in satnogs-network).

Most of the TLEs in satnogs-network and -db are downloaded from CelesTrak (with the exception of 8 satellites not available via CelesTrak, see this notebook). Unfortunately as of now, satnogs doesn’t have the right to redistribute TLEs downloaded from Celestrak (citation-needed).

Sincerely,
Fabian


#4

Thanks @kerel and @KD9KCK for the replies.

I will check notebook for code to download TLE directly fom Celestrak instead of scraping tles from jobs api.

Looking forward to new observation request api feature. @KD9KCK I will check your gist code and test if it still works.

–JFerns


#5

I can tell you right now its broken. I am working on fixing it over this week some time.


#6

I will make a simple script that if given start and stop time, The transmitter for the sat. (UUID from satnogs Database) ,the norad and your station number. Can schedule the observation. but you will have to provide the information as I can’t seem to get the UI’s internal-use api to give me prediction information.

Also be aware this could break at any time because yea all these apis are undocumented and internal use only type of apis.


#7

A while back I wrote some a tool, based on SatNOGS network code, to compute satellite passes given a ground station and a set of TLEs, and create firefox commands to network scheduling API to quickly schedule passes one at a time. The tool would sort the satellite passes on user defined priorities (NORAD IDs for now), or on their maximum altitude of the pass. The tool is at https://gitlab.com/cgbsat/auto-scheduler/tree/master/

The ability to automatically schedule passes on the network would allow me to use this tool to regularly, say every few hours, check if my stations have scheduled passes, and if not, fill up the schedule using user define satellite/transmitter priorities.

The tool needs to be expanded to:

  • automatically download TLEs using @kerel’s TLE downloading script
  • use the SatNOGS DB to get transmitter information
  • use transmitter information (frequency, success rate, status) for computing passes
  • use the future scheduling API to schedule passes

#8

I threw together this.


Which allows one to make an observation on the network.

BUT you must provide the times in UTC in the format below as a string.

“2018-11-16 18:14:41.423”

For specifically it has to be a date in which pyephem can understand.


#9

Many thanks @KD9KCK!

I’m trying it out, but it doesn’t appear to work. Can the NORAD ID and sation ID be given as integers and the rest as strings?


#10

Yes. I should put together some docs for that annoying function.


#11

I used the following.

MakeObservation(USERNAME,PASSWORD,40967,“ZyjKNJ9KqnTHBCUzAPN5G5”,41,“2018-11-19 07:38:28.000”,“2018-11-19 07:52:00.00”)
To make the following observation https://network.satnogs.org/observations/327952/

I got the times right from GPredict in the default UTC times mode.


#12

It works! Turns out it needed fractional seconds in the timestamps (e.g. 2018-11-18 23:04:20.0 instead of 2018-11-18 23:04:20).

Nice work!


#13

I also just wrote some code to be able to get all the nextPasses displayed on the station page. Along with the overlap. (Took a bit for that)

I also made a function to pick the best transmitter that the station can receive.

I will update the file later today.


#14

There’s some code for that in my repository, but it might not do exactly what you need.

Incidentally, I’ve been working on fetching transmitter information given the antennas present at a station, and then fetching the TLEs for it.


#15

I mean I already write the code to do that yesterday, (it gets all passes.)

And if you giver this function the NORAD the function finds the transmitters your station ad actually hear and then gives them in order of success rate.


#16

On a side note I will have to rewrite this once the new OAuth changes happen.


#17

@KD9KCK Thanks for sharing the code for creating new observations.

My station is 22 is now back online, all observations are created and scheduled locally and satnogs-client logs the observations to the network.


#18

Just be aware that the code is going to break as soon as the new OAuth stuff is built. But hopefully by then I will have a MR for a public api.


#19

After a few days of testing my auto-scheduler tool appears to be stable.

When given a ground station ID, it obtains the antenna properties of that ground station (mostly frequency ranges) from the SatNOGS database. Next, it gets the properties of all active transmitters in the database, and downloads TLEs using @kerel’s TLE downloader. These TLEs are cached locally, and only updated if the last download was more than 24h ago. Passes for these satellites are computed for a given time frame, and these passes are given a priority based on the transmitter success rate as well as the maximum pass elevation. This list is then sorted on priority, and each successive pass that does not overlap with previously scheduled passes is scheduled on the network.

On my stations (#39 and #40) I run my auto scheduler hourly from a cronjob, scheduling passes for the coming hour. This way, the auto scheduler keeps the stations busy, while allowing other users to schedule passes further in the future.


#20

Nice work @cgbsat. I noticed in the patch from @kerel that it requires matplotlib; however, at a quick glance, I can not find any references to it. Is it actually required. If yes, for what?