Too easy to schedule observations

1

I hate to be the person that brings this up, but I fear it is something that needs to be discussed.

It is simply far too easy for an observer to go crazy scheduling observations, and this makes it incredibly easy to abuse the system. For example, I can easily monopolize another stations time by picking very broad parameters for some satellites. The ISS is a perfect example.

A person can literally schedule HUNDREDS of observations PER DAY, but in order for me to “free up” time on my own station, I must manually delete each individual observation one at a time.

This simply is not fair to the station owners who were getting into the program to make their stations available “when not in use”.

I really see this as a fundamental flaw with the system. So what is the solution? I don’t know for sure, but my suggestions are as follows:

  1. Station owners can easily “overwrite” any scheduled passes by other observers with no need to manually delete existing observations. Owners’ stations are always “available” to the owner, unless the owner themselves had already scheduled a pass.

  2. Limit remote observations for any observer. I don’t know if a limit already exists, but to allow any one person to schedule hundreds of observations per day (if not more) seems like it is inviting abuse.

I love this system - I really do. However I really need to be in control of my own equipment when I need it. The current system does not allow that, unfortunately. I’d love to hear counterpoints to my concerns, however. Thanks for taking a moment to hear me out.

–Roy
K3RLD

2 Likes
2

Hi Roy

From:

station 144
operator bob

Like you, I don’t want to accuse anyone of abusing another gs. But, for several days, my gs is almost totally filled with scheduled passes. My Rpi is on a relaively slow wireless network which is shared with other devices. This causes a major problem as my gs cannot upload all the data collected before another pass starts. So, I get a failed pass and for some reason, the next upload apparently overflows the buffer, and both passes fail. This almost always locks the Rpi and it stops uploading data until i manually reboot the Pi, which sometimes does not work because the Pi is locked, and disconnecting the power source is the only way to restart.

As you can imagine, this is time consuming and harmful to the Pi.

Last week, I put the gs in test mode until all the scheduled passes were cleared. Updated Stretch, ansible and satnogs-client. Rebooted and the gs ran fine for several hours.

I missed some passes that I really wanted to capture this afternoon because the Pi was locked again.

I will put the gs back in test mode until I have some way of limiting the number of passes that are scheduled.I want anyone that needs to use this station to feel free to use it. But the physical characteristics of this gs cannot handle a full load of schedules.

I hope everyone understands.

This gs is offline until I can clear the scheduled buffers.

3

Hi @K3RLD,

Thanks for opening this thread, indeed the last months the number of the stations and the one of observers has grown up. In the beginning when we had a couple of station, scheduling was a job for a couple of people (sometimes we refer to them as observation team) who had a main role for organizing scheduling, by spreading observations among satellites, following special events and also try to keep up with the requests of station owners, like preferred satellites or less observations due to bandwidth issues.

Today, more stations means more observers which means that scheduling becomes a little unorganized and sometime touches the edge of abusing the system. Until now the focus was to help observers to schedule and keep busy the stations, but as you described we need to make changes to help station owners to have better control of their station and be useful for the network.

The ideal system and this is the one we try to reach with slow steps (for several reasons) is an automatic scheduling algorithm which will consider station characteristics, owner’s requests and preferences and satellites observation needs in order to achieve the best utilization of the network. However we are far from it.

So, we need to do another step on this direction. I’ll try to describe the issues and suggest some solutions that can be implemented almost right away. If there are any extra issues, please let me know.

  1. Owner wants always to follow some satellites

Example: As an owner I want to observe NOAAs on every pass above my station

Solution: Creating a list of preferred satellites for each station. Satellites in this list will be automatically scheduled before other observers get the slot of the pass

  1. Owner wants to be able to schedule satellites dynamically, even if someone else has already scheduled something

Example: As an owner I want to observer this NOAA pass because there are rare weather conditions

Solution: The owner will be able to overwrite already scheduled observations. This means that if a pass scheduled from the owner overlaps with passes from other observers then the latter will be removed. This makes the manual process of deleting one by one observations of other observers and then scheduling the one that owner wants automatic. The manual process is available already so make it automatic will help us save some time.

In this point I want to make a remark about deleting future observations. As we all offer our stations’ spare time to the network, a scheduled pass on a station means that there are good reasons for it and that observer was sure that the slot was available on this station. It is kind of unfair to have a free slot and then delete a scheduled observation on this slot. if a pass is really important for the owner it would be better to mark this slot as unavailable and let observers to know and schedule their pass in another station. With that remark I move to the next problem.

  1. Owner wants to mark his/her station as unavailable for some timeslots

Example: As an owner I want to make changes in my station, or I want to use it manually from this datetime to this datetime.

Solution: I’ve described it with details in this thread: [Network] Station status and availability

The above were a result of quick brainstorming and I would like to hear your opinion on them.

2 Likes
4

Hi @bob

I think that your issue could be solved partially by the solution on issue 3 I described on my previous reply. You can change (I don’t remember if it is available in satnogs-setup but we can make it if not) the parameter of how often the uploading task is executed.

With the solution in issue 3 and the right frequency of running the uploading task, you could make your station unavailable 2-3 times (depending on free available space) in a day and during these periods you can perform all the uploads.

However I think it is strange that rpi3 isn’t able to perform uploading at the same time with an observation. I think it is capable to do this, as far as I know many stations have back to back observations and perform them without any problem.

5

How do I solve this problem? This is pretty much a daily occurrence with the same person scheduling my gs. Take a look at the number of schedules he/she has logged.

This is not a contest: points do not count.

satnogs
satnogs1920×1080 288 KB

6

I really don’t think the problem is with the Rpi. My internet service carrier, which is Windstream.com. has not worked correctly for years, but since they have a monopoly here, the only other option is direct to the sats, and that is too expensive for a hobby.

I have run a packet sniffer for weeks, at intervals, and found that the internet service completely stops at random periods from a few seconds to several hours.

The short time failures of the internet service is the most troublsome, since they happen frequently and are not visible on my modem, until it looses connection totally, and not just a reduction in data transfer to as low as 2 KB/sec, in which my internet connection to the provider’s server is only a ping. Hopefully that will be solved soon as Windstream is replacing some of the equipment in this area, but they do not have a scheduled finish time.

7

As we all offer our stations’ spare time to the network, a scheduled pass on a station means that there are good reasons for it and that observer was sure that the slot was available on this station. It is kind of unfair to have a free slot and then delete a scheduled observation on this slot. if a pass is really important for the owner it would be better to mark this slot as unavailable and let observers to know and schedule their pass in another station. With that remark I move to the next problem.

As Bob has illustrated with his screen grab, there is NO “good reason” for an observer scheduling 300+ observations PER DAY, many (most?) of which are NOAA satellites on the other side of the planet. This is flat out abuse. Additionally, there is no good way for me to “mark this slot” if the station use is “first come, first serve”. I am a slave to the clock, like everybody else, correct?

–Roy
K3RLD

8

I think I gave some insight on the other thread, but let me write it quickly here too.

The good reason is that the specific observer has a general view of the stations (requests/abilities) and is able to utilize better the network as a result of his daily contribution on scheduling. As I pointed out on the other thread, this doesn’t mean that he can not reduce/stop scheduling so many observations and leave space to others to chime in.

About NOAAs I think that many owners have them in their preferred satellites list (in description until we fix this) as it is impressive to get earth images :slight_smile: . So they get some priority. Also we have 128 satellites in VHF instead of 328 in UHF, and 28 of the 57 online stations have only VHF antennas (+ those with both VHF and UHF), so I think it isn’t that strange to see many NOAA observations.

I would be happy to hear alternatives to “first come first served” solution, which is not exactly the way we use, as if there is a need (a missing satellite, special events, something we want to observe (a contact between two radio amateurs) etc), owners and admins can change the current schedule with the initial observers’ understanding.

9

As I mentioned in another post, why not give each gs the ability to limit the number of remote schedules from any one remote gs.Seems that would resove the problem of one remote scheduler taking almost 90% of available slots and leaving the owner with little recourse.

10

I am thinking of another approach: getting close to 100% of “free time” would be the goal for a SatNOGS station. But give the owner a priority on scheduling this “free time”. So if you, as an owner, want to have the “next two hours” of time with your GS, this should be prio #1, regardless what is scheduled. And as an addition: add such free times in the future - let’s say: every day from 16:00 to 21:00 as an example.

But a SatNOGS GS should be “busy” - in this manner - at nearly 100%, including a pre-defined duty-cylce.

1 Like
11

Possibly, there may be situations in which stations need to be remotely controlled, such as a life/death situation on ISS. But, this option needs to be used with prudence.

Or possibly a natural disaster in which amateur radio communication through sats are used for emergency communications.

my 2c

12

That’s easy you unplug the pi and plug it into your computer.

So what if the obs fail. The task would be more important.

1 Like
13

I was thinking more of using the satnogs system to monitor the sats and relaying emergency messages to places that have no other communication. The Pi with a thumbdrive is an excellent portablilty system that can operated for days from a vehicle battery. Receiving info in a disaster at times can be a life saver, as the rescue teams can know when and where the nearest shelters/food/water is located, Every bit of info about the circumstances of a disaster is very useful to the hero grunt groups in the field with a boat and paddle. pun intended.

14

Hello all, I’ve been following this thread and would like to address some issues

I want to make something crystal clear:

The scheduling goal for SatNOGS is 100% utilization of a station’s time.
We simply cannot have enough observations. (and no @K3RLD most are not NOAA, but even if it where we should be fine). The end goal for 100% coverage of any satellite we track requires we go down this road of 100% utilization of stations. This should not be considered an abuse. If anything those “schedulers” should be praised for keeping the network active and observations coming, so we can also debug, check stations and of course track as many satellites as frequently as possible. And as you can see here they are becoming more and more which is great! So yes it should be “extremely easy” to schedule observations by people that have access (active station owners), and we should work on automating the scheduling as much as possible. Check this thread for automation.

We have though to recognize the following situations:

  • Some stations will not be able to cope with 100% utilization (due to bandwidth and/or processing constraints). Thus, we should consider the targeted utilization being an editable (by the station owner) attribute of the station and the auto-scheduler should respect that.
  • Owners should be able to reserve time on their ground station (check this thread for details)
  • Owners should be able to easily override existing observations. They already can delete anything on their stations, but it should be indeed easier for them to just override existing obs.

This is not a contest, but every single observation counts. Blame the UI/UX (we are working on them!) and not the volunteers that keep the network active and are contributing tirelessly everyday (don’t forget the vetting!)

@all: assume best intentions in all cases and communicate when you have an issue or something seems off! A simple comment in your ground station description can go a long way!

9 Likes
15

Why? As Bob suggested, this is not a contest. Also, this is not a “for profit” business. 100% utilization of a ground station isn’t making or losing any money for anybody. It’s not like an airliner, where “downtime” is lost money. These are volunteer groundstations. As soon as you achieve a goal of 100% station utilization, you then completely eliminate the possibility of “last minute” type observations. For example, the recent non-scheduled ISS voice contacts could have sparked last minute observation requests from people, and if GS’s time is already 100% utilized, we have a wasted opportunity to collect data of tangible value.

What good is collecting hundreds of NOAA images (especially when the observations are vetted automatically - and the likelihood of ANYBODY ever actually looking at them is virtually zero)?

Having said that, I still very much want my stations to be available to EVERYBODY, for passes that are of genuine specific interest to any actual person. I have thus added to the description of the stations that they are free to be used by anybody with specific and targeted observation needs, but are NOT to be used for mass scheduling of observations for the sole purpose of “100% system utilization”. I will consider usage of this type to be abuse of my equipment.

It already appears that traffic has been cut down significantly, so I appreciate all users of the system, and encourage every body to use my stations for any targeted (specific) observation needs.

I also appreciate all the work being put into the code to allow for GS owner controls (scheduling priorities, duty cycle definition, etc.).

–Roy
K3RLD

16

Hi Roy

Good point. But there is a cost to running a GS. That is internet usage. Lot of GS are on limited internet plans and give up some/all of their internet usage to satnogs. Thanks to everyone for the changes and hope more decoders will be added soon.

17

So far I haven’t seen that much data usage and it doesn’t degrade my 3 down 0.7 up internet at all.

2 Likes
18

I’m watching with attention this very interesting discussion. I think it is very constructive and It has already helped me to understand better some things. So these are some comments on it:
About “The scheduling goal for SatNOGS is 100% utilization of a station’s time”.
The goal of utilizing a station by 100% doesn’t violate the owner’s right to use the time of his station in any way.
The solution we already work on, is that each owner will be able to set priorities of satellites and transmitters for his station.
An automated mechanism will be able to schedule the next observation in a short future period and in the same time will try to satisfy owner’s requirements.
It will also consider the already scheduled observations either if they were scheduled one by one or by massive scheduling.
And of course the owner will be always able to remove scheduled observations.
With the automated mechanism will not be any need for massive scheduling but only in cases of emergency, like new deployed satellites, targeted satellites and other cases like ARISS ones.
Unfortunately this mechanism can not be implemented quickly and soon as it is complicated and there are a lot of load of work for other things.
Also I think that it is good to have some time with manual observation in order to get experience as community on this field.
Today there is massive scheduling or scheduling by station with certain goals:

  • Solving problems related with the load of the network
  • Studying problems with the station related with their location and their strength
  • Studying and trying equipment, like how rotator and rotatorless stations perform.
  • Finding as soon as possible malfunctions or functionality of a Satellite, which is crucial for satellite owners
  • Getting experience in vetting observations (there is plan to automate vetting but always there will be need for manually doing it)
    About the NOAAs
    It looks useless to gather images which not many people seeing them due to automate vetting, however is it like that?
    When vetting will be automated for all the satellite modes, observation will be useless as no one will vetting them, so no one will sees them.
    One more thing is that having some powerful signal help to evaluate the performance of a station.

And something, maybe a little crazy, in the future someone could get all the data of the network and with the big coverage will be able to create a live earth image from space.
By closing I want to emphasize that we don’t have any contest here.
Thank you and I hope soon we will have a global meeting for satnogs in order to discuss and work on our concerns.

4 Likes
19

Sure, it’s a good thing, for valid observations.

But why do some observers, Motionlab Berlin for example, schedule so many obs then vet most of them as “Failed”. What’s the point? A lot of those birds have failed obs as the majority. All I see is red, red, red :frowning:

What I’d like to know is how is this affecting my “Success Rate” sats?

20

Maybe there could be a page per “schedule” an observer gets for his schedule plans.