Tips and Trick: How I Vet 100 observations in less than 5 minutes

log to your satno dashboard,
and click the orange number at top right corner:
image

or make custom vet:
image

and then click Vet Search Results button
image

and to make vetting faster, dont use mouse.
just put your left middle finger standby on G and left index finger on B of your keyboard, and just press:
g = good = there is a signal from the expected satellite
b = bad = no signal on waterfall
u = unknown = if you want keep it in orange status

if you have another faster trick for vet, please share with us in this thread.

thank you for your attention & good luck!
73!

7 Likes

Instead of using u you can also use the right arrow with your other hand to skip the current one… also it is easy to go back by using the left arrow if needed.

Depending on the amount of observations, it may be easier to go per satellite, this is very helpful as you have only one pattern in your mind to identify.

As I’m usually vet per satellite here are some more advices for this method:

If the satellite is pretty active, vet only the clearly good/easy to spot signal in waterfall, so only hitting g or right arrow and then go for the unvetted ones that need better attention.

If the satellite is less active, vet only the clearly bad/easy to spot no signal in the watefall, so only hitting b or right arrow and then go for the unvetted with signals and determine if the signals belong to the satellite or not.

The above methods can be used also for vetting multiple satellites but more difficult to spot or not the signal as per satellite there is a different pattern.


As sometimes there is a need for re-vetting a satellite’s observation, here are the steps I’m choosing to follow:

  1. Filter by choosing With Signal in “Rated Artifacts” filter (in more filters). This give you all the observations with waterfall vetted with signal, so with a quick pass you can agree with the vetting and hit the right arrow or disagree and hit the b button.
  2. Then filter good observations (keep enabled only the green tick) and choose Non Rated in “Rated Artifacts” filter (in more filters). This will give you the observations with data that automatically marked as good. In general case the data belong to the satellite, and vetting waterfalls is mostly limited to hit the g button.
  3. Then filter Without Signal in “Rated Artifacts” filter (in more filters). This give you all the observations with waterfall vetted without signal, so with a quick pass you can agree with the vetting and hit the right arrow or disagree and hit the b button.
  4. Then filter Non Rated in “Rated Artifacts” filter (in more filters). This give you all the remaining observations with unvetted waterfall, so you follow the standard vetting process.

Part of the above can also be used when you don’t focus only on one satellite, but as I said focus to one satellite makes things easier.


While the previous method is good, there is a case that I prefer not to use it, this is when the satellite has issues and we have few or none good observations or in other words when searching if the satellite is alive.

In this case I choose to go with getting all the observations and vet them carefully using what @bali described, but keeping the already vetted once too. In this way due to some overlapping observations, it is easier to say if the signal you see in this observation is also visible on the next/previous which performed on the same time, as the observations are ordered by start time. This way is easy to spot terrestrial signals or signals from other satellites, however…

Caution! Not seeing signal in one observation but see on another at the same time doesn’t necessarily mean that the signal doesn’t belong to the satellite as if station has observed signal or not depends on various parameters, for example different pass elevation, different duration (which currently affect waterfall size and resolution), station status, station calibration, station receiving capabilities, noise levels, satellite pointing etc.

By the way vetting by keeping in mind the (stations) networked nature can give advantages, like the one described above.


Finally I take the chance to talk about the upcoming and currently developed feature of multi-vetting. That in its first phase will allow vetting of one waterfall from multiple sources/ursers (no need to be a station owner to vet as it is now), that will result in more automation (tools that will analyse the waterfall (or other artifacts in the future) and auto-vet) and by suitable tuning (simple for this phase but better in the next ones) will give more accurate/trusted results.

3 Likes

wow . thank you for deep advice.. left / right button, focus on one satellite, etc are very helpfull.

1 Like

before 16885

now 9982:
image

2 Likes

I appreciate your efforts… seriously…

As a station owner, I’m looking at “foreign” obs on my station that are >2 weeks old, and doing a looksy. I find that I have a lot of terrestrial interference ~400-401 MHz, and that’s giving me a couple ideas for experimentation station(s)… testing other antenna to see if I can reduce/eliminate this noise.

So much to do…

1 Like

thank you! vetting a thousands waterfall make me burnout. hope there is some smart guy that can create genious / clever trick to auto vet this. so lazy noob like me can take more rest, with just one click.

btw i just wondering how @freddy not burnout… incredible!

also @PE0SAT , salute!

note: you can check everyone orange vetting number.. all is open. just use observations > browse

but the positive is, it increase my muscle memory to identify signal on waterfall with just look.. not much. but better than one month ago.. lol

Just to mention here, that vetting isn’t obligatory. You can check more details behind the reasoning in older threads of this forum (not able to find and link them right now).

1 Like

image

i found genious trick, that make my vetting faster 50% and comfort especially for my eyes and hand, no need scroll up and down.

how? just rotate your monitor 90 degree / vertical mode.

update:

loading my final stage:

You can DO it!!!

1 Like

thank you friend!

finally.. after vetting more than 16K observations in 20 days

73!

1 Like

And you wrapped it up at 11:11… not only did you do a mountain of work, you did it with STYLE!

1 Like

@verdantfoster , i’m sure your image_processor.py with some modification can be used for this job…

73!!

poke @pjm , @fredy , interesting…


vetting 16K observations vs 5K observations. lol

1 Like

Wow, thank you very much, its way much more easier than opening each observation status!

1 Like

Indeed! I guess this is the way we are going to move forward.

We can train different models, per satellite, per signal etc and do the massive work with the human to intervene in special cases, like near deployment/re-entry cases, not regular transmissions etc.

This is why we are moving towards to multi-vetting and opening API endpoints for vetting.

1 Like

Whatoh chaps,

This is where my project is at present; its running on a nvidia jetson TX2, yes its a bit old but was sat around not doing anything useful.

The aim is to have a computer vision (CV) solution that can examine the waterfall and decide if there is a signal there, even in the presence of an interferer.

Its mostly cobbled in python. I have a script that polls satnogs every 5 minutes for new observations from my station (91), it then downloads each waterfall image, decodes it (inverting the colour map back to relative power, reading the frequency axis for calibration), it extracts roughly 40 features (column sum peak sig/noise radio, persistence tracking, burst periodicity, curve vs vertical discrimination for interfering carriers, etc..), and runs them through the currently active vision classifier to get a confidence score and a verdict bucket which determines signal, no_signal, or uncertain if the confidence falls between the 85 - 15% thresholds.

Anything the CV cannot determine is reviewed by me and is queued up so I can pick over it each morning. The queue is sorted most-uncertain-first, I’ve laid it out with the CV analysys and the satnogs waterfalls next to each other so its very quick to vet, plus the CV analysys image is pre built so reviews are fast (about the same speed as the satnogs obs vetting page). All of my reviews feed back into the model that the CV model is ultimately trying to match.

Completly automatic vetting is a separate process. every 15 minutes it rechecks confidence values for my station and if it clears a strict bar (>80% signal or <5% no-signal), it labels and submits the vetting status straight to the network without my involcement.

Underneath all of this another script runs nightly, pulling in fresh community vetted training examples from other stations network wide and retraining the classifier on up to 1k5 labeled examples, reporting cross validated accuracy and only promoting a new model if it isnt meaningfully worse than the one currently live, so the traning / vetting process is self-reinforcing.

The idea of the process is that with each day the model becomes better, there are stats to let me easily see how much the CV vet has submitted vs how many I manually do.

My station has done 431K observations since its been running and its high time a computer learnt to do the needful :wink:

IT is a work in progress, I’ll report back after a few more weeks of CV training. Currently looking at saving off the IQ from my station, and processing that with the jetson etc. Will let you know how it looks, however this isnt a magic silver bullet that will also solve all problems with vetting.

Happy to run it on other stations obs to classify easy to identify signals if that is of interest, just let me know your station number etc.

regards,

Paul (pjm @SatNOGS IRC)

1 Like

can you screen record how the cv vetting and submit to network?

my station number 4451