Install the satnogs_gr-satellites "addon" and use your station more!

SatNOGS Software
63

Many thanks Daniel for your help

64

Given the back-and-forth above, what’s the best script to follow right now? It’s not clear to me that the instructions on GitHub are as up to date as some of the instructions in this thread, but I’m very keen on giving this a try.

65

If you are new to Linux and not experienced with the command line, maybe somebody can help you who is.

66

I have read this thread (and the related one “integrating-gr-satellites-into-satnogs”) with great interest. We have taken a somewhat different approach to the problem of generating decoded data from satellites that SatNOGs doesn’t currently support – the AO-40 satellites such as Funcube-1, Nayif-1, and others such as DhabiSat – and thought the community might be interested in a description of what’s been done, should anyone else want to try this.
Firstly I should say that our approach is only possible because it builds on all the heavy lifting already done by @EA4GPZ and @SA2KNG, in fact the extra components required are rather minor.
Secondly, our motivation was in fact the opposite of integration – we wanted a way of adding decoding capability for a minority of targets which interfered as little as possible with the existing SatNOGs installation. For me, one of the great attractions of SatNOGs is the ease of use and automation. We do not want to be tweaking / customising the installation at all if this can be avoided, as this creates potential upgrade headaches, the potential for malfunctions etc etc.
So, we didn’t want to modify the flowgraphs to export IQ over UDP. Whilst we could have achieved IQ export without modifying the flowgraphs by turning on IQ file logging and sending the output to a fifo, there is always the risk that a problem down the line blocks the fifo and stops the flowgraph – bad!
Also, we have had quite a good experience with using gr-satellites to post process SatNOGs audio recordings – whilst at the margin, no doubt having the original IQ is better, in most cases we are interested in, we know the signal quality is good and so there is little or no practical difference in results whether the source is IQ or audio (provided you implement the 12KHz audio offset…).
So the result of this thinking is that we concluded we didn’t need to interface with the SatNOGs flowgraphs at all – all the data needed is online in the SatNOGs network. Gr-satellites can be run as a post process on SatNOGs audio in quasi real time provided you have the observation start time, as implemented by @EA4GPZ and @SA2KNG.
To make this work, you need to capture the following pieces of information, from these all the elements needed to download the audio, use the correct start time and configure gr-satellites for the correct decoder profile can be determined.
These are:

  1. The observation start time
  2. The satellite ID, eg the NORAD ID
  3. The observation ID.

That’s all you need.
Our first attempt used the pre-observation script to capture these elements. As this script is the first thing that happens when an observation kicks off, “now” as captured by the script should be well within the first second of the observation and so should give an accurate start. The observation ID can be captured from the parameters passed by the SatNOGs client, and the NORAD ID extracted from the TLEs. The url of the audio recording on the SatNOGs network can then be constructed from the start time and observation ID.
This worked, and we were able to automate everything so that:

  1. The url of the audio .ogg is generated and the file downloaded using wget
  2. Depending on your version of GNURadio, the .ogg can either be read directly by gr-satellites or can be converted to a .wav using sox.
  3. The command line for gr-satellites is configured using the NORAD ID and the audio file, so that .kss files are generated – see @SA2KNG’s script here satnogs_gr-satellites/grsat-wrapper.sh at main · kng/satnogs_gr-satellites · GitHub for the principles
  4. @SA2KNG’s script here satnogs_gr-satellites/kiss_satnogs.py at main · kng/satnogs_gr-satellites · GitHub then reads the .kss file, JSONifies the output and creates “per packet” files which are placed where the SatNOGs client expects, so they can be uploaded.

However this approach was not ideal – one of the main problems is that the url construction changes depending on the age of the observation. Very recent observations have one format, after a few days though they are archived and the url has a different format. So this is a bit cumbersome, and might require trying both formats to capture the file.
We then realised that each observation has a webpage and unique url solely dependent on the observation ID. In fact, if you have the observation ID, all the information you need to configure post processing with gr-satellites is on that page. So all you actually need to capture is a series of observation IDs, from there the information / data needed is all available via the webpage.
How to capture / extract this? We used the well known python web automation package, selenium. Also our system, although on a desktop, sits in a rack with remote access so runs headless. So the browser instance accessed by the webdriver used by selenium has to be configured to run headless so it doesn’t try to open a browser on a (non existent) screen.
The code you will need to replicate this approach looks something like this, firstly to set up the selenium components:

#!/usr/bin/python3 
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.firefox.options import Options as FirefoxOptions
options = FirefoxOptions()
options.add_argument("--headless")

Here is an example function to capture the required information:

def get_attributes(observation_ID):
    observation = observation_ID
    target_url = "https://network.satnogs.org/observations/"+ observation + "/"
    driver = webdriver.Firefox(options=options)
    driver.get(target_url)
    elem1 = driver.find_element(By.CLASS_NAME, "datetime-date").text
    elem2 = driver.find_element(By.CLASS_NAME,"datetime-time").text
    timestamp = elem1 + "T" + elem2
    fileurl = driver.find_element(By.XPATH, "//*[contains(@href,'ogg')]").get_attribute('href')
    norad_id = driver.find_element(By.XPATH, "//*[contains(@data-target,'#SatelliteModal')]").get_attribute('data-id')
    driver.close()
    return(observation,norad_id, timestamp, fileurl)

(Note the selenium syntax has changed fairly recently, so sources such as stackoverflow might be confusing if they refer to the previous style)
 As you can see, the web page elements which contain the information needed are identified by a combination of unique classes, the unique hyperlink referencing the .ogg file, and the identifier of the element containing the NORAD ID. You can find these the usual way by opening the browser developer tools, right clicking on the element and selecting inspect.

We have also tested this on a Raspberry Pi, here you have to use Chromium and the Chrome webdriver rather than Firefox, but otherwise everything is the same and works headless.
Hopefully this write up might be useful to anyone wanting to add currently non decoding satellites to their SatNOGs installation whilst making as few changes as possible.

4 Likes
67

Thanks for documenting your approach!

If you want to get rid of selenium, you can also use the satnogs-network API to get the required information. The NETWORK_API_TOKEN is the same as the token used to register stations.
If you are already logged in to the network, the API is browsable.

import requests
NETWORK_BASE_URL = 'https://network.satnogs.org'
NETWORK_API_TOKEN = 'YOUR_NEWORK_API_TOKEN'

def get_attributes(observation_id):
    url = f'{NETWORK_BASE_URL}/api/observations/?id={observation_id}'
    header = {'Authorization': f'Token {NETWORK_API_TOKEN}'}
    r = requests.get(url=url, headers=header)
    obs = r.json()[0]

    return (observation_id, obs['norad_cat_id'], obs['start'], obs['payload'])

I assume that this works with old and new observations, if not please let me know.

Best wishes,
Fabian

2 Likes
68

Interesting! I actually working with Selenium in my professional life as a Q&A developer, but this was a new way of using it. But the API solution is a cleaner way.
Will see if I can get any spare time to test this.
Thank you all.

2 Likes
69

Hi Fabian, yes this works with one easily fixed proviso. For archived observations, ‘payload’ is null and the url for the .ogg files moves to ‘archive.url’. So you could retrieve both and use ‘archive_url’ if ‘payload’ returns None.

Regards

Brian

1 Like
70

You could also try to pull older observation data files from archive.org to save on Libre Space bandwidth, when possible. I have a scriptlet that hits archive.org to download the audio file, for example at:

It pulls the XML file from archive.org and then pulls the audio file from it. Here’s a sample URL:

  • https://archive.org/download/satnogs-observation-$OBSERVATION-ID/satnogs-observation-$OBSERVATION-ID_files.xml
2 Likes
71

I’ve been wanting to add gr-satellites to my SatNOGS station for some time but I’ve been leery due to my ignorance of Linux.

I took the plunge yesterday and got gr-satellites installed and it appears I have no errors.

My question is how do I implement more than one pre & post observation script?

My station uses the RTL-SDR as a receiver and I am using the bias-T script to power an LNA on and off for each observation. I’ve searched everywhere I can think of and can’t find the answer.

Sorry for such a noob question.

Thanks,
K3TZ

2 Likes
72

You can include what you want in the script itself. For example, you could have the satnogs-pre script call another script or just do whatever it is you want done. So if you want to turn the bias-T on (adjust to suit your system), your satnogs-pre script would look like this, with a bias-T line added:

#!/bin/bash
# SATNOGS_PRE_OBSERVATION_SCRIPT
# /usr/local/bin/satnogs-pre {{ID}} {{FREQ}} {{TLE}} {{TIMESTAMP}} {{BAUD}} {{SCRIPT_NAME}}

/usr/local/bin/grsat-wrapper.sh start "$@"

/usr/local/bin/rtl_biast 1

Then the satnogs-post would be something like this:

#!/bin/bash
# SATNOGS_POST_OBSERVATION_SCRIPT
# /usr/local/bin/satnogs-post {{ID}} {{FREQ}} {{TLE}} {{TIMESTAMP}} {{BAUD}} {{SCRIPT_NAME}}

/usr/local/bin/grsat-wrapper.sh stop "$@"

/usr/local/bin/rtl_biast 0

You may have to change /usr/local/bin/rtl_biast 0 to match how it is enabled on your system. (As a side note, some SDRs used to need that script, but don’t anymore, such as the Nooelec).

(Maybe someone more familiar with satnogs-gr-satellites could clarify if it is non-blocking or not–I’ll need to check. If so, the rtl_blast line would go before the grsat-wrapper.sh line.)

2 Likes
73

@jebba I appreciate your reply but I am still having trouble.

Since I’m such a Linux beginner, I use WinSCP to navigate around to the files and folders. I could not find the grsat-wrapper.sh file anywhere. I used the search function of WinSCP starting at the root directory and it couldn’t find it either.

I used the instructions to install gr-satellites from here.

Is this file something I have to copy from here? Was it supposed to be installed with the gr-satellites package?

I also didn’t find the satnogs-pre or satnogs-post, I assume I would create these. I also assume that they would have to be made executable with chmod? (see, I told you I was a noob) Do they have to have a particular extension? .sh?

Once I get this sorted I think I’ll be on my way.

Thanks again for your help,
-K3TZ

74

grsat-wrapper.sh gets installed when you install satnogs-gr-satellites, so you should have it installed already. The scripts it installs get put in /usr/local/bin/.

They don’t need to have a particular extension, it is just convention. They do need to be executable, but they should be already by the installer.

1 Like
75

Ok, I see what I did wrong…

I have the files I need and they are in /usr/local/bin

My next issue is that the owner is root and I am logged in as pi so I can’t edit them.

How do I get around that?

image

Thanks once again,
-K3TZ

76

This is kind of crufty, but one way if you get ssh access, you could run:

sudo chown pi /usr/local/bin/satnogs-pre
sudo chown pi /usr/local/bin/satnogs-post

Then you can run your sftp client as user pi and edit those files.

Another way would be to upload the files as use pi to /home/pi then use mv or cp with sudo to put them in place.

77

That spawned a memory that I could use ssh and edit them with a sudo nano [filename] so, that’s what I did.

I’m pleased to announce that it is working and decoded my first observation of NAYIF-1!

Now to pick out some birds to decode…

Thank you so much @jebba for all of your help!

73,
K3TZ

4 Likes
78

Hi,
is gr-satellites can demod LRPT …?
(Meteor M2)
ve2dsk

79

According to the supported satellites list, no.
If you have any software for that task that can run from a terminal and use either udp input or stdin, it could be doable. The wrapper could launch that on LRPT observations based on norad id perhaps.

80

@SA2KNG, We have that already available. See my howto:

@ve2dsk is one of the users of this howto.

81

Hi SA2KNG…
Before gr-satellites option on my system, i had a previous installation with LRPT demod/decode scripts…
Disapear with new installation client v1.8.1.
I chose gr_satellites instead… but i’m not shure if i can brick LRPT inside… ve2dsk

82

Hi Eelke…
I have already participate to this topic…
Work very well.
Now new installation with gr_satellites option… ve2dsk

1 Like