Glouton : a satnogs data downloader


#1

Hello everybody,

I’m new in satnogs community :slight_smile:
I have started to release a satnogs data downloader. It’s a console app where you can enter a norad id, a start date and an end date, and it downloads all the waterfalls and payloads available. For me it’s juste the skeleton, in futur i would like to add more features (all related on the github page).

I didn’t find that kind of tools actually. I hope it can help some of you in any how. But if you are not interested i wont talk about it anymore :slight_smile: i just wanted you to know if some of you are looking for that kind of stuff.

Of course any advises/ideas or contributions are welcome. I’m pretty new in python/radio world so, it’s the occasion to learn. And like i said its the beginning so, i 'll constantly refactor the code.

The github repo : https://github.com/deckbsd/glouton-satnogs-data-downloader

I hope in the futur i ll be able to add my ground station into the network. I actually have a discone , but i’m not able to transmit data H24 7/7 for the moment.

Best regards and sorry for my bad english


#2

Hey @deckbsd – that looks really cool! I’ll be sure to take a look at it.


#3

Yop @saintaardvark thanks for your reply. I’m happy by you interest.

For informations i just added the docker flow (Dockerfile, automated build, …)


#4

Hey @deckbsd great work.

A small suggestion, please include a license in your code repository.


#5

yop @elkos You are right, but i’m really not a expert in term of license. Which kind of license can i use for free open source software. I don’t want it to be used as a comercial product and everybody can modify it (MIT ?)


#6

This page might help you: https://choosealicense.com/


#7

thanks you :slight_smile:


#8

Hello everybody,

Juste un quick add on glouton. Now we can filter by stations, yeay nothing really big, but some other things are coming.

See ya


#9

Hello,

Now glouton can also download demoddata :slight_smile:

have a nice weekend everybody


#10

Hi,

You can now select which data you want to download (all by default)

see ya


#12

Hi!

I appreciate your python script development. But when I start your script on my Linux Mint computer, I see the following error message appear on my screen. Downloaded latest glouton version today.

I probably forget something very important

===============================================================
$ python ./glouton.py -h
Traceback (most recent call last):
File “./glouton.py”, line 4, in
from services.observation.observationsService import ObservationsService
File “/home/frank/Downloads/glouton-satnogs-data-downloader-master/services/observation/observationsService.py”, line 1, in
from repositories.demoddata.demoddataRepo import DemoddataRepo
File “/home/frank/Downloads/glouton-satnogs-data-downloader-master/repositories/demoddata/demoddataRepo.py”, line 3, in
from commands.download.downloadCommandParams import DownloadCommandParams
ImportError: No module named download.downloadCommandParams

Frank


#13

Hi,

Solved my own question from yesterday :slight_smile:

Did a fast docker Youtube course, great stuff.
(1) Learn docker in 12 minutes: https://www.youtube.com/watch?v=YFl2mCHdv24
(2) From zero to docker : Link: https://www.youtube.com/watch?v=JprTjTViaEA

Answer to my own question:

Create docker environment…

$ docker build -t gloutimagename .
$ docker run -it gloutimagename

Your are now in the docker container environment and can submit the command as in de README.md file:

python ./glouton.py -s 2018-01-20T00:51:54 -e 2018-01-21T00:51:54 -n 28654

Tip: if possible I would advise developer those 2 additional commands in the README.md file

Great stuff, thanks!
Frank,


#14

it works fine on ubuntu


#15

Hey,

Sorry, i didn t see the notification (the mail landed in spam) :slight_smile: Yes, it’s more simple with the docker image as you have all you need :slight_smile:

i ll put the docker command lines into the readme file asap.

anyway thanks for at least tested my stuff :stuck_out_tongue:


#16

Hello,

Thanks for your test on ubuntu :stuck_out_tongue:


#17

And btw, if you want, you don’t need to build your image yourself. Glouton is on the docker hub : https://hub.docker.com/r/deckbsd/glouton-satnogs-data-downloader/ And there is an automatic build linked to it so, if i push new code on github, the image is automatically rebuilded.

I ll put all that details into the readme file :stuck_out_tongue:


#18

Hi there,

Here is a new feature :slight_smile: You are now able to create your own modules to do whatever post processing you want after the download of a ressource.

The module (py file) must be placed into the modules folder. The module class must herite from ModuleBase and implement the runAfterDownload method. And last thing the file and the module class must have the same name.

A very simple module is included into the sources.

here is the repo

Enjoy :slight_smile:


#19

Hi there,

Just this message to announces some improvements about Glouton. I added the followings :

  • Logs : now the entire process (informations, warnings, error) is logged into a file that you can configure.
  • Session : I added the ‘–last’ option, if there were some issues during a process, or you just want to re-execute the last command, you just now have to type --last.
  • Now if some files from satnogs exists already, there aren’t downloaded again.
  • I did a little refactoring of the error handling.
  • All observation metadata are now accessible into a module (Before it was only the file name and the full path).

That’s all for this time. There is a lot more improvements that i 'll do. But it seems that not a lot of people are interested about Glouton, so i 'll take my time :slight_smile:

The repo

Best regards,


#20

Hello everybody,

Some improvements have been made on Glouton :

  • Now the executions of the modules after getting the file are completely asynchronous. So all the process (data download + operations on data) is faster.
  • Now you can specify which modules to execute on what type of satnogs data directly from the config.json file.

Very soon, i’ll rewrite the documentation also, because it’s a little bit outdated.

That’s it for this time folks. So as usual, every ideas and remarks are welcome.

The repo
The docker image

Best regards,


#21

Hi folks,

I did some updates on glouton :slight_smile:

  • I added the new filters of the satnogs observation api as parameters of the cli program (user id, transmitter type, transmitter mode, transmitter uuid)
  • I fixed the docker automated build, apparently docker made some changes and i had to re link my github account to docker.

the repo
the docker image

Best regards :slight_smile: ,