Cannot start Satnogs

I cannot start Satnogs client with sudo systemctl restart satnogs-client. I get:

AttributeError: module 'distutils' has no attribute 'version'

Any suggestion or direction is welcome.

Welcome PE1MEW,

Can please share more information on your setup and what procedure you used to install the satnogs-client.

Installation was using instructions from the website.

  1. Raspberry Pi - SatNOGS Wiki
  2. SatNOGS Client Setup - SatNOGS Wiki
  3. Updating SatNOGS Client Software: sudo satnogs-setup etc.
    I configured it according to the instructions.

I use an RTLSDR dongle and the system has been working since some months ago.
When I moved the system to the rooftop (reboot) I inspected the operation and it failed.
I can imagine that a dependency was updated.

distutils: python3-distutils is already the newest version (3.9.2-1).

For your information the last attempt of starting the service in the log:

Jun 06 08:12:59 satnogs systemd[1]: satnogs-client.service: Scheduled restart job, restart counter is at 4.
Jun 06 08:12:59 satnogs systemd[1]: Stopped SatNOGS client.
Jun 06 08:12:59 satnogs systemd[1]: Started SatNOGS client.
Jun 06 08:13:00 satnogs satnogs-client[12703]: Traceback (most recent call last):
Jun 06 08:13:00 satnogs satnogs-client[12703]:   File "/var/lib/satnogs/bin/satnogs-client", line 5, in <module>
Jun 06 08:13:00 satnogs satnogs-client[12703]:     from satnogsclient import main
Jun 06 08:13:00 satnogs satnogs-client[12703]:   File "/var/lib/satnogs/lib/python3.9/site-packages/satnogsclient/__init__.py", line 9, in <module>
Jun 06 08:13:00 satnogs satnogs-client[12703]:     import sentry_sdk
Jun 06 08:13:00 satnogs satnogs-client[12703]:   File "/var/lib/satnogs/lib/python3.9/site-packages/sentry_sdk/__init__.py", line 1, in <module>
Jun 06 08:13:00 satnogs satnogs-client[12703]:     from sentry_sdk.hub import Hub, init
Jun 06 08:13:00 satnogs satnogs-client[12703]:   File "/var/lib/satnogs/lib/python3.9/site-packages/sentry_sdk/hub.py", line 8, in <module>
Jun 06 08:13:00 satnogs satnogs-client[12703]:     from sentry_sdk.scope import Scope
Jun 06 08:13:00 satnogs satnogs-client[12703]:   File "/var/lib/satnogs/lib/python3.9/site-packages/sentry_sdk/scope.py", line 7, in <module>
Jun 06 08:13:00 satnogs satnogs-client[12703]:     from sentry_sdk.utils import logger, capture_internal_exceptions
Jun 06 08:13:00 satnogs satnogs-client[12703]:   File "/var/lib/satnogs/lib/python3.9/site-packages/sentry_sdk/utils.py", line 887, in <module>
Jun 06 08:13:00 satnogs satnogs-client[12703]:     HAS_REAL_CONTEXTVARS, ContextVar = _get_contextvars()
Jun 06 08:13:00 satnogs satnogs-client[12703]:   File "/var/lib/satnogs/lib/python3.9/site-packages/sentry_sdk/utils.py", line 857, in _get_contextvars
Jun 06 08:13:00 satnogs satnogs-client[12703]:     if not _is_contextvars_broken():
Jun 06 08:13:00 satnogs satnogs-client[12703]:   File "/var/lib/satnogs/lib/python3.9/site-packages/sentry_sdk/utils.py", line 818, in _is_contextvars_broken
Jun 06 08:13:00 satnogs satnogs-client[12703]:     from eventlet.patcher import is_monkey_patched  # type: ignore
Jun 06 08:13:00 satnogs satnogs-client[12703]:   File "/usr/local/lib/python3.9/dist-packages/eventlet/__init__.py", line 17, in <module>
Jun 06 08:13:00 satnogs satnogs-client[12703]:     from eventlet import convenience
Jun 06 08:13:00 satnogs satnogs-client[12703]:   File "/usr/local/lib/python3.9/dist-packages/eventlet/convenience.py", line 4, in <module>
Jun 06 08:13:00 satnogs satnogs-client[12703]:     from eventlet import greenpool
Jun 06 08:13:00 satnogs satnogs-client[12703]:   File "/usr/local/lib/python3.9/dist-packages/eventlet/greenpool.py", line 4, in <module>
Jun 06 08:13:00 satnogs satnogs-client[12703]:     from eventlet import queue
Jun 06 08:13:00 satnogs satnogs-client[12703]:   File "/usr/local/lib/python3.9/dist-packages/eventlet/queue.py", line 49, in <module>
Jun 06 08:13:00 satnogs satnogs-client[12703]:     from eventlet.event import Event
Jun 06 08:13:00 satnogs satnogs-client[12703]:   File "/usr/local/lib/python3.9/dist-packages/eventlet/event.py", line 3, in <module>
Jun 06 08:13:00 satnogs satnogs-client[12703]:     from eventlet import hubs
Jun 06 08:13:00 satnogs satnogs-client[12703]:   File "/usr/local/lib/python3.9/dist-packages/eventlet/hubs/__init__.py", line 7, in <module>
Jun 06 08:13:00 satnogs satnogs-client[12703]:     from eventlet.support import greenlets as greenlet
Jun 06 08:13:00 satnogs satnogs-client[12703]:   File "/usr/local/lib/python3.9/dist-packages/eventlet/support/__init__.py", line 6, in <module>
Jun 06 08:13:00 satnogs satnogs-client[12703]:     from eventlet.support import greenlets
Jun 06 08:13:00 satnogs satnogs-client[12703]:   File "/usr/local/lib/python3.9/dist-packages/eventlet/support/greenlets.py", line 6, in <module>
Jun 06 08:13:00 satnogs satnogs-client[12703]:     preserves_excinfo = (distutils.version.LooseVersion(greenlet.__version__)
Jun 06 08:13:00 satnogs satnogs-client[12703]: AttributeError: module 'distutils' has no attribute 'version'
Jun 06 08:13:00 satnogs systemd[1]: satnogs-client.service: Main process exited, code=exited, status=1/FAILURE
Jun 06 08:13:00 satnogs systemd[1]: satnogs-client.service: Failed with result 'exit-code'.
Jun 06 08:13:00 satnogs systemd[1]: satnogs-client.service: Scheduled restart job, restart counter is at 5.
Jun 06 08:13:00 satnogs systemd[1]: Stopped SatNOGS client.
Jun 06 08:13:00 satnogs systemd[1]: satnogs-client.service: Start request repeated too quickly.
Jun 06 08:13:00 satnogs systemd[1]: satnogs-client.service: Failed with result 'exit-code'.
Jun 06 08:13:00 satnogs systemd[1]: Failed to start SatNOGS client.

Thanks for the information,

This behavior is new to me on a clean install, so we need some extra information.
You mention update, was this system previous running based on debian buster?

Can you share some additional information:

  • I saw your tweet is the Pi combining tinyGS and the satnogs-client. Maybe there are conflicting software versions?
  • Was the system previously running debian buster? Here a detailed post on the upgrade to bullseye
  • sudo satnogs-setup option Advanced option Support and share the output.

Hi,

  • I saw your tweet is the Pi combining tinyGS and the satnogs-client. Maybe there are conflicting software versions?

This is impossible because TinyGS is running on a dedicated LoRa module (heltec) and is connected over the WiFi AP that is configured on the RPI. Also, the setup worked before I moved it a couple of moths ago.

  • Was the system previously running debian buster? Here a detailed post on the upgrade to bullseye

It was running “Linux satnogs 5.15.84-v7l+ #1613 SMP Thu Jan 5 12:01:26 GMT 2023 armv7l” from the beginning as I download the image.

  • sudo satnogs-setup option Advanced option Support and share the output.

As requested:

Generating support report. Please wait...
------------[ copy here ]------------
{
    "versions": {
        "satnogs-client": "1.8.1",
        "satnogs-client-ansible": "202209101521",
        "satnogs-flowgraphs": "1.4-1",
        "gr-satnogs": "2.3.4.0-1",
        "gr-soapy": "2.1.3.1-1",
        "gnuradio": "3.8.2.0-14",
        "satnogs-config": "0.13.2"
    },
    "state": {
        "is-applied": true,
        "pending-tags": null
    },
    "system": {
        "date": "2023-06-06T09:35:22.553403+00:00",
        "distribution": {
            "DESCRIPTION": "Raspbian GNU/Linux 11 (bullseye)",
            "RELEASE": "11",
            "CODENAME": "bullseye",
            "ID": "Raspbian"
        },
        "pending-updates": false,
        "platform": {
            "system": "Linux",
            "node": "satnogs",
            "release": "5.15.84-v7l+",
            "version": "#1613 SMP Thu Jan 5 12:01:26 GMT 2023",
            "machine": "armv7l"
        },
        "memory": {
            "total": 4025397248,
            "available": 3706580992,
            "percent": 7.9,
            "used": 157184000,
            "free": 2559819776,
            "active": 543281152,
            "inactive": 768983040,
            "buffers": 120283136,
            "cached": 1188110336,
            "shared": 954368,
            "slab": 97710080
        },
        "disk": {
            "total": 15336914944,
            "used": 5792321536,
            "free": 8866578432,
            "percent": 39.5
        }
    },
    "configuration": {
        "satnogs_antenna": "RX",
        "satnogs_api_token": "[redacted]",
        "satnogs_rx_samp_rate": "2.048e6",
        "satnogs_soapy_rx_device": "driver=rtlsdr",
        "satnogs_station_elev": "25",
        "satnogs_station_id": "2849",
        "satnogs_station_lat": "51.98773",
        "satnogs_station_lon": "5.95154"
    }
}
------------[ copy end ]-------------

Weird,

I just compared it to a running Pi based satnogs-client and it is the same. The only thing missing but that can’t be related to the error are the following options:

"satnogs_ppm_error": "-2" (optional)
"satnogs_rf_gain": "42.1" (must be set when operational, RTL gain setting)

Can you share the output from the following command dpkg -l | grep setuptools

I will in the meantime also reach out to satnogs-developers

I’d say this is a clue:
/usr/local/lib/python3.9/dist-packages/eventlet/

 $ dpkg -l | grep setuptools
ii  python3-setuptools                     52.0.0-4                         all          Python3 Distutils Enhancements

A, mixing repository python libraries and local builds.

That was my initial idea. Now, how can we solve this? what is required?
It would be better to update the software to meet the latest version?

What other software did you install?

There seems to be a library conflict

The problem seems to be created by /usr/local/lib/python3.9/dist-packages/eventlet is it possible to remove this lib and all the dependencies it installed.

Hi,

The other software running on the raspberry pi is this: GitHub - Stefal/rtkbase: Your own GNSS base station for RTK localization with a Web GUI

I’ll have to see if I can remove the suggested dependency. let me look at it.

I am still puzzled because the system was completely working. I would have considered that an update of a library requires a change in calling it.

Hi,
I think the two possible routes are, make satnogs-client avoid /usr/local, the other is to install the other software inside a venv, with all the dependencies installed just for that environment.
the client already works in a venv, but uses site-packages so will pull in the dist/local stuff.
regarding it working earlier, the -client will just happily run until restarted, then look at new libs etc, so if it was running for a long time until moving, I’d say that is a probable cause.

Looking at the RTKBase install script, my best guess is that the web_server install is adding python
libraries and is influencing this behavior.

cat requirements.txt

cryptography==38.0.0
itsdangerous==2.1.2
Flask==2.2.2
Flask-SocketIO==5.3.2
eventlet==0.33.3
greenlet==1.1.3
Bootstrap-Flask==2.2.0
Flask-WTF==1.1.1
Flask-Login==0.6.2
#On some linux distribution, pexpect and psutil are already installed
#and can't be upgraded with pip. In these cases, pip fail, leaving
#some package uninstalled.
#With pexpect <=4.8.0, pip will not fail.
#With psutil <= 5.8.0, pip will not fail.
pexpect<=4.8.0
psutil<=5.9.4
pyOpenSSL==23.0.0
pyserial==3.5
pystemd==0.10.0
requests==2.28.1
dnspython==2.3.0
distro==1.8.0

This has my preference. In both cases, I yet lack the knowledge at short notice to change this.
Can I receive some aid In making such changes?

I agree.

I have been looking around for how to exclude /usr/local from python’s include path, but no easy or quick fixes that I can see.
it should be applied to the virtualenv that is in /var/lib/satnogs
one way would be to disable system-site-packages and link the gps and hamlib packages in manually somehow.
I think the route to this would be to add .pth files in /var/lib/satnogs/lib/python3.9/site-packages.
or perhaps pip install -e for these two libraries.
this needs a bit of experimentation.

I have noted down this as the way to create the venv in the current dir:

virtualenv -p python3 --system-site-packages .
. bin/activate
pip3 install satnogs-client~=1.8 --prefix=. --prefer-binary --extra-index-url https://www.piwheels.org/simple

changing that to --no-site-packages and bringing the gpsd & hamlib in before running pip3 install.

or just go for the docker route and be done with it (: