Failed observations due to USB traffic

SatNOGS Software
1

Hello,

i’m owner of station #2136 ( IU2JWG - San Siro Space Ground Station OMNI) . I looked for same problem in this community but without success. Any time interval from 10 to 20 hours the station stops working and all observations get ‘failed’ status without any data in waterfalls. The journal after one of this event is reported below:

satnogs-journal
satnogs-journal1920×1020 113 KB

I looked for error explanation on the net and i understand the USB system locks up in same way, and apparently easiest way to solve the issue is to reboot the station. Doing so the station restarts, but for me is complicated doing so with daily frequency since it is not phisically close to me.

My question is: is there a way (hardware/software/both) to solve definitively this issue ?
Thanks

Maurizio Morini

2

Login remotely via command ssh username@hostname/ip and execute sudo shutdown -r now

3

I have similar issue with rtl-sdr. After some time no samples are coming from the stick. It can be detected just fine and used by rtl_test or rtl_sdr. Just no samples are coming. Must be something to do with the driver.

Do you know if somebody else experience this?

4

Let me guess, RTL-SDR v4 and or the updated librtlsdr ?

I have the same experience and I also suspect that the lo and or usb interface is generating extra noise that is picked up by other receivers.

For now I still haven’t found the real root cause and need to make some time to investigate.

5

No it is v3 version. I’m turning it on and off for many observations during the day. So suspect some race condition somewhere in the libusb code.

6

Are you using the updated librtlsdr that supports the V4 ?

7

No, old one. Parched version that supports bias-t. Maybe 4 or 5 years old.

8

Thanks for this information,

So the challenges becomes even bigger, now we also have the option that more generic Linux kernel drivers are part of the issue.

For now I have tried the standard SatNOGS/Stable drivers and also the ones from https://github.com/librtlsdr/librtlsdr.git

Both introduce the issue, but I still use the https://github.com/librtlsdr/librtlsdr.git drivers because they provide extra options.

In the past a sudo systemctl restart satnogs-client would solve it but now I have the reboot the system to get the SDR streaming again.

And it only happens with RTL-SDR v3 and v4 models, no issues with Airspy Mini nor R2 models connected.

Some driver and module information:

librtlsdr0:amd64        0.6.0-3satnogs1  amd64 Software defined radio receiver for Realtek RTL2832U (library)
libusb-1.0-0-dev:amd64  2:1.0.24-3       amd64 userspace USB programming library development files

librtlsdr.so.0.8git (https://github.com/librtlsdr/librtlsdr.git)

Module found: modules0.7/librtlsdrSupport.so  (0.3.3-068aa77) (https://github.com/pothosware/SoapyRTLSDR.git)
1 Like
9

I have yet to find a more stable rtl-sdr lib than librespace:satnogs/rtl-sdr. There is pretty old work on making it handle the occasional errors, I don’t think those are part of either of the other repos unfortunately.

1 Like
10

I get this error once in awhile using Ubuntu 20.04. What works for me is issuing a usbreset command for the RTL-SDR.

2 Likes
11

I was able to reproduce the error and captured USB traffic during the issue. Then restarted the system and captured “good” traffic. Then compared both. See no difference between them. During the issue USB device returns same data and responds to the write register commands.

However all URB transfers timeout. librtlsdr doesn’t handle LIBUSB_TRANSFER_TIMED_OUT, thus no error when running rtl_sdr program.

I suspect it might be:

  • Some electrical issue that require physical power down/up. Might be due to old hardware or some static electricity.
  • Some internal i2c registers flipped. I’ll try to write a program to dump all register states.
12

Some extra information on remote resetting USB devices

apt-cache show usbutils

Package: usbutils
Version: 1:014-1+deb12u1
Installed-Size: 320
Maintainer: Aurelien Jarno <aurel32@debian.org>
Architecture: amd64
Depends: libc6 (>= 2.34), libudev1 (>= 196), libusb-1.0-0 (>= 2:1.0.22)
Breaks: hwdata (<< 0.334-1~), isenkram (<< 0.45~), kinfocenter (<< 4:5.14.5-2~), libosinfo-1.0-0 (<< 1.8.0-1~), usbip (<< 2.0+5.10.4-1~)
Description-en: Linux USB utilities
 This package contains the lsusb utility for inspecting the devices
 connected to the USB bus. It shows a graphical representation of the
 devices that are currently plugged in, showing the topology of the
 USB bus. It also displays information on each individual device on
 the bus.
Description-md5: 928f8c65aee445506dc03a844a66adac
Multi-Arch: foreign
Homepage: https://github.com/gregkh/usbutils
Tag: hardware::usb, implemented-in::c, interface::commandline, role::program,
 scope::utility, use::monitor, use::viewing
Section: utils
Priority: optional
Filename: pool/main/u/usbutils/usbutils_014-1+deb12u1_amd64.deb
Size: 73488
MD5sum: 59da4e3bb2be6bf5a89e9360571ef755
SHA256: 1b60c6922fe5f121ac22b7ae164552746896604614d5822ff5b3c76260feebf9

sudo apt-get install usbutils

sudo usbreset

Usage:
  usbreset PPPP:VVVV - reset by product and vendor id
  usbreset BBB/DDD   - reset by bus and device number
  usbreset "Product" - reset by product name

Devices:
  Number 001/006  ID 1d50:60a1  AIRSPY
  Number 002/002  ID 8087:8008  
  Number 001/008  ID 8087:07dc  
  Number 001/002  ID 0bda:2832  RTL2832U
  Number 004/002  ID 8087:8000  
  Number 001/005  ID 0bda:0129  USB2.0-CRW
  Number 001/003  ID 0bda:2838  Blog V4

sudo usbreset 001/003

Resetting Blog V4 ... ok
2 Likes