Station 3702: rotctl + RTC-200 G5500 controller troubleshooting

yasiu · August 16, 2024, 9:06pm

Dear all!

I’m working on station 3702, I hope to bring it online very soon.

The Setup

My setup is:

Raspberry Pi running latest SatNOGS image
Pluto SDR + Arrow II 14 element 2m/70cm Yagi
Yaesu G5500 rotator
RTC-200 (GS232B compatible) rotator controller

Currently I’m focused on getting the rotator to fully work. So far I’ve managed to configure everything via satnogs-setup, here is my config:

satnogs-client config

“hamlib_utils_rig_opts”: “-T 0.0.0.0 -m 1”,
“hamlib_utils_rot_enabled”: true,
“hamlib_utils_rot_opts”: “-m 603 -r /dev/ttyUSB0 -s 9600 -T 0.0.0.0 -t 4533 --set-conf=max_el=180 -vvvv”,
“satnogs_antenna”: “A_BALANCED”,
“satnogs_log_level”: “DEBUG”,
“satnogs_rf_gain”: “50”,
“satnogs_rig_ip”: “0.0.0.0”,
“satnogs_rot_baud”: “9600”,
“satnogs_rot_enabled”: true,
“satnogs_rot_model”: “ROT_MODEL_NETROTCTL”,
“satnogs_rot_port”: “127.0.0.1:4533”,
“satnogs_rot_threshold”: “4”,
“satnogs_rx_samp_rate”: “2e6”,
“satnogs_scheduler_log_level”: “DEBUG”,
“satnogs_soapy_rx_device”: “driver=plutosdr”,
“satnogs_station_elev”: “120”,
“satnogs_station_id”: “3702”,

I’ve configured rotctld service to run in background and satnogs-client to connect to it via localhost. This allows me to specify more settings for rotctl, like the supported azimuth range.

The problem

When I connect to rotctl (by using telnet localhost 4533), I can send the commands and the rotator responds to the commands. Here is an example log from rotctld.service:

rotctld.service + telnet connection

Aug 16 22:44:00 vega rotctld[20292]: Connection opened from 192.168.1.95:39466 // telnet connects
Aug 16 22:44:03 vega rotctld[20292]: rot_get_position called
Aug 16 22:44:03 vega rotctld[20292]: serial_flush called
Aug 16 22:44:03 vega rotctld[20292]: tcflush
Aug 16 22:44:03 vega rotctld[20292]: write_block called
Aug 16 22:44:03 vega rotctld[20292]: rot_get_position: got az=184.00, el=90.00 // p command
Aug 16 22:44:14 vega rotctld[20292]: rot_set_position called az=180.00 el=180.00 // P 180 180
Aug 16 22:44:14 vega rotctld[20292]: rot_set_position: south_zero=0
Aug 16 22:44:14 vega rotctld[20292]: serial_flush called
Aug 16 22:44:14 vega rotctld[20292]: tcflush
Aug 16 22:44:14 vega rotctld[20292]: write_block called
Aug 16 22:44:14 vega rotctld[20292]: write_block called
Aug 16 22:44:18 vega rotctld[20292]: rot_get_position called
Aug 16 22:44:18 vega rotctld[20292]: serial_flush called
Aug 16 22:44:18 vega rotctld[20292]: tcflush
Aug 16 22:44:18 vega rotctld[20292]: write_block called
Aug 16 22:44:18 vega rotctld[20292]: rot_get_position: got az=180.00, el=99.00 // Rotator responds to the command and starts moving
Aug 16 22:44:19 vega rotctld[20292]: rot_get_position called
Aug 16 22:44:19 vega rotctld[20292]: serial_flush called
Aug 16 22:44:19 vega rotctld[20292]: tcflush
Aug 16 22:44:19 vega rotctld[20292]: write_block called
Aug 16 22:44:19 vega rotctld[20292]: rot_get_position: got az=183.00, el=103.00 // I spam the p command couple of times to show progress…
Aug 16 22:44:20 vega rotctld[20292]: rot_get_position called
Aug 16 22:44:20 vega rotctld[20292]: serial_flush called
Aug 16 22:44:20 vega rotctld[20292]: tcflush
Aug 16 22:44:20 vega rotctld[20292]: write_block called
Aug 16 22:44:20 vega rotctld[20292]: rot_get_position: got az=180.00, el=105.00
Aug 16 22:44:21 vega rotctld[20292]: rot_get_position called
Aug 16 22:44:21 vega rotctld[20292]: serial_flush called
Aug 16 22:44:21 vega rotctld[20292]: tcflush
Aug 16 22:44:21 vega rotctld[20292]: write_block called
Aug 16 22:44:21 vega rotctld[20292]: rot_get_position: got az=183.00, el=109.00
Aug 16 22:44:22 vega rotctld[20292]: rot_get_position called
Aug 16 22:44:22 vega rotctld[20292]: serial_flush called
Aug 16 22:44:22 vega rotctld[20292]: tcflush
Aug 16 22:44:22 vega rotctld[20292]: write_block called
Aug 16 22:44:22 vega rotctld[20292]: rot_get_position: got az=183.00, el=112.00
Aug 16 22:44:23 vega rotctld[20292]: rot_get_position called
Aug 16 22:44:23 vega rotctld[20292]: serial_flush called
Aug 16 22:44:23 vega rotctld[20292]: tcflush
Aug 16 22:44:23 vega rotctld[20292]: write_block called
Aug 16 22:44:23 vega rotctld[20292]: rot_get_position: got az=183.00, el=115.00
Aug 16 22:44:25 vega rotctld[20292]: rot_get_position called
Aug 16 22:44:25 vega rotctld[20292]: serial_flush called
Aug 16 22:44:25 vega rotctld[20292]: tcflush
Aug 16 22:44:25 vega rotctld[20292]: write_block called
Aug 16 22:44:25 vega rotctld[20292]: rot_get_position: got az=183.00, el=119.00
Aug 16 22:44:26 vega rotctld[20292]: rot_get_position called
Aug 16 22:44:26 vega rotctld[20292]: serial_flush called
Aug 16 22:44:26 vega rotctld[20292]: tcflush
Aug 16 22:44:26 vega rotctld[20292]: write_block called
Aug 16 22:44:26 vega rotctld[20292]: rot_get_position: got az=183.00, el=122.00
Aug 16 22:44:27 vega rotctld[20292]: rot_get_position called
Aug 16 22:44:27 vega rotctld[20292]: serial_flush called
Aug 16 22:44:27 vega rotctld[20292]: tcflush
Aug 16 22:44:27 vega rotctld[20292]: write_block called
Aug 16 22:44:27 vega rotctld[20292]: rot_get_position: got az=183.00, el=126.00
Aug 16 22:44:42 vega rotctld[20292]: rot_get_position called
Aug 16 22:44:42 vega rotctld[20292]: serial_flush called
Aug 16 22:44:42 vega rotctld[20292]: tcflush
Aug 16 22:44:42 vega rotctld[20292]: write_block called
Aug 16 22:44:42 vega rotctld[20292]: rot_get_position: got az=180.00, el=168.00
Aug 16 22:44:43 vega rotctld[20292]: rot_get_position called
Aug 16 22:44:43 vega rotctld[20292]: serial_flush called
Aug 16 22:44:43 vega rotctld[20292]: tcflush
Aug 16 22:44:43 vega rotctld[20292]: write_block called
Aug 16 22:44:43 vega rotctld[20292]: rot_get_position: got az=183.00, el=171.00
Aug 16 22:44:44 vega rotctld[20292]: rot_get_position called
Aug 16 22:44:44 vega rotctld[20292]: serial_flush called
Aug 16 22:44:44 vega rotctld[20292]: tcflush
Aug 16 22:44:44 vega rotctld[20292]: write_block called
Aug 16 22:44:44 vega rotctld[20292]: rot_get_position: got az=180.00, el=173.00
Aug 16 22:44:45 vega rotctld[20292]: rot_get_position called
Aug 16 22:44:45 vega rotctld[20292]: serial_flush called
Aug 16 22:44:45 vega rotctld[20292]: tcflush
Aug 16 22:44:45 vega rotctld[20292]: write_block called
Aug 16 22:44:45 vega rotctld[20292]: rot_get_position: got az=183.00, el=178.00
Aug 16 22:44:46 vega rotctld[20292]: rot_get_position called
Aug 16 22:44:46 vega rotctld[20292]: serial_flush called
Aug 16 22:44:46 vega rotctld[20292]: tcflush
Aug 16 22:44:46 vega rotctld[20292]: write_block called
Aug 16 22:44:46 vega rotctld[20292]: rot_get_position: got az=181.00, el=179.00
Aug 16 22:44:47 vega rotctld[20292]: rot_get_position called
Aug 16 22:44:47 vega rotctld[20292]: serial_flush called
Aug 16 22:44:47 vega rotctld[20292]: tcflush
Aug 16 22:44:47 vega rotctld[20292]: write_block called
Aug 16 22:44:47 vega rotctld[20292]: rot_get_position: got az=181.00, el=179.00 // Movement finished
Aug 16 22:46:38 vega rotctld[20292]: Connection closed from 192.168.1.95:39466 // q - telnet exits

Meanwhile when satnogs-client connects during a scheduled pass, I can see that the commands are sent, but the rotor does not respond. Here is an example log from rotctld.service:

rotctld.service + satnogs-client connection

Aug 16 23:00:48 vega rotctld[20292]: Connection opened from localhost:36618 // satnogs-client connects
Aug 16 23:00:48 vega rotctld[20292]: rot_get_position called
Aug 16 23:00:48 vega rotctld[20292]: serial_flush called
Aug 16 23:00:48 vega rotctld[20292]: tcflush
Aug 16 23:00:48 vega rotctld[20292]: write_block called
Aug 16 23:00:48 vega rotctld[20292]: rot_get_position: got az=183.00, el=180.00 // get position
Aug 16 23:00:48 vega rotctld[20292]: rot_set_position called az=178.42 el=0.44 // set position
Aug 16 23:00:48 vega rotctld[20292]: rot_set_position: south_zero=0
Aug 16 23:00:48 vega rotctld[20292]: serial_flush called
Aug 16 23:00:48 vega rotctld[20292]: tcflush
Aug 16 23:00:48 vega rotctld[20292]: write_block called
Aug 16 23:00:48 vega rotctld[20292]: write_block called
Aug 16 23:00:51 vega rotctld[20292]: rot_get_position called
Aug 16 23:00:51 vega rotctld[20292]: serial_flush called
Aug 16 23:00:51 vega rotctld[20292]: tcflush
Aug 16 23:00:51 vega rotctld[20292]: write_block called
Aug 16 23:00:51 vega rotctld[20292]: rot_get_position: got az=183.00, el=179.00 // no movement
Aug 16 23:00:51 vega rotctld[20292]: rot_set_position called az=178.53 el=0.61 // set position
Aug 16 23:00:51 vega rotctld[20292]: rot_set_position: south_zero=0
Aug 16 23:00:51 vega rotctld[20292]: serial_flush called
Aug 16 23:00:51 vega rotctld[20292]: tcflush
Aug 16 23:00:51 vega rotctld[20292]: write_block called
Aug 16 23:00:51 vega rotctld[20292]: write_block called
Aug 16 23:00:54 vega rotctld[20292]: rot_get_position called
Aug 16 23:00:54 vega rotctld[20292]: serial_flush called
Aug 16 23:00:54 vega rotctld[20292]: tcflush
Aug 16 23:00:54 vega rotctld[20292]: write_block called
Aug 16 23:00:54 vega rotctld[20292]: rot_get_position: got az=182.00, el=179.00 // no movement
Aug 16 23:00:54 vega rotctld[20292]: rot_set_position called az=178.64 el=0.79
Aug 16 23:00:54 vega rotctld[20292]: rot_set_position: south_zero=0
Aug 16 23:00:54 vega rotctld[20292]: serial_flush called
Aug 16 23:00:54 vega rotctld[20292]: tcflush
Aug 16 23:00:54 vega rotctld[20292]: write_block called
Aug 16 23:00:54 vega rotctld[20292]: write_block called
Aug 16 23:00:57 vega rotctld[20292]: rot_get_position called
Aug 16 23:00:57 vega rotctld[20292]: serial_flush called
Aug 16 23:00:57 vega rotctld[20292]: tcflush
Aug 16 23:00:57 vega rotctld[20292]: write_block called
Aug 16 23:00:57 vega rotctld[20292]: rot_get_position: got az=183.00, el=180.00 // no movement
Aug 16 23:00:57 vega rotctld[20292]: rot_set_position called az=178.75 el=0.98
Aug 16 23:00:57 vega rotctld[20292]: rot_set_position: south_zero=0
Aug 16 23:00:57 vega rotctld[20292]: serial_flush called
Aug 16 23:00:57 vega rotctld[20292]: tcflush
Aug 16 23:00:57 vega rotctld[20292]: write_block called
Aug 16 23:00:57 vega rotctld[20292]: write_block called
Aug 16 23:01:00 vega rotctld[20292]: rot_get_position called
Aug 16 23:01:00 vega rotctld[20292]: serial_flush called
Aug 16 23:01:00 vega rotctld[20292]: tcflush
Aug 16 23:01:00 vega rotctld[20292]: write_block called

etc.

From what I can tell, when sending the commands by hand (I send one, and wait for the audible response of the rotator) it works fine. On the other hand satnogs-client seems to set the position every second or so. This makes me think that it might somehow “break” my controller.

I’m trying to pin down at which level of the SatNOGS stack the issue is. Some questions:

Is there a difference between how telnet and satnogs-client connects to the rotctl daemon, which could brake it?
Is my RTC-200 somehow overwhelmed, and the quicker pace of commands sent by satnogs-client breaks it? Is there a way to further debug the connection between rotctl and the device?
Are there any config changes I could make, or additional logs I could provide to help debug it?

I feel like this issue goes beyond the scope of SatNOGS itself, but resources about the RTC-200 device and Hamlib/rotctl seem sparse. Perhaps some experts here could point me in the right direction and help troubleshoot this issue.

SA2KNG · August 17, 2024, 7:45am

I am also running rotator on -client, it sure has it’s peculiarities

You are talking about rotctld.service, this is running on systemd with the config in /etc/default/hamlib iirc.
Also running it from satnogs-client with hamlib_utils_rot_enabled etc, this would result in two rotctl(d) connecting to the serial port.

The hamlib_utils_rig_opts should not be needed, this is for flowgraph doppler correction and works fine with the defaults.

The only two that is needed for the -client to connect to a running rotctld is:

SATNOGS_ROT_MODEL=ROT_MODEL_NETROTCTL
SATNOGS_ROT_PORT=127.0.0.1:4533

I would recommend clearing out as many of the other RIG/ROT/HAMLIB stuff as possible to start with.
Using the systemd service is the better way imho, as you can use it without the -client running etc.

As -client is running on DEBUG log level, you should also get a lot of messages from that. I think INFO could be enough, there’s something with the rotator.py that does not use the logger properly, resulting in spewage of Hamlib.Rot() unfiltered to the log.

Regarding the update rate, it should not be a problem even with less than 1s. I don’t particularly like the current method thou, it has the tendency to spam the controller unnecessarily. This is a change that I have running on my station (althou I see a minor mistake in the code now).

That branch rotator-flip has the added config of SATNOGS_ROT_INVERT that uses a 0-180 alt rotator to use the 90-180 part to flip the az rotation by 180 degrees so it doesn’t hit endstop and start spinning 360 in the middle of a pass.

yasiu · August 17, 2024, 11:34am

Just to be sure, I reinstalled everything brand-new, starting from the latest Raspberry Pi Image - 2023111400.

I did minimal changes to the config, I:

configured the PlutoSDR
enabled rotctld
configured ROT_MODEL_NETROTCTL

Here is my config after:

satnogs config

    "hamlib_utils_rot_enabled": true,
    "hamlib_utils_rot_opts": "-m 603 -r /dev/ttyUSB0 -s 9600 -T 0.0.0.0 -vvvv",
    "satnogs_antenna": "A_BALANCED",
    "satnogs_api_token": "[redacted]",
    "satnogs_rot_enabled": true,
    "satnogs_rot_model": "ROT_MODEL_NETROTCTL",
    "satnogs_rot_port": "127.0.0.1:4533",
    "satnogs_rx_samp_rate": "2e6",
    "satnogs_soapy_rx_device": "driver=plutosdr",

The result is the same: I can telnet, issue commands and the rotator moves. Meanwhile when the satnogs-client connects, the commands are issued but rotator doesn’t move.

Here is part of non-debug log from satnogs-client.service:

satnogs-client.service

Aug 17 12:51:13 vega systemd[1]: Started SatNOGS client.
Aug 17 13:06:46 vega satnogs-client[12743]: rot_init called
Aug 17 13:06:46 vega satnogs-client[12743]: initrots4_dummy: _init called
Aug 17 13:06:46 vega satnogs-client[12743]: rot_register (1)
Aug 17 13:06:46 vega satnogs-client[12743]: rot_register (2)
Aug 17 13:06:46 vega satnogs-client[12743]: rot_open called
Aug 17 13:06:46 vega satnogs-client[12743]: rot_open: using network address 127.0.0.1:4533
Aug 17 13:06:46 vega satnogs-client[12743]: network_open called
Aug 17 13:06:46 vega satnogs-client[12743]: network_open version 1.0
Aug 17 13:06:46 vega satnogs-client[12743]: network_open: hoststr=127.0.0.1, portstr=4533
Aug 17 13:06:46 vega satnogs-client[12743]: netrotctl_open called
Aug 17 13:06:46 vega satnogs-client[12743]: rig_flush: called for network device
Aug 17 13:06:46 vega satnogs-client[12743]: network_flush called
Aug 17 13:06:46 vega satnogs-client[12743]: write_block called
Aug 17 13:06:46 vega satnogs-client[12743]: write_block(): TX 12 bytes
Aug 17 13:06:46 vega satnogs-client[12743]: 0000 5c 64 75 6d 70 5f 73 74 61 74 65 0a \dump_state.
Aug 17 13:06:46 vega satnogs-client[12743]: read_string called, rxmax=64
Aug 17 13:06:46 vega satnogs-client[12743]: read_string(): RX 2 characters
Aug 17 13:06:46 vega satnogs-client[12743]: 0000 30 0a 0.
Aug 17 13:06:46 vega satnogs-client[12743]: read_string called, rxmax=64
Aug 17 13:06:46 vega satnogs-client[12743]: read_string(): RX 4 characters
Aug 17 13:06:46 vega satnogs-client[12743]: 0000 36 30 33 0a 603.
Aug 17 13:06:46 vega satnogs-client[12743]: read_string called, rxmax=64
Aug 17 13:06:46 vega satnogs-client[12743]: read_string(): RX 12 characters
Aug 17 13:06:46 vega satnogs-client[12743]: 0000 2d 31 38 30 2e 30 30 30 30 30 30 0a -180.000000.
Aug 17 13:06:46 vega satnogs-client[12743]: read_string called, rxmax=64
Aug 17 13:06:46 vega satnogs-client[12743]: read_string(): RX 11 characters
Aug 17 13:06:46 vega satnogs-client[12743]: 0000 34 35 30 2e 30 30 30 30 30 30 0a 450.000000.
Aug 17 13:06:46 vega satnogs-client[12743]: read_string called, rxmax=64
Aug 17 13:06:46 vega satnogs-client[12743]: read_string(): RX 9 characters
Aug 17 13:06:46 vega satnogs-client[12743]: 0000 30 2e 30 30 30 30 30 30 0a 0.000000.
Aug 17 13:06:46 vega satnogs-client[12743]: read_string called, rxmax=64
Aug 17 13:06:46 vega satnogs-client[12743]: read_string(): RX 11 characters
Aug 17 13:06:46 vega satnogs-client[12743]: 0000 31 38 30 2e 30 30 30 30 30 30 0a 180.000000.
Aug 17 13:06:46 vega satnogs-client[12743]: rot_get_position called
Aug 17 13:06:46 vega satnogs-client[12743]: netrotctl_get_position called
Aug 17 13:06:46 vega satnogs-client[12743]: rig_flush: called for network device
Aug 17 13:06:46 vega satnogs-client[12743]: network_flush called
Aug 17 13:06:46 vega satnogs-client[12743]: network_flush: network data clear d: ret=0, len=2, ‘’
Aug 17 13:06:46 vega satnogs-client[12743]: network_flush: network data cleared: ret=0, len_read=2/0x2, '0
Aug 17 13:06:46 vega satnogs-client[12743]: ’
Aug 17 13:06:46 vega satnogs-client[12743]: write_block called
Aug 17 13:06:46 vega satnogs-client[12743]: write_block(): TX 2 bytes
Aug 17 13:06:46 vega satnogs-client[12743]: 0000 70 0a p.
Aug 17 13:06:46 vega satnogs-client[12743]: read_string called, rxmax=64
Aug 17 13:06:46 vega satnogs-client[12743]: read_string(): RX 7 characters
Aug 17 13:06:46 vega satnogs-client[12743]: 0000 31 38 30 2e 30 30 0a 180.00.
Aug 17 13:06:46 vega satnogs-client[12743]: read_string called, rxmax=64
Aug 17 13:06:46 vega satnogs-client[12743]: read_string(): RX 6 characters
Aug 17 13:06:46 vega satnogs-client[12743]: 0000 39 30 2e 30 30 0a 90.00.
Aug 17 13:06:46 vega satnogs-client[12743]: rot_get_position: got az=180.00, el=90.00
Aug 17 13:06:46 vega satnogs-client[12743]: rot_set_position called az=356.80 el=0.45
Aug 17 13:06:46 vega satnogs-client[12743]: rot_set_position: south_zero=0
Aug 17 13:06:46 vega satnogs-client[12743]: netrotctl_set_position called: 356.801849 0.447346
Aug 17 13:06:46 vega satnogs-client[12743]: rig_flush: called for network device
Aug 17 13:06:46 vega satnogs-client[12743]: network_flush called
Aug 17 13:06:46 vega satnogs-client[12743]: write_block called
Aug 17 13:06:46 vega satnogs-client[12743]: write_block(): TX 22 bytes
Aug 17 13:06:46 vega satnogs-client[12743]: 0000 50 20 33 35 36 2e 38 30 31 38 34 39 20 30 2e 34 P 356.801849 0.4
Aug 17 13:06:46 vega satnogs-client[12743]: 0010 34 37 33 34 36 0a 47346.
Aug 17 13:06:46 vega satnogs-client[12743]: read_string called, rxmax=64
Aug 17 13:06:46 vega satnogs-client[12743]: read_string(): RX 7 characters
Aug 17 13:06:46 vega satnogs-client[12743]: 0000 52 50 52 54 20 30 0a RPRT 0.
Aug 17 13:06:46 vega satnogs-client[12743]: rig_init: rig does not have rx_range!!
… the non-debug log doesn’t show any more rot_set commands, but they seem to be sent in the background.

Other side of the coin, the rotctld.service log:

rotctld.service

Aug 17 13:06:46 vega rotctld[12739]: Connection opened from localhost:46054
Aug 17 13:06:46 vega rotctld[12739]: rot_get_position called
Aug 17 13:06:46 vega rotctld[12739]: serial_flush called
Aug 17 13:06:46 vega rotctld[12739]: tcflush
Aug 17 13:06:46 vega rotctld[12739]: write_block called
Aug 17 13:06:46 vega rotctld[12739]: rot_get_position: got az=180.00, el=90.00
Aug 17 13:06:46 vega rotctld[12739]: rot_set_position called az=356.80 el=0.45
Aug 17 13:06:46 vega rotctld[12739]: rot_set_position: south_zero=0
Aug 17 13:06:46 vega rotctld[12739]: serial_flush called
Aug 17 13:06:46 vega rotctld[12739]: tcflush
Aug 17 13:06:46 vega rotctld[12739]: write_block called
Aug 17 13:06:46 vega rotctld[12739]: write_block called
Aug 17 13:06:49 vega rotctld[12739]: rot_get_position called
Aug 17 13:06:49 vega rotctld[12739]: serial_flush called
Aug 17 13:06:49 vega rotctld[12739]: tcflush
Aug 17 13:06:49 vega rotctld[12739]: write_block called
Aug 17 13:06:49 vega rotctld[12739]: rot_get_position: got az=180.00, el=90.00
Aug 17 13:06:49 vega rotctld[12739]: rot_set_position called az=356.45 el=0.58
Aug 17 13:06:49 vega rotctld[12739]: rot_set_position: south_zero=0
Aug 17 13:06:49 vega rotctld[12739]: serial_flush called
Aug 17 13:06:49 vega rotctld[12739]: tcflush
Aug 17 13:06:49 vega rotctld[12739]: write_block called
Aug 17 13:06:49 vega rotctld[12739]: write_block called
Aug 17 13:06:52 vega rotctld[12739]: rot_get_position called
Aug 17 13:06:52 vega rotctld[12739]: serial_flush called
Aug 17 13:06:52 vega rotctld[12739]: tcflush
Aug 17 13:06:52 vega rotctld[12739]: write_block called
Aug 17 13:06:52 vega rotctld[12739]: rot_get_position: got az=180.00, el=90.00 // no movement
Aug 17 13:06:52 vega rotctld[12739]: rot_set_position called az=356.09 el=0.71
Aug 17 13:06:52 vega rotctld[12739]: rot_set_position: south_zero=0
Aug 17 13:06:52 vega rotctld[12739]: serial_flush called
Aug 17 13:06:52 vega rotctld[12739]: tcflush
Aug 17 13:06:52 vega rotctld[12739]: write_block called
Aug 17 13:06:52 vega rotctld[12739]: write_block called
etc.

I might add, that I’m disconnected from telnet when satnogs-client is connecting.

Honestly I have no idea what the issue is…

PE0SAT · August 17, 2024, 11:39am

Lets check the /dev/USB0 permissions and if the user satnogs is allowed to write to this device.

Please share the output of ls -l /dev/USB0

Looking at my own Linux system when connecting a USB to Serial converter, I get the following:

ls -l /dev/ttyUSB*

crw-rw---- 1 root dialout 188, 0 Aug 17 08:44 ttyUSB0
crw-rw---- 1 root dialout 188, 1 Aug 17 08:44 ttyUSB1
crw-rw---- 1 root dialout 188, 2 Aug 17 08:44 ttyUSB2
crw-rw---- 1 root dialout 188, 3 Aug 17 13:41 ttyUSB3

And the user satnogs is a member of dialout, so that should work.
Maybe you USB0 has another group membership

yasiu · August 17, 2024, 11:41am

Sure, here it is:

$ ls -l /dev/ttyUSB0
crw-rw-rw- 1 root dialout 188, 0 Aug 17 13:34 /dev/ttyUSB0

As I understand its rotctld that connects to the device in the end.

EDIT:

$ groups satnogs
satnogs : satnogs dialout plugdev

PE0SAT · August 17, 2024, 11:48am

What is the process owner of rotctld, looking at the systemd service file, it is hamlib-utils.
Then we maybe need to add hamlib-utils to dailout.

SA2KNG · August 17, 2024, 11:50am

It still has two rotctl(d) running, right ?

hamlib_utils_rot_enabled": true,

run a ps ax | grep rotctl and make sure there isn’t one running on systemd and one on -client

yasiu · August 17, 2024, 12:00pm

This is during a pass:

$ ps -ef | grep rotctl
hamlib-+ 16524 1 0 13:35 ? 00:00:00 /usr/bin/rotctld -m 603 -r /dev/ttyUSB0 -s 9600
yasiu 16644 848 0 13:53 pts/0 00:00:00 grep --color=auto

(In the meantime I removed -T and -vvvv from rotctld launch parameters, to make sure it doesn’t break anything.)

$ groups hamlib-utils
hamlib-utils : hamlib-utils dialout

There is only one instance of rotctld started by user hamlib-utils. The user has access to dialout which in turn has access to /dev/ttyUSB0.

This doesn’t suprise me, because I can connect to the rotctld instance via telnet, and the commands work. During a pass satnogs-client doesn’t spawn its own rotctl instance, because it is configured to connect to one at 127.0.0.1:4533.

I can telnet to the same instance outside and during a pass and my hand-issued commands move the rotator, but the ones issued by satnogs-client do not.

SA2KNG · August 17, 2024, 12:03pm

I think you should set all 4 min/max values and not let the default play tricks.
min_az=0,max_az=360,min_el=0,max_el=180

\dump_state.
-180.000000.
450.000000.
0.000000.
180.000000.

Also, having several things connected to the rotctld is’nt a problem, just don’t have more than one calling set pos.
Like, you can have telnet, gpredict (in monitor mode) and -client connected at the same time, no issues.

yasiu · August 17, 2024, 12:18pm

Thanks, sounds like a good idea overall, but it didn’t seem to change anything in this case. Telnet works (although I cannot move to az=420 for example - I’ll change it later) but during a scheduled pass the rotator doesn’t move.

I’m playing around some more with the manual commands and I notice one thing:

When I command the rotor to move in one direction (for example clock-wise), and then set another position which would need the rotator to move in the other direction (for example changing direction to counter clock-wise), instead of changing directions, it just stops moving.

To make the rotor move in the other direction I need to issue the command twice, or first issue a S-top command and then enter the new position.

Is this expected?

I’m not sure how does satnogs-client issues commands. Does it take into account changing directions, or does it just sends new positions?

On another occasion I was looking into satnogs log and trying to issue the same position commands manually, and they did work, although I was much slower at typing them in.

PS: Just to be sure, the version of Hamlib installed is 4.0.
I had an idea to use the –read-history and —save-history flag to debug it, but it’s not available in this version:

Aug 17 14:31:24 vega rotctld[17675]: /usr/bin/rotctld: unrecognized option ‘–read-history’
Aug 17 14:32:33 vega rotctld[18021]: /usr/bin/rotctld: unrecognized option ‘–save-history’

SA2KNG · August 17, 2024, 1:00pm

Normally, a stop command is only issued before a program is closing the connection, just to make sure it isn’t just left in a unpredictable state.
If the controller doesn’t handle a set position that changes the direction, there’s something really odd or wrong with it. Have never seen it on a K3NG rotator for instance.
satnogs-client and gpredict just sends a new set position when it wants it to move, nothing else is needed.

Big sidetrack, consider this risky:
If you want a newer hamlib, you can try building the one from a later debian release. YMMV (: a bunch of dependencies needed to build btw.

dget -q -u http://deb.debian.org/debian/pool/main/h/hamlib/hamlib_4.5.4-1.dsc
cd hamlib-4.5.4
DEB_BUILD_OPTIONS=noautodbgsym dpkg-buildpackage -b -us -uc --build=binary
cd ..
ls -l *.deb

make sure to install all the packages that you already have installed at once, else the versions will conflict and be a pita. `sudo dpkg -i xxx.deb yyy.deb zzz.deb"

yasiu · August 17, 2024, 1:09pm

Okay, I’ve done some more testing, and I come to the conclusion that only telnet works. I’ve installed SatDump and set it up to send commands to my rotctld instance. The outcome is the same - I can see the commands coming in, but no movement from the rotator.

I’ll try to configure a direct connection between satnogs-client with the rotator without the daemon. I might play around with bumping the hamlib version later.

SA2KNG · August 17, 2024, 1:55pm

If SatDump or gpredict isn’t able to control it, something is seriously wrong.
Is it refusing to run with floating point decimals ?!
P 180 90 vs P 180.0 90.0

yasiu · August 17, 2024, 2:14pm

Via telnet, it works. I tried all of the combinations:

p
\get_pos
P 180 90
P 180.00 90.00
P 180.0000001 90.0000001
\set_pos (+all of the above)

Of course with different positions, so its not in one place all the time. It works via telnet.

Now I tried running it without the daemon. Here is my config:

satnogs-client config

    "hamlib_utils_rot_opts": "--vvvv",
    "satnogs_antenna": "A_BALANCED",
    "satnogs_api_token": "[redacted]",
    "satnogs_log_level": "DEBUG",
    "satnogs_rot_baud": "9600",
    "satnogs_rot_enabled": true,
    "satnogs_rot_model": "ROT_MODEL_GS232B",
    "satnogs_rx_samp_rate": "2e6",
    "satnogs_soapy_rx_device": "driver=plutosdr",

I see that the SATNOGS_ROT_PORT doesn’t show up while running the Advanced → Support. It is set to /dev/ttyUSB0.

During a scheduled pass I see no errors in the satnogs-client.service DEBUG log:

satnogs-client.service log

Aug 17 16:09:07 vega satnogs-client[21681]: satnogsclient.scheduler.tasks - INFO - Spawning observer worker.
Aug 17 16:09:07 vega satnogs-client[21681]: satnogsclient.observer.observer - INFO - Start rotctrl thread.
Aug 17 16:09:07 vega satnogs-client[21681]: satnogsclient.observer.observer - DEBUG - TLE: {‘tle0’: ‘DEKART’, ‘tle1’: ‘1 46493U 20068H 24229.85783387 .00020301 00000+0 10409-2 0 9991’, ‘tle2’: ‘2 46493 97.8194 192.2344 0016697 106.1439 254.1633 15.16471996213586’}
Aug 17 16:09:07 vega satnogs-client[21681]: satnogsclient.observer.observer - DEBUG - Observation end: 2024-08-17 14:19:38+00:00
Aug 17 16:09:07 vega satnogs-client[21681]: satnogsclient.observer.worker - INFO - Tracking initiated
Aug 17 16:09:07 vega satnogs-client[21681]: rot_init called
Aug 17 16:09:07 vega satnogs-client[21681]: initrots4_gs232a called
Aug 17 16:09:07 vega satnogs-client[21681]: rot_register (601)
Aug 17 16:09:07 vega satnogs-client[21681]: rot_register (609)
Aug 17 16:09:07 vega satnogs-client[21681]: rot_register (610)
Aug 17 16:09:07 vega satnogs-client[21681]: rot_register (602)
Aug 17 16:09:07 vega satnogs-client[21681]: rot_register (603)
Aug 17 16:09:07 vega satnogs-client[21681]: rot_register (611)
Aug 17 16:09:07 vega satnogs-client[21681]: rot_register (612)
Aug 17 16:09:07 vega satnogs-client[21681]: rot_register (604)
Aug 17 16:09:07 vega satnogs-client[21681]: rot_register (605)
Aug 17 16:09:07 vega satnogs-client[21681]: rot_register (606)
Aug 17 16:09:07 vega satnogs-client[21681]: rot_register (607)
Aug 17 16:09:07 vega satnogs-client[21681]: rot_register (608)
Aug 17 16:09:07 vega satnogs-client[21681]: gs232b_rot_init called
Aug 17 16:09:07 vega satnogs-client[21681]: rot_open called
Aug 17 16:09:07 vega satnogs-client[21681]: serial_open called
Aug 17 16:09:07 vega satnogs-client[21681]: serial_setup called
Aug 17 16:09:07 vega satnogs-client[21681]: serial_setup: tcgetattr
Aug 17 16:09:07 vega satnogs-client[21681]: serial_setup: cfmakeraw
Aug 17 16:09:07 vega satnogs-client[21681]: serial_setup: cfsetispeed=9600,0x000d
Aug 17 16:09:07 vega satnogs-client[21681]: serial_setup: cfsetospeed=9600,0x000d
Aug 17 16:09:07 vega satnogs-client[21681]: serial_setup: data_bits=8
Aug 17 16:09:07 vega satnogs-client[21681]: serial_setup: parity=0
Aug 17 16:09:07 vega satnogs-client[21681]: serial_setup: tcsetattr TCSANOW
Aug 17 16:09:07 vega satnogs-client[21681]: serial_flush called
Aug 17 16:09:07 vega satnogs-client[21681]: tcflush
Aug 17 16:09:07 vega satnogs-client[21681]: satnogsclient.observer.observer - INFO - Start rigctrl thread.
Aug 17 16:09:07 vega satnogs-client[21681]: satnogsclient.observer.orbital - DEBUG - Observer data: {‘lon’: 19.691, ‘lat’: 53.7635, ‘elev’: 120}
Aug 17 16:09:07 vega satnogs-client[21681]: satnogsclient.observer.observer - DEBUG - Rig Frequency 437000000
Aug 17 16:09:07 vega satnogs-client[21681]: satnogsclient.observer.orbital - DEBUG - Satellite data: {‘tle0’: ‘DEKART’, ‘tle1’: ‘1 46493U 20068H 24229.85783387 .00020301 00000+0 10409-2 0 9991’, ‘tle2’: ‘2 46493 97.8194 192.2344 0016697 106.1439 254.1633 15.16471996213586’}
Aug 17 16:09:07 vega satnogs-client[21681]: satnogsclient.observer.observer - DEBUG - Observation end: 2024-08-17 14:19:38+00:00
Aug 17 16:09:07 vega satnogs-client[21681]: satnogsclient.observer.orbital - DEBUG - Calculated data: {‘alt’: 0.007631667889654636, ‘az’: 3.5551822185516357, ‘rng’: 2622207.25, ‘rng_vlct’: -6232.37890625, ‘ok’: True}
Aug 17 16:09:07 vega satnogs-client[21681]: rot_get_position called
Aug 17 16:09:07 vega satnogs-client[21681]: gs232b_rot_get_position called
Aug 17 16:09:07 vega satnogs-client[21681]: rig_flush: called for serial device
Aug 17 16:09:07 vega satnogs-client[21681]: serial_flush called
Aug 17 16:09:07 vega satnogs-client[21681]: tcflush
Aug 17 16:09:07 vega satnogs-client[21681]: write_block called
Aug 17 16:09:07 vega satnogs-client[21681]: write_block(): TX 3 bytes
Aug 17 16:09:07 vega satnogs-client[21681]: 0000 43 32 0d C2.
Aug 17 16:09:07 vega satnogs-client[21681]: read_string called, rxmax=32
Aug 17 16:09:07 vega satnogs-client[21681]: read_string(): RX 16 characters
Aug 17 16:09:07 vega satnogs-client[21681]: 0000 41 5a 3d 31 37 38 20 20 45 4c 3d 30 39 30 0d 0a AZ=178 EL=090…
Aug 17 16:09:07 vega satnogs-client[21681]: gs232b_rot_get_position: (az, el) = (178, 90)
Aug 17 16:09:07 vega satnogs-client[21681]: rot_get_position: got az=178.00, el=90.00
Aug 17 16:09:07 vega satnogs-client[21681]: satnogsclient.observer.worker - INFO - Tracking initiated
Aug 17 16:09:07 vega satnogs-client[21681]: satnogsclient.observer.worker - DEBUG - Rotctld msg: (203.6969365229654, 0.43726236072272234)
etc.

The rotator still doesn’t move…

I’m assuming the hamlib version is broken. I’ll try to bump it now.

SA2KNG · August 17, 2024, 2:22pm

I’m running 4.0-7 and it seems to be working just fine. tested on easycomm, gs232 and spid.
ii libhamlib4:amd64 4.0-7 amd64 Run-time library to control radio transceivers and receivers
you can raise the verbosity one or two steps so you can see the serial commands sent as well, but it gets very spammy

yasiu · August 17, 2024, 2:52pm

I seem to run the same version, but for ARM.

I’m loosing my marbles…

I look at the log file at the moment SatDump or satnogs-client sends commands (they don’t move the rotator), copy-paste the command and send it via telnet - they work. The commands reported as sent via serial are bit-perfect the same…

The only difference is the time… Maybe the initialization after connection is somehow different between telnet and anything else?

SA2KNG · August 17, 2024, 3:40pm

There is no initialization, I have this in a parking script:

echo "+P 180 90" | nc -w 1 localhost 4533

yasiu · August 17, 2024, 3:41pm

And it works for me too! It looks like I can script around the hamlib and communicate with my rotator via telnet only.

I’m out of ideas.

You make it seem like there are alternatives…

Any ideas how else can I connect my rotator to the SatNOGS network? The only alternative I see is to start with a fresh Raspberry Pi image and set up everything manually / via ansible.

SA2KNG · August 17, 2024, 3:51pm

I’m also out of ideas, this is very odd.

I don’t run it on the ansible (satnogs-setup, Pi image) install anymore thou, I have moved everything to docker.

It is a bit more messing around as it isn’t ocfficially released, but I have one system that runs rigctld in it’s own stack for different software to interface, satnogs-client one of them.
I have tested dockerized client with rotator, gps and whatnot.

yasiu · August 17, 2024, 5:03pm

Ok, I’ll try configuring the docker. Hopefully I find more time to finish this

I’ll keep you updated.

EDIT:

I tried docker, and exactly the same thing happens. I changed to Debian bookworm, installed docker and cofigured rotctld using your instructions.

This leads me to believe there must be something wrong with my hardware, but I fail to see how…