Auto-Scheduler log output errors

m0roj · July 21, 2022, 7:33am

For each set of timed observations using satnogs-auto-scheduler with cron it shows the following as part of the preamble prior to displaying the small list of scheduled sats’ in the logs. The observations appear unaffected.
I would welcome comment on the cause(s) and the fix please?

Jul 20 20:41:58 SatGOLD python3[1326]: 2022-07-20 20:41:58,418 - auto_scheduler.io - WARNING - Malformed line, expected 3 parameters but found 1
Jul 20 20:41:58 SatGOLD python3[1326]: 2022-07-20 20:41:58,418 - auto_scheduler.io - WARNING - Malformed line, expected 3 parameters but found 1
Jul 20 20:41:58 SatGOLD python3[1326]: 2022-07-20 20:41:58,418 - auto_scheduler.io - WARNING - Malformed line, expected 3 parameters but found 18
Jul 20 20:41:58 SatGOLD python3[1326]: 2022-07-20 20:41:58,419 - auto_scheduler.io - WARNING - Malformed line, expected 3 parameters but found 18
Jul 20 20:41:58 SatGOLD python3[1326]: 2022-07-20 20:41:58,419 - auto_scheduler.io - WARNING - Malformed line, expected 3 parameters but found 16

kerel · July 21, 2022, 10:03am

Hi M0ROJ,

This log message originates here in the code for parsing the transmitter priorities. So apparently your transmitter priority file is malformed.

Feel free to post your transmitter prority file if you can’t find the syntax error and I’ll have a look.

Sincerely,
Fabian

PS: Please avoid all-bold messages in forum posts. Instead to separate code from text you can create a code block by enclosing the code section with lines of three backticks (more details and a short example on helpful markdown tutorial page).

m0roj · July 21, 2022, 5:04pm

Thanks Fabian. And my apologies for the bold text. Thanks for the tip.
I cannot identify syntax error, so herewith the priorities file.
Many thanks for your help.

STATION_ID=2122
TRM_FILE="/tmp/cache/transmitters_2122.txt"
PRIO_FILE="priorities_2122.txt"

awk '{if ($3>=80) print $0 }' ${TRM_FILE} | grep -e "FSK" | awk '{printf("%s 1.0 %s\n",$1,$2)}' > ${PRIO_FILE}
awk '{if ($3>=0) print $0 }' ${TRM_FILE} | grep -e "BPSK1k2" | awk '{printf("%s 1.0 %s\n",$1,$2)}' >> ${PRIO_FILE}
awk '{if ($3>=0) print $0 }' ${TRM_FILE} | grep -e "BPSK9k6" | awk '{printf("%s 1.0 %s\n",$1,$2)}' >> ${PRIO_FILE}
awk '{if ($3>=80) print $0 }' ${TRM_FILE} | grep -e "MSK" | awk '{printf("%s 1.0 %s\n",$1,$2)}' >> ${PRIO_FILE}
sort -n -k 4 ${TRM_FILE} | grep -e "DUV" | awk '{printf("%s 1.0 %s\n",$1,$2)}' >> ${PRIO_FILE}

jebba · July 21, 2022, 5:30pm

That doesn’t look like a proper priorities file. Per help, it is supposed to be like:

help="File with transmitter priorities. Should have " +
"columns of the form |NORAD priority UUID| like |43017 0.9" +
" KgazZMKEa74VnquqXLwAvD|. Priority is fractional, one transmitter " +
 "per line, 1.0 gets maximum priority.",

Edit: I see that what you have there is a script from the docs that generates the priorities file, not the file itself. I think they’ll want to see the latter.

m0roj · July 21, 2022, 5:48pm

Oops! Apologies. Do you mean like this?

26931 2dCbF2jmsxFpHw8ApevTMd 11 125 1117 AFSK
05680 2GvxE2TC9BsEVdqLoG7F3D 8 120 1448 CW
32789 2GyWowDxwCzVjbUW5H2R5Y 44 238 531 BPSK
44885 37STiZxCY6Yft8sZ68LWfE 17 533 3006 GFSK
43137 3rLGJWqj3XZ6Z8vADCRwiW 73 52171 70682 DUV
49273 3wSKbpe2xXy74jGeZpQBQ3 6 47 686 AFSK
42778 52jVE9w8EnSpF8YFssD5KD 78 37206 47455 CW
43770 5AqszaBQhFwfRF74XHUXY5 81 513 626 FM
52175 5RFYLP5dKvt3Vb3YcNN2vm 30 130 430 GFSK
22491 696NHJGiRZYuxM8qzJkFgc 44 8 18 PSK
40032 6DQp67crsBzih2LWRGFNFM 4 109 2454 BPSK
47309 6eXkz6niXrNxgEQGSGvQNJ 14 134 936 FSK
40906 6PkyCNyEaqRbADNZXj94WM 66 179 271 USB
44398 6PMJaNzeqyaLHDazrHWHHK 8 20 247 CW
33591 6sVpSCgyPBskf5Mt6qp49N 26 672 2514 BPSK
40074 7AchptE4yLHTtdLHSZQR4o 46 45 97 CW
40074 7FGfFQtV3iYR8anFpbZKz3 54 149 275 CW
42761 7fV9xpVQEQfSHipnioTrFg 81 51933 63450 CW
04237 83t3GeyBEcXWqXaCvdLn9K 22 648 2833 CW
39427 88v7p7asctEGk47ZrqF7yN 35 77 214 AFSK
52738 9Ba9rwPsPQhArn4Y2g4s7R 39 308 789 GFSK
40074 9UHihpacwFrRmgZNAweK2g 37 1540 4074 BPSK
39432 a5kpE63CcUXcGwLN4fY26k 6 186 2713 BPSK
14781 aBPWui23BqUhVc62nySNuQ 35 3484 9875 AFSK
14129 ae83giGVVtiqwjkiyPPvAa 33 39 115 CW
43139 ahu2AFiiiRGcqxX3Gz5Whk 12 67 532 LoRa
43792 aj32dry5ZZkoA84yLYz8y8 22 25 113 FM
42017 aKFXGsXLrQFvgJFRW7cCCh 34 142 415 USB
43770 aQiKaeWH7YPEMNgGo5Qg58 78 153 195 AFSK
39428 aRboijNaKGSDZu3ZVyTFX2 35 3201 9102 BPSK
40025 AWujQt3WDBqLAmjXYtBgPZ 8 4 48 BPSK
44400 bnnhacLTJVVt5kVJpouHrY 9 15 158 CW
41171 btX4S4t58zxggKoLHYWGMR 57 2111 3644 CW
01291 bVkDZUhYzLQzjgMxACbJwU 28 1006 3519 USB
02412 BvWvsP5U9GgFTsR6xEYPsB 17 300 1724 CW
43770 bxfwWfvm9UaXRvVfyhcjt6 88 59444 66932 DUV
40909 BZVHvyW5jm2owcx7L5gYt8 74 155 207 USB
40907 bzvvEibPpi3aRsrUjBTH7V 65 2911 4463 GMSK
42761 CaxhYGFfYkuzJ7JRgQcQU7 79 830 1038 USB
03669 CDtgaCLGBG8JRPDK7NChdL 31 943 2993 USB
43137 ckTw5Du4VKjWGouE36z5b3 80 1222 1513 FM
40069 CojkGDaq3u42nRdLdfczng 75 49023 64767 LRPT
40903 cpx9PNK4TVM2nWMsQCNAph 82 32472 39589 CW
43140 cvzGBFRi8gaf882DL4iiqX 20 7 34 LoRa
25338 D3ZUeiSUnUQkq5EuEAepxi 72 216 300 PSK
40021 D5hpsmx8LgSCS3u9rpLMVC 3 1 27 BPSK

jebba · July 21, 2022, 6:17pm

When I run the script, my output looks like this.

48850 1.0 6pfjjBe2aEmpnJrsukKZy5
40014 1.0 NSXo8tGxmxpTUMsmSH34FF
40379 1.0 PMnr7eXunTq4zsS3WnbqxV
30776 1.0 rjz4x8XfpQCvmM8qgArr9Z
44103 1.0 TGP8nqLVquNWuYYytpbQ7A
51085 1.0 7RKVacYwn9iHxWwDpqKhQs
41460 1.0 HhPiuNj3whyDNCFCYBDykV
41460 1.0 JLNq6FzBfaCvK7H5neMKiC
43908 1.0 oXe5XCa53jwWzCac5W75Bm
42761 1.0 s7WSyh6UFkfkzeW4rro6Ze
40014 1.0 VqUvcE3MLq2sfsgvwJBuyC
41168 1.0 XsKJgznKU3tFxKKVsNRXGY
40967 1.0 ZyjKNJ9KqnTHBCUzAPN5G5
43137 1.0 3rLGJWqj3XZ6Z8vADCRwiW
43770 1.0 bxfwWfvm9UaXRvVfyhcjt6
43017 1.0 KgazZMKEa74VnquqXLwAvD

Per your error, expected 3 parameters but found 16, it appears the file is malformed. You have more fields than I do. I cut & pasted the scriptlet from here:

STATION_ID=37
TRM_FILE="/tmp/cache/transmitters_${STATION_ID}.txt"
PRIO_FILE="priorities_${STATION_ID}.txt"

awk '{if ($3>=80) print $0 }' ${TRM_FILE} | grep -e "FSK" | awk '{printf("%s 1.0 %s\n",$1,$2)}' > ${PRIO_FILE}
awk '{if ($3>=0) print $0 }' ${TRM_FILE} | grep -e "BPSK1k2" | awk '{printf("%s 1.0 %s\n",$1,$2)}' >> ${PRIO_FILE}
awk '{if ($3>=0) print $0 }' ${TRM_FILE} | grep -e "BPSK9k6" | awk '{printf("%s 1.0 %s\n",$1,$2)}' >> ${PRIO_FILE}
awk '{if ($3>=80) print $0 }' ${TRM_FILE} | grep -e "MSK" | awk '{printf("%s 1.0 %s\n",$1,$2)}' >> ${PRIO_FILE}
sort -n -k 4 ${TRM_FILE} | grep -e "DUV" | awk '{printf("%s 1.0 %s\n",$1,$2)}' >> ${PRIO_FILE}

m0roj · July 22, 2022, 3:33pm

[SOLVED] - well perhaps, but I don’t know how, other than having reinstalled the latest iteration from gitlab (noting that there had been a new commit only 22 hrs ago by Fabian). Previously floundering around by hunting, pecking and trying various changes without success.
But there may well be more to come if not.
Thanks @jebba and @kerel

jebba · July 22, 2022, 3:39pm

FWIW, I have never re-installed satnogs-client from the initial install a couple years ago. You really shouldn’t ever have to re-install.

kerel · July 22, 2022, 7:16pm

This commit only changed the log message to be more verbose (Merge Request !94).

Good to hear you solved the problem though!

m0roj · July 25, 2022, 6:00pm

Sorry to say that the problem is NOT solved as I’m still receiving this error, and I think its caused by my misunderstanding of the coding and placements:

2022-07-25 17:12:51,246 - auto_scheduler.io - WARNING - Malformed line in priority file /home/usr/satnogs-auto-scheduler/priorities_2122.txt,
Jul 25 17:12:51 SatGOLD python3[2497]:  expected 3 parameters but found 18

Q1. Where do I place the scheduling script file?
Q2. Where to place the PRIO_FILE?
Q3. Where should the <path_to_priority_list>/<priority_file> be located?
Q4. References are made to both auto_scheduler and satnogs-auto-scheduler - are they the same?
I’m a Linux mature novice and a Python beginner, so please excuse what may be thought of as elementary questions. But I am a long-in-the-tooth radio ham, willing to learn.
Many thanks in advance.

jebba · July 26, 2022, 12:22am

The locations don’t necessarily matter, you’ll just likely want them in your home directory. Whatever path you use, then that’s what you have to call when running it.

Open a text editor and enter this text, saving the file as gen-priorities.py:

STATION_ID=2122
TRM_FILE="/tmp/cache/transmitters_${STATION_ID}.txt"
PRIO_FILE="priorities_${STATION_ID}.txt"

awk '{if ($3>=80) print $0 }' ${TRM_FILE} | grep -e "FSK" | awk '{printf("%s 1.0 %s\n",$1,$2)}' > ${PRIO_FILE}
awk '{if ($3>=0) print $0 }' ${TRM_FILE} | grep -e "BPSK1k2" | awk '{printf("%s 1.0 %s\n",$1,$2)}' >> ${PRIO_FILE}
awk '{if ($3>=0) print $0 }' ${TRM_FILE} | grep -e "BPSK9k6" | awk '{printf("%s 1.0 %s\n",$1,$2)}' >> ${PRIO_FILE}
awk '{if ($3>=80) print $0 }' ${TRM_FILE} | grep -e "MSK" | awk '{printf("%s 1.0 %s\n",$1,$2)}' >> ${PRIO_FILE}
sort -n -k 4 ${TRM_FILE} | grep -e "DUV" | awk '{printf("%s 1.0 %s\n",$1,$2)}' >> ${PRIO_FILE}

Then run:

bash ./gen-priorities.py

Not:

python3 ./gen-priorities.py

That will output THREE columns which look like this:

48850 1.0 6pfjjBe2aEmpnJrsukKZy5
40014 1.0 NSXo8tGxmxpTUMsmSH34FF
40379 1.0 PMnr7eXunTq4zsS3WnbqxV
30776 1.0 rjz4x8XfpQCvmM8qgArr9Z
44103 1.0 TGP8nqLVquNWuYYytpbQ7A
51085 1.0 7RKVacYwn9iHxWwDpqKhQs
41460 1.0 HhPiuNj3whyDNCFCYBDykV
41460 1.0 JLNq6FzBfaCvK7H5neMKiC
43908 1.0 oXe5XCa53jwWzCac5W75Bm
42761 1.0 s7WSyh6UFkfkzeW4rro6Ze
40014 1.0 VqUvcE3MLq2sfsgvwJBuyC
41168 1.0 XsKJgznKU3tFxKKVsNRXGY
40967 1.0 ZyjKNJ9KqnTHBCUzAPN5G5
43137 1.0 3rLGJWqj3XZ6Z8vADCRwiW
43770 1.0 bxfwWfvm9UaXRvVfyhcjt6
43017 1.0 KgazZMKEa74VnquqXLwAvD

That output is for the priorities file, priorities_2122.txt, so just save that where you like such as your path from earlier:

/home/usr/satnogs-auto-scheduler/priorities_2122.txt

If you DON’T get that, please upload the exact command you ran with the exact output.

m0roj · November 9, 2022, 2:59pm

Having never getting satnogs-auto-scheduler to run successfully - despite my attempts back in July, I’ve revisited the topic having become bored with scheduling manually. But I keep on receiving a syntax error in priorities_2122.txt which I cannot find. The error given is:

:~/satnogs-auto-scheduler $ python3 gen-priorities.py
  File "/home/roj/satnogs-auto-scheduler/gen-priorities.py", line 5
    awk '{if ($3>=80) print $0 }' ${TRM_FILE} | grep -e "FSK" | awk '{printf("%s 1.0 %s\n",$1,$2)}' > ${PRIO_FILE}
        ^
SyntaxError: invalid syntax

And yet the coding is identical to that given on the gitlab instructions website and as @jebba suggested.
Does anyone please have an explanation?

PE0SAT · November 9, 2022, 3:47pm

Can share some extra information, what git repo, how did you install the schedular, share your priorities file etc. The information you can share the better.

mfalkvidd · November 9, 2022, 8:51pm

The code jenbba posted doesn’t look like python. Looks more like shell script to me. But I’ll leave to @jebba to let us know.

jebba · November 10, 2022, 1:07am

Ya, awk script is shell.

mfalkvidd · November 10, 2022, 5:25am

So this part of your instructions won’t work very well:

Maybe edit the post to avoid confusing people?

m0roj · November 11, 2022, 5:50pm

The README is not sufficiently explicit and therefore somewhat confusing as regards the scheduling script; especially to limited Linux/Python users like me. Although I must admit I’ve learned quite a lot in solving the problem. Some of the erroneous comments in this posting didn’t help. Anyway, Its now working well and I’m now looking to modifying and enhancing the coding give a larger choice of decoders; including the intermittent ones. I would be interested to hear how others may have improved upon it?
Thanks to all.

Auto-Scheduler log output errors

For each set of timed observations using satnogs-auto-scheduler with cron it shows the following as part of the preamble prior to displaying the small list of scheduled sats’ in the logs. The observations appear unaffected. I would welcome comment on the cause(s) and the fix please?

For each set of timed observations using satnogs-auto-scheduler with cron it shows the following as part of the preamble prior to displaying the small list of scheduled sats’ in the logs. The observations appear unaffected.
I would welcome comment on the cause(s) and the fix please?