Broken Pipe Error - Doesn't appear to be getting jobs

SatNOGS Software
1

Hey all, I’m working on a ground station using a pi, and I can’t seem to get the pi to receive jobs either from the network or the local server. The website says the station is in the testing phase, so I’m pretty sure it’s not offline/it’s properly connected to the internet. In the journal I see

[13/Jun/2018 15:42:21] "GET /socket.io/?EIO=3&transport=polling&t=MFwgqi6&sid=d55dc62fd6e242aa8911055af750781a HTTP/1.1" 200 -
Jun 13 15:42:21 raspberrypi satnogs-client[463]: Traceback (most recent call last):
Jun 13 15:42:21 raspberrypi satnogs-client[463]:   File "/usr/lib/python2.7/SocketServer.py", line 596, in process_request_thread
Jun 13 15:42:21 raspberrypi satnogs-client[463]:     self.finish_request(request, client_address)
Jun 13 15:42:21 raspberrypi satnogs-client[463]:   File "/usr/lib/python2.7/SocketServer.py", line 331, in finish_request
Jun 13 15:42:21 raspberrypi satnogs-client[463]:     self.RequestHandlerClass(request, client_address, self)
Jun 13 15:42:21 raspberrypi satnogs-client[463]:   File "/usr/lib/python2.7/SocketServer.py", line 654, in __init__
Jun 13 15:42:21 raspberrypi satnogs-client[463]:     self.finish()
Jun 13 15:42:21 raspberrypi satnogs-client[463]:   File "/usr/lib/python2.7/SocketServer.py", line 713, in finish
Jun 13 15:42:21 raspberrypi satnogs-client[463]:     self.wfile.close()
Jun 13 15:42:21 raspberrypi satnogs-client[463]:   File "/usr/lib/python2.7/socket.py", line 283, in close
Jun 13 15:42:21 raspberrypi satnogs-client[463]:     self.flush()
Jun 13 15:42:21 raspberrypi satnogs-client[463]:   File "/usr/lib/python2.7/socket.py", line 307, in flush
Jun 13 15:42:21 raspberrypi satnogs-client[463]:     self._sock.sendall(view[write_offset:write_offset+buffer_size])
Jun 13 15:42:21 raspberrypi satnogs-client[463]: error: [Errno 32] Broken pipe

I’ve tried upgrading socket and socket.io, but I can’t get it to work. The jobs.sql file just has a header and no jobs listed.

Any help is greatly appreciated. Thanks a bunch!

2

Hey @tqk0611 and welcome!

So, I think you are almost ready…

According to the log your station indeed was online and connected to the network. Testing mode is the default status for the new stations and it is shown only if station is online, if it is not the station shows up as offline.

The error you see in your logs is a known error that doesn’t play any role, you can safely ignore it.

About zero jobs, isn’t an issue… it is just that no one (in testing mode only you and admins) had scheduled any observation. We currently add observations manually while moving slowly to automate this process. So, as soon as you schedule some observations, you will start getting results.

Based on these results try to adjust the settings of your station like gain, ppm error etc, in the wiki you can find some guidance.

After you feel that your station is giving the best it can you can go to your station’s settings and uncheck the testing mode checkbox.

EDIT: Feel free to join our irc/matrix channels for live chatting/questions etc… you can find more details in the wiki link above.

3

Thanks so much for the response, and I apologize in advance for my short reply - I’m on a mobile device right now…

We actually did try to schedule jobs. I scheduled 5-6 jobs and they all came back as “need vetting” with no waterfall or audio data uploaded to the site, and I couldn’t find any trace of the job on the pi (eg. Audio, waterfall, job list, etc…) which is why I’m a little concerned. I should mention I installed the packages on the pi manually and didn’t use the custom raspian image. I made sure to upgrade all of the packages I felt were needed. If there is any more information I can provide, please let me know. Thanks so much again for the assistance!

4

I guess then you have deleted them… this is why I didn’t see any observation :slight_smile:

In such cases we mark them/vet them as failed no need to remove them, there is no problem as you are in testing mode.

Two questions to help you debugging the issue:

  1. What OS have you installed in rpi?
  2. Which guide(s) did you followed for setting rpi up?

It would be great if you have already or if you want to document your manual installation steps in order to update any guides as some of them may need update. :wink:

5

I did write a guide actually but it might not include every single package I needed to update but I do believe it includes most if not all of them. It’s running on raspian.

When I add a job via the network, how can I verify that my ground station received and will only execute the job? I will post the guide once I get back to my desk in a bit

6

If everything goes well you should see something like this in logs:
2017-07-24 12:11:53,323 - apscheduler.executors.default - INFO - Job "get_jobs (trigger: interval[0:01:00], next run at: 2017-07-24 04:12:50 UTC)" executed successfully

Forgot to ask… which branch did you use for client, master or 0? If it is 0 then you can check also the web interface at http://:5000.

7

We used 0, I think. We followed the instructions here. I have been using that page, but it never seems to update the log unless I change from stand-alone to network or vice-versa. Here is my log

pi@raspberrypi:~ $ journalctl -f -u satnogs-client.service
-- Logs begin at Wed 2018-06-13 16:09:20 EDT. --
Jun 14 08:09:34 raspberrypi satnogs-client[462]: 2018-06-14 08:09:34,993 - apscheduler.executors.default - INFO - Job "post_data (trigger: interval[0:02:00], next run at: 2018-06-14 12:09:34 UTC)" executed successfully
Jun 14 08:09:35 raspberrypi satnogs-client[462]: 2018-06-14 08:09:35,926 - apscheduler.executors.default - INFO - Job "get_jobs (trigger: interval[0:01:00], next run at: 2018-06-14 12:10:34 UTC)" executed successfully
Jun 14 08:10:35 raspberrypi satnogs-client[462]: 2018-06-14 08:10:35,010 - apscheduler.executors.default - INFO - Running job "get_jobs (trigger: interval[0:01:00], next run at: 2018-06-14 12:10:34 UTC)" (scheduled at 2018-06-14 12:10:34.933626+00:00)
Jun 14 08:10:36 raspberrypi satnogs-client[462]: 2018-06-14 08:10:36,326 - apscheduler.executors.default - INFO - Job "get_jobs (trigger: interval[0:01:00], next run at: 2018-06-14 12:11:34 UTC)" executed successfully
Jun 14 08:11:34 raspberrypi satnogs-client[462]: 2018-06-14 08:11:34,956 - apscheduler.executors.default - INFO - Running job "get_jobs (trigger: interval[0:01:00], next run at: 2018-06-14 12:11:34 UTC)" (scheduled at 2018-06-14 12:11:34.933626+00:00)
Jun 14 08:11:35 raspberrypi satnogs-client[462]: 2018-06-14 08:11:35,024 - apscheduler.executors.default - INFO - Running job "post_data (trigger: interval[0:02:00], next run at: 2018-06-14 12:11:34 UTC)" (scheduled at 2018-06-14 12:11:34.949433+00:00)
Jun 14 08:11:35 raspberrypi satnogs-client[462]: 2018-06-14 08:11:35,034 - apscheduler.executors.default - INFO - Job "post_data (trigger: interval[0:02:00], next run at: 2018-06-14 12:13:34 UTC)" executed successfully
Jun 14 08:11:35 raspberrypi satnogs-client[462]: 2018-06-14 08:11:35,596 - apscheduler.executors.default - INFO - Job "get_jobs (trigger: interval[0:01:00], next run at: 2018-06-14 12:12:34 UTC)" executed successfully
Jun 14 08:12:34 raspberrypi satnogs-client[462]: 2018-06-14 08:12:34,954 - apscheduler.executors.default - INFO - Running job "get_jobs (trigger: interval[0:01:00], next run at: 2018-06-14 12:12:34 UTC)" (scheduled at 2018-06-14 12:12:34.933626+00:00)
Jun 14 08:12:35 raspberrypi satnogs-client[462]: 2018-06-14 08:12:35,874 - apscheduler.executors.default - INFO - Job "get_jobs (trigger: interval[0:01:00], next run at: 2018-06-14 12:13:34 UTC)" executed successfully
Jun 14 08:13:34 raspberrypi satnogs-client[462]: 2018-06-14 08:13:34,963 - apscheduler.executors.default - INFO - Running job "get_jobs (trigger: interval[0:01:00], next run at: 2018-06-14 12:13:34 UTC)" (scheduled at 2018-06-14 12:13:34.933626+00:00)
Jun 14 08:13:34 raspberrypi satnogs-client[462]: 2018-06-14 08:13:34,997 - apscheduler.executors.default - INFO - Running job "post_data (trigger: interval[0:02:00], next run at: 2018-06-14 12:13:34 UTC)" (scheduled at 2018-06-14 12:13:34.949433+00:00)
Jun 14 08:13:35 raspberrypi satnogs-client[462]: 2018-06-14 08:13:35,014 - apscheduler.executors.default - INFO - Job "post_data (trigger: interval[0:02:00], next run at: 2018-06-14 12:15:34 UTC)" executed successfully
Jun 14 08:13:35 raspberrypi satnogs-client[462]: 2018-06-14 08:13:35,631 - apscheduler.executors.default - INFO - Job "get_jobs (trigger: interval[0:01:00], next run at: 2018-06-14 12:14:34 UTC)" executed successfully

Seems OK, right? What’s the next step in checking if the job was executed? I can manually add jobs using the :5000 page, correct? Should I be seeing temp files somewhere as the job is executed?

Thanks so much for the assistance

EDIT: It may be working now… I’ll let you know after this next observation

1 Like
8

Yes logs look ok. You can manually add jobs but in stadalone mode… this means that you will stop any communication with the network and you will probably not run network’s jobs.

Yes during the observation running you should be able to see at least two files in /tmp/.satnogs/data (except if you have changed this default directory). There should be one for waterfall and one for audio and optionally files for decoded data.

9

Hello @tqk0611

One thing that would be worth a try if it still doesn’t work is seeing what date/time your raspberry pi is running. For me, none of my scheduled observations would actually run because 1) the raspberry pi thought it was still 2014 and 2) it was running off local time instead of UTC. Because of this, the raspberry pi never started the job because it didn’t think it was time.

You can correct the date and time by typing:

sudo date -s “Wed Jun 14 12:12:12 UTC 2018”

But with the correct time and whatnot.

1 Like
10

@elkins_ka Good point! :slight_smile:

11

I can get the jobs and the uploads worked occasionally, but I’m now getting “Error on request %s” in the log and the waterfall doesn’t upload…

EDIT: Working again…