It seems that more and more, it is taking very long to vet my observations. Sometimes a single observation will take over 30 seconds to load the full waterfall. I know this is being worked on, so at this point are there any things that I can do on my end to speed this up?
For example, if I load 30 observations for vetting, generally only 5 waterfalls will completely load. The rest will stall, and any progress made on the downloading of the waterfall seems to get “lost” to the internet. Is there any setting that I can make on my browser to “keep” what is already downloaded (and hopefully continue the download from that point when refresh is clicked)? Any other suggestions?
Seems like Satnogs network is growing, so it may be outgrowing its current hardware or bandwidth capabilities along with the user growth. Could a real solution to this be a network or bandwidth upgrade? Perhaps a funding campaign with a fixed goal would cover the costs to upgrade the infrastructure. It wouldn’t be bad to point out how much funding is needed to keep the operation going for 1 month, 6 months and a full year.
LSF patreon exists and that’s great, but campaigns with clear goals would probably bring in more funds. I’m sure there are those would would prefer to make larger donations once or twice a year, instead of smaller amount every month.
I had no idea there even was a Patreon. Having said that, looks like not many others know about it, either (current take is $8 a month). I prefer straight donations over “subscription” donations, but I’ll look into this.
No, it’s still slow. I’m coming up on 15 minutes trying to vet 25 observations. Even after an observation is fully loaded, sometimes just selecting the vet button will result in 10 to 15 seconds of wait time until the vet results are shown.
[edit] I just loaded a single observation. 26 seconds to load the full waterfall.
oof… ok, back to the drawing board. Thanks for reporting.
The worst I can produce is 5s. If you are familiar with developer tools in Firefox or Chrome could you send a screenshot of the Network tab when loading a slow page, along with any errors that come up in the console?
No errors popped up. Here’s the network tab with likely the most pertinent information highlight. This observation also took about 30 seconds to load the waterfall.
As an update, it has taken roughly 45 minutes of loading, reloading, reloading, etc. to vet 30 observations.
In the last few months, this has been the case for only certain parts of the day. However, lately - it seems that no matter when I try and vet observations, it takes ages to load the waterfalls. I will also add that I have no problem streaming HD movies, or slow loading speeds on any other pages - so I don’t think this is a connection issue on my end.
FWIW, I’ve been able to vet 50+ tabs loading using “Open all tabs” and they’ve been loading fine. I’m even on a pretty mediocre Internet connection. So it may not be the SatNOGS server itself. It could be your path from your location to the server. You could do a traceroute to network.satnogs.org to find out more.
Just took 48 seconds to load a single observation. It’s always the waterfall .png holding up the download.
If this continues I will not be able to vet observations anymore (vacation ends today… can’t spend 40 minutes every day vetting observations, unfortunately).
[edit - here’s the traceroute for anybody that’s interested]
traceroute to network.satnogs.org (94.130.162.100), 30 hops max, 60 byte packets
1 _gateway (192.168.1.1) 3.855 ms 3.789 ms 3.750 ms
2 * * *
3 B3373.PHLAPA-LCR-22.verizon-gni.net (100.41.13.154) 19.991 ms 20.524 ms 20.497 ms
4 * * *
5 0.ae2.BR1.PHIL.ALTER.NET (140.222.237.229) 14.183 ms 17.867 ms 0.ae1.BR1.PHIL.ALTER.NET (140.222.237.227) 19.023 ms
6 1-1-2 (63.65.164.121) 18.687 ms 9.155 ms 7.389 ms
7 * ash-bb3-link.telia.net (62.115.124.225) 115.709 ms *
8 prs-bb4-link.telia.net (62.115.122.158) 115.695 ms 115.682 ms 115.631 ms
9 ffm-bb2-link.telia.net (62.115.122.139) 109.575 ms 111.763 ms 108.354 ms
10 ffm-b5-link.telia.net (62.115.114.91) 106.645 ms 104.666 ms 106.239 ms
11 hetzner-ic-326013-ffm-b4.c.telia.net (213.248.70.3) 122.174 ms 121.886 ms 108.391 ms
12 core21.fsn1.hetzner.com (213.239.224.241) 113.082 ms 115.146 ms 119.930 ms
13 ex9k2.dc4.fsn1.hetzner.com (213.239.229.134) 118.506 ms ex9k2.dc4.fsn1.hetzner.com (213.239.229.130) 119.904 ms ex9k2.dc4.fsn1.hetzner.com (213.239.229.134) 118.209 ms
14 srv01.libre.space (94.130.162.100) 115.893 ms !X 115.838 ms !X 115.257 ms !X
The traceroute is pretty normal for crossing the ocean. Any chance you could try your workstation on another connection or try another computer at your current connection? I just say this because I haven’t seen the issues you encounter and I vet observations every day. If it was on the server end, I would occassionally see it slow too (I’m in USA too). If you just download an image with wget or similar, is it also slow? You could right click on a waterfall, and wget that URL to see if it is fast or slow.
You could also try using a VPN proxy or similar. Proxies usually slow things down, but if you have a bad route, it may actually speed it up, since the route will be different.
Hate to say it, but I have been on vacation as well and have been vetting up a storm.
Done a few hundred and have noticed no slow down at all. Just as snappy as usual (so love those ‘link bombs’ that open 50 tabs at once, what an awesome feature!).
I am in Southern California.
