SatNOGs Network API - pagination change

SatNOGS Software
1

Hi, we have been using the network api for a long time now to capture a list of observations with no decoded data - we use this to run a “backfill” operation using the audio recording and gr-satellites to decode the observations. This has been pretty successful and has run trouble free for a long time.
About a week ago it stopped working. The problem seems to be something to do with the api access to multiple pages of observations. We are running a request in the following form:
url = f’{NETWORK_BASE_URL}/api/observations/?ground_station=2433&start={start_time}&end={end_time}&page={page}’
We retrieve the observations using requests.get(), stepping through from page 1 by incrementing the value of {page} each time until we run out of pages (tested by the type of response to the request).
Since a week ago, this approach always retrieves the same page of up to 25 observations and fails to find any more pages so gets stuck in an endless loop. It is as though setting page={page} no longer works. How do we fix this?

Brian M0GKK

2

Hi @briany1000,

Indeed the API has changed the pagination technique that it uses due to some performance issues.

Now the page parameter doesn’t work anymore. On your code you have to remove the page parameter and run something like that for the first page:

url = f’{NETWORK_BASE_URL}/api/observations/?ground_station=2433&start={start_time}&end={end_time}

and for getting the next page, you should use the header Link in the response headers of your request. Link header should look like that:

Link: 
 <https://network.satnogs.org/api/observations/?cursor=cD0yMDI0LTA0LTE1KzEwJTNBMTYlM0EyMiUyQjAwJTNBMDA%3D>; rel="next", 
 <https://network.satnogs.org/api/observations/?cursor=cj0xJnA9MjAyNC0wNC0xNSsxMiUzQTEzJTNBMTclMkIwMCUzQTAw>; rel="prev"

So, you need to extract the "next" link from the header and use it for requesting the next page. In case you want to go to the previous one you use the "prev" link.

Let me know if you have any questions.

3

Thanks Fredy, I got it working!

Brian

1 Like