For uploading data, POST request, should work without any issue.
However for downloading data (your checking with URL in browser), GET request, you need to provide credentials and filter, with NORAD ID, to a specific satellite.
Depending on how you access the endpoint you need to use the right credentials. Accessing from the browser UI you need to have an active session in SatNOGS DB simply by logging in the site. Accessing directly the api it requires to use the API key.
There isn’t any response at all? Does this mean that the requests time out?
… so, clearly there is an issue specific to my Arduino code. Since the same thing is happening with multiple satellites and using different internet (ISP) connections, I suspect that a library update caused something to stop working properly. I will dig and get to the bottom of it. Many thanks!
While this is almost certainly a shortcoming (or, lack of understanding) on my part, I wanted to pass this finding along since it appears to be the root cause of my difficulty.
It “appears” that both the db and db-dev servers are including non-printable / non-ASCII characters in response to my POST commands.
Viewed in a serial monitor or copy/pasted to a text editor, of course you can’t see the detail. I’ll also include a hex editor view here so that the exact bytes can be observed. Please note that it’s not this one line - the entire HTTPS response continues to include a mix of printable & non-ASCII bytes. Also please note that this happens using multiple different serial monitor apps and on both the DB and DB-DEV instances.
If this is NOT a server issue, if anyone is aware of some particular character set, LOCALE, or other setting that I apparently now need to have my SIDS client be able to interpret, I’d be very grateful if you would pass that info along. Thanks!
Here is the first few lines of what I am seeing in the server responses to my POSTs:
To be honest I don’t know… I’ve just ran the curl command and the server respond with HTTP/2, so I thought to try also with HTTP/1.1. I’m not sure what’s the difference (or if it makes a difference in our case) and what’s the preferred way.
Maybe you are getting gzipped content as response, so that’s why there are non-ascii characters?
Also, I notice that the timestamp in the print of the response is mangled (it ends abruptly). This might be due to the Arduino code. Could you share the code that prints the response?
Thanks for the offer ref. how I’m receiving the server response!
This will be embarrassing since I’m a beginner & the faults in my code will be nearly infinite. But, this is how we learn, so thank you.
For a couple of months, everything worked perfectly where I cut out a small portion of the server response to only print out the ‘201’ reply code section. However, as you’ll see below, I have commented that parsing out to display a larger volume of the response which is where I see the non-ASCII characters. I should mention that this is not just a response display issue… sometimes my uploads DO reach the database and sometimes they do not - I have not yet found where the shortcoming is.
So, the function that displays the server response for me:
/* -------------------------------------------------------------------------- */
void read_response() {
/* -------------------------------------------------------------------------- */
uint32_t received_data_num = 0; // used w/ code below if full server response is displayed
int i=0;
while (client.available()) {
/* actual data reception */
char c = client.read();
// if(i < 64){ // if only interested in beginning of server response
if(i < 9999){
httpserver_RESPONSE[i] += c;
i ++;
}
}
/* print data to serial port */
String httpserver_CODE = String(httpserver_RESPONSE);
// httpserver_CODE.remove(20); // only first 20 characters of Server Response
Serial.println("SatNogs Server Response:");
Serial.println("***********************");
Serial.println(httpserver_CODE);
Serial.println("***********************");
/* wrap data to 80 columns*/ // un-comment below if printing FULL server response
received_data_num++;
if(received_data_num % 80 == 0) {
Serial.println();
}
Serial.print("\n");
// if the server's disconnected, stop the client:
if (!client.connected()) {
Serial.println();
Serial.println("disconnecting from server.");
client.stop();
}
memset(httpserver_RESPONSE, 0, sizeof httpserver_RESPONSE); // empty array
httpserver_CODE = "";
} //END:void-read_response
Btw, client.read() returns an int, not a char, and the value will be -1 if there is no more data to read. Not sure if that happens, but if it does the code will add -1 to httpserver_RESPONSE which might have unintended consequences. For example, it could result in printing characters from ram that were not received from the server.
Also, the code for the 80 character limit will not work but you might know that already and it is probably not relevant for this dicussion anyway.
The microcontroller being used is the new Arduino Uno-R4 (wifi model), so the compile report is:
Sketch uses 118380 bytes (45%) of program storage space. Maximum is 262144 bytes.
Global variables use 12348 bytes (37%) of dynamic memory, leaving 20420 bytes for local variables. Maximum is 32768 bytes.
… but I believe you found why my display becomes corrupt at the exact same point each time when I try to get the full server response! ( no surprise that the error is on my end )
Since I eventually settled on only parsing out a small portion of the reply, there was no need for any huge size in the character array to capture that info. So, the answer to your question:
How is httpserver_RESPONSE declared? How big is it?
… is going to be my issue. It’s declared up above but with only enough size for me to parse out the part I’m normally interested in. It would need to be larger to view a greater volume of the display. So, currently it is:
char httpserver_RESPONSE[64];
… but of course needs to be larger if I want to view more than the first 20 characters or so. I guess that I should allow enough space so that it doesn’t fill up & cause unintended errors - will get right on that.
… you guys are great - thanks a million! If the day comes that a microcontroller platform like this becomes something that is commonly used to contribute to SatNogs, I’ll have code that now contains fewer flaws.
Has anything changed in the past couple of months on the format or requirements for a SIDS upload to SatNogs?
The code that was working perfectly no longer gets any response at all from the server (either db.satnogs.org -or- db-dev.satnogs.org).
When I test my string w/ a CURL command, it shows up under the appropriate NORAD_ID, but my Arduino code gives the appearance of not getting any reply at all.
Thanks!
-Scott, K4KDR
Additional: using the ‘POSTMAN’ app to test my POST manually, the response that I get is:
{
"detail": "Authentication credentials were not provided."
}
… so perhaps I need to incorporate the API key associated w/ my SatNogs login (which I have obtained & saved for reference)?? I tried using that API key as a ‘bearer token’ in POSTMAN but I get the same response. Thanks for any tips!
That’s very strange, nothing has changed recently in the code that should affect this endpoint. Also on my tests with curl, I get my data uploaded in the db-dev and I get a 201 response as expected.
Can you send the arduino code and the POSTMAN request/settings?
EDIT:
I’ve just seen that you have already pasted the code in the previous messages… I’ll check it from there.
But give me the POSTMAN details if you can.
I’m a novice w/ POSTMAN, so I can understand that not working because of some mistake on my part. But it’s very odd that the Arduino code that was working perfectly is apparently not connecting to the DB at all. Perhaps it’s as simple as some update to the wifisslclient library, etc. I’ll have to dig deeper on that side w/ WireShark, etc.
As for POSTMAN, (or the POST upload to SatNogs more generally), I don’t believe that ANY authorization is required, correct? Is it correct that only certain GET requests require the API key?
Anyway, here are my current header & body screens. Very grateful to anyone that can spot what I’m doing wrong w/ this valuable tool:
Not sure about the arduino but for postman after checking some logs and some testing locally with a similar tool, I have some comments, please double check just in case:
Request should be POST
Check URL that doesn’t have any spaces before/after the address
The values in the body don’t need quotes
Make sure that Postman doesn’t sent any OPTIONS request
The last two are from the logs, all the requests that get 401 authorization error are either GET or OPTIONS, while all the POST requests get 400 bad request, probably due to the quotes.
If you don’t find any solution, check if you can generate a curl link through Postman interface. maybe we will be able to find out there what’s going on.
