NOAA APT slanting issues

Hi,

now that I’ve got a little more free time on my hands,
I’ve decided to look int the issues with the APT decoder again.

So back in January I open a merge request with a gnuardio port of my code from GitHub - LongHairedHacker/apt-decoder: Automatic picture transmission encoder in Rust
The code syncs to A and B markers in the signal continuously,
instead of just once, which pretty much corrects all of the slanting.
Initially it seemed to work perfectly.

However at some point it stopped working without any changes done to my code, producing even worse images than before.

https://network.satnogs.org/observations/164929/

I’m trying to reproduce the issue by replicating the decoder does using the audio files for this observations.

apttest.grc.txt (20.2 KB)

This flowgraphs takes the 49kHz audio from observations, re-samples it to 62.5kHz like in the original flow graph, does the exact same filtering and feeds it into the NOAA APT sink from the gr-satnogs master.
Now we’ve got a new problem: The resulting png looks perfectly alright.

So there must be some difference between the audio from the file and what is being used in the regular decoder flowgraph.

So hints on what I might be missing here would be highly appreciated.

Currently the only thing that I can think of is that audio written to the file been down-sampled to 48kHz and hence has a smaller bandwidth.
This may filter out some artefacts, that otherwise cause trouble for my line syncing code.

Other than that I’m currently out of ideas.

Also thanks to Alexander for spotting my typo and opening a merge request:

Unfortunately this is not cause of the problem.

Update:

After @DL4PD helped me to stare at things and poke flowgraphs,
we noticed something.

Running my sample flowgraph on the exact same file on a Pi gives me a slanted image as well.
My Pi has a recent enough gr-satnogs installed that it has the same APT sink code as the one on my desktop machine.

So something seems to go wrong when running on ARM.

1 Like

Maybe it’s compiled with software floating point? Instead of hardware?

Interesting idea might have to look into that at some point.

I had the idea that I should take an image decoded on the satnogs station
where the syncing code just didn’t do anything at all and compare it to the same one decoded on my computer with syncing disabled.
In an ideal world that would be pixel perfect identical images.

I picked SatNOGS Network - Observation 24624 as my test subject.

From the station:

From my computer:

They are pretty much the same, right?
Well I used gimp to compute the difference between the two.
Ideally the result would be completely black.

And there’s our problem.
Everything with large changes from one pixel to the net,
such as the noise at the beginning and end and of course the sync markers is ever so slightly different on the when decoded on the station.

That’s why the line sync code can not find the markers.

1 Like

Oh well, this just keeps getting better.

So I wrote little python script to look for the sync markers in slanted images from the network.
It uses the same algorithm than the sink, as I was hoping to prototype something more robust with it.

Turns out it identifies the markers just fine for all my test images.

Red indicates potential type A markers, blue potential type B markers.
Since the brightness of the pixels in the PNG are 1:1 the samples processed by the gnuradio module it should work on the station just as well.

And once again I’m out of ideas.

1 Like

And to add insult to injury here is that totally messed up image from 164929.

I can’t even spot the point where it wrongly since and jumped all over the place.
And even if there was one it should sync fine on the next line.
This is starting to get really weird.

1 Like

Very interesting. This is very strange.

After freeing my raspberry pi 3 from an other permanent test installation I can debug the code directly on the target platform.

Here’s what I found so far:
For some reason the first sample passed to the sink on the Pi will always be NaN.
Since arithmetic operations with NaN will also yield NaN this sample ‘poisons’ the floating average obtained exponential smoothing, which is used by the code looking for the markers.

For some bizarre reason this never occurred on x86 systems.

I’ve added code to skip the NaN samples, which seems to work okay,
but I’ll need a little while to fully test it.

Here’s the first slant free image it produced:

4 Likes

Look how beautiful 164929 turned out:

Also there is a merge request now: Fixed slating issues caused by NaN samples (!157) · Merge requests · librespacefoundation / SatNOGS / gr-satnogs · GitLab

5 Likes

So I have made three observations with the fix from the MR merged into gr-satnogs.

  1. Obs 25408
  2. Obs 25409
  3. Obs 25410

They all worked okay.

1 Like