(Satnogs) Satellite ID

[fredy]

After many and long discussions we are almost ready to start implement (SatNOGS) Satellite ID. This is a quick FAQ and an overview on how we are going to move. Any feedback or questions are more than welcome.

Why another ID, or why not using NORAD ID?
We need an ID that will allow us to uniquely identify satellites and to uniquely relate Artifacts with them. NORAD ID can not be used, as a satellite can have 0, 1 or more than one possible NORAD IDs, making this ID unusable.

What is an Artifact?
For those not familiar with the term, that is used lately, artifact is a combination of a result and its metadata of an observation. We are still looking for the best way to store artifacts in DB, but we have found and we currently experiment with HDF5 for transferring artifacts from clients to DB. About the metadata, Metasat project is in progress. Metasat is focused on describe the structure and the fields of the metadata that are related with space missions.

Are we going to change SiDS or how SiDS will change?
The DB endpoint for Artifacts will not be compatible with SiDS. SiDS endpoint will remain, until is fully deprecated. So, the goal is to move to better ways/protocols that will allow us to keep observations results in a better and more scientific way. And a way to do this is by implementing Satellite ID.

Who is going to create these IDs?
There will be two ways to create a Satellite ID:

1. Creating a Satellite object in DB
2. Receive Artifact with a new Satellite ID

Will this be a UUID?
Technically yes, practically due to the nature of DB and how we are going to accept Artifacts, no. It would be possible to merge or split “Satellite” objects as there have been cases and there will be ones that demand to be able to merge or split.

How clients, that send Artifacts, would be able to generate an ID?
For generating these IDs, we are going to use a library called Nano ID which is available in several programming languages. So any client would be able by using Nano ID to generate Satellite IDs.
Note: Generating an ID on a client will be necessary in case that there isn’t one available on DB, also developers and users of such clients will be encouraged to create satellite entries directly in DB, instead of using the “Sending Artifacts” way.

[surligas]

@fredy

Receive Artifact with a new Satellite ID

How the producer of this artifact will generate a valid ID? Does the process rely on specific attributes? How we avoid that x stations are producing x UUIDs for the same tracking object, making the merging process a PITA?

[fredy]

@surligas

How the producer of this artifact will generate a valid ID?

By using the same library (Nano ID).

@surligas

Does the process rely on specific attributes?

Not sure if this answers your question… Nano ID ensures that the ID will be unique enough to have any collisions. But even in this very very rare occasion we will have the split mechanism.

@surligas

How we avoid that x stations are producing x UUIDs for the same tracking object, making the merging process a PITA?

Unfortunately you can not, as you can not right now avoid the fact that stations could generate random NORAD IDs. Merging would be a simple action. To be honest I don’t expect many clients/stations to generate their own IDs, usually would be easier and more convenient to use the existing or create a new one in DB.
However if we end up in a strange situation that independent stations generate many IDs for their artifacts, then we can create extra toolset that will allow us to recognize duplicates, for example checking transmitter, NORAD IDs, TLEs, demodulated or decoded data etc.

[fredy]

I’m pumping up this thread in case someone missed it.

[pierros]

Some previous context and discussion that led to our developments:

Unique Satellite identifier definition in SatNOGS

(this post is based on latest discussions on this topic in IRC) New Satellites in SatNOGS SatNOGS has an identification of a new satellite of interest consisting of the satellite name and a set of transmitters (in edge cases only of the transmitter and not the satellite name) usually before the launch as provided by the satellite owners (or other sources) or shortly after the launch by scan observations. NORAD IDs and COSPAR designators JSpOC tracks new satellites and assigns a sequential 5 di…

[kerel]

A User Story I’m thinking about since a few days:

I discovered a previously unknown satellite which is not tracked by NORAD and thus does not have a NORAD ID which is transmitting in amateur band. I was able to reverse engineer the protocol and decoded some frames. Now I want to upload those frames (& artifacts for doppler analysis) to SatNOGS.

I generate a new SatNOGS Satellite ID using Nano ID and upload the frames & artifacts.
SatNOGS DB accepts this data and creates a new satellite.

Now a second observer receives frames from this satellite and wants to upload those to db. She has to browse the SatNOGS DB Web UI for this satellite and copy the SatNOGS Satellite ID into her software/client.

How to avoid manually handling the Nano ID (21 char string) in this case? The satellite doesn’t have a unique name (maybe it is dubbed “Mystery Sat”).

Currently (using SiDS) our temporary assigned “NORAD ID” would be used which is much easier to work with (but which is only valid for a limited time).

[fredy]

@kerel

Now a second observer receives frames from this satellite and wants to upload those to db. She has to browse the SatNOGS DB Web UI for this satellite and copy the SatNOGS Satellite ID into her software/client.

Let’s split the example above into different use cases:

1. Observer knows about the mystery satellite, so the intention is to observe this satellite. In this case observer’s client/software should have requested from DB the satellites, so the observer be able to choose from a list of satellites (with their names or other properties) the mystery one. Artifacts from observations will have the right ID as the selected satellite has already ID.
2. Observer doesn’t know about the mystery satellite but receives it. Then there are two options:
   a) Observer search in DB for the mystery satellite and finds it and uses the ID to upload observation’s artifacts.
   b) Observer generate a new ID (satellite entry) and upload the artifacts in DB. Now in DB when we realize that the two satellites are the same one, we associate/merge the two entries. In this case both observers would be able to continue using the initial IDs they used, but in DB would be associated and count as one satellite.

@kerel let me know if on the cases above I missed something.

[kerel]

@fredy

let me know if on the cases above I missed something.

Nothing missing in the description.

@fredy

In this case observer’s client/software should have requested from DB the satellites, so the observer be able to choose from a list of satellites (with their names or other properties) the mystery one

The observer’s software might be some non-interactive CLI tool, so I assume ultimately the user will have to manually handle the Satellite ID. This is a UX issue when the Satellite ID is a long alphanumeric string, but not a show-stopper.

The proposed support of truncated Satellite IDs on UI would fix this issue.

[Acinonyx]

Having an ID which is “memorizable” would be a nice feature. Inspired by car license plates, which are somewhat possible to memorize, I would suggest a few improvements:

1. Split into fixed size segments
2. Use a limited character set per segment
3. Keep it truncate-able
4. Make sure it is extendable to increase the size in the future in a backwards compatible way, if required

Some facts:

• Standard Nano ID size/charset equivalent is 126bits (64^21 = 2^126)
• IPv6 ULAs are 120bits.

I am setting a target of 1% chance of collision on ~10 billion generated IDs. This is close to 72 bits in size.

With Nano ID full charset, the above equivalent number of characters for 72bits is 12. That could be split into 4 segments of 3 characters each to satisfy (1). It also satisfies (3) and (4) easily. But (2) is a problem since the large (64) character set of uppercase, lowercase, numbers and two symbols makes it very difficult to memorize, even truncated.

What I’m proposing is this form:
CCCC-NNNN-NNNN-NNNN-NNNN

where:
CCCC = { A-Z }{ A-Z }{ A-Z }{ A-Z }
NNNN = { 0-9 }{ 0-9 }{ 0-9 }{ 0-9 }

The equivalent bits are ~72 (26^4 * 10^16).

The above form can be truncated into its segments. E.g. JAHK-2812-9284-1248-4812 can be truncated to JAHK-2812 (which is “memorizable” as a typical license plate is equivalent to ~32bits) or JAHK-2812-9284 if a collision occurs.

Also, if at some point we get close to collisions, the size can be increased to 85bits by adding an additional numeric segment while the existing 72bit IDs can be converted to 85bit IDs by adding a -0000 segment at the end.

[dernasherbrezon]

For generating these IDs

Oh! This could exploited in so many ways. I would assume no one would bother to map his/her generated satellite ID back to the standard. Why? Look at the vetting algorithm right now. Half of observations not vetted, half of positively vetted observations doesn’t contain anything.

NORAD ID can not be used, as a satellite can have 0, 1 or more than one possible NORAD IDs

Can you elaborate on this? Doesn’t look like correct. Do you refer to a situation just after the launch? When satellite cannot be reliably identified?

[fredy]

@dernasherbrezon

Oh! This could exploited in so many ways. I would assume no one would bother to map his/her generated satellite ID back to the standard. Why? Look at the vetting algorithm right now. Half of observations not vetted, half of positively vetted observations doesn’t contain anything.

To be honest I don’t expect many clients of DB not using the already generated satellite IDs. But in any case new satellite IDs/entries will stand as suggested satellites in DB. Operations team will review these new entries and either keep them, merge them with another entry or reject them, depending on the rest of the metadata and actual data of the artifact. The suggestion/review process would be similar to the current one for transmitters and clients will be able to use this new ID all the time, before and after review.

The main reason for allowing clients to generate their own IDs is to cover cases that satellite is not known or access to IDs is not possible. In these cases you need the client(s) to be able to group their results/artifacts with an ID.

@dernasherbrezon

Can you elaborate on this? Doesn’t look like correct. Do you refer to a situation just after the launch? When satellite cannot be reliably identified?

We have 3 states:

• Before launch/deployment, 0 NORAD ID assigned which means no way (except from a temporary NORAD ID) to test integration of the mission with the SatNOGS in different levels (Client, Network, DB, Dashboards etc).
• After launch/deployment but before identification, 0 to many (possible) NORAD IDs. In this state again there are issues with integration with SatNOGS. Also external stations/clients/observers are not sure which NORAD ID should use to send data to SatNOGS DB (the temporary one, one of the possible NORAD IDs?).
• After identification, 1 NORAD ID but this may never happen. If identified then well no problem but there are plenty examples of satellites that never have been identified and you need to somehow be able to still refer to these satellites. Also in these cases it would be useful to be able to document which NORAD IDs could be the satellite, allowing anyone to choose one of the objects to track (currently we are limited to the one NORAD ID we set to follow).

I hope I answered your questions, let me know If I missed any or if there are more questions.

[kerel]

@fredy

But in any case new satellite IDs/entries will stand as suggested satellites in DB. Operations team will review these new entries and either keep them, merge them with another entry or reject them, depending on the rest of the metadata and actual data of the artifact.

I do see a DoS vulnerability on review of new satellites when (malicious) users submits many artifacts with unknown satellite IDs.

But as just discussed with @fredy we consider this not a blocking issue for the implementation of this proposal (especially given that we are discussing SatNOGS Satellite ID proposals since a long time and have to settle with some solution).