In the works, November 2025

A post to summarise various aspects that are being worked on; some are additions, others are improvements and/or evolutions of the engine, both server- and client-side and being worked on more or less in parallel. Here goes:

Server-side changes

Even though the server-side procedural engine was completely upgraded a few weeks back does not mean it should stop evolving. :-) So here are the changes that are coming soon(ish) - not in order of importance or anything.

Entity bookmarks

As promised, the ability to bookmark more than just coordinates is coming and is running on localhost at the moment. Thanks to it, it is now possible to bookmark:

• Stars
• Satellites
• … and subsatellites

There is a clear distinction between bookmarking coordinates (which remains perfectly valid) and bookmarking an entity: you see, bookmarking coordinates close to a planet or moon will not anchor whatever object you think is worthy of being bookmarked; no, the coordinates are absolute in 3D space and the planets (or satellites) keep on moving server-side. I know I must still implement orbital movement animations, but this requires some synching work that I have not even started yet.

Enter entity bookmarks, which allow you to come back to the entity's current position, wherever that may be. Which means if you come back 2 days after bookmarking an entity, you will come back close and in view of it. An arbitrary choice was made: when teleporting to an entity bookmark, the engine will try and rotate your view so that you face not only the entity itself, but also the centre of the galaxy. In a very real way, the bookmark position/orientation prolongs the origin->entity position vector and looks back.

To bookmark an entity, simply access its "details view" and click the bookmark icon next to the close cross. You must be logged in for this to work.

And voilà. I hope this satisfies deziheyward and uhhhdie requests! I will in any case start thinking about a "my bookmarks" section where anyone will be able to annotate their bookmarks with comments, etc.

High-precision coordinates

Recently, while I was working on entity bookmarks, I thought I might try and reproduce a video I watched on YT a while back, where a crazy No Man's Sky player sent his ship to the other side of the galaxy, and found the/one of the most distant planets possible. It was an old one, but fun to watch, especially due to how long it had taken in real time.

Anyway, I went to a data explorer test galaxy, entered 10,000 for both X and Y coordinates, ended up in a cube very distant from the galactic centre and bookmarked a planet.
I then proceeded to the 3D explorer, loaded my bookmarks, clicked goto and teleported not far from the planet.

Great.

But long story short: I noticed the dreaded signs of precision loss. Hold on, isn't Adlumens supposed to use a floating origin, and so precision loss is not a problem?
Yes, Adlumens does make use of a floating origin camera and rebases it against the origin every frame; standard stuff which works very well for close-up objects who retain precision. Said precision is lost for faraway objects, but we don't really care because they are so small their wobble does not even show.

The problem came from the usage of f64s (server-side) and floats (client-side). Both have an impressive precision, but they could not discern between fractions of coordinates as far from the origin as 10,000 light years. Here's a breakdown of the problem I encountered:

1. Server-side coordinates are expressed in light years
2. The client uses a ratio light year/units, which allows it to draw distances; usually the ratio is 1e7 per light year as it seems to work quite well.
3. A 64-bit float has 52 bits for its fractional part.
4. 10000 (light years) is between 8192 and 16384 light years; expressed in powers of two, that is between 2¹³ and 2¹⁴. Take 13.
5. 2¹³ divided by 2⁵² is 2^(13-52) = 2^-39; that represents a precision of around 1.819e-12
6. That feels like and actually is an amazing precision. But multiply it by the amount of meters in a light year (our unit) (around 9.4607e+15) and you end up with a cannot-be-gone-around precision of around 17209 metres
7. That's 17 kilometres, give or take.
8. Past that scale, two distinct positions closer than ~17 km become indistinguishable in f64.
9. Past that scale, server-side calculations cannot be more accurate than ~17km. For player-centric activities such as rangefinding, surface movement/navigation, short-range combat (whether ground-based, from metres to kilometres) or space-based (tens/hundreds/thousands or kilometres), it is easy to see how huge a problem this can become.

It gets worse as you jump to the next "up" power ot two. Plus the infuriating part where precision is absolutely fine at low light-year coordinates, only to degrade as you move further from the origin. Inconsistent!

The problem is actually precision loss happens even before coordinates can reach the client. The precision is gone as soon as it is generated. I like to think about it in that way: the further out one goes from the galactic centre, the more "coarse" the unit substrate becomes. A less granular spacetime. I digress :-)

I do not claim to 100% master the above, but a solution seems to have presented itself. I read quite a bit about lossless large unit systems and one presented itself quite naturally. I skipped the exotic ones like prime factorisation form (gni?), symbolic representation (not too bad but really?) and even the logarithmic form and settled for a integer/float split.

How it (is starting to) work

This was my pick because it was conceptually simple: a normal human brain like mine (wife doesn't agree though) can very easily grasp that any coordinate can be expressed not via a single number, but using two of them:

• An integer (i64/BigInt) for the whole part of the coordinates
• A float (f64/Number) the fractional part "in between ones"

Woah hold on, why such a large integer? Surely an i32 can hold light year coordinates inside a galaxy, whose radii are between very small and very large - say 200,000 light years? Well because a i32 would not be able to hold large light year coordinates times the client-side unit ratio. It overflows. Like, badly. :-) I tried and saw it fail, so i64/BigInt it is.

Overall principles

The overarching idea is the following:

1. All coordinates, both server-side and client-side, are internally kept in this dual format.
2. Only when they need to interact with the outside do they end up as f64s. Such interaction might be to set positions in a Babylonjs Vector3 or run a matrix transform in rust.
3. Whichever the use case, the engine tries to leave the full precision untouched up until the very last moment.

Due to this need, I had to implement class methods such as:

etc, etc.

And yes, high-precision coordinates are implemented client-side as well. But since it started server-side…

Results and performance

The final outcome is constant precision maintained to a full 64-bit float, wherever coordinates point to in a galaxy; whether at 0,1,4 light years or 10000/97640/4566. Cool. :-)

To compare to the brief (and I hope correct) calculation above:

• Since we store fractional parts separately, we enjoy the full bits for precision
• Which should be around 1.11 × 10^-16 and smaller!
• Which leaves us with a sub-metre precision, in a way that is completely independent from the integer part of coordinates.

Performance-wise, all of the above is of course slower than simply toying around with floats. However, I have tried to:

• Server-side: minimise heap allocations and try to work on the stack as much as possible. I tried to inject some SIMD but frankly, that is a little bit beyond me and most calculations are done on object instancing. In other words, not many things run on large lists of such new structs; and when they do, it is for initial pseudorandom creation. Maybe later.
• Client-side: try and minimise GC pressure in hot loops and avoid new() calls whenever possible. Quick tests seem to indicate that shifting origin nodes using Position3D is about 40% slower than the native BabylonJS Vector3 subtract. On ~2500 origin node (a rare occurence), that's (locally, on a nice i9 14900HX), around 0.25ms. Which does not seem to clobber the CPU too much.

Useful code :-)

python…:

import math
value = 10000
precision = math.ulp(value)
print(f'At value {value}, precision is {precision}')

… or rust:

let x: f64 = 10000.0;
let precision = x.ulp();
println!("At value {:?}, precision is {:?}", value, precision);

Update November 28^th

I am also in the process of removing the client-side multiplier/ratio. Why? Because it was originally used to try and maintain precision, but a single float context. Given that the new internal coordinate system now always enjoys full precision, this has now become obsolete.

One more less multiplication! Always nice.

3D explorer streaming data optimisations

In a previous version, the data streamed by the server to the client - as the user moves around a galaxy - was all sent as very long strings. While this works, I managed to:

1. return tuples which are transparently interpreted by python/msgpack and then msgpack/javacript. This saves time both on the server and its now useless joins() and the client which now no longer has to split those large strings into discreet bits.
2. I also had to find a way to go around pyo3's limitations in the number of values it can return to python from rust. That came via pub type declarations, often wrapped within one another to "fool" pyo3 into believing less values are returned to python

Typical values returned are now structures like this, instead of glued heterogenous values:

pub type SystemPassiveLoeData = (
    String,
    [f64; 3], // x,y,z
    Vec<StarPassiveLoeData>
);

pub type StarPassiveLoeData = (
    String, // id
    String, // designation
    String, // star type symbol
    f64, // radius
    f64, // luminosity sun
    [u8; 3], // color
    [f64; 3], // ecliptic
    Vec<SatellitePassiveLoeData>,
);

pub type SatellitePassiveLoeData = (
    String, // id 
    String, // designation
    String, //satellite_type_symbol
    f64, // radius
    [f64; 2], // orbit radius, orbit_eccentricity
    f64, // rotation_period
    [f64; 3], // position
    [f64; 3], // ecliptic
    // subsatellites
    Vec<(
        String, // id
        String, // designation
        String, // satellite_type_symbol
        f64, // radius
        [f64; 2], // orbit radius, orbit_eccentricity
        f64, // rotation_period
        [f64; 3], // position
        [f64; 3], // ecliptic
    )>,
);

The sharp-eyed have of course noticed that all positions are still expressed as single f64s. :o) I am working on migrating this at the moment. So all those positions fields will be replaced by (i64, i64, i64, f64, f64, f64) or similar.

All the float values above are returned at a forced/arbritrary level of precision that reflect graphical needs. Why? Well, systems at LOE_SYSTEM are very very far away and do not require visual precision at all. Which is why their fractional precision will always be forced to 0.0. All the extra bytes would be wasted anyway!

Procedural data tweaks

Those are under constant scrutiny for weird outcomes after the procedural engine uses them; I am also working on wording reviews for descriptions, which could gain by being a bit more immersive…

Client-side changes

Custom shaders

I confess this is a major one for me as GLSL is not really my forte. But the change are not only about shaders themselves - this is also about how shaders are loaded and made available to scenes.

BabylonJS offers multiple ways of loading custom shaders.

Parallel to this, I had been thinking about server-side database storage and serving of shaders. This, for me, had several advantages:

• the ability to consider shaders not as code, but as content
• the ability to make shader code dynamic if ever needed (ie, pass them through a server-side template or similar)
• the ability to have someone else than me (better than me at GLSL which is not hard) create/modify them, without having access to the main codebase; all that is needed is an admin access
• the ability to serve them via websockets, during scene initialisation
• the ability to store them "in the clear" and serve a compressed/obfuscated version of them, via tools such as dnload

That's a lot of advantages; I do not think inlining <script>, using external js files or harcoding in the source - or anything else, for that matter - would work as well for me.

An example of custom shader for stars

A picture has more weight than a thousand words so here goes:

This is not 100% finished; as you can see, the star sphere is bleeding and a weird ring appears around it. I plan on working on this as well as polishing the integration of the following stellar statistics as attributes/varyings:

• Luminosity of course
• Colour variations
• Impact of being a remnant (of what type?), a giant, etc. All with impacts in terms of flare colour and range, speed of animation, etc. Endless fun! :-)

My only frustration is that this is billboarded… and I would love the flares to be volumetric. But that is way beyond me at the moment!
Last, this is also good training for me for when I come around and integrate planetary composition into the (still deactivated as of writing this) planetary surface wasm!

3D explorer: render worker reorganisation

A lot of work has been done on the reorganisation of many components of the render worker. Previously, most of it was centered around a central Galaxy class, which exposed many methods for state management, moving stuff around from and to various LOEs, rendering, etc. The new paradigm is now to extract most components out of the galaxy and insert them at the root of the singleton (for the main thread) or self (for workers).

The below illustrates an extremely simplified galaxy, before the changes:

class Galaxy extends OriginNode{
  constructor(data){
    this.systems_loe_sys = new Map();
    this.systems_loe_star = new Map();
    this.systems_adjacent = new Map();
  }
  init_systems_loe_system(data_from_websocket){
    this.do_something();
    // ...
  }
  init_systems_loe_star(data_from_websocket){
    this.do_something_else();
    // ...
  }
  init_systems_adjacent(data_from_websocket){
    this.do_something_else_more();
    // ...
  }
  // etc
}

After the change, the systems are now full-fledged classes; for example:

class Galaxy extends OriginNode{
  constructor(data){
    // ...
  }
  @include 'galaxy/foo.sjs';
  @include 'galaxy/bar.sjs';
}

class SystemsSys extends Map{
  constructor(data){
    // ...
  }
  @include 'systems_sys/foo.sjs';
  @include 'systems_sys/bar.sjs';
}

class SystemsStar extends Map{
  constructor(data){
    // ...
  }
  @include 'systems_star/foo.sjs';
  @include 'systems_star/bar.sjs';
}

class SystemsAdjacent extends Map{
  constructor(data){
    // ...
  }
  @include 'systems_adjacent/foo.sjs';
  @include 'systems_adjacent/bar.sjs';
  @if dev
  @include 'systems_adjacent/debug.sjs';
  @end
}

It has all become cleaner:

• In the logical structure itself, i.e., objects are more personified and it becomes easier (at least for me) to think of actions as methods on the relevant objects instead of a generic galaxy tasked with doing pretty much everything. I think I was here guilty of transferring the logic of the server-side structure to the client. Which was of course ridiculous.
• But also in my custom made aggregator which is quite convenient for handling one file == one method, including by taking into account optional files during gulp "compilation"

To inline or not to inline CSS and JS? And other questions

Frankly I am torn on this one. Not so long ago both CSS and JS files were external assets to a page and called via standard <link> and <script> elements.

The table below summarises my thoughts:

Basic stuff, but worth remembering when trying to decide. :-) I struggled (and still am) to decide so I ended up writing gulp tasks that support both solutions, and more:

• CSS and JS files were made far more specialised. The result is that there are now far more of them, but each of them far less generic. The two main advantages of this are: 1) the skipping of useless CSS/JS downloading and parsing, which is a little performance gain and 2) better code coverage. Drawback: there is slightly more maintenance.
• Each file is generated is two different places and in two slightly different formats:
  • Inline version, included directly from django templates
  • External asset version, pushed to STATIC_FOLDER and served via a dedicated static asset server (s.adlumens.org). Each external asset file generation is also associated with a cache busting hash file, whose role is to make sure the right file version is served.
• A global --dev flag generates the flags in production or dev build (minified/tersed and potentially obfuscated or fully in the clear with indentation, etc). Convenient.

And now it becomes easy to switch to and from both worlds: external or inlined or both or none, in development or production mode. The solution includes only switching CSS and not JS, or vice-versa. At the moment, the full site runs in full inline mode, by the way.

NPC factions

LLM playgrounds

I have managed to squeeze a few playful sessions to work on an extremely basic LLM server and client pair. Both python based and handmade to use sockets. The models used are of course limited to the hardware resources at my disposal.

What is this for? Well, it is an extremely early series of tests for AI-driven NPCs dialogues, along with the embryo of core values, memory, objectives and relationship management injection, via Qdrant and RAG. I am but a mere dabbler, but after trying the following models:

• mistralai/Mistral-7B-Instruct-v0.3
• meta-llama/Meta-Llama-3-8B-Instruct
• NousResearch/Nous-Hermes-2-Mistral-7B-DPO
• meta-llama/Meta-Llama-3-13B-Instruct
• mistralai/Mixtral-8x7B-Instruct-v0.1 ERROR NOT ENOUGH MEMORYYYYYYYY

… all quantized to 4 bits (due to my having only a RTX 4090 on a laptop, hence 16GB), I can say that pure dialogue is quite fun. Of course prompts need to be carefully crafted and I am a bit lost in NLP, vector embeddings and semantic retrieval.

Core values and personality

I will post soon on the subject, but here's what I am thinking at the moment:

• All factions, be they PC or NPC, will have to define their core values. In summary, is your faction a bunch of grim scientists, a crew of bold explorers, a band of ruthless but romantic pirates, jumpy settlers or just the Firefly crew?
• Since I like the principle of immersion, PC factions should be made to respect their core values. In other words, if a player declared he was A and starts playing his faction as if it was B, then penalties will ensue. Whether those penalties are morale-related or perhaps even cause crew member defections remains to be seen.
• NPC factions are both easier and harder to handle: easier, because they are fully automated and the engine that drives them is not subjected to mood swings/metaplay like humans/players are. Harder because maintaining alignment to core values is hard both in the short and long term

It's all very basic for now; all I have done is start to define:

1. The dimensions/traits along which a faction is defined: aspects such as Freedom, Dignity, Harmony, etc. All scores from 0.0 to 1.0. This I hope will be extremely helpful in crafting long-terms behaviours into NPCs.
2. In a similar fashion, the traits for individual interacting NPCs such as Empathy, Generosity, Ambition or Patience. All scored from 0.0 to 1.0 too.
3. … and then ways to store such information (or rather a compacted version) as vectors to be used when simulating not only dialogue, but also monologue/inner dialogue and consistent decision making. On top of course of memory.

Comme dirait l'autre, on est pas rendus !

In closing for now

Thanks for taking the time and that's it for now! Most of the above is still in development, especially the big precision migration and NPC thingie, as they both require a lot of attention, vigilance about silent bugs/errors, or plain learning new stuff. Interesting times ahead.