Architecture decisions¶
Why the engine is built the way it is. Each record (an ADR, architecture decision record) says what was decided, why, and what it costs. They are short on purpose: the code and its tests are the detail. They are written for contributors and for anyone who wants to know why Ubunye behaves as it does.
Three words used on these pages, in plain terms:
- Hexagon: the engine's core, drawn as a shape with many sides, one per kind of outside thing it talks to. The core holds the rules and decisions and knows nothing about any particular engine or cloud.
- Port: a side of that shape: the contract the core sets for one kind of outside thing, such as "something that holds a table" or "somewhere to store run records".
- Adapter: the code that makes one real thing (Spark, pandas, a database, a cloud) fit a port. Swapping an adapter changes what Ubunye talks to without changing the core.
| ADR | Decision | Since |
|---|---|---|
| 001 | Backends are plugins, found by name | 0.7.0 |
| 002 | Backends say what they can do; tasks are checked before they run | 0.7.0 |
| 003 | One order for choosing a backend; the task never names it | 0.7.0 |
| 004 | Your transform gets native frames; the engine gets the port | 0.7.0 |
| 005 | One transform for every engine: Narwhals, detected not declared | 0.7.0 |
| 006 | The run record is correct before it is sold: every row, any order, same on every engine | 0.7.0 |
| 007 | One time zone for every backend, UTC unless the task says | 0.8.0 |
| 008 | A rerun is safe: one live run per batch, appends land once | 0.8.0 |
| 009 | Each output is computed once; checks, write and run record get that copy | 0.8.0 |
| 010 | Registry backends are plugins, found by name | 0.9.0 |
| 011 | Run records are stored through a plugin, found by name | 0.9.0 |
| 012 | Model runs on the receipt; features checked at predict | 0.9.0 |
| 013 | Job health and upstream checks are receipt queries | 0.9.0 |
| 014 | The MLflow registry backend uses the Model Registry: stages are aliases, the rest are tags | 0.9.0 |
| 015 | Job settings and the run identity come from platform plugins | 0.9.2 |
| 016 | A notebook's with block is one recorded run; every failure names its stage |
0.9.2 |
| 017 | A SQL name from a variable is quoted by one built-in filter, ident |
0.9.2 |
| 018 | insert_new inserts rows with new keys and changes nothing else |
0.9.2 |
The rule behind all of them¶
The engine is a hexagon. The core (ubunye/core) holds the contract and the
decisions; everything that touches the outside world (Spark, pandas, a database,
a cloud) is an adapter behind a port, found through an entry point. The core
never imports an engine. A test reads every import in ubunye/core, even ones
inside functions, and fails if one names Spark, pandas or pyarrow.