OpenRA3/README.md

# OpenRA3

A from-scratch, portable re-implementation of **Command & Conquer: Red Alert 3**
in **C++26** using **C++ modules** and `import std;`, built with **Clang** for
both **Linux** and **Windows**.

Red Alert 3 runs SAGE 2.0. EA never released that engine's C++ source, but it
did open-source the closely related SAGE 1.0 engine as
[`electronicarts/CnC_Generals_Zero_Hour`](https://github.com/electronicarts/CnC_Generals_Zero_Hour)
(GPLv3). OpenRA3 uses that tree as the architectural reference and reconstructs
the RA3-specific behaviour by decompiling the retail `ra3_1.12.game` binary with
Ghidra.

> OpenRA3 ships **no game assets or binaries**. You must own Red Alert 3. The
> engine reads your own local installation at runtime; nothing from it is ever
> copied into this repository. Command & Conquer and Red Alert are trademarks of
> Electronic Arts; this project is unaffiliated with and unsupported by EA.

## Status

OpenRA3 is at **v0.8.0**. The engine compiles and runs headless, and a **minimal
skirmish** is playable: it reads a real multiplayer map out of your install,
recovers the player start waypoints, and simulates two sides building a base,
extracting ore and fighting until one side is wiped out. The balance is the
**retail Red Alert 3 balance**, pinned in `ra3.data` from EA's open RA3 XML:
damage types, `ArmorTemplate` percentages, weapon target masks, build costs and
the ore economy. Presentation offers **Vulkan** (`ra3.vulkan`), **Direct3D 11 / 12**
(`ra3.dx`), **WebGPU** (`ra3.webgpu`, the wasm worker backend) and **WebGL 2**
(`ra3.wasmgl`, the SDL-free wasm worker fallback) GPU backends with an SDL
software blit and null fallbacks; the backend is selectable from the in-game menu
and the software renderer still produces headless images. Terrain tiles
cross-fade the way the retail `Terrain.fx` does (a per-cell blend ramp plus a
gutter-padded atlas), so material boundaries are smooth instead of a grid of
hard lines; the **buildings and props a map places** are decoded from the retail
compiled `W3DMesh` art (the uncompressed `worldbuilder.bin` stream and its
embedded DDS textures) and drawn with a depth test over the terrain. An
in-window **menu** lists the maps by their localized name and
exposes every render/skirmish option for tweaking before launch. The whole tree
builds for **Linux** (clang + libc++) and cross-compiles to **Windows**
(`openra3.exe` + `SDL3.dll`) with llvm-mingw — both using C++26 modules and
`import std;`.

```text
$ openra3 skirmish --game-dir "/game" --map map_mp_2_feasel4
OpenRA3 skirmish  map=map_mp_2_feasel4  source=archive:map_mp_2_feasel4  seed=1
start positions:
  P0 (1338, 1940)
  P1 (1291, 1404)
result: decided  winner=0  frames=9293 (309.8 s)
  P0 Commander  Allied  money=   700 units=15 kills=21 losses=15
  P1 AI         Soviet  money=   376 units= 0 kills=15 losses=21
```

## Offline only

OpenRA3 implements **offline skirmish and, eventually, LAN/single-player**.
There is deliberately **no online mode**: no EA account, no online service, no
matchmaking, no GameSpy/Steam integration, and no networking in the simulation.
The match loop is deterministic and self-contained so it can be replayed and
tested without any external service.

## Assets

The runtime reads a portable `assets/` folder **next to the executable** — there
is no `--game-dir` at run time. The build extracts it from your install:

- `openra3_assets` (build target) runs `cmake/extract_assets.cmake`, which
  - extracts every map (double-unwrapped `CkMp`) and terrain TGA into `assets/`
    via the engine's own `openra3 extract`;
  - with `-DOPENRA3_EXTRACT_ALL=ON` (default) also dumps **every** `.big` entry
    plus models, textures (`.png`), sound effects/voice and movie audio via the
    [`ra3-headless/ra3tools`](https://github.com/) scripts.
- CMake cache vars: `RA3_GAME_DIR` (default `C:/Red Alert 3`), `RA3TOOLS_DIR`
  (the `ra3tools` scripts), `PYTHON_EXECUTABLE`.
- The step is skipped when the exe cannot run on the build host (a Windows
  cross-build in a Linux container); the exe then extracts maps/terrain on first
  launch, and the full dump can be run explicitly:
  `cmake --build <build> --target openra3_assets` on a native host, or
  `openra3 extract --game-dir DIR --out DIR`.

**No game data is committed or distributed.** `reference/`, `assets/` and the
ra3tools checkout are git-ignored. Without an install the engine falls back to a
built-in test map so the project still builds and runs in CI.

## Layout

| Path | Purpose |
| --- | --- |
| `src/core/ra3.core.cppm` | fundamental types, math, strings, random, message stream |
| `src/logic/ra3.logic.cppm` | objects, players, teams, spatial partition, game loop |
| `src/client/ra3.client.cppm` | `display` abstraction + shared interactive loops + client facade |
| `src/display/ra3.display.cppm` | picks the backend (Vulkan/D3D11/D3D12/WebGL, then SDL) for the app |
| `src/game/ra3.game.cppm` | RA3 sides, player templates, skirmish defaults |
| `src/data/ra3.data.cppm` | retail RA3 balance: damage types, armour, weapons, units, economy |
| `src/fs/ra3.fs.cppm` | `BIG4` archives, RefPack codec, local install locator |
| `src/map/ra3.map.cppm` | map catalog, `EAR`/RefPack unwrap, start waypoints, `gamestrings.csf` display names |
| `src/skirmish/ra3.skirmish.cppm` | base building, economy, AI, combat, win condition |
| `src/terrain/ra3.terrain.cppm` | `CkMp` terrain: `HeightMapData`, `BlendTileData` (tiles + blends), `Terrain.big` tiles |
| `src/models/ra3.models.cppm` | compiled `W3DMesh` art (BAB stream + DDS textures) for the objects a map places |
| `src/render/ra3.render.cppm` | ARGB framebuffer, TGA decode, BMP encode, map compositing, bitmap-font text |
| `src/ui/ra3.ui.*.cppm` | SDL3 window viewer and menu (null backend when SDL3 is absent) |
| `src/vulkan/ra3.vulkan.*.cppm` | Vulkan presentation backend and menu (null fallback without a loader) |
| `src/dx/ra3.dx.*.cppm` | Direct3D 11/12 presentation backend (runtime HLSL; null fallback off Windows) |
| `src/wasmgl/ra3.wasmgl.*.cppm` | SDL-free wasm backend: engine on a Web Worker, WebGL2 on an OffscreenCanvas, lazy assets (null fallback off Emscripten) |
| `src/webgpu/ra3.webgpu.*.cppm` | WebGPU wasm backend (Dawn `emdawnwebgpu`, WGSL; the default on wasm, falls back to `ra3.wasmgl`) |
| `third_party/libenderlog/` | vendored C++26 module logger (`import ender.log;`, MIT) with a native stack fallback for libc++ |
| `src/ra3.cppm` | umbrella module re-exporting the SDK |
| `apps/openra3/main.cpp` | `menu` / `maps` / `skirmish` / `render` CLI |
| `tests/ra3_tests.cpp` | smoke tests (run via `ctest`) |
| `tools/` | reference fetch + Ghidra-driven reconstruction helpers |
| `docs/` | architecture & master plan, reverse-engineering notes |
| `Dockerfile` | Linux build image (`dev` toolchain + `deploy` runtime) |
| `Dockerfile.win` | isolated Windows cross-build image (llvm-mingw + SDL3 MinGW) |
| `cmake/toolchains/` | `llvm-mingw-x86_64.cmake` cross toolchain |
| `scripts/` | `build-linux.sh` / `build-windows.sh` one-shot builders |
| `.gitlab-ci.yml` | build both targets → test → package pipeline |

## Why Clang + `import std;`

The engine never `#include`s the standard library: every module does
`import std;`. CMake's support for that (`CXX_MODULE_STD`) works today with
Clang + libc++. Linux uses the distro clang (LLVM 21); Windows cross-compiles
with [llvm-mingw](https://github.com/mstorsjo/llvm-mingw) (clang 23 + libc++ +
the libc++ `std` module). The two toolchains live in **separate images** so
their compilers and standard libraries never interfere.

## Build

The host needs no toolchain: each target builds inside its own image.

```bash
# Linux (amd64)   -> build/linux/bin/openra3 + .deb
scripts/build-linux.sh

# Linux/arm64     -> build/linux-arm64/bin/openra3 + .deb   (cross)
scripts/build-linux-arm64.sh

# Debian (amd64)  -> build/debian/bin/openra3 + .deb
scripts/build-debian.sh

# Debian (arm64)  -> build/debian-arm64/bin/openra3 + .deb   (cross)
scripts/build-debian-arm64.sh

# Windows (x86_64) -> build/windows/bin/openra3.exe + SDL3.dll, .zip, .msi
scripts/build-windows.sh

# Windows (arm64)  -> build/windows-arm64/bin/openra3.exe + SDL3.dll, .zip   (cross)
scripts/build-windows-arm64.sh

# WebAssembly      -> build/wasm/bin/index.html (+ openra3.js/.wasm, openra3.worker.js)
scripts/build-wasm.sh
```

Every target is cross-built from x86_64 Linux in its own image: llvm-mingw for
the Windows targets, clang + libc++ with dpkg multiarch for the arm64 targets,
and Debian/Ubuntu-specific images for the `.deb` packages.

Or drive Docker directly:

```bash
docker build --target dev -t openra3-linux:local .
docker run --rm -v "$PWD:/work" -w /work openra3-linux:local \
    cmake -S . -B build/linux -G Ninja -DCMAKE_BUILD_TYPE=Release
docker run --rm -v "$PWD:/work" -w /work openra3-linux:local cmake --build build/linux -j
docker run --rm -v "$PWD:/work" -w /work openra3-linux:local ctest --test-dir build/linux --output-on-failure

docker build -f Dockerfile.win --target dev -t openra3-windows:local .
docker run --rm -v "$PWD:/work" -w /work openra3-windows:local \
    cmake -S . -B build/windows -G Ninja \
        -DCMAKE_TOOLCHAIN_FILE=cmake/toolchains/llvm-mingw-x86_64.cmake \
        -DCMAKE_BUILD_TYPE=Release
docker run --rm -v "$PWD:/work" -w /work openra3-windows:local cmake --build build/windows -j
```

Behind a slow or blocked mirror, pass `--build-arg APT_MIRROR=<url>`.

The two images are deliberately separate so the Linux (clang + libc++) and
Windows (llvm-mingw + MinGW SDL3) toolchains never interfere. The Windows demo
is `build/windows/bin/openra3.exe` next to `SDL3.dll`:

```powershell
# show a real map in a Vulkan window
openra3.exe render --game-dir "C:\Red Alert 3" --vulkan
```

Vulkan is provided by **vendored volk + headers** (`third_party/`), resolved at
runtime, so neither image needs a Vulkan SDK.

## WebAssembly

OpenRA3 also builds to **WebAssembly** with Emscripten (`scripts/build-wasm.sh`),
producing `openra3.js` + `.wasm` plus the `index.html` page and
`openra3.worker.js` bootstrap. The browser gets the same GPU renderer as the
desktop builds: the engine runs on a **Web Worker** (no SDL, no blocking of the
page) and draws into an **OffscreenCanvas** handed over by the page. The default
wasm backend is **`ra3.webgpu`** (WebGPU, WGSL shaders via Emscripten's
`emdawnwebgpu` port); it falls back to **`ra3.wasmgl`** (WebGL2, the GLSL ES
ports of the desktop shaders) when WebGPU is unavailable. The page keeps the DOM
(input, resize) and forwards events to the worker, so the main thread stays
responsive.

Because the runtime's main thread lives in the worker, its filesystem is local
to it and assets load **on demand**: point `OPENRA3_WEB_ASSETS` at the extracted
asset tree and the build embeds an `assets.manifest.json`; at startup the worker
registers every listed file as a lazy file, so the browser fetches a file only
when the engine first opens it (entering a map pulls just that map and the tiles
it uses, not the whole tree).

```bash
# build (generates the manifest from the extracted assets, embeds it)
OPENRA3_WEB_ASSETS=/path/to/assets scripts/build-wasm.sh

# serve the build output (Range-capable, COOP/COEP; --assets is the lazy tree)
python3 apps/web/serve.py --root build/wasm/bin --assets /path/to/assets
# open http://localhost:8199/index.html
```

The engine's frame loops are blocking; the wasm build yields to the worker's
event loop via Asyncify (`display::sleep_frame` calls `emscripten_sleep`), so
the page repaints and handles input. Log records go to `console.log` at their
real level (the console sink uses stdout on the web instead of stderr). Without
`OPENRA3_WEB_ASSETS` the module still loads and starts, but exits at the menu
because no maps are found.

## Packages

CPack produces the release artifacts; the Docker images and CI run it for every
target.

| Target | Portable | Installer |
| --- | --- | --- |
| Linux amd64 (Ubuntu 26.04) | `.tar.gz` | `openra3_<v>_amd64ubuntu26.04.deb` |
| Linux arm64 (Ubuntu 26.04) | `.tar.gz` | `openra3_<v>_arm64ubuntu26.04.deb` |
| Linux amd64 (Debian 13) | `.tar.gz` | `openra3_<v>_amd64debian13.deb` |
| Linux arm64 (Debian 13) | `.tar.gz` | `openra3_<v>_arm64debian13.deb` |
| Windows x86_64 | `openra3-<v>-windows-x86_64.zip` | `openra3-<v>-windows-x86_64.msi` |
| Windows arm64 | `openra3-<v>-windows-arm64.zip` | (WiX-only; see below) |

The `.deb` dependencies are derived from the binary with `dpkg-shlibdeps`. The
`.msi` is built with **wixl** (msitools) straight from the Linux cross image;
wixl 0.106 has no arm64 support, so Windows/ARM64 ships the portable `.zip`
(the MSI toolchain for ARM64 would be WiX v4 via the .NET SDK).

## Logging

Every run writes `openra3.log` in a per-user `logs/` folder — on Windows
`%LOCALAPPDATA%\OpenRA3\logs`, elsewhere `$XDG_STATE_HOME/openra3/logs`
(falling back to `~/.local/state/openra3/logs`) — created on first use and kept
out of the binary's own directory, through the vendored
[`libenderlog`](third_party/libenderlog) module (`import ender.log;`). A file
sink archives the previous log to `openra3.<YYYYmmdd-HHMMSS>.log` on open, so
each run gets its own file; the active file rotates at 4 MiB and the last 10
archives are kept. Records at **`warn` and above** carry a call stack (Windows
`CaptureStackBackTrace` / POSIX `execinfo`, because libc++ has no
`<stacktrace>`). A hard crash also writes `openra3_crash.log` alongside it with
the faulting module and a raw backtrace.

## Running a skirmish

```bash
# list the maps in your install
docker run --rm -v "/path/to/Red Alert 3:/game:ro" openra3-linux:local \
    /work/build/linux/bin/openra3 maps --game-dir /game

# play a headless skirmish on a real map
docker run --rm -v "/path/to/Red Alert 3:/game:ro" openra3-linux:local \
    /work/build/linux/bin/openra3 skirmish --game-dir /game --map map_mp_2_feasel4 --seed 7
```

The Windows build is a native `openra3.exe` — copy it next to `SDL3.dll` and run
it from `cmd`/PowerShell, e.g. `openra3.exe skirmish --game-dir "C:\Red Alert 3"`.

`--frames N` caps the simulation length (default 15 minutes of game time at
30 Hz). The result is deterministic for a given map and seed.

## Menu

Run `openra3` with no arguments (or `openra3 menu`) to open a window listing the
maps by their **localized display name** (read from the install's
`gamestrings.csf`, e.g. `map_mp_2_feasel4` → "Battlebase Beta") with their id
below. The `Options` column exposes every `render`/`skirmish` parameter — mode
(3D terrain / top-down / skirmish), window size, camera pitch/yaw/height, FOV,
zoom, terrain scale, terrain pitch, world size, seed, frame cap, the overview
thumbnail toggle and a BMP output path. `Up`/`Down` selects, `Left`/`Right`
changes a value (or moves the text caret on the BMP field), `Tab` switches
between the map list and the options, `Enter` starts and `Esc` quits. Starting a
3D or top-down view opens the viewer; closing it returns to the menu. When no
window backend is available the same flow falls back to a console picker, and
`openra3 menu-preview` renders one menu frame to a BMP for inspection.

## Rendering the map

`render` draws the **real terrain**: it parses the map's `HeightMapData`
(elevation grid) and `BlendTileData` (per-cell tile index plus the per-cell
`Blends`/`ThreeWayBlends` and their `BlendDescription`s), loads the tile
textures from `Data\Terrain.big` (RefPack + TGA), and rasterises the map with an
elevation shade. Each cell cross-fades into its blend neighbour with the same
linear ramp the retail `Terrain.fx` uses, so material transitions are smooth; on
the GPU path the tile atlas is padded with a replicated gutter so filtering
never bleeds between tiles. It also draws the **buildings and props the map
places**: the `ObjectsList` chunk is decoded into `(type, position, angle)`
instances, the type resolves to its compiled `W3DMesh` parts in the map's art
stream (`Data\WBData.big`'s uncompressed `worldbuilder.bin`), the embedded DDS
textures are decoded, bind-pose skinning places each mesh's bone-space vertices
(`W3DHierarchy`), and the scene is flattened into one world-space triangle soup.
The terrain raymarcher writes a depth value so the model pass depth-tests
against the relief; the GPU path draws it in a second pipeline (shared camera
basis, with a small depth bias so ground decals do not z-fight) and the software
path rasterises it with a z-buffer. Match state is
overlaid (start markers in yellow, player 0 in blue, player 1 in red).
`--thumbnail` uses the old `<map>_art.tga` overview instead.

> **Roads / sidewalks** are drawn too: the map stores each as a
> `RoadType::Start`/`End` pair of objects, and `render` emits a flat textured
> ribbon along each segment, lifted onto the terrain with a small depth bias so
> it does not clip into the ground.

**Offscreen image** (works anywhere, no display needed):

```bash
docker run --rm -v "/path/to/Red Alert 3:/game:ro" -v "$PWD/out:/out" openra3-linux:local \
    /work/build/linux/bin/openra3 render --game-dir /game --map map_mp_2_feasel4 --out /out/map.bmp
```

On Windows the same command runs natively: `openra3.exe render --game-dir
"C:\Red Alert 3" --map map_mp_2_feasel4` opens an SDL3 window.

**Interactive window** (Vulkan with `--vulkan`, otherwise SDL3). The camera
follows the retail tactical view: it opens centred on the first player's start,
the wheel zooms, pushing the cursor against a screen edge scrolls, dragging with
the left button pans, and Esc quits. `--zoom Z` sets the initial zoom (1 fits
the whole map). In a container you need an X server on the host — on Windows run
[VcXsrv](https://sourceforge.net/projects/vcxsrv/) and launch it with "Disable
access control", then:

```bash
docker run --rm -v "/path/to/Red Alert 3:/game:ro" -e DISPLAY=host.docker.internal:0.0 \
    openra3-linux:local /work/build/linux/bin/openra3 render --game-dir /game --map map_mp_2_feasel4
```

If no display is available the viewer falls back to writing `openra3_view.bmp`.
The unit overlay uses an approximate world scale (`--world-size`, default
5120); exact calibration from the map's heightmap is on the roadmap.

## Reverse engineering

The reconstruction is driven by Ghidra against the retail binary. See
[`docs/REVERSE_ENGINEERING.md`](docs/REVERSE_ENGINEERING.md) for the workflow and
the recovered symbol map, and [`tools/`](tools/) for the helpers.

## License

Project code: GPLv3, matching the `CnC_Generals_Zero_Hour` reference (see
[`LICENSE`](LICENSE)).