# Reverse engineering workflow OpenRA3 is reconstructed from two sources: 1. **Architecture** — the GPLv3 SAGE 1.0 tree, [`electronicarts/CnC_Generals_Zero_Hour`](https://github.com/electronicarts/CnC_Generals_Zero_Hour). RA3 runs SAGE 2.0, so class names, message flow and subsystem boundaries carry over even though the code does not. 2. **Facts** — the retail `ra3_1.12.game` binary (image base `0x400000`), analysed in Ghidra and, when needed, observed live. Nothing in OpenRA3 should assert a structure or constant that is not either copied from the reference or cited to a retail address. ## Fetching the reference ```bash tools/fetch_reference.sh # sparse-clone Code/GameEngine into reference/ ``` The checkout is git-ignored (large, and GPLv3 terms differ from this repo's). ## Ghidra The retail module is loaded into Ghidra as `ra3_1.12.game` (`x86:LE:32:default`, 34k+ functions). The analysis is driven through the Ghidra MCP bridge, so every recovered fact can be re-derived: | Question | Tool call | | --- | --- | | What does an address do? | `decompile_function(address=0x…)` | | What is this function? | `get_function_by_address(address=0x…)` | | Who touches a global? | `get_xrefs_to(address=0x…)` | | What are the vtable slots? | `list_class_members` / `analyze_data_region` | | Where is a string referenced? | `search_strings` + `get_xrefs_to` | ### Recovery loop 1. Pick a subsystem from the reference tree (e.g. `MessageStream`). 2. Find its RTTI/vtable in the binary via `search_strings` and xrefs. 3. Decompile the constructor to recover object size and field init order. 4. Decompile the hot methods to recover field meaning. 5. Write the OpenRA3 module with a comment citing the address. 6. Add a smoke test that pins the behaviour. ## Recovered symbol map (retail `ra3_1.12.game`) ### Engine singletons Each holds an object pointer (0 when the subsystem is down). | Global | Address | Object vtable | | --- | --- | --- | | `TheGameLogic` | `0x00cd8ce4` | `0x00beb630` | | `ThePlayerList` | `0x00ce8c9c` | `0x00c5b9e0` | | `ThePartitionManager` | `0x00ce2f9c` | `0x00c6af98` | | `TheShroudManager` | `0x00ce2fa0` | `0x00c6aefc` | | `ThePlacementGrid` | `0x00cd8d0c` | `0x00bea560` | | `TheRecorder` | `0x00ce2fd0` | `0x00c10544` | | `TheGlobalObjectRegistry` | `0x00cd8d08` | `0x00bea644` | | `ThePlayerTemplateStore` | `0x00ce8ca0` | `0x00c5bc70` | | `TheGameState` | `0x00cdbbc4` | `0x00bef0c0` | | `GlobalData` | `0x00ce2fa8` | `0x00c0d8e4` | | `TheMessageStream` | `0x00ce2fb8` | `0x00c0ecd4` | ### Reconstructed layouts - **`GameLogic`** — tick counter at `+0x50` (incremented once per 30 Hz simulation step); `starting` guard flag at `+0xa7`, raised while a new match initialises. - **`MessageStream`** — intrusive doubly linked list; head at `+0x24`, tail at `+0x28`. `appendMessage` (`0x0060c4a0`) allocates a `0x74`-byte node: `+0x00` next, `+0x04` prev, `+0x08` owner stream, `+0x0c` message type, `+0x10` player index, `+0x18` capacity, `+0x1c` data pointer (`node + 0x20`). - **`Player`** — money via `std::vector` at `+0xe4`; power at `+0x74`; team at `+0xac`; relation maps at `+0xfc` / `+0x100`. - **Skirmish setup** (`SkirmishGameInfo`, pointer at `[0x00ce3a78]`) — starting cash `+0x64`, player slots `+0xfc` (stride `0x5c`, 6 slots), faction at `slot + 0x18` (`Empire=2`, `Allied=4`, `Soviet=8`, `Random=7`). ### Tactical view (camera) The in-game camera is the `TheTacticalView` object, held in the global at `0x00cdb7b4`. Its vtable accessors return zoom, pitch (current/target), yaw, world position and FOV; the debug overlay `FUN_005ef0a0` prints them through the format string at `0x00c0b900`. Mouse state is the singleton at `0x00ce9284` (cursor position vtable slot `+0x3c`, button down `+0x48`); the keyboard manager is `0x00ce927c` (modifier mask `+0x38`). Per-map camera tuning is a named-field table in `.rdata` (around `0x00c11a54`): `cameraMinHeight`, `cameraMaxHeight`, `cameraPitchAngle`, `cameraYawAngle`, `cameraScrollSpeedScalar`, `cameraGroundMinHeight`, `cameraGroundMaxHeight`. The map-load chunk `CHUNK_TacticalView` (`0x00beea14`, consumed near `0x00548a00`) seeds the view from the map; the tutorial actions `LOCK_CAMERA_SCROLL` / `LOCK_CAMERA_ZOOM` / `LOCK_CAMERA_ROTATION` gate the controls. OpenRA3 has no 3D terrain yet, so `ra3::render::view_camera` reproduces the *controls* - wheel zoom, screen-edge scroll, clamped pan and opening on the player's start - over the 2D map overview, not the retail perspective camera. ### Match start The BEGIN button calls `SkirmishGameOptionsMenu::start` (`0x00b28d60`), which copies the map, calls `GameInfo::startGame(0)`, seeds the logic random and appends `MSG_NEW_GAME` (`0x2`). `startNewGame` itself is `0x00623e40`. OpenRA3 mirrors this two-phase `prepare_new_game` / `start_new_game` split. ## Container and map formats Recovered by inspection of `Data\*.big` (see `ra3.fs` / `ra3.map`). ### `BIG4` archive All integers little-endian except where noted: ``` offset 0 magic "BIG4" offset 4 fileSize u32 LE total archive size offset 8 fileCount u32 BE number of entries offset 12 indexSize u32 offset 16 entries fileCount * { offset u32 BE, size u32 BE, name cstring } ``` Entry offsets are absolute; payloads are RefPack-compressed. ### RefPack EA's `10 FB` stream. `ra3.fs::refpack_decompress` implements the 2/3/4-byte commands and the long-literal/stop opcodes; `refpack_output_size` reads the declared output length without decompressing. ### Map file (`.map`) Two layers of compression: ``` BIG4 payload = RefPack -> "EAR\0" + u32 unpacked_size + RefPack -> "CkMp" ... ``` The `CkMp` payload is the compiled SAGE map: a type/field name table followed by chunk data. Player start positions appear as waypoints named `Player_1_Start`, `Player_2_Start`, ... Each waypoint record carries a `Coord3D` (three little-endian `float`s) shortly after the name; `ra3.map` scans forward from the name for the first plausible `(x, y, 0)` triple. Maps that store starts in `MPPositionList` instead yield no waypoints and fall back. Verified example (`map_mp_2_feasel4`): `Player_1_Start` = `(1338.9, 1940.5, 0)`, `Player_2_Start` = `(1290.8, 1404.9, 0)`. ### Terrain (`ra3.terrain`) The `CkMp` tree is a flat chunk list: `"CkMp"`, `u32 assetCount`, the name table (`{ u8 len, name, u32 index }`, index descending from `assetCount`), then `{ u32 index, u16 version, u32 size, data }` per chunk. On `map_mp_2_feasel4` the terrain is `HeightMapData` v6 (540 x 600, border 20, `u16` elevations, scale `0.0390625`) and `BlendTileData` v27. `BlendTileData` opens with `NumTiles`, the `u16` tile grid, then the blend/three-way/cliff tables (`u16` for v27); the passability flag arrays that follow are not needed for rendering, so the texture table is located by scanning for its `{ cellStart, cellCount, cellSize, magic }` + `u16`-prefixed name entries. A tile value is `(cellIndex << 2) | variant`, and `cellIndex` indexes the global `TextureCellCount`-cell table (each texture owning `cellSize^2` 64 px cells). The textures themselves are `art\terrain\.tga` in `Terrain.big` / `Core11.big` (RefPack + 256x256 TGA). Rendered top-down, `map_mp_2_feasel4` correlates 0.94 with the official `_art.tga` overview. ### Terrain blending (`BlendTileData` tail) After the tile grid, `BlendTileData` stores three per-cell `u16` tables — `Blends`, `ThreeWayBlends` and `CliffTextures` — then `TextureCellCount`, `BlendsCount`, the texture table, two magic words and `BlendsCount - 1` blend descriptions (18 bytes each: `u32 secondaryTile`, four direction bytes, `u8 flags`, `u8 twoSided`, `u32 0xFFFFFFFF`, `u32 0x7ADA0000`). A non-zero `Blends[cell]` is a 1-based index into the descriptions; `secondaryTile` is a packed tile value (`secondaryTile >> 2` is its cell). The retail `Terrain.fx` (compiled `terrain.fxo`, parameters `Terrain.BaseTexture`, `Terrain.MacroTexture`, `MapCellSize`, `IsTerrainAtlasEnabled`; technique `TerrainTile`) cross-fades a cell's base tile into `secondaryTile` with a linear ramp selected by `BlendDirection`: `1` right, `2` top, `4` top-right, `8` top-left, where `flags` bit 0 flips the axis and bit 1 marks a two-sided diagonal. OpenSAGE's `Terrain.frag` reconstructs the exact `CalculateBlendFactor`; `ra3::terrain::blend_factor` and `shaders/terrain.frag` mirror it. The row axis is *not* inverted (73% of long-axis blends point at a neighbour of the same texture, versus 25% inverted). On `map_mp_2_feasel4`, 33828 of 324000 cells carry a blend and there are 7094 descriptions. The GPU atlas pads every 64 px tile with a 2-texel replicated gutter. Packed edge-to-edge, bilinear/mipmap filtering averaged two unrelated tiles at every cell border — that cross-tile bleed was the visible grid line the hardware path drew. (`shaders/terrain.frag` samples `cell_stride = cell_texels + 2 * gutter`.) ### Open question: per-cell tile sampling Measured on `map_mp_2_feasel4`, sampling each cell as its own 64 px block and restarting the UV every cell leaves a 1.39x edge spike at cell boundaries (43.4 vs 31.1 mean gradient at 8 px/cell). Two candidate mappings reduce it and need a visual decision against the retail art: | Mapping | Boundary/interior | | --- | --- | | per-cell 64 px block (current) | 1.39 | | OpenSAGE `BlendTileTextureIndex` Morton layout, 32 px block | 1.18 | | continuous `uv / (cellSize * 2)`, per OpenSAGE `Terrain.frag` | 1.11 | The retail `Terrain.frag` (OpenSAGE) samples the tile texture *continuously* (`uv / (CellSize * 2)`), so adjacent cells never restart the texture; our per-cell restart is the remaining source of grid lines. Switching the atlas from 64 px blocks to continuous 32 px regions (or the Morton 8x8 layout) is the next step, pending an art correlation check. ### Compiled art (BinaryAssetBuilder) and map objects Retail RA3 ships no `.w3x`/`.w3d` files: BinaryAssetBuilder bakes every model, texture and script into a *binary asset stream* — a `.manifest` index plus a `.bin` of relocatable instance data (and optional `.relo`/`.imp` fixups). The layout (little-endian) is: ``` ManifestHeader (48 B) isBigEndian u8, isLinked u8, version u16, streamChecksum, allTypesHash, assetCount u32, totalInstanceDataSize, maxInstance/maxRelocation/ maxImportsChunkSize, assetReferenceBufferSize, referenceManifestNameBufferSize, assetNameBufferSize, sourceFileNameBufferSize AssetEntry (48 B) * count typeId, instanceId, typeHash, instanceHash, assetReferenceOffset i32, assetReferenceCount i32, nameOffset i32, sourceFileNameOffset i32, instanceDataSize i32, relocationDataSize i32, importsDataSize i32, tokenized u32 then the reference / referenced-name / asset-name / source-name buffers ``` Asset names are `Type:Instance` (e.g. `W3DMesh:BB_GRASS02`). Instance pointers are stored as offsets from the start of the instance data (which begins at byte 4 of `.bin`, after the stream checksum), so a slice is readable without the relocation stream. Each multiplayer map carries its own stream (`data\maps\official\\map.bin`) but it is **linked**: only map-specific assets (the terrain texture atlas, scripts, `GameMap`) have data; the rendered props are imported and therefore have `instanceDataSize == 0`. The complete prop art (meshes + textures) lives in `Data\WBData.big`'s `data\worldbuilder.bin`, which is **uncompressed** (first four bytes are the stream checksum, not `10 FB`), so `ra3.models` reads the 1.1 GB stream lazily — the manifest is parsed and only the needed instance slices are read. `W3DMesh` compiled layout (offsets from the instance start): ``` +4 vertexBufferPtr +52 triangleCount +56 triangleItemPtr +60 shaderNameLength +64 shaderNamePtr vertexBuffer: +0 numVertices, +4 stride, +8 elementDataPtr, +12 declarationBytes, +16 declarationPtr declaration: text "p0:00:3f32 n0:0C:3f32 t0:1C:2f32" (D3D9 usage:index:offset:type) triangles: triangleCount * { u32 indexCount, u32 indexPtr } (24 B each), u32 indices ``` The diffuse texture is found through the mesh's `FXShaderConstant`s (`+76` count, `+80` items): a texture-valued constant (TypeId `0xA59096A6`) names its role (`DiffuseTexture`, `NormalMap`, `SpecMap`) and points at a 1-based index into the mesh's cross-asset references, which resolve by `(typeId, instanceId)`. The `Texture` instance embeds a standard DDS file (at `u32@+4`, or scan for `"DDS "`); `ra3.models::decode_dds` decodes DXT1/3/5 and uncompressed 16/24/32-bit. **Vertices are stored in bone space, not object space.** A mesh whose vertex declaration carries blend data (`i0:..:4u8 w0:..:4u8n`, e.g. buildings and vehicles) must be skinned; props without it (`BB_GRASS02`, `IF_STREETSEGMENT01`) are already in object space. The skeleton is a `W3DHierarchy` asset named after the mesh's instance prefix (`W3DMesh:FI_STRUCTURE_02.NEWSKIN_CIV01` → `W3DHierarchy:FI_STRUCTURE_02`). Compiled layout: ``` W3DHierarchy: u32 pad, u32 boneCount, u32 headerBytes, then boneCount records 100 B each: u32 nameHash, i32 parent (-1 = root), f32 translation[3], f32 quaternion[4] (x, y, z, w), f32 matrix[12] ``` The default (bind) pose is rebuilt by composing each bone's local translation/quaternion down the parent chain, then `skinnedPos = Σ wᵢ · (Rᵢ·p + Tᵢ)` (and the normal by the rotation only). `ra3.models` does this before placing the mesh at the map object's `(x, y, angle)`. The compiled shader (below) binds **one joint per vertex** — `WorldBones` holds 64 bones as 2 `float4` each (quaternion `c[128+2j]`, translation `c[129+2j]`) — so the skin is rigid: `blendindices.x` selects the joint, remapped through the mesh's per-model **bone table** (vertex-descriptor `+0x14` = bone count, `+0x18` = `u16` bone indices into the `W3DHierarchy`). Applying the raw blend index without that remap tears models apart (`FI_BUILDING01`'s table is `[0,14,15,16,17,18]`, not `[0..5]`). The map objects that lie flat on the ground (sidewalks, deck pieces) are coplanar with the terrain; retail biases their depth in the shader so they do not z-fight. The object pass reproduces that with a small negative depth bias (Vulkan `depthBiasConstant/SlopeFactor`, and a half-unit bias in the software rasteriser). Meshes whose material has no diffuse texture (`DefaultW3D.fx`, `BasicW3D.fx` — `FXLIGHTS`/ambient helper billboards) are not opaque geometry and are skipped; drawing them fills the frame with garbage triangles. Likewise the `BuildingsGenericDamageFill.fx` **damage-fill** sub-meshes are skipped: they are the wrecked-interior shell (e.g. `CBBuilding_Wood`, an orange plank texture) that retail only reveals through damage holes, but our opaque pass would paint it over the main shell and tint whole buildings warm. ### Official shader behaviour (`Shaders.big` → `*.fxo`) The compiled D3D9 effects in `Data\Shaders.big` name their parameters, so the model pipeline is recoverable. `buildingsgeneric.fxo` (`BuildingsGeneric.fx`) vertex stage: skinning (above), `World`/`ViewProjection`, and vertex color `c0` multiplied into the lit color (`(Ambient·AmbientColor + Σ DLᵢ.Color·max(N·DLᵢ,0)) · DiffuseColor · vertexColor`). Pixel stage samples `DiffuseTexture`/`NormalMap`/`SpecMap`/`DamagedTexture`/ `CloudTexture` (all at **UV0**, except `DamagedTexture` at `v0.wz` = transposed UV1), then `final.rgb *= TintColor` and `*= ShroudTexture.rgb`. `DiffuseVelocity` is not used for static structures. So the diffuse texture is UV0 and is tinted by vertex color and `TintColor`; `basicw3d.fxo` instead modulates a single macro/lightmap with `(vertexColor + additive) * diffuse * 2` and has no normal map. (Recovered by disassembling the embedded `vs_3_0`/`ps_3_0` bytecode.) The map's `ObjectsList` chunk is a list of nested `Object` assets — `Coord3D`, Z `angle`, `RoadType` u32, a `u16`-prefixed type-name and a property list whose keys index the shared name table (`ra3.map::parse_objects`). Each type resolves to the `W3DMesh` parts whose instance name equals it or starts with `.`. ### Roads / sidewalks (`Road` assets) The flat sidewalk and road decals are not meshes: the map places them as **consecutive pairs** of objects — a `RoadType::Start` (`2`) and a `RoadType::End` (`4`) at the segment's two ends (`Angled` `8`, `TightCurve` `64` and `EndCap` `128` are curve/cap flags; `BridgeStart/End` `16/32` are bridges). The road's type-name (`IslandFortressSidewalk01`, ...) resolves to a compiled `Road` asset in the same art stream: ``` Road: u32 version, u32 pad, u32 textureCount, f32 roadWidth, f32 pad, f32 ???, then textureCount texture references (diffuse, normal) ``` `ra3.models` pairs the objects in file order, emits a flat quad ribbon of `roadWidth` along each segment (overlapping the joins by half a width), sampled with the diffuse texture, and lifts it onto the terrain. Retail gives roads a small depth offset toward the screen so they do not clip into the ground; the object pass reproduces that with the same negative depth bias used for the other ground decals (`depthBiasConstant/SlopeFactor` on Vulkan, a half-unit bias in the software rasteriser). ### Map display names The skirmish map list labels live in `Data\English.big`'s `data\gamestrings.csf` (the newest `Lang-English*.big` wins) under `MAP:`, e.g. `MAP:MAP_MP_2_FEASEL4` = "Battlebase Beta". CSF values are UTF-16 code units whose low byte is XORed with `0xFF` (`ra3::map::parse_map_names`). `openra3 extract` writes the decoded table to `maps/map_names.tsv`; `openra3 menu` shows them instead of the raw map id. ### Still to recover - `MPPositionList` layout (per-player starts for maps without waypoints). - Cliff textures and the `CliffTextureMapping` UV remap (`CliffTextures` is parsed but not yet drawn). - The `Road` network mesher (the map's sidewalk/road objects reference `Road` templates, not `W3DMesh` assets). - W3D container/hierarchy assembly and animation (props are drawn as their static mesh parts; skinned/animated in-game models are not). - Compiled asset blobs (`global.bin`, `static.*.bin`) and the `.manifest` schema used to deserialise them.