EnderTheCoder
b6d619254d
- ra3.client::display: shared interactive loops; SDL and Vulkan backends implement only the primitives (init/present/poll_event/window_size/ key_down/present_terrain). ra3.display picks the backend. - Menu: maps by localized name (gamestrings.csf), red/gold theme, hover highlight, mouse + keyboard, wheel scroll, fullscreen and FPS/vsync options, loading progress bar. - Terrain: continuous tile sampling via a texture array (REPEAT, uv = cell/(2*cellSize)) removes per-cell grid seams; SAGE blend ramp for material transitions; FPS label + top-right minimap overlays. - Skip the skirmish sim for map views; reuse the Vulkan texture; no idle terrain redraw.
232 lines
10 KiB
Markdown
232 lines
10 KiB
Markdown
# Reverse engineering workflow
|
|
|
|
OpenRA3 is reconstructed from two sources:
|
|
|
|
1. **Architecture** — the GPLv3 SAGE 1.0 tree,
|
|
[`electronicarts/CnC_Generals_Zero_Hour`](https://github.com/electronicarts/CnC_Generals_Zero_Hour).
|
|
RA3 runs SAGE 2.0, so class names, message flow and subsystem boundaries
|
|
carry over even though the code does not.
|
|
2. **Facts** — the retail `ra3_1.12.game` binary (image base `0x400000`),
|
|
analysed in Ghidra and, when needed, observed live.
|
|
|
|
Nothing in OpenRA3 should assert a structure or constant that is not either
|
|
copied from the reference or cited to a retail address.
|
|
|
|
## Fetching the reference
|
|
|
|
```bash
|
|
tools/fetch_reference.sh # sparse-clone Code/GameEngine into reference/
|
|
```
|
|
|
|
The checkout is git-ignored (large, and GPLv3 terms differ from this repo's).
|
|
|
|
## Ghidra
|
|
|
|
The retail module is loaded into Ghidra as `ra3_1.12.game`
|
|
(`x86:LE:32:default`, 34k+ functions). The analysis is driven through the
|
|
Ghidra MCP bridge, so every recovered fact can be re-derived:
|
|
|
|
| Question | Tool call |
|
|
| --- | --- |
|
|
| What does an address do? | `decompile_function(address=0x…)` |
|
|
| What is this function? | `get_function_by_address(address=0x…)` |
|
|
| Who touches a global? | `get_xrefs_to(address=0x…)` |
|
|
| What are the vtable slots? | `list_class_members` / `analyze_data_region` |
|
|
| Where is a string referenced? | `search_strings` + `get_xrefs_to` |
|
|
|
|
### Recovery loop
|
|
|
|
1. Pick a subsystem from the reference tree (e.g. `MessageStream`).
|
|
2. Find its RTTI/vtable in the binary via `search_strings` and xrefs.
|
|
3. Decompile the constructor to recover object size and field init order.
|
|
4. Decompile the hot methods to recover field meaning.
|
|
5. Write the OpenRA3 module with a comment citing the address.
|
|
6. Add a smoke test that pins the behaviour.
|
|
|
|
## Recovered symbol map (retail `ra3_1.12.game`)
|
|
|
|
### Engine singletons
|
|
|
|
Each holds an object pointer (0 when the subsystem is down).
|
|
|
|
| Global | Address | Object vtable |
|
|
| --- | --- | --- |
|
|
| `TheGameLogic` | `0x00cd8ce4` | `0x00beb630` |
|
|
| `ThePlayerList` | `0x00ce8c9c` | `0x00c5b9e0` |
|
|
| `ThePartitionManager` | `0x00ce2f9c` | `0x00c6af98` |
|
|
| `TheShroudManager` | `0x00ce2fa0` | `0x00c6aefc` |
|
|
| `ThePlacementGrid` | `0x00cd8d0c` | `0x00bea560` |
|
|
| `TheRecorder` | `0x00ce2fd0` | `0x00c10544` |
|
|
| `TheGlobalObjectRegistry` | `0x00cd8d08` | `0x00bea644` |
|
|
| `ThePlayerTemplateStore` | `0x00ce8ca0` | `0x00c5bc70` |
|
|
| `TheGameState` | `0x00cdbbc4` | `0x00bef0c0` |
|
|
| `GlobalData` | `0x00ce2fa8` | `0x00c0d8e4` |
|
|
| `TheMessageStream` | `0x00ce2fb8` | `0x00c0ecd4` |
|
|
|
|
### Reconstructed layouts
|
|
|
|
- **`GameLogic`** — tick counter at `+0x50` (incremented once per 30 Hz
|
|
simulation step); `starting` guard flag at `+0xa7`, raised while a new match
|
|
initialises.
|
|
- **`MessageStream`** — intrusive doubly linked list; head at `+0x24`, tail at
|
|
`+0x28`. `appendMessage` (`0x0060c4a0`) allocates a `0x74`-byte node:
|
|
`+0x00` next, `+0x04` prev, `+0x08` owner stream, `+0x0c` message type,
|
|
`+0x10` player index, `+0x18` capacity, `+0x1c` data pointer (`node + 0x20`).
|
|
- **`Player`** — money via `std::vector<Money*>` at `+0xe4`; power at `+0x74`;
|
|
team at `+0xac`; relation maps at `+0xfc` / `+0x100`.
|
|
- **Skirmish setup** (`SkirmishGameInfo`, pointer at `[0x00ce3a78]`) — starting
|
|
cash `+0x64`, player slots `+0xfc` (stride `0x5c`, 6 slots), faction at
|
|
`slot + 0x18` (`Empire=2`, `Allied=4`, `Soviet=8`, `Random=7`).
|
|
|
|
### Tactical view (camera)
|
|
|
|
The in-game camera is the `TheTacticalView` object, held in the global at
|
|
`0x00cdb7b4`. Its vtable accessors return zoom, pitch (current/target), yaw,
|
|
world position and FOV; the debug overlay `FUN_005ef0a0` prints them through the
|
|
format string at `0x00c0b900`. Mouse state is the singleton at `0x00ce9284`
|
|
(cursor position vtable slot `+0x3c`, button down `+0x48`); the keyboard manager
|
|
is `0x00ce927c` (modifier mask `+0x38`).
|
|
|
|
Per-map camera tuning is a named-field table in `.rdata` (around
|
|
`0x00c11a54`): `cameraMinHeight`, `cameraMaxHeight`, `cameraPitchAngle`,
|
|
`cameraYawAngle`, `cameraScrollSpeedScalar`, `cameraGroundMinHeight`,
|
|
`cameraGroundMaxHeight`. The map-load chunk `CHUNK_TacticalView`
|
|
(`0x00beea14`, consumed near `0x00548a00`) seeds the view from the map; the
|
|
tutorial actions `LOCK_CAMERA_SCROLL` / `LOCK_CAMERA_ZOOM` / `LOCK_CAMERA_ROTATION`
|
|
gate the controls.
|
|
|
|
OpenRA3 has no 3D terrain yet, so `ra3::render::view_camera` reproduces the
|
|
*controls* - wheel zoom, screen-edge scroll, clamped pan and opening on the
|
|
player's start - over the 2D map overview, not the retail perspective camera.
|
|
|
|
### Match start
|
|
|
|
The BEGIN button calls `SkirmishGameOptionsMenu::start` (`0x00b28d60`), which
|
|
copies the map, calls `GameInfo::startGame(0)`, seeds the logic random and
|
|
appends `MSG_NEW_GAME` (`0x2`). `startNewGame` itself is `0x00623e40`. OpenRA3
|
|
mirrors this two-phase `prepare_new_game` / `start_new_game` split.
|
|
|
|
## Container and map formats
|
|
|
|
Recovered by inspection of `Data\*.big` (see `ra3.fs` / `ra3.map`).
|
|
|
|
### `BIG4` archive
|
|
|
|
All integers little-endian except where noted:
|
|
|
|
```
|
|
offset 0 magic "BIG4"
|
|
offset 4 fileSize u32 LE total archive size
|
|
offset 8 fileCount u32 BE number of entries
|
|
offset 12 indexSize u32
|
|
offset 16 entries fileCount * { offset u32 BE, size u32 BE, name cstring }
|
|
```
|
|
|
|
Entry offsets are absolute; payloads are RefPack-compressed.
|
|
|
|
### RefPack
|
|
|
|
EA's `10 FB` stream. `ra3.fs::refpack_decompress` implements the 2/3/4-byte
|
|
commands and the long-literal/stop opcodes; `refpack_output_size` reads the
|
|
declared output length without decompressing.
|
|
|
|
### Map file (`.map`)
|
|
|
|
Two layers of compression:
|
|
|
|
```
|
|
BIG4 payload = RefPack -> "EAR\0" + u32 unpacked_size + RefPack -> "CkMp" ...
|
|
```
|
|
|
|
The `CkMp` payload is the compiled SAGE map: a type/field name table followed by
|
|
chunk data. Player start positions appear as waypoints named
|
|
`Player_1_Start`, `Player_2_Start`, ... Each waypoint record carries a `Coord3D`
|
|
(three little-endian `float`s) shortly after the name; `ra3.map` scans forward
|
|
from the name for the first plausible `(x, y, 0)` triple. Maps that store starts
|
|
in `MPPositionList` instead yield no waypoints and fall back.
|
|
|
|
Verified example (`map_mp_2_feasel4`): `Player_1_Start` = `(1338.9, 1940.5, 0)`,
|
|
`Player_2_Start` = `(1290.8, 1404.9, 0)`.
|
|
|
|
### Terrain (`ra3.terrain`)
|
|
|
|
The `CkMp` tree is a flat chunk list: `"CkMp"`, `u32 assetCount`, the name
|
|
table (`{ u8 len, name, u32 index }`, index descending from `assetCount`), then
|
|
`{ u32 index, u16 version, u32 size, data }` per chunk. On
|
|
`map_mp_2_feasel4` the terrain is `HeightMapData` v6 (540 x 600, border 20,
|
|
`u16` elevations, scale `0.0390625`) and `BlendTileData` v27.
|
|
|
|
`BlendTileData` opens with `NumTiles`, the `u16` tile grid, then the
|
|
blend/three-way/cliff tables (`u16` for v27); the passability flag arrays that
|
|
follow are not needed for rendering, so the texture table is located by
|
|
scanning for its `{ cellStart, cellCount, cellSize, magic }` + `u16`-prefixed
|
|
name entries. A tile value is `(cellIndex << 2) | variant`, and `cellIndex`
|
|
indexes the global `TextureCellCount`-cell table (each texture owning
|
|
`cellSize^2` 64 px cells). The textures themselves are
|
|
`art\terrain\<stem>.tga` in `Terrain.big` / `Core11.big` (RefPack + 256x256
|
|
TGA). Rendered top-down, `map_mp_2_feasel4` correlates 0.94 with the official
|
|
`_art.tga` overview.
|
|
|
|
### Terrain blending (`BlendTileData` tail)
|
|
|
|
After the tile grid, `BlendTileData` stores three per-cell `u16` tables —
|
|
`Blends`, `ThreeWayBlends` and `CliffTextures` — then `TextureCellCount`,
|
|
`BlendsCount`, the texture table, two magic words and `BlendsCount - 1` blend
|
|
descriptions (18 bytes each: `u32 secondaryTile`, four direction bytes,
|
|
`u8 flags`, `u8 twoSided`, `u32 0xFFFFFFFF`, `u32 0x7ADA0000`). A non-zero
|
|
`Blends[cell]` is a 1-based index into the descriptions; `secondaryTile` is a
|
|
packed tile value (`secondaryTile >> 2` is its cell).
|
|
|
|
The retail `Terrain.fx` (compiled `terrain.fxo`, parameters `Terrain.BaseTexture`,
|
|
`Terrain.MacroTexture`, `MapCellSize`, `IsTerrainAtlasEnabled`; technique
|
|
`TerrainTile`) cross-fades a cell's base tile into `secondaryTile` with a linear
|
|
ramp selected by `BlendDirection`: `1` right, `2` top, `4` top-right, `8`
|
|
top-left, where `flags` bit 0 flips the axis and bit 1 marks a two-sided
|
|
diagonal. OpenSAGE's `Terrain.frag` reconstructs the exact
|
|
`CalculateBlendFactor`; `ra3::terrain::blend_factor` and `shaders/terrain.frag`
|
|
mirror it. The row axis is *not* inverted (73% of long-axis blends point at a
|
|
neighbour of the same texture, versus 25% inverted). On `map_mp_2_feasel4`,
|
|
33828 of 324000 cells carry a blend and there are 7094 descriptions.
|
|
|
|
The GPU atlas pads every 64 px tile with a 2-texel replicated gutter. Packed
|
|
edge-to-edge, bilinear/mipmap filtering averaged two unrelated tiles at every
|
|
cell border — that cross-tile bleed was the visible grid line the hardware path
|
|
drew. (`shaders/terrain.frag` samples `cell_stride = cell_texels + 2 * gutter`.)
|
|
|
|
### Open question: per-cell tile sampling
|
|
|
|
Measured on `map_mp_2_feasel4`, sampling each cell as its own 64 px block and
|
|
restarting the UV every cell leaves a 1.39x edge spike at cell boundaries
|
|
(43.4 vs 31.1 mean gradient at 8 px/cell). Two candidate mappings reduce it and
|
|
need a visual decision against the retail art:
|
|
|
|
| Mapping | Boundary/interior |
|
|
| --- | --- |
|
|
| per-cell 64 px block (current) | 1.39 |
|
|
| OpenSAGE `BlendTileTextureIndex` Morton layout, 32 px block | 1.18 |
|
|
| continuous `uv / (cellSize * 2)`, per OpenSAGE `Terrain.frag` | 1.11 |
|
|
|
|
The retail `Terrain.frag` (OpenSAGE) samples the tile texture *continuously*
|
|
(`uv / (CellSize * 2)`), so adjacent cells never restart the texture; our
|
|
per-cell restart is the remaining source of grid lines. Switching the atlas
|
|
from 64 px blocks to continuous 32 px regions (or the Morton 8x8 layout) is the
|
|
next step, pending an art correlation check.
|
|
|
|
### Map display names
|
|
|
|
The skirmish map list labels live in `Data\English.big`'s
|
|
`data\gamestrings.csf` (the newest `Lang-English*.big` wins) under
|
|
`MAP:<UPPERCASE_ID>`, e.g. `MAP:MAP_MP_2_FEASEL4` = "Battlebase Beta". CSF
|
|
values are UTF-16 code units whose low byte is XORed with `0xFF`
|
|
(`ra3::map::parse_map_names`). `openra3 extract` writes the decoded table to
|
|
`maps/map_names.tsv`; `openra3 menu` shows them instead of the raw map id.
|
|
|
|
|
|
### Still to recover
|
|
|
|
- `MPPositionList` layout (per-player starts for maps without waypoints).
|
|
- Cliff textures and the `CliffTextureMapping` UV remap (`CliffTextures` is
|
|
parsed but not yet drawn).
|
|
- Compiled asset blobs (`global.bin`, `static.*.bin`) and the `.manifest`
|
|
schema used to deserialise them.
|