OpenRA3/docs/REVERSE_ENGINEERING.md

Reverse engineering workflow

OpenRA3 is reconstructed from two sources:

1. Architecture — the GPLv3 SAGE 1.0 tree, electronicarts/CnC_Generals_Zero_Hour. RA3 runs SAGE 2.0, so class names, message flow and subsystem boundaries carry over even though the code does not.
2. Facts — the retail ra3_1.12.game binary (image base 0x400000), analysed in Ghidra and, when needed, observed live.

Nothing in OpenRA3 should assert a structure or constant that is not either copied from the reference or cited to a retail address.

Fetching the reference

tools/fetch_reference.sh            # sparse-clone Code/GameEngine into reference/

The checkout is git-ignored (large, and GPLv3 terms differ from this repo's).

Ghidra

The retail module is loaded into Ghidra as ra3_1.12.game (x86:LE:32:default, 34k+ functions). The analysis is driven through the Ghidra MCP bridge, so every recovered fact can be re-derived:

Question	Tool call
What does an address do?	decompile_function(address=0x…)
What is this function?	get_function_by_address(address=0x…)
Who touches a global?	get_xrefs_to(address=0x…)
What are the vtable slots?	list_class_members / analyze_data_region
Where is a string referenced?	search_strings + get_xrefs_to

Recovery loop

1. Pick a subsystem from the reference tree (e.g. MessageStream).
2. Find its RTTI/vtable in the binary via search_strings and xrefs.
3. Decompile the constructor to recover object size and field init order.
4. Decompile the hot methods to recover field meaning.
5. Write the OpenRA3 module with a comment citing the address.
6. Add a smoke test that pins the behaviour.

Recovered symbol map (retail ra3_1.12.game)

Engine singletons

Each holds an object pointer (0 when the subsystem is down).

Global	Address	Object vtable
TheGameLogic	0x00cd8ce4	0x00beb630
ThePlayerList	0x00ce8c9c	0x00c5b9e0
ThePartitionManager	0x00ce2f9c	0x00c6af98
TheShroudManager	0x00ce2fa0	0x00c6aefc
ThePlacementGrid	0x00cd8d0c	0x00bea560
TheRecorder	0x00ce2fd0	0x00c10544
TheGlobalObjectRegistry	0x00cd8d08	0x00bea644
ThePlayerTemplateStore	0x00ce8ca0	0x00c5bc70
TheGameState	0x00cdbbc4	0x00bef0c0
GlobalData	0x00ce2fa8	0x00c0d8e4
TheMessageStream	0x00ce2fb8	0x00c0ecd4

Reconstructed layouts

• GameLogic — tick counter at +0x50 (incremented once per 30 Hz simulation step); starting guard flag at +0xa7, raised while a new match initialises.
• MessageStream — intrusive doubly linked list; head at +0x24, tail at +0x28. appendMessage (0x0060c4a0) allocates a 0x74-byte node: +0x00 next, +0x04 prev, +0x08 owner stream, +0x0c message type, +0x10 player index, +0x18 capacity, +0x1c data pointer (node + 0x20).
• Player — money via std::vector<Money*> at +0xe4; power at +0x74; team at +0xac; relation maps at +0xfc / +0x100.
• Skirmish setup (SkirmishGameInfo, pointer at [0x00ce3a78]) — starting cash +0x64, player slots +0xfc (stride 0x5c, 6 slots), faction at slot + 0x18 (Empire=2, Allied=4, Soviet=8, Random=7).

Tactical view (camera)

The in-game camera is the TheTacticalView object, held in the global at 0x00cdb7b4. Its vtable accessors return zoom, pitch (current/target), yaw, world position and FOV; the debug overlay FUN_005ef0a0 prints them through the format string at 0x00c0b900. Mouse state is the singleton at 0x00ce9284 (cursor position vtable slot +0x3c, button down +0x48); the keyboard manager is 0x00ce927c (modifier mask +0x38).

Per-map camera tuning is a named-field table in .rdata (around 0x00c11a54): cameraMinHeight, cameraMaxHeight, cameraPitchAngle, cameraYawAngle, cameraScrollSpeedScalar, cameraGroundMinHeight, cameraGroundMaxHeight. The map-load chunk CHUNK_TacticalView (0x00beea14, consumed near 0x00548a00) seeds the view from the map; the tutorial actions LOCK_CAMERA_SCROLL / LOCK_CAMERA_ZOOM / LOCK_CAMERA_ROTATION gate the controls.

OpenRA3 has no 3D terrain yet, so ra3::render::view_camera reproduces the controls - wheel zoom, screen-edge scroll, clamped pan and opening on the player's start - over the 2D map overview, not the retail perspective camera.

Match start

The BEGIN button calls SkirmishGameOptionsMenu::start (0x00b28d60), which copies the map, calls GameInfo::startGame(0), seeds the logic random and appends MSG_NEW_GAME (0x2). startNewGame itself is 0x00623e40. OpenRA3 mirrors this two-phase prepare_new_game / start_new_game split.

Container and map formats

Recovered by inspection of Data\*.big (see ra3.fs / ra3.map).

BIG4 archive

All integers little-endian except where noted:

offset 0   magic      "BIG4"
offset 4   fileSize   u32 LE   total archive size
offset 8   fileCount  u32 BE   number of entries
offset 12  indexSize  u32
offset 16  entries    fileCount * { offset u32 BE, size u32 BE, name cstring }

Entry offsets are absolute; payloads are RefPack-compressed.

RefPack

EA's 10 FB stream. ra3.fs::refpack_decompress implements the 2/3/4-byte commands and the long-literal/stop opcodes; refpack_output_size reads the declared output length without decompressing.

Map file (.map)

Two layers of compression:

BIG4 payload          = RefPack  ->  "EAR\0" + u32 unpacked_size + RefPack  ->  "CkMp" ...

The CkMp payload is the compiled SAGE map: a type/field name table followed by chunk data. Player start positions appear as waypoints named Player_1_Start, Player_2_Start, ... Each waypoint record carries a Coord3D (three little-endian floats) shortly after the name; ra3.map scans forward from the name for the first plausible (x, y, 0) triple. Maps that store starts in MPPositionList instead yield no waypoints and fall back.

Verified example (map_mp_2_feasel4): Player_1_Start = (1338.9, 1940.5, 0), Player_2_Start = (1290.8, 1404.9, 0).

Terrain (ra3.terrain)

The CkMp tree is a flat chunk list: "CkMp", u32 assetCount, the name table ({ u8 len, name, u32 index }, index descending from assetCount), then { u32 index, u16 version, u32 size, data } per chunk. On map_mp_2_feasel4 the terrain is HeightMapData v6 (540 x 600, border 20, u16 elevations, scale 0.0390625) and BlendTileData v27.

BlendTileData opens with NumTiles, the u16 tile grid, then the blend/three-way/cliff tables (u16 for v27); the passability flag arrays that follow are not needed for rendering, so the texture table is located by scanning for its { cellStart, cellCount, cellSize, magic } + u16-prefixed name entries. A tile value is (cellIndex << 2) | variant, and cellIndex indexes the global TextureCellCount-cell table (each texture owning cellSize^2 64 px cells). The textures themselves are art\terrain\<stem>.tga in Terrain.big / Core11.big (RefPack + 256x256 TGA). Rendered top-down, map_mp_2_feasel4 correlates 0.94 with the official _art.tga overview.

Terrain blending (BlendTileData tail)

After the tile grid, BlendTileData stores three per-cell u16 tables — Blends, ThreeWayBlends and CliffTextures — then TextureCellCount, BlendsCount, the texture table, two magic words and BlendsCount - 1 blend descriptions (18 bytes each: u32 secondaryTile, four direction bytes, u8 flags, u8 twoSided, u32 0xFFFFFFFF, u32 0x7ADA0000). A non-zero Blends[cell] is a 1-based index into the descriptions; secondaryTile is a packed tile value (secondaryTile >> 2 is its cell).

The retail Terrain.fx (compiled terrain.fxo, parameters Terrain.BaseTexture, Terrain.MacroTexture, MapCellSize, IsTerrainAtlasEnabled; technique TerrainTile) cross-fades a cell's base tile into secondaryTile with a linear ramp selected by BlendDirection: 1 right, 2 top, 4 top-right, 8 top-left, where flags bit 0 flips the axis and bit 1 marks a two-sided diagonal. OpenSAGE's Terrain.frag reconstructs the exact CalculateBlendFactor; ra3::terrain::blend_factor and shaders/terrain.frag mirror it. The row axis is not inverted (73% of long-axis blends point at a neighbour of the same texture, versus 25% inverted). On map_mp_2_feasel4, 33828 of 324000 cells carry a blend and there are 7094 descriptions.

The GPU atlas pads every 64 px tile with a 2-texel replicated gutter. Packed edge-to-edge, bilinear/mipmap filtering averaged two unrelated tiles at every cell border — that cross-tile bleed was the visible grid line the hardware path drew. (shaders/terrain.frag samples cell_stride = cell_texels + 2 * gutter.)

Open question: per-cell tile sampling

Measured on map_mp_2_feasel4, sampling each cell as its own 64 px block and restarting the UV every cell leaves a 1.39x edge spike at cell boundaries (43.4 vs 31.1 mean gradient at 8 px/cell). Two candidate mappings reduce it and need a visual decision against the retail art:

Mapping	Boundary/interior
per-cell 64 px block (current)	1.39
OpenSAGE BlendTileTextureIndex Morton layout, 32 px block	1.18
continuous uv / (cellSize * 2), per OpenSAGE Terrain.frag	1.11

The retail Terrain.frag (OpenSAGE) samples the tile texture continuously (uv / (CellSize * 2)), so adjacent cells never restart the texture; our per-cell restart is the remaining source of grid lines. Switching the atlas from 64 px blocks to continuous 32 px regions (or the Morton 8x8 layout) is the next step, pending an art correlation check.

Compiled art (BinaryAssetBuilder) and map objects

Retail RA3 ships no .w3x/.w3d files: BinaryAssetBuilder bakes every model, texture and script into a binary asset stream — a .manifest index plus a .bin of relocatable instance data (and optional .relo/.imp fixups). The layout (little-endian) is:

ManifestHeader (48 B)  isBigEndian u8, isLinked u8, version u16,
                       streamChecksum, allTypesHash, assetCount u32,
                       totalInstanceDataSize, maxInstance/maxRelocation/
                       maxImportsChunkSize, assetReferenceBufferSize,
                       referenceManifestNameBufferSize, assetNameBufferSize,
                       sourceFileNameBufferSize
AssetEntry (48 B) * count
                       typeId, instanceId, typeHash, instanceHash,
                       assetReferenceOffset i32, assetReferenceCount i32,
                       nameOffset i32, sourceFileNameOffset i32,
                       instanceDataSize i32, relocationDataSize i32,
                       importsDataSize i32, tokenized u32
then the reference / referenced-name / asset-name / source-name buffers

Asset names are Type:Instance (e.g. W3DMesh:BB_GRASS02). Instance pointers are stored as offsets from the start of the instance data (which begins at byte 4 of .bin, after the stream checksum), so a slice is readable without the relocation stream.

Each multiplayer map carries its own stream (data\maps\official\<id>\map.bin) but it is linked: only map-specific assets (the terrain texture atlas, scripts, GameMap) have data; the rendered props are imported and therefore have instanceDataSize == 0. The complete prop art (meshes + textures) lives in Data\WBData.big's data\worldbuilder.bin, which is uncompressed (first four bytes are the stream checksum, not 10 FB), so ra3.models reads the 1.1 GB stream lazily — the manifest is parsed and only the needed instance slices are read.

W3DMesh compiled layout (offsets from the instance start):

+4   vertexBufferPtr        +52 triangleCount      +56 triangleItemPtr
+60  shaderNameLength       +64 shaderNamePtr
vertexBuffer: +0 numVertices, +4 stride, +8 elementDataPtr,
              +12 declarationBytes, +16 declarationPtr
declaration:  text "p0:00:3f32 n0:0C:3f32 t0:1C:2f32" (D3D9 usage:index:offset:type)
triangles:    triangleCount * { u32 indexCount, u32 indexPtr } (24 B each), u32 indices

The diffuse texture is found through the mesh's FXShaderConstants (+76 count, +80 items): a texture-valued constant (TypeId 0xA59096A6) names its role (DiffuseTexture, NormalMap, SpecMap) and points at a 1-based index into the mesh's cross-asset references, which resolve by (typeId, instanceId). The Texture instance embeds a standard DDS file (at u32@+4, or scan for "DDS "); ra3.models::decode_dds decodes DXT1/3/5 and uncompressed 16/24/32-bit.

Vertices are stored in bone space, not object space. A mesh whose vertex declaration carries blend data (i0:..:4u8 w0:..:4u8n, e.g. buildings and vehicles) must be skinned; props without it (BB_GRASS02, IF_STREETSEGMENT01) are already in object space. The skeleton is a W3DHierarchy asset named after the mesh's instance prefix (W3DMesh:FI_STRUCTURE_02.NEWSKIN_CIV01 → W3DHierarchy:FI_STRUCTURE_02). Compiled layout:

W3DHierarchy: u32 pad, u32 boneCount, u32 headerBytes, then boneCount records
100 B each:
    u32 nameHash, i32 parent (-1 = root), f32 translation[3],
    f32 quaternion[4] (x, y, z, w), f32 matrix[12]

The default (bind) pose is rebuilt by composing each bone's local translation/quaternion down the parent chain, then skinnedPos = Σ wᵢ · (Rᵢ·p + Tᵢ) (and the normal by the rotation only). ra3.models does this before placing the mesh at the map object's (x, y, angle).

The compiled shader (below) binds one joint per vertex — WorldBones holds 64 bones as 2 float4 each (quaternion c[128+2j], translation c[129+2j]) — so the skin is rigid: blendindices.x selects the joint, remapped through the mesh's per-model bone table (vertex-descriptor +0x14 = bone count, +0x18 = u16 bone indices into the W3DHierarchy). Applying the raw blend index without that remap tears models apart (FI_BUILDING01's table is [0,14,15,16,17,18], not [0..5]).

The map objects that lie flat on the ground (sidewalks, deck pieces) are coplanar with the terrain; retail biases their depth in the shader so they do not z-fight. The object pass reproduces that with a small negative depth bias (Vulkan depthBiasConstant/SlopeFactor, and a half-unit bias in the software rasteriser).

Meshes whose material has no diffuse texture (DefaultW3D.fx, BasicW3D.fx — FXLIGHTS/ambient helper billboards) are not opaque geometry and are skipped; drawing them fills the frame with garbage triangles. Likewise the BuildingsGenericDamageFill.fx damage-fill sub-meshes are skipped: they are the wrecked-interior shell (e.g. CBBuilding_Wood, an orange plank texture) that retail only reveals through damage holes, but our opaque pass would paint it over the main shell and tint whole buildings warm.

Official shader behaviour (Shaders.big → *.fxo)

The compiled D3D9 effects in Data\Shaders.big name their parameters, so the model pipeline is recoverable. buildingsgeneric.fxo (BuildingsGeneric.fx) vertex stage: skinning (above), World/ViewProjection, and vertex color c0 multiplied into the lit color ((Ambient·AmbientColor + Σ DLᵢ.Color·max(N·DLᵢ,0)) · DiffuseColor · vertexColor). Pixel stage samples DiffuseTexture/NormalMap/SpecMap/DamagedTexture/ CloudTexture (all at UV0, except DamagedTexture at v0.wz = transposed UV1), then final.rgb *= TintColor and *= ShroudTexture.rgb. DiffuseVelocity is not used for static structures. So the diffuse texture is UV0 and is tinted by vertex color and TintColor; basicw3d.fxo instead modulates a single macro/lightmap with (vertexColor + additive) * diffuse * 2 and has no normal map. (Recovered by disassembling the embedded vs_3_0/ps_3_0 bytecode.)

Water / ocean (Ocean.fx, OceanDisplacement.fx, RiverWater.fx, UnderwaterDeferred.fx)

The SAGE water surface is reconstructed by OpenSAGE as Assets/Shaders/Water.vert+Water.frag (same family as retail Ocean.fx). The surface is a world-space mesh drawn with alpha blending, fed by two render targets rendered before it: a reflection map (scene from the mirrored camera about GlobalWaterSettings.ReflectionPlaneZ) and a refraction map + depth. The fragment model (Water.frag) is:

• waterUV = worldPos.xy / 320; a scrolling WaterTexture supplies both a flow distortion ((tex.xy*2-1)*0.05) and the flow layer; a BumpTexture supplies the surface worldNormal.
• fresnelFactor = dot(viewVector, +Z); reflection/refraction are sampled in screen space (gl_FragCoord / ViewportSize), each displaced by the distortion.
• linearWaterDepth = linearize(RefractionDepth) - linearize(gl_FragCoord.z); alpha = clamp((linearWaterDepth/2)/TransparentWaterDepth, 0, TransparentWaterMinOpacity).
• final = diffuseColor * textureColor * cloudColor, then mixed with mix(reflectionColor, refractionColor, fresnelFactor) (both maps on) or just one of them, per IsRenderReflection / IsRenderRefraction.
• Per-time-of-day WaterSet: WaterTexture, UScrollPerMS/VScrollPerMS, DiffuseColor, TransparentDiffuseColor; WaterTransparency supplies TransparentWaterDepth/TransparentWaterMinOpacity, the skybox faces, RiverTransparencyMultiplier, ReflectionPlaneZ/ReflectionOn.

OpenRA3 has no water mesh or reflection/refraction targets (its terrain is raymarched), so the model is folded into the terrain pass (shaders/terrain.frag and its HLSL/GLSL-ES/WGSL twins): the ray's water-plane hit takes a scrolling wave normal built from the retail bump map combined with a de-gridded procedural wave, Schlick fresnel (F0 = 0.02), a sky reflection and a depth-graded refraction, SAGE diffuse + specular lighting, a depth-based transparency fade, and — when the camera is below ReflectionPlaneZ — an underwater tint/fog (UnderwaterDeferred.fx). The two retail maps art/terrain/ra3_deepocean.tga (flow/distortion) and ra3_deepocean_nrm.tga (bump normal) are appended as the last two layers of the terrain atlas (ra3::terrain::build_gpu_terrain, water_flow = layer_count - 2, water_normal = layer_count - 1), so every backend samples them with the existing atlas binding; absent maps fall back to a neutral layer. True reflection/refraction render targets are still the next step. GPUParticleOceanDisplacement.fx drives wave displacement from a GPU particle buffer and has no analogue here.

The map's ObjectsList chunk is a list of nested Object assets — Coord3D, Z angle, RoadType u32, a u16-prefixed type-name and a property list whose keys index the shared name table (ra3.map::parse_objects). Each type resolves to the W3DMesh parts whose instance name equals it or starts with <type>..

Roads / sidewalks (Road assets)

The flat sidewalk and road decals are not meshes: the map places them as consecutive pairs of objects — a RoadType::Start (2) and a RoadType::End (4) at the segment's two ends (Angled 8, TightCurve 64 and EndCap 128 are curve/cap flags; BridgeStart/End 16/32 are bridges). The road's type-name (IslandFortressSidewalk01, ...) resolves to a compiled Road asset in the same art stream:

Road: u32 version, u32 pad, u32 textureCount, f32 roadWidth, f32 pad,
      f32 ???, then textureCount texture references (diffuse, normal)

ra3.models pairs the objects in file order, emits a flat quad ribbon of roadWidth along each segment (overlapping the joins by half a width), sampled with the diffuse texture, and lifts it onto the terrain. Retail gives roads a small depth offset toward the screen so they do not clip into the ground; the object pass reproduces that with the same negative depth bias used for the other ground decals (depthBiasConstant/SlopeFactor on Vulkan, a half-unit bias in the software rasteriser).

Map display names

The skirmish map list labels live in Data\English.big's data\gamestrings.csf (the newest Lang-English*.big wins) under MAP:<UPPERCASE_ID>, e.g. MAP:MAP_MP_2_FEASEL4 = "Battlebase Beta". CSF values are UTF-16 code units whose low byte is XORed with 0xFF (ra3::map::parse_map_names). openra3 extract writes the decoded table to maps/map_names.tsv; openra3 menu shows them instead of the raw map id.

Still to recover

• MPPositionList layout (per-player starts for maps without waypoints).
• Cliff textures and the CliffTextureMapping UV remap (CliffTextures is parsed but not yet drawn).
• The Road network mesher (the map's sidewalk/road objects reference Road templates, not W3DMesh assets).
• W3D container/hierarchy assembly and animation (props are drawn as their static mesh parts; skinned/animated in-game models are not).
• Compiled asset blobs (global.bin, static.*.bin) and the .manifest schema used to deserialise them.