OpenRA3/third_party/libra3assets/README.md

libra3assets

A dependency-free C++26 library that reads and writes the resource files Red Alert 3 ships, so tooling (map editors in particular) can work with the retail assets directly. No engine, no game install required to build - only to feed it data.

It is a sub-project of this repository, a sibling of libra3replay, and follows the same build conventions (C++26 modules, import std;, GCC 16, CMake 4 + Ninja).

What it handles

Module	Format	Read	Write
ra3.assets:big	BIG4 archives (Data\*.big)	yes	yes
ra3.assets:refpack	EA RefPack codec (10 FB)	yes	yes
ra3.assets:binary	compiled BinaryAsset streams (.bin + .manifest, cdata)	yes	-
ra3.assets:csf	SAGE .csf string tables (gamestrings.csf)	yes	yes
ra3.assets:map	SAGE .map containers (CkMp) incl. HeightMapData/objects	yes	yes
ra3.assets:bytes	bounds-checked little-/big-endian readers and writers	yes	yes

The map module is the centrepiece for a map editor. A .map is modelled as an ordered list of named, versioned CkMp chunks; chunks the library does not type (e.g. BlendTileData, SidesList) are preserved byte-for-byte and the asset-name table keeps its original indices, so an edit/ serialise cycle is lossless. Typed accessors cover the terrain grid (HeightMapData), every placed object (ObjectsList, including the *Waypoints/Waypoint objects that carry Player_N_Start), MPPositionList, WorldInfo and WaypointsList.

The binary module parses the compiled BinaryAsset streams that BinaryAssetBuilder produces (data\static.bin, data\global.bin, ...): the manifest index, each asset's instance slice, its relocation/import sidecars and its cdata blob, plus the hash used to name assets.

Build

cmake -S libra3assets -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=g++-16
cmake --build build
ctest --test-dir build --output-on-failure

The library is a set of C++20/26 modules (ra3.assets plus the :error, :bytes, :refpack, :big, :binary, :csf and :map partitions) that imports the standard library (import std;). That needs CMake 4.0+ with the Ninja generator and a compiler whose standard library ships a std module - GCC 16 in practice. The tests need nothing but the library itself.

Building with the distro GCC 16: pass -DCMAKE_CXX_COMPILER=g++-16.

Use

import std;
import ra3.assets;

using namespace ra3::assets;

// --- a map, for a map editor -------------------------------------------------
auto document = map_document::open("map_mp_2_black1b.map");

if (const auto height = document.height_map()) {
    std::println("terrain {}x{}", height->width, height->height);
    auto grid = *height;                       // copy, then edit
    grid.set(10, 10, grid.at(10, 10) + 1);     // raise a cell
    document.set_height_map(std::move(grid));  // re-encode the chunk
}

for (const auto &start: document.player_starts())
    std::println("Player {} at ({}, {})", start.index, start.position.x, start.position.y);

for (const auto &object: document.objects())
    if (object.type_name == "*Waypoints/Waypoint")
        std::println("{} -> {}", object.property("waypointName")->as_ascii(), object.position.y);

// Lossless: unknown chunks and the name table survive the round-trip.
write_file("edited.map", document.serialize(/* compress = */ true));

// --- a BIG4 archive ----------------------------------------------------------
const auto archive = big_archive::open("Data/GlobalStream.big");
std::println("{} entries", archive.size());
for (const auto *entry: archive.find("audio"))
    std::println("{} ({} bytes)", entry->name, entry->size);

// --- a compiled BinaryAsset stream ------------------------------------------
const auto stream = binary_stream_from_big(archive, binary_stream::global, /* need_data = */ false);
for (const auto &[type, count]: stream.type_counts())
    std::println("{:6}  {}", count, type);

The API deliberately avoids integer IDs for anything selectable: chunk kinds are chunk_kind (e.g. chunk_kind::height_map_data), streams are binary_stream (binary_stream::global), property types are property_type, and an asset_property carries a std::variant<bool, std::int32_t, float, std::string, std::u16string>.

Command-line tool

ra3assets-cli is built alongside the library:

ra3assets-cli big    list    <archive.big> [match]
ra3assets-cli big    extract <archive.big> <out-dir> [match]
ra3assets-cli binary list    <path> [type]
ra3assets-cli binary types   <path> [static|global|locale|static_l|static_m]
ra3assets-cli binary cat     <path> <Type:Instance|#index> <out-file>
ra3assets-cli map    info    <map-file>
ra3assets-cli map    starts  <map-file>
ra3assets-cli map    repack  <map-file> <out-file> [--compress]
ra3assets-cli csf    <csf-or-game-dir-or-big> [label]
ra3assets-cli hash   <text>...

Notes and limits

• A BIG4 archive opened from disk keeps only its index in memory and reads payloads on demand, so enumerating the ~700 MB retail StaticStream.big costs a few megabytes; pass need_data = false to binary_stream_from_big when you only need the manifest. A parsed .map document is held in full - that is what its typed accessors edit.
• set_height_map keeps the on-disk HeightMapData version (RA3 uses 6, i.e. 16-bit elevations). Older 8-bit maps serialise back as 8-bit.
• RefPack compression is a plain greedy LZ77 matcher. It is lossless against the bundled decoder and produces streams the game's decoder accepts, but it is slightly less dense than the retail compressor (≈3% on a typical map).
• BlendTileData (terrain texture blending) is preserved but not yet type-modelled; a map editor should treat the chunk as opaque for now. The planned follow-up covers BlendTileData and SidesList.
• Only BIG4 is supported (RA3's format); older BIGF archives are not.

The on-disk layouts were reverse engineered from the shipped files and cross-checked against ra3tools, OpenRA3's ra3.fs/ra3.map modules and the OpenSAGE re-implementation (reference/OpenSAGE).