EnderTheCoder
23f8be394c
Vendor libra3assets (C++26 modules: BIG4, RefPack, BinaryAsset, CSF, CkMp map) under third_party/ and add the ra3_assets target. ra3.fs, ra3.map and ra3.terrain become thin adapters over it: - ra3.fs delegates RefPack and the BIG4 index/payload reads (index-only, payloads read on demand). - ra3.map decodes ObjectsList and CSF via map_document/csf_table; starts come from player_starts(), replacing the off-by-one whole-buffer scan. - ra3.terrain takes the CkMp container and HeightMapData from map_document, with BlendTileData (not modelled by the library) parsed from the chunk payload. Adds an opt-in real-asset check (OPENRA3_TEST_ASSETS) plus the library's own unit suite as ra3assets_unit.
135 lines
6.4 KiB
Markdown
135 lines
6.4 KiB
Markdown
# libra3assets
|
|
|
|
A dependency-free C++26 library that reads **and writes** the resource files
|
|
Red Alert 3 ships, so tooling (map editors in particular) can work with the
|
|
retail assets directly. No engine, no game install required to build - only to
|
|
feed it data.
|
|
|
|
It is a sub-project of this repository, a sibling of
|
|
[`libra3replay`](../libra3replay/README.md), and follows the same build
|
|
conventions (C++26 modules, `import std;`, GCC 16, CMake 4 + Ninja).
|
|
|
|
## What it handles
|
|
|
|
| Module | Format | Read | Write |
|
|
| ------------------ | ------------------------------------------------------------------- | :--: | :---: |
|
|
| `ra3.assets:big` | `BIG4` archives (`Data\*.big`) | yes | yes |
|
|
| `ra3.assets:refpack` | EA RefPack codec (`10 FB`) | yes | yes |
|
|
| `ra3.assets:binary` | compiled `BinaryAsset` streams (`.bin` + `.manifest`, `cdata`) | yes | - |
|
|
| `ra3.assets:csf` | SAGE `.csf` string tables (`gamestrings.csf`) | yes | yes |
|
|
| `ra3.assets:map` | SAGE `.map` containers (`CkMp`) incl. `HeightMapData`/objects | yes | yes |
|
|
| `ra3.assets:bytes` | bounds-checked little-/big-endian readers and writers | yes | yes |
|
|
|
|
The **map** module is the centrepiece for a map editor. A `.map` is modelled as
|
|
an ordered list of named, versioned `CkMp` chunks; chunks the library does not
|
|
type (e.g. `BlendTileData`, `SidesList`) are preserved byte-for-byte and the
|
|
asset-name table keeps its original indices, so an edit/ serialise cycle is
|
|
lossless. Typed accessors cover the terrain grid (`HeightMapData`), every placed
|
|
object (`ObjectsList`, including the `*Waypoints/Waypoint` objects that carry
|
|
`Player_N_Start`), `MPPositionList`, `WorldInfo` and `WaypointsList`.
|
|
|
|
The **binary** module parses the compiled `BinaryAsset` streams that
|
|
BinaryAssetBuilder produces (`data\static.bin`, `data\global.bin`, ...): the
|
|
manifest index, each asset's instance slice, its relocation/import sidecars and
|
|
its `cdata` blob, plus the hash used to name assets.
|
|
|
|
## Build
|
|
|
|
```bash
|
|
cmake -S libra3assets -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=g++-16
|
|
cmake --build build
|
|
ctest --test-dir build --output-on-failure
|
|
```
|
|
|
|
The library is a set of C++20/26 modules (`ra3.assets` plus the `:error`,
|
|
`:bytes`, `:refpack`, `:big`, `:binary`, `:csf` and `:map` partitions) that
|
|
imports the standard library (`import std;`). That needs **CMake 4.0+ with the
|
|
Ninja generator** and a compiler whose standard library ships a `std` module -
|
|
**GCC 16** in practice. The tests need nothing but the library itself.
|
|
|
|
> Building with the distro GCC 16: pass `-DCMAKE_CXX_COMPILER=g++-16`.
|
|
|
|
## Use
|
|
|
|
```cpp
|
|
import std;
|
|
import ra3.assets;
|
|
|
|
using namespace ra3::assets;
|
|
|
|
// --- a map, for a map editor -------------------------------------------------
|
|
auto document = map_document::open("map_mp_2_black1b.map");
|
|
|
|
if (const auto height = document.height_map()) {
|
|
std::println("terrain {}x{}", height->width, height->height);
|
|
auto grid = *height; // copy, then edit
|
|
grid.set(10, 10, grid.at(10, 10) + 1); // raise a cell
|
|
document.set_height_map(std::move(grid)); // re-encode the chunk
|
|
}
|
|
|
|
for (const auto &start: document.player_starts())
|
|
std::println("Player {} at ({}, {})", start.index, start.position.x, start.position.y);
|
|
|
|
for (const auto &object: document.objects())
|
|
if (object.type_name == "*Waypoints/Waypoint")
|
|
std::println("{} -> {}", object.property("waypointName")->as_ascii(), object.position.y);
|
|
|
|
// Lossless: unknown chunks and the name table survive the round-trip.
|
|
write_file("edited.map", document.serialize(/* compress = */ true));
|
|
|
|
// --- a BIG4 archive ----------------------------------------------------------
|
|
const auto archive = big_archive::open("Data/GlobalStream.big");
|
|
std::println("{} entries", archive.size());
|
|
for (const auto *entry: archive.find("audio"))
|
|
std::println("{} ({} bytes)", entry->name, entry->size);
|
|
|
|
// --- a compiled BinaryAsset stream ------------------------------------------
|
|
const auto stream = binary_stream_from_big(archive, binary_stream::global, /* need_data = */ false);
|
|
for (const auto &[type, count]: stream.type_counts())
|
|
std::println("{:6} {}", count, type);
|
|
```
|
|
|
|
The API deliberately avoids integer IDs for anything selectable: chunk kinds are
|
|
`chunk_kind` (e.g. `chunk_kind::height_map_data`), streams are `binary_stream`
|
|
(`binary_stream::global`), property types are `property_type`, and an
|
|
`asset_property` carries a `std::variant<bool, std::int32_t, float, std::string,
|
|
std::u16string>`.
|
|
|
|
## Command-line tool
|
|
|
|
`ra3assets-cli` is built alongside the library:
|
|
|
|
```
|
|
ra3assets-cli big list <archive.big> [match]
|
|
ra3assets-cli big extract <archive.big> <out-dir> [match]
|
|
ra3assets-cli binary list <path> [type]
|
|
ra3assets-cli binary types <path> [static|global|locale|static_l|static_m]
|
|
ra3assets-cli binary cat <path> <Type:Instance|#index> <out-file>
|
|
ra3assets-cli map info <map-file>
|
|
ra3assets-cli map starts <map-file>
|
|
ra3assets-cli map repack <map-file> <out-file> [--compress]
|
|
ra3assets-cli csf <csf-or-game-dir-or-big> [label]
|
|
ra3assets-cli hash <text>...
|
|
```
|
|
|
|
## Notes and limits
|
|
|
|
- A `BIG4` archive opened from disk keeps only its index in memory and reads
|
|
payloads on demand, so enumerating the ~700 MB retail `StaticStream.big` costs
|
|
a few megabytes; pass `need_data = false` to `binary_stream_from_big` when you
|
|
only need the manifest. A parsed `.map` document is held in full - that is what
|
|
its typed accessors edit.
|
|
- `set_height_map` keeps the on-disk `HeightMapData` version (RA3 uses 6, i.e.
|
|
16-bit elevations). Older 8-bit maps serialise back as 8-bit.
|
|
- RefPack compression is a plain greedy LZ77 matcher. It is lossless against the
|
|
bundled decoder and produces streams the game's decoder accepts, but it is
|
|
slightly less dense than the retail compressor (≈3% on a typical map).
|
|
- `BlendTileData` (terrain texture blending) is preserved but not yet
|
|
type-modelled; a map editor should treat the chunk as opaque for now. The
|
|
planned follow-up covers `BlendTileData` and `SidesList`.
|
|
- Only `BIG4` is supported (RA3's format); older `BIGF` archives are not.
|
|
|
|
The on-disk layouts were reverse engineered from the shipped files and
|
|
cross-checked against [`ra3tools`](../ra3tools/), OpenRA3's `ra3.fs`/`ra3.map`
|
|
modules and the OpenSAGE re-implementation (`reference/OpenSAGE`).
|