12 Commits
Author SHA1 Message Date
EnderTheCoder 2e284a16e5 chore: release v0.10.1 2026-10-01 01:10:42 +08:00
EnderTheCoder 2f6ff74670 chore(vendor): update vendored libenderlog to v0.0.3
Dated record timestamps, per-sink timestamp format, and archive names
that keep the extension last (<stem>.<timestamp>.<ext>).
2026-10-01 01:05:21 +08:00
EnderTheCoder 2c348cb590 chore: release v0.10.0 2026-09-30 22:18:27 +08:00
EnderTheCoder 26e1934a5d feat(render): SAGE water/ocean shaders, per-user logs, camera & culling
Terrain pass now ports the SAGE water model (Ocean.fx / OpenSAGE Water.frag): a de-gridded procedural wave normal combined with the retail ra3_deepocean flow and ra3_deepocean_nrm bump maps (appended as the last two terrain-atlas layers, no new backend binding), Schlick fresnel, sky reflection + depth-graded refraction, SAGE diffuse/specular lighting, depth-based transparency, and an underwater tint (UnderwaterDeferred.fx). Mirrored across terrain.frag / dx_terrain.hlsl / webgl_terrain_frag.glsl / webgpu_terrain.wgsl.

Also: logs move to the per-user state dir (%LOCALAPPDATA%\\OpenRA3\\logs, else XDG) and archive as openra3.<stamp>.log; the FPS label shows the active backend; middle-drag camera reset; objects and roads below the water plane are culled.
2026-09-30 22:18:08 +08:00
EnderTheCoder 10a1963eec chore: release v0.9.0 2026-09-29 23:59:32 +08:00
EnderTheCoder 23f8be394c feat(assets): read retail assets through the vendored libra3assets library
Vendor libra3assets (C++26 modules: BIG4, RefPack, BinaryAsset, CSF, CkMp map)
under third_party/ and add the ra3_assets target. ra3.fs, ra3.map and ra3.terrain
become thin adapters over it:

- ra3.fs delegates RefPack and the BIG4 index/payload reads (index-only, payloads
  read on demand).
- ra3.map decodes ObjectsList and CSF via map_document/csf_table; starts come from
  player_starts(), replacing the off-by-one whole-buffer scan.
- ra3.terrain takes the CkMp container and HeightMapData from map_document, with
  BlendTileData (not modelled by the library) parsed from the chunk payload.

Adds an opt-in real-asset check (OPENRA3_TEST_ASSETS) plus the library's own unit
suite as ra3assets_unit.
2026-09-29 23:58:13 +08:00
EnderTheCoder 8047e80ed6 v0.8.0: map roads/sidewalks, per-vertex depth bias, retail camera controls
- ra3.map: capture the object RoadType; ra3.models pairs the map's
  Start/End road objects and drapes a flat textured ribbon along each
  segment (tessellated to the relief).
- Ground decals (roads/sidewalks) get a per-vertex screen-space depth
  offset (scene::vertex.bias) applied in the object shader and the
  software rasteriser, so they no longer z-fight or clip into the
  terrain at top-down angles.
- Tactical view aligned with retail (SAGE LookAtTranslator): middle-drag
  orbits yaw/pitch, a middle click resets angle/pitch/zoom, arrow keys
  pan, wheel zooms within the retail zoom range; WASD pan removed and
  left-drag no longer rotates.
- ui_event gains middle/right/released buttons; Vulkan + SDL map them.
- docs: roads, depth bias and camera controls.
2026-09-29 20:37:24 +08:00
EnderTheCoder 70f35d8382 render map objects: decode compiled W3DMesh art, skin and draw over terrain
Read the buildings and props a map places from its compiled art (the
uncompressed worldbuilder stream + embedded DDS textures):

- ra3.models: BAB asset-stream parser (lazy slices), W3DMesh + D3DHierarchy
  decode, DDS (DXT1/3/5 + uncompressed) decode, per-mesh bone remap and
  bind-pose single-joint skinning, flattened to one world-space triangle soup.
- ra3.map: parse the ObjectsList chunk into (type, x, y, z, angle).
- ra3.terrain: render3d rasterises the scene over the raymarched terrain with
  a z-buffer; gpu_terrain carries the scene for the GPU backends.
- ra3.vulkan: second pipeline + depth attachment, terrain.frag writes
  gl_FragDepth, and a small depth bias keeps ground decals from z-fighting.
- objects.fx shaders (compiled to SPIR-V), embedded like scene/terrain.

Only opaque parts are drawn: FX-light billboards (DefaultW3D.fx / BasicW3D.fx)
and damage-fill shells (BuildingsGenericDamageFill.fx) are skipped, since the
latter paint the wrecked interior (e.g. orange CBBuilding_Wood) over the shell.

Ground-decal meshes with no diffuse role and the Road templates themselves are
still not drawn.
2026-09-29 18:08:11 +08:00
EnderTheCoder 6e5df350c7 fix ci: use the Debian apt mirror for the Debian images
debian_image/debian_arm64_image got the Ubuntu TUNA mirror, so apt-get update 404'd on deb.debian.org paths and the images never built (red since v0.5.0). Add DEBIAN_APT_MIRROR and pass it to both debian builds.
2026-09-29 12:16:31 +08:00
EnderTheCoder 889ce945f2 v0.7.0: SDL-free wasm worker backend (WebGPU + WebGL2) with on-demand assets
The engine now runs on a plain Web Worker (no SDL, no PROXY_TO_PTHREAD: in a pthread Emscripten proxies every filesystem syscall to the main browser thread, where synchronous XHR - and thus FS.createLazyFile - is forbidden). A page/worker pair transfers an OffscreenCanvas and forwards DOM input; assets load lazily from an embedded manifest, so entering a map fetches only that map and its tiles.

Presentation: ra3.webgpu (WebGPU via Emscripten's emdawnwebgpu port, WGSL shaders, the default on wasm) and ra3.wasmgl (WebGL2, GLSL ES). The legacy SDL ra3.webgl backend is removed.
2026-09-29 12:08:07 +08:00
EnderTheCoder 70d4beca4f v0.6.0: WebAssembly build with a WebGL backend (playable in the browser)
- ra3.webgl: a Web peer of Vulkan/Direct3D (SDL3 canvas + GLES3/WebGL2), with
  2D blit and the GPU terrain raymarch; GLSL ES shaders; null fallback off
  Emscripten. backend::webgl + default_backend() (webgl under Emscripten).
- Emscripten toolchain + Dockerfile.wasm + scripts/build-wasm.sh; import std,
  global -fexceptions, and Asyncify so the blocking frame loop yields to the
  browser (display::sleep_frame calls emscripten_sleep).
- apps/web/shell.html + apps/web/serve.py (Range-capable dev server).
- Assets: browsers forbid synchronous on-demand reads on the main thread (and
  FS.createLazyFile / the WasmFS fetch backend are worker-only; SDL3's Emscripten
  backend is main-thread DOM only, so the engine cannot run in a worker). The
  wasm build therefore preloads a compact per-map set via OPENRA3_WEB_ASSETS.
- New openra3 textures --map ID lists the loose terrain TGAs a map resolves to
  (terrain::resolve_texture_files, shared with load_textures_from_dir).
- Fixes: webgl heightmap used GL_R16 = 0x8229 (that is R8) -> upload rejected,
  terrain flattened to water; now 0x822A. Logger uses a stdout console sink on
  the web (stderr maps to console.error); a GL error check logs bad uploads.
- CI: wasm image + build jobs.
2026-09-28 22:57:19 +08:00
EnderTheCoder d0ae4c76ba v0.5.0: Windows/ARM64 and Linux/ARM64 cross targets, .deb/.msi packaging
- Cross toolchains: llvm-mingw-aarch64 (Windows ARM64; SDL3 staged from MSYS2's
  clangarm64 repo since SDL ships no MinGW ARM64 dev package) and
  clang-aarch64-linux-gnu (Linux ARM64; arm64 libc++/SDL3 via dpkg multiarch).
- Distro images: Dockerfile.linux-arm64, Dockerfile.win-arm64, Dockerfile.debian
  and Dockerfile.debian-arm64 (Debian 13 with upstream CMake, whose 3.31 gate
  cannot enable import std).
- Packaging via CPack: .deb for Ubuntu 26.04 and Debian 13 on amd64 and arm64
  (dependencies derived by dpkg-shlibdeps), portable .zip/.tar.gz, and a
  Windows .msi built with wixl (msitools) straight from the Linux cross image.
  wixl 0.106 lacks arm64, so Windows/ARM64 ships the portable zip only.
- CMake: parameterize the SDL3 MinGW triple, add install() + CPack config.
- CI: build/package jobs for both arm64 targets and both Debian packages;
  existing Linux/Windows jobs now also emit the .deb / .zip + .msi.
- scripts/: build-linux-arm64, build-windows-arm64, build-debian,
  build-debian-arm64.
2026-09-28 20:38:40 +08:00
71 changed files with 9835 additions and 727 deletions
+216 -3
View File
@@ -8,6 +8,9 @@ variables:
DOCKER_BUILDKIT: "1" DOCKER_BUILDKIT: "1"
# apt mirror used inside both images (override per pipeline if needed) # apt mirror used inside both images (override per pipeline if needed)
APT_MIRROR: "http://mirrors.tuna.tsinghua.edu.cn/ubuntu" APT_MIRROR: "http://mirrors.tuna.tsinghua.edu.cn/ubuntu"
# Debian images need the Debian (not Ubuntu) mirror path: the two repos live
# under /debian and /debian-security on the same host.
DEBIAN_APT_MIRROR: "http://mirrors.tuna.tsinghua.edu.cn/debian"
# Images are shared through Harbor (plain HTTP), never as artifacts. # Images are shared through Harbor (plain HTTP), never as artifacts.
HARBOR_HOST: "192.168.1.11:9090" HARBOR_HOST: "192.168.1.11:9090"
HARBOR_PROJECT: "openra3" HARBOR_PROJECT: "openra3"
@@ -89,10 +92,11 @@ build_linux:
script: script:
- retry sh -c 'echo "$HARBOR_PASSWORD" | docker login "$HARBOR_HOST" -u "$HARBOR_USERNAME" --password-stdin' - retry sh -c 'echo "$HARBOR_PASSWORD" | docker login "$HARBOR_HOST" -u "$HARBOR_USERNAME" --password-stdin'
- retry docker pull $IMAGE:$CI_COMMIT_SHORT_SHA-linux - retry docker pull $IMAGE:$CI_COMMIT_SHORT_SHA-linux
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-linux cmake -S . -B build/linux -G Ninja -DCMAKE_BUILD_TYPE=Release - docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-linux cmake -S . -B build/linux -G Ninja -DCMAKE_BUILD_TYPE=Release -DOPENRA3_DISTRO_TAG=ubuntu26.04
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-linux cmake --build build/linux -j - docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-linux cmake --build build/linux -j
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-linux ctest --test-dir build/linux --output-on-failure - docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-linux ctest --test-dir build/linux --output-on-failure
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-linux bash -c "mkdir -p artifacts/linux && cp build/linux/bin/openra3 artifacts/linux/" - docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-linux bash -c "cd build/linux && cpack -G DEB"
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-linux bash -c "mkdir -p artifacts/linux && cp build/linux/bin/openra3 build/linux/*.deb artifacts/linux/"
artifacts: artifacts:
name: "$CI_PROJECT_NAME-linux-$CI_COMMIT_SHORT_SHA" name: "$CI_PROJECT_NAME-linux-$CI_COMMIT_SHORT_SHA"
paths: paths:
@@ -116,9 +120,218 @@ build_windows:
-DCMAKE_TOOLCHAIN_FILE=cmake/toolchains/llvm-mingw-x86_64.cmake -DCMAKE_TOOLCHAIN_FILE=cmake/toolchains/llvm-mingw-x86_64.cmake
-DCMAKE_BUILD_TYPE=Release -DCMAKE_BUILD_TYPE=Release
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-windows cmake --build build/windows -j - docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-windows cmake --build build/windows -j
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-windows bash -c "mkdir -p artifacts/windows && cp build/windows/bin/openra3.exe build/windows/bin/SDL3.dll artifacts/windows/" - docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-windows cmake --build build/windows --target openra3_msi
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-windows bash -c "cd build/windows && cpack -G ZIP"
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-windows bash -c "mkdir -p artifacts/windows && cp build/windows/bin/openra3.exe build/windows/bin/SDL3.dll build/windows/bin/*.msi build/windows/*.zip artifacts/windows/"
artifacts: artifacts:
name: "$CI_PROJECT_NAME-windows-$CI_COMMIT_SHORT_SHA" name: "$CI_PROJECT_NAME-windows-$CI_COMMIT_SHORT_SHA"
paths: paths:
- artifacts/windows/ - artifacts/windows/
expire_in: 7 days expire_in: 7 days
# --- arm64 + Debian toolchain images -----------------------------------------
linux_arm64_image:
stage: image
image: docker:latest
services: *dind
tags:
- docker
before_script: *retry
script:
- retry sh -c 'echo "$HARBOR_PASSWORD" | docker login "$HARBOR_HOST" -u "$HARBOR_USERNAME" --password-stdin'
- |
retry docker build \
--cache-from type=registry,ref=$IMAGE:cache-linux-arm64,insecure=true \
--cache-to type=registry,ref=$IMAGE:cache-linux-arm64,mode=max,insecure=true \
--build-arg APT_MIRROR=$APT_MIRROR \
--target dev \
-t $IMAGE:$CI_COMMIT_SHORT_SHA-linux-arm64 \
-f Dockerfile.linux-arm64 .
- retry docker push $IMAGE:$CI_COMMIT_SHORT_SHA-linux-arm64
windows_arm64_image:
stage: image
image: docker:latest
services: *dind
tags:
- docker
before_script: *retry
script:
- retry sh -c 'echo "$HARBOR_PASSWORD" | docker login "$HARBOR_HOST" -u "$HARBOR_USERNAME" --password-stdin'
- |
retry docker build \
--cache-from type=registry,ref=$IMAGE:cache-windows-arm64,insecure=true \
--cache-to type=registry,ref=$IMAGE:cache-windows-arm64,mode=max,insecure=true \
--build-arg APT_MIRROR=$APT_MIRROR \
--target dev \
-t $IMAGE:$CI_COMMIT_SHORT_SHA-windows-arm64 \
-f Dockerfile.win-arm64 .
- retry docker push $IMAGE:$CI_COMMIT_SHORT_SHA-windows-arm64
debian_image:
stage: image
image: docker:latest
services: *dind
tags:
- docker
before_script: *retry
script:
- retry sh -c 'echo "$HARBOR_PASSWORD" | docker login "$HARBOR_HOST" -u "$HARBOR_USERNAME" --password-stdin'
- |
retry docker build \
--cache-from type=registry,ref=$IMAGE:cache-debian,insecure=true \
--cache-to type=registry,ref=$IMAGE:cache-debian,mode=max,insecure=true \
--build-arg APT_MIRROR=$DEBIAN_APT_MIRROR \
--target dev \
-t $IMAGE:$CI_COMMIT_SHORT_SHA-debian \
-f Dockerfile.debian .
- retry docker push $IMAGE:$CI_COMMIT_SHORT_SHA-debian
debian_arm64_image:
stage: image
image: docker:latest
services: *dind
tags:
- docker
before_script: *retry
script:
- retry sh -c 'echo "$HARBOR_PASSWORD" | docker login "$HARBOR_HOST" -u "$HARBOR_USERNAME" --password-stdin'
- |
retry docker build \
--cache-from type=registry,ref=$IMAGE:cache-debian-arm64,insecure=true \
--cache-to type=registry,ref=$IMAGE:cache-debian-arm64,mode=max,insecure=true \
--build-arg APT_MIRROR=$DEBIAN_APT_MIRROR \
--target dev \
-t $IMAGE:$CI_COMMIT_SHORT_SHA-debian-arm64 \
-f Dockerfile.debian-arm64 .
- retry docker push $IMAGE:$CI_COMMIT_SHORT_SHA-debian-arm64
# --- build each target (binary + packages) -----------------------------------
build_linux_arm64:
stage: build
image: docker:latest
services: *dind
tags:
- docker
needs:
- linux_arm64_image
before_script: *retry
script:
- retry sh -c 'echo "$HARBOR_PASSWORD" | docker login "$HARBOR_HOST" -u "$HARBOR_USERNAME" --password-stdin'
- retry docker pull $IMAGE:$CI_COMMIT_SHORT_SHA-linux-arm64
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-linux-arm64 cmake -S . -B build/linux-arm64 -G Ninja -DCMAKE_TOOLCHAIN_FILE=cmake/toolchains/clang-aarch64-linux-gnu.cmake -DCMAKE_BUILD_TYPE=Release -DOPENRA3_DISTRO_TAG=ubuntu26.04
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-linux-arm64 cmake --build build/linux-arm64 -j
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-linux-arm64 bash -c "cd build/linux-arm64 && cpack -G DEB"
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-linux-arm64 bash -c "mkdir -p artifacts/linux-arm64 && cp build/linux-arm64/bin/openra3 build/linux-arm64/*.deb artifacts/linux-arm64/"
artifacts:
name: "$CI_PROJECT_NAME-linux-arm64-$CI_COMMIT_SHORT_SHA"
paths:
- artifacts/linux-arm64/
expire_in: 7 days
build_windows_arm64:
stage: build
image: docker:latest
services: *dind
tags:
- docker
needs:
- windows_arm64_image
before_script: *retry
script:
- retry sh -c 'echo "$HARBOR_PASSWORD" | docker login "$HARBOR_HOST" -u "$HARBOR_USERNAME" --password-stdin'
- retry docker pull $IMAGE:$CI_COMMIT_SHORT_SHA-windows-arm64
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-windows-arm64 cmake -S . -B build/windows-arm64 -G Ninja -DCMAKE_TOOLCHAIN_FILE=cmake/toolchains/llvm-mingw-aarch64.cmake -DCMAKE_BUILD_TYPE=Release
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-windows-arm64 cmake --build build/windows-arm64 -j
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-windows-arm64 bash -c "cd build/windows-arm64 && cpack -G ZIP"
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-windows-arm64 bash -c "mkdir -p artifacts/windows-arm64 && cp build/windows-arm64/bin/openra3.exe build/windows-arm64/bin/SDL3.dll build/windows-arm64/*.zip artifacts/windows-arm64/"
artifacts:
name: "$CI_PROJECT_NAME-windows-arm64-$CI_COMMIT_SHORT_SHA"
paths:
- artifacts/windows-arm64/
expire_in: 7 days
build_debian:
stage: build
image: docker:latest
services: *dind
tags:
- docker
needs:
- debian_image
before_script: *retry
script:
- retry sh -c 'echo "$HARBOR_PASSWORD" | docker login "$HARBOR_HOST" -u "$HARBOR_USERNAME" --password-stdin'
- retry docker pull $IMAGE:$CI_COMMIT_SHORT_SHA-debian
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-debian cmake -S . -B build/debian -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_STDLIB_MODULES_JSON=/usr/lib/llvm-19/lib/libc++.modules.json -DOPENRA3_DISTRO_TAG=debian13
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-debian cmake --build build/debian -j
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-debian ctest --test-dir build/debian --output-on-failure
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-debian bash -c "cd build/debian && cpack -G DEB"
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-debian bash -c "mkdir -p artifacts/debian && cp build/debian/bin/openra3 build/debian/*.deb artifacts/debian/"
artifacts:
name: "$CI_PROJECT_NAME-debian-$CI_COMMIT_SHORT_SHA"
paths:
- artifacts/debian/
expire_in: 7 days
build_debian_arm64:
stage: build
image: docker:latest
services: *dind
tags:
- docker
needs:
- debian_arm64_image
before_script: *retry
script:
- retry sh -c 'echo "$HARBOR_PASSWORD" | docker login "$HARBOR_HOST" -u "$HARBOR_USERNAME" --password-stdin'
- retry docker pull $IMAGE:$CI_COMMIT_SHORT_SHA-debian-arm64
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-debian-arm64 cmake -S . -B build/debian-arm64 -G Ninja -DCMAKE_TOOLCHAIN_FILE=cmake/toolchains/clang-aarch64-linux-gnu.cmake -DCMAKE_BUILD_TYPE=Release -DOPENRA3_AARCH64_CC=clang-19 -DOPENRA3_AARCH64_CXX=clang++-19 -DOPENRA3_AARCH64_STDLIB_MODULES_JSON=/usr/lib/llvm-19/lib/libc++.modules.json -DOPENRA3_DISTRO_TAG=debian13
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-debian-arm64 cmake --build build/debian-arm64 -j
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-debian-arm64 bash -c "cd build/debian-arm64 && cpack -G DEB"
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-debian-arm64 bash -c "mkdir -p artifacts/debian-arm64 && cp build/debian-arm64/bin/openra3 build/debian-arm64/*.deb artifacts/debian-arm64/"
artifacts:
name: "$CI_PROJECT_NAME-debian-arm64-$CI_COMMIT_SHORT_SHA"
paths:
- artifacts/debian-arm64/
expire_in: 7 days
# --- WebAssembly (Emscripten) -------------------------------------------------
wasm_image:
stage: image
image: docker:latest
services: *dind
tags:
- docker
before_script: *retry
script:
- retry sh -c 'echo "$HARBOR_PASSWORD" | docker login "$HARBOR_HOST" -u "$HARBOR_USERNAME" --password-stdin'
- |
retry docker build \
--cache-from type=registry,ref=$IMAGE:cache-wasm,insecure=true \
--cache-to type=registry,ref=$IMAGE:cache-wasm,mode=max,insecure=true \
--build-arg APT_MIRROR=$APT_MIRROR \
--target dev \
-t $IMAGE:$CI_COMMIT_SHORT_SHA-wasm \
-f Dockerfile.wasm .
- retry docker push $IMAGE:$CI_COMMIT_SHORT_SHA-wasm
build_wasm:
stage: build
image: docker:latest
services: *dind
tags:
- docker
needs:
- wasm_image
before_script: *retry
script:
- retry sh -c 'echo "$HARBOR_PASSWORD" | docker login "$HARBOR_HOST" -u "$HARBOR_USERNAME" --password-stdin'
- retry docker pull $IMAGE:$CI_COMMIT_SHORT_SHA-wasm
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-wasm cmake -S . -B build/wasm -G Ninja -DCMAKE_TOOLCHAIN_FILE=cmake/toolchains/emscripten-wasm.cmake -DCMAKE_BUILD_TYPE=Release
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-wasm cmake --build build/wasm -j
- docker run --rm -v "$(pwd):/work" -w /work $IMAGE:$CI_COMMIT_SHORT_SHA-wasm bash -c "mkdir -p artifacts/wasm && cp build/wasm/bin/index.html build/wasm/bin/openra3.js build/wasm/bin/openra3.wasm build/wasm/bin/openra3.worker.js artifacts/wasm/ && cp build/wasm/bin/openra3.data artifacts/wasm/ 2>/dev/null || true"
artifacts:
name: "$CI_PROJECT_NAME-wasm-$CI_COMMIT_SHORT_SHA"
paths:
- artifacts/wasm/
expire_in: 7 days
+296 -20
View File
@@ -7,8 +7,9 @@ set(CMAKE_EXPERIMENTAL_CXX_IMPORT_STD "d0edc3af-4c50-42ea-a356-e2862fe7a444")
# Path to the standard-library module description. A cross toolchain overrides # Path to the standard-library module description. A cross toolchain overrides
# this before the compiler is probed; this default targets the Linux LLVM # this before the compiler is probed; this default targets the Linux LLVM
# package. # package. Skipped for Emscripten, whose toolchain supplies its own sysroot
if(NOT CMAKE_CXX_STDLIB_MODULES_JSON) # modules.json.
if(NOT CMAKE_CXX_STDLIB_MODULES_JSON AND NOT DEFINED ENV{EMSDK})
set(CMAKE_CXX_STDLIB_MODULES_JSON "/usr/lib/llvm-21/lib/libc++.modules.json") set(CMAKE_CXX_STDLIB_MODULES_JSON "/usr/lib/llvm-21/lib/libc++.modules.json")
endif() endif()
@@ -25,7 +26,7 @@ set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF) set(CMAKE_CXX_EXTENSIONS OFF)
set(CMAKE_CXX_SCAN_FOR_MODULES ON) set(CMAKE_CXX_SCAN_FOR_MODULES ON)
project(OpenRA3 VERSION 0.4.0 LANGUAGES C CXX) project(OpenRA3 VERSION 0.9.0 LANGUAGES C CXX)
if(NOT CMAKE_BUILD_TYPE AND NOT CMAKE_CONFIGURATION_TYPES) if(NOT CMAKE_BUILD_TYPE AND NOT CMAKE_CONFIGURATION_TYPES)
set(CMAKE_BUILD_TYPE Release CACHE STRING "Build type" FORCE) set(CMAKE_BUILD_TYPE Release CACHE STRING "Build type" FORCE)
@@ -52,9 +53,16 @@ endfunction()
# --- SDL3: pkg-config on Linux, or an explicit MinGW root for Windows -------- # --- SDL3: pkg-config on Linux, or an explicit MinGW root for Windows --------
add_library(openra3_sdl3 INTERFACE) add_library(openra3_sdl3 INTERFACE)
set(OPENRA3_HAS_SDL3 OFF) set(OPENRA3_HAS_SDL3 OFF)
if(OPENRA3_SDL3_ROOT) # The triple-named subdirectory inside a MinGW SDL3 development package
target_include_directories(openra3_sdl3 INTERFACE "${OPENRA3_SDL3_ROOT}/x86_64-w64-mingw32/include") # (`<root>/<triple>/{include,lib,bin}`); x86_64 unless a toolchain overrides it.
target_link_libraries(openra3_sdl3 INTERFACE "${OPENRA3_SDL3_ROOT}/x86_64-w64-mingw32/lib/libSDL3.dll.a") set(OPENRA3_SDL3_MINGW_TRIPLE "x86_64-w64-mingw32" CACHE STRING "SDL3 MinGW triple subdirectory")
if(EMSCRIPTEN)
# The wasm build runs the engine on a Web Worker and renders into an
# OffscreenCanvas (ra3.webgpu / ra3.wasmgl), so it links no SDL: SDL3's
# Emscripten port is main-thread DOM only and cannot run in a worker.
elseif(OPENRA3_SDL3_ROOT)
target_include_directories(openra3_sdl3 INTERFACE "${OPENRA3_SDL3_ROOT}/${OPENRA3_SDL3_MINGW_TRIPLE}/include")
target_link_libraries(openra3_sdl3 INTERFACE "${OPENRA3_SDL3_ROOT}/${OPENRA3_SDL3_MINGW_TRIPLE}/lib/libSDL3.dll.a")
set(OPENRA3_HAS_SDL3 ON) set(OPENRA3_HAS_SDL3 ON)
else() else()
find_package(PkgConfig QUIET) find_package(PkgConfig QUIET)
@@ -75,9 +83,15 @@ endif()
# --- Vulkan: vendored volk + headers, so no Vulkan SDK is needed --------------- # --- Vulkan: vendored volk + headers, so no Vulkan SDK is needed ---------------
# The loader is resolved at runtime (volk dlopen/LoadLibrary), which lets the # The loader is resolved at runtime (volk dlopen/LoadLibrary), which lets the
# same source build the Vulkan backend on Linux and on the MinGW Windows target. # same source build the Vulkan backend on Linux and on the MinGW Windows target.
option(OPENRA3_VULKAN "Build the Vulkan viewer" ON) # WebAssembly has no Vulkan, so the viewer is off there (the null backend).
if(EMSCRIPTEN)
set(OPENRA3_VULKAN_DEFAULT OFF)
else()
set(OPENRA3_VULKAN_DEFAULT ON)
endif()
option(OPENRA3_VULKAN "Build the Vulkan viewer" ${OPENRA3_VULKAN_DEFAULT})
set(OPENRA3_HAS_VULKAN OFF) set(OPENRA3_HAS_VULKAN OFF)
if(OPENRA3_VULKAN AND OPENRA3_HAS_SDL3) if(OPENRA3_VULKAN AND OPENRA3_HAS_SDL3 AND NOT EMSCRIPTEN)
set(OPENRA3_HAS_VULKAN ON) set(OPENRA3_HAS_VULKAN ON)
endif() endif()
if(OPENRA3_HAS_VULKAN) if(OPENRA3_HAS_VULKAN)
@@ -86,9 +100,11 @@ else()
message(STATUS "OpenRA3: Vulkan viewer disabled - offscreen image only") message(STATUS "OpenRA3: Vulkan viewer disabled - offscreen image only")
endif() endif()
add_library(ra3_volk STATIC third_party/volk/volk.c) if(NOT EMSCRIPTEN)
target_include_directories(ra3_volk PUBLIC third_party/volk third_party/vulkan/include) add_library(ra3_volk STATIC third_party/volk/volk.c)
target_compile_definitions(ra3_volk PUBLIC VK_NO_PROTOTYPES) target_include_directories(ra3_volk PUBLIC third_party/volk third_party/vulkan/include)
target_compile_definitions(ra3_volk PUBLIC VK_NO_PROTOTYPES)
endif()
# Embed the committed SPIR-V blobs into a generated header at configure time. # Embed the committed SPIR-V blobs into a generated header at configure time.
# Regenerate with shaders/compile.sh after editing a .vert/.frag. # Regenerate with shaders/compile.sh after editing a .vert/.frag.
@@ -132,6 +148,44 @@ function(openra3_embed_hlsl out_header)
file(WRITE "${out_header}" "${content}") file(WRITE "${out_header}" "${content}")
endfunction() endfunction()
# Embed the GLSL ES sources for the WebGL backend as string literals (compiled
# at runtime by the browser's GL, like the HLSL for Direct3D).
function(openra3_embed_glsl out_header)
set(content "// Generated by CMake from shaders/*.glsl.\n")
string(APPEND content "#pragma once\n\nnamespace ra3_shaders {\n")
foreach(src IN LISTS ARGN)
if(NOT EXISTS "${src}")
message(FATAL_ERROR "Missing GLSL shader ${src}")
endif()
get_filename_component(base "${src}" NAME_WE)
string(MAKE_C_IDENTIFIER "${base}" ident)
string(APPEND ident "_glsl")
file(READ "${src}" text)
string(APPEND content "inline constexpr const char ${ident}[] = R\"RA3GLSL(${text})RA3GLSL\";\n\n")
endforeach()
string(APPEND content "} // namespace ra3_shaders\n")
file(WRITE "${out_header}" "${content}")
endfunction()
# Embed the WGSL sources for the WebGPU backend as string literals (compiled at
# runtime by the browser's WebGPU, like the GLSL/HLSL for the other backends).
function(openra3_embed_wgsl out_header)
set(content "// Generated by CMake from shaders/*.wgsl.\n")
string(APPEND content "#pragma once\n\nnamespace ra3_shaders {\n")
foreach(src IN LISTS ARGN)
if(NOT EXISTS "${src}")
message(FATAL_ERROR "Missing WGSL shader ${src}")
endif()
get_filename_component(base "${src}" NAME_WE)
string(MAKE_C_IDENTIFIER "${base}" ident)
string(APPEND ident "_wgsl")
file(READ "${src}" text)
string(APPEND content "inline constexpr const char ${ident}[] = R\"RA3WGSL(${text})RA3WGSL\";\n\n")
endforeach()
string(APPEND content "} // namespace ra3_shaders\n")
file(WRITE "${out_header}" "${content}")
endfunction()
# --- logging (vendored libenderlog, MIT) ------------------------------------- # --- logging (vendored libenderlog, MIT) -------------------------------------
# A standalone C++26 module logger (`import ender.log;`) with a per-run # A standalone C++26 module logger (`import ender.log;`) with a per-run
# archiving file sink. libc++ has no <stacktrace>, so on this toolchain the # archiving file sink. libc++ has no <stacktrace>, so on this toolchain the
@@ -142,6 +196,25 @@ target_sources(ra3_enderlog PUBLIC FILE_SET CXX_MODULES FILES third_party/libend
target_compile_features(ra3_enderlog PUBLIC cxx_std_26) target_compile_features(ra3_enderlog PUBLIC cxx_std_26)
openra3_target_defaults(ra3_enderlog) openra3_target_defaults(ra3_enderlog)
# --- RA3 built-in asset containers (vendored libra3assets) --------------------
# Reader/writer for the files the retail game ships: `BIG4` archives, the EA
# RefPack codec, SAGE `.csf` string tables and `.map` (`CkMp`) containers, plus
# the compiled `BinaryAsset` streams. A sibling of libenderlog: C++26 modules
# (`import ra3.assets;`) importing the standard library. It owns the on-disk
# format knowledge, so `ra3.fs` / `ra3.map` / `ra3.terrain` become thin adapters.
add_library(ra3_assets STATIC)
target_sources(ra3_assets PUBLIC FILE_SET CXX_MODULES FILES
third_party/libra3assets/src/assets.cppm
third_party/libra3assets/src/error.cppm
third_party/libra3assets/src/bytes.cppm
third_party/libra3assets/src/refpack.cppm
third_party/libra3assets/src/big.cppm
third_party/libra3assets/src/binary.cppm
third_party/libra3assets/src/csf.cppm
third_party/libra3assets/src/map.cppm)
target_compile_features(ra3_assets PUBLIC cxx_std_26)
openra3_target_defaults(ra3_assets)
# --- engine core ------------------------------------------------------------- # --- engine core -------------------------------------------------------------
add_library(ra3_core STATIC) add_library(ra3_core STATIC)
target_sources(ra3_core PUBLIC FILE_SET CXX_MODULES FILES src/core/ra3.core.cppm) target_sources(ra3_core PUBLIC FILE_SET CXX_MODULES FILES src/core/ra3.core.cppm)
@@ -171,16 +244,16 @@ target_sources(ra3_game PUBLIC FILE_SET CXX_MODULES FILES src/game/ra3.game.cppm
target_link_libraries(ra3_game PUBLIC ra3_core ra3_logic) target_link_libraries(ra3_game PUBLIC ra3_core ra3_logic)
openra3_target_defaults(ra3_game) openra3_target_defaults(ra3_game)
# --- filesystem: BIG4 + RefPack --------------------------------------------- # --- filesystem: BIG4 + RefPack (adapter over libra3assets) -------------------
add_library(ra3_fs STATIC) add_library(ra3_fs STATIC)
target_sources(ra3_fs PUBLIC FILE_SET CXX_MODULES FILES src/fs/ra3.fs.cppm) target_sources(ra3_fs PUBLIC FILE_SET CXX_MODULES FILES src/fs/ra3.fs.cppm)
target_link_libraries(ra3_fs PUBLIC ra3_core) target_link_libraries(ra3_fs PUBLIC ra3_core ra3_assets)
openra3_target_defaults(ra3_fs) openra3_target_defaults(ra3_fs)
# --- map discovery/loading --------------------------------------------------- # --- map discovery/loading ---------------------------------------------------
add_library(ra3_map STATIC) add_library(ra3_map STATIC)
target_sources(ra3_map PUBLIC FILE_SET CXX_MODULES FILES src/map/ra3.map.cppm) target_sources(ra3_map PUBLIC FILE_SET CXX_MODULES FILES src/map/ra3.map.cppm)
target_link_libraries(ra3_map PUBLIC ra3_core ra3_fs) target_link_libraries(ra3_map PUBLIC ra3_core ra3_fs ra3_assets)
openra3_target_defaults(ra3_map) openra3_target_defaults(ra3_map)
# --- minimal skirmish simulation -------------------------------------------- # --- minimal skirmish simulation --------------------------------------------
@@ -198,9 +271,18 @@ openra3_target_defaults(ra3_render)
# --- real map terrain (HeightMapData / BlendTileData) ------------------------ # --- real map terrain (HeightMapData / BlendTileData) ------------------------
add_library(ra3_terrain STATIC) add_library(ra3_terrain STATIC)
target_sources(ra3_terrain PUBLIC FILE_SET CXX_MODULES FILES src/terrain/ra3.terrain.cppm) target_sources(ra3_terrain PUBLIC FILE_SET CXX_MODULES FILES src/terrain/ra3.terrain.cppm)
target_link_libraries(ra3_terrain PUBLIC ra3_core ra3_fs ra3_render) target_link_libraries(ra3_terrain PUBLIC ra3_core ra3_fs ra3_render ra3_assets)
openra3_target_defaults(ra3_terrain) openra3_target_defaults(ra3_terrain)
# --- map static art (compiled W3D meshes) ------------------------------------
add_library(ra3_models STATIC)
target_sources(ra3_models PUBLIC FILE_SET CXX_MODULES FILES src/models/ra3.models.cppm)
target_link_libraries(ra3_models PUBLIC ra3_core ra3_fs ra3_render)
openra3_target_defaults(ra3_models)
# A map's terrain carries the objects placed on it (props/buildings).
target_link_libraries(ra3_terrain PUBLIC ra3_models)
# The presentation facade imports render + terrain (for the terrain capability). # The presentation facade imports render + terrain (for the terrain capability).
target_link_libraries(ra3_client PUBLIC ra3_render ra3_terrain) target_link_libraries(ra3_client PUBLIC ra3_render ra3_terrain)
@@ -224,13 +306,17 @@ if(OPENRA3_HAS_VULKAN AND OPENRA3_HAS_SDL3)
"${CMAKE_CURRENT_SOURCE_DIR}/shaders/generated/scene.frag.spv" "${CMAKE_CURRENT_SOURCE_DIR}/shaders/generated/scene.frag.spv"
"${CMAKE_CURRENT_SOURCE_DIR}/shaders/generated/terrain.vert.spv" "${CMAKE_CURRENT_SOURCE_DIR}/shaders/generated/terrain.vert.spv"
"${CMAKE_CURRENT_SOURCE_DIR}/shaders/generated/terrain.frag.spv" "${CMAKE_CURRENT_SOURCE_DIR}/shaders/generated/terrain.frag.spv"
"${CMAKE_CURRENT_SOURCE_DIR}/shaders/generated/object.vert.spv"
"${CMAKE_CURRENT_SOURCE_DIR}/shaders/generated/object.frag.spv"
) )
# The blobs are read at configure time, so re-run CMake when they change. # The blobs are read at configure time, so re-run CMake when they change.
set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS
"${CMAKE_CURRENT_SOURCE_DIR}/shaders/generated/scene.vert.spv" "${CMAKE_CURRENT_SOURCE_DIR}/shaders/generated/scene.vert.spv"
"${CMAKE_CURRENT_SOURCE_DIR}/shaders/generated/scene.frag.spv" "${CMAKE_CURRENT_SOURCE_DIR}/shaders/generated/scene.frag.spv"
"${CMAKE_CURRENT_SOURCE_DIR}/shaders/generated/terrain.vert.spv" "${CMAKE_CURRENT_SOURCE_DIR}/shaders/generated/terrain.vert.spv"
"${CMAKE_CURRENT_SOURCE_DIR}/shaders/generated/terrain.frag.spv") "${CMAKE_CURRENT_SOURCE_DIR}/shaders/generated/terrain.frag.spv"
"${CMAKE_CURRENT_SOURCE_DIR}/shaders/generated/object.vert.spv"
"${CMAKE_CURRENT_SOURCE_DIR}/shaders/generated/object.frag.spv")
target_sources(ra3_vulkan PUBLIC FILE_SET CXX_MODULES FILES src/vulkan/ra3.vulkan.sdl.cppm) target_sources(ra3_vulkan PUBLIC FILE_SET CXX_MODULES FILES src/vulkan/ra3.vulkan.sdl.cppm)
target_include_directories(ra3_vulkan PRIVATE "${CMAKE_CURRENT_BINARY_DIR}/generated") target_include_directories(ra3_vulkan PRIVATE "${CMAKE_CURRENT_BINARY_DIR}/generated")
target_link_libraries(ra3_vulkan PUBLIC ra3_core ra3_render ra3_terrain ra3_client openra3_sdl3 ra3_volk) target_link_libraries(ra3_vulkan PUBLIC ra3_core ra3_render ra3_terrain ra3_client openra3_sdl3 ra3_volk)
@@ -272,17 +358,92 @@ else()
endif() endif()
openra3_target_defaults(ra3_dx) openra3_target_defaults(ra3_dx)
# --- wasm worker viewer (SDL-free OffscreenCanvas + WebGL2, or null fallback) --
# The default wasm backend: the engine runs on a plain Web Worker and renders
# into an OffscreenCanvas transferred from the page, with no SDL. Non-wasm builds
# link the null stub (init fails, so desktop backends are unaffected).
option(OPENRA3_WASMGL "Build the SDL-free wasm worker viewer (Emscripten)" ON)
set(OPENRA3_HAS_WASMGL OFF)
if(OPENRA3_WASMGL AND EMSCRIPTEN)
set(OPENRA3_HAS_WASMGL ON)
endif()
add_library(ra3_wasmgl STATIC)
if(OPENRA3_HAS_WASMGL)
message(STATUS "OpenRA3: SDL-free wasm worker viewer enabled")
openra3_embed_glsl(
"${CMAKE_CURRENT_BINARY_DIR}/generated/wasmgl_glsl_embedded.hpp"
"${CMAKE_CURRENT_SOURCE_DIR}/shaders/webgl_scene_vert.glsl"
"${CMAKE_CURRENT_SOURCE_DIR}/shaders/webgl_scene_frag.glsl"
"${CMAKE_CURRENT_SOURCE_DIR}/shaders/webgl_terrain_vert.glsl"
"${CMAKE_CURRENT_SOURCE_DIR}/shaders/webgl_terrain_frag.glsl"
)
set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS
"${CMAKE_CURRENT_SOURCE_DIR}/shaders/webgl_scene_vert.glsl"
"${CMAKE_CURRENT_SOURCE_DIR}/shaders/webgl_scene_frag.glsl"
"${CMAKE_CURRENT_SOURCE_DIR}/shaders/webgl_terrain_vert.glsl"
"${CMAKE_CURRENT_SOURCE_DIR}/shaders/webgl_terrain_frag.glsl")
target_sources(ra3_wasmgl PUBLIC FILE_SET CXX_MODULES FILES src/wasmgl/ra3.wasmgl.cppm)
target_include_directories(ra3_wasmgl PRIVATE "${CMAKE_CURRENT_BINARY_DIR}/generated")
target_link_libraries(ra3_wasmgl PUBLIC ra3_core ra3_render ra3_terrain ra3_client ra3_enderlog)
# Target WebGL2 so GLES3-only entry points (glTexImage3D, VAOs, ...) link.
target_link_options(ra3_wasmgl PUBLIC -sMIN_WEBGL_VERSION=2 -sMAX_WEBGL_VERSION=2)
else()
message(STATUS "OpenRA3: SDL-free wasm worker viewer disabled (non-wasm) - null backend")
target_sources(ra3_wasmgl PUBLIC FILE_SET CXX_MODULES FILES src/wasmgl/ra3.wasmgl.null.cppm)
target_link_libraries(ra3_wasmgl PUBLIC ra3_core ra3_render ra3_client)
endif()
openra3_target_defaults(ra3_wasmgl)
# --- wasm WebGPU viewer (Dawn emdawnwebgpu, or null fallback) ------------------
# A peer of ra3.wasmgl in the same Web Worker; web WebGPU is WGSL-only, so it has
# its own WGSL shaders (the Vulkan SPIR-V blobs cannot be used on the web). The
# `--use-port=emdawnwebgpu` port is downloaded/cached by Emscripten and linked
# into the final module.
option(OPENRA3_WEBGPU "Build the WebGPU wasm viewer (Emscripten emdawnwebgpu)" ON)
set(OPENRA3_HAS_WEBGPU OFF)
if(OPENRA3_WEBGPU AND EMSCRIPTEN)
set(OPENRA3_HAS_WEBGPU ON)
endif()
add_library(ra3_webgpu STATIC)
if(OPENRA3_HAS_WEBGPU)
message(STATUS "OpenRA3: WebGPU wasm viewer enabled (emdawnwebgpu)")
openra3_embed_wgsl(
"${CMAKE_CURRENT_BINARY_DIR}/generated/webgpu_wgsl_embedded.hpp"
"${CMAKE_CURRENT_SOURCE_DIR}/shaders/webgpu_scene.wgsl"
"${CMAKE_CURRENT_SOURCE_DIR}/shaders/webgpu_terrain.wgsl"
)
set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS
"${CMAKE_CURRENT_SOURCE_DIR}/shaders/webgpu_scene.wgsl"
"${CMAKE_CURRENT_SOURCE_DIR}/shaders/webgpu_terrain.wgsl")
target_sources(ra3_webgpu PUBLIC FILE_SET CXX_MODULES FILES src/webgpu/ra3.webgpu.cppm)
target_include_directories(ra3_webgpu PRIVATE "${CMAKE_CURRENT_BINARY_DIR}/generated")
# CMake's module dependency scanner does not process `--use-port`, so point
# the include path at the (image-baked) port package explicitly.
set(OPENRA3_EMDAWNWEBGPU_DIR "$ENV{EMSDK}/upstream/emscripten/cache/ports/emdawnwebgpu/emdawnwebgpu_pkg")
target_include_directories(ra3_webgpu SYSTEM PRIVATE
"${OPENRA3_EMDAWNWEBGPU_DIR}/webgpu_cpp/include"
"${OPENRA3_EMDAWNWEBGPU_DIR}/webgpu/include")
target_compile_options(ra3_webgpu PUBLIC --use-port=emdawnwebgpu)
target_link_options(ra3_webgpu PUBLIC --use-port=emdawnwebgpu)
target_link_libraries(ra3_webgpu PUBLIC ra3_core ra3_render ra3_terrain ra3_client ra3_wasmgl ra3_enderlog)
else()
message(STATUS "OpenRA3: WebGPU wasm viewer disabled (non-wasm) - null backend")
target_sources(ra3_webgpu PUBLIC FILE_SET CXX_MODULES FILES src/webgpu/ra3.webgpu.null.cppm)
target_link_libraries(ra3_webgpu PUBLIC ra3_core ra3_render ra3_client)
endif()
openra3_target_defaults(ra3_webgpu)
# --- display backend selection ------------------------------------------------ # --- display backend selection ------------------------------------------------
add_library(ra3_display STATIC) add_library(ra3_display STATIC)
target_sources(ra3_display PUBLIC FILE_SET CXX_MODULES FILES src/display/ra3.display.cppm) target_sources(ra3_display PUBLIC FILE_SET CXX_MODULES FILES src/display/ra3.display.cppm)
target_link_libraries(ra3_display PUBLIC ra3_core ra3_render ra3_terrain ra3_client ra3_ui ra3_vulkan ra3_dx) target_link_libraries(ra3_display PUBLIC ra3_core ra3_render ra3_terrain ra3_client ra3_ui ra3_vulkan ra3_dx ra3_wasmgl ra3_webgpu)
openra3_target_defaults(ra3_display) openra3_target_defaults(ra3_display)
# --- umbrella ----------------------------------------------------------------- # --- umbrella -----------------------------------------------------------------
add_library(ra3 STATIC) add_library(ra3 STATIC)
target_sources(ra3 PUBLIC FILE_SET CXX_MODULES FILES src/ra3.cppm) target_sources(ra3 PUBLIC FILE_SET CXX_MODULES FILES src/ra3.cppm)
target_link_libraries(ra3 PUBLIC ra3_core ra3_logic ra3_client ra3_data ra3_game ra3_fs ra3_map ra3_skirmish ra3_render ra3_terrain ra3_display target_link_libraries(ra3 PUBLIC ra3_core ra3_logic ra3_client ra3_data ra3_game ra3_fs ra3_map ra3_skirmish ra3_render ra3_terrain ra3_models ra3_display
ra3_ui ra3_vulkan ra3_dx ra3_enderlog) ra3_ui ra3_vulkan ra3_dx ra3_wasmgl ra3_webgpu ra3_enderlog)
openra3_target_defaults(ra3) openra3_target_defaults(ra3)
# --- executable --------------------------------------------------------------- # --- executable ---------------------------------------------------------------
@@ -293,10 +454,113 @@ openra3_target_defaults(openra3)
if(WIN32 AND OPENRA3_SDL3_ROOT) if(WIN32 AND OPENRA3_SDL3_ROOT)
add_custom_command(TARGET openra3 POST_BUILD add_custom_command(TARGET openra3 POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different COMMAND ${CMAKE_COMMAND} -E copy_if_different
"${OPENRA3_SDL3_ROOT}/x86_64-w64-mingw32/bin/SDL3.dll" "${OPENRA3_SDL3_ROOT}/${OPENRA3_SDL3_MINGW_TRIPLE}/bin/SDL3.dll"
"$<TARGET_FILE_DIR:openra3>") "$<TARGET_FILE_DIR:openra3>")
endif() endif()
if(EMSCRIPTEN)
# Emit openra3.js/.wasm. The engine runs inside apps/web/openra3.worker.js
# (a plain Web Worker), which the page (apps/web/index.html) loads and hands
# an OffscreenCanvas; so there is no Emscripten HTML shell here - both web
# files are copied next to the module.
set_target_properties(openra3 PROPERTIES SUFFIX ".js")
target_link_options(openra3 PRIVATE
-sFORCE_FILESYSTEM=1
# index.html input glue calls the exported openra3_* functions via ccall.
"-sEXPORTED_RUNTIME_METHODS=ccall,cwrap")
add_custom_command(TARGET openra3 POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different
"${CMAKE_CURRENT_SOURCE_DIR}/apps/web/index.html"
"${CMAKE_CURRENT_SOURCE_DIR}/apps/web/openra3.worker.js"
"$<TARGET_FILE_DIR:openra3>"
COMMENT "Copying the OpenRA3 web page and worker bootstrap")
# On-demand assets: `OPENRA3_WEB_MANIFEST` is a JSON `{"files": [...]}` listing
# every asset relative to the assets root (generate it with
# scripts/make_web_manifest.py). Only that small manifest is preloaded, to
# /assets.manifest.json; at startup the worker registers every listed file as
# an Emscripten lazy file, so the browser fetches each asset when the engine
# first opens it (serve the tree at <root>/assets, or pass serve.py --assets).
set(OPENRA3_WEB_MANIFEST "" CACHE FILEPATH "Assets manifest JSON, embedded as /assets.manifest.json for lazy loading")
if(OPENRA3_WEB_MANIFEST AND EXISTS "${OPENRA3_WEB_MANIFEST}")
target_link_options(openra3 PRIVATE "--embed-file=${OPENRA3_WEB_MANIFEST}@/assets.manifest.json")
endif()
# Legacy: preload a whole assets directory into the virtual FS at /assets (the
# monolithic path the lazy worker backend replaces). Only used when no manifest
# is given, since lazy files cannot shadow a preloaded tree.
set(OPENRA3_WEB_ASSETS "" CACHE PATH "Assets directory preloaded into the wasm FS at /assets (legacy)")
if(NOT (OPENRA3_WEB_MANIFEST AND EXISTS "${OPENRA3_WEB_MANIFEST}") AND OPENRA3_WEB_ASSETS AND EXISTS "${OPENRA3_WEB_ASSETS}")
target_link_options(openra3 PRIVATE "--preload-file=${OPENRA3_WEB_ASSETS}@/assets")
endif()
endif()
# --- packaging (.deb / .msi / portable archives) -----------------------------
# CPack produces the release artifacts in one shot:
# Linux -> DEB (dependencies derived from the binary with dpkg-shlibdeps)
# plus a portable .tar.gz
# Windows -> a Windows Installer .msi (built with wixl from msitools when the
# WiX tools are absent, so the Linux cross image can make it) plus a
# portable .zip with the exe and SDL3.dll
# OPENRA3_DISTRO_TAG (e.g. ubuntu26.04) is appended to the Linux file name so the
# per-distro images do not collide.
install(TARGETS openra3 RUNTIME DESTINATION bin)
if(WIN32 AND OPENRA3_SDL3_ROOT)
install(FILES "${OPENRA3_SDL3_ROOT}/${OPENRA3_SDL3_MINGW_TRIPLE}/bin/SDL3.dll" DESTINATION bin)
endif()
set(OPENRA3_DISTRO_TAG "" CACHE STRING "Distro tag appended to package file names (e.g. ubuntu26.04)")
set(CPACK_PACKAGE_NAME "openra3")
set(CPACK_PACKAGE_VENDOR "OpenRA3")
set(CPACK_PACKAGE_CONTACT "OpenRA3 <noreply@openra3.invalid>")
set(CPACK_PACKAGE_DESCRIPTION_SUMMARY "OpenRA3 - a from-scratch reimplementation of Red Alert 3")
set(CPACK_PACKAGE_HOMEPAGE_URL "https://git.ender.cool/EnderTheCoder/openra3")
set(CPACK_PACKAGE_VERSION "${PROJECT_VERSION}")
set(CPACK_PACKAGE_EXECUTABLES "openra3" "OpenRA3")
set(CPACK_STRIP_FILES ON)
if(EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/LICENSE")
set(CPACK_RESOURCE_FILE_LICENSE "${CMAKE_CURRENT_SOURCE_DIR}/LICENSE")
endif()
if(WIN32)
if(CMAKE_SYSTEM_PROCESSOR STREQUAL "aarch64")
set(OPENRA3_WIN_ARCH "arm64")
else()
set(OPENRA3_WIN_ARCH "x86_64")
endif()
set(CPACK_PACKAGE_FILE_NAME "openra3-${PROJECT_VERSION}-windows-${OPENRA3_WIN_ARCH}")
# Portable .zip (exe + SDL3.dll).
set(CPACK_GENERATOR "ZIP")
# A Windows Installer (.msi) straight from the Linux cross build via wixl
# (msitools). wixl 0.106 has no arm64 support, so the MSI is offered for
# x86_64 and ARM64 ships the portable zip.
find_program(OPENRA3_WIXL wixl)
if(OPENRA3_WIXL AND NOT CMAKE_SYSTEM_PROCESSOR STREQUAL "aarch64")
configure_file("${CMAKE_CURRENT_SOURCE_DIR}/cmake/packaging/openra3.wxs.in"
"${CMAKE_CURRENT_BINARY_DIR}/openra3.wxs" @ONLY)
add_custom_target(openra3_msi
COMMAND "${OPENRA3_WIXL}" --arch x64
-D "exe=$<TARGET_FILE:openra3>"
-D "dll=${OPENRA3_SDL3_ROOT}/${OPENRA3_SDL3_MINGW_TRIPLE}/bin/SDL3.dll"
-o "$<TARGET_FILE_DIR:openra3>/${CPACK_PACKAGE_FILE_NAME}.msi"
"${CMAKE_CURRENT_BINARY_DIR}/openra3.wxs"
DEPENDS openra3
COMMENT "Building Windows Installer (${CPACK_PACKAGE_FILE_NAME}.msi) with wixl"
VERBATIM)
endif()
else()
if(CMAKE_SYSTEM_PROCESSOR STREQUAL "aarch64")
set(CPACK_DEBIAN_PACKAGE_ARCHITECTURE "arm64")
else()
set(CPACK_DEBIAN_PACKAGE_ARCHITECTURE "amd64")
endif()
set(CPACK_PACKAGE_FILE_NAME "openra3_${PROJECT_VERSION}_${CPACK_DEBIAN_PACKAGE_ARCHITECTURE}${OPENRA3_DISTRO_TAG}")
set(CPACK_GENERATOR "TGZ;DEB")
set(CPACK_DEBIAN_PACKAGE_SHLIBDEPS ON)
set(CPACK_DEBIAN_PACKAGE_SECTION "games")
set(CPACK_DEBIAN_PACKAGE_PRIORITY "optional")
endif()
include(CPack)
# --- extracted assets --------------------------------------------------------- # --- extracted assets ---------------------------------------------------------
# At build time, extract the retail assets into <exe_dir>/assets so the # At build time, extract the retail assets into <exe_dir>/assets so the
# executable is self-contained (no --game-dir at run time). This drives the # executable is self-contained (no --game-dir at run time). This drives the
@@ -335,3 +599,15 @@ add_executable(ra3_tests tests/ra3_tests.cpp)
target_link_libraries(ra3_tests PRIVATE ra3) target_link_libraries(ra3_tests PRIVATE ra3)
openra3_target_defaults(ra3_tests) openra3_target_defaults(ra3_tests)
add_test(NAME ra3_tests COMMAND ra3_tests) add_test(NAME ra3_tests COMMAND ra3_tests)
# libra3assets' own dependency-free unit tests (RefPack + BIG4 + BinaryAsset +
# CSF + CkMp round-trips), built against the vendored copy so the upstream
# suite keeps guarding the readers OpenRA3 now relies on.
add_executable(ra3assets_unit
third_party/libra3assets/tests/test_main.cpp
third_party/libra3assets/tests/test_assets.cpp)
target_include_directories(ra3assets_unit PRIVATE third_party/libra3assets/tests)
target_link_libraries(ra3assets_unit PRIVATE ra3_assets)
target_compile_features(ra3assets_unit PRIVATE cxx_std_26)
openra3_target_defaults(ra3assets_unit)
add_test(NAME ra3assets_unit COMMAND ra3assets_unit)
+5 -2
View File
@@ -29,6 +29,8 @@ RUN --mount=type=cache,target=/var/lib/apt,sharing=locked \
clang-21 \ clang-21 \
clang-tools-21 \ clang-tools-21 \
cmake \ cmake \
dpkg-dev \
file \
gdb \ gdb \
git \ git \
libc++-21-dev \ libc++-21-dev \
@@ -45,9 +47,10 @@ ENV CXX=clang++-21
WORKDIR /work WORKDIR /work
COPY . . COPY . .
RUN cmake -S . -B build/linux -G Ninja -DCMAKE_BUILD_TYPE=Release \ RUN cmake -S . -B build/linux -G Ninja -DCMAKE_BUILD_TYPE=Release -DOPENRA3_DISTRO_TAG=ubuntu26.04 \
&& cmake --build build/linux -j \ && cmake --build build/linux -j \
&& ctest --test-dir build/linux --output-on-failure && ctest --test-dir build/linux --output-on-failure \
&& (cd build/linux && cpack -G DEB)
# deploy target: runtime dependencies + final binary, minimal and fast. # deploy target: runtime dependencies + final binary, minimal and fast.
FROM ubuntu:26.04 AS deploy FROM ubuntu:26.04 AS deploy
+87
View File
@@ -0,0 +1,87 @@
# syntax=docker/dockerfile:1
# Debian build environment (latest stable, Debian 13 "trixie") producing a .deb
# alongside the binary. Separate from Dockerfile (Ubuntu 26.04) because the
# distro and its libc++/SDL3 versions differ, and a .deb's dependencies must be
# generated against the distro it targets.
#
# Toolchain: clang-19 + libc++-19 (for `import std;`). Debian 13 ships CMake
# 3.31, whose experimental `import std` gate value the project does not target,
# so an upstream CMake is installed from the Kitware tarball.
FROM debian:stable AS dev
ENV DEBIAN_FRONTEND=noninteractive
ARG APT_MIRROR=""
RUN if [ -n "$APT_MIRROR" ]; then \
sed -i -e "s|http://deb.debian.org/debian|${APT_MIRROR}|g" \
-e "s|http://security.debian.org/debian-security|${APT_MIRROR}|g" \
/etc/apt/sources.list /etc/apt/sources.list.d/*.sources 2>/dev/null || true; \
fi
RUN --mount=type=cache,target=/var/lib/apt,sharing=locked \
--mount=type=cache,target=/var/cache/apt,sharing=locked \
for attempt in 1 2 3 4 5; do \
apt-get -o Acquire::Retries=5 update && break; \
echo "apt-get update failed (attempt $attempt), retrying"; \
sleep 5; \
done \
&& apt-get -o Acquire::Retries=5 install -y --no-install-recommends \
ca-certificates \
clang-19 \
clang-tools-19 \
curl \
dpkg-dev \
file \
libc++-19-dev \
libc++abi-19-dev \
libsdl3-dev \
libunwind-19-dev \
lld-19 \
ninja-build \
pkg-config \
python3 \
&& rm -rf /var/lib/apt/lists/*
# Upstream CMake, because Debian 13's 3.31 experimental gate does not enable the
# `import std;` support the project relies on.
ARG CMAKE_VERSION=4.2.3
RUN mkdir -p /opt/cmake \
&& curl -fL "https://github.com/Kitware/CMake/releases/download/v${CMAKE_VERSION}/cmake-${CMAKE_VERSION}-linux-x86_64.tar.gz" \
| tar -xz -C /opt/cmake --strip-components=1
ENV PATH="/opt/cmake/bin:${PATH}"
ENV CC=clang-19
ENV CXX=clang++-19
WORKDIR /work
COPY . .
RUN cmake -S . -B build/debian -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_CXX_STDLIB_MODULES_JSON=/usr/lib/llvm-19/lib/libc++.modules.json \
-DOPENRA3_DISTRO_TAG=debian13 \
&& cmake --build build/debian -j \
&& ctest --test-dir build/debian --output-on-failure \
&& (cd build/debian && cpack -G DEB)
# deploy target: runtime dependencies + final binary, minimal and fast.
FROM debian:stable AS deploy
ENV DEBIAN_FRONTEND=noninteractive
RUN --mount=type=cache,target=/var/lib/apt,sharing=locked \
--mount=type=cache,target=/var/cache/apt,sharing=locked \
apt-get -o Acquire::Retries=5 update \
&& apt-get -o Acquire::Retries=5 install -y --no-install-recommends \
libc++1-19 \
libc++abi1-19 \
libsdl3-0 \
libvulkan1 \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY --from=dev /work/build/debian/bin/openra3 /app/openra3
ENTRYPOINT ["/app/openra3"]
+86
View File
@@ -0,0 +1,86 @@
# syntax=docker/dockerfile:1
# Debian build environment for Linux on ARM64 (aarch64-linux-gnu), producing an
# arm64 .deb. Cross-compiled from amd64 with Debian's clang-19 + libc++-19 and
# the arm64 packages installed via dpkg multiarch.
FROM debian:stable AS dev
ENV DEBIAN_FRONTEND=noninteractive
ARG APT_MIRROR=""
RUN if [ -n "$APT_MIRROR" ]; then \
sed -i -e "s|http://deb.debian.org/debian|${APT_MIRROR}|g" \
-e "s|http://security.debian.org/debian-security|${APT_MIRROR}|g" \
/etc/apt/sources.list /etc/apt/sources.list.d/*.sources 2>/dev/null || true; \
fi
RUN dpkg --add-architecture arm64
RUN --mount=type=cache,target=/var/lib/apt,sharing=locked \
--mount=type=cache,target=/var/cache/apt,sharing=locked \
for attempt in 1 2 3 4 5; do \
apt-get -o Acquire::Retries=5 update && break; \
echo "apt-get update failed (attempt $attempt), retrying"; \
sleep 5; \
done \
&& apt-get -o Acquire::Retries=5 install -y --no-install-recommends \
ca-certificates \
clang-19 \
clang-tools-19 \
curl \
dpkg-dev \
file \
gcc-aarch64-linux-gnu \
libc++-19-dev:arm64 \
libc++abi-19-dev:arm64 \
libsdl3-dev:arm64 \
libunwind-19-dev:arm64 \
lld-19 \
ninja-build \
pkg-config \
python3 \
&& rm -rf /var/lib/apt/lists/*
ARG CMAKE_VERSION=4.2.3
RUN mkdir -p /opt/cmake \
&& curl -fL "https://github.com/Kitware/CMake/releases/download/v${CMAKE_VERSION}/cmake-${CMAKE_VERSION}-linux-x86_64.tar.gz" \
| tar -xz -C /opt/cmake --strip-components=1
ENV PATH="/opt/cmake/bin:${PATH}"
ENV CC=clang-19
ENV CXX=clang++-19
# Point pkg-config at the arm64 packages so SDL3 resolves to the aarch64 build.
ENV PKG_CONFIG_LIBDIR=/usr/lib/aarch64-linux-gnu/pkgconfig
WORKDIR /work
COPY . .
RUN cmake -S . -B build/debian-arm64 -G Ninja \
-DCMAKE_TOOLCHAIN_FILE=cmake/toolchains/clang-aarch64-linux-gnu.cmake \
-DCMAKE_BUILD_TYPE=Release \
-DOPENRA3_AARCH64_CC=clang-19 \
-DOPENRA3_AARCH64_CXX=clang++-19 \
-DOPENRA3_AARCH64_STDLIB_MODULES_JSON=/usr/lib/llvm-19/lib/libc++.modules.json \
-DOPENRA3_DISTRO_TAG=debian13 \
&& cmake --build build/debian-arm64 -j \
&& (cd build/debian-arm64 && cpack -G DEB)
FROM --platform=linux/arm64 debian:stable AS deploy
ENV DEBIAN_FRONTEND=noninteractive
RUN --mount=type=cache,target=/var/lib/apt,sharing=locked \
--mount=type=cache,target=/var/cache/apt,sharing=locked \
apt-get -o Acquire::Retries=5 update \
&& apt-get -o Acquire::Retries=5 install -y --no-install-recommends \
libc++1-19 \
libc++abi1-19 \
libsdl3-0 \
libvulkan1 \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY --from=dev /work/build/debian-arm64/bin/openra3 /app/openra3
ENTRYPOINT ["/app/openra3"]
+84
View File
@@ -0,0 +1,84 @@
# syntax=docker/dockerfile:1
# Linux on ARM64 cross-build environment (aarch64-linux-gnu), isolated from the
# x86_64 Linux image and the Windows images.
#
# Toolchain: clang-21 + libc++ with the arm64 libc++ (for `import std;`), the
# arm64 glibc from `gcc-aarch64-linux-gnu`, arm64 SDL3 from `libsdl3-dev:arm64`,
# and lld as the linker. The arm64 packages are installed with dpkg multiarch
# (`arm64` added to the sources); the native clang/cmake build tools stay amd64.
FROM ubuntu:26.04 AS dev
ENV DEBIAN_FRONTEND=noninteractive
ARG APT_MIRROR=""
RUN if [ -n "$APT_MIRROR" ]; then \
sed -i -e "s|http://archive.ubuntu.com/ubuntu|${APT_MIRROR}|g" \
-e "s|http://security.ubuntu.com/ubuntu|${APT_MIRROR}|g" \
-e "s|http://ports.ubuntu.com/ubuntu-ports|${APT_MIRROR}|g" \
/etc/apt/sources.list /etc/apt/sources.list.d/*.sources 2>/dev/null || true; \
fi
# Enable the arm64 architecture on top of the emulated amd64 userland.
RUN dpkg --add-architecture arm64
RUN --mount=type=cache,target=/var/lib/apt,sharing=locked \
--mount=type=cache,target=/var/cache/apt,sharing=locked \
for attempt in 1 2 3 4 5; do \
apt-get -o Acquire::Retries=5 update && break; \
echo "apt-get update failed (attempt $attempt), retrying"; \
sleep 5; \
done \
&& apt-get -o Acquire::Retries=5 install -y --no-install-recommends \
ca-certificates \
clang-21 \
cmake \
dpkg-dev \
file \
gcc-aarch64-linux-gnu \
libc++-21-dev:arm64 \
libc++abi-21-dev:arm64 \
libsdl3-dev:arm64 \
libunwind-21-dev:arm64 \
lld-21 \
ninja-build \
pkg-config \
python3 \
&& rm -rf /var/lib/apt/lists/*
ENV CC=clang-21
ENV CXX=clang++-21
# Point pkg-config at the arm64 packages so SDL3 resolves to the aarch64 build.
ENV PKG_CONFIG_LIBDIR=/usr/lib/aarch64-linux-gnu/pkgconfig
WORKDIR /work
COPY . .
RUN cmake -S . -B build/linux-arm64 -G Ninja \
-DCMAKE_TOOLCHAIN_FILE=cmake/toolchains/clang-aarch64-linux-gnu.cmake \
-DCMAKE_BUILD_TYPE=Release \
-DOPENRA3_DISTRO_TAG=ubuntu26.04 \
&& cmake --build build/linux-arm64 -j \
&& (cd build/linux-arm64 && cpack -G DEB)
# deploy target: runtime dependencies + final binary, minimal and fast. This is
# an arm64 image; build it on (or with emulation for) an arm64 host.
FROM --platform=linux/arm64 ubuntu:26.04 AS deploy
ENV DEBIAN_FRONTEND=noninteractive
RUN --mount=type=cache,target=/var/lib/apt,sharing=locked \
--mount=type=cache,target=/var/cache/apt,sharing=locked \
apt-get -o Acquire::Retries=5 update \
&& apt-get -o Acquire::Retries=5 install -y --no-install-recommends \
libc++1 \
libc++abi1 \
libsdl3-0 \
libvulkan1 \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY --from=dev /work/build/linux-arm64/bin/openra3 /app/openra3
ENTRYPOINT ["/app/openra3"]
+56
View File
@@ -0,0 +1,56 @@
# syntax=docker/dockerfile:1
# WebAssembly build environment (Emscripten), producing openra3.js/.wasm plus the
# `index.html` page and `openra3.worker.js` bootstrap. The engine runs on a plain
# Web Worker (WebGL2 and WebGPU backends), with lazy on-demand assets.
#
# The emsdk image ships CMake 3.28, whose experimental `import std` support is
# too old for the project, so an upstream CMake is layered on top.
FROM emscripten/emsdk:6.0.10 AS dev
ENV DEBIAN_FRONTEND=noninteractive
ARG APT_MIRROR=""
RUN if [ -n "$APT_MIRROR" ]; then \
sed -i -e "s|http://archive.ubuntu.com/ubuntu|${APT_MIRROR}|g" \
-e "s|http://security.ubuntu.com/ubuntu|${APT_MIRROR}|g" \
-e "s|http://ports.ubuntu.com/ubuntu-ports|${APT_MIRROR}|g" \
/etc/apt/sources.list /etc/apt/sources.list.d/*.sources 2>/dev/null || true; \
fi
RUN --mount=type=cache,target=/var/lib/apt,sharing=locked \
--mount=type=cache,target=/var/cache/apt,sharing=locked \
for attempt in 1 2 3 4 5; do \
apt-get -o Acquire::Retries=5 update && break; \
echo "apt-get update failed (attempt $attempt), retrying"; \
sleep 5; \
done \
&& apt-get -o Acquire::Retries=5 install -y --no-install-recommends \
ca-certificates \
curl \
ninja-build \
&& rm -rf /var/lib/apt/lists/*
# Upstream CMake, because the emsdk image's 3.28 cannot build the project.
ARG CMAKE_VERSION=4.2.3
RUN mkdir -p /opt/cmake \
&& curl -fL "https://github.com/Kitware/CMake/releases/download/v${CMAKE_VERSION}/cmake-${CMAKE_VERSION}-linux-x86_64.tar.gz" \
| tar -xz -C /opt/cmake --strip-components=1
ENV PATH="/opt/cmake/bin:${PATH}"
# Pre-fetch the emdawnwebgpu port into the image. The WebGPU backend needs
# <webgpu/webgpu_cpp.h>, and CMake's module dependency scanner does not process
# `--use-port`, so a warm cache is required before the project is configured.
RUN . /emsdk/emsdk_env.sh \
&& printf '#include <webgpu/webgpu_cpp.h>\nint main(){}\n' > /tmp/wgpu_port.cpp \
&& em++ -c /tmp/wgpu_port.cpp -o /tmp/wgpu_port.o --use-port=emdawnwebgpu \
&& rm -f /tmp/wgpu_port.cpp /tmp/wgpu_port.o
WORKDIR /work
COPY . .
RUN cmake -S . -B build/wasm -G Ninja \
-DCMAKE_TOOLCHAIN_FILE=cmake/toolchains/emscripten-wasm.cmake \
-DCMAKE_BUILD_TYPE=Release \
&& cmake --build build/wasm -j
+4 -1
View File
@@ -31,6 +31,7 @@ RUN --mount=type=cache,target=/var/lib/apt,sharing=locked \
curl \ curl \
ninja-build \ ninja-build \
python3 \ python3 \
wixl \
xz-utils \ xz-utils \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
@@ -54,7 +55,9 @@ COPY . .
RUN cmake -S . -B build/windows -G Ninja \ RUN cmake -S . -B build/windows -G Ninja \
-DCMAKE_TOOLCHAIN_FILE=cmake/toolchains/llvm-mingw-x86_64.cmake \ -DCMAKE_TOOLCHAIN_FILE=cmake/toolchains/llvm-mingw-x86_64.cmake \
-DCMAKE_BUILD_TYPE=Release \ -DCMAKE_BUILD_TYPE=Release \
&& cmake --build build/windows -j && cmake --build build/windows -j \
&& cmake --build build/windows --target openra3_msi \
&& (cd build/windows && cpack -G ZIP)
# package target: the .exe plus the SDL3 runtime DLL. # package target: the .exe plus the SDL3 runtime DLL.
FROM ubuntu:26.04 AS package FROM ubuntu:26.04 AS package
+77
View File
@@ -0,0 +1,77 @@
# syntax=docker/dockerfile:1
# Windows on ARM64 cross-build environment (aarch64-w64-mingw32), isolated from
# the x86_64 Windows image and the Linux images.
#
# Toolchain: the same multi-target llvm-mingw as Dockerfile.win (clang + libc++ +
# the libc++ `std` module, MSVCRT runtime), targeting aarch64. The official SDL
# release ships no MinGW ARM64 dev package, so SDL3 comes from MSYS2's clangarm64
# repository, staged into the `<triple>/{include,lib,bin}` layout the toolchain
# expects.
FROM ubuntu:26.04 AS dev
ENV DEBIAN_FRONTEND=noninteractive
ARG APT_MIRROR=""
RUN if [ -n "$APT_MIRROR" ]; then \
sed -i -e "s|http://archive.ubuntu.com/ubuntu|${APT_MIRROR}|g" \
-e "s|http://security.ubuntu.com/ubuntu|${APT_MIRROR}|g" \
-e "s|http://ports.ubuntu.com/ubuntu-ports|${APT_MIRROR}|g" \
/etc/apt/sources.list /etc/apt/sources.list.d/*.sources 2>/dev/null || true; \
fi
RUN --mount=type=cache,target=/var/lib/apt,sharing=locked \
--mount=type=cache,target=/var/cache/apt,sharing=locked \
for attempt in 1 2 3 4 5; do \
apt-get -o Acquire::Retries=5 update && break; \
echo "apt-get update failed (attempt $attempt), retrying"; \
sleep 5; \
done \
&& apt-get -o Acquire::Retries=5 install -y --no-install-recommends \
ca-certificates \
cmake \
curl \
ninja-build \
python3 \
xz-utils \
zstd \
&& rm -rf /var/lib/apt/lists/*
# Pinned toolchain + SDL3 versions (override with --build-arg to bump).
ARG LLVM_MINGW_VERSION=20260908
ARG LLVM_MINGW_FLAVOR=msvcrt
# SDL3 for aarch64-w64-mingw32 from MSYS2's clangarm64 repo (name includes the
# MSYS2 package release, hence the `-1`).
ARG MSYS2_SDL3_PKG=mingw-w64-clang-aarch64-sdl3-3.4.16-1-any.pkg.tar.zst
RUN mkdir -p /opt/llvm-mingw \
&& curl -fL "https://github.com/mstorsjo/llvm-mingw/releases/download/${LLVM_MINGW_VERSION}/llvm-mingw-${LLVM_MINGW_VERSION}-${LLVM_MINGW_FLAVOR}-ubuntu-22.04-x86_64.tar.xz" \
| tar -xJ -C /opt/llvm-mingw --strip-components=1
RUN mkdir -p /opt/sdl3-aarch64/aarch64-w64-mingw32/include /opt/sdl3-aarch64/aarch64-w64-mingw32/lib /opt/sdl3-aarch64/aarch64-w64-mingw32/bin \
&& curl -fL "https://repo.msys2.org/mingw/clangarm64/${MSYS2_SDL3_PKG}" -o /tmp/sdl3.pkg.tar.zst \
&& mkdir -p /tmp/sdl3 && tar --zstd -xf /tmp/sdl3.pkg.tar.zst -C /tmp/sdl3 \
&& cp -r /tmp/sdl3/clangarm64/include/SDL3 /opt/sdl3-aarch64/aarch64-w64-mingw32/include/ \
&& cp /tmp/sdl3/clangarm64/lib/libSDL3.dll.a /opt/sdl3-aarch64/aarch64-w64-mingw32/lib/ \
&& cp /tmp/sdl3/clangarm64/bin/SDL3.dll /opt/sdl3-aarch64/aarch64-w64-mingw32/bin/ \
&& rm -rf /tmp/sdl3 /tmp/sdl3.pkg.tar.zst
ENV OPENRA3_LLVM_MINGW_ROOT=/opt/llvm-mingw
ENV OPENRA3_SDL3_ROOT=/opt/sdl3-aarch64
WORKDIR /work
COPY . .
RUN cmake -S . -B build/windows-arm64 -G Ninja \
-DCMAKE_TOOLCHAIN_FILE=cmake/toolchains/llvm-mingw-aarch64.cmake \
-DCMAKE_BUILD_TYPE=Release \
&& cmake --build build/windows-arm64 -j \
&& (cd build/windows-arm64 && cpack -G ZIP)
# package target: the .exe plus the SDL3 runtime DLL.
FROM ubuntu:26.04 AS package
WORKDIR /app
COPY --from=dev /work/build/windows-arm64/bin/openra3.exe /app/
COPY --from=dev /work/build/windows-arm64/bin/SDL3.dll /app/
+114 -16
View File
@@ -18,19 +18,23 @@ Ghidra.
## Status ## Status
OpenRA3 is at **v0.4.0**. The engine compiles and runs headless, and a **minimal OpenRA3 is at **v0.8.0**. The engine compiles and runs headless, and a **minimal
skirmish** is playable: it reads a real multiplayer map out of your install, skirmish** is playable: it reads a real multiplayer map out of your install,
recovers the player start waypoints, and simulates two sides building a base, recovers the player start waypoints, and simulates two sides building a base,
extracting ore and fighting until one side is wiped out. The balance is the extracting ore and fighting until one side is wiped out. The balance is the
**retail Red Alert 3 balance**, pinned in `ra3.data` from EA's open RA3 XML: **retail Red Alert 3 balance**, pinned in `ra3.data` from EA's open RA3 XML:
damage types, `ArmorTemplate` percentages, weapon target masks, build costs and damage types, `ArmorTemplate` percentages, weapon target masks, build costs and
the ore economy. Presentation offers **Vulkan** (`ra3.vulkan`) and **Direct3D 11 / 12** the ore economy. Presentation offers **Vulkan** (`ra3.vulkan`), **Direct3D 11 / 12**
(`ra3.dx`) GPU backends with an SDL software blit and null fallbacks; the (`ra3.dx`), **WebGPU** (`ra3.webgpu`, the wasm worker backend) and **WebGL 2**
backend is selectable from the in-game menu and the software renderer still (`ra3.wasmgl`, the SDL-free wasm worker fallback) GPU backends with an SDL
produces headless images. Terrain tiles software blit and null fallbacks; the backend is selectable from the in-game menu
and the software renderer still produces headless images. Terrain tiles
cross-fade the way the retail `Terrain.fx` does (a per-cell blend ramp plus a cross-fade the way the retail `Terrain.fx` does (a per-cell blend ramp plus a
gutter-padded atlas), so material boundaries are smooth instead of a grid of gutter-padded atlas), so material boundaries are smooth instead of a grid of
hard lines. An in-window **menu** lists the maps by their localized name and hard lines; the **buildings and props a map places** are decoded from the retail
compiled `W3DMesh` art (the uncompressed `worldbuilder.bin` stream and its
embedded DDS textures) and drawn with a depth test over the terrain. An
in-window **menu** lists the maps by their localized name and
exposes every render/skirmish option for tweaking before launch. The whole tree exposes every render/skirmish option for tweaking before launch. The whole tree
builds for **Linux** (clang + libc++) and cross-compiles to **Windows** builds for **Linux** (clang + libc++) and cross-compiles to **Windows**
(`openra3.exe` + `SDL3.dll`) with llvm-mingw — both using C++26 modules and (`openra3.exe` + `SDL3.dll`) with llvm-mingw — both using C++26 modules and
@@ -85,17 +89,20 @@ built-in test map so the project still builds and runs in CI.
| `src/core/ra3.core.cppm` | fundamental types, math, strings, random, message stream | | `src/core/ra3.core.cppm` | fundamental types, math, strings, random, message stream |
| `src/logic/ra3.logic.cppm` | objects, players, teams, spatial partition, game loop | | `src/logic/ra3.logic.cppm` | objects, players, teams, spatial partition, game loop |
| `src/client/ra3.client.cppm` | `display` abstraction + shared interactive loops + client facade | | `src/client/ra3.client.cppm` | `display` abstraction + shared interactive loops + client facade |
| `src/display/ra3.display.cppm` | picks the backend (Vulkan/D3D11/D3D12, then SDL) for the app | | `src/display/ra3.display.cppm` | picks the backend (Vulkan/D3D11/D3D12/WebGL, then SDL) for the app |
| `src/game/ra3.game.cppm` | RA3 sides, player templates, skirmish defaults | | `src/game/ra3.game.cppm` | RA3 sides, player templates, skirmish defaults |
| `src/data/ra3.data.cppm` | retail RA3 balance: damage types, armour, weapons, units, economy | | `src/data/ra3.data.cppm` | retail RA3 balance: damage types, armour, weapons, units, economy |
| `src/fs/ra3.fs.cppm` | `BIG4` archives, RefPack codec, local install locator | | `src/fs/ra3.fs.cppm` | `BIG4` archives, RefPack codec, local install locator |
| `src/map/ra3.map.cppm` | map catalog, `EAR`/RefPack unwrap, start waypoints, `gamestrings.csf` display names | | `src/map/ra3.map.cppm` | map catalog, `EAR`/RefPack unwrap, start waypoints, `gamestrings.csf` display names |
| `src/skirmish/ra3.skirmish.cppm` | base building, economy, AI, combat, win condition | | `src/skirmish/ra3.skirmish.cppm` | base building, economy, AI, combat, win condition |
| `src/terrain/ra3.terrain.cppm` | `CkMp` terrain: `HeightMapData`, `BlendTileData` (tiles + blends), `Terrain.big` tiles | | `src/terrain/ra3.terrain.cppm` | `CkMp` terrain: `HeightMapData`, `BlendTileData` (tiles + blends), `Terrain.big` tiles |
| `src/models/ra3.models.cppm` | compiled `W3DMesh` art (BAB stream + DDS textures) for the objects a map places |
| `src/render/ra3.render.cppm` | ARGB framebuffer, TGA decode, BMP encode, map compositing, bitmap-font text | | `src/render/ra3.render.cppm` | ARGB framebuffer, TGA decode, BMP encode, map compositing, bitmap-font text |
| `src/ui/ra3.ui.*.cppm` | SDL3 window viewer and menu (null backend when SDL3 is absent) | | `src/ui/ra3.ui.*.cppm` | SDL3 window viewer and menu (null backend when SDL3 is absent) |
| `src/vulkan/ra3.vulkan.*.cppm` | Vulkan presentation backend and menu (null fallback without a loader) | | `src/vulkan/ra3.vulkan.*.cppm` | Vulkan presentation backend and menu (null fallback without a loader) |
| `src/dx/ra3.dx.*.cppm` | Direct3D 11/12 presentation backend (runtime HLSL; null fallback off Windows) | | `src/dx/ra3.dx.*.cppm` | Direct3D 11/12 presentation backend (runtime HLSL; null fallback off Windows) |
| `src/wasmgl/ra3.wasmgl.*.cppm` | SDL-free wasm backend: engine on a Web Worker, WebGL2 on an OffscreenCanvas, lazy assets (null fallback off Emscripten) |
| `src/webgpu/ra3.webgpu.*.cppm` | WebGPU wasm backend (Dawn `emdawnwebgpu`, WGSL; the default on wasm, falls back to `ra3.wasmgl`) |
| `third_party/libenderlog/` | vendored C++26 module logger (`import ender.log;`, MIT) with a native stack fallback for libc++ | | `third_party/libenderlog/` | vendored C++26 module logger (`import ender.log;`, MIT) with a native stack fallback for libc++ |
| `src/ra3.cppm` | umbrella module re-exporting the SDK | | `src/ra3.cppm` | umbrella module re-exporting the SDK |
| `apps/openra3/main.cpp` | `menu` / `maps` / `skirmish` / `render` CLI | | `apps/openra3/main.cpp` | `menu` / `maps` / `skirmish` / `render` CLI |
@@ -122,13 +129,32 @@ their compilers and standard libraries never interfere.
The host needs no toolchain: each target builds inside its own image. The host needs no toolchain: each target builds inside its own image.
```bash ```bash
# Linux -> build/linux/bin/openra3 # Linux (amd64) -> build/linux/bin/openra3 + .deb
scripts/build-linux.sh scripts/build-linux.sh
# Windows -> build/windows/bin/openra3.exe (+ SDL3.dll) # Linux/arm64 -> build/linux-arm64/bin/openra3 + .deb (cross)
scripts/build-linux-arm64.sh
# Debian (amd64) -> build/debian/bin/openra3 + .deb
scripts/build-debian.sh
# Debian (arm64) -> build/debian-arm64/bin/openra3 + .deb (cross)
scripts/build-debian-arm64.sh
# Windows (x86_64) -> build/windows/bin/openra3.exe + SDL3.dll, .zip, .msi
scripts/build-windows.sh scripts/build-windows.sh
# Windows (arm64) -> build/windows-arm64/bin/openra3.exe + SDL3.dll, .zip (cross)
scripts/build-windows-arm64.sh
# WebAssembly -> build/wasm/bin/index.html (+ openra3.js/.wasm, openra3.worker.js)
scripts/build-wasm.sh
``` ```
Every target is cross-built from x86_64 Linux in its own image: llvm-mingw for
the Windows targets, clang + libc++ with dpkg multiarch for the arm64 targets,
and Debian/Ubuntu-specific images for the `.deb` packages.
Or drive Docker directly: Or drive Docker directly:
```bash ```bash
@@ -160,16 +186,74 @@ openra3.exe render --game-dir "C:\Red Alert 3" --vulkan
Vulkan is provided by **vendored volk + headers** (`third_party/`), resolved at Vulkan is provided by **vendored volk + headers** (`third_party/`), resolved at
runtime, so neither image needs a Vulkan SDK. runtime, so neither image needs a Vulkan SDK.
## WebAssembly
OpenRA3 also builds to **WebAssembly** with Emscripten (`scripts/build-wasm.sh`),
producing `openra3.js` + `.wasm` plus the `index.html` page and
`openra3.worker.js` bootstrap. The browser gets the same GPU renderer as the
desktop builds: the engine runs on a **Web Worker** (no SDL, no blocking of the
page) and draws into an **OffscreenCanvas** handed over by the page. The default
wasm backend is **`ra3.webgpu`** (WebGPU, WGSL shaders via Emscripten's
`emdawnwebgpu` port); it falls back to **`ra3.wasmgl`** (WebGL2, the GLSL ES
ports of the desktop shaders) when WebGPU is unavailable. The page keeps the DOM
(input, resize) and forwards events to the worker, so the main thread stays
responsive.
Because the runtime's main thread lives in the worker, its filesystem is local
to it and assets load **on demand**: point `OPENRA3_WEB_ASSETS` at the extracted
asset tree and the build embeds an `assets.manifest.json`; at startup the worker
registers every listed file as a lazy file, so the browser fetches a file only
when the engine first opens it (entering a map pulls just that map and the tiles
it uses, not the whole tree).
```bash
# build (generates the manifest from the extracted assets, embeds it)
OPENRA3_WEB_ASSETS=/path/to/assets scripts/build-wasm.sh
# serve the build output (Range-capable, COOP/COEP; --assets is the lazy tree)
python3 apps/web/serve.py --root build/wasm/bin --assets /path/to/assets
# open http://localhost:8199/index.html
```
The engine's frame loops are blocking; the wasm build yields to the worker's
event loop via Asyncify (`display::sleep_frame` calls `emscripten_sleep`), so
the page repaints and handles input. Log records go to `console.log` at their
real level (the console sink uses stdout on the web instead of stderr). Without
`OPENRA3_WEB_ASSETS` the module still loads and starts, but exits at the menu
because no maps are found.
## Packages
CPack produces the release artifacts; the Docker images and CI run it for every
target.
| Target | Portable | Installer |
| --- | --- | --- |
| Linux amd64 (Ubuntu 26.04) | `.tar.gz` | `openra3_<v>_amd64ubuntu26.04.deb` |
| Linux arm64 (Ubuntu 26.04) | `.tar.gz` | `openra3_<v>_arm64ubuntu26.04.deb` |
| Linux amd64 (Debian 13) | `.tar.gz` | `openra3_<v>_amd64debian13.deb` |
| Linux arm64 (Debian 13) | `.tar.gz` | `openra3_<v>_arm64debian13.deb` |
| Windows x86_64 | `openra3-<v>-windows-x86_64.zip` | `openra3-<v>-windows-x86_64.msi` |
| Windows arm64 | `openra3-<v>-windows-arm64.zip` | (WiX-only; see below) |
The `.deb` dependencies are derived from the binary with `dpkg-shlibdeps`. The
`.msi` is built with **wixl** (msitools) straight from the Linux cross image;
wixl 0.106 has no arm64 support, so Windows/ARM64 ships the portable `.zip`
(the MSI toolchain for ARM64 would be WiX v4 via the .NET SDK).
## Logging ## Logging
Every run writes `openra3.log` next to the executable through the vendored Every run writes `openra3.log` in a per-user `logs/` folder — on Windows
`%LOCALAPPDATA%\OpenRA3\logs`, elsewhere `$XDG_STATE_HOME/openra3/logs`
(falling back to `~/.local/state/openra3/logs`) — created on first use and kept
out of the binary's own directory, through the vendored
[`libenderlog`](third_party/libenderlog) module (`import ender.log;`). A file [`libenderlog`](third_party/libenderlog) module (`import ender.log;`). A file
sink archives the previous log to `openra3.log.<YYYYmmdd-HHMMSS>` on open, so sink archives the previous log to `openra3.<YYYYmmdd-HHMMSS>.log` on open, so
each run gets its own file; the active file rotates at 4 MiB and the last 10 each run gets its own file; the active file rotates at 4 MiB and the last 10
archives are kept. Records at **`warn` and above** carry a call stack (Windows archives are kept. Records at **`warn` and above** carry a call stack (Windows
`CaptureStackBackTrace` / POSIX `execinfo`, because libc++ has no `CaptureStackBackTrace` / POSIX `execinfo`, because libc++ has no
`<stacktrace>`). A hard crash also writes `openra3_crash.log` with the faulting `<stacktrace>`). A hard crash also writes `openra3_crash.log` alongside it with
module and a raw backtrace. the faulting module and a raw backtrace.
## Running a skirmish ## Running a skirmish
@@ -213,9 +297,23 @@ textures from `Data\Terrain.big` (RefPack + TGA), and rasterises the map with an
elevation shade. Each cell cross-fades into its blend neighbour with the same elevation shade. Each cell cross-fades into its blend neighbour with the same
linear ramp the retail `Terrain.fx` uses, so material transitions are smooth; on linear ramp the retail `Terrain.fx` uses, so material transitions are smooth; on
the GPU path the tile atlas is padded with a replicated gutter so filtering the GPU path the tile atlas is padded with a replicated gutter so filtering
never bleeds between tiles. Match state is overlaid (start markers in yellow, never bleeds between tiles. It also draws the **buildings and props the map
player 0 in blue, player 1 in red). `--thumbnail` uses the old `<map>_art.tga` places**: the `ObjectsList` chunk is decoded into `(type, position, angle)`
overview instead. instances, the type resolves to its compiled `W3DMesh` parts in the map's art
stream (`Data\WBData.big`'s uncompressed `worldbuilder.bin`), the embedded DDS
textures are decoded, bind-pose skinning places each mesh's bone-space vertices
(`W3DHierarchy`), and the scene is flattened into one world-space triangle soup.
The terrain raymarcher writes a depth value so the model pass depth-tests
against the relief; the GPU path draws it in a second pipeline (shared camera
basis, with a small depth bias so ground decals do not z-fight) and the software
path rasterises it with a z-buffer. Match state is
overlaid (start markers in yellow, player 0 in blue, player 1 in red).
`--thumbnail` uses the old `<map>_art.tga` overview instead.
> **Roads / sidewalks** are drawn too: the map stores each as a
> `RoadType::Start`/`End` pair of objects, and `render` emits a flat textured
> ribbon along each segment, lifted onto the terrain with a small depth bias so
> it does not clip into the ground.
**Offscreen image** (works anywhere, no display needed): **Offscreen image** (works anywhere, no display needed):
+1 -1
View File
@@ -1 +1 @@
0.4.0 0.10.1
+234 -27
View File
@@ -10,14 +10,56 @@ import ender.log;
namespace { namespace {
#if defined(_WIN32) #if defined(_WIN32)
/** Path of the crash report written next to the executable. */ /** `%LOCALAPPDATA%` as a wide path, or empty when the variable is unset. */
[[nodiscard]] auto crash_log_path() -> const std::filesystem::path & { auto local_appdata() -> std::filesystem::path {
static const auto path = [] { const DWORD needed = GetEnvironmentVariableW(L"LOCALAPPDATA", nullptr, 0U);
std::wstring buffer(32768U, L'\0'); if (needed == 0U || needed > 32768U) return {};
const DWORD length = GetModuleFileNameW(nullptr, buffer.data(), static_cast<DWORD>(buffer.size())); std::wstring buffer(needed, L'\0');
buffer.resize(length); const DWORD written = GetEnvironmentVariableW(L"LOCALAPPDATA", buffer.data(), needed);
return std::filesystem::path{buffer}.parent_path() / L"openra3_crash.log"; if (written == 0U || written >= needed) return {};
buffer.resize(written);
return std::filesystem::path{buffer};
}
#endif
/**
* Directory that holds per-run logs and crash reports: a `logs` folder
* under the platform's per-user state location, created on first use. It
* deliberately does not sit beside the executable, which may be read-only
* or a shared build tree.
*
* Windows: `%LOCALAPPDATA%\OpenRA3\logs`; elsewhere
* `$XDG_STATE_HOME/openra3/logs` (falling back to
* `~/.local/state/openra3/logs`).
*/
[[maybe_unused]] auto log_directory() -> const std::filesystem::path & {
static const auto directory = [] {
#if defined(_WIN32)
auto base = local_appdata();
if (base.empty()) base = std::filesystem::temp_directory_path();
base /= L"OpenRA3";
#else
std::filesystem::path base;
if (const char *state = std::getenv("XDG_STATE_HOME"); state != nullptr && *state != '\0')
base = state;
else if (const char *home = std::getenv("HOME"); home != nullptr && *home != '\0')
base = std::filesystem::path{home} / ".local" / "state";
else
base = std::filesystem::temp_directory_path();
base /= "openra3";
#endif
auto result = base / "logs";
std::error_code ec;
std::filesystem::create_directories(result, ec);
return result;
}(); }();
return directory;
}
#if defined(_WIN32)
/** Path of the crash report, under the per-user `logs` folder. */
[[nodiscard]] auto crash_log_path() -> const std::filesystem::path & {
static const auto path = log_directory() / L"openra3_crash.log";
return path; return path;
} }
@@ -92,10 +134,17 @@ namespace {
* timestamped file on open, so every run gets its own log and the previous * timestamped file on open, so every run gets its own log and the previous
* run's log is preserved. * run's log is preserved.
*/ */
auto setup_logging(const std::filesystem::path &exe_dir) -> void { auto setup_logging() -> void {
namespace log = ender::log; namespace log = ender::log;
log::configure({.minimum = log::level::info, .stacktrace_from = log::level::warn}); log::configure({.minimum = log::level::info, .stacktrace_from = log::level::warn});
log::add_file_sink(exe_dir / "openra3.log", {.max_file_size = 4U * 1024U * 1024U, .max_archives = 10U}); #if defined(__EMSCRIPTEN__)
// The default sink writes to stderr, which the browser maps to
// console.error regardless of level; use stdout so INFO/WARN appear at
// their real level. There is no file sink on the web.
log::set_sinks({std::make_shared<log::console_sink>(std::cout)});
#else
log::add_file_sink(log_directory() / "openra3.log", {.max_file_size = 4U * 1024U * 1024U, .max_archives = 10U});
#endif
log::info("OpenRA3 started"); log::info("OpenRA3 started");
} }
@@ -106,13 +155,14 @@ namespace {
std::puts(" openra3 menu (pick a map, then view it)"); std::puts(" openra3 menu (pick a map, then view it)");
std::puts(" openra3 menu-preview [--out FILE.bmp] (headless render of the menu)"); std::puts(" openra3 menu-preview [--out FILE.bmp] (headless render of the menu)");
std::puts(" openra3 maps"); std::puts(" openra3 maps");
std::puts(" openra3 textures --map ID (loose terrain TGAs a map uses)");
std::puts(" openra3 skirmish [--map ID] [--frames N] [--seed N]"); std::puts(" openra3 skirmish [--map ID] [--frames N] [--seed N]");
std::puts(" openra3 render [--map ID] [--frames N] [--seed N] [--zoom Z]"); std::puts(" openra3 render [--map ID] [--frames N] [--seed N] [--zoom Z]");
std::puts(" [--out FILE.bmp] [--vulkan] [--no-window] [--no-sim] [--thumbnail]"); std::puts(" [--out FILE.bmp] [--vulkan] [--no-window] [--no-sim] [--thumbnail]");
std::puts(" [--3d] [--cam-pitch DEG] [--cam-yaw DEG] [--cam-x U] [--cam-y U]"); std::puts(" [--3d] [--cam-pitch DEG] [--cam-yaw DEG] [--cam-x U] [--cam-y U]");
std::puts(" [--cam-height U] [--fov DEG] [--width W] [--height H]"); std::puts(" [--cam-height U] [--fov DEG] [--width W] [--height H]");
std::puts(" [--fullscreen] [--fps N] (0 = vsync)"); std::puts(" [--fullscreen] [--fps N] (0 = vsync)");
std::puts(" [--vulkan] [--dx11] [--dx12] [--sdl] (preferred display backend)"); std::puts(" [--vulkan] [--dx11] [--dx12] [--webgpu] [--wasmgl] [--sdl] (preferred display backend)");
std::puts("assets: read from <exe_dir>/assets (extracted at build time by the openra3_assets target)"); std::puts("assets: read from <exe_dir>/assets (extracted at build time by the openra3_assets target)");
} }
@@ -131,6 +181,8 @@ namespace {
[[nodiscard]] auto display_backend_from_args(const std::vector<std::string> &args) -> ra3::display::backend { [[nodiscard]] auto display_backend_from_args(const std::vector<std::string> &args) -> ra3::display::backend {
if (has_flag(args, "--dx11") || has_flag(args, "--d3d11")) return ra3::display::backend::d3d11; if (has_flag(args, "--dx11") || has_flag(args, "--d3d11")) return ra3::display::backend::d3d11;
if (has_flag(args, "--dx12") || has_flag(args, "--d3d12")) return ra3::display::backend::d3d12; if (has_flag(args, "--dx12") || has_flag(args, "--d3d12")) return ra3::display::backend::d3d12;
if (has_flag(args, "--webgpu")) return ra3::display::backend::webgpu;
if (has_flag(args, "--wasmgl")) return ra3::display::backend::wasmgl;
if (has_flag(args, "--sdl")) return ra3::display::backend::sdl; if (has_flag(args, "--sdl")) return ra3::display::backend::sdl;
return ra3::display::backend::vulkan; return ra3::display::backend::vulkan;
} }
@@ -139,7 +191,9 @@ namespace {
auto executable_dir(const char *argv0) -> std::filesystem::path { auto executable_dir(const char *argv0) -> std::filesystem::path {
std::error_code ec; std::error_code ec;
const auto path = std::filesystem::absolute(argv0, ec); const auto path = std::filesystem::absolute(argv0, ec);
return ec ? std::filesystem::current_path() : path.parent_path(); // lexically_normal drops the "." element Emscripten leaves in argv[0]
// ("/./openra3.js" -> "/"), which the wasm FS does not collapse itself.
return ec ? std::filesystem::current_path() : path.parent_path().lexically_normal();
} }
auto read_file(const std::filesystem::path &path) -> std::vector<ra3::core::uint8> { auto read_file(const std::filesystem::path &path) -> std::vector<ra3::core::uint8> {
@@ -177,6 +231,32 @@ namespace {
return maps; return maps;
} }
/** Case-insensitive ASCII ordering (`a` before `b`). */
auto name_less(std::string_view a, std::string_view b) -> bool {
const auto length = std::min(a.size(), b.size());
for (std::size_t i = 0; i < length; ++i) {
const auto ca = static_cast<unsigned char>(std::tolower(static_cast<unsigned char>(a[i])));
const auto cb = static_cast<unsigned char>(std::tolower(static_cast<unsigned char>(b[i])));
if (ca != cb) return ca < cb;
}
return a.size() < b.size();
}
/**
* Order the map list the way the retail skirmish screen does: alphabetically
* by the localized display name (e.g. "Battlebase Beta" before "Cabana
* Republic"), falling back to the id for entries with equal names.
*/
auto sort_maps_by_name(std::vector<asset_map> &maps, const ra3::map::map_name_table &names) -> void {
std::sort(maps.begin(), maps.end(), [&](const asset_map &a, const asset_map &b) {
const auto name_a = names.lookup(a.id);
const auto name_b = names.lookup(b.id);
if (name_less(name_a, name_b)) return true;
if (name_less(name_b, name_a)) return false;
return a.id < b.id;
});
}
/** Find `<id>_art.tga` anywhere under `root`. */ /** Find `<id>_art.tga` anywhere under `root`. */
auto find_art(const std::filesystem::path &root, std::string_view id) -> std::optional<std::filesystem::path> { auto find_art(const std::filesystem::path &root, std::string_view id) -> std::optional<std::filesystem::path> {
const auto want = std::string{id} + "_art.tga"; const auto want = std::string{id} + "_art.tga";
@@ -187,6 +267,83 @@ namespace {
return std::nullopt; return std::nullopt;
} }
/** A map's compiled art stream (`map.manifest` + `map.bin`). */
struct model_stream_paths {
std::filesystem::path manifest;
std::filesystem::path bin;
};
/**
* Locate the compiled art stream that holds a map's buildings/props.
*
* Retail bakes the shared static art (the props a map places) into
* `Data\WBData.big`'s uncompressed `worldbuilder.bin`, so that stream is
* preferred; the per-map `map.bin` (which links to it) is the fallback.
*/
auto find_map_stream(const std::filesystem::path &root, std::string_view id) -> std::optional<model_stream_paths> {
std::error_code ec;
const auto consider = [&](const std::filesystem::path &dir) -> std::optional<model_stream_paths> {
const auto manifest = dir / "map.manifest";
const auto bin = dir / "map.bin";
if (std::filesystem::exists(manifest, ec) && std::filesystem::exists(bin, ec)) return model_stream_paths{manifest, bin};
return std::nullopt;
};
// A `worldbuilder` stream (the full prop art) takes precedence.
for (const auto &dir: {root / "models", root / "raw" / "WBData" / "data", root}) {
const auto manifest = dir / "worldbuilder.manifest";
const auto bin = dir / "worldbuilder.bin";
if (std::filesystem::exists(manifest, ec) && std::filesystem::exists(bin, ec)) return model_stream_paths{manifest, bin};
}
for (const auto &entry: std::filesystem::recursive_directory_iterator(root, ec)) {
if (entry.is_regular_file() && entry.path().filename() == "worldbuilder.bin") {
const auto manifest = entry.path().parent_path() / "worldbuilder.manifest";
if (std::filesystem::exists(manifest, ec)) return model_stream_paths{manifest, entry.path()};
}
}
if (auto found = consider(root / "models" / id); found) return found;
if (auto found = consider(root / "maps" / id); found) return found;
for (const auto &entry: std::filesystem::recursive_directory_iterator(root, ec)) {
if (!entry.is_directory() || entry.path().filename().string() != id) continue;
if (auto found = consider(entry.path()); found) return found;
}
return std::nullopt;
}
/** Build the map's building/prop scene from its compiled art stream. */
auto build_object_scene(const std::filesystem::path &assets, std::string_view id, const ra3::terrain::map_data &terrain,
const ra3::terrain::render_options &options, std::span<const ra3::core::uint8> ckmp) -> ra3::models::scene {
ra3::models::scene scene;
try {
const auto paths = find_map_stream(assets, id);
if (!paths) {
std::puts("objects: no compiled art stream found");
return scene;
}
const auto stream = ra3::models::asset_stream::load_files(paths->manifest, paths->bin);
std::vector<ra3::models::placement> placements;
for (const auto &object: ra3::map::parse_objects(ckmp)) {
placements.push_back({object.type, object.x, object.y, object.z, object.angle, object.scale, object.road_type});
}
const auto world_w = terrain.world_width();
const auto world_h = terrain.world_height();
// An opaque water surface hides anything below it, so cull objects
// and roads submerged under the map's water plane (sunken ships,
// underwater props, ...) instead of drawing them on top of the sea.
const auto cull_below_z = terrain.has_water ? terrain.water_plane_z : -3.4e38F;
scene = ra3::models::build_scene(stream, placements, [&](float x, float y) -> float {
if (x < 0.0F || y < 0.0F || x >= world_w || y >= world_h) return 0.0F;
const auto cx = std::min(terrain.width - 1U, static_cast<ra3::core::uint32>(x / ra3::terrain::cell_size));
const auto cy = std::min(terrain.height - 1U, static_cast<ra3::core::uint32>((world_h - y) / ra3::terrain::cell_size));
return static_cast<float>(terrain.elevation(cx, cy)) * options.z_scale;
}, 128U, cull_below_z);
std::printf("objects: %zu placed, %zu missing, %zu hidden, %zu road segments, %zu triangles, %zu textures (%s)\n", scene.placed, scene.missing,
scene.hidden, scene.roads, scene.triangle_count(), scene.textures.size(), paths->bin.filename().string().c_str());
} catch (const std::exception &error) {
std::printf("objects: failed to build scene (%s)\n", error.what());
}
return scene;
}
/** Write `<assets>/maps/map_names.tsv` from the install's localized string table. */ /** Write `<assets>/maps/map_names.tsv` from the install's localized string table. */
auto write_map_names(const std::filesystem::path &data_dir, const std::filesystem::path &assets) -> bool { auto write_map_names(const std::filesystem::path &data_dir, const std::filesystem::path &assets) -> bool {
try { try {
@@ -296,10 +453,37 @@ namespace {
return ready; return ready;
} }
/** Print the loose terrain TGA files a map resolves to (for a wasm preload). */
auto command_textures(const std::vector<std::string> &args, const std::filesystem::path &assets) -> int {
if (!ensure_assets(assets)) return 1;
const auto requested = option_value(args, "--map");
if (!requested) {
std::puts("usage: openra3 textures --map ID");
return 2;
}
const auto maps = list_asset_maps(assets);
const asset_map *picked = nullptr;
for (const auto &map: maps) {
if (map.id == *requested) picked = &map;
}
if (picked == nullptr) {
std::printf("no map '%s'\n", requested->c_str());
return 2;
}
const auto parsed = ra3::terrain::parse_map(ra3::map::to_ckmp(read_file(picked->file)));
const auto files = ra3::terrain::resolve_texture_files(parsed, assets / "terrain");
std::printf("map %s: %zu resolved textures\n", picked->id.c_str(), files.size());
for (const auto &file: files) {
std::printf("%s\n", file.string().c_str());
}
return 0;
}
auto command_maps(const std::filesystem::path &assets) -> int { auto command_maps(const std::filesystem::path &assets) -> int {
if (!ensure_assets(assets)) return 1; if (!ensure_assets(assets)) return 1;
const auto maps = list_asset_maps(assets); auto maps = list_asset_maps(assets);
const auto names = ra3::map::load_map_names(assets); const auto names = ra3::map::load_map_names(assets);
sort_maps_by_name(maps, names);
std::printf("maps: %zu\n", maps.size()); std::printf("maps: %zu\n", maps.size());
for (const auto &m: maps) { for (const auto &m: maps) {
std::error_code ec; std::error_code ec;
@@ -312,8 +496,9 @@ namespace {
auto command_skirmish(const std::vector<std::string> &args, const std::filesystem::path &assets) -> int { auto command_skirmish(const std::vector<std::string> &args, const std::filesystem::path &assets) -> int {
using namespace ra3; using namespace ra3;
if (!ensure_assets(assets)) return 1; if (!ensure_assets(assets)) return 1;
const auto maps = list_asset_maps(assets); auto maps = list_asset_maps(assets);
if (maps.empty()) return 1; if (maps.empty()) return 1;
sort_maps_by_name(maps, map::load_map_names(assets));
const auto requested = option_value(args, "--map"); const auto requested = option_value(args, "--map");
const asset_map *picked = &maps.front(); const asset_map *picked = &maps.front();
if (requested) { if (requested) {
@@ -351,8 +536,10 @@ namespace {
auto command_render(const std::vector<std::string> &args, const std::filesystem::path &assets) -> int { auto command_render(const std::vector<std::string> &args, const std::filesystem::path &assets) -> int {
using namespace ra3; using namespace ra3;
if (!ensure_assets(assets)) return 1; if (!ensure_assets(assets)) return 1;
const auto maps = list_asset_maps(assets); auto maps = list_asset_maps(assets);
if (maps.empty()) return 1; if (maps.empty()) return 1;
const auto names = map::load_map_names(assets);
sort_maps_by_name(maps, names);
const auto requested = option_value(args, "--map"); const auto requested = option_value(args, "--map");
const asset_map *picked = &maps.front(); const asset_map *picked = &maps.front();
if (requested) { if (requested) {
@@ -361,7 +548,6 @@ namespace {
} }
render::scene_options scene; render::scene_options scene;
const auto names = map::load_map_names(assets);
scene.title = "OpenRA3 - " + names.lookup(picked->id); scene.title = "OpenRA3 - " + names.lookup(picked->id);
if (const auto size = option_value(args, "--world-size")) scene.world_width = scene.world_height = std::stod(*size); if (const auto size = option_value(args, "--world-size")) scene.world_width = scene.world_height = std::stod(*size);
@@ -426,8 +612,9 @@ namespace {
const bool offscreen = option_value(args, "--out").has_value() || has_flag(args, "--no-window"); const bool offscreen = option_value(args, "--out").has_value() || has_flag(args, "--no-window");
bool gpu_shown = false; bool gpu_shown = false;
const auto objects = build_object_scene(assets, picked->id, terrain, terrain_options, bytes);
if (!offscreen) { if (!offscreen) {
const auto gpu = terrain::build_gpu_terrain(terrain, textures, terrain_options); const auto gpu = terrain::build_gpu_terrain(terrain, textures, terrain_options, objects);
ra3::client::display_options options; ra3::client::display_options options;
options.title = scene.title; options.title = scene.title;
options.width = static_cast<int>(width); options.width = static_cast<int>(width);
@@ -447,7 +634,7 @@ namespace {
} }
if (offscreen) { if (offscreen) {
// Offscreen: the software raymarcher produces the 3D image. // Offscreen: the software raymarcher produces the 3D image.
composed = terrain::render3d(terrain, textures, camera, width, height, terrain_options); composed = terrain::render3d(terrain, textures, camera, width, height, terrain_options, &objects);
perspective = true; perspective = true;
} else { } else {
// Windowed without a GPU: the cheap top-down raster. // Windowed without a GPU: the cheap top-down raster.
@@ -574,9 +761,12 @@ namespace {
* then the others in turn, so an unavailable backend degrades gracefully. * then the others in turn, so an unavailable backend degrades gracefully.
* Direct3D is Windows-only; on other hosts those entries fall through. * Direct3D is Windows-only; on other hosts those entries fall through.
*/ */
inline constexpr std::array<ra3::display::backend, 4> renderer_backends{ra3::display::backend::vulkan, ra3::display::backend::d3d11, inline constexpr std::array<ra3::display::backend, 6> renderer_backends{ra3::display::backend::vulkan, ra3::display::backend::d3d11,
ra3::display::backend::d3d12, ra3::display::backend::sdl}; ra3::display::backend::d3d12, ra3::display::backend::webgpu,
inline constexpr std::array<std::string_view, 4> renderer_names{"Vulkan", "Direct3D 11", "Direct3D 12", "SDL (software)"}; ra3::display::backend::wasmgl, ra3::display::backend::sdl};
inline constexpr std::array<std::string_view, 6> renderer_names{"Vulkan", "Direct3D 11", "Direct3D 12", "WebGPU", "WebGL (worker)",
"SDL (software)"};
inline constexpr int renderer_count = static_cast<int>(renderer_backends.size());
[[nodiscard]] inline auto renderer_backend(int index) -> ra3::display::backend { [[nodiscard]] inline auto renderer_backend(int index) -> ra3::display::backend {
return renderer_backends[static_cast<std::size_t>(std::clamp(index, 0, static_cast<int>(renderer_backends.size()) - 1))]; return renderer_backends[static_cast<std::size_t>(std::clamp(index, 0, static_cast<int>(renderer_backends.size()) - 1))];
@@ -661,7 +851,7 @@ namespace {
case 15: return s.fullscreen ? "yes" : "no"; case 15: return s.fullscreen ? "yes" : "no";
case 16: return std::string{fps_names[static_cast<std::size_t>(std::clamp(s.fps_index, 0, 4))]} + " fps"; case 16: return std::string{fps_names[static_cast<std::size_t>(std::clamp(s.fps_index, 0, 4))]} + " fps";
case 17: return "start"; case 17: return "start";
case 18: return std::string{renderer_names[static_cast<std::size_t>(std::clamp(s.renderer, 0, 3))]}; case 18: return std::string{renderer_names[static_cast<std::size_t>(std::clamp(s.renderer, 0, renderer_count - 1))]};
default: return "exit"; default: return "exit";
} }
} }
@@ -684,7 +874,7 @@ namespace {
case 13: s.thumbnail = !s.thumbnail; break; case 13: s.thumbnail = !s.thumbnail; break;
case 15: s.fullscreen = !s.fullscreen; break; case 15: s.fullscreen = !s.fullscreen; break;
case 16: s.fps_index = (s.fps_index + delta + 5) % 5; break; case 16: s.fps_index = (s.fps_index + delta + 5) % 5; break;
case 18: s.renderer = (s.renderer + delta + 4) % 4; break; case 18: s.renderer = (s.renderer + delta + renderer_count) % renderer_count; break;
default: break; default: break;
} }
} }
@@ -1025,7 +1215,9 @@ namespace {
const bool want_gpu = s.mode == 0 && s.out.empty() && s.thumbnail == false; const bool want_gpu = s.mode == 0 && s.out.empty() && s.thumbnail == false;
if (want_gpu) { if (want_gpu) {
view.gpu = terrain::build_gpu_terrain(view.map, view.textures, {}, [&](float t) { report(0.70F + 0.18F * t, "Building terrain..."); }); report(0.62F, "Loading objects...");
const auto objects = build_object_scene(assets, map_file.stem().string(), view.map, terrain::render_options{}, bytes);
view.gpu = terrain::build_gpu_terrain(view.map, view.textures, {}, objects, [&](float t) { report(0.70F + 0.18F * t, "Building terrain..."); });
if (!starts.empty()) { if (!starts.empty()) {
view.camera3d.target_x = starts[0].x; view.camera3d.target_x = starts[0].x;
view.camera3d.target_y = starts[0].y; view.camera3d.target_y = starts[0].y;
@@ -1069,12 +1261,13 @@ namespace {
auto command_menu(const std::vector<std::string> &args, const std::filesystem::path &assets) -> int { auto command_menu(const std::vector<std::string> &args, const std::filesystem::path &assets) -> int {
if (!ensure_assets(assets)) return 1; if (!ensure_assets(assets)) return 1;
const auto maps = list_asset_maps(assets); auto maps = list_asset_maps(assets);
if (maps.empty()) { if (maps.empty()) {
std::puts("no maps found"); std::puts("no maps found");
return 1; return 1;
} }
const auto names = ra3::map::load_map_names(assets); const auto names = ra3::map::load_map_names(assets);
sort_maps_by_name(maps, names);
menu_state st; menu_state st;
// Seed the menu from any command-line flags so they are all visible/editable. // Seed the menu from any command-line flags so they are all visible/editable.
@@ -1084,7 +1277,9 @@ namespace {
if (has_flag(args, "--vulkan")) st.settings.renderer = 0; if (has_flag(args, "--vulkan")) st.settings.renderer = 0;
if (has_flag(args, "--dx11") || has_flag(args, "--d3d11")) st.settings.renderer = 1; if (has_flag(args, "--dx11") || has_flag(args, "--d3d11")) st.settings.renderer = 1;
if (has_flag(args, "--dx12") || has_flag(args, "--d3d12")) st.settings.renderer = 2; if (has_flag(args, "--dx12") || has_flag(args, "--d3d12")) st.settings.renderer = 2;
if (has_flag(args, "--sdl")) st.settings.renderer = 3; if (has_flag(args, "--webgpu")) st.settings.renderer = 3;
if (has_flag(args, "--wasmgl")) st.settings.renderer = 4;
if (has_flag(args, "--sdl")) st.settings.renderer = 5;
if (const auto value = option_value(args, "--width")) st.settings.width = std::stoi(*value); if (const auto value = option_value(args, "--width")) st.settings.width = std::stoi(*value);
if (const auto value = option_value(args, "--height")) st.settings.height = std::stoi(*value); if (const auto value = option_value(args, "--height")) st.settings.height = std::stoi(*value);
if (const auto value = option_value(args, "--cam-pitch")) st.settings.cam_pitch = std::stof(*value); if (const auto value = option_value(args, "--cam-pitch")) st.settings.cam_pitch = std::stof(*value);
@@ -1189,8 +1384,9 @@ namespace {
/** Render one menu frame to a BMP (headless preview of the menu layout). */ /** Render one menu frame to a BMP (headless preview of the menu layout). */
auto command_menu_preview(const std::vector<std::string> &args, const std::filesystem::path &assets) -> int { auto command_menu_preview(const std::vector<std::string> &args, const std::filesystem::path &assets) -> int {
if (!ensure_assets(assets)) return 1; if (!ensure_assets(assets)) return 1;
const auto maps = list_asset_maps(assets); auto maps = list_asset_maps(assets);
const auto names = ra3::map::load_map_names(assets); const auto names = ra3::map::load_map_names(assets);
sort_maps_by_name(maps, names);
menu_state st; menu_state st;
if (const auto requested = option_value(args, "--map")) { if (const auto requested = option_value(args, "--map")) {
for (std::size_t i = 0; i < maps.size(); ++i) { for (std::size_t i = 0; i < maps.size(); ++i) {
@@ -1220,7 +1416,17 @@ auto main(int argc, char **argv) -> int {
const std::vector<std::string> args{argv + 1, argv + argc}; const std::vector<std::string> args{argv + 1, argv + argc};
const auto exe_dir = executable_dir(argc > 0 ? argv[0] : "."); const auto exe_dir = executable_dir(argc > 0 ? argv[0] : ".");
const auto assets = exe_dir / "assets"; const auto assets = exe_dir / "assets";
setup_logging(exe_dir); setup_logging();
#if defined(__EMSCRIPTEN__)
// The wasm build runs on a Web Worker (its runtime's main thread lives
// there), where synchronous XHR is legal, so the asset tree is mounted
// lazily from a manifest instead of being preloaded: the browser fetches
// only the files the engine actually opens. On wasm the executable dir is
// "/", so the manifest sits at /assets.manifest.json and the lazy file URLs
// are "assets/<rel>" relative to the page.
const int mounted = ra3::wasmgl::mount_assets((exe_dir / "assets.manifest.json").string(), assets.string(), "assets/");
ender::log::info(std::format("wasm assets: {} files mounted lazily under {}", mounted, assets.string()));
#endif
if (args.empty()) return command_menu({}, assets); if (args.empty()) return command_menu({}, assets);
@@ -1243,6 +1449,7 @@ auto main(int argc, char **argv) -> int {
if (command == "menu") return command_menu(rest, assets); if (command == "menu") return command_menu(rest, assets);
if (command == "menu-preview") return command_menu_preview(rest, assets); if (command == "menu-preview") return command_menu_preview(rest, assets);
if (command == "maps") return command_maps(assets); if (command == "maps") return command_maps(assets);
if (command == "textures") return command_textures(rest, assets);
if (command == "skirmish") return command_skirmish(rest, assets); if (command == "skirmish") return command_skirmish(rest, assets);
if (command == "render") return command_render(rest, assets); if (command == "render") return command_render(rest, assets);
+101
View File
@@ -0,0 +1,101 @@
<!doctype html>
<!-- OpenRA3 web page. The engine runs inside openra3.worker.js: the page owns the
DOM (input, resize) and transfers #canvas to the worker as an OffscreenCanvas,
so the engine never blocks the main thread. -->
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no">
<title>OpenRA3</title>
<style>
html, body { margin: 0; height: 100%; background: #1a0a08; overflow: hidden; }
#canvas { display: block; width: 100vw; height: 100vh; outline: none; cursor: default; }
#loading { position: absolute; inset: 0; display: flex; align-items: center; justify-content: center;
color: #ecbe50; font: 16px/1.4 monospace; pointer-events: none; }
</style>
</head>
<body>
<canvas id="canvas" tabindex="-1" oncontextmenu="event.preventDefault()"></canvas>
<div id="loading">Loading OpenRA3…</div>
<script>
(function () {
var canvas = document.getElementById('canvas');
var loading = document.getElementById('loading');
// `?nocache` busts the worker, module and wasm fetches (development),
// so a rebuild is picked up without clearing the browser cache.
var bust = location.search.indexOf('nocache') >= 0 ? Date.now() : 0;
var worker = new Worker('openra3.worker.js' + (bust ? '?v=' + bust : ''));
var ready = false;
var pending = []; // input queued until the runtime is up
function post(message) {
if (ready) {
worker.postMessage(message);
} else {
pending.push(message);
}
}
function backingSize() {
var dpr = window.devicePixelRatio || 1;
return {
width: Math.max(1, Math.round(window.innerWidth * dpr)),
height: Math.max(1, Math.round(window.innerHeight * dpr)),
};
}
worker.onmessage = function (event) {
var data = event.data || {};
if (data.log) { console.log(data.log); return; }
if (data.ready) {
ready = true;
if (loading) loading.remove();
for (var i = 0; i < pending.length; i++) worker.postMessage(pending[i]);
pending = [];
return;
}
if (typeof data.openra3Frames === 'number') { window.openra3Frames = data.openra3Frames; }
};
// Hand the canvas to the worker as an OffscreenCanvas. After this the
// page can no longer size or draw it; the worker owns it.
var size = backingSize();
var offscreen = canvas.transferControlToOffscreen();
worker.postMessage({ canvas: offscreen, width: size.width, height: size.height, bust: bust }, [offscreen]);
// ---- input: the page owns the DOM, the worker owns the game --------
window.addEventListener('keydown', function (event) {
post({ type: 'key', code: event.code, down: true });
if (event.code === 'Tab' || event.code === 'Space' || event.code.indexOf('Arrow') === 0) event.preventDefault();
}, false);
window.addEventListener('keyup', function (event) {
post({ type: 'key', code: event.code, down: false });
}, false);
window.addEventListener('keypress', function (event) {
if (event.charCode) post({ type: 'text', char: event.charCode });
}, false);
canvas.addEventListener('mousedown', function (event) {
post({ type: 'mousebutton', x: event.clientX, y: event.clientY, left: event.button === 0 });
}, false);
window.addEventListener('mousemove', function (event) {
post({ type: 'mousemove', x: event.clientX, y: event.clientY, dx: event.movementX, dy: event.movementY,
left: (event.buttons & 1) !== 0 });
}, false);
window.addEventListener('wheel', function (event) {
post({ type: 'wheel', deltaY: event.deltaY });
}, { passive: true });
window.addEventListener('contextmenu', function (event) { event.preventDefault(); });
window.addEventListener('resize', function () {
var next = backingSize();
post({ type: 'resize', width: next.width, height: next.height });
});
window.addEventListener('beforeunload', function () {
if (ready) worker.postMessage({ type: 'quit' });
});
})();
</script>
</body>
</html>
+114
View File
@@ -0,0 +1,114 @@
'use strict';
// OpenRA3 WebAssembly worker bootstrap.
//
// The whole Emscripten runtime runs here (not on the page main thread), so the
// runtime's filesystem is local to this worker: `FS.createLazyFile` can use
// synchronous XHR to fetch assets on demand without blocking the page.
//
// Protocol with index.html:
// page -> worker { canvas, width, height } (once, transfers the OffscreenCanvas)
// page -> worker { type: 'key' | 'text' | 'mousebutton' | 'mousemove' | 'wheel' | 'resize' | 'quit', ... }
// worker -> page { log } | { ready } | { openra3Frames }
//
// The Emscripten runtime is started by importScripts('openra3.js') once the
// canvas has arrived; main() then runs here and yields via Asyncify.
// Forward worker warnings/errors (including WebGPU validation messages from the
// runtime) to the page, which logs them to the main console.
(function () {
var origError = console.error.bind(console);
var origWarn = console.warn.bind(console);
console.error = function () {
origError.apply(null, arguments);
try { self.postMessage({ log: '[worker error] ' + Array.prototype.join.call(arguments, ' ') }); } catch (e) {}
};
console.warn = function () {
origWarn.apply(null, arguments);
try { self.postMessage({ log: '[worker warn] ' + Array.prototype.join.call(arguments, ' ') }); } catch (e) {}
};
})();
self.Module = {
print: function (text) {
self.postMessage({ log: String(text) });
},
printErr: function (text) {
self.postMessage({ log: String(text) });
},
onRuntimeInitialized: function () {
self.postMessage({ ready: true });
},
};
var openra3Started = false;
self.onmessage = function (event) {
var data = event.data;
if (!data) return;
if (!openra3Started) {
if (!data.canvas) return;
openra3Started = true;
var canvas = data.canvas;
if (data.width && data.height) {
canvas.width = data.width;
canvas.height = data.height;
}
self.Module.canvas = canvas;
var start = function () {
// `?nocache` on the page forces a fresh module + wasm fetch, so a
// rebuild is picked up without clearing the browser cache.
var bust = data.bust ? ('?v=' + data.bust) : '';
if (bust) self.Module.locateFile = function (path) { return path + bust; };
try {
importScripts('openra3.js' + bust);
} catch (error) {
self.postMessage({ log: 'OpenRA3 worker failed to start: ' + error });
}
};
// The WebGPU backend starts from a device created here (`Module.
// preinitializedWebGPUDevice`); the WebGL backend ignores it. If WebGPU
// is unavailable the runtime falls back to WebGL.
if (typeof navigator !== 'undefined' && navigator.gpu) {
navigator.gpu.requestAdapter().then(function (adapter) {
if (!adapter) { start(); return; }
adapter.requestDevice().then(function (device) {
self.Module.preinitializedWebGPUDevice = device;
start();
}).catch(start);
}).catch(start);
} else {
start();
}
return;
}
// Input is only meaningful once the runtime has initialized its exports.
switch (data.type) {
case 'key':
if (self.Module.ccall) self.Module.ccall('openra3_key', null, ['string', 'number'], [data.code, data.down ? 1 : 0]);
break;
case 'text':
if (self.Module._openra3_text) self.Module._openra3_text(data.char | 0);
break;
case 'mousebutton':
if (self.Module._openra3_mouse_button) self.Module._openra3_mouse_button(data.x, data.y, data.left ? 1 : 0);
break;
case 'mousemove':
if (self.Module._openra3_mouse_move) self.Module._openra3_mouse_move(data.x, data.y, data.dx, data.dy, data.left ? 1 : 0);
break;
case 'wheel':
if (self.Module._openra3_wheel) self.Module._openra3_wheel(data.deltaY);
break;
case 'resize':
if (self.Module._openra3_resize) self.Module._openra3_resize(data.width | 0, data.height | 0);
break;
case 'quit':
if (self.Module._openra3_quit) self.Module._openra3_quit();
break;
default:
break;
}
};
+132
View File
@@ -0,0 +1,132 @@
#!/usr/bin/env python3
"""Static file server with HTTP Range support, for the OpenRA3 wasm build.
Emscripten's lazy file reads stream assets back as byte ranges, so the server
must honour the ``Range`` header -- Python's stdlib ``http.server`` does not.
python3 apps/web/serve.py --root build/wasm/bin --port 8199
# then open http://localhost:8199/index.html
Serve ``--root`` containing ``index.html``, ``openra3.js``, ``openra3.wasm``,
``openra3.worker.js`` and ``assets.manifest.json``. The lazy asset tree is
mounted from ``--assets`` (its files are then fetched from ``/assets/<rel>`` on
demand); without ``--assets`` the tree is expected under ``--root`` itself.
"""
import argparse
import functools
import os
import re
import sys
from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer
_RANGE = re.compile(r"bytes=(\d*)-(\d*)$")
class RangeHandler(SimpleHTTPRequestHandler):
# Directory served for ``/assets/...`` (set in main); None means "under root".
assets_dir = None
def translate_path(self, path):
if self.assets_dir:
prefix = "/assets/"
if path == "/assets" or path.startswith(prefix):
rel = path[len(prefix):] if path.startswith(prefix) else ""
rel = rel.split("?", 1)[0].split("#", 1)[0]
parts = [p for p in rel.split("/") if p and p != ".."]
return os.path.join(self.assets_dir, *parts)
return super().translate_path(path)
# Cross-origin isolation (kept for SharedArrayBuffer-style features and the
# WebGPU/pthread paths). no-store keeps the dev loop honest: a stale
# openra3.js/.wasm in the browser cache hides new builds (the worker's
# importScripts especially).
def end_headers(self):
self.send_header("Cross-Origin-Opener-Policy", "same-origin")
self.send_header("Cross-Origin-Embedder-Policy", "require-corp")
self.send_header("Cache-Control", "no-store, max-age=0")
super().end_headers()
def send_head(self):
header = self.headers.get("Range")
if not header:
return super().send_head()
path = self.translate_path(self.path)
if os.path.isdir(path):
return super().send_head()
try:
handle = open(path, "rb")
except OSError:
self.send_error(404, "File not found")
return None
match = _RANGE.match(header.strip())
if not match:
handle.close()
return super().send_head()
size = os.fstat(handle.fileno()).st_size
start = int(match.group(1)) if match.group(1) else 0
end = int(match.group(2)) if match.group(2) else size - 1
if start > end or start >= size:
handle.close()
self.send_error(416, "Requested Range Not Satisfiable")
return None
end = min(end, size - 1)
self.send_response(206)
self.send_header("Content-Type", self.guess_type(path))
self.send_header("Accept-Ranges", "bytes")
self.send_header("Content-Range", f"bytes {start}-{end}/{size}")
self.send_header("Content-Length", str(end - start + 1))
self.send_header("Cache-Control", "no-store")
self.end_headers()
handle.seek(start)
self._remaining = end - start + 1
return handle
def copyfile(self, source, outputfile):
remaining = getattr(self, "_remaining", None)
if remaining is None:
return super().copyfile(source, outputfile)
try:
while remaining > 0:
chunk = source.read(min(65536, remaining))
if not chunk:
break
outputfile.write(chunk)
remaining -= len(chunk)
finally:
self._remaining = None
def main() -> int:
parser = argparse.ArgumentParser(description="OpenRA3 wasm dev server")
parser.add_argument("--root", default="build/wasm/bin", help="directory to serve")
parser.add_argument("--assets", default=None, help="directory served for /assets (the lazy asset tree)")
parser.add_argument("--port", type=int, default=8199)
parser.add_argument("--bind", default="127.0.0.1")
args = parser.parse_args()
if not os.path.isdir(args.root):
print(f"root not found: {args.root}", file=sys.stderr)
return 1
if args.assets and not os.path.isdir(args.assets):
print(f"assets not found: {args.assets}", file=sys.stderr)
return 1
RangeHandler.assets_dir = os.path.abspath(args.assets) if args.assets else None
handler = functools.partial(RangeHandler, directory=args.root)
server = ThreadingHTTPServer((args.bind, args.port), handler)
print(f"serving {args.root} at http://{args.bind}:{args.port}/index.html")
if RangeHandler.assets_dir:
print(f" /assets -> {RangeHandler.assets_dir}")
try:
server.serve_forever()
except KeyboardInterrupt:
pass
return 0
if __name__ == "__main__":
raise SystemExit(main())
+39
View File
@@ -0,0 +1,39 @@
<?xml version='1.0' encoding='utf-8'?>
<!--
Windows Installer source for OpenRA3, driven by wixl (msitools) so the Linux
cross-build image can produce the .msi. `exe` and `dll` are passed with
wixl -D; the version is substituted at configure time.
-->
<Wix xmlns='http://schemas.microsoft.com/wix/2006/wi'>
<Product Id='*'
Name='OpenRA3'
Language='1033'
Version='@PROJECT_VERSION@'
Manufacturer='OpenRA3'
UpgradeCode='6C6B1F4E-0B9A-4C2E-9E2A-2E4D7B5C1A31'>
<Package Id='*' InstallerVersion='500' Compressed='yes' InstallScope='perMachine'/>
<Media Id='1' Cabinet='openra3.cab' EmbedCab='yes'/>
<Directory Id='TARGETDIR' Name='SourceDir'>
<Directory Id='ProgramFiles64Folder'>
<Directory Id='INSTALLFOLDER' Name='OpenRA3'>
<Component Id='MainExecutable' Guid='1F2E3D4C-5B6A-4798-8C1D-2E3F4A5B6C7D'>
<File Id='OpenRA3Exe' Name='openra3.exe' Source='$(var.exe)' KeyPath='yes'>
<Shortcut Id='StartMenuShortcut' Directory='ProgramMenuFolder' Name='OpenRA3'
WorkingDirectory='INSTALLFOLDER' Advertise='no'/>
</File>
</Component>
<Component Id='SdlRuntime' Guid='2A3B4C5D-6E7F-4809-9D2E-3F4A5B6C7D8E'>
<File Id='Sdl3Dll' Name='SDL3.dll' Source='$(var.dll)' KeyPath='yes'/>
</Component>
</Directory>
</Directory>
<Directory Id='ProgramMenuFolder'/>
</Directory>
<Feature Id='Main' Title='OpenRA3' Level='1'>
<ComponentRef Id='MainExecutable'/>
<ComponentRef Id='SdlRuntime'/>
</Feature>
</Product>
</Wix>
@@ -0,0 +1,53 @@
# Cross-compile for Linux on ARM64 (aarch64-linux-gnu) from x86_64.
#
# clang-21 + libc++ (for `import std;`) with the arm64 libc++ and SDL3 taken
# from Ubuntu's arm64 packages (installed with dpkg multiarch), and the arm64
# glibc headers/libs from `gcc-aarch64-linux-gnu`. lld links the result; the C
# multiarch include directory is added explicitly because clang does not always
# infer Debian's `bits/` layout from the target triple alone.
# The `import std` gate must be set before the compiler is probed; a toolchain
# file is processed earlier than the project's own top-level settings.
set(CMAKE_EXPERIMENTAL_CXX_IMPORT_STD "d0edc3af-4c50-42ea-a356-e2862fe7a444")
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_PROCESSOR aarch64)
set(OPENRA3_AARCH64_TRIPLE "aarch64-linux-gnu")
# Multiarch paths the arm64 packages install into.
set(OPENRA3_AARCH64_INCLUDE "/usr/include/${OPENRA3_AARCH64_TRIPLE}")
set(OPENRA3_AARCH64_LIB "/usr/lib/${OPENRA3_AARCH64_TRIPLE}")
# Compiler + libc++ module description are cache vars so a distro with a
# differently-versioned clang (e.g. Debian's clang-19) can override on the
# command line.
set(OPENRA3_AARCH64_CC "clang-21" CACHE STRING "aarch64 C compiler")
set(OPENRA3_AARCH64_CXX "clang++-21" CACHE STRING "aarch64 C++ compiler")
set(OPENRA3_AARCH64_STDLIB_MODULES_JSON "/usr/lib/llvm-21/lib/libc++.modules.json" CACHE FILEPATH
"libc++ std module description for the aarch64 target")
set(CMAKE_C_COMPILER ${OPENRA3_AARCH64_CC})
set(CMAKE_CXX_COMPILER ${OPENRA3_AARCH64_CXX})
set(CMAKE_C_COMPILER_TARGET ${OPENRA3_AARCH64_TRIPLE})
set(CMAKE_CXX_COMPILER_TARGET ${OPENRA3_AARCH64_TRIPLE})
# CMake's standard-library detection does not pass COMPILER_TARGET, so without
# this it probes the host standard library.
set(CMAKE_C_COMPILER_ARG1 "--target=${OPENRA3_AARCH64_TRIPLE}")
set(CMAKE_CXX_COMPILER_ARG1 "--target=${OPENRA3_AARCH64_TRIPLE}")
# libc++ standard-library module description for the aarch64 target. On a
# multiarch image only the arm64 libc++ is installed, so the standard path
# already resolves to the aarch64 module description.
set(CMAKE_CXX_STDLIB_MODULES_JSON "${OPENRA3_AARCH64_STDLIB_MODULES_JSON}")
# Strip with the cross binutils, so CPack can strip the arm64 binary.
set(CMAKE_STRIP aarch64-linux-gnu-strip)
set(CMAKE_C_FLAGS_INIT "-isystem ${OPENRA3_AARCH64_INCLUDE}")
set(CMAKE_CXX_FLAGS_INIT "-stdlib=libc++ -isystem ${OPENRA3_AARCH64_INCLUDE}")
set(CMAKE_EXE_LINKER_FLAGS_INIT "-stdlib=libc++ -fuse-ld=lld -L${OPENRA3_AARCH64_LIB}")
set(CMAKE_SHARED_LINKER_FLAGS_INIT "-stdlib=libc++ -fuse-ld=lld -L${OPENRA3_AARCH64_LIB}")
# Run arm64 test binaries through qemu when it is installed, so `ctest` works
# during the image build; harmless when the tests are only cross-compiled.
set(CMAKE_CROSSCOMPILING_EMULATOR qemu-aarch64-static)
+55
View File
@@ -0,0 +1,55 @@
# WebAssembly cross-build via Emscripten.
#
# Includes Emscripten's own platform toolchain (found through $EMSDK, set by the
# emsdk image / `emsdk_env.sh`) and then layers the project's requirements:
#
# * C++26 `import std;`. Emscripten ships the libc++ `std` module source and a
# modules.json in its sysroot; its toolchain wires that up, but only if
# `CMAKE_CXX_MODULE_STD` is on and no std-modules JSON has been set yet, so
# both are arranged before Emscripten.cmake is included and before the
# compiler is probed.
# * Global `-fexceptions`. Emscripten disables exceptions by default, and the
# libc++ `std` module must be built with the same setting as its consumers,
# or clang rejects the prebuilt module ("configuration mismatch"). The engine
# throws, so exceptions are enabled everywhere.
# * Asyncify turns `emscripten_sleep` (used by display::sleep_frame on
# Emscripten) into a yield: the engine's frame loops are blocking, and without
# yielding the worker freezes and never paints.
#
# No pthreads: the runtime's main thread runs inside a plain Web Worker started
# by `apps/web/openra3.worker.js`, so its virtual filesystem is local to that
# worker and `FS.createLazyFile` (synchronous XHR) is legal. Under
# `PROXY_TO_PTHREAD` Emscripten proxies every filesystem syscall to the main
# browser thread, where synchronous XHR is forbidden, so lazy assets cannot work
# there - hence the plain worker. The SDL3 web port is only pulled in by the
# legacy main-thread backend (OPENRA3_WASM_SDL).
# `import std` support: the gate and the module switch must be in place before
# Emscripten.cmake runs and before the compiler is probed.
set(CMAKE_EXPERIMENTAL_CXX_IMPORT_STD "d0edc3af-4c50-42ea-a356-e2862fe7a444")
set(CMAKE_CXX_MODULE_STD ON)
# Let Emscripten.cmake point CMake at its own sysroot modules.json.
unset(CMAKE_CXX_STDLIB_MODULES_JSON)
if(DEFINED ENV{EMSDK})
set(OPENRA3_EMSCRIPTEN_CMAKE "$ENV{EMSDK}/upstream/emscripten/cmake/Modules/Platform/Emscripten.cmake")
elseif(DEFINED ENV{EMSCRIPTEN_ROOT})
set(OPENRA3_EMSCRIPTEN_CMAKE "$ENV{EMSCRIPTEN_ROOT}/cmake/Modules/Platform/Emscripten.cmake")
endif()
if(NOT EXISTS "${OPENRA3_EMSCRIPTEN_CMAKE}")
message(FATAL_ERROR "Emscripten.cmake not found (set EMSDK); looked at '${OPENRA3_EMSCRIPTEN_CMAKE}'")
endif()
include("${OPENRA3_EMSCRIPTEN_CMAKE}")
# Exceptions are used by the engine; keep them on globally so the std module and
# its consumers agree.
#
# Asyncify turns `emscripten_sleep` (used by display::sleep_frame on Emscripten)
# into a yield to the browser: the engine's frame loops are blocking, and
# without yielding the worker freezes and never paints.
set(CMAKE_CXX_FLAGS_INIT "-fexceptions -sASYNCIFY=1")
set(CMAKE_C_FLAGS_INIT "-sASYNCIFY=1")
set(CMAKE_EXE_LINKER_FLAGS_INIT
"-fexceptions -sDISABLE_EXCEPTION_CATCHING=0 -sALLOW_MEMORY_GROWTH=1 -sASYNCIFY=1")
set(CMAKE_SHARED_LINKER_FLAGS_INIT
"-fexceptions -sDISABLE_EXCEPTION_CATCHING=0 -sALLOW_MEMORY_GROWTH=1 -sASYNCIFY=1")
+50
View File
@@ -0,0 +1,50 @@
# Cross-compile for Windows on ARM64 (aarch64-w64-mingw32) with llvm-mingw:
# clang + libc++ + the libc++ `std` module, targeting the MSVCRT MinGW runtime.
# The same llvm-mingw release is multi-target, so only the target triple, the
# SDL3 SDK and the sysroot paths differ from the x86_64 toolchain.
# The `import std` gate must be set before the compiler is probed; a toolchain
# file is processed earlier than the project's own top-level settings.
set(CMAKE_EXPERIMENTAL_CXX_IMPORT_STD "d0edc3af-4c50-42ea-a356-e2862fe7a444")
set(CMAKE_SYSTEM_NAME Windows)
set(CMAKE_SYSTEM_PROCESSOR aarch64)
set(OPENRA3_LLVM_MINGW_ROOT "/opt/llvm-mingw" CACHE PATH "llvm-mingw installation root")
if(NOT EXISTS "${OPENRA3_LLVM_MINGW_ROOT}/bin/clang++")
message(FATAL_ERROR "llvm-mingw not found at ${OPENRA3_LLVM_MINGW_ROOT} (expected bin/clang++)")
endif()
# llvm-mingw's per-target prefix/sysroot for the ARM64 target.
set(OPENRA3_MINGW_TRIPLE "aarch64-w64-mingw32")
set(CMAKE_C_COMPILER "${OPENRA3_LLVM_MINGW_ROOT}/bin/clang")
set(CMAKE_CXX_COMPILER "${OPENRA3_LLVM_MINGW_ROOT}/bin/clang++")
set(CMAKE_C_COMPILER_TARGET ${OPENRA3_MINGW_TRIPLE})
set(CMAKE_CXX_COMPILER_TARGET ${OPENRA3_MINGW_TRIPLE})
# CMake's standard-library detection does not pass COMPILER_TARGET, so without
# this it probes the host standard library. Putting the target in ARG1 makes
# the probe see libc++ (the MinGW target's library).
set(CMAKE_C_COMPILER_ARG1 "--target=${OPENRA3_MINGW_TRIPLE}")
set(CMAKE_CXX_COMPILER_ARG1 "--target=${OPENRA3_MINGW_TRIPLE}")
# libc++ standard-library module description for the MinGW target. Must be set
# before the compiler is probed so CMake can build `import std;`.
set(CMAKE_CXX_STDLIB_MODULES_JSON "${OPENRA3_LLVM_MINGW_ROOT}/${OPENRA3_MINGW_TRIPLE}/lib/libc++.modules.json")
# Select libc++ explicitly so CMake's `import std` detection recognises the
# standard library (llvm-mingw defaults to it, but CMake keys off -stdlib=).
set(CMAKE_CXX_FLAGS_INIT "-stdlib=libc++")
set(CMAKE_EXE_LINKER_FLAGS_INIT "-stdlib=libc++")
set(CMAKE_SHARED_LINKER_FLAGS_INIT "-stdlib=libc++")
# SDL3 for aarch64-w64-mingw32. The official SDL release ships no MinGW ARM64
# dev package, so the image stages one from MSYS2's clangarm64 repo under this
# root in the same `<triple>/{include,lib,bin}` layout.
set(OPENRA3_SDL3_ROOT "/opt/sdl3-aarch64" CACHE PATH "SDL3 aarch64 MinGW development package root")
set(OPENRA3_SDL3_MINGW_TRIPLE "${OPENRA3_MINGW_TRIPLE}" CACHE STRING "SDL3 MinGW triple subdirectory")
set(CMAKE_FIND_ROOT_PATH "${OPENRA3_LLVM_MINGW_ROOT}/${OPENRA3_MINGW_TRIPLE}")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
+2
View File
@@ -36,6 +36,8 @@ set(CMAKE_SHARED_LINKER_FLAGS_INIT "-stdlib=libc++")
# Official SDL3 MinGW development package (headers + import library + DLL). # Official SDL3 MinGW development package (headers + import library + DLL).
set(OPENRA3_SDL3_ROOT "/opt/sdl3-mingw" CACHE PATH "SDL3 MinGW development package root") set(OPENRA3_SDL3_ROOT "/opt/sdl3-mingw" CACHE PATH "SDL3 MinGW development package root")
# The triple-named subdirectory inside the SDL3 MinGW package.
set(OPENRA3_SDL3_MINGW_TRIPLE "x86_64-w64-mingw32" CACHE STRING "SDL3 MinGW triple subdirectory")
set(CMAKE_FIND_ROOT_PATH "${OPENRA3_LLVM_MINGW_ROOT}/x86_64-w64-mingw32") set(CMAKE_FIND_ROOT_PATH "${OPENRA3_LLVM_MINGW_ROOT}/x86_64-w64-mingw32")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER) set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
+87 -14
View File
@@ -62,9 +62,9 @@ umbrella module re-exports the SDK; applications import `ra3` only.
| \ | | \ |
| \ v | \ v
| ra3.display (backend pick) | ra3.display (backend pick)
| / | \ | / | \ \
v ra3.ui ra3.vulkan ra3.dx v ra3.ui ra3.vulkan ra3.dx ra3.webgpu / ra3.wasmgl
| (SDL3) (Vulkan) (D3D11/12) | (SDL3) (Vulkan) (D3D11/12) (wasm workers)
--------------------------------------------------------------------------- ---------------------------------------------------------------------------
v v
L2 simulation ra3.logic ra3.modules ra3.combat ra3.movement L2 simulation ra3.logic ra3.modules ra3.combat ra3.movement
@@ -79,8 +79,8 @@ umbrella module re-exports the SDK; applications import `ra3` only.
``` ```
The current concrete modules (`ra3.core`, `ra3.logic`, `ra3.data`, The current concrete modules (`ra3.core`, `ra3.logic`, `ra3.data`,
`ra3.skirmish`, `ra3.fs`, `ra3.map`, `ra3.terrain`, `ra3.render`, `ra3.ui.*`, `ra3.skirmish`, `ra3.fs`, `ra3.map`, `ra3.terrain`, `ra3.models`, `ra3.render`, `ra3.ui.*`,
`ra3.vulkan.*`, `ra3.dx.*`, `ra3.display`, `ra3.game`, `ra3.client`, and the `ra3.vulkan.*`, `ra3.dx.*`, `ra3.webgpu.*`, `ra3.wasmgl.*`, `ra3.display`, `ra3.game`, `ra3.client`, and the
vendored `ender.log`) are the **seeds** of the vendored `ender.log`) are the **seeds** of the
target modules below. `ra3.skirmish` and `ra3.game` will be absorbed into target modules below. `ra3.skirmish` and `ra3.game` will be absorbed into
`ra3.ai` / `ra3.match`; new modules are added as their subsystems are recovered. `ra3.ai` / `ra3.match`; new modules are added as their subsystems are recovered.
@@ -206,10 +206,11 @@ Mirrors SAGE `GameEngine/Common`. Everything else speaks this.
- `[ ]` DDS / DXT compressed textures `(v0.5)` - `[ ]` DDS / DXT compressed textures `(v0.5)`
- `[ ]` atlas + mip generation, gutter padding (GPU) `[~]` - `[ ]` atlas + mip generation, gutter padding (GPU) `[~]`
- `[ ]` async upload / streaming `(v0.6)` - `[ ]` async upload / streaming `(v0.6)`
- **F2 Models** `[ ]` `!!` - **F2 Models** `[~]`
- `[ ]` W3D container parse (chunks, hierarchy, meshes) `(v0.6)` - `[x]` compiled `W3DMesh` decode (vertex buffer + D3D9 declaration + triangle list) from the BAB static/worldbuilder stream
- `[ ]` materials, shaders, texture references - `[x]` embedded DDS textures (DXT1/3/5 + uncompressed RGB) → ARGB
- `[ ]` LOD sets, collision meshes - `[x]` `W3DHierarchy` decode + static bind-pose skinning (bone-space vertices)
- `[ ]` W3D container/hierarchy animation, LOD sets, collision meshes `!!` `(v0.6)`
- **F3 Animation** `[ ]` `!!` - **F3 Animation** `[ ]` `!!`
- `[ ]` W3D animation chunks, bone poses `(v0.6)` - `[ ]` W3D animation chunks, bone poses `(v0.6)`
- `[ ]` blend trees / transition animations - `[ ]` blend trees / transition animations
@@ -532,9 +533,12 @@ Mirrors SAGE `GameEngine/Common`. Everything else speaks this.
- `[ ]` pass scheduling, barriers, transient/aliased resources `(v0.6)` - `[ ]` pass scheduling, barriers, transient/aliased resources `(v0.6)`
- **F3 Terrain render** `[~]` - **F3 Terrain render** `[~]`
- `[x]` top-down software + GPU heightfield with blend ramp + gutter atlas - `[x]` top-down software + GPU heightfield with blend ramp + gutter atlas
- `[ ]` perspective terrain mesh, LOD, cliff, water `(v0.6)` - `[ ]` perspective terrain mesh, LOD, cliff `(v0.6)`
- **F4 Model render** `[ ]` `!!` - `[x]` water surface in the raymarch: SAGE `Water.frag` port (ocean/river) `[~]`
- `[ ]` W3D draw, skinning, materials, team colors `(v0.6)` - **F4 Model render** `[~]`
- `[x]` static map-object triangle soup, world-space, depth-tested over the terrain (Vulkan + software)
- `[x]` bind-pose skinning (bone-space vertices) + ground-decal depth bias
- `[ ]` animated W3D draw, materials, team colors `!!` `(v0.6)`
- `[ ]` shadows, decals, ground marks `(v0.6)` - `[ ]` shadows, decals, ground marks `(v0.6)`
- **F5 VFX** `[ ]` - **F5 VFX** `[ ]`
- `[ ]` particle systems, beams, muzzle flashes, explosions `(v0.6)` - `[ ]` particle systems, beams, muzzle flashes, explosions `(v0.6)`
@@ -547,8 +551,35 @@ Mirrors SAGE `GameEngine/Common`. Everything else speaks this.
- `[x]` ARGB framebuffer, blit/line/circle/text, TGA decode, BMP encode - `[x]` ARGB framebuffer, blit/line/circle/text, TGA decode, BMP encode
- `[x]` map compositing, grid, markers; headless output - `[x]` map compositing, grid, markers; headless output
- **F9 Post-processing** `[ ]` - **F9 Post-processing** `[ ]`
- `[x]` underwater tint/fog in the terrain pass (retail `UnderwaterDeferred.fx`) `[~]`
- `[ ]` bloom, color grading, AA, resolution scaling `(v0.7)` - `[ ]` bloom, color grading, AA, resolution scaling `(v0.7)`
### M16b shader porting status (retail `Data\Shaders.big` → `*.fxo`)
The retail set is **88 compiled effects** (`Core12\shaders\compiled`, duplicated
in `Misc`/`Shaders`; `Core5`/`Core8` ship only `terrain`; plus a 60-byte
`null` stub). OpenRA3 implements the static-map subset only:
- **Ported (approximate stand-ins, shared by all backends):** `Terrain.fx`
(`terrain.*`), the opaque diffuse subset of `BuildingsGeneric.fx` /
`BasicW3D.fx` / `ObjectsGeneric.fx` (`object.*`), and the SAGE water model
`Ocean.fx` (+ `OceanDisplacement`/`OceanNoVertexTexture`/`RiverWater`/
`RiverReflection`/`UnderwaterDeferred`, folded into the `terrain.*` water
branch — see `docs/REVERSE_ENGINEERING.md`). The retail water flow and bump
maps (`art/terrain/ra3_deepocean.tga`, `ra3_deepocean_nrm.tga`) ride as the
last two terrain-atlas layers, so no backend adds a binding.
- **Not ported:** all faction/variant model shaders (`buildings*`, `objects*`,
`basicw3d*`, `defaultw3d*`, `normalmapped`, `tree`/`treesway`), instances/
animation (`infantry*`), particles and beams (`cpuparticle`, `gpuparticle*`,
`swarmparticle`, `laser*`, `lightning`, `fx*`, `tracer`, `trail`,
`connectionline`, `linerenderers`, `stream`, `rain`, `simple*`), shadows and
ground decals (`shadow`, `decal`, `outlines`, `occlusion`, `terraintracks`),
post-processing (`postfx_*`), and the 2D/misc shaders (`render2d`, `video`,
`bootupscreen`, `debug`, `errormissing`, `rotateenvironmentmap`,
`distortingobject`). The shared retail includes `shadowmap.fxh`, `ssao.fxh`,
`macrotexture.fxh`, `gamma.fxh` are likewise absent (`skinning.fxh` is done at
bind pose only).
### M17 `ra3.ui` — platform layer & backends `[D]` ### M17 `ra3.ui` — platform layer & backends `[D]`
- **F1 Display abstraction** `[D]` - **F1 Display abstraction** `[D]`
@@ -575,8 +606,9 @@ Mirrors SAGE `GameEngine/Common`. Everything else speaks this.
- `[ ]` formal RHI integration (see M16 F1) `(v0.6)` - `[ ]` formal RHI integration (see M16 F1) `(v0.6)`
- **F2 Terrain presentation** `[D]` - **F2 Terrain presentation** `[D]`
- `[x]` GPU heightfield raymarch, mipmapped atlas, gutter, retail blend ramp - `[x]` GPU heightfield raymarch, mipmapped atlas, gutter, retail blend ramp
- **F3 Materials & pipelines** `[ ]` - **F3 Materials & pipelines** `[~]`
- `[ ]` model/particle/HUD pipelines `(v0.6)` - `[x]` static-model pipeline (vertex/index buffers, texture array, depth test)
- `[ ]` particle/HUD pipelines `(v0.6)`
- **F4 Null fallback** `[D]` - **F4 Null fallback** `[D]`
- `[x]` report failure when no Vulkan loader is present - `[x]` report failure when no Vulkan loader is present
@@ -595,6 +627,47 @@ Mirrors SAGE `GameEngine/Common`. Everything else speaks this.
- **F5 Null fallback** `[D]` - **F5 Null fallback** `[D]`
- `[x]` non-Windows builds link a stub that fails `init` - `[x]` non-Windows builds link a stub that fails `init`
### M18d `ra3.wasmgl` — SDL-free wasm worker backend `[P]`
- **F1 Device & canvas** `[D]`
- `[x]` Emscripten-only; engine runs on a plain Web Worker, WebGL2 on an
OffscreenCanvas transferred by the page (no SDL, no `PROXY_TO_PTHREAD`)
- `[x]` `specialHTMLTargets['#canvas']` lets `emscripten_webgl_create_context` find
the worker's OffscreenCanvas
- **F2 2D image path** `[D]`
- `[x]` GLSL ES BGRA texture + fullscreen triangle (`.bgra` swizzle)
- **F3 Terrain presentation** `[D]`
- `[x]` GPU heightfield raymarch (GLSL ES), including the `GL_R16`
(`0x822A`) heightmap and `GL_RGBA16` cell textures
- **F4 On-demand assets** `[D]`
- `[x]` `assets.manifest.json` (from `scripts/make_web_manifest.py`) is embedded;
the worker registers every file with `FS.createLazyFile`, so only opened files
are fetched (a map pulls just its map + tiles)
- **F5 Input** `[D]`
- `[x]` page owns DOM listeners and posts events to the worker's exported
`openra3_*` functions (no DOM callbacks in the worker)
- **F6 Null fallback** `[D]`
- `[x]` non-Emscripten builds link a stub that fails `init`
### M18e `ra3.webgpu` — WebGPU wasm backend `[P]`
- **F1 Device & surface** `[D]`
- `[x]` Emscripten `emdawnwebgpu` port (Dawn C++ wrapper); the worker pre-creates
the device (`Module.preinitializedWebGPUDevice`) and a surface is configured on
the transferred OffscreenCanvas
- **F2 WGSL shaders** `[D]`
- `[x]` web WebGPU is WGSL-only, so the scene/terrain shaders are WGSL ports
(the Vulkan SPIR-V blobs cannot be reused); the uniform layout and terrain data
plumbing are shared with Vulkan/WebGL
- **F3 Terrain** `[D]`
- `[x]` GPU heightfield raymarch in WGSL (R16Uint heightmap, RGBA16Uint cell
record, RGBA8 atlas)
- **F4 Selection & fallback** `[D]`
- `[x]` preferred wasm backend; falls back to `ra3.wasmgl` when no adapter is
available
- **F5 Null fallback** `[D]`
- `[x]` non-Emscripten builds link a stub that fails `init`
### M19 `ra3.audio` — audio `[ ]` ### M19 `ra3.audio` — audio `[ ]`
- **F1 SFX** `[ ]` - **F1 SFX** `[ ]`
+186
View File
@@ -212,6 +212,188 @@ per-cell restart is the remaining source of grid lines. Switching the atlas
from 64 px blocks to continuous 32 px regions (or the Morton 8x8 layout) is the from 64 px blocks to continuous 32 px regions (or the Morton 8x8 layout) is the
next step, pending an art correlation check. next step, pending an art correlation check.
### Compiled art (BinaryAssetBuilder) and map objects
Retail RA3 ships no `.w3x`/`.w3d` files: BinaryAssetBuilder bakes every model,
texture and script into a *binary asset stream* — a `.manifest` index plus a
`.bin` of relocatable instance data (and optional `.relo`/`.imp` fixups). The
layout (little-endian) is:
```
ManifestHeader (48 B) isBigEndian u8, isLinked u8, version u16,
streamChecksum, allTypesHash, assetCount u32,
totalInstanceDataSize, maxInstance/maxRelocation/
maxImportsChunkSize, assetReferenceBufferSize,
referenceManifestNameBufferSize, assetNameBufferSize,
sourceFileNameBufferSize
AssetEntry (48 B) * count
typeId, instanceId, typeHash, instanceHash,
assetReferenceOffset i32, assetReferenceCount i32,
nameOffset i32, sourceFileNameOffset i32,
instanceDataSize i32, relocationDataSize i32,
importsDataSize i32, tokenized u32
then the reference / referenced-name / asset-name / source-name buffers
```
Asset names are `Type:Instance` (e.g. `W3DMesh:BB_GRASS02`). Instance pointers
are stored as offsets from the start of the instance data (which begins at byte
4 of `.bin`, after the stream checksum), so a slice is readable without the
relocation stream.
Each multiplayer map carries its own stream (`data\maps\official\<id>\map.bin`)
but it is **linked**: only map-specific assets (the terrain texture atlas,
scripts, `GameMap`) have data; the rendered props are imported and therefore
have `instanceDataSize == 0`. The complete prop art (meshes + textures) lives in
`Data\WBData.big`'s `data\worldbuilder.bin`, which is **uncompressed** (first
four bytes are the stream checksum, not `10 FB`), so `ra3.models` reads the
1.1 GB stream lazily — the manifest is parsed and only the needed instance
slices are read.
`W3DMesh` compiled layout (offsets from the instance start):
```
+4 vertexBufferPtr +52 triangleCount +56 triangleItemPtr
+60 shaderNameLength +64 shaderNamePtr
vertexBuffer: +0 numVertices, +4 stride, +8 elementDataPtr,
+12 declarationBytes, +16 declarationPtr
declaration: text "p0:00:3f32 n0:0C:3f32 t0:1C:2f32" (D3D9 usage:index:offset:type)
triangles: triangleCount * { u32 indexCount, u32 indexPtr } (24 B each), u32 indices
```
The diffuse texture is found through the mesh's `FXShaderConstant`s
(`+76` count, `+80` items): a texture-valued constant (TypeId `0xA59096A6`)
names its role (`DiffuseTexture`, `NormalMap`, `SpecMap`) and points at a
1-based index into the mesh's cross-asset references, which resolve by
`(typeId, instanceId)`. The `Texture` instance embeds a standard DDS file (at
`u32@+4`, or scan for `"DDS "`); `ra3.models::decode_dds` decodes DXT1/3/5 and
uncompressed 16/24/32-bit.
**Vertices are stored in bone space, not object space.** A mesh whose vertex
declaration carries blend data (`i0:..:4u8 w0:..:4u8n`, e.g. buildings and
vehicles) must be skinned; props without it (`BB_GRASS02`, `IF_STREETSEGMENT01`)
are already in object space. The skeleton is a `W3DHierarchy` asset named after
the mesh's instance prefix (`W3DMesh:FI_STRUCTURE_02.NEWSKIN_CIV01` →
`W3DHierarchy:FI_STRUCTURE_02`). Compiled layout:
```
W3DHierarchy: u32 pad, u32 boneCount, u32 headerBytes, then boneCount records
100 B each:
u32 nameHash, i32 parent (-1 = root), f32 translation[3],
f32 quaternion[4] (x, y, z, w), f32 matrix[12]
```
The default (bind) pose is rebuilt by composing each bone's local
translation/quaternion down the parent chain, then `skinnedPos = Σ wᵢ ·
(Rᵢ·p + Tᵢ)` (and the normal by the rotation only). `ra3.models` does this before
placing the mesh at the map object's `(x, y, angle)`.
The compiled shader (below) binds **one joint per vertex** — `WorldBones` holds
64 bones as 2 `float4` each (quaternion `c[128+2j]`, translation `c[129+2j]`) —
so the skin is rigid: `blendindices.x` selects the joint, remapped through the
mesh's per-model **bone table** (vertex-descriptor `+0x14` = bone count,
`+0x18` = `u16` bone indices into the `W3DHierarchy`). Applying the raw blend
index without that remap tears models apart (`FI_BUILDING01`'s table is
`[0,14,15,16,17,18]`, not `[0..5]`).
The map objects that lie flat on the ground (sidewalks, deck pieces) are
coplanar with the terrain; retail biases their depth in the shader so they do
not z-fight. The object pass reproduces that with a small negative depth bias
(Vulkan `depthBiasConstant/SlopeFactor`, and a half-unit bias in the software
rasteriser).
Meshes whose material has no diffuse texture (`DefaultW3D.fx`, `BasicW3D.fx` —
`FXLIGHTS`/ambient helper billboards) are not opaque geometry and are skipped;
drawing them fills the frame with garbage triangles. Likewise the
`BuildingsGenericDamageFill.fx` **damage-fill** sub-meshes are skipped: they are
the wrecked-interior shell (e.g. `CBBuilding_Wood`, an orange plank texture) that
retail only reveals through damage holes, but our opaque pass would paint it over
the main shell and tint whole buildings warm.
### Official shader behaviour (`Shaders.big` → `*.fxo`)
The compiled D3D9 effects in `Data\Shaders.big` name their parameters, so the
model pipeline is recoverable. `buildingsgeneric.fxo` (`BuildingsGeneric.fx`)
vertex stage: skinning (above), `World`/`ViewProjection`, and vertex color
`c0` multiplied into the lit color
(`(Ambient·AmbientColor + Σ DLᵢ.Color·max(N·DLᵢ,0)) · DiffuseColor · vertexColor`).
Pixel stage samples `DiffuseTexture`/`NormalMap`/`SpecMap`/`DamagedTexture`/
`CloudTexture` (all at **UV0**, except `DamagedTexture` at `v0.wz` = transposed
UV1), then `final.rgb *= TintColor` and `*= ShroudTexture.rgb`. `DiffuseVelocity`
is not used for static structures. So the diffuse texture is UV0 and is tinted by
vertex color and `TintColor`; `basicw3d.fxo` instead modulates a single
macro/lightmap with `(vertexColor + additive) * diffuse * 2` and has no normal
map. (Recovered by disassembling the embedded `vs_3_0`/`ps_3_0` bytecode.)
### Water / ocean (`Ocean.fx`, `OceanDisplacement.fx`, `RiverWater.fx`, `UnderwaterDeferred.fx`)
The SAGE water surface is reconstructed by OpenSAGE as
`Assets/Shaders/Water.vert`+`Water.frag` (same family as retail `Ocean.fx`). The
surface is a world-space mesh drawn with alpha blending, fed by two render
targets rendered before it: a **reflection** map (scene from the mirrored camera
about `GlobalWaterSettings.ReflectionPlaneZ`) and a **refraction** map + depth.
The fragment model (`Water.frag`) is:
- `waterUV = worldPos.xy / 320`; a scrolling `WaterTexture` supplies both a
flow distortion (`(tex.xy*2-1)*0.05`) and the flow layer; a `BumpTexture`
supplies the surface `worldNormal`.
- `fresnelFactor = dot(viewVector, +Z)`; reflection/refraction are sampled in
screen space (`gl_FragCoord / ViewportSize`), each displaced by the distortion.
- `linearWaterDepth = linearize(RefractionDepth) - linearize(gl_FragCoord.z)`;
`alpha = clamp((linearWaterDepth/2)/TransparentWaterDepth, 0, TransparentWaterMinOpacity)`.
- `final = diffuseColor * textureColor * cloudColor`, then mixed with
`mix(reflectionColor, refractionColor, fresnelFactor)` (both maps on) or just
one of them, per `IsRenderReflection` / `IsRenderRefraction`.
- Per-time-of-day `WaterSet`: `WaterTexture`, `UScrollPerMS`/`VScrollPerMS`,
`DiffuseColor`, `TransparentDiffuseColor`; `WaterTransparency` supplies
`TransparentWaterDepth`/`TransparentWaterMinOpacity`, the skybox faces,
`RiverTransparencyMultiplier`, `ReflectionPlaneZ`/`ReflectionOn`.
OpenRA3 has no water mesh or reflection/refraction targets (its terrain is
raymarched), so the model is folded into the terrain pass (`shaders/terrain.frag`
and its HLSL/GLSL-ES/WGSL twins): the ray's water-plane hit takes a scrolling
wave normal built from the retail bump map combined with a de-gridded procedural
wave, Schlick fresnel (F0 = 0.02), a sky reflection and a depth-graded
refraction, SAGE diffuse + specular lighting, a depth-based transparency fade,
and — when the camera is below `ReflectionPlaneZ` — an underwater tint/fog
(`UnderwaterDeferred.fx`). The two retail maps
`art/terrain/ra3_deepocean.tga` (flow/distortion) and `ra3_deepocean_nrm.tga`
(bump normal) are appended as the **last two layers of the terrain atlas**
(`ra3::terrain::build_gpu_terrain`, `water_flow = layer_count - 2`,
`water_normal = layer_count - 1`), so every backend samples them with the
existing atlas binding; absent maps fall back to a neutral layer. True
reflection/refraction render targets are still the next step.
`GPUParticleOceanDisplacement.fx` drives wave displacement from a GPU particle
buffer and has no analogue here.
The map's `ObjectsList` chunk is a list of nested `Object` assets —
`Coord3D`, Z `angle`, `RoadType` u32, a `u16`-prefixed type-name and a property
list whose keys index the shared name table (`ra3.map::parse_objects`). Each
type resolves to the `W3DMesh` parts whose instance name equals it or starts
with `<type>.`.
### Roads / sidewalks (`Road` assets)
The flat sidewalk and road decals are not meshes: the map places them as
**consecutive pairs** of objects — a `RoadType::Start` (`2`) and a
`RoadType::End` (`4`) at the segment's two ends (`Angled` `8`, `TightCurve`
`64` and `EndCap` `128` are curve/cap flags; `BridgeStart/End` `16/32` are
bridges). The road's type-name (`IslandFortressSidewalk01`, ...) resolves to a
compiled `Road` asset in the same art stream:
```
Road: u32 version, u32 pad, u32 textureCount, f32 roadWidth, f32 pad,
f32 ???, then textureCount texture references (diffuse, normal)
```
`ra3.models` pairs the objects in file order, emits a flat quad ribbon of
`roadWidth` along each segment (overlapping the joins by half a width), sampled
with the diffuse texture, and lifts it onto the terrain. Retail gives roads a
small depth offset toward the screen so they do not clip into the ground; the
object pass reproduces that with the same negative depth bias used for the other
ground decals (`depthBiasConstant/SlopeFactor` on Vulkan, a half-unit bias in the
software rasteriser).
### Map display names ### Map display names
The skirmish map list labels live in `Data\English.big`'s The skirmish map list labels live in `Data\English.big`'s
@@ -227,5 +409,9 @@ values are UTF-16 code units whose low byte is XORed with `0xFF`
- `MPPositionList` layout (per-player starts for maps without waypoints). - `MPPositionList` layout (per-player starts for maps without waypoints).
- Cliff textures and the `CliffTextureMapping` UV remap (`CliffTextures` is - Cliff textures and the `CliffTextureMapping` UV remap (`CliffTextures` is
parsed but not yet drawn). parsed but not yet drawn).
- The `Road` network mesher (the map's sidewalk/road objects reference `Road`
templates, not `W3DMesh` assets).
- W3D container/hierarchy assembly and animation (props are drawn as their
static mesh parts; skinned/animated in-game models are not).
- Compiled asset blobs (`global.bin`, `static.*.bin`) and the `.manifest` - Compiled asset blobs (`global.bin`, `static.*.bin`) and the `.manifest`
schema used to deserialise them. schema used to deserialise them.
+19
View File
@@ -0,0 +1,19 @@
#!/usr/bin/env bash
# Build the Debian (latest stable) Linux/ARM64 target (.deb + binary).
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
IMAGE="${OPENRA3_DEBIAN_ARM64_IMAGE:-openra3-debian-arm64:local}"
BUILD_DIR="${1:-build/debian-arm64}"
docker build --target dev -f "$ROOT/Dockerfile.debian-arm64" -t "$IMAGE" "$ROOT"
docker run --rm -v "$ROOT:/work" -w /work "$IMAGE" cmake -S . -B "$BUILD_DIR" -G Ninja \
-DCMAKE_TOOLCHAIN_FILE=cmake/toolchains/clang-aarch64-linux-gnu.cmake \
-DCMAKE_BUILD_TYPE=Release \
-DOPENRA3_AARCH64_CC=clang-19 -DOPENRA3_AARCH64_CXX=clang++-19 \
-DOPENRA3_AARCH64_STDLIB_MODULES_JSON=/usr/lib/llvm-19/lib/libc++.modules.json \
-DOPENRA3_DISTRO_TAG=debian13
docker run --rm -v "$ROOT:/work" -w /work "$IMAGE" cmake --build "$BUILD_DIR" -j
docker run --rm -v "$ROOT:/work" -w /work "$IMAGE" bash -c "cd $BUILD_DIR && cpack -G DEB"
echo "Debian arm64: $ROOT/$BUILD_DIR/bin/openra3 (+ .deb)"
+17
View File
@@ -0,0 +1,17 @@
#!/usr/bin/env bash
# Build the Debian (latest stable) target (.deb + binary) with the isolated image.
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
IMAGE="${OPENRA3_DEBIAN_IMAGE:-openra3-debian:local}"
BUILD_DIR="${1:-build/debian}"
docker build --target dev -f "$ROOT/Dockerfile.debian" -t "$IMAGE" "$ROOT"
docker run --rm -v "$ROOT:/work" -w /work "$IMAGE" cmake -S . -B "$BUILD_DIR" -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_CXX_STDLIB_MODULES_JSON=/usr/lib/llvm-19/lib/libc++.modules.json \
-DOPENRA3_DISTRO_TAG=debian13
docker run --rm -v "$ROOT:/work" -w /work "$IMAGE" cmake --build "$BUILD_DIR" -j
docker run --rm -v "$ROOT:/work" -w /work "$IMAGE" bash -c "cd $BUILD_DIR && cpack -G DEB"
echo "Debian: $ROOT/$BUILD_DIR/bin/openra3 (+ .deb)"
+16
View File
@@ -0,0 +1,16 @@
#!/usr/bin/env bash
# Build the Linux/ARM64 target (.deb + binary) with the isolated cross image.
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
IMAGE="${OPENRA3_LINUX_ARM64_IMAGE:-openra3-linux-arm64:local}"
BUILD_DIR="${1:-build/linux-arm64}"
docker build --target dev -f "$ROOT/Dockerfile.linux-arm64" -t "$IMAGE" "$ROOT"
docker run --rm -v "$ROOT:/work" -w /work "$IMAGE" cmake -S . -B "$BUILD_DIR" -G Ninja \
-DCMAKE_TOOLCHAIN_FILE=cmake/toolchains/clang-aarch64-linux-gnu.cmake \
-DCMAKE_BUILD_TYPE=Release -DOPENRA3_DISTRO_TAG=ubuntu26.04
docker run --rm -v "$ROOT:/work" -w /work "$IMAGE" cmake --build "$BUILD_DIR" -j
docker run --rm -v "$ROOT:/work" -w /work "$IMAGE" bash -c "cd $BUILD_DIR && cpack -G DEB"
echo "Linux arm64: $ROOT/$BUILD_DIR/bin/openra3 (+ .deb)"
+38
View File
@@ -0,0 +1,38 @@
#!/usr/bin/env bash
# Build the WebAssembly target (openra3.js/.wasm + worker + page) with the
# Emscripten image.
#
# The engine runs inside a plain Web Worker and renders into an OffscreenCanvas
# with no SDL, so assets load on demand: point OPENRA3_WEB_ASSETS at the
# extracted asset tree and the build generates an assets manifest, embeds it, and
# registers every file as an Emscripten lazy file at startup. The browser then
# fetches each asset only when the engine opens it.
#
# OPENRA3_WEB_ASSETS=/path/to/assets scripts/build-wasm.sh
#
# open http://localhost:8199/index.html
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
IMAGE="${OPENRA3_WASM_IMAGE:-openra3-wasm:local}"
BUILD_DIR="${1:-build/wasm}"
docker build --target dev -f "$ROOT/Dockerfile.wasm" -t "$IMAGE" "$ROOT"
mounts=(-v "$ROOT:/work")
configure=(-DCMAKE_TOOLCHAIN_FILE=cmake/toolchains/emscripten-wasm.cmake -DCMAKE_BUILD_TYPE=Release)
serve_assets=()
if [ -n "${OPENRA3_WEB_ASSETS:-}" ]; then
mounts+=(-v "$OPENRA3_WEB_ASSETS:/assets:ro")
manifest="$ROOT/$BUILD_DIR/assets.manifest.json"
mkdir -p "$(dirname "$manifest")"
python3 "$ROOT/scripts/make_web_manifest.py" --root "$OPENRA3_WEB_ASSETS" --out "$manifest"
configure+=(-DOPENRA3_WEB_MANIFEST="/work/$BUILD_DIR/assets.manifest.json")
serve_assets=(--assets "$OPENRA3_WEB_ASSETS")
fi
docker run --rm "${mounts[@]}" -w /work "$IMAGE" cmake -S . -B "$BUILD_DIR" -G Ninja "${configure[@]}"
docker run --rm "${mounts[@]}" -w /work "$IMAGE" cmake --build "$BUILD_DIR" -j
echo "WebAssembly: $ROOT/$BUILD_DIR/bin/index.html (+ openra3.js/.wasm, openra3.worker.js)"
echo "serve: python3 \"$ROOT/apps/web/serve.py\" --root \"$ROOT/$BUILD_DIR/bin\" ${serve_assets[*]:-}"
+16
View File
@@ -0,0 +1,16 @@
#!/usr/bin/env bash
# Build the Windows/ARM64 target (portable .zip) with the isolated cross image.
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
IMAGE="${OPENRA3_WINDOWS_ARM64_IMAGE:-openra3-windows-arm64:local}"
BUILD_DIR="${1:-build/windows-arm64}"
docker build --target dev -f "$ROOT/Dockerfile.win-arm64" -t "$IMAGE" "$ROOT"
docker run --rm -v "$ROOT:/work" -w /work "$IMAGE" cmake -S . -B "$BUILD_DIR" -G Ninja \
-DCMAKE_TOOLCHAIN_FILE=cmake/toolchains/llvm-mingw-aarch64.cmake \
-DCMAKE_BUILD_TYPE=Release
docker run --rm -v "$ROOT:/work" -w /work "$IMAGE" cmake --build "$BUILD_DIR" -j
docker run --rm -v "$ROOT:/work" -w /work "$IMAGE" bash -c "cd $BUILD_DIR && cpack -G ZIP"
echo "Windows arm64: $ROOT/$BUILD_DIR/bin/openra3.exe (+ .zip)"
+46
View File
@@ -0,0 +1,46 @@
#!/usr/bin/env python3
"""Generate the wasm lazy-asset manifest.
The wasm build runs the engine on a worker, where synchronous XHR is legal, so
assets are registered with ``FS.createLazyFile`` instead of being preloaded: the
browser fetches each file only when the engine first opens it. This script lists
every file under the assets root (as POSIX-relative paths) as
``{"files": [...]}``; CMake preloads the manifest to ``/assets.manifest.json``.
scripts/make_web_manifest.py --root /path/to/assets --out assets.manifest.json
Serve the tree at ``<wasm-root>/assets`` (or use ``serve.py --assets``).
"""
import argparse
import json
import os
import sys
def main() -> int:
parser = argparse.ArgumentParser(description="Generate the OpenRA3 wasm asset manifest")
parser.add_argument("--root", required=True, help="assets root to enumerate")
parser.add_argument("--out", required=True, help="manifest JSON to write")
args = parser.parse_args()
if not os.path.isdir(args.root):
print(f"root not found: {args.root}", file=sys.stderr)
return 1
files = []
for dirpath, dirnames, filenames in os.walk(args.root):
dirnames.sort()
for name in sorted(filenames):
rel = os.path.relpath(os.path.join(dirpath, name), args.root).replace(os.sep, "/")
files.append(rel)
with open(args.out, "w", encoding="utf-8") as handle:
json.dump({"files": files}, handle, separators=(",", ":"))
handle.write("\n")
print(f"wrote {args.out} ({len(files)} files)")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+2 -2
View File
@@ -25,9 +25,9 @@ compile() {
fi fi
} }
for name in scene terrain; do for name in scene terrain object; do
compile "$DIR/$name.vert" "$OUT/$name.vert.spv" compile "$DIR/$name.vert" "$OUT/$name.vert.spv"
compile "$DIR/$name.frag" "$OUT/$name.frag.spv" compile "$DIR/$name.frag" "$OUT/$name.frag.spv"
done done
echo "wrote $OUT/{scene,terrain}.{vert,frag}.spv" echo "wrote $OUT/{scene,terrain,object}.{vert,frag}.spv"
+162 -40
View File
@@ -8,6 +8,10 @@
// `Terrain.fx` / OpenSAGE `Terrain.frag` do, so it never restarts at a cell // `Terrain.fx` / OpenSAGE `Terrain.frag` do, so it never restarts at a cell
// edge; material boundaries cross-fade with the SAGE blend ramp. // edge; material boundaries cross-fade with the SAGE blend ramp.
// //
// The water plane is shaded with a port of the SAGE water effect
// (`Ocean.fx` / OpenSAGE `Water.frag`); the camera-below-water tint stands in
// for `UnderwaterDeferred.fx`. See docs/REVERSE_ENGINEERING.md.
//
// The Vulkan push constants (20 floats) become a constant buffer. // The Vulkan push constants (20 floats) become a constant buffer.
cbuffer TerrainCB : register(b0) { cbuffer TerrainCB : register(b0) {
@@ -15,7 +19,7 @@ cbuffer TerrainCB : register(b0) {
float4 params; // x=pitch, y=fov, z=water_z, w=has_water float4 params; // x=pitch, y=fov, z=water_z, w=has_water
float4 sun; // xyz=sun dir, w=ambient float4 sun; // xyz=sun dir, w=ambient
float4 mapinfo; // x=W, y=H, z=unused, w=z_scale float4 mapinfo; // x=W, y=H, z=unused, w=z_scale
float4 misc; // x=time, y=unused, z=cells per texture repeat, w=aspect float4 misc; // x=time, y=water kind (0=ocean,1=river), z=cells per repeat, w=aspect
}; };
Texture2D<float> heightmap : register(t0); Texture2D<float> heightmap : register(t0);
@@ -27,6 +31,12 @@ SamplerState atlas_smp : register(s2);
static const float CELL = 10.0; // must match ra3::terrain::cell_size static const float CELL = 10.0; // must match ra3::terrain::cell_size
// SAGE water model constants (see docs/REVERSE_ENGINEERING.md).
static const float WATER_SCALE = 1.0 / 320.0;
static const float WATER_TRANSPARENT_DEPTH = 10.0;
static const float WATER_MIN_OPACITY = 0.70;
static const float WATER_RIVER_MULTIPLIER = 1.0;
struct VSOut { struct VSOut {
float4 pos : SV_Position; float4 pos : SV_Position;
float2 uv : TEXCOORD0; float2 uv : TEXCOORD0;
@@ -104,6 +114,105 @@ float3 sample_layer(uint layer, float wx, float wy) {
return atlas.Sample(atlas_smp, float3(float2(wx, wy) / span, l)).rgb; return atlas.Sample(atlas_smp, float3(float2(wx, wy) / span, l)).rgb;
} }
// ---- SAGE water (Ocean.fx / RiverWater.fx / Water.frag port) ----------------
// Sample an atlas layer by an explicit layer index (the water flow/bump maps
// are appended as the last two layers of the tile atlas).
float3 water_tex(int layer, float2 uv) {
uint w = 0;
uint h = 0;
uint lc = 0;
atlas.GetDimensions(w, h, lc);
float l = (float) clamp(layer, 0, (int) lc - 1);
return atlas.Sample(atlas_smp, float3(uv, l)).rgb;
}
// Scrolling wave normal on the water plane: the retail bump map (atlas's last
// layer), offset by the flow map (second-last layer) and combined with a
// de-gridded procedural wave so the sun glint is irregular and always moving.
float3 water_normal(float2 world_xy, float time) {
float2 q = world_xy * (WATER_SCALE * 6.0);
float a1 = q.x * 0.80 + q.y * 0.30 + time * 1.10;
float a2 = q.y * 0.95 - q.x * 0.45 - time * 1.30;
float a3 = (q.x + q.y) * 1.60 + time * 2.10;
float a4 = (q.x - q.y) * 2.30 - time * 1.70;
float dx = 0.224 * cos(a1) - 0.099 * cos(a2) + 0.256 * cos(a3) + 0.230 * cos(a4);
float dy = 0.084 * cos(a1) + 0.209 * cos(a2) + 0.256 * cos(a3) - 0.230 * cos(a4);
uint w = 0;
uint h = 0;
uint lc = 0;
atlas.GetDimensions(w, h, lc);
float3 flow = water_tex((int) lc - 2, q - float2(time * 0.010, time * 0.014)) * 2.0 - 1.0;
float3 bump = water_tex((int) lc - 1, q + flow.xy * 0.05 + float2(time * 0.006, time * 0.008)) * 2.0 - 1.0;
float sx = -dx * 0.30 + bump.x * 0.45;
float sy = -dy * 0.30 + bump.y * 0.45;
return normalize(float3(sx, sy, 1.0));
}
// Water.frag distortionPower * the flow texture: a small scrolling UV offset.
float water_distortion(float2 world_xy, float time) {
float2 q = world_xy * (WATER_SCALE * 6.0);
return 0.05 * (sin(q.x * 0.9 + time * 0.7) + sin(q.y * 1.1 - time * 0.5));
}
// Water.frag GetCloudColor: no cloud texture is bound, so a slow low-frequency
// scroll stands in.
float3 water_cloud(float2 world_xy, float time) {
return float3(0.85 + 0.15 * sin((world_xy.x + world_xy.y) * 0.0007 - time * 0.05));
}
// Retail UnderwaterDeferred.fx: absorbs red and fogs with distance. Applied to
// every output while the camera is below the water plane.
float3 apply_underwater(float3 color, float dist, float cam_z, float water_z) {
if (cam_z >= water_z - 0.5) return color;
const float3 absorb = float3(0.35, 0.62, 0.75);
float fog = clamp(1.0 - exp(-dist * 0.00022), 0.0, 0.9);
return lerp(color * absorb, float3(0.02, 0.10, 0.16), fog);
}
// Water.frag: fresnel-mixed reflection (sky) and refraction (depth-graded
// seabed), SAGE diffuse + specular lighting, cloud term, and a depth-based
// transparency fade. `river` (>0.5) applies RiverTransparencyMultiplier.
float3 water_shade(float3 hitpos, float3 dir, float dist) {
float time = misc.x;
float river = misc.y;
float seabed = world_height(hitpos.x, hitpos.y);
float depth = max(0.0, params.z - seabed);
float3 n = water_normal(hitpos.xy, time + water_distortion(hitpos.xy, time));
float3 sun_dir = normalize(sun.xyz);
// Schlick fresnel, water F0 = 0.02.
float cos_theta = clamp(dot(-dir, n), 0.0, 1.0);
float fresnel = 0.02 + 0.98 * pow(1.0 - cos_theta, 5.0);
// Reflection: the sky the surface mirrors. Refraction: the seabed, graded
// from shallow to deep and lit by the SAGE diffuse + specular model.
float3 reflection = sky_color(reflect(dir, n));
float3 shallow = float3(0.10, 0.34, 0.38);
float3 deep = float3(0.02, 0.12, 0.22);
float3 refraction = lerp(shallow, deep, clamp(depth / 40.0, 0.0, 1.0));
float ndotl = max(dot(n, sun_dir), 0.0);
float ambient = sun.w;
float3 diffuse = float3(ambient + (1.0 - ambient) * ndotl);
float3 half_v = normalize(sun_dir - dir);
float spec = pow(max(dot(n, half_v), 0.0), 90.0);
float3 color = lerp(refraction, reflection, clamp(fresnel, 0.0, 1.0)) * diffuse * water_cloud(hitpos.xy, time);
color += float3(1.0, 0.97, 0.9) * spec * 0.45;
// Depth-based transparency: shallow water shows the seabed, deep water goes
// opaque toward the deep colour.
float alpha = clamp(depth / WATER_TRANSPARENT_DEPTH, 0.0, 1.0) * WATER_MIN_OPACITY;
if (river > 0.5) alpha *= WATER_RIVER_MULTIPLIER;
color = lerp(refraction, color, clamp(alpha + 0.15, 0.0, 1.0));
// Distance haze toward the horizon, as the terrain.
float wfog = clamp(1.0 - exp(-dist * 0.00009), 0.0, 0.75);
return lerp(color, sky_color(float3(dir.x, dir.y, 0.0)), wfog);
}
float4 PSMain(VSOut input) : SV_Target { float4 PSMain(VSOut input) : SV_Target {
float4 p = cam; float4 p = cam;
float pitch = clamp(params.x, 0.15, 1.45); float pitch = clamp(params.x, 0.15, 1.45);
@@ -127,48 +236,75 @@ float4 PSMain(VSOut input) : SV_Target {
float3 dir = normalize(fwd + right * ndc.x * th * aspect + up * ndc.y * th); float3 dir = normalize(fwd + right * ndc.x * th * aspect + up * ndc.y * th);
if (dir.z >= -1e-4) { if (dir.z >= -1e-4) {
return float4(sky_color(dir), 1.0); return float4(apply_underwater(sky_color(dir), 0.0, cam_pos.z, params.z), 1.0);
} }
// March the heightfield (bounded work: the step grows toward the horizon). // Clip the ray to the map's XY rectangle: the boundary is an exact plane,
float t = CELL * 0.5; // so the silhouette there stays razor-sharp instead of stair-stepping
float dt = CELL * 0.5; // across it. Outside the map is sky.
float t_enter = 0.0;
float t_exit = 1.0e30;
bool inside = true;
if (abs(dir.x) < 1e-6) {
inside = (cam_pos.x >= 0.0 && cam_pos.x <= world_w);
} else {
float a = (0.0 - cam_pos.x) / dir.x;
float b = (world_w - cam_pos.x) / dir.x;
t_enter = max(t_enter, min(a, b));
t_exit = min(t_exit, max(a, b));
}
if (inside) {
if (abs(dir.y) < 1e-6) {
inside = (cam_pos.y >= 0.0 && cam_pos.y <= world_h);
} else {
float a = (0.0 - cam_pos.y) / dir.y;
float b = (world_h - cam_pos.y) / dir.y;
t_enter = max(t_enter, min(a, b));
t_exit = min(t_exit, max(a, b));
}
}
if (!inside || t_exit <= 0.0) {
return float4(apply_underwater(sky_color(dir), 0.0, cam_pos.z, params.z), 1.0);
}
// March the heightfield cell by cell: the step is never longer than the
// time to cross one cell (dominant horizontal axis), while a clearance term
// lets the ray skip the empty air above the surface. Resolving every cell is
// what keeps cliff and map-edge silhouettes from quantising into huge
// stair-steps that crawl as the camera pans.
float horiz = max(abs(dir.x), abs(dir.y));
float cell_step = min(CELL / max(horiz, 1e-4), CELL * 32.0);
float t = max(t_enter, CELL * 0.5);
float prev = t; float prev = t;
bool hit = false; bool hit = false;
float hit_t = 0.0; float hit_t = 0.0;
for (int i = 0; i < 512 && t < 60000.0; ++i) { for (int i = 0; i < 1024 && t <= t_exit; ++i) {
float3 w = cam_pos + dir * t; float3 w = cam_pos + dir * t;
if (w.x < 0.0 || w.y < 0.0 || w.x >= world_w || w.y >= world_h) { float h = world_height(w.x, w.y);
prev = t; float surface = (params.w > 0.5) ? max(h, params.z) : h;
dt *= 1.06; if (w.z <= surface) {
t += dt;
continue;
}
if (params.w > 0.5 && w.z <= params.z) {
hit = true;
hit_t = t;
break;
}
if (w.z <= world_height(w.x, w.y)) {
hit = true; hit = true;
hit_t = t; hit_t = t;
break; break;
} }
float clearance = (w.z - surface) / max(-dir.z, 1e-4);
prev = t; prev = t;
dt *= 1.06; t += clamp(clearance, cell_step, cell_step * 8.0);
t += dt;
} }
if (!hit) { if (!hit) {
return float4(sky_color(dir), 1.0); return float4(apply_underwater(sky_color(dir), 0.0, cam_pos.z, params.z), 1.0);
} }
// Refine the first crossing; with a sub-cell bracket this converges to the
// exact surface point.
float lo = prev; float lo = prev;
float hi = hit_t; float hi = hit_t;
for (int i = 0; i < 6; ++i) { for (int i = 0; i < 18; ++i) {
float mid = 0.5 * (lo + hi); float mid = 0.5 * (lo + hi);
float3 w = cam_pos + dir * mid; float3 w = cam_pos + dir * mid;
bool water = params.w > 0.5 && w.z <= params.z; float h = world_height(w.x, w.y);
if (water || w.z <= world_height(w.x, w.y)) { float surface = (params.w > 0.5) ? max(h, params.z) : h;
if (w.z <= surface) {
hi = mid; hi = mid;
} else { } else {
lo = mid; lo = mid;
@@ -180,21 +316,7 @@ float4 PSMain(VSOut input) : SV_Target {
float ambient = sun.w; float ambient = sun.w;
if (params.w > 0.5 && hitpos.z <= params.z + 0.01) { if (params.w > 0.5 && hitpos.z <= params.z + 0.01) {
// Water: animated normal from a procedural wave, sky reflection + fresnel. return float4(apply_underwater(water_shade(hitpos, dir, hi), hi, cam_pos.z, params.z), 1.0);
float time = misc.x;
float2 q = hitpos.xy * 0.015;
float nx = sin(q.x * 1.3 + time * 1.7) + 0.5 * sin(q.x * 3.1 - time * 2.3);
float ny = sin(q.y * 1.1 - time * 1.3) + 0.5 * sin(q.y * 2.7 + time * 1.9);
float3 n = normalize(float3(nx * 0.06, ny * 0.06, 1.0));
float fres = pow(1.0 - clamp(-dir.z, 0.0, 1.0), 3.0);
float3 deep = float3(0.03, 0.16, 0.28);
float3 refl = sky_color(reflect(dir, n));
float lam = max(0.0, dot(n, sun_dir));
float3 water = lerp(deep, refl, clamp(0.25 + 0.55 * fres, 0.0, 0.9));
water += float3(1.0, 0.98, 0.9) * pow(lam, 64.0) * 0.6; // sun glint
float wfog = clamp(1.0 - exp(-hi * 0.00009), 0.0, 0.75);
water = lerp(water, sky_color(float3(dir.x, dir.y, 0.0)), wfog);
return float4(water, 1.0);
} }
// Terrain: read the per-cell blend record, sample the base/blend/three-way // Terrain: read the per-cell blend record, sample the base/blend/three-way
@@ -232,5 +354,5 @@ float4 PSMain(VSOut input) : SV_Target {
// Distance haze toward the horizon so the map edge blends into the sky. // Distance haze toward the horizon so the map edge blends into the sky.
float fog = clamp(1.0 - exp(-hi * 0.00009), 0.0, 0.75); float fog = clamp(1.0 - exp(-hi * 0.00009), 0.0, 0.75);
lit = lerp(lit, sky_color(float3(dir.x, dir.y, 0.0)), fog); lit = lerp(lit, sky_color(float3(dir.x, dir.y, 0.0)), fog);
return float4(lit, 1.0); return float4(apply_underwater(lit, hi, cam_pos.z, params.z), 1.0);
} }
Binary file not shown.
Binary file not shown.
Binary file not shown.
+36
View File
@@ -0,0 +1,36 @@
#version 450
// Static-map model fragment stage: sample the shared texture array and apply
// the same directional sun the terrain uses. Cut-out props (trees, fences)
// carry an alpha mask in their diffuse texture; discard the transparent texels
// so the ground shows through.
layout(binding = 0) uniform sampler2DArray atlas;
layout(push_constant) uniform Push {
vec4 cam_pos;
vec4 fwd;
vec4 right;
vec4 up;
vec4 sun;
} pc;
layout(location = 0) in vec2 in_uv;
layout(location = 1) in vec3 in_normal;
layout(location = 2) flat in float in_layer;
layout(location = 0) out vec4 out_color;
void main() {
int layers = textureSize(atlas, 0).z;
int layer = clamp(int(in_layer + 0.5), 0, layers - 1);
vec4 tex = texture(atlas, vec3(in_uv, float(layer)));
if (tex.a < 0.5) discard;
vec3 n = normalize(in_normal);
if (!gl_FrontFacing) n = -n;
float lambert = max(0.0, dot(n, normalize(pc.sun.xyz)));
float ambient = pc.sun.w;
vec3 lit = tex.rgb * (ambient + (1.0 - ambient) * lambert);
out_color = vec4(lit, 1.0);
}
+45
View File
@@ -0,0 +1,45 @@
#version 450
// Static-map model vertex stage. Draws the world-space triangle soup built by
// ra3::models (buildings and props the map places) against the exact camera the
// terrain raymarcher uses, so both passes share one projection and depth test.
//
// The camera is passed as its orthonormal basis so the same math as
// terrain.frag applies: ndc.x = dot(r, right) / (a * tan(fov/2) * aspect),
// ndc.y = -dot(r, up) / (a * tan(fov/2)), depth = (a - NEAR) / (FAR - NEAR)
// with `a = dot(r, fwd)` the view-space depth.
layout(location = 0) in vec3 in_pos;
layout(location = 1) in vec3 in_normal;
layout(location = 2) in vec2 in_uv;
layout(location = 3) in float in_layer;
layout(location = 4) in float in_bias;
layout(push_constant) uniform Push {
vec4 cam_pos; // xyz = eye position
vec4 fwd; // xyz = forward
vec4 right; // xyz = right, w = tan(fov / 2)
vec4 up; // xyz = up, w = tan(fov / 2) * aspect
vec4 sun; // xyz = sun direction, w = ambient
} pc;
layout(location = 0) out vec2 out_uv;
layout(location = 1) out vec3 out_normal;
layout(location = 2) flat out float out_layer;
const float NEAR = 10.0;
const float FAR = 60000.0;
void main() {
vec3 r = in_pos - pc.cam_pos.xyz;
float a = dot(r, pc.fwd.xyz);
float b = dot(r, pc.right.xyz);
float c = dot(r, pc.up.xyz);
float th = pc.right.w;
float th_aspect = pc.up.w;
float depth = clamp((a - in_bias - NEAR) / (FAR - NEAR), 0.0, 1.0);
gl_Position = vec4(b / th_aspect, -c / th, depth * a, a);
out_uv = in_uv;
out_normal = in_normal;
out_layer = in_layer;
}
+169 -32
View File
@@ -8,6 +8,10 @@
// The material is sampled **continuously** (`uv = cell / (2 * cellSize)`), as the // The material is sampled **continuously** (`uv = cell / (2 * cellSize)`), as the
// retail `Terrain.fx` / OpenSAGE `Terrain.frag` do, so it never restarts at a // retail `Terrain.fx` / OpenSAGE `Terrain.frag` do, so it never restarts at a
// cell edge; material boundaries cross-fade with the SAGE blend ramp. // cell edge; material boundaries cross-fade with the SAGE blend ramp.
//
// The water plane is shaded with a port of the SAGE water effect
// (`Ocean.fx` / OpenSAGE `Water.frag`); the camera-below-water tint stands in
// for `UnderwaterDeferred.fx`. See docs/REVERSE_ENGINEERING.md.
layout(binding = 0) uniform sampler2D heightmap; layout(binding = 0) uniform sampler2D heightmap;
layout(binding = 1) uniform sampler2D celldata; layout(binding = 1) uniform sampler2D celldata;
layout(binding = 2) uniform sampler2DArray atlas; layout(binding = 2) uniform sampler2DArray atlas;
@@ -17,7 +21,7 @@ layout(push_constant) uniform Push {
vec4 params; // x=pitch, y=fov, z=water_z, w=has_water vec4 params; // x=pitch, y=fov, z=water_z, w=has_water
vec4 sun; // xyz=sun dir, w=ambient vec4 sun; // xyz=sun dir, w=ambient
vec4 mapinfo; // x=W, y=H, z=unused, w=z_scale vec4 mapinfo; // x=W, y=H, z=unused, w=z_scale
vec4 misc; // x=time, y=unused, z=cells per texture repeat, w=aspect vec4 misc; // x=time, y=water kind (0=ocean,1=river), z=cells per repeat, w=aspect
} pc; } pc;
layout(location = 0) in vec2 in_uv; layout(location = 0) in vec2 in_uv;
@@ -25,6 +29,17 @@ layout(location = 0) out vec4 out_color;
const float CELL = 10.0; // must match ra3::terrain::cell_size const float CELL = 10.0; // must match ra3::terrain::cell_size
// Must match object.vert: the shared projection writes the view-space depth
// into gl_FragDepth so the static-map models depth-test against the terrain.
const float NEAR = 10.0;
const float FAR = 60000.0;
// SAGE water model constants (see docs/REVERSE_ENGINEERING.md).
const float WATER_SCALE = 1.0 / 320.0; // Water.frag: worldPos.xy / 320
const float WATER_TRANSPARENT_DEPTH = 10.0; // WaterTransparency.TransparentWaterDepth
const float WATER_MIN_OPACITY = 0.70; // WaterTransparency.TransparentWaterMinOpacity
const float WATER_RIVER_MULTIPLIER = 1.0; // WaterTransparency.RiverTransparencyMultiplier
float height_at(ivec2 c) { float height_at(ivec2 c) {
c = clamp(c, ivec2(0), ivec2(pc.mapinfo.xy) - 1); c = clamp(c, ivec2(0), ivec2(pc.mapinfo.xy) - 1);
return texelFetch(heightmap, c, 0).r * 65535.0 * pc.mapinfo.w; return texelFetch(heightmap, c, 0).r * 65535.0 * pc.mapinfo.w;
@@ -87,6 +102,99 @@ vec3 sample_layer(uint layer, float wx, float wy) {
return texture(atlas, vec3(vec2(wx, wy) / span, l)).rgb; return texture(atlas, vec3(vec2(wx, wy) / span, l)).rgb;
} }
// ---- SAGE water (Ocean.fx / RiverWater.fx / Water.frag port) ----------------
// Sample an atlas layer by an explicit float layer index (the water flow/bump
// maps are appended as the last two layers of the tile atlas).
vec3 water_tex(int layer, vec2 uv) {
int lc = textureSize(atlas, 0).z;
float l = float(clamp(layer, 0, max(lc - 1, 0)));
return texture(atlas, vec3(uv, l)).rgb;
}
// Scrolling wave normal on the water plane: the retail bump map (atlas's last
// layer), offset by the flow map (second-last layer) and combined with a
// de-gridded procedural wave so the sun glint is irregular and always moving.
vec3 water_normal(vec2 world_xy, float time) {
vec2 q = world_xy * (WATER_SCALE * 6.0);
float a1 = q.x * 0.80 + q.y * 0.30 + time * 1.10;
float a2 = q.y * 0.95 - q.x * 0.45 - time * 1.30;
float a3 = (q.x + q.y) * 1.60 + time * 2.10;
float a4 = (q.x - q.y) * 2.30 - time * 1.70;
float dx = 0.224 * cos(a1) - 0.099 * cos(a2) + 0.256 * cos(a3) + 0.230 * cos(a4);
float dy = 0.084 * cos(a1) + 0.209 * cos(a2) + 0.256 * cos(a3) - 0.230 * cos(a4);
int lc = textureSize(atlas, 0).z;
vec3 flow = water_tex(lc - 2, q - vec2(time * 0.010, time * 0.014)) * 2.0 - 1.0;
vec3 bump = water_tex(lc - 1, q + flow.xy * 0.05 + vec2(time * 0.006, time * 0.008)) * 2.0 - 1.0;
float sx = -dx * 0.30 + bump.x * 0.45;
float sy = -dy * 0.30 + bump.y * 0.45;
return normalize(vec3(sx, sy, 1.0));
}
// Water.frag distortionPower * the flow texture: a small scrolling UV offset.
float water_distortion(vec2 world_xy, float time) {
vec2 q = world_xy * (WATER_SCALE * 6.0);
return 0.05 * (sin(q.x * 0.9 + time * 0.7) + sin(q.y * 1.1 - time * 0.5));
}
// Water.frag GetCloudColor: no cloud texture is bound, so a slow low-frequency
// scroll stands in.
vec3 water_cloud(vec2 world_xy, float time) {
return vec3(0.85 + 0.15 * sin((world_xy.x + world_xy.y) * 0.0007 - time * 0.05));
}
// Retail UnderwaterDeferred.fx: absorbs red and fogs with distance. Applied to
// every output while the camera is below the water plane.
vec3 apply_underwater(vec3 color, float dist, float cam_z, float water_z) {
if (cam_z >= water_z - 0.5) return color;
const vec3 absorb = vec3(0.35, 0.62, 0.75);
float fog = clamp(1.0 - exp(-dist * 0.00022), 0.0, 0.9);
return mix(color * absorb, vec3(0.02, 0.10, 0.16), fog);
}
// Water.frag: fresnel-mixed reflection (sky) and refraction (depth-graded
// seabed), SAGE diffuse + specular lighting, cloud term, and a depth-based
// transparency fade. `river` (>0.5) applies RiverTransparencyMultiplier.
vec3 water_shade(vec3 hitpos, vec3 dir, float dist) {
float time = pc.misc.x;
float river = pc.misc.y;
float seabed = world_height(hitpos.x, hitpos.y);
float depth = max(0.0, pc.params.z - seabed);
vec3 n = water_normal(hitpos.xy, time + water_distortion(hitpos.xy, time));
vec3 sun = normalize(pc.sun.xyz);
// Schlick fresnel, water F0 = 0.02.
float cos_theta = clamp(dot(-dir, n), 0.0, 1.0);
float fresnel = 0.02 + 0.98 * pow(1.0 - cos_theta, 5.0);
// Reflection: the sky the surface mirrors. Refraction: the seabed, graded
// from shallow to deep and lit by the SAGE diffuse + specular model.
vec3 reflection = sky_color(reflect(dir, n));
vec3 shallow = vec3(0.10, 0.34, 0.38);
vec3 deep = vec3(0.02, 0.12, 0.22);
vec3 refraction = mix(shallow, deep, clamp(depth / 40.0, 0.0, 1.0));
float ndotl = max(dot(n, sun), 0.0);
float ambient = pc.sun.w;
vec3 diffuse = vec3(ambient + (1.0 - ambient) * ndotl);
vec3 half_v = normalize(sun - dir);
float spec = pow(max(dot(n, half_v), 0.0), 90.0);
vec3 color = mix(refraction, reflection, clamp(fresnel, 0.0, 1.0)) * diffuse * water_cloud(hitpos.xy, time);
color += vec3(1.0, 0.97, 0.9) * spec * 0.45;
// Depth-based transparency: shallow water shows the seabed, deep water goes
// opaque toward the deep colour.
float alpha = clamp(depth / WATER_TRANSPARENT_DEPTH, 0.0, 1.0) * WATER_MIN_OPACITY;
if (river > 0.5) alpha *= WATER_RIVER_MULTIPLIER;
color = mix(refraction, color, clamp(alpha + 0.15, 0.0, 1.0));
// Distance haze toward the horizon, as the terrain.
float wfog = clamp(1.0 - exp(-dist * 0.00009), 0.0, 0.75);
return mix(color, sky_color(vec3(dir.x, dir.y, 0.0)), wfog);
}
void main() { void main() {
vec4 p = pc.cam; vec4 p = pc.cam;
float pitch = clamp(pc.params.x, 0.15, 1.45); float pitch = clamp(pc.params.x, 0.15, 1.45);
@@ -112,55 +220,84 @@ void main() {
vec3 dir = normalize(fwd + right * ndc.x * th * aspect + up * ndc.y * th); vec3 dir = normalize(fwd + right * ndc.x * th * aspect + up * ndc.y * th);
if (dir.z >= -1e-4) { if (dir.z >= -1e-4) {
out_color = vec4(sky_color(dir), 1.0); gl_FragDepth = 1.0;
out_color = vec4(apply_underwater(sky_color(dir), 0.0, cam.z, pc.params.z), 1.0);
return; return;
} }
// March the heightfield (bounded work: the step grows toward the horizon). // Clip the ray to the map's XY rectangle: the boundary is an exact plane,
float t = CELL * 0.5; // so the silhouette there stays razor-sharp instead of stair-stepping
float dt = CELL * 0.5; // across it. Outside the map is sky.
float t_enter = 0.0;
float t_exit = 1.0e30;
bool inside = true;
if (abs(dir.x) < 1e-6) {
inside = (cam.x >= 0.0 && cam.x <= world_w);
} else {
float a = (0.0 - cam.x) / dir.x;
float b = (world_w - cam.x) / dir.x;
t_enter = max(t_enter, min(a, b));
t_exit = min(t_exit, max(a, b));
}
if (inside) {
if (abs(dir.y) < 1e-6) {
inside = (cam.y >= 0.0 && cam.y <= world_h);
} else {
float a = (0.0 - cam.y) / dir.y;
float b = (world_h - cam.y) / dir.y;
t_enter = max(t_enter, min(a, b));
t_exit = min(t_exit, max(a, b));
}
}
if (!inside || t_exit <= 0.0) {
gl_FragDepth = 1.0;
out_color = vec4(apply_underwater(sky_color(dir), 0.0, cam.z, pc.params.z), 1.0);
return;
}
// March the heightfield cell by cell: the step is never longer than the
// time to cross one cell (dominant horizontal axis), while a clearance term
// lets the ray skip the empty air above the surface. Resolving every cell is
// what keeps cliff and map-edge silhouettes from quantising into huge
// stair-steps that crawl as the camera pans.
float horiz = max(abs(dir.x), abs(dir.y));
float cell_step = min(CELL / max(horiz, 1e-4), CELL * 32.0);
float t = max(t_enter, CELL * 0.5);
float prev = t; float prev = t;
bool hit = false; bool hit = false;
float hit_t = 0.0; float hit_t = 0.0;
for (int i = 0; i < 512 && t < 60000.0; ++i) { for (int i = 0; i < 1024 && t <= t_exit; ++i) {
vec3 w = cam + dir * t; vec3 w = cam + dir * t;
if (w.x < 0.0 || w.y < 0.0 || w.x >= world_w || w.y >= world_h) { float h = world_height(w.x, w.y);
prev = t; dt *= 1.06; t += dt; continue; float surface = (pc.params.w > 0.5) ? max(h, pc.params.z) : h;
if (w.z <= surface) { hit = true; hit_t = t; break; }
float clearance = (w.z - surface) / max(-dir.z, 1e-4);
prev = t;
t += clamp(clearance, cell_step, cell_step * 8.0);
} }
if (pc.params.w > 0.5 && w.z <= pc.params.z) { hit = true; hit_t = t; break; } if (!hit) { gl_FragDepth = 1.0; out_color = vec4(apply_underwater(sky_color(dir), 0.0, cam.z, pc.params.z), 1.0); return; }
if (w.z <= world_height(w.x, w.y)) { hit = true; hit_t = t; break; }
prev = t; dt *= 1.06; t += dt;
}
if (!hit) { out_color = vec4(sky_color(dir), 1.0); return; }
// Refine the first crossing; with a sub-cell bracket this converges to the
// exact surface point.
float lo = prev, hi = hit_t; float lo = prev, hi = hit_t;
for (int i = 0; i < 6; ++i) { for (int i = 0; i < 18; ++i) {
float mid = 0.5 * (lo + hi); float mid = 0.5 * (lo + hi);
vec3 w = cam + dir * mid; vec3 w = cam + dir * mid;
bool water = pc.params.w > 0.5 && w.z <= pc.params.z; float h = world_height(w.x, w.y);
if (water || w.z <= world_height(w.x, w.y)) hi = mid; else lo = mid; float surface = (pc.params.w > 0.5) ? max(h, pc.params.z) : h;
if (w.z <= surface) hi = mid; else lo = mid;
} }
vec3 hitpos = cam + dir * hi; vec3 hitpos = cam + dir * hi;
// View-space depth of the hit (project onto the forward axis), matching the
// projection object.vert applies to the model vertices.
gl_FragDepth = clamp((dot(hitpos - cam, fwd) - NEAR) / (FAR - NEAR), 0.0, 1.0);
vec3 sun = normalize(pc.sun.xyz); vec3 sun = normalize(pc.sun.xyz);
float ambient = pc.sun.w; float ambient = pc.sun.w;
if (pc.params.w > 0.5 && hitpos.z <= pc.params.z + 0.01) { if (pc.params.w > 0.5 && hitpos.z <= pc.params.z + 0.01) {
// Water: animated normal from a procedural wave, sky reflection + fresnel. out_color = vec4(apply_underwater(water_shade(hitpos, dir, hi), hi, cam.z, pc.params.z), 1.0);
float time = pc.misc.x;
vec2 q = hitpos.xy * 0.015;
float nx = sin(q.x * 1.3 + time * 1.7) + 0.5 * sin(q.x * 3.1 - time * 2.3);
float ny = sin(q.y * 1.1 - time * 1.3) + 0.5 * sin(q.y * 2.7 + time * 1.9);
vec3 n = normalize(vec3(nx * 0.06, ny * 0.06, 1.0));
float fres = pow(1.0 - clamp(-dir.z, 0.0, 1.0), 3.0);
vec3 deep = vec3(0.03, 0.16, 0.28);
vec3 refl = sky_color(reflect(dir, n));
float lam = max(0.0, dot(n, sun));
vec3 water = mix(deep, refl, clamp(0.25 + 0.55 * fres, 0.0, 0.9));
water += vec3(1.0, 0.98, 0.9) * pow(lam, 64.0) * 0.6; // sun glint
float wfog = clamp(1.0 - exp(-hi * 0.00009), 0.0, 0.75);
water = mix(water, sky_color(vec3(dir.x, dir.y, 0.0)), wfog);
out_color = vec4(water, 1.0);
return; return;
} }
@@ -199,5 +336,5 @@ void main() {
// Distance haze toward the horizon so the map edge blends into the sky. // Distance haze toward the horizon so the map edge blends into the sky.
float fog = clamp(1.0 - exp(-hi * 0.00009), 0.0, 0.75); float fog = clamp(1.0 - exp(-hi * 0.00009), 0.0, 0.75);
lit = mix(lit, sky_color(vec3(dir.x, dir.y, 0.0)), fog); lit = mix(lit, sky_color(vec3(dir.x, dir.y, 0.0)), fog);
out_color = vec4(lit, 1.0); out_color = vec4(apply_underwater(lit, hi, cam.z, pc.params.z), 1.0);
} }
+15
View File
@@ -0,0 +1,15 @@
#version 300 es
// Samples the software-rendered scene image (the WebGL port of scene.frag).
precision highp float;
precision highp sampler2D;
uniform sampler2D u_scene;
in vec2 v_uv;
out vec4 frag_color;
void main() {
if (v_uv.x < 0.0 || v_uv.x > 1.0 || v_uv.y < 0.0 || v_uv.y > 1.0) discard;
// The engine stores 0xAARRGGBB (BGRA in memory), uploaded as GL_RGBA.
frag_color = texture(u_scene, v_uv).bgra;
}
+15
View File
@@ -0,0 +1,15 @@
#version 300 es
// Fullscreen triangle for the 2D image path (the WebGL port of scene.vert).
// WebGL clip space is +Y up, so the vertex position flips Y relative to the
// Vulkan shader; the sampled UVs and the image's top-left origin are unchanged.
precision highp float;
uniform vec4 u_rect; // xy = top-left (0..1), zw = size (0..1)
out vec2 v_uv;
void main() {
vec2 p = vec2(float((gl_VertexID << 1) & 2), float(gl_VertexID & 2));
v_uv = (p - u_rect.xy) / u_rect.zw;
gl_Position = vec4(p.x * 2.0 - 1.0, 1.0 - p.y * 2.0, 0.0, 1.0);
}
+318
View File
@@ -0,0 +1,318 @@
#version 300 es
// GPU heightfield raymarcher for the real RA3 terrain (the WebGL port of
// terrain.frag). The Vulkan push constants become a set of vec4 uniforms.
precision highp float;
precision highp int;
precision highp sampler2D;
precision highp sampler2DArray;
uniform sampler2D u_heightmap; // R16 heights
uniform sampler2D u_celldata; // per-cell blend record (RGBA16)
uniform sampler2DArray u_atlas; // tile material array (RGBA8)
// One contiguous array so the host can upload all five vec4s with a single
// glUniform4fv; the names keep the shader body identical to terrain.frag.
uniform vec4 u_data[5];
#define u_cam u_data[0] // x=target_x, y=target_y, z=yaw, w=height
#define u_params u_data[1] // x=pitch, y=fov, z=water_z, w=has_water
#define u_sun u_data[2] // xyz=sun dir, w=ambient
#define u_mapinfo u_data[3] // x=W, y=H, z=unused, w=z_scale
#define u_misc u_data[4] // x=time, y=unused, z=cells per texture repeat, w=aspect
in vec2 v_uv;
out vec4 frag_color;
const float CELL = 10.0; // must match ra3::terrain::cell_size
// SAGE water model constants (see docs/REVERSE_ENGINEERING.md).
const float WATER_SCALE = 1.0 / 320.0;
const float WATER_TRANSPARENT_DEPTH = 10.0;
const float WATER_MIN_OPACITY = 0.70;
const float WATER_RIVER_MULTIPLIER = 1.0;
float height_at(ivec2 c) {
c = clamp(c, ivec2(0), ivec2(u_mapinfo.xy) - 1);
return texelFetch(u_heightmap, c, 0).r * 65535.0 * u_mapinfo.w;
}
float world_height(float wx, float wy) {
float world_w = u_mapinfo.x * CELL;
float world_h = u_mapinfo.y * CELL;
if (wx < 0.0 || wy < 0.0 || wx >= world_w || wy >= world_h) return -1.0e9;
ivec2 c = ivec2(int(wx / CELL), int((world_h - wy) / CELL));
return height_at(c);
}
vec3 sky_color(vec3 dir) {
vec3 d = normalize(dir);
vec3 sun_dir = normalize(u_sun.xyz);
float t = clamp(d.z, 0.0, 1.0);
vec3 horizon = vec3(0.70, 0.78, 0.85);
vec3 zenith = vec3(0.28, 0.48, 0.80);
vec3 col = mix(horizon, zenith, pow(t, 0.6));
float sun = max(dot(d, sun_dir), 0.0);
col += vec3(1.0, 0.95, 0.82) * pow(sun, 300.0) * 1.6; // sun disk
col += vec3(1.0, 0.90, 0.72) * pow(sun, 8.0) * 0.18; // glow
return col;
}
// The retail SAGE blend ramp (see terrain.frag).
float blend_factor(uint direction, uint flags, vec2 f) {
bool flipped = (flags & 1u) != 0u;
bool two_sided = (flags & 2u) != 0u;
if (flipped) {
if (direction == 1u) {
f.x = 1.0 - f.x;
} else if (direction == 2u || direction == 4u || direction == 8u) {
f.y = 1.0 - f.y;
}
}
if (direction == 1u) return f.x;
if (direction == 2u) return f.y;
if (direction == 4u) {
float s = (1.0 - f.x) + (1.0 - f.y);
return two_sided ? 1.0 - clamp(s - 1.0, 0.0, 1.0) : clamp(1.0 - s, 0.0, 1.0);
}
if (direction == 8u) {
float s = f.x + (1.0 - f.y);
return two_sided ? 1.0 - clamp(s - 1.0, 0.0, 1.0) : clamp(1.0 - s, 0.0, 1.0);
}
return 0.0;
}
vec3 sample_layer(uint layer, float wx, float wy) {
float span = max(u_misc.z, 1.0);
int layer_count = textureSize(u_atlas, 0).z;
float l = float(min(layer, uint(layer_count - 1)));
// The atlas is 0xAARRGGBB (BGRA in memory), uploaded as GL_RGBA.
return texture(u_atlas, vec3(vec2(wx, wy) / span, l)).bgr;
}
// ---- SAGE water (Ocean.fx / RiverWater.fx / Water.frag port) ----------------
// Sample an atlas layer by an explicit layer index (the water flow/bump maps
// are appended as the last two layers of the tile atlas). The atlas is
// 0xAARRGGBB, so `.bgr` restores RGB (as `sample_layer`).
vec3 water_tex(int layer, vec2 uv) {
int lc = textureSize(u_atlas, 0).z;
float l = float(clamp(layer, 0, max(lc - 1, 0)));
return texture(u_atlas, vec3(uv, l)).bgr;
}
// Scrolling wave normal on the water plane: the retail bump map (atlas's last
// layer), offset by the flow map (second-last layer) and combined with a
// de-gridded procedural wave so the sun glint is irregular and always moving.
vec3 water_normal(vec2 world_xy, float time) {
vec2 q = world_xy * (WATER_SCALE * 6.0);
float a1 = q.x * 0.80 + q.y * 0.30 + time * 1.10;
float a2 = q.y * 0.95 - q.x * 0.45 - time * 1.30;
float a3 = (q.x + q.y) * 1.60 + time * 2.10;
float a4 = (q.x - q.y) * 2.30 - time * 1.70;
float dx = 0.224 * cos(a1) - 0.099 * cos(a2) + 0.256 * cos(a3) + 0.230 * cos(a4);
float dy = 0.084 * cos(a1) + 0.209 * cos(a2) + 0.256 * cos(a3) - 0.230 * cos(a4);
int lc = textureSize(u_atlas, 0).z;
vec3 flow = water_tex(lc - 2, q - vec2(time * 0.010, time * 0.014)) * 2.0 - 1.0;
vec3 bump = water_tex(lc - 1, q + flow.xy * 0.05 + vec2(time * 0.006, time * 0.008)) * 2.0 - 1.0;
float sx = -dx * 0.30 + bump.x * 0.45;
float sy = -dy * 0.30 + bump.y * 0.45;
return normalize(vec3(sx, sy, 1.0));
}
// Water.frag distortionPower * the flow texture: a small scrolling UV offset.
float water_distortion(vec2 world_xy, float time) {
vec2 q = world_xy * (WATER_SCALE * 6.0);
return 0.05 * (sin(q.x * 0.9 + time * 0.7) + sin(q.y * 1.1 - time * 0.5));
}
// Water.frag GetCloudColor: no cloud texture is bound, so a slow scroll stands in.
vec3 water_cloud(vec2 world_xy, float time) {
return vec3(0.85 + 0.15 * sin((world_xy.x + world_xy.y) * 0.0007 - time * 0.05));
}
// Retail UnderwaterDeferred.fx: absorbs red and fogs with distance.
vec3 apply_underwater(vec3 color, float dist, float cam_z, float water_z) {
if (cam_z >= water_z - 0.5) return color;
const vec3 absorb = vec3(0.35, 0.62, 0.75);
float fog = clamp(1.0 - exp(-dist * 0.00022), 0.0, 0.9);
return mix(color * absorb, vec3(0.02, 0.10, 0.16), fog);
}
// Water.frag: fresnel-mixed sky reflection and depth-graded seabed refraction,
// SAGE diffuse + specular lighting, cloud term, depth-based transparency fade.
vec3 water_shade(vec3 hitpos, vec3 dir, float dist) {
float time = u_misc.x;
float river = u_misc.y;
float seabed = world_height(hitpos.x, hitpos.y);
float depth = max(0.0, u_params.z - seabed);
vec3 n = water_normal(hitpos.xy, time + water_distortion(hitpos.xy, time));
vec3 sun_dir = normalize(u_sun.xyz);
float cos_theta = clamp(dot(-dir, n), 0.0, 1.0);
float fresnel = 0.02 + 0.98 * pow(1.0 - cos_theta, 5.0);
vec3 reflection = sky_color(reflect(dir, n));
vec3 shallow = vec3(0.10, 0.34, 0.38);
vec3 deep = vec3(0.02, 0.12, 0.22);
vec3 refraction = mix(shallow, deep, clamp(depth / 40.0, 0.0, 1.0));
float ndotl = max(dot(n, sun_dir), 0.0);
float ambient = u_sun.w;
vec3 diffuse = vec3(ambient + (1.0 - ambient) * ndotl);
vec3 half_v = normalize(sun_dir - dir);
float spec = pow(max(dot(n, half_v), 0.0), 90.0);
vec3 color = mix(refraction, reflection, clamp(fresnel, 0.0, 1.0)) * diffuse * water_cloud(hitpos.xy, time);
color += vec3(1.0, 0.97, 0.9) * spec * 0.45;
float alpha = clamp(depth / WATER_TRANSPARENT_DEPTH, 0.0, 1.0) * WATER_MIN_OPACITY;
if (river > 0.5) alpha *= WATER_RIVER_MULTIPLIER;
color = mix(refraction, color, clamp(alpha + 0.15, 0.0, 1.0));
float wfog = clamp(1.0 - exp(-dist * 0.00009), 0.0, 0.75);
return mix(color, sky_color(vec3(dir.x, dir.y, 0.0)), wfog);
}
void main() {
vec4 p = u_cam;
float pitch = clamp(u_params.x, 0.15, 1.45);
float fov = clamp(u_params.y, 0.3, 1.4);
float world_w = u_mapinfo.x * CELL;
float world_h = u_mapinfo.y * CELL;
float cp = cos(pitch);
vec3 fwd = vec3(cp * sin(p.z), cp * cos(p.z), -sin(pitch));
vec3 right = normalize(cross(fwd, vec3(0, 0, 1)));
vec3 up = cross(right, fwd);
float target_z = world_height(p.x, p.y);
if (target_z < -1.0e8) target_z = 0.0;
float dist = p.w / sin(pitch);
vec3 cam = vec3(p.x, p.y, target_z + p.w) - fwd * dist;
vec2 ndc = vec2(v_uv.x * 2.0 - 1.0, 1.0 - v_uv.y * 2.0);
float aspect = u_misc.w;
float th = tan(fov * 0.5);
vec3 dir = normalize(fwd + right * ndc.x * th * aspect + up * ndc.y * th);
if (dir.z >= -1e-4) {
frag_color = vec4(apply_underwater(sky_color(dir), 0.0, cam.z, u_params.z), 1.0);
return;
}
// Clip the ray to the map's XY rectangle: the boundary is an exact plane,
// so the silhouette there stays razor-sharp instead of stair-stepping
// across it. Outside the map is sky.
float t_enter = 0.0;
float t_exit = 1.0e30;
bool inside = true;
if (abs(dir.x) < 1e-6) {
inside = (cam.x >= 0.0 && cam.x <= world_w);
} else {
float a = (0.0 - cam.x) / dir.x;
float b = (world_w - cam.x) / dir.x;
t_enter = max(t_enter, min(a, b));
t_exit = min(t_exit, max(a, b));
}
if (inside) {
if (abs(dir.y) < 1e-6) {
inside = (cam.y >= 0.0 && cam.y <= world_h);
} else {
float a = (0.0 - cam.y) / dir.y;
float b = (world_h - cam.y) / dir.y;
t_enter = max(t_enter, min(a, b));
t_exit = min(t_exit, max(a, b));
}
}
if (!inside || t_exit <= 0.0) {
frag_color = vec4(apply_underwater(sky_color(dir), 0.0, cam.z, u_params.z), 1.0);
return;
}
// March the heightfield cell by cell: the step is never longer than the
// time to cross one cell (dominant horizontal axis), while a clearance term
// lets the ray skip the empty air above the surface. Resolving every cell is
// what keeps cliff and map-edge silhouettes from quantising into huge
// stair-steps that crawl as the camera pans.
float horiz = max(abs(dir.x), abs(dir.y));
float cell_step = min(CELL / max(horiz, 1e-4), CELL * 32.0);
float t = max(t_enter, CELL * 0.5);
float prev = t;
bool hit = false;
float hit_t = 0.0;
for (int i = 0; i < 1024 && t <= t_exit; ++i) {
vec3 w = cam + dir * t;
float h = world_height(w.x, w.y);
float surface = (u_params.w > 0.5) ? max(h, u_params.z) : h;
if (w.z <= surface) {
hit = true;
hit_t = t;
break;
}
float clearance = (w.z - surface) / max(-dir.z, 1e-4);
prev = t;
t += clamp(clearance, cell_step, cell_step * 8.0);
}
if (!hit) {
frag_color = vec4(apply_underwater(sky_color(dir), 0.0, cam.z, u_params.z), 1.0);
return;
}
// Refine the first crossing; with a sub-cell bracket this converges to the
// exact surface point.
float lo = prev;
float hi = hit_t;
for (int i = 0; i < 18; ++i) {
float mid = 0.5 * (lo + hi);
vec3 w = cam + dir * mid;
float h = world_height(w.x, w.y);
float surface = (u_params.w > 0.5) ? max(h, u_params.z) : h;
if (w.z <= surface) {
hi = mid;
} else {
lo = mid;
}
}
vec3 hitpos = cam + dir * hi;
vec3 sun = normalize(u_sun.xyz);
float ambient = u_sun.w;
if (u_params.w > 0.5 && hitpos.z <= u_params.z + 0.01) {
frag_color = vec4(apply_underwater(water_shade(hitpos, dir, hi), hi, cam.z, u_params.z), 1.0);
return;
}
float wx = hitpos.x / CELL;
float wy = (world_h - hitpos.y) / CELL;
int cx = clamp(int(wx), 0, int(u_mapinfo.x) - 1);
int cy = clamp(int(wy), 0, int(u_mapinfo.y) - 1);
float fx = wx - floor(wx);
float fy = wy - floor(wy);
uvec4 record = uvec4(texelFetch(u_celldata, ivec2(cx, cy), 0) * 65535.0 + 0.5);
uint packed = record.w;
uint dir1 = packed & 0xFu;
uint flags1 = (packed >> 4u) & 0x3u;
uint dir2 = (packed >> 8u) & 0xFu;
uint flags2 = (packed >> 12u) & 0x3u;
vec2 fracUV = vec2(fx, fy);
vec3 c0 = sample_layer(record.x, wx, wy);
vec3 c1 = sample_layer(record.y, wx, wy);
vec3 c2 = sample_layer(record.z, wx, wy);
float f1 = blend_factor(dir1, flags1, fracUV);
float f2 = blend_factor(dir2, flags2, fracUV);
vec3 albedo = mix(mix(c0, c1, f1), c2, f2);
float hl = world_height(hitpos.x - CELL, hitpos.y);
float hr = world_height(hitpos.x + CELL, hitpos.y);
float hd = world_height(hitpos.x, hitpos.y - CELL);
float hu = world_height(hitpos.x, hitpos.y + CELL);
vec3 n = normalize(vec3(hl - hr, hd - hu, 2.0 * CELL));
float lambert = max(0.0, dot(n, sun));
vec3 lit = albedo * (ambient + (1.0 - ambient) * lambert);
float fog = clamp(1.0 - exp(-hi * 0.00009), 0.0, 0.75);
lit = mix(lit, sky_color(vec3(dir.x, dir.y, 0.0)), fog);
frag_color = vec4(apply_underwater(lit, hi, cam.z, u_params.z), 1.0);
}
+11
View File
@@ -0,0 +1,11 @@
#version 300 es
// Fullscreen triangle for the terrain raymarcher (the WebGL port of terrain.vert).
precision highp float;
out vec2 v_uv;
void main() {
vec2 p = vec2(float((gl_VertexID << 1) & 2), float(gl_VertexID & 2));
v_uv = p;
gl_Position = vec4(p.x * 2.0 - 1.0, 1.0 - p.y * 2.0, 0.0, 1.0);
}
+38
View File
@@ -0,0 +1,38 @@
// WebGPU counterpart of the 2D image path (the WGSL port of
// webgl_scene_vert/frag.glsl and the Vulkan scene shaders).
//
// Bind group 0: uniform { rect: vec4<f32>; }, the scene texture and a sampler.
// WebGPU clip space is +Y up, matching GL, so the vertex flips Y the same way.
struct SceneUniforms {
rect: vec4<f32>, // xy = top-left (0..1), zw = size (0..1)
};
@group(0) @binding(0) var<uniform> u_scene: SceneUniforms;
@group(0) @binding(1) var u_scene_tex: texture_2d<f32>;
@group(0) @binding(2) var u_scene_samp: sampler;
struct SceneOut {
@builtin(position) position: vec4<f32>,
@location(0) uv: vec2<f32>,
};
@vertex
fn vs_main(@builtin(vertex_index) vertex_index: u32) -> SceneOut {
let p = vec2<f32>(f32((vertex_index << 1u) & 2u), f32(vertex_index & 2u));
var out: SceneOut;
out.uv = (p - u_scene.rect.xy) / u_scene.rect.zw;
out.position = vec4<f32>(p.x * 2.0 - 1.0, 1.0 - p.y * 2.0, 0.0, 1.0);
return out;
}
@fragment
fn fs_main(in: SceneOut) -> @location(0) vec4<f32> {
// Only the destination rect is drawn; outside it the previous contents stay.
if (in.uv.x < 0.0 || in.uv.x > 1.0 || in.uv.y < 0.0 || in.uv.y > 1.0) {
discard;
}
// The image is 0xAARRGGBB (BGRA in memory); swizzle it back.
let c = textureSampleLevel(u_scene_tex, u_scene_samp, in.uv, 0.0);
return vec4<f32>(c.b, c.g, c.r, c.a);
}
+334
View File
@@ -0,0 +1,334 @@
// WebGPU counterpart of the GPU heightfield raymarcher (the WGSL port of
// webgl_terrain_frag.glsl / terrain.frag). The Vulkan push constants become a
// uniform buffer of five vec4s, laid out exactly as `ra3::wasmgl::detail::terrain_uniforms`.
//
// Bind group 0: uniform data[5], the R16 heightmap (u32), the RGBA16 cell record
// (vec4<u32>), the RGBA8 tile atlas (array), and an atlas sampler.
struct TerrainUniforms {
data: array<vec4<f32>, 5>,
};
@group(0) @binding(0) var<uniform> u: TerrainUniforms;
@group(0) @binding(1) var u_heightmap: texture_2d<u32>; // R16Uint heights
@group(0) @binding(2) var u_celldata: texture_2d<u32>; // RGBA16Uint blend record (texel is vec4<u32>)
@group(0) @binding(3) var u_atlas: texture_2d_array<f32>; // RGBA8 tile materials
@group(0) @binding(4) var u_atlas_samp: sampler;
const CELL: f32 = 10.0; // must match ra3::terrain::cell_size
// SAGE water model constants (see docs/REVERSE_ENGINEERING.md).
const WATER_SCALE: f32 = 1.0 / 320.0;
const WATER_TRANSPARENT_DEPTH: f32 = 10.0;
const WATER_MIN_OPACITY: f32 = 0.70;
const WATER_RIVER_MULTIPLIER: f32 = 1.0;
fn cam_uniform() -> vec4<f32> { return u.data[0]; } // x=target_x, y=target_y, z=yaw, w=height
fn params_uniform() -> vec4<f32> { return u.data[1]; } // x=pitch, y=fov, z=water_z, w=has_water
fn sun_uniform() -> vec4<f32> { return u.data[2]; } // xyz=sun dir, w=ambient
fn mapinfo_uniform() -> vec4<f32> { return u.data[3]; }// x=W, y=H, z=unused, w=z_scale
fn misc_uniform() -> vec4<f32> { return u.data[4]; } // x=time, y=unused, z=cells per repeat, w=aspect
struct TerrainOut {
@builtin(position) position: vec4<f32>,
@location(0) uv: vec2<f32>,
};
@vertex
fn vs_main(@builtin(vertex_index) vertex_index: u32) -> TerrainOut {
let p = vec2<f32>(f32((vertex_index << 1u) & 2u), f32(vertex_index & 2u));
var out: TerrainOut;
out.uv = p;
out.position = vec4<f32>(p.x * 2.0 - 1.0, 1.0 - p.y * 2.0, 0.0, 1.0);
return out;
}
fn height_at(cell: vec2<i32>) -> f32 {
let limit = vec2<i32>(mapinfo_uniform().xy) - vec2<i32>(1);
let c = clamp(cell, vec2<i32>(0), limit);
return f32(textureLoad(u_heightmap, c, 0).r) * mapinfo_uniform().w;
}
fn world_height(wx: f32, wy: f32) -> f32 {
let world_w = mapinfo_uniform().x * CELL;
let world_h = mapinfo_uniform().y * CELL;
if (wx < 0.0 || wy < 0.0 || wx >= world_w || wy >= world_h) {
return -1.0e9;
}
let c = vec2<i32>(i32(wx / CELL), i32((world_h - wy) / CELL));
return height_at(c);
}
fn sky_color(dir: vec3<f32>) -> vec3<f32> {
let d = normalize(dir);
let sun_dir = normalize(sun_uniform().xyz);
let t = clamp(d.z, 0.0, 1.0);
let horizon = vec3<f32>(0.70, 0.78, 0.85);
let zenith = vec3<f32>(0.28, 0.48, 0.80);
var col = mix(horizon, zenith, pow(t, 0.6));
let sun = max(dot(d, sun_dir), 0.0);
col += vec3<f32>(1.0, 0.95, 0.82) * pow(sun, 300.0) * 1.6;
col += vec3<f32>(1.0, 0.90, 0.72) * pow(sun, 8.0) * 0.18;
return col;
}
// The retail SAGE blend ramp (see terrain.frag).
fn blend_factor(direction: u32, flags: u32, f_in: vec2<f32>) -> f32 {
var f = f_in;
let flipped = (flags & 1u) != 0u;
let two_sided = (flags & 2u) != 0u;
if (flipped) {
if (direction == 1u) {
f.x = 1.0 - f.x;
} else if (direction == 2u || direction == 4u || direction == 8u) {
f.y = 1.0 - f.y;
}
}
if (direction == 1u) { return f.x; }
if (direction == 2u) { return f.y; }
if (direction == 4u) {
let s = (1.0 - f.x) + (1.0 - f.y);
return select(clamp(1.0 - s, 0.0, 1.0), 1.0 - clamp(s - 1.0, 0.0, 1.0), two_sided);
}
if (direction == 8u) {
let s = f.x + (1.0 - f.y);
return select(clamp(1.0 - s, 0.0, 1.0), 1.0 - clamp(s - 1.0, 0.0, 1.0), two_sided);
}
return 0.0;
}
fn sample_layer(layer: u32, wx: f32, wy: f32) -> vec3<f32> {
let span = max(misc_uniform().z, 1.0);
let layer_count = textureNumLayers(u_atlas);
let l = f32(min(layer, layer_count - 1u));
// The atlas is 0xAARRGGBB (BGRA in memory), uploaded as RGBA8.
let c = textureSampleLevel(u_atlas, u_atlas_samp, vec2<f32>(wx, wy) / span, i32(l), 0.0);
return vec3<f32>(c.b, c.g, c.r);
}
// ---- SAGE water (Ocean.fx / RiverWater.fx / Water.frag port) ----------------
// Sample an atlas layer by an explicit layer index (the water flow/bump maps
// are appended as the last two layers of the tile atlas). The atlas is
// 0xAARRGGBB, so the b/g/r swap restores RGB (as `sample_layer`).
fn water_tex(layer: i32, uv: vec2<f32>) -> vec3<f32> {
let lc = i32(textureNumLayers(u_atlas));
let l = clamp(layer, 0, max(lc - 1, 0));
let c = textureSampleLevel(u_atlas, u_atlas_samp, uv, l, 0.0);
return vec3<f32>(c.b, c.g, c.r);
}
// Scrolling wave normal on the water plane: the retail bump map (atlas's last
// layer), offset by the flow map (second-last layer) and combined with a
// de-gridded procedural wave so the sun glint is irregular and always moving.
fn water_normal(world_xy: vec2<f32>, time: f32) -> vec3<f32> {
let q = world_xy * (WATER_SCALE * 6.0);
let a1 = q.x * 0.80 + q.y * 0.30 + time * 1.10;
let a2 = q.y * 0.95 - q.x * 0.45 - time * 1.30;
let a3 = (q.x + q.y) * 1.60 + time * 2.10;
let a4 = (q.x - q.y) * 2.30 - time * 1.70;
let dx = 0.224 * cos(a1) - 0.099 * cos(a2) + 0.256 * cos(a3) + 0.230 * cos(a4);
let dy = 0.084 * cos(a1) + 0.209 * cos(a2) + 0.256 * cos(a3) - 0.230 * cos(a4);
let lc = i32(textureNumLayers(u_atlas));
let flow = water_tex(lc - 2, q - vec2<f32>(time * 0.010, time * 0.014)) * 2.0 - 1.0;
let bump = water_tex(lc - 1, q + flow.xy * 0.05 + vec2<f32>(time * 0.006, time * 0.008)) * 2.0 - 1.0;
let sx = -dx * 0.30 + bump.x * 0.45;
let sy = -dy * 0.30 + bump.y * 0.45;
return normalize(vec3<f32>(sx, sy, 1.0));
}
// Water.frag distortionPower * the flow texture: a small scrolling UV offset.
fn water_distortion(world_xy: vec2<f32>, time: f32) -> f32 {
let q = world_xy * (WATER_SCALE * 6.0);
return 0.05 * (sin(q.x * 0.9 + time * 0.7) + sin(q.y * 1.1 - time * 0.5));
}
// Water.frag GetCloudColor: no cloud texture is bound, so a slow scroll stands in.
fn water_cloud(world_xy: vec2<f32>, time: f32) -> vec3<f32> {
return vec3<f32>(0.85 + 0.15 * sin((world_xy.x + world_xy.y) * 0.0007 - time * 0.05));
}
// Retail UnderwaterDeferred.fx: absorbs red and fogs with distance.
fn apply_underwater(color: vec3<f32>, distance: f32, cam_z: f32, water_z: f32) -> vec3<f32> {
if (cam_z >= water_z - 0.5) { return color; }
let absorb = vec3<f32>(0.35, 0.62, 0.75);
let fog = clamp(1.0 - exp(-distance * 0.00022), 0.0, 0.9);
return mix(color * absorb, vec3<f32>(0.02, 0.10, 0.16), fog);
}
// Water.frag: fresnel-mixed sky reflection and depth-graded seabed refraction,
// SAGE diffuse + specular lighting, cloud term, depth-based transparency fade.
fn water_shade(hitpos: vec3<f32>, dir: vec3<f32>, distance: f32) -> vec3<f32> {
let time = misc_uniform().x;
let river = misc_uniform().y;
let seabed = world_height(hitpos.x, hitpos.y);
let depth = max(0.0, params_uniform().z - seabed);
let n = water_normal(hitpos.xy, time + water_distortion(hitpos.xy, time));
let sun_dir = normalize(sun_uniform().xyz);
let cos_theta = clamp(dot(-dir, n), 0.0, 1.0);
let fresnel = 0.02 + 0.98 * pow(1.0 - cos_theta, 5.0);
let reflection = sky_color(reflect(dir, n));
let shallow = vec3<f32>(0.10, 0.34, 0.38);
let deep = vec3<f32>(0.02, 0.12, 0.22);
let refraction = mix(shallow, deep, clamp(depth / 40.0, 0.0, 1.0));
let ndotl = max(dot(n, sun_dir), 0.0);
let ambient = sun_uniform().w;
let diffuse = vec3<f32>(ambient + (1.0 - ambient) * ndotl);
let half_v = normalize(sun_dir - dir);
let spec = pow(max(dot(n, half_v), 0.0), 90.0);
var color = mix(refraction, reflection, clamp(fresnel, 0.0, 1.0)) * diffuse * water_cloud(hitpos.xy, time);
color += vec3<f32>(1.0, 0.97, 0.9) * spec * 0.45;
var alpha = clamp(depth / WATER_TRANSPARENT_DEPTH, 0.0, 1.0) * WATER_MIN_OPACITY;
if (river > 0.5) { alpha *= WATER_RIVER_MULTIPLIER; }
color = mix(refraction, color, clamp(alpha + 0.15, 0.0, 1.0));
let wfog = clamp(1.0 - exp(-distance * 0.00009), 0.0, 0.75);
return mix(color, sky_color(vec3<f32>(dir.x, dir.y, 0.0)), wfog);
}
@fragment
fn fs_main(in: TerrainOut) -> @location(0) vec4<f32> {
let p = cam_uniform();
let pitch = clamp(params_uniform().x, 0.15, 1.45);
let fov = clamp(params_uniform().y, 0.3, 1.4);
let world_w = mapinfo_uniform().x * CELL;
let world_h = mapinfo_uniform().y * CELL;
let cp = cos(pitch);
let fwd = vec3<f32>(cp * sin(p.z), cp * cos(p.z), -sin(pitch));
let right = normalize(cross(fwd, vec3<f32>(0.0, 0.0, 1.0)));
let up = cross(right, fwd);
var target_z = world_height(p.x, p.y);
if (target_z < -1.0e8) { target_z = 0.0; }
let dist = p.w / sin(pitch);
let cam = vec3<f32>(p.x, p.y, target_z + p.w) - fwd * dist;
let ndc = vec2<f32>(in.uv.x * 2.0 - 1.0, 1.0 - in.uv.y * 2.0);
let aspect = misc_uniform().w;
let th = tan(fov * 0.5);
let dir = normalize(fwd + right * ndc.x * th * aspect + up * ndc.y * th);
if (dir.z >= -1e-4) {
return vec4<f32>(apply_underwater(sky_color(dir), 0.0, cam.z, params_uniform().z), 1.0);
}
// Clip the ray to the map's XY rectangle: the boundary is an exact plane,
// so the silhouette there stays razor-sharp instead of stair-stepping
// across it. Outside the map is sky.
var t_enter = 0.0;
var t_exit = 1.0e30;
var inside = true;
if (abs(dir.x) < 1e-6) {
inside = (cam.x >= 0.0 && cam.x <= world_w);
} else {
let a = (0.0 - cam.x) / dir.x;
let b = (world_w - cam.x) / dir.x;
t_enter = max(t_enter, min(a, b));
t_exit = min(t_exit, max(a, b));
}
if (inside) {
if (abs(dir.y) < 1e-6) {
inside = (cam.y >= 0.0 && cam.y <= world_h);
} else {
let a = (0.0 - cam.y) / dir.y;
let b = (world_h - cam.y) / dir.y;
t_enter = max(t_enter, min(a, b));
t_exit = min(t_exit, max(a, b));
}
}
if (!inside || t_exit <= 0.0) {
return vec4<f32>(apply_underwater(sky_color(dir), 0.0, cam.z, params_uniform().z), 1.0);
}
// March the heightfield cell by cell: the step is never longer than the
// time to cross one cell (dominant horizontal axis), while a clearance term
// lets the ray skip the empty air above the surface. Resolving every cell is
// what keeps cliff and map-edge silhouettes from quantising into huge
// stair-steps that crawl as the camera pans.
let horiz = max(abs(dir.x), abs(dir.y));
let cell_step = min(CELL / max(horiz, 1e-4), CELL * 32.0);
var t = max(t_enter, CELL * 0.5);
var prev = t;
var hit = false;
var hit_t = 0.0;
for (var i = 0; i < 1024 && t <= t_exit; i = i + 1) {
let w = cam + dir * t;
let h = world_height(w.x, w.y);
let surface = select(h, max(h, params_uniform().z), params_uniform().w > 0.5);
if (w.z <= surface) {
hit = true;
hit_t = t;
break;
}
let clearance = (w.z - surface) / max(-dir.z, 1e-4);
prev = t;
t += clamp(clearance, cell_step, cell_step * 8.0);
}
if (!hit) {
return vec4<f32>(apply_underwater(sky_color(dir), 0.0, cam.z, params_uniform().z), 1.0);
}
// Refine the first crossing; with a sub-cell bracket this converges to the
// exact surface point.
var lo = prev;
var hi = hit_t;
for (var i = 0; i < 18; i = i + 1) {
let mid = 0.5 * (lo + hi);
let w = cam + dir * mid;
let h = world_height(w.x, w.y);
let surface = select(h, max(h, params_uniform().z), params_uniform().w > 0.5);
if (w.z <= surface) {
hi = mid;
} else {
lo = mid;
}
}
let hitpos = cam + dir * hi;
let sun = normalize(sun_uniform().xyz);
let ambient = sun_uniform().w;
if (params_uniform().w > 0.5 && hitpos.z <= params_uniform().z + 0.01) {
return vec4<f32>(apply_underwater(water_shade(hitpos, dir, hi), hi, cam.z, params_uniform().z), 1.0);
}
let wx = hitpos.x / CELL;
let wy = (world_h - hitpos.y) / CELL;
let cx = clamp(i32(wx), 0, i32(mapinfo_uniform().x) - 1);
let cy = clamp(i32(wy), 0, i32(mapinfo_uniform().y) - 1);
let fx = wx - floor(wx);
let fy = wy - floor(wy);
let record = textureLoad(u_celldata, vec2<i32>(cx, cy), 0);
let packed = record.w;
let dir1 = packed & 0xFu;
let flags1 = (packed >> 4u) & 0x3u;
let dir2 = (packed >> 8u) & 0xFu;
let flags2 = (packed >> 12u) & 0x3u;
let frac_uv = vec2<f32>(fx, fy);
let c0 = sample_layer(record.x, wx, wy);
let c1 = sample_layer(record.y, wx, wy);
let c2 = sample_layer(record.z, wx, wy);
let f1 = blend_factor(dir1, flags1, frac_uv);
let f2 = blend_factor(dir2, flags2, frac_uv);
let albedo = mix(mix(c0, c1, f1), c2, f2);
let hl = world_height(hitpos.x - CELL, hitpos.y);
let hr = world_height(hitpos.x + CELL, hitpos.y);
let hd = world_height(hitpos.x, hitpos.y - CELL);
let hu = world_height(hitpos.x, hitpos.y + CELL);
let n = normalize(vec3<f32>(hl - hr, hd - hu, 2.0 * CELL));
let lambert = max(0.0, dot(n, sun));
var lit = albedo * (ambient + (1.0 - ambient) * lambert);
let fog = clamp(1.0 - exp(-hi * 0.00009), 0.0, 0.75);
lit = mix(lit, sky_color(vec3<f32>(dir.x, dir.y, 0.0)), fog);
return vec4<f32>(apply_underwater(lit, hi, cam.z, params_uniform().z), 1.0);
}
+47 -7
View File
@@ -1,3 +1,10 @@
module;
#if defined(__EMSCRIPTEN__)
// Yields to the browser (requires -sASYNCIFY); see display::sleep_frame.
extern "C" void emscripten_sleep(unsigned int ms);
#endif
export module ra3.client; export module ra3.client;
import std; import std;
@@ -85,12 +92,19 @@ export namespace ra3::client {
/** Throttle an idle frame according to the configured limit. */ /** Throttle an idle frame according to the configured limit. */
auto sleep_frame() const -> void { auto sleep_frame() const -> void {
#if defined(__EMSCRIPTEN__)
// The frame loops are blocking; on the web the only way to let the
// browser paint and handle input is to yield (Asyncify sleep).
const int fps = fps_limit_ > 0 ? fps_limit_ : 60;
emscripten_sleep(static_cast<unsigned>(std::max(1, 1000 / fps)));
#else
if (fps_limit_ > 0) { if (fps_limit_ > 0) {
std::this_thread::sleep_for(std::chrono::milliseconds(std::max(1, 1000 / fps_limit_))); std::this_thread::sleep_for(std::chrono::milliseconds(std::max(1, 1000 / fps_limit_)));
} else if (fps_limit_ < 0) { } else if (fps_limit_ < 0) {
std::this_thread::yield(); std::this_thread::yield();
} }
// fps_limit_ == 0: vertical sync already blocks in present(). // fps_limit_ == 0: vertical sync already blocks in present().
#endif
} }
// ---- shared interactive loops (assume `init` succeeded) -------------- // ---- shared interactive loops (assume `init` succeeded) --------------
@@ -263,15 +277,17 @@ export namespace ra3::client {
} else if (event.type == ui_event_type::mouse_move) { } else if (event.type == ui_event_type::mouse_move) {
mouse_x = event.x; mouse_x = event.x;
mouse_y = event.y; mouse_y = event.y;
if (event.left) { if (event.middle) {
drag_x += event.dx; drag_x += event.dx;
drag_y += event.dy; drag_y += event.dy;
} }
} else if (event.type == ui_event_type::wheel) { } else if (event.type == ui_event_type::wheel) {
if (event.wheel != 0.0F) {
camera.height = std::clamp(camera.height * (event.wheel > 0.0F ? (1.0F / 1.15F) : 1.15F), camera.min_height, camera.max_height); camera.height = std::clamp(camera.height * (event.wheel > 0.0F ? (1.0F / 1.15F) : 1.15F), camera.min_height, camera.max_height);
dirty = true; dirty = true;
} }
} }
}
if (!running) break; if (!running) break;
const auto now = std::chrono::steady_clock::now(); const auto now = std::chrono::steady_clock::now();
@@ -331,8 +347,9 @@ export namespace ra3::client {
/** /**
* GPU terrain viewer. `present_terrain` draws the heightfield; the loop * GPU terrain viewer. `present_terrain` draws the heightfield; the loop
* here owns the camera controls. The top-left shows the FPS (current / * here owns the camera controls. The top-left shows the FPS (current /
* cap) and, when `minimap_overview` is not empty, a corner minimap with * cap) tagged with the active backend name (e.g. `[vulkan]`) and, when
* the camera location is drawn. * `minimap_overview` is not empty, a corner minimap with the camera
* location is drawn.
*/ */
[[nodiscard]] auto terrain_loop(const ra3::terrain::gpu_terrain &terrain, ra3::render::camera3d camera, const image &minimap_overview) -> bool { [[nodiscard]] auto terrain_loop(const ra3::terrain::gpu_terrain &terrain, ra3::render::camera3d camera, const image &minimap_overview) -> bool {
if (terrain.width == 0U || terrain.height == 0U || !this->supports_terrain()) return false; if (terrain.width == 0U || terrain.height == 0U || !this->supports_terrain()) return false;
@@ -356,6 +373,9 @@ export namespace ra3::client {
bool running = true; bool running = true;
bool presented = false; bool presented = false;
const auto default_camera = camera;
bool middle_dragged = false;
bool middle_down = false;
while (running) { while (running) {
ui_event event; ui_event event;
float drag_x = 0.0F; float drag_x = 0.0F;
@@ -374,14 +394,32 @@ export namespace ra3::client {
} else if (event.type == ui_event_type::mouse_move) { } else if (event.type == ui_event_type::mouse_move) {
mouse_x = event.x; mouse_x = event.x;
mouse_y = event.y; mouse_y = event.y;
if (event.left) { // Retail RA3: the middle button orbits the camera; the
// left button is selection only, not camera rotation.
if (event.middle) {
drag_x += event.dx; drag_x += event.dx;
drag_y += event.dy; drag_y += event.dy;
if (event.dx != 0.0F || event.dy != 0.0F) middle_dragged = true;
} }
} else if (event.type == ui_event_type::wheel) { } else if (event.type == ui_event_type::wheel) {
if (event.wheel != 0.0F) {
camera.height = std::clamp(camera.height * (event.wheel > 0.0F ? (1.0F / 1.15F) : 1.15F), camera.min_height, camera.max_height); camera.height = std::clamp(camera.height * (event.wheel > 0.0F ? (1.0F / 1.15F) : 1.15F), camera.min_height, camera.max_height);
camera_moved = true; camera_moved = true;
} }
} else if (event.type == ui_event_type::mouse_button && event.middle) {
if (event.released) {
if (middle_down && !middle_dragged) {
camera.yaw = default_camera.yaw;
camera.pitch = default_camera.pitch;
camera.height = default_camera.height;
camera_moved = true;
}
middle_down = false;
} else {
middle_down = true;
middle_dragged = false;
}
}
} }
if (!running) break; if (!running) break;
@@ -390,8 +428,10 @@ export namespace ra3::client {
last = now; last = now;
if (drag_x != 0.0F || drag_y != 0.0F) { if (drag_x != 0.0F || drag_y != 0.0F) {
camera.yaw -= drag_x * 0.005F; // SAGE LookAtTranslator: middle-drag rotates yaw (X) and
camera.pitch = std::clamp(camera.pitch + drag_y * 0.004F, 0.15F, 1.45F); // pitch (Y) at 0.01 rad per pixel; pitch is clamped to +-36 deg.
camera.yaw -= drag_x * 0.01F;
camera.pitch = std::clamp(camera.pitch + drag_y * 0.01F, 0.15F, 1.45F);
camera_moved = true; camera_moved = true;
} }
@@ -438,7 +478,7 @@ export namespace ra3::client {
fps = static_cast<uint32>(std::lround(static_cast<float>(fps_frames) / window_s)); fps = static_cast<uint32>(std::lround(static_cast<float>(fps_frames) / window_s));
fps_frames = 0; fps_frames = 0;
fps_window = now; fps_window = now;
overlay.label = ra3::render::compose_fps_label(fps, fps_limit_); overlay.label = ra3::render::compose_fps_label(fps, fps_limit_, this->name());
overlay.label_changed = true; overlay.label_changed = true;
} }
} else { } else {
+29 -12
View File
@@ -9,17 +9,19 @@ import ra3.client;
import ra3.ui; import ra3.ui;
import ra3.vulkan; import ra3.vulkan;
import ra3.dx; import ra3.dx;
import ra3.wasmgl;
import ra3.webgpu;
/** /**
* Backend selection for the presentation layer. * Backend selection for the presentation layer.
* *
* The app talks only to `ra3::client::display`; this module picks the concrete * The app talks only to `ra3::client::display`; this module picks the concrete
* backend so no caller has to know which one is in use. The available backends * backend so no caller has to know which one is in use. The available backends
* are Vulkan, Direct3D 11 and Direct3D 12 (both GPU terrain), and the SDL * are Vulkan, Direct3D 11 and Direct3D 12 (desktop), the WebGPU and SDL-free
* software blit fallback. The caller may name a preferred backend (the in-game * WebGL2 wasm worker backends (wasm) and the SDL software blit fallback. The
* menu exposes this); if it does not start, the others are tried in turn. The * caller may name a preferred backend (the in-game menu exposes this); if it
* GPU terrain path is Vulkan/D3D-only and reports failure so the caller can fall * does not start, the others are tried in turn. The GPU terrain path reports
* back to the software renderer. * failure so the caller can fall back to the software renderer.
*/ */
export namespace ra3::display { export namespace ra3::display {
using ra3::client::display_options; using ra3::client::display_options;
@@ -32,13 +34,24 @@ export namespace ra3::display {
using camera_frame = std::function<image(const camera3d &, ra3::core::uint32, ra3::core::uint32)>; using camera_frame = std::function<image(const camera3d &, ra3::core::uint32, ra3::core::uint32)>;
/** A concrete presentation backend, or `none`. */ /** A concrete presentation backend, or `none`. */
enum class backend { none, vulkan, d3d11, d3d12, sdl }; enum class backend { none, vulkan, d3d11, d3d12, wasmgl, webgpu, sdl };
/** The backend preferred on this host: WebGPU under Emscripten (falling back to the WebGL worker), else Vulkan. */
[[nodiscard]] inline constexpr auto default_backend() -> backend {
#if defined(__EMSCRIPTEN__)
return backend::webgpu;
#else
return backend::vulkan;
#endif
}
[[nodiscard]] inline auto backend_name(backend which) -> std::string_view { [[nodiscard]] inline auto backend_name(backend which) -> std::string_view {
switch (which) { switch (which) {
case backend::vulkan: return "vulkan"; case backend::vulkan: return "vulkan";
case backend::d3d11: return "d3d11"; case backend::d3d11: return "d3d11";
case backend::d3d12: return "d3d12"; case backend::d3d12: return "d3d12";
case backend::wasmgl: return "wasmgl";
case backend::webgpu: return "webgpu";
case backend::sdl: return "sdl"; case backend::sdl: return "sdl";
default: return "none"; default: return "none";
} }
@@ -49,19 +62,23 @@ export namespace ra3::display {
case backend::vulkan: return std::make_unique<ra3::vulkan::vulkan_display>(); case backend::vulkan: return std::make_unique<ra3::vulkan::vulkan_display>();
case backend::d3d11: return std::make_unique<ra3::dx::d3d11_display>(); case backend::d3d11: return std::make_unique<ra3::dx::d3d11_display>();
case backend::d3d12: return std::make_unique<ra3::dx::d3d12_display>(); case backend::d3d12: return std::make_unique<ra3::dx::d3d12_display>();
case backend::wasmgl: return std::make_unique<ra3::wasmgl::wasmgl_display>();
case backend::webgpu: return std::make_unique<ra3::webgpu::webgpu_display>();
case backend::sdl: return std::make_unique<ra3::ui::sdl_display>(); case backend::sdl: return std::make_unique<ra3::ui::sdl_display>();
default: return nullptr; default: return nullptr;
} }
} }
/** The order in which backends are attempted for a preferred one. */ /** The order in which backends are attempted for a preferred one. */
[[nodiscard]] inline auto backend_order(backend preferred) -> std::array<backend, 4> { [[nodiscard]] inline auto backend_order(backend preferred) -> std::array<backend, 6> {
switch (preferred) { switch (preferred) {
case backend::d3d11: return {backend::d3d11, backend::d3d12, backend::vulkan, backend::sdl}; case backend::d3d11: return {backend::d3d11, backend::d3d12, backend::vulkan, backend::webgpu, backend::wasmgl, backend::sdl};
case backend::d3d12: return {backend::d3d12, backend::d3d11, backend::vulkan, backend::sdl}; case backend::d3d12: return {backend::d3d12, backend::d3d11, backend::vulkan, backend::webgpu, backend::wasmgl, backend::sdl};
case backend::sdl: return {backend::sdl, backend::d3d11, backend::d3d12, backend::vulkan}; case backend::wasmgl: return {backend::wasmgl, backend::webgpu, backend::vulkan, backend::d3d11, backend::d3d12, backend::sdl};
case backend::webgpu: return {backend::webgpu, backend::wasmgl, backend::vulkan, backend::d3d11, backend::d3d12, backend::sdl};
case backend::sdl: return {backend::sdl, backend::d3d11, backend::d3d12, backend::vulkan, backend::webgpu, backend::wasmgl};
case backend::vulkan: case backend::vulkan:
default: return {backend::vulkan, backend::d3d11, backend::d3d12, backend::sdl}; default: return {backend::vulkan, backend::d3d11, backend::d3d12, backend::webgpu, backend::wasmgl, backend::sdl};
} }
} }
@@ -105,7 +122,7 @@ export namespace ra3::display {
} }
/** Interactive menu on the preferred (or first available) backend. */ /** Interactive menu on the preferred (or first available) backend. */
[[nodiscard]] inline auto run_menu(const display_options &options, const menu_frame &frame, backend preferred = backend::vulkan) -> bool { [[nodiscard]] inline auto run_menu(const display_options &options, const menu_frame &frame, backend preferred = default_backend()) -> bool {
return with_display(preferred, [&](ra3::client::display &d) { return d.run_menu(options, frame); }); return with_display(preferred, [&](ra3::client::display &d) { return d.run_menu(options, frame); });
} }
+57 -189
View File
@@ -3,13 +3,16 @@ export module ra3.fs;
import std; import std;
export import ra3.core; export import ra3.core;
import ra3.assets;
/** /**
* Reading of the retail game's on-disk assets. * Reading of the retail game's on-disk assets.
* *
* Red Alert 3 ships its data in `BIG4` archives under `<install>\Data`, with * Red Alert 3 ships its data in `BIG4` archives under `<install>\Data`, with
* individual payloads compressed by EA's RefPack codec. This module implements * individual payloads compressed by EA's RefPack codec. The container and codec
* the container and codec so OpenRA3 can read a user's own installation. * are implemented by the vendored `libra3assets` (`ra3.assets`, a sibling of
* `libenderlog`); this module is the thin adapter OpenRA3's loader talks to, so
* the format knowledge lives in one place.
* *
* No game data is ever written into the repository; callers point the loader at * No game data is ever written into the repository; callers point the loader at
* their local install (`--game-dir` / `RA3_GAME_DIR`, default `C:\Red Alert 3`). * their local install (`--game-dir` / `RA3_GAME_DIR`, default `C:\Red Alert 3`).
@@ -42,10 +45,20 @@ export namespace ra3::fs {
uint32 size = 0; uint32 size = 0;
}; };
/** True when `data` starts with a RefPack header (`0b??010000`, `0xFB`). */ namespace detail {
[[nodiscard]] inline auto is_refpack(std::span<const uint8> data) -> bool { /** View a `uint8` range as bytes, the currency of `libra3assets`. */
return data.size() >= 2 && (data[0] & refpack_mask) == 0x10U && data[1] == refpack_magic2; [[nodiscard]] inline auto as_bytes(std::span<const uint8> data) -> std::span<const std::byte> { return std::as_bytes(data); }
/** Copy a `libra3assets` byte buffer into OpenRA3's `uint8` vector. */
[[nodiscard]] inline auto to_u8(std::vector<std::byte> bytes) -> std::vector<uint8> {
std::vector<uint8> out(bytes.size());
if (!bytes.empty()) std::memcpy(out.data(), bytes.data(), bytes.size());
return out;
} }
}
/** True when `data` starts with a RefPack header (`0b??010000`, `0xFB`). */
[[nodiscard]] inline auto is_refpack(std::span<const uint8> data) -> bool { return ra3::assets::is_refpack(detail::as_bytes(data)); }
/** /**
* Decompress an EA RefPack stream. * Decompress an EA RefPack stream.
@@ -55,80 +68,17 @@ export namespace ra3::fs {
* @throws refpack_error if the stream is malformed or the length disagrees. * @throws refpack_error if the stream is malformed or the length disagrees.
*/ */
[[nodiscard]] inline auto refpack_decompress(std::span<const uint8> data) -> std::vector<uint8> { [[nodiscard]] inline auto refpack_decompress(std::span<const uint8> data) -> std::vector<uint8> {
if (!is_refpack(data)) throw refpack_error("not a RefPack stream"); try {
return detail::to_u8(ra3::assets::refpack_decompress(detail::as_bytes(data)));
usize pos = 0; } catch (const ra3::assets::refpack_error &error) {
const auto header = data[pos++]; throw refpack_error(error.what());
const bool large_files = (header & 0x80U) != 0;
const bool compressed_size_present = (header & 0x01U) != 0;
pos++; // 0xFB
const usize size_bytes = large_files ? 4U : 3U;
auto read_size = [&]() -> uint32 {
uint32 value = 0;
for (usize i = 0; i < size_bytes; ++i) {
if (pos >= data.size()) throw refpack_error("truncated RefPack size field");
value = (value << 8) | data[pos++];
} }
return value;
};
if (compressed_size_present) (void)read_size();
const auto out_len = read_size();
std::vector<uint8> out;
out.reserve(out_len);
auto copy_literals = [&](usize count) {
if (pos + count > data.size()) throw refpack_error("truncated RefPack literals");
out.insert(out.end(), data.begin() + static_cast<std::ptrdiff_t>(pos), data.begin() + static_cast<std::ptrdiff_t>(pos + count));
pos += count;
};
auto copy_reference = [&](usize length, usize distance) {
if (distance == 0 || distance > out.size()) throw refpack_error("RefPack back-reference out of range");
usize start = out.size() - distance;
for (usize i = 0; i < length; ++i) out.push_back(out[start + i]);
};
while (pos < data.size()) {
const auto cmd = data[pos++];
if ((cmd & 0x80U) == 0) { // 2-byte command
if (pos >= data.size()) throw refpack_error("truncated 2-byte command");
const auto b2 = data[pos++];
copy_literals(cmd & 0x03U);
copy_reference(((cmd & 0x1CU) >> 2) + 3, ((cmd & 0x60U) << 3) + b2 + 1);
} else if ((cmd & 0x40U) == 0) { // 3-byte command
if (pos + 1 >= data.size()) throw refpack_error("truncated 3-byte command");
const auto b2 = data[pos];
const auto b3 = data[pos + 1];
pos += 2;
copy_literals((b2 & 0xC0U) >> 6);
copy_reference((cmd & 0x3FU) + 4, ((b2 & 0x3FU) << 8) + b3 + 1);
} else if ((cmd & 0x20U) == 0) { // 4-byte command
if (pos + 2 >= data.size()) throw refpack_error("truncated 4-byte command");
const auto b2 = data[pos];
const auto b3 = data[pos + 1];
const auto b4 = data[pos + 2];
pos += 3;
copy_literals(cmd & 0x03U);
copy_reference(((cmd & 0x0CU) << 6) + b4 + 5, ((cmd & 0x10U) << 12) + (b2 << 8) + b3 + 1);
} else if (cmd < 0xFCU) { // long literal run
copy_literals(((cmd & 0x1FU) + 1) << 2);
} else { // stop
copy_literals(cmd & 0x03U);
break;
}
}
if (out.size() != out_len) throw refpack_error("RefPack length mismatch");
return out;
} }
/** Decompress `data` when it is RefPack, otherwise copy it unchanged. */ /** Decompress `data` when it is RefPack, otherwise copy it unchanged. */
[[nodiscard]] inline auto maybe_decompress(std::span<const uint8> data) -> std::vector<uint8> { [[nodiscard]] inline auto maybe_decompress(std::span<const uint8> data) -> std::vector<uint8> {
if (is_refpack(data)) return refpack_decompress(data); if (!is_refpack(data)) return {data.begin(), data.end()};
return {data.begin(), data.end()}; return refpack_decompress(data);
} }
/** /**
@@ -137,23 +87,11 @@ export namespace ra3::fs {
* @throws refpack_error if the stream is malformed. * @throws refpack_error if the stream is malformed.
*/ */
[[nodiscard]] inline auto refpack_output_size(std::span<const uint8> data) -> uint32 { [[nodiscard]] inline auto refpack_output_size(std::span<const uint8> data) -> uint32 {
if (!is_refpack(data)) throw refpack_error("not a RefPack stream"); try {
usize pos = 0; return ra3::assets::refpack_output_size(detail::as_bytes(data));
const auto header = data[pos++]; } catch (const ra3::assets::refpack_error &error) {
const bool large_files = (header & 0x80U) != 0; throw refpack_error(error.what());
const bool compressed_size_present = (header & 0x01U) != 0;
pos++; // 0xFB
const usize size_bytes = large_files ? 4U : 3U;
auto read_size = [&]() -> uint32 {
uint32 value = 0;
for (usize i = 0; i < size_bytes; ++i) {
if (pos >= data.size()) throw refpack_error("truncated RefPack size field");
value = (value << 8) | data[pos++];
} }
return value;
};
if (compressed_size_present) (void)read_size();
return read_size();
} }
[[nodiscard]] constexpr auto read_be32(const uint8 *p) -> uint32 { [[nodiscard]] constexpr auto read_be32(const uint8 *p) -> uint32 {
@@ -165,81 +103,32 @@ export namespace ra3::fs {
} }
/** /**
* A parsed `BIG4` archive. Only the index is held in memory; payloads are * A parsed `BIG4` archive.
* read from disk on demand so multi-hundred-megabyte archives stay cheap. *
* The index and payloads are held by a `ra3::assets::big_archive`; this
* adapter exposes the OpenRA3-facing surface (`big_entry`, `uint8` buffers)
* over it. Payloads are RefPack-decompressed on request.
*/ */
class big_archive { class big_archive {
public: public:
/** /**
* Parse the index of a `BIG4` archive. * Parse a `BIG4` archive.
* *
* @param path Archive path. * @param path Archive path.
* @throws archive_error if the file is missing, not `BIG4`, or truncated. * @throws archive_error if the file is missing, not `BIG4`, or truncated.
*/ */
[[nodiscard]] static auto open(const std::filesystem::path &path) -> big_archive { [[nodiscard]] static auto open(const std::filesystem::path &path) -> big_archive {
big_archive archive; try {
archive.path_ = path; return big_archive{ra3::assets::big_archive::open(path), path};
std::error_code ec; } catch (const ra3::assets::asset_error &error) {
archive.file_size_ = static_cast<usize>(std::filesystem::file_size(path, ec)); throw archive_error(error.what());
if (ec) throw archive_error("cannot stat archive: " + path.string());
std::ifstream in(path, std::ios::binary);
if (!in) throw archive_error("cannot open archive: " + path.string());
std::array<uint8, 16> header{};
in.read(reinterpret_cast<char *>(header.data()), static_cast<std::streamsize>(header.size()));
if (!in || std::memcmp(header.data(), big_magic.data(), big_magic.size()) != 0) throw archive_error("not a BIG4 archive: " + path.string());
const auto count = read_be32(header.data() + 8);
// Read the variable-length index, growing the window until parsed.
usize window = std::min(archive.file_size_, std::max<usize>(1U << 20U, static_cast<usize>(count) * 256U));
std::vector<uint8> index;
for (;;) {
index.resize(window);
in.clear();
in.seekg(0);
in.read(reinterpret_cast<char *>(index.data()), static_cast<std::streamsize>(window));
const auto got = static_cast<usize>(in.gcount());
index.resize(got);
archive.entries_.clear();
archive.entries_.reserve(count);
usize pos = 16;
bool complete = true;
for (uint32 i = 0; i < count; ++i) {
if (pos + 8 > index.size()) {
complete = false;
break;
} }
big_entry entry;
entry.offset = read_be32(index.data() + pos);
entry.size = read_be32(index.data() + pos + 4);
pos += 8;
const auto *begin = reinterpret_cast<const char *>(index.data() + pos);
const auto *end = reinterpret_cast<const char *>(std::memchr(begin, '\0', index.size() - pos));
if (end == nullptr) {
complete = false;
break;
}
entry.name.assign(begin, end);
pos += static_cast<usize>(end - begin) + 1U;
archive.entries_.push_back(std::move(entry));
}
if (complete) break;
if (window >= archive.file_size_) throw archive_error("truncated BIG4 index: " + path.string());
window = std::min(archive.file_size_, window * 2U);
}
archive.index_.reserve(archive.entries_.size());
for (usize i = 0; i < archive.entries_.size(); ++i) archive.index_.emplace(archive.entries_[i].name, i);
return archive;
} }
[[nodiscard]] auto path() const -> const std::filesystem::path & { return path_; } [[nodiscard]] auto path() const -> const std::filesystem::path & { return path_; }
[[nodiscard]] auto entries() const -> const std::vector<big_entry> & { return entries_; } [[nodiscard]] auto entries() const -> const std::vector<big_entry> & { return entries_; }
[[nodiscard]] auto size() const -> usize { return entries_.size(); } [[nodiscard]] auto size() const -> usize { return entries_.size(); }
[[nodiscard]] auto contains(std::string_view name) const -> bool { return index_.contains(std::string{name}); } [[nodiscard]] auto contains(std::string_view name) const -> bool { return archive_.contains(name); }
/** Entry names whose path contains `needle`, in index order. */ /** Entry names whose path contains `needle`, in index order. */
[[nodiscard]] auto find(std::string_view needle) const -> std::vector<const big_entry *> { [[nodiscard]] auto find(std::string_view needle) const -> std::vector<const big_entry *> {
@@ -258,42 +147,33 @@ export namespace ra3::fs {
* @throws archive_error if the entry is missing or unreadable. * @throws archive_error if the entry is missing or unreadable.
*/ */
[[nodiscard]] auto read(std::string_view name, bool decompress = true) const -> std::vector<uint8> { [[nodiscard]] auto read(std::string_view name, bool decompress = true) const -> std::vector<uint8> {
const auto it = index_.find(std::string{name}); try {
if (it == index_.end()) throw archive_error("no such entry: " + std::string{name}); return detail::to_u8(archive_.read(name, decompress));
const auto &entry = entries_[it->second]; } catch (const ra3::assets::refpack_error &error) {
throw refpack_error(error.what());
std::ifstream in(path_, std::ios::binary); } catch (const ra3::assets::asset_error &error) {
if (!in) throw archive_error("cannot open archive: " + path_.string()); throw archive_error(error.what());
in.seekg(static_cast<std::streamoff>(entry.offset)); }
std::vector<uint8> raw(entry.size);
in.read(reinterpret_cast<char *>(raw.data()), static_cast<std::streamsize>(raw.size()));
if (!in) throw archive_error("short read for entry: " + entry.name);
return decompress ? maybe_decompress(raw) : raw;
} }
/** Read the first `count` stored bytes of an entry (no decompression). */ /** Read the first `count` stored bytes of an entry (no decompression). */
[[nodiscard]] auto read_prefix(std::string_view name, usize count) const -> std::vector<uint8> { [[nodiscard]] auto read_prefix(std::string_view name, usize count) const -> std::vector<uint8> {
const auto it = index_.find(std::string{name}); try {
if (it == index_.end()) throw archive_error("no such entry: " + std::string{name}); return detail::to_u8(archive_.read_prefix(name, count));
const auto &entry = entries_[it->second]; } catch (const ra3::assets::asset_error &error) {
throw archive_error(error.what());
std::ifstream in(path_, std::ios::binary); }
if (!in) throw archive_error("cannot open archive: " + path_.string());
in.seekg(static_cast<std::streamoff>(entry.offset));
const auto want = std::min(count, static_cast<usize>(entry.size));
std::vector<uint8> raw(want);
in.read(reinterpret_cast<char *>(raw.data()), static_cast<std::streamsize>(raw.size()));
raw.resize(static_cast<usize>(in.gcount()));
return raw;
} }
private: private:
big_archive() = default; big_archive(ra3::assets::big_archive archive, std::filesystem::path path) : archive_(std::move(archive)), path_(std::move(path)) {
entries_.reserve(archive_.size());
for (const auto &entry: archive_.entries()) entries_.push_back({entry.name, entry.offset, entry.size});
}
ra3::assets::big_archive archive_;
std::filesystem::path path_; std::filesystem::path path_;
usize file_size_ = 0;
std::vector<big_entry> entries_; std::vector<big_entry> entries_;
std::unordered_map<std::string, usize> index_;
}; };
/** /**
@@ -306,18 +186,6 @@ export namespace ra3::fs {
* @return The install root, or `std::nullopt` when none is found. * @return The install root, or `std::nullopt` when none is found.
*/ */
[[nodiscard]] inline auto find_game_dir(const std::optional<std::filesystem::path> &explicit_dir = std::nullopt) -> std::optional<std::filesystem::path> { [[nodiscard]] inline auto find_game_dir(const std::optional<std::filesystem::path> &explicit_dir = std::nullopt) -> std::optional<std::filesystem::path> {
auto qualifies = [](const std::filesystem::path &candidate) { return ra3::assets::find_game_dir(explicit_dir);
std::error_code ec;
return !candidate.empty() && std::filesystem::is_directory(candidate / "Data", ec);
};
if (explicit_dir && qualifies(*explicit_dir)) return explicit_dir;
if (const auto *env = std::getenv("RA3_GAME_DIR"); env != nullptr && *env != '\0') {
const std::filesystem::path candidate{env};
if (qualifies(candidate)) return candidate;
}
const std::filesystem::path default_dir{"C:/Red Alert 3"};
if (qualifies(default_dir)) return default_dir;
return std::nullopt;
} }
} }
+85 -129
View File
@@ -4,6 +4,7 @@ import std;
export import ra3.core; export import ra3.core;
export import ra3.fs; export import ra3.fs;
import ra3.assets;
/** /**
* Red Alert 3 map discovery and loading. * Red Alert 3 map discovery and loading.
@@ -19,8 +20,9 @@ export import ra3.fs;
export namespace ra3::map { export namespace ra3::map {
using ra3::core::coord3d; using ra3::core::coord3d;
using ra3::core::real; using ra3::core::real;
using ra3::core::uint32;
using ra3::core::uint8; using ra3::core::uint8;
using ra3::core::uint16;
using ra3::core::uint32;
using ra3::core::usize; using ra3::core::usize;
/** A player start location extracted from a map's waypoints. */ /** A player start location extracted from a map's waypoints. */
@@ -49,6 +51,9 @@ export namespace ra3::map {
}; };
namespace detail { namespace detail {
/** View a `uint8` range as bytes, the currency of `libra3assets`. */
[[nodiscard]] inline auto as_bytes(std::span<const uint8> data) -> std::span<const std::byte> { return std::as_bytes(data); }
[[nodiscard]] inline auto split_path(std::string_view path) -> std::vector<std::string> { [[nodiscard]] inline auto split_path(std::string_view path) -> std::vector<std::string> {
std::vector<std::string> parts; std::vector<std::string> parts;
usize start = 0; usize start = 0;
@@ -81,54 +86,6 @@ export namespace ra3::map {
return file; return file;
} }
[[nodiscard]] inline auto find_bytes(std::span<const uint8> haystack, std::string_view needle) -> usize {
if (needle.empty()) return 0;
const auto *needle_begin = reinterpret_cast<const uint8 *>(needle.data());
const auto it = std::search(haystack.begin(), haystack.end(), needle_begin, needle_begin + needle.size());
return it == haystack.end() ? static_cast<usize>(-1) : static_cast<usize>(it - haystack.begin());
}
[[nodiscard]] inline auto read_le_f32(const uint8 *p) -> real {
const auto bits = fs::read_le32(p);
real value = 0.0F;
std::memcpy(&value, &bits, sizeof(value));
return value;
}
/** First plausible `(x, y, 0)` float triple at or after `from`. */
[[nodiscard]] inline auto scan_triple(std::span<const uint8> data, usize from, usize window) -> std::optional<start_position> {
const auto end = std::min(data.size(), from + window);
for (usize p = from; p + 12U <= end; ++p) {
const auto x = read_le_f32(data.data() + p);
const auto y = read_le_f32(data.data() + p + 4U);
const auto z = read_le_f32(data.data() + p + 8U);
if (z == 0.0F && std::isfinite(x) && std::isfinite(y) && x > 0.0F && x < 20000.0F && y > 0.0F && y < 20000.0F) {
return start_position{x, y, z};
}
}
return std::nullopt;
}
/** Extract `Player_1_Start` .. `Player_N_Start` waypoint coordinates. */
[[nodiscard]] inline auto extract_start_positions(std::span<const uint8> ckmp, int max_players = 8) -> std::vector<start_position> {
std::vector<start_position> result;
for (int n = 1; n <= max_players; ++n) {
const auto needle = std::string{"Player_"} + std::to_string(n) + "_Start";
usize cursor = 0;
while (cursor < ckmp.size()) {
const auto at = find_bytes(ckmp.subspan(cursor), needle);
if (at == static_cast<usize>(-1)) break;
const auto abs = cursor + at;
if (const auto triple = scan_triple(ckmp, abs + needle.size(), 128); triple) {
result.push_back(*triple);
break;
}
cursor = abs + 1;
}
}
return result;
}
/** Number of distinct integer positions (used to reject degenerate sets). */ /** Number of distinct integer positions (used to reject degenerate sets). */
[[nodiscard]] inline auto distinct_positions(const std::vector<start_position> &starts) -> usize { [[nodiscard]] inline auto distinct_positions(const std::vector<start_position> &starts) -> usize {
std::vector<std::pair<int, int>> seen; std::vector<std::pair<int, int>> seen;
@@ -195,13 +152,78 @@ export namespace ra3::map {
return fs::maybe_decompress(payload); return fs::maybe_decompress(payload);
} }
/** Recover `Player_N_Start` waypoints from raw, uncompressed `CkMp` bytes. */ /**
* Recover `Player_N_Start` waypoints from raw, uncompressed `CkMp` bytes.
*
* `libra3assets` decodes the `ObjectsList` `*Waypoints/Waypoint` objects and
* returns the position of each `waypointName == "Player_<n>_Start"`. This
* replaces the previous whole-buffer heuristic, which scanned for the string
* and then took the first plausible float triple *after* it — off by one
* (the position precedes the name in each object) and prone to matching
* arbitrary bytes.
*/
[[nodiscard]] inline auto starts_from_ckmp(std::span<const uint8> ckmp) -> std::vector<start_position> { [[nodiscard]] inline auto starts_from_ckmp(std::span<const uint8> ckmp) -> std::vector<start_position> {
auto starts = detail::extract_start_positions(ckmp); std::vector<start_position> starts;
try {
const auto document = ra3::assets::map_document::parse(detail::as_bytes(ckmp));
for (const auto &start: document.player_starts()) starts.push_back({start.position.x, start.position.y, start.position.z});
} catch (const std::exception &) {
return {};
}
if (detail::distinct_positions(starts) < 2U) starts.clear(); if (detail::distinct_positions(starts) < 2U) starts.clear();
return starts; return starts;
} }
/**
* One object the map places on the ground: a building, a prop, a lamp, a
* tree — anything in the `ObjectsList` chunk.
*
* `type` is the SAGE `ThingTemplate` name (e.g. `BB_GRASS02`,
* `AlliedBarracks`); the per-map compiled art stream resolves it to the
* `W3DMesh` assets that draw it. `angle` is the Z rotation in radians.
*/
struct map_object {
std::string type;
real x = 0.0F;
real y = 0.0F;
real z = 0.0F;
real angle = 0.0F;
real scale = 1.0F;
uint32 road_type = 0U; ///< SAGE `RoadType` flags; 0 for a non-road object.
};
/**
* Every object the map places (the `ObjectsList` chunk).
*
* Decoded by `libra3assets` (`ra3.assets`): the map is parsed as a
* `map_document`, whose `ObjectsList` accessor walks the nested `Object`
* assets (a `Coord3D`, a Z `angle`, a `RoadType`, a `u16`-prefixed
* type-name and a property list keyed into the shared name table).
*
* @return The objects in chunk order; empty when the map is malformed or has
* no object list.
*/
[[nodiscard]] inline auto parse_objects(std::span<const uint8> ckmp) -> std::vector<map_object> {
std::vector<map_object> objects;
try {
const auto document = ra3::assets::map_document::parse(detail::as_bytes(ckmp));
for (const auto &object: document.objects()) {
map_object out;
out.type = object.type_name;
out.x = object.position.x;
out.y = object.position.y;
out.z = object.position.z;
out.angle = object.angle;
out.scale = 1.0F;
out.road_type = static_cast<uint32>(object.road);
objects.push_back(std::move(out));
}
} catch (const std::exception &) {
return {};
}
return objects;
}
/** Map id -> localized display name, keyed by lowercased id. */ /** Map id -> localized display name, keyed by lowercased id. */
struct map_name_table { struct map_name_table {
std::unordered_map<std::string, std::string> names; std::unordered_map<std::string, std::string> names;
@@ -217,96 +239,30 @@ export namespace ra3::map {
} }
}; };
namespace detail {
/**
* Decode one SAGE `.csf` string value.
*
* RA3's `gamestrings.csf` stores each UTF-16 code unit with the low byte
* XORed by `0xFF` (the high byte is the padding `0xFF`); undoing that
* yields the plain ASCII/UTF-8 text.
*/
[[nodiscard]] inline auto decode_csf_string(std::span<const uint8> raw) -> std::string {
std::string value;
for (usize i = 0; i + 1U < raw.size(); i += 2U) {
const auto ch = static_cast<char>(static_cast<uint8>(raw[i] ^ 0xFFU));
if (ch == '\0') break;
value.push_back(ch);
}
return value;
}
}
/** /**
* Parse `MAP:<id>` display names out of a SAGE `.csf` string table. * Parse `MAP:<id>` display names out of a SAGE `.csf` string table.
* *
* The map list UI reads its labels from the install's `data\gamestrings.csf` * The map list UI reads its labels from the install's `data\gamestrings.csf`
* under the key `MAP:<UPPERCASE_ID>` (e.g. `MAP:MAP_MP_2_FEASEL4` is * under the key `MAP:<UPPERCASE_ID>` (e.g. `MAP:MAP_MP_2_FEASEL4` is
* "Battlebase Beta"). Values are byte-XORed with `0xFF`. * "Battlebase Beta"). Decoding is `libra3assets`' `csf_table` (each UTF-16
* code unit is bit-inverted; the table lookup is case-insensitive).
*/ */
[[nodiscard]] inline auto parse_map_names(std::span<const uint8> csf) -> map_name_table { [[nodiscard]] inline auto parse_map_names(std::span<const uint8> csf) -> map_name_table {
map_name_table table; map_name_table table;
if (csf.size() < 24U || std::memcmp(csf.data(), " FSC", 4) != 0) return table; try {
const auto strings = ra3::assets::csf_table::parse(detail::as_bytes(csf));
const auto read = [&](usize p) -> uint32 { for (const auto &entry: strings.entries()) {
return static_cast<uint32>(csf[p]) | (static_cast<uint32>(csf[p + 1U]) << 8U) | (static_cast<uint32>(csf[p + 2U]) << 16U) |
(static_cast<uint32>(csf[p + 3U]) << 24U);
};
usize pos = 24; // header: magic + version + label/string counts + 8 reserved bytes
while (pos + 4U <= csf.size()) {
if (std::memcmp(csf.data() + pos, " LBL", 4) != 0) {
++pos;
continue;
}
pos += 4;
if (pos + 4U > csf.size()) break;
const auto count = read(pos);
pos += 4;
std::vector<std::string> labels;
labels.reserve(count);
bool ok = true;
for (uint32 i = 0; i < count; ++i) {
if (pos + 4U > csf.size()) {
ok = false;
break;
}
const auto len = read(pos);
pos += 4;
if (pos + len > csf.size()) {
ok = false;
break;
}
labels.emplace_back(reinterpret_cast<const char *>(csf.data() + pos), len);
pos += len;
}
if (!ok) break;
for (const auto &label: labels) {
if (pos + 4U > csf.size() || std::memcmp(csf.data() + pos, " RTS", 4) != 0) {
ok = false;
break;
}
pos += 4;
if (pos + 4U > csf.size()) {
ok = false;
break;
}
const auto chars = read(pos);
pos += 4;
const auto bytes = static_cast<usize>(chars) * 2U;
if (pos + bytes > csf.size()) {
ok = false;
break;
}
auto value = detail::decode_csf_string(csf.subspan(pos, bytes));
pos += bytes;
constexpr std::string_view prefix = "MAP:"; constexpr std::string_view prefix = "MAP:";
if (label.size() > prefix.size() && label.compare(0, prefix.size(), prefix) == 0) { if (entry.label.size() <= prefix.size() || entry.label.compare(0, prefix.size(), prefix) != 0) continue;
auto id = label.substr(prefix.size()); auto id = entry.label.substr(prefix.size());
std::transform(id.begin(), id.end(), id.begin(), [](unsigned char ch) { return static_cast<char>(std::tolower(ch)); }); std::transform(id.begin(), id.end(), id.begin(), [](unsigned char ch) { return static_cast<char>(std::tolower(ch)); });
auto value = ra3::assets::utf16_to_utf8(entry.value());
// CSF strings are NUL-terminated; the terminator is not part of the text.
if (const auto nul = value.find('\0'); nul != std::string::npos) value.resize(nul);
if (!value.empty()) table.names.try_emplace(std::move(id), std::move(value)); if (!value.empty()) table.names.try_emplace(std::move(id), std::move(value));
} }
} } catch (const std::exception &) {
if (!ok) break; return {};
} }
return table; return table;
} }
File diff suppressed because it is too large Load Diff
+3
View File
@@ -16,7 +16,10 @@ export import ra3.map;
export import ra3.skirmish; export import ra3.skirmish;
export import ra3.render; export import ra3.render;
export import ra3.terrain; export import ra3.terrain;
export import ra3.models;
export import ra3.ui; export import ra3.ui;
export import ra3.vulkan; export import ra3.vulkan;
export import ra3.dx; export import ra3.dx;
export import ra3.wasmgl;
export import ra3.webgpu;
export import ra3.display; export import ra3.display;
+19 -7
View File
@@ -277,6 +277,9 @@ export namespace ra3::render {
float dy = 0.0F; float dy = 0.0F;
float wheel = 0.0F; float wheel = 0.0F;
bool left = false; bool left = false;
bool middle = false;
bool right = false;
bool released = false; ///< mouse_button: true for a button-up event.
}; };
/** A warm red/gold loading screen with a progress bar (0..1). */ /** A warm red/gold loading screen with a progress bar (0..1). */
@@ -311,10 +314,11 @@ export namespace ra3::render {
} }
/** /**
* A small translucent label for the top-left corner, e.g. `FPS: 155/160`. * A small translucent label for the top-left corner, e.g.
* `cap == 0` means vertical sync, `cap < 0` means uncapped. * `FPS: 155/160 [vulkan]`. `cap == 0` means vertical sync, `cap < 0` means
* uncapped. `backend` is the active renderer backend name (empty omits it).
*/ */
[[nodiscard]] inline auto compose_fps_label(uint32 fps, int cap) -> image { [[nodiscard]] inline auto compose_fps_label(uint32 fps, int cap, std::string_view backend = {}) -> image {
char text[64]; char text[64];
if (cap == 0) { if (cap == 0) {
std::snprintf(text, sizeof(text), "FPS: %u/vsync", fps); std::snprintf(text, sizeof(text), "FPS: %u/vsync", fps);
@@ -323,10 +327,16 @@ export namespace ra3::render {
} else { } else {
std::snprintf(text, sizeof(text), "FPS: %u/%d", fps, cap); std::snprintf(text, sizeof(text), "FPS: %u/%d", fps, cap);
} }
const auto w = text_width(text, 1U) + 8U; std::string label{text};
if (!backend.empty()) {
label += " [";
label += backend;
label += ']';
}
const auto w = text_width(label, 1U) + 8U;
const auto h = detail::glyph_height + 6U; const auto h = detail::glyph_height + 6U;
image img(w, h, argb(30, 6, 6, 200)); // warm translucent backing image img(w, h, argb(30, 6, 6, 200)); // warm translucent backing
draw_text(img, 4, 3, text, argb(240, 200, 90), 1U); draw_text(img, 4, 3, label, argb(240, 200, 90), 1U);
return img; return img;
} }
@@ -675,7 +685,9 @@ export namespace ra3::render {
float pitch = 1.02F; ///< Radians above the horizon (~58 degrees down). float pitch = 1.02F; ///< Radians above the horizon (~58 degrees down).
float height = 420.0F; ///< Camera height above the target's ground. float height = 420.0F; ///< Camera height above the target's ground.
float fov = 0.85F; ///< Vertical field of view, radians. float fov = 0.85F; ///< Vertical field of view, radians.
float min_height = 120.0F; // Zoom range mirrors the retail TacticalView (zoom 0.2..1.3 around the
float max_height = 1600.0F; // default height): closer/farther than that is clamped.
float min_height = 320.0F;
float max_height = 2100.0F;
}; };
} }
+364 -162
View File
@@ -5,6 +5,8 @@ import std;
export import ra3.core; export import ra3.core;
export import ra3.fs; export import ra3.fs;
export import ra3.render; export import ra3.render;
export import ra3.models;
import ra3.assets;
/** /**
* The map's real terrain, read from the compiled `CkMp` chunk tree. * The map's real terrain, read from the compiled `CkMp` chunk tree.
@@ -90,82 +92,24 @@ export namespace ra3::terrain {
}; };
namespace detail { namespace detail {
/** View a `uint8` range as bytes, the currency of `libra3assets`. */
[[nodiscard]] inline auto as_bytes(std::span<const uint8> data) -> std::span<const std::byte> { return std::as_bytes(data); }
/** View a `libra3assets` byte range as OpenRA3's `uint8`. */
[[nodiscard]] inline auto as_u8(std::span<const std::byte> data) -> std::span<const uint8> {
return {reinterpret_cast<const uint8 *>(data.data()), data.size()};
}
[[nodiscard]] inline auto read_u16(const uint8 *p) -> uint16 { return static_cast<uint16>(p[0]) | (static_cast<uint16>(p[1]) << 8U); } [[nodiscard]] inline auto read_u16(const uint8 *p) -> uint16 { return static_cast<uint16>(p[0]) | (static_cast<uint16>(p[1]) << 8U); }
[[nodiscard]] inline auto read_u32(const uint8 *p) -> uint32 { [[nodiscard]] inline auto read_u32(const uint8 *p) -> uint32 {
return static_cast<uint32>(p[0]) | (static_cast<uint32>(p[1]) << 8U) | (static_cast<uint32>(p[2]) << 16U) | return static_cast<uint32>(p[0]) | (static_cast<uint32>(p[1]) << 8U) | (static_cast<uint32>(p[2]) << 16U) |
(static_cast<uint32>(p[3]) << 24U); (static_cast<uint32>(p[3]) << 24U);
} }
struct chunk { // The `CkMp` container and `HeightMapData` chunk are modelled by
std::string name; // `libra3assets` (`ra3.assets`); `parse_map` reads them through a
uint16 version = 0; // `map_document`. Only `BlendTileData` (which the library does not type
usize offset = 0; // yet) is parsed here, over the chunk payload the document exposes.
usize size = 0;
};
/**
* Parse the `CkMp` chunk tree into a flat chunk list.
*
* Layout: `"CkMp"`, `u32 assetCount`, then `assetCount` entries of
* `{ u8 nameLen, name, u32 index }` (index descending from the count),
* then `{ u32 index, u16 version, u32 dataSize, data[dataSize] }` per
* chunk until the end.
*/
[[nodiscard]] inline auto parse_chunks(std::span<const uint8> data) -> std::vector<chunk> {
if (data.size() < 8U || std::memcmp(data.data(), "CkMp", 4) != 0) throw terrain_error("not a CkMp map");
usize pos = 4;
const auto count = read_u32(data.data() + pos);
pos += 4;
std::vector<std::string> names(count + 1U);
for (uint32 i = count; i >= 1U; --i) {
if (pos >= data.size()) throw terrain_error("truncated asset-name table");
const auto len = data[pos++];
if (pos + len + 4U > data.size()) throw terrain_error("truncated asset-name");
names[i] = std::string{reinterpret_cast<const char *>(data.data() + pos), len};
pos += len;
pos += 4; // asset index (== i)
}
std::vector<chunk> chunks;
while (pos + 10U <= data.size()) {
const auto index = read_u32(data.data() + pos);
const auto version = read_u16(data.data() + pos + 4U);
const auto size = read_u32(data.data() + pos + 6U);
pos += 10;
if (index >= names.size() || pos + size > data.size()) throw terrain_error("bad chunk header");
chunks.push_back({names[index], version, pos, size});
pos += size;
}
return chunks;
}
[[nodiscard]] inline auto find(const std::vector<chunk> &chunks, std::string_view name) -> const chunk * {
for (const auto &c: chunks) {
if (c.name == name) return &c;
}
return nullptr;
}
inline auto parse_heightmap(std::span<const uint8> data, const chunk &c, map_data &out) -> void {
usize p = c.offset;
out.width = read_u32(data.data() + p);
out.height = read_u32(data.data() + p + 4U);
out.border_width = read_u32(data.data() + p + 8U);
const auto border_count = read_u32(data.data() + p + 12U);
p += 16;
p += static_cast<usize>(border_count) * (c.version >= 6U ? 16U : 8U);
p += 4U; // area
if (out.width == 0U || out.height == 0U) throw terrain_error("empty heightmap");
const auto area = static_cast<usize>(out.width) * out.height;
out.elevations.resize(area);
for (usize i = 0; i < area; ++i) {
if (p >= data.size()) throw terrain_error("truncated heightmap");
out.elevations[i] = c.version >= 5U ? read_u16(data.data() + p) : data[p];
p += c.version >= 5U ? 2U : 1U;
}
}
/** Where the texture table ends and how many `BlendDescription`s follow. */ /** Where the texture table ends and how many `BlendDescription`s follow. */
struct texture_table_info { struct texture_table_info {
@@ -245,20 +189,20 @@ export namespace ra3::terrain {
} }
} }
inline auto parse_blend(std::span<const uint8> data, const chunk &c, map_data &out) -> void { inline auto parse_blend(std::span<const uint8> payload, uint16 version, map_data &out) -> void {
const auto area = static_cast<usize>(out.width) * out.height; const auto area = static_cast<usize>(out.width) * out.height;
usize p = c.offset; usize p = 0;
const auto num_tiles = read_u32(data.data() + p); const auto num_tiles = read_u32(payload.data() + p);
p += 4; p += 4;
if (num_tiles != area) throw terrain_error("BlendTileData tile count mismatch"); if (num_tiles != area) throw terrain_error("BlendTileData tile count mismatch");
out.tiles.resize(area); out.tiles.resize(area);
for (usize i = 0; i < area; ++i) out.tiles[i] = read_u16(data.data() + p + i * 2U); for (usize i = 0; i < area; ++i) out.tiles[i] = read_u16(payload.data() + p + i * 2U);
p += area * 2U; p += area * 2U;
const auto bits = (c.version >= 14U && c.version < 24U) ? 32U : 16U; const auto bits = (version >= 14U && version < 24U) ? 32U : 16U;
const auto word = bits / 8U; const auto word = bits / 8U;
const auto read_index = [&](usize off) -> uint16 { const auto read_index = [&](usize off) -> uint16 {
return word == 4U ? static_cast<uint16>(read_u32(data.data() + off)) : read_u16(data.data() + off); return word == 4U ? static_cast<uint16>(read_u32(payload.data() + off)) : read_u16(payload.data() + off);
}; };
out.blends.resize(area); out.blends.resize(area);
for (usize i = 0; i < area; ++i) out.blends[i] = read_index(p + i * word); for (usize i = 0; i < area; ++i) out.blends[i] = read_index(p + i * word);
@@ -268,38 +212,55 @@ export namespace ra3::terrain {
p += area * word; p += area * word;
p += area * word; // CliffTextures (not rendered yet) p += area * word; // CliffTextures (not rendered yet)
const auto chunk_end = c.offset + c.size; const auto chunk_end = payload.size();
const auto table = parse_textures(data, p, chunk_end, out); const auto table = parse_textures(payload, p, chunk_end, out);
parse_blend_descriptions(data, table, chunk_end, out); parse_blend_descriptions(payload, table, chunk_end, out);
} }
} }
/** /**
* Parse the terrain chunks out of a `CkMp` (uncompressed) map payload. * Parse the terrain chunks out of a `CkMp` (uncompressed) map payload.
* *
* The container and `HeightMapData` come from `libra3assets`
* (`map_document`); `BlendTileData` (which the library does not model yet)
* is decoded here from the chunk payload the document exposes.
*
* @throws terrain_error if the chunk tree or terrain chunks are malformed. * @throws terrain_error if the chunk tree or terrain chunks are malformed.
*/ */
[[nodiscard]] inline auto parse_map(std::span<const uint8> ckmp) -> map_data { [[nodiscard]] inline auto parse_map(std::span<const uint8> ckmp) -> map_data {
const auto chunks = detail::parse_chunks(ckmp); try {
const auto *heightmap = detail::find(chunks, "HeightMapData"); auto document = ra3::assets::map_document::parse(detail::as_bytes(ckmp));
const auto *blend = detail::find(chunks, "BlendTileData");
if (heightmap == nullptr || blend == nullptr) throw terrain_error("map has no terrain chunks");
map_data out; map_data out;
detail::parse_heightmap(ckmp, *heightmap, out); const auto height = document.height_map();
detail::parse_blend(ckmp, *blend, out); if (!height) throw terrain_error("map has no HeightMapData chunk");
if (const auto *water = detail::find(chunks, "GlobalWaterSettings"); water != nullptr && water->size >= 8U) { out.width = height->width;
out.has_water = detail::read_u32(ckmp.data() + water->offset) != 0U; out.height = height->height;
const auto bits = detail::read_u32(ckmp.data() + water->offset + 4U); out.border_width = height->border_width;
out.elevations.assign(height->elevations.begin(), height->elevations.end());
const auto *blend = document.find_chunk("BlendTileData");
if (blend == nullptr) throw terrain_error("map has no BlendTileData chunk");
detail::parse_blend(detail::as_u8(blend->payload), blend->version, out);
if (const auto *water = document.find_chunk("GlobalWaterSettings"); water != nullptr && water->payload.size() >= 8U) {
const auto *bytes = reinterpret_cast<const uint8 *>(water->payload.data());
out.has_water = detail::read_u32(bytes) != 0U;
const auto bits = detail::read_u32(bytes + 4U);
std::memcpy(&out.water_plane_z, &bits, sizeof(out.water_plane_z)); std::memcpy(&out.water_plane_z, &bits, sizeof(out.water_plane_z));
} }
out.valid = true; out.valid = true;
return out; return out;
} catch (const ra3::assets::asset_error &error) {
throw terrain_error(error.what());
}
} }
/** The decoded terrain textures, parallel to `map_data::textures`. */ /** The decoded terrain textures, parallel to `map_data::textures`. */
struct texture_set { struct texture_set {
std::vector<image> images; std::vector<image> images;
image water_flow; ///< `ra3_deepocean.tga`: SAGE water flow/distortion (RG), optional.
image water_normal; ///< `ra3_deepocean_nrm.tga`: SAGE water bump normal, optional.
[[nodiscard]] auto resolved() const -> usize { [[nodiscard]] auto resolved() const -> usize {
usize n = 0; usize n = 0;
@@ -335,6 +296,10 @@ export namespace ra3::terrain {
}; };
std::vector<fs::big_archive> archives; std::vector<fs::big_archive> archives;
std::unordered_map<std::string, source> files; std::unordered_map<std::string, source> files;
source flow_src{};
source nrm_src{};
bool has_flow = false;
bool has_nrm = false;
for (const auto &name: {"Terrain.big", "Core11.big"}) { for (const auto &name: {"Terrain.big", "Core11.big"}) {
const auto path = data_dir / name; const auto path = data_dir / name;
std::error_code ec; std::error_code ec;
@@ -343,9 +308,13 @@ export namespace ra3::terrain {
} }
for (const auto &archive: archives) { for (const auto &archive: archives) {
for (const auto &entry: archive.entries()) { for (const auto &entry: archive.entries()) {
auto stem = detail::tga_stem(entry.name);
if (stem.size() > 4U && stem.ends_with("_nrm")) continue;
if (!entry.name.ends_with(".tga") && !entry.name.ends_with(".TGA")) continue; if (!entry.name.ends_with(".tga") && !entry.name.ends_with(".TGA")) continue;
auto stem = detail::tga_stem(entry.name);
// The global ocean flow/normal pair is grabbed separately: the
// tile index deliberately drops `_nrm` files.
if (stem == "ra3_deepocean") { flow_src = source{&archive, entry.name}; has_flow = true; continue; }
if (stem == "ra3_deepocean_nrm") { nrm_src = source{&archive, entry.name}; has_nrm = true; continue; }
if (stem.size() > 4U && stem.ends_with("_nrm")) continue;
files.try_emplace(stem, source{&archive, entry.name}); files.try_emplace(stem, source{&archive, entry.name});
} }
} }
@@ -377,6 +346,16 @@ export namespace ra3::terrain {
// Leave the slot empty; the renderer falls back to a palette. // Leave the slot empty; the renderer falls back to a palette.
} }
} }
const auto decode_water = [](const source &src, bool present) -> image {
if (!present) return {};
try {
return ra3::render::decode_tga(src.archive->read(src.entry, true));
} catch (const std::exception &) {
return {};
}
};
set.water_flow = decode_water(flow_src, has_flow);
set.water_normal = decode_water(nrm_src, has_nrm);
return set; return set;
} }
@@ -394,12 +373,13 @@ export namespace ra3::terrain {
uint32 supersample = 2U; ///< Render at Nx and box-downsample (antialiasing). uint32 supersample = 2U; ///< Render at Nx and box-downsample (antialiasing).
}; };
/** Load tile textures from a directory of loose `*.tga` files (extracted assets). */ namespace detail {
[[nodiscard]] inline auto load_textures_from_dir(const map_data &map, const std::filesystem::path &dir, /** Index `*.tga` under a terrain dir by stem (lower-cased). */
const std::function<void(float)> &progress = {}) -> texture_set { [[nodiscard]] inline auto terrain_file_index(const std::filesystem::path &dir, bool include_normals = false)
-> std::unordered_map<std::string, std::filesystem::path> {
std::unordered_map<std::string, std::filesystem::path> files; std::unordered_map<std::string, std::filesystem::path> files;
std::error_code ec; std::error_code ec;
if (std::filesystem::is_directory(dir, ec)) { if (!std::filesystem::is_directory(dir, ec)) return files;
// Recursive: our own extract writes flat `terrain/*.tga`, ra3tools // Recursive: our own extract writes flat `terrain/*.tga`, ra3tools
// writes nested `.../art/terrain/*.tga`. Only index `terrain` dirs // writes nested `.../art/terrain/*.tga`. Only index `terrain` dirs
// so a full asset dump does not pull in every unrelated TGA. // so a full asset dump does not pull in every unrelated TGA.
@@ -409,40 +389,71 @@ export namespace ra3::terrain {
auto parent = path.parent_path().filename().string(); auto parent = path.parent_path().filename().string();
std::transform(parent.begin(), parent.end(), parent.begin(), [](unsigned char ch) { return static_cast<char>(std::tolower(ch)); }); std::transform(parent.begin(), parent.end(), parent.begin(), [](unsigned char ch) { return static_cast<char>(std::tolower(ch)); });
if (parent != "terrain") continue; if (parent != "terrain") continue;
auto stem = detail::tga_stem(path.filename().string()); auto stem = tga_stem(path.filename().string());
if (stem.size() > 4U && stem.ends_with("_nrm")) continue; if (!include_normals && stem.size() > 4U && stem.ends_with("_nrm")) continue;
files.try_emplace(stem, path); files.try_emplace(stem, path);
} }
return files;
} }
/** Match one map texture name against the index (see `load_textures_from_dir`). */
[[nodiscard]] inline auto match_terrain_file(const std::unordered_map<std::string, std::filesystem::path> &files, std::string name)
-> std::filesystem::path {
std::transform(name.begin(), name.end(), name.begin(), [](unsigned char ch) { return static_cast<char>(std::tolower(ch)); });
for (const auto &candidate: {name, "t" + name, "tmisc_" + name}) {
if (const auto it = files.find(candidate); it != files.end()) return it->second;
}
for (const auto &[stem, path]: files) {
if (stem.ends_with(name)) return path;
}
return {};
}
} // namespace detail
/**
* The loose `*.tga` files `map` resolves to under `dir`, de-duplicated.
*
* Used to stage just the tiles a single map needs (e.g. the wasm preload).
*/
[[nodiscard]] inline auto resolve_texture_files(const map_data &map, const std::filesystem::path &dir) -> std::vector<std::filesystem::path> {
// Include normals so the global ocean flow/normal pair is staged too:
// the terrain pass appends them to the atlas.
const auto files = detail::terrain_file_index(dir, true);
std::vector<std::filesystem::path> resolved;
for (const auto &texture: map.textures) {
if (auto found = detail::match_terrain_file(files, texture.name); !found.empty()) resolved.push_back(std::move(found));
}
for (const auto *water: {"ra3_deepocean", "ra3_deepocean_nrm"}) {
if (const auto it = files.find(water); it != files.end()) resolved.push_back(it->second);
}
std::sort(resolved.begin(), resolved.end());
resolved.erase(std::unique(resolved.begin(), resolved.end()), resolved.end());
return resolved;
}
/** Load tile textures from a directory of loose `*.tga` files (extracted assets). */
[[nodiscard]] inline auto load_textures_from_dir(const map_data &map, const std::filesystem::path &dir,
const std::function<void(float)> &progress = {}) -> texture_set {
const auto files = detail::terrain_file_index(dir, true);
texture_set set; texture_set set;
set.images.resize(map.textures.size()); set.images.resize(map.textures.size());
for (usize i = 0; i < map.textures.size(); ++i) { const auto decode_file = [](const std::filesystem::path &path) -> image {
auto name = map.textures[i].name; if (path.empty()) return {};
std::transform(name.begin(), name.end(), name.begin(), [](unsigned char ch) { return static_cast<char>(std::tolower(ch)); });
std::filesystem::path found;
for (const auto &candidate: {name, "t" + name, "tmisc_" + name}) {
if (const auto it = files.find(candidate); it != files.end()) {
found = it->second;
break;
}
}
if (found.empty()) {
for (const auto &[stem, path]: files) {
if (stem.ends_with(name)) {
found = path;
break;
}
}
}
if (found.empty()) continue;
try { try {
std::ifstream in(found, std::ios::binary); std::ifstream in(path, std::ios::binary);
std::vector<uint8> raw((std::istreambuf_iterator<char>(in)), std::istreambuf_iterator<char>()); std::vector<uint8> raw((std::istreambuf_iterator<char>(in)), std::istreambuf_iterator<char>());
set.images[i] = ra3::render::decode_tga(raw); return ra3::render::decode_tga(raw);
} catch (const std::exception &) { } catch (const std::exception &) {
return {};
} }
};
for (usize i = 0; i < map.textures.size(); ++i) {
const auto found = detail::match_terrain_file(files, map.textures[i].name);
set.images[i] = decode_file(found);
if (progress && !map.textures.empty()) progress(static_cast<float>(i + 1U) / static_cast<float>(map.textures.size())); if (progress && !map.textures.empty()) progress(static_cast<float>(i + 1U) / static_cast<float>(map.textures.size()));
} }
if (const auto it = files.find("ra3_deepocean"); it != files.end()) set.water_flow = decode_file(it->second);
if (const auto it = files.find("ra3_deepocean_nrm"); it != files.end()) set.water_normal = decode_file(it->second);
return set; return set;
} }
@@ -459,6 +470,7 @@ export namespace ra3::terrain {
float z_scale = 0.0390625F; float z_scale = 0.0390625F;
bool has_water = false; bool has_water = false;
float water_z = 0.0F; float water_z = 0.0F;
ra3::models::scene objects; ///< Buildings and props placed on the map (world-space triangle soup).
}; };
/** /**
@@ -470,14 +482,15 @@ export namespace ra3::terrain {
* base texture layer, the blend (and three-way) secondary layer and the * base texture layer, the blend (and three-way) secondary layer and the
* packed blend direction/flags, which the shader ramps across the cell. * packed blend direction/flags, which the shader ramps across the cell.
*/ */
[[nodiscard]] inline auto build_gpu_terrain(const map_data &map, const texture_set &set, const render_options &options = {}, [[nodiscard]] inline auto build_gpu_terrain(const map_data &map, const texture_set &set, const render_options &options,
const std::function<void(float)> &progress = {}) -> gpu_terrain { const ra3::models::scene &objects, const std::function<void(float)> &progress = {}) -> gpu_terrain {
gpu_terrain out; gpu_terrain out;
out.width = map.width; out.width = map.width;
out.height = map.height; out.height = map.height;
out.z_scale = options.z_scale; out.z_scale = options.z_scale;
out.has_water = map.has_water; out.has_water = map.has_water;
out.water_z = map.water_plane_z; out.water_z = map.water_plane_z;
out.objects = objects;
// Cell index -> texture layer. // Cell index -> texture layer.
const auto layer_of = [&](uint32 cell_index) -> uint16 { const auto layer_of = [&](uint32 cell_index) -> uint16 {
@@ -514,7 +527,12 @@ export namespace ra3::terrain {
out.cell_data[i * 4U + 3U] = static_cast<uint16>(packed); out.cell_data[i * 4U + 3U] = static_cast<uint16>(packed);
} }
out.layer_count = static_cast<uint32>(std::max<usize>(1U, map.textures.size())); // Two extra atlas layers hold the SAGE water flow map and bump normal so
// the water shader can sample them without a new binding on any backend:
// they are always the last two layers (water_flow = layer_count - 2,
// water_normal = layer_count - 1).
const auto tile_layers = static_cast<uint32>(std::max<usize>(1U, map.textures.size()));
out.layer_count = tile_layers + 2U;
uint32 layer_size = 64U; uint32 layer_size = 64U;
for (const auto &img: set.images) { for (const auto &img: set.images) {
if (!img.empty()) layer_size = std::max(layer_size, img.width()); if (!img.empty()) layer_size = std::max(layer_size, img.width());
@@ -529,21 +547,36 @@ export namespace ra3::terrain {
} }
out.layers.assign(static_cast<usize>(out.layer_count) * layer_size * layer_size, 0xFF3A4550U); out.layers.assign(static_cast<usize>(out.layer_count) * layer_size * layer_size, 0xFF3A4550U);
for (usize i = 0; i < map.textures.size(); ++i) { // Copy `img` into atlas layer `index`, box-nearest downscaled to
const auto &img = set.images[i]; // `layer_size`; `fallback` is the ARGB used when the image is absent.
if (img.empty()) continue; const auto blit_layer = [&](uint32 index, const image &img, uint32 fallback) {
for (uint32 y = 0; y < layer_size; ++y) { for (uint32 y = 0; y < layer_size; ++y) {
const auto sy = std::min(img.height() - 1U, y * img.height() / layer_size);
for (uint32 x = 0; x < layer_size; ++x) { for (uint32 x = 0; x < layer_size; ++x) {
uint32 px = fallback;
if (!img.empty()) {
const auto sy = std::min(img.height() - 1U, y * img.height() / layer_size);
const auto sx = std::min(img.width() - 1U, x * img.width() / layer_size); const auto sx = std::min(img.width() - 1U, x * img.width() / layer_size);
out.layers[(i * layer_size + y) * layer_size + x] = img.data()[static_cast<usize>(sy) * img.width() + sx]; px = img.data()[static_cast<usize>(sy) * img.width() + sx];
}
out.layers[(static_cast<usize>(index) * layer_size + y) * layer_size + x] = px;
} }
} }
};
for (usize i = 0; i < map.textures.size(); ++i) {
blit_layer(static_cast<uint32>(i), set.images[i], 0xFF3A4550U);
} }
blit_layer(tile_layers, set.water_flow, 0xFF808080U); // neutral flow (0, 0)
blit_layer(tile_layers + 1U, set.water_normal, 0xFF8080FFU); // flat normal (0, 0, 1)
if (progress) progress(1.0F); if (progress) progress(1.0F);
return out; return out;
} }
/** Terrain without any placed objects. */
[[nodiscard]] inline auto build_gpu_terrain(const map_data &map, const texture_set &set, const render_options &options = {},
const std::function<void(float)> &progress = {}) -> gpu_terrain {
return build_gpu_terrain(map, set, options, ra3::models::scene{}, progress);
}
namespace detail { namespace detail {
/** /**
* The source texture a tile cell maps to. The texture is sampled * The source texture a tile cell maps to. The texture is sampled
@@ -699,6 +732,146 @@ export namespace ra3::terrain {
const auto c2 = f2 > 0.0F ? sample_layer(layers.three, wx, wy) : c0; const auto c2 = f2 > 0.0F ? sample_layer(layers.three, wx, wy) : c0;
return mix_color(mix_color(c0, c1, f1), c2, f2); return mix_color(mix_color(c0, c1, f1), c2, f2);
} }
/**
* Rasterise the map's building/prop scene over an already ray-marched
* terrain image, depth-testing against it.
*
* The camera basis is the one `render3d` used, so the two passes agree;
* `zbuf` holds the terrain's view-space depth per pixel (large where the
* ray hit nothing). Triangles are z-tested and perspective-correct.
*/
inline auto rasterize_objects(image &hi, std::vector<float> &zbuf, const ra3::models::scene &scene, const std::array<float, 3> &cam,
const std::array<float, 3> &f, const std::array<float, 3> &r, const std::array<float, 3> &u, float tan_half,
float aspect) -> void {
const auto rw = static_cast<int>(hi.width());
const auto rh = static_cast<int>(hi.height());
if (rw <= 0 || rh <= 0) return;
const auto dot3 = [](const std::array<float, 3> &a, const std::array<float, 3> &b) { return a[0] * b[0] + a[1] * b[1] + a[2] * b[2]; };
float sun[3] = {0.45F, 0.35F, 0.82F};
const auto sl = std::sqrt(sun[0] * sun[0] + sun[1] * sun[1] + sun[2] * sun[2]);
sun[0] /= sl;
sun[1] /= sl;
sun[2] /= sl;
constexpr float ambient = 0.38F;
constexpr float near_plane = 10.0F;
const auto sample = [&](uint32 layer, float tu, float tv) -> uint32 {
if (layer >= scene.textures.size() || scene.textures[layer].empty()) return argb(140, 140, 140);
const auto &img = scene.textures[layer];
const auto wrap = [](float x) { return x - std::floor(x); };
const auto sx = std::min(img.width() - 1U, static_cast<uint32>(wrap(tu) * static_cast<float>(img.width())));
const auto sy = std::min(img.height() - 1U, static_cast<uint32>(wrap(tv) * static_cast<float>(img.height())));
return img.data()[static_cast<usize>(sy) * img.width() + sx];
};
// Average colour per texture, used when a triangle covers fewer pixels
// than texels (minification) — a cheap mip-0-away fallback that keeps
// distant props from shimmering.
std::vector<uint32> average(scene.textures.size(), argb(140, 140, 140));
for (usize i = 0; i < scene.textures.size(); ++i) {
const auto &img = scene.textures[i];
if (img.empty()) continue;
usize r = 0;
usize g = 0;
usize b = 0;
for (usize p = 0; p < static_cast<usize>(img.width()) * img.height(); ++p) {
r += (img.data()[p] >> 16U) & 0xFFU;
g += (img.data()[p] >> 8U) & 0xFFU;
b += img.data()[p] & 0xFFU;
}
const auto n = static_cast<usize>(img.width()) * img.height();
average[i] = argb(static_cast<uint8>(r / n), static_cast<uint8>(g / n), static_cast<uint8>(b / n));
}
struct projected {
float sx = 0.0F;
float sy = 0.0F;
float inv_a = 0.0F; ///< 1 / view-space depth
};
const auto project = [&](const ra3::models::vertex &v, projected &out) -> bool {
const std::array<float, 3> rel{v.x - cam[0], v.y - cam[1], v.z - cam[2]};
const auto a = dot3(rel, f);
if (a <= near_plane) return false;
const auto ndc_x = (dot3(rel, r) / a) / (tan_half * aspect);
const auto ndc_y = (dot3(rel, u) / a) / tan_half;
out.sx = (ndc_x * 0.5F + 0.5F) * static_cast<float>(rw);
out.sy = (0.5F - ndc_y * 0.5F) * static_cast<float>(rh);
out.inv_a = 1.0F / a;
return true;
};
for (usize t = 0; t + 2U < scene.indices.size(); t += 3U) {
const auto &v0 = scene.vertices[scene.indices[t]];
const auto &v1 = scene.vertices[scene.indices[t + 1U]];
const auto &v2 = scene.vertices[scene.indices[t + 2U]];
projected p0, p1, p2;
if (!project(v0, p0) || !project(v1, p1) || !project(v2, p2)) continue;
const auto area = (p1.sx - p0.sx) * (p2.sy - p0.sy) - (p1.sy - p0.sy) * (p2.sx - p0.sx);
if (std::abs(area) < 1.0e-6F) continue;
const auto sign = area < 0.0F ? -1.0F : 1.0F;
// Texture footprint: if the triangle covers more texels than
// pixels it is minified, so fall back to the texture average.
const auto uv_area = std::abs((v1.u - v0.u) * (v2.v - v0.v) - (v2.u - v0.u) * (v1.v - v0.v));
const auto texel_footprint = uv_area * static_cast<float>(scene.texture_size) * static_cast<float>(scene.texture_size);
const auto minified = texel_footprint > 2.0F * std::abs(area);
const auto flat_layer = static_cast<uint32>(v0.layer + 0.5F);
const auto flat_color = flat_layer < average.size() ? average[flat_layer] : argb(140, 140, 140);
const auto min_x = std::max(0, static_cast<int>(std::floor(std::min({p0.sx, p1.sx, p2.sx}))));
const auto max_x = std::min(rw - 1, static_cast<int>(std::ceil(std::max({p0.sx, p1.sx, p2.sx}))));
const auto min_y = std::max(0, static_cast<int>(std::floor(std::min({p0.sy, p1.sy, p2.sy}))));
const auto max_y = std::min(rh - 1, static_cast<int>(std::ceil(std::max({p0.sy, p1.sy, p2.sy}))));
for (int y = min_y; y <= max_y; ++y) {
for (int x = min_x; x <= max_x; ++x) {
const auto px = static_cast<float>(x) + 0.5F;
const auto py = static_cast<float>(y) + 0.5F;
auto w0 = ((p1.sx - p0.sx) * (py - p0.sy) - (p1.sy - p0.sy) * (px - p0.sx)) * sign;
auto w1 = ((p2.sx - p1.sx) * (py - p1.sy) - (p2.sy - p1.sy) * (px - p1.sx)) * sign;
auto w2 = ((p0.sx - p2.sx) * (py - p2.sy) - (p0.sy - p2.sy) * (px - p2.sx)) * sign;
if (w0 < 0.0F || w1 < 0.0F || w2 < 0.0F) continue;
const auto sum = w0 + w1 + w2;
if (sum <= 0.0F) continue;
w0 /= sum;
w1 /= sum;
w2 /= sum;
// Perspective-correct depth and attributes.
const auto inv_a = w0 * p0.inv_a + w1 * p1.inv_a + w2 * p2.inv_a;
const auto depth = 1.0F / inv_a;
const auto pixel = static_cast<usize>(y) * static_cast<usize>(rw) + static_cast<usize>(x);
// Ground decals carry a per-vertex bias toward the camera
// (the retail shader's screen-space depth offset); it
// keeps roads/sidewalks from z-fighting the terrain.
const auto bias = w0 * v0.bias + w1 * v1.bias + w2 * v2.bias;
if (depth - bias - 1.0F >= zbuf[pixel]) continue;
const auto tu = (w0 * v0.u * p0.inv_a + w1 * v1.u * p1.inv_a + w2 * v2.u * p2.inv_a) * depth;
const auto tv = (w0 * v0.v * p0.inv_a + w1 * v1.v * p1.inv_a + w2 * v2.v * p2.inv_a) * depth;
float nx = w0 * v0.nx + w1 * v1.nx + w2 * v2.nx;
float ny = w0 * v0.ny + w1 * v1.ny + w2 * v2.ny;
float nz = w0 * v0.nz + w1 * v1.nz + w2 * v2.nz;
const auto nl = std::sqrt(nx * nx + ny * ny + nz * nz);
if (nl > 1.0e-6F) {
nx /= nl;
ny /= nl;
nz /= nl;
}
const auto lambert = std::max(0.0F, std::abs(nx * sun[0] + ny * sun[1] + nz * sun[2]));
const auto shade = ambient + (1.0F - ambient) * lambert;
const auto texel = minified ? flat_color : sample(flat_layer, tu, tv);
const auto mod = [&](uint32 shift) {
return static_cast<uint8>(std::clamp(static_cast<float>((texel >> shift) & 0xFFU) * shade, 0.0F, 255.0F));
};
hi.data()[pixel] = argb(mod(16U), mod(8U), mod(0U));
zbuf[pixel] = depth;
}
}
}
}
} }
/** /**
@@ -829,7 +1002,7 @@ export namespace ra3::terrain {
* a transformed 2D image. * a transformed 2D image.
*/ */
[[nodiscard]] inline auto render3d(const map_data &map, const texture_set &set, const camera3d &camera, uint32 out_w, uint32 out_h, [[nodiscard]] inline auto render3d(const map_data &map, const texture_set &set, const camera3d &camera, uint32 out_w, uint32 out_h,
const render_options &options = {}) -> image { const render_options &options = {}, const ra3::models::scene *objects = nullptr) -> image {
if (!map.valid) throw terrain_error("terrain not parsed"); if (!map.valid) throw terrain_error("terrain not parsed");
out_w = std::max(1U, out_w); out_w = std::max(1U, out_w);
out_h = std::max(1U, out_h); out_h = std::max(1U, out_h);
@@ -946,6 +1119,7 @@ export namespace ra3::terrain {
}; };
image hi(rw, rh, argb(0, 0, 0)); image hi(rw, rh, argb(0, 0, 0));
std::vector<float> zbuf(static_cast<usize>(rw) * rh, 1.0e30F);
for (uint32 py = 0; py < rh; ++py) { for (uint32 py = 0; py < rh; ++py) {
const auto ndc_y = 1.0F - 2.0F * (static_cast<float>(py) + 0.5F) / static_cast<float>(rh); const auto ndc_y = 1.0F - 2.0F * (static_cast<float>(py) + 0.5F) / static_cast<float>(rh);
for (uint32 px = 0; px < rw; ++px) { for (uint32 px = 0; px < rw; ++px) {
@@ -957,66 +1131,94 @@ export namespace ra3::terrain {
dx /= dlen; dx /= dlen;
dy /= dlen; dy /= dlen;
dz /= dlen; dz /= dlen;
const auto pixel = static_cast<usize>(py) * rw + px;
if (dz >= -1.0e-4F) { if (dz >= -1.0e-4F) {
const auto t = std::clamp(std::abs(dz) * 2.0F, 0.0F, 1.0F); const auto t = std::clamp(std::abs(dz) * 2.0F, 0.0F, 1.0F);
hi.data()[static_cast<usize>(py) * rw + px] = hi.data()[pixel] =
argb(static_cast<uint8>(120.0F + 60.0F * t), static_cast<uint8>(150.0F + 45.0F * t), static_cast<uint8>(190.0F + 40.0F * t)); argb(static_cast<uint8>(120.0F + 60.0F * t), static_cast<uint8>(150.0F + 45.0F * t), static_cast<uint8>(190.0F + 40.0F * t));
continue; continue;
} }
auto t = cell_size * 0.5F; // Clip the ray to the map's XY rectangle. The boundary is an
auto dt = cell_size * 0.5F; // exact plane, so the silhouette there stays razor-sharp
auto prev_t = t; // instead of stair-stepping across it; outside the map is sky.
bool hit = false; auto t_enter = 0.0F;
float hit_t = 0.0F; auto t_exit = 1.0e30F;
for (int iter = 0; iter < 4000 && t < 60000.0F; ++iter) { const auto slab = [](float origin, float dir, float span, float &lo_t, float &hi_t) -> bool {
const auto wx = cam_x + dx * t; if (std::abs(dir) < 1.0e-6F) return origin >= 0.0F && origin <= span;
const auto wy = cam_y + dy * t; const auto a = (0.0F - origin) / dir;
const auto wz = cam_z + dz * t; const auto b = (span - origin) / dir;
if (wx < 0.0F || wy < 0.0F || wx >= world_w || wy >= world_h) { lo_t = std::max(lo_t, std::min(a, b));
prev_t = t; hi_t = std::min(hi_t, std::max(a, b));
dt *= 1.03F; return true;
t += dt; };
continue; if (!slab(cam_x, dx, world_w, t_enter, t_exit) || !slab(cam_y, dy, world_h, t_enter, t_exit) || t_exit <= 0.0F) {
} hi.data()[pixel] = argb(150, 170, 200);
if (map.has_water && wz <= map.water_plane_z) {
hit = true;
hit_t = t;
break;
}
if (wz <= sample_height(wx, wy)) {
hit = true;
hit_t = t;
break;
}
prev_t = t;
dt *= 1.03F;
t += dt;
}
if (!hit) {
hi.data()[static_cast<usize>(py) * rw + px] = argb(150, 170, 200);
continue; continue;
} }
const auto surface_at = [&](float wx, float wy) -> float {
const auto h = sample_height(wx, wy);
return map.has_water ? std::max(h, static_cast<float>(map.water_plane_z)) : h;
};
// March the heightfield cell by cell: the step is never longer
// than the time to cross one cell (in the dominant horizontal
// axis), while a clearance term lets the ray skip the empty air
// above the surface. Resolving every cell is what keeps cliff and
// map-edge silhouettes from quantising into huge stair-steps that
// crawl/wave as the camera pans.
const auto horiz = std::max(std::abs(dx), std::abs(dy));
const auto cell_step = std::min(cell_size / std::max(horiz, 1.0e-4F), cell_size * 32.0F);
auto t = std::max(t_enter, cell_size * 0.5F);
auto prev_t = t;
bool hit = false;
float hit_t = 0.0F;
for (int iter = 0; iter < 4096 && t <= t_exit; ++iter) {
const auto wx = cam_x + dx * t;
const auto wy = cam_y + dy * t;
const auto wz = cam_z + dz * t;
const auto surface = surface_at(wx, wy);
if (wz <= surface) {
hit = true;
hit_t = t;
break;
}
const auto clearance = (wz - surface) / std::max(-dz, 1.0e-4F);
prev_t = t;
t += std::clamp(clearance, cell_step, cell_step * 8.0F);
}
if (!hit) {
hi.data()[pixel] = argb(150, 170, 200);
continue;
}
// Refine the first crossing; with a sub-cell bracket this
// converges to the exact surface point.
auto lo = prev_t; auto lo = prev_t;
auto up = hit_t; auto up = hit_t;
for (int i = 0; i < 6; ++i) { for (int i = 0; i < 18; ++i) {
const auto mid = 0.5F * (lo + up); const auto mid = 0.5F * (lo + up);
const auto wx = cam_x + dx * mid; const auto wx = cam_x + dx * mid;
const auto wy = cam_y + dy * mid; const auto wy = cam_y + dy * mid;
const auto wz = cam_z + dz * mid; const auto wz = cam_z + dz * mid;
const auto water = map.has_water && wz <= map.water_plane_z; if (wz <= surface_at(wx, wy)) {
if (water || wz <= sample_height(wx, wy)) {
up = mid; up = mid;
} else { } else {
lo = mid; lo = mid;
} }
} }
hi.data()[static_cast<usize>(py) * rw + px] = hit_color(cam_x + dx * up, cam_y + dy * up, cam_z + dz * up); hi.data()[pixel] = hit_color(cam_x + dx * up, cam_y + dy * up, cam_z + dz * up);
// View-space depth of the hit, for the object pass below.
zbuf[pixel] = up * (dx * fx + dy * fy + dz * fz);
} }
} }
if (objects != nullptr && !objects->empty()) {
detail::rasterize_objects(hi, zbuf, *objects, {cam_x, cam_y, cam_z}, {fx, fy, fz}, {rx, ry, rz}, {ux, uy, uz}, tan_half, aspect);
}
if (ss == 1U) return hi; if (ss == 1U) return hi;
image out(out_w, out_h, argb(0, 0, 0)); image out(out_w, out_h, argb(0, 0, 0));
for (uint32 y = 0; y < out_h; ++y) { for (uint32 y = 0; y < out_h; ++y) {
+18 -4
View File
@@ -101,6 +101,8 @@ export namespace ra3::ui {
out.dx = event.motion.xrel; out.dx = event.motion.xrel;
out.dy = event.motion.yrel; out.dy = event.motion.yrel;
out.left = (event.motion.state & SDL_BUTTON_LMASK) != 0U; out.left = (event.motion.state & SDL_BUTTON_LMASK) != 0U;
out.middle = (event.motion.state & SDL_BUTTON_MMASK) != 0U;
out.right = (event.motion.state & SDL_BUTTON_RMASK) != 0U;
return true; return true;
case SDL_EVENT_MOUSE_BUTTON_DOWN: case SDL_EVENT_MOUSE_BUTTON_DOWN:
out = {}; out = {};
@@ -108,6 +110,18 @@ export namespace ra3::ui {
out.x = event.button.x; out.x = event.button.x;
out.y = event.button.y; out.y = event.button.y;
out.left = event.button.button == SDL_BUTTON_LEFT; out.left = event.button.button == SDL_BUTTON_LEFT;
out.middle = event.button.button == SDL_BUTTON_MIDDLE;
out.right = event.button.button == SDL_BUTTON_RIGHT;
return true;
case SDL_EVENT_MOUSE_BUTTON_UP:
out = {};
out.type = ra3::render::ui_event_type::mouse_button;
out.x = event.button.x;
out.y = event.button.y;
out.released = true;
out.left = event.button.button == SDL_BUTTON_LEFT;
out.middle = event.button.button == SDL_BUTTON_MIDDLE;
out.right = event.button.button == SDL_BUTTON_RIGHT;
return true; return true;
case SDL_EVENT_MOUSE_WHEEL: case SDL_EVENT_MOUSE_WHEEL:
out = {}; out = {};
@@ -132,10 +146,10 @@ export namespace ra3::ui {
const bool *keys = SDL_GetKeyboardState(nullptr); const bool *keys = SDL_GetKeyboardState(nullptr);
if (keys == nullptr) return false; if (keys == nullptr) return false;
switch (key) { switch (key) {
case ra3::render::ui_key::up: return keys[SDL_SCANCODE_W] || keys[SDL_SCANCODE_UP]; case ra3::render::ui_key::up: return keys[SDL_SCANCODE_UP];
case ra3::render::ui_key::down: return keys[SDL_SCANCODE_S] || keys[SDL_SCANCODE_DOWN]; case ra3::render::ui_key::down: return keys[SDL_SCANCODE_DOWN];
case ra3::render::ui_key::left: return keys[SDL_SCANCODE_A] || keys[SDL_SCANCODE_LEFT]; case ra3::render::ui_key::left: return keys[SDL_SCANCODE_LEFT];
case ra3::render::ui_key::right: return keys[SDL_SCANCODE_D] || keys[SDL_SCANCODE_RIGHT]; case ra3::render::ui_key::right: return keys[SDL_SCANCODE_RIGHT];
default: return false; default: return false;
} }
} }
+392 -21
View File
@@ -92,6 +92,7 @@ export namespace ra3::vulkan {
if (!this->create_device()) return false; if (!this->create_device()) return false;
if (!this->create_swapchain()) return false; if (!this->create_swapchain()) return false;
if (!this->create_render_pass()) return false; if (!this->create_render_pass()) return false;
if (!this->create_framebuffers()) return false;
if (!this->create_pipeline()) return false; if (!this->create_pipeline()) return false;
if (!this->create_commands()) return false; if (!this->create_commands()) return false;
if (!this->create_sync()) return false; if (!this->create_sync()) return false;
@@ -294,6 +295,8 @@ export namespace ra3::vulkan {
out.dx = event.motion.xrel; out.dx = event.motion.xrel;
out.dy = event.motion.yrel; out.dy = event.motion.yrel;
out.left = (event.motion.state & SDL_BUTTON_LMASK) != 0U; out.left = (event.motion.state & SDL_BUTTON_LMASK) != 0U;
out.middle = (event.motion.state & SDL_BUTTON_MMASK) != 0U;
out.right = (event.motion.state & SDL_BUTTON_RMASK) != 0U;
return true; return true;
case SDL_EVENT_MOUSE_BUTTON_DOWN: case SDL_EVENT_MOUSE_BUTTON_DOWN:
out = {}; out = {};
@@ -301,6 +304,18 @@ export namespace ra3::vulkan {
out.x = event.button.x; out.x = event.button.x;
out.y = event.button.y; out.y = event.button.y;
out.left = event.button.button == SDL_BUTTON_LEFT; out.left = event.button.button == SDL_BUTTON_LEFT;
out.middle = event.button.button == SDL_BUTTON_MIDDLE;
out.right = event.button.button == SDL_BUTTON_RIGHT;
return true;
case SDL_EVENT_MOUSE_BUTTON_UP:
out = {};
out.type = ra3::render::ui_event_type::mouse_button;
out.x = event.button.x;
out.y = event.button.y;
out.released = true;
out.left = event.button.button == SDL_BUTTON_LEFT;
out.middle = event.button.button == SDL_BUTTON_MIDDLE;
out.right = event.button.button == SDL_BUTTON_RIGHT;
return true; return true;
case SDL_EVENT_MOUSE_WHEEL: case SDL_EVENT_MOUSE_WHEEL:
out = {}; out = {};
@@ -325,10 +340,10 @@ export namespace ra3::vulkan {
const bool *keys = SDL_GetKeyboardState(nullptr); const bool *keys = SDL_GetKeyboardState(nullptr);
if (keys == nullptr) return false; if (keys == nullptr) return false;
switch (key) { switch (key) {
case ra3::render::ui_key::up: return keys[SDL_SCANCODE_W] || keys[SDL_SCANCODE_UP]; case ra3::render::ui_key::up: return keys[SDL_SCANCODE_UP];
case ra3::render::ui_key::down: return keys[SDL_SCANCODE_S] || keys[SDL_SCANCODE_DOWN]; case ra3::render::ui_key::down: return keys[SDL_SCANCODE_DOWN];
case ra3::render::ui_key::left: return keys[SDL_SCANCODE_A] || keys[SDL_SCANCODE_LEFT]; case ra3::render::ui_key::left: return keys[SDL_SCANCODE_LEFT];
case ra3::render::ui_key::right: return keys[SDL_SCANCODE_D] || keys[SDL_SCANCODE_RIGHT]; case ra3::render::ui_key::right: return keys[SDL_SCANCODE_RIGHT];
default: return false; default: return false;
} }
} }
@@ -341,6 +356,12 @@ export namespace ra3::vulkan {
if (!this->create_terrain_pipeline()) return false; if (!this->create_terrain_pipeline()) return false;
if (!this->create_terrain_images(terrain)) return false; if (!this->create_terrain_images(terrain)) return false;
if (!this->create_terrain_descriptors()) return false; if (!this->create_terrain_descriptors()) return false;
if (!terrain.objects.empty()) {
if (!this->create_object_pipeline()) return false;
if (!this->create_object_buffers(terrain)) return false;
if (!this->create_object_descriptors()) return false;
objects_ready_ = true;
}
terrain_ready_ = true; terrain_ready_ = true;
} }
if (overlay.label_changed) this->update_overlay(overlay_label_, overlay_label_set_, overlay_label_w_, overlay_label_h_, overlay.label); if (overlay.label_changed) this->update_overlay(overlay_label_, overlay_label_set_, overlay_label_w_, overlay_label_h_, overlay.label);
@@ -669,6 +690,11 @@ export namespace ra3::vulkan {
VkPipelineMultisampleStateCreateInfo multisample{}; VkPipelineMultisampleStateCreateInfo multisample{};
multisample.sType = VK_STRUCTURE_TYPE_PIPELINE_MULTISAMPLE_STATE_CREATE_INFO; multisample.sType = VK_STRUCTURE_TYPE_PIPELINE_MULTISAMPLE_STATE_CREATE_INFO;
multisample.rasterizationSamples = VK_SAMPLE_COUNT_1_BIT; multisample.rasterizationSamples = VK_SAMPLE_COUNT_1_BIT;
VkPipelineDepthStencilStateCreateInfo depth_stencil{};
depth_stencil.sType = VK_STRUCTURE_TYPE_PIPELINE_DEPTH_STENCIL_STATE_CREATE_INFO;
depth_stencil.depthTestEnable = VK_TRUE;
depth_stencil.depthWriteEnable = VK_TRUE;
depth_stencil.depthCompareOp = VK_COMPARE_OP_LESS_OR_EQUAL;
VkPipelineColorBlendAttachmentState blend_attachment{}; VkPipelineColorBlendAttachmentState blend_attachment{};
blend_attachment.colorWriteMask = VK_COLOR_COMPONENT_R_BIT | VK_COLOR_COMPONENT_G_BIT | VK_COLOR_COMPONENT_B_BIT | VK_COLOR_COMPONENT_A_BIT; blend_attachment.colorWriteMask = VK_COLOR_COMPONENT_R_BIT | VK_COLOR_COMPONENT_G_BIT | VK_COLOR_COMPONENT_B_BIT | VK_COLOR_COMPONENT_A_BIT;
VkPipelineColorBlendStateCreateInfo blend{}; VkPipelineColorBlendStateCreateInfo blend{};
@@ -690,6 +716,7 @@ export namespace ra3::vulkan {
pipeline_info.pViewportState = &viewport; pipeline_info.pViewportState = &viewport;
pipeline_info.pRasterizationState = &raster; pipeline_info.pRasterizationState = &raster;
pipeline_info.pMultisampleState = &multisample; pipeline_info.pMultisampleState = &multisample;
pipeline_info.pDepthStencilState = &depth_stencil;
pipeline_info.pColorBlendState = &blend; pipeline_info.pColorBlendState = &blend;
pipeline_info.pDynamicState = &dynamic; pipeline_info.pDynamicState = &dynamic;
pipeline_info.layout = terrain_pipeline_layout_; pipeline_info.layout = terrain_pipeline_layout_;
@@ -737,6 +764,198 @@ export namespace ra3::vulkan {
return true; return true;
} }
/** Host-visible vertex/index buffer, filled once (the map's geometry is static). */
[[nodiscard]] auto make_host_buffer(const void *data, VkDeviceSize bytes, VkBufferUsageFlags usage, VkBuffer &buffer,
VkDeviceMemory &memory) -> bool {
VkBufferCreateInfo info{};
info.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO;
info.size = bytes;
info.usage = usage;
info.sharingMode = VK_SHARING_MODE_EXCLUSIVE;
if (!detail::check(vkCreateBuffer(device_, &info, nullptr, &buffer), "vkCreateBuffer(object)")) return false;
VkMemoryRequirements requirements{};
vkGetBufferMemoryRequirements(device_, buffer, &requirements);
VkMemoryAllocateInfo allocate{};
allocate.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO;
allocate.allocationSize = requirements.size;
allocate.memoryTypeIndex = this->memory_type(requirements.memoryTypeBits, VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT);
if (!detail::check(vkAllocateMemory(device_, &allocate, nullptr, &memory), "vkAllocateMemory(object)")) return false;
if (!detail::check(vkBindBufferMemory(device_, buffer, memory, 0), "vkBindBufferMemory(object)")) return false;
void *mapped = nullptr;
if (!detail::check(vkMapMemory(device_, memory, 0, bytes, 0, &mapped), "vkMapMemory(object)")) return false;
std::memcpy(mapped, data, static_cast<std::size_t>(bytes));
vkUnmapMemory(device_, memory);
return true;
}
/** Upload the static-map geometry and its texture array. */
[[nodiscard]] auto create_object_buffers(const ra3::terrain::gpu_terrain &terrain) -> bool {
const auto &scene = terrain.objects;
if (scene.empty()) return true;
const auto vertex_bytes = static_cast<VkDeviceSize>(scene.vertices.size()) * sizeof(ra3::models::vertex);
const auto index_bytes = static_cast<VkDeviceSize>(scene.indices.size()) * sizeof(uint32_t);
if (!this->make_host_buffer(scene.vertices.data(), vertex_bytes, VK_BUFFER_USAGE_VERTEX_BUFFER_BIT, object_vertex_buffer_, object_vertex_memory_)) return false;
if (!this->make_host_buffer(scene.indices.data(), index_bytes, VK_BUFFER_USAGE_INDEX_BUFFER_BIT, object_index_buffer_, object_index_memory_)) return false;
object_index_count_ = static_cast<uint32_t>(scene.indices.size());
const auto size = std::max(1U, scene.texture_size);
const auto layers = std::max<size_t>(1U, scene.textures.size());
std::vector<uint32_t> pixels(static_cast<size_t>(size) * size * layers, 0xFFFFFFFFU);
for (size_t layer = 0; layer < scene.textures.size(); ++layer) {
const auto &texture = scene.textures[layer];
if (texture.empty()) continue;
// The scene already resized every texture to `texture_size`.
for (uint32_t y = 0; y < std::min(size, texture.height()); ++y) {
for (uint32_t x = 0; x < std::min(size, texture.width()); ++x) {
pixels[(static_cast<size_t>(layer) * size + y) * size + x] = texture.data()[static_cast<size_t>(y) * texture.width() + x];
}
}
}
return this->make_gpu_image(object_texture_, size, size, VK_FORMAT_B8G8R8A8_UNORM, pixels.data(),
static_cast<VkDeviceSize>(pixels.size()) * 4U, this->mip_count(size), VK_FILTER_LINEAR, VK_SAMPLER_MIPMAP_MODE_LINEAR,
static_cast<uint32_t>(layers), VK_SAMPLER_ADDRESS_MODE_CLAMP_TO_EDGE);
}
/** Full mip chain length for a square texture. */
[[nodiscard]] static auto mip_count(uint32_t size) -> uint32_t {
uint32_t mips = 1U;
while (size > 1U) {
size >>= 1U;
++mips;
}
return mips;
}
[[nodiscard]] auto create_object_pipeline() -> bool {
VkPushConstantRange push_range{};
push_range.stageFlags = VK_SHADER_STAGE_VERTEX_BIT | VK_SHADER_STAGE_FRAGMENT_BIT;
push_range.size = sizeof(float) * 20U;
VkPipelineLayoutCreateInfo layout_info{};
layout_info.sType = VK_STRUCTURE_TYPE_PIPELINE_LAYOUT_CREATE_INFO;
layout_info.setLayoutCount = 1U;
layout_info.pSetLayouts = &descriptor_layout_; // reused: one combined image sampler
layout_info.pushConstantRangeCount = 1U;
layout_info.pPushConstantRanges = &push_range;
if (!detail::check(vkCreatePipelineLayout(device_, &layout_info, nullptr, &object_pipeline_layout_), "vkCreatePipelineLayout(object)")) return false;
VkShaderModule vertex = VK_NULL_HANDLE;
VkShaderModule fragment = VK_NULL_HANDLE;
if (!this->make_shader_module(ra3_shaders::object_vert_spv, ra3_shaders::object_vert_spv_words, vertex)) return false;
if (!this->make_shader_module(ra3_shaders::object_frag_spv, ra3_shaders::object_frag_spv_words, fragment)) return false;
VkPipelineShaderStageCreateInfo stages[2]{};
stages[0].sType = VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO;
stages[0].stage = VK_SHADER_STAGE_VERTEX_BIT;
stages[0].module = vertex;
stages[0].pName = "main";
stages[1].sType = VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO;
stages[1].stage = VK_SHADER_STAGE_FRAGMENT_BIT;
stages[1].module = fragment;
stages[1].pName = "main";
VkVertexInputBindingDescription binding{};
binding.binding = 0U;
binding.stride = sizeof(ra3::models::vertex);
binding.inputRate = VK_VERTEX_INPUT_RATE_VERTEX;
VkVertexInputAttributeDescription attributes[5]{};
attributes[0] = {0U, 0U, VK_FORMAT_R32G32B32_SFLOAT, 0U};
attributes[1] = {1U, 0U, VK_FORMAT_R32G32B32_SFLOAT, 12U};
attributes[2] = {2U, 0U, VK_FORMAT_R32G32_SFLOAT, 24U};
attributes[3] = {3U, 0U, VK_FORMAT_R32_SFLOAT, 32U};
attributes[4] = {4U, 0U, VK_FORMAT_R32_SFLOAT, 36U};
static_assert(sizeof(ra3::models::vertex) == 40U, "object vertex layout changed");
VkPipelineVertexInputStateCreateInfo vertex_input{};
vertex_input.sType = VK_STRUCTURE_TYPE_PIPELINE_VERTEX_INPUT_STATE_CREATE_INFO;
vertex_input.vertexBindingDescriptionCount = 1U;
vertex_input.pVertexBindingDescriptions = &binding;
vertex_input.vertexAttributeDescriptionCount = 5U;
vertex_input.pVertexAttributeDescriptions = attributes;
VkPipelineInputAssemblyStateCreateInfo assembly{};
assembly.sType = VK_STRUCTURE_TYPE_PIPELINE_INPUT_ASSEMBLY_STATE_CREATE_INFO;
assembly.topology = VK_PRIMITIVE_TOPOLOGY_TRIANGLE_LIST;
VkPipelineViewportStateCreateInfo viewport{};
viewport.sType = VK_STRUCTURE_TYPE_PIPELINE_VIEWPORT_STATE_CREATE_INFO;
viewport.viewportCount = 1U;
viewport.scissorCount = 1U;
VkPipelineRasterizationStateCreateInfo raster{};
raster.sType = VK_STRUCTURE_TYPE_PIPELINE_RASTERIZATION_STATE_CREATE_INFO;
raster.polygonMode = VK_POLYGON_MODE_FILL;
raster.cullMode = VK_CULL_MODE_NONE;
raster.frontFace = VK_FRONT_FACE_COUNTER_CLOCKWISE;
raster.lineWidth = 1.0F;
// Ground decals (sidewalks/roads/deck pieces) sit exactly on the
// terrain; a small negative depth bias keeps them from z-fighting.
raster.depthBiasEnable = VK_TRUE;
raster.depthBiasConstantFactor = -4.0F;
raster.depthBiasSlopeFactor = -4.0F;
VkPipelineMultisampleStateCreateInfo multisample{};
multisample.sType = VK_STRUCTURE_TYPE_PIPELINE_MULTISAMPLE_STATE_CREATE_INFO;
multisample.rasterizationSamples = VK_SAMPLE_COUNT_1_BIT;
VkPipelineDepthStencilStateCreateInfo depth_stencil{};
depth_stencil.sType = VK_STRUCTURE_TYPE_PIPELINE_DEPTH_STENCIL_STATE_CREATE_INFO;
depth_stencil.depthTestEnable = VK_TRUE;
depth_stencil.depthWriteEnable = VK_TRUE;
depth_stencil.depthCompareOp = VK_COMPARE_OP_LESS_OR_EQUAL;
VkPipelineColorBlendAttachmentState blend_attachment{};
blend_attachment.colorWriteMask = VK_COLOR_COMPONENT_R_BIT | VK_COLOR_COMPONENT_G_BIT | VK_COLOR_COMPONENT_B_BIT | VK_COLOR_COMPONENT_A_BIT;
blend_attachment.blendEnable = VK_FALSE;
VkPipelineColorBlendStateCreateInfo blend{};
blend.sType = VK_STRUCTURE_TYPE_PIPELINE_COLOR_BLEND_STATE_CREATE_INFO;
blend.attachmentCount = 1U;
blend.pAttachments = &blend_attachment;
const VkDynamicState dynamic_states[] = {VK_DYNAMIC_STATE_VIEWPORT, VK_DYNAMIC_STATE_SCISSOR};
VkPipelineDynamicStateCreateInfo dynamic{};
dynamic.sType = VK_STRUCTURE_TYPE_PIPELINE_DYNAMIC_STATE_CREATE_INFO;
dynamic.dynamicStateCount = 2U;
dynamic.pDynamicStates = dynamic_states;
VkGraphicsPipelineCreateInfo pipeline_info{};
pipeline_info.sType = VK_STRUCTURE_TYPE_GRAPHICS_PIPELINE_CREATE_INFO;
pipeline_info.stageCount = 2U;
pipeline_info.pStages = stages;
pipeline_info.pVertexInputState = &vertex_input;
pipeline_info.pInputAssemblyState = &assembly;
pipeline_info.pViewportState = &viewport;
pipeline_info.pRasterizationState = &raster;
pipeline_info.pMultisampleState = &multisample;
pipeline_info.pDepthStencilState = &depth_stencil;
pipeline_info.pColorBlendState = &blend;
pipeline_info.pDynamicState = &dynamic;
pipeline_info.layout = object_pipeline_layout_;
pipeline_info.renderPass = render_pass_;
pipeline_info.subpass = 0U;
const auto created = detail::check(vkCreateGraphicsPipelines(device_, VK_NULL_HANDLE, 1U, &pipeline_info, nullptr, &object_pipeline_),
"vkCreateGraphicsPipelines(object)");
vkDestroyShaderModule(device_, vertex, nullptr);
vkDestroyShaderModule(device_, fragment, nullptr);
return created;
}
[[nodiscard]] auto create_object_descriptors() -> bool {
if (object_texture_.view == VK_NULL_HANDLE) return false;
VkDescriptorSetAllocateInfo set_info{};
set_info.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_ALLOCATE_INFO;
set_info.descriptorPool = descriptor_pool_;
set_info.descriptorSetCount = 1U;
set_info.pSetLayouts = &descriptor_layout_;
if (!detail::check(vkAllocateDescriptorSets(device_, &set_info, &object_set_), "vkAllocateDescriptorSets(object)")) return false;
VkDescriptorImageInfo info{};
info.imageLayout = VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL;
info.imageView = object_texture_.view;
info.sampler = object_texture_.sampler;
VkWriteDescriptorSet write{};
write.sType = VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET;
write.dstSet = object_set_;
write.dstBinding = 0U;
write.descriptorCount = 1U;
write.descriptorType = VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER;
write.pImageInfo = &info;
vkUpdateDescriptorSets(device_, 1U, &write, 0U, nullptr);
return true;
}
[[nodiscard]] auto draw_terrain(const ra3::terrain::gpu_terrain &terrain, const ra3::render::camera3d &camera, float time_s, float aspect) -> bool { [[nodiscard]] auto draw_terrain(const ra3::terrain::gpu_terrain &terrain, const ra3::render::camera3d &camera, float time_s, float aspect) -> bool {
vkWaitForFences(device_, 1U, &in_flight_[current_frame_], VK_TRUE, UINT64_MAX); vkWaitForFences(device_, 1U, &in_flight_[current_frame_], VK_TRUE, UINT64_MAX);
@@ -754,13 +973,16 @@ export namespace ra3::vulkan {
VkClearValue clear{}; VkClearValue clear{};
clear.color = {{0.45F, 0.55F, 0.70F, 1.0F}}; clear.color = {{0.45F, 0.55F, 0.70F, 1.0F}};
VkClearValue depth_clear{};
depth_clear.depthStencil = {1.0F, 0U};
const VkClearValue clears[2] = {clear, depth_clear};
VkRenderPassBeginInfo render_pass_begin{}; VkRenderPassBeginInfo render_pass_begin{};
render_pass_begin.sType = VK_STRUCTURE_TYPE_RENDER_PASS_BEGIN_INFO; render_pass_begin.sType = VK_STRUCTURE_TYPE_RENDER_PASS_BEGIN_INFO;
render_pass_begin.renderPass = render_pass_; render_pass_begin.renderPass = render_pass_;
render_pass_begin.framebuffer = framebuffers_[image_index]; render_pass_begin.framebuffer = framebuffers_[image_index];
render_pass_begin.renderArea = {{0U, 0U}, swapchain_extent_}; render_pass_begin.renderArea = {{0U, 0U}, swapchain_extent_};
render_pass_begin.clearValueCount = 1U; render_pass_begin.clearValueCount = 2U;
render_pass_begin.pClearValues = &clear; render_pass_begin.pClearValues = clears;
vkCmdBeginRenderPass(cmd, &render_pass_begin, VK_SUBPASS_CONTENTS_INLINE); vkCmdBeginRenderPass(cmd, &render_pass_begin, VK_SUBPASS_CONTENTS_INLINE);
VkViewport viewport{0.0F, 0.0F, static_cast<float>(swapchain_extent_.width), static_cast<float>(swapchain_extent_.height), 0.0F, 1.0F}; VkViewport viewport{0.0F, 0.0F, static_cast<float>(swapchain_extent_.width), static_cast<float>(swapchain_extent_.height), 0.0F, 1.0F};
@@ -793,6 +1015,49 @@ export namespace ra3::vulkan {
vkCmdPushConstants(cmd, terrain_pipeline_layout_, VK_SHADER_STAGE_FRAGMENT_BIT, 0U, sizeof(push), push); vkCmdPushConstants(cmd, terrain_pipeline_layout_, VK_SHADER_STAGE_FRAGMENT_BIT, 0U, sizeof(push), push);
vkCmdDraw(cmd, 3U, 1U, 0U, 0U); vkCmdDraw(cmd, 3U, 1U, 0U, 0U);
// Static-map models (buildings/props) depth-test against the terrain.
if (objects_ready_ && object_index_count_ > 0U) {
const auto z_scale = terrain.z_scale;
const auto world_w = static_cast<float>(terrain.width) * 10.0F;
const auto world_h = static_cast<float>(terrain.height) * 10.0F;
const auto sample = [&](float wx, float wy) -> float {
if (wx < 0.0F || wy < 0.0F || wx >= world_w || wy >= world_h) return -1.0e9F;
const auto cx = std::min(terrain.width - 1U, static_cast<uint32_t>(wx / 10.0F));
const auto cy = std::min(terrain.height - 1U, static_cast<uint32_t>((world_h - wy) / 10.0F));
return static_cast<float>(terrain.heights[static_cast<size_t>(cy) * terrain.width + cx]) * z_scale;
};
float target_z = sample(camera.target_x, camera.target_y);
if (target_z < -1.0e8F) target_z = 0.0F;
const auto pitch = std::clamp(camera.pitch, 0.15F, 1.45F);
const auto fov = std::clamp(camera.fov, 0.3F, 1.4F);
const auto cp = std::cos(pitch);
const float fwd[3] = {cp * std::sin(camera.yaw), cp * std::cos(camera.yaw), -std::sin(pitch)};
float right[3] = {fwd[1], -fwd[0], 0.0F};
const auto rl = std::sqrt(right[0] * right[0] + right[1] * right[1]);
right[0] /= rl;
right[1] /= rl;
float up[3] = {right[1] * fwd[2], -right[0] * fwd[2], right[0] * fwd[1] - right[1] * fwd[0]};
const auto ul = std::sqrt(up[0] * up[0] + up[1] * up[1] + up[2] * up[2]);
up[0] /= ul;
up[1] /= ul;
up[2] /= ul;
const auto dist = camera.height / std::sin(pitch);
const float cam[3] = {camera.target_x - fwd[0] * dist, camera.target_y - fwd[1] * dist, target_z + camera.height - fwd[2] * dist};
const auto th = std::tan(fov * 0.5F);
const float obj_push[20] = {cam[0], cam[1], cam[2], 0.0F,
fwd[0], fwd[1], fwd[2], 0.0F,
right[0], right[1], right[2], th,
up[0], up[1], up[2], th * aspect,
0.45F, 0.35F, 0.82F, 0.38F};
vkCmdBindPipeline(cmd, VK_PIPELINE_BIND_POINT_GRAPHICS, object_pipeline_);
vkCmdBindDescriptorSets(cmd, VK_PIPELINE_BIND_POINT_GRAPHICS, object_pipeline_layout_, 0U, 1U, &object_set_, 0U, nullptr);
vkCmdPushConstants(cmd, object_pipeline_layout_, VK_SHADER_STAGE_VERTEX_BIT | VK_SHADER_STAGE_FRAGMENT_BIT, 0U, sizeof(obj_push), obj_push);
const VkDeviceSize offset = 0U;
vkCmdBindVertexBuffers(cmd, 0U, 1U, &object_vertex_buffer_, &offset);
vkCmdBindIndexBuffer(cmd, object_index_buffer_, 0U, VK_INDEX_TYPE_UINT32);
vkCmdDrawIndexed(cmd, object_index_count_, 1U, 0U, 0, 0U);
}
// Overlays (top-left FPS label, bottom-right minimap) use the image // Overlays (top-left FPS label, bottom-right minimap) use the image
// pipeline with alpha blending, drawn into the same render pass. // pipeline with alpha blending, drawn into the same render pass.
const auto draw_overlay = [&](const gpu_image &overlay, VkDescriptorSet set, uint32_t w, uint32_t h, float x, float y) { const auto draw_overlay = [&](const gpu_image &overlay, VkDescriptorSet set, uint32_t w, uint32_t h, float x, float y) {
@@ -861,6 +1126,27 @@ export namespace ra3::vulkan {
terrain_set_ = VK_NULL_HANDLE; terrain_set_ = VK_NULL_HANDLE;
overlay_label_set_ = VK_NULL_HANDLE; overlay_label_set_ = VK_NULL_HANDLE;
overlay_minimap_set_ = VK_NULL_HANDLE; overlay_minimap_set_ = VK_NULL_HANDLE;
if (object_vertex_buffer_ != VK_NULL_HANDLE) vkDestroyBuffer(device_, object_vertex_buffer_, nullptr);
if (object_index_buffer_ != VK_NULL_HANDLE) vkDestroyBuffer(device_, object_index_buffer_, nullptr);
if (object_vertex_memory_ != VK_NULL_HANDLE) vkFreeMemory(device_, object_vertex_memory_, nullptr);
if (object_index_memory_ != VK_NULL_HANDLE) vkFreeMemory(device_, object_index_memory_, nullptr);
if (object_texture_.sampler != VK_NULL_HANDLE) vkDestroySampler(device_, object_texture_.sampler, nullptr);
if (object_texture_.view != VK_NULL_HANDLE) vkDestroyImageView(device_, object_texture_.view, nullptr);
if (object_texture_.image != VK_NULL_HANDLE) vkDestroyImage(device_, object_texture_.image, nullptr);
if (object_texture_.memory != VK_NULL_HANDLE) vkFreeMemory(device_, object_texture_.memory, nullptr);
if (object_pipeline_ != VK_NULL_HANDLE) vkDestroyPipeline(device_, object_pipeline_, nullptr);
if (object_pipeline_layout_ != VK_NULL_HANDLE) vkDestroyPipelineLayout(device_, object_pipeline_layout_, nullptr);
object_vertex_buffer_ = VK_NULL_HANDLE;
object_index_buffer_ = VK_NULL_HANDLE;
object_vertex_memory_ = VK_NULL_HANDLE;
object_index_memory_ = VK_NULL_HANDLE;
object_texture_ = {};
object_pipeline_ = VK_NULL_HANDLE;
object_pipeline_layout_ = VK_NULL_HANDLE;
object_set_ = VK_NULL_HANDLE;
object_index_count_ = 0U;
objects_ready_ = false;
} }
@@ -1009,23 +1295,78 @@ export namespace ra3::vulkan {
} }
framebuffers_.resize(actual); framebuffers_.resize(actual);
for (uint32_t i = 0; i < actual; ++i) { return true;
VkImageView attachments[] = {swapchain_views_[i]}; }
/** Create one color+depth framebuffer per swapchain image (needs `render_pass_`). */
[[nodiscard]] auto create_framebuffers() -> bool {
this->create_depth_resources();
for (size_t i = 0; i < framebuffers_.size(); ++i) {
VkImageView attachments[] = {swapchain_views_[i], depth_views_[i]};
VkFramebufferCreateInfo framebuffer_info{}; VkFramebufferCreateInfo framebuffer_info{};
framebuffer_info.sType = VK_STRUCTURE_TYPE_FRAMEBUFFER_CREATE_INFO; framebuffer_info.sType = VK_STRUCTURE_TYPE_FRAMEBUFFER_CREATE_INFO;
framebuffer_info.renderPass = render_pass_; framebuffer_info.renderPass = render_pass_;
framebuffer_info.attachmentCount = 1U; framebuffer_info.attachmentCount = 2U;
framebuffer_info.pAttachments = attachments; framebuffer_info.pAttachments = attachments;
framebuffer_info.width = extent.width; framebuffer_info.width = swapchain_extent_.width;
framebuffer_info.height = extent.height; framebuffer_info.height = swapchain_extent_.height;
framebuffer_info.layers = 1U; framebuffer_info.layers = 1U;
if (!detail::check(vkCreateFramebuffer(device_, &framebuffer_info, nullptr, &framebuffers_[i]), "vkCreateFramebuffer")) return false; if (!detail::check(vkCreateFramebuffer(device_, &framebuffer_info, nullptr, &framebuffers_[i]), "vkCreateFramebuffer")) return false;
} }
return true; return true;
} }
/** Allocate a depth image + view per swapchain image (static-map models depth-test against the terrain). */
auto create_depth_resources() -> void {
this->destroy_depth_resources();
depth_images_.resize(swapchain_images_.size());
depth_memories_.resize(swapchain_images_.size());
depth_views_.resize(swapchain_images_.size());
for (size_t i = 0; i < swapchain_images_.size(); ++i) {
VkImageCreateInfo image_info{};
image_info.sType = VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO;
image_info.imageType = VK_IMAGE_TYPE_2D;
image_info.extent = {swapchain_extent_.width, swapchain_extent_.height, 1U};
image_info.mipLevels = 1U;
image_info.arrayLayers = 1U;
image_info.format = VK_FORMAT_D32_SFLOAT;
image_info.tiling = VK_IMAGE_TILING_OPTIMAL;
image_info.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED;
image_info.usage = VK_IMAGE_USAGE_DEPTH_STENCIL_ATTACHMENT_BIT;
image_info.samples = VK_SAMPLE_COUNT_1_BIT;
image_info.sharingMode = VK_SHARING_MODE_EXCLUSIVE;
if (vkCreateImage(device_, &image_info, nullptr, &depth_images_[i]) != VK_SUCCESS) return;
VkMemoryRequirements requirements{};
vkGetImageMemoryRequirements(device_, depth_images_[i], &requirements);
VkMemoryAllocateInfo allocate{};
allocate.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO;
allocate.allocationSize = requirements.size;
allocate.memoryTypeIndex = this->memory_type(requirements.memoryTypeBits, VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT);
if (vkAllocateMemory(device_, &allocate, nullptr, &depth_memories_[i]) != VK_SUCCESS) return;
vkBindImageMemory(device_, depth_images_[i], depth_memories_[i], 0);
VkImageViewCreateInfo view_info{};
view_info.sType = VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO;
view_info.image = depth_images_[i];
view_info.viewType = VK_IMAGE_VIEW_TYPE_2D;
view_info.format = VK_FORMAT_D32_SFLOAT;
view_info.subresourceRange = {VK_IMAGE_ASPECT_DEPTH_BIT, 0U, 1U, 0U, 1U};
if (vkCreateImageView(device_, &view_info, nullptr, &depth_views_[i]) != VK_SUCCESS) return;
}
}
auto destroy_depth_resources() -> void {
if (device_ == VK_NULL_HANDLE) return;
for (const auto view: depth_views_) vkDestroyImageView(device_, view, nullptr);
for (const auto image: depth_images_) vkDestroyImage(device_, image, nullptr);
for (const auto memory: depth_memories_) vkFreeMemory(device_, memory, nullptr);
depth_views_.clear();
depth_images_.clear();
depth_memories_.clear();
}
auto cleanup_swapchain() -> void { auto cleanup_swapchain() -> void {
for (const auto framebuffer: framebuffers_) vkDestroyFramebuffer(device_, framebuffer, nullptr); for (const auto framebuffer: framebuffers_) vkDestroyFramebuffer(device_, framebuffer, nullptr);
this->destroy_depth_resources();
for (const auto view: swapchain_views_) vkDestroyImageView(device_, view, nullptr); for (const auto view: swapchain_views_) vkDestroyImageView(device_, view, nullptr);
framebuffers_.clear(); framebuffers_.clear();
swapchain_views_.clear(); swapchain_views_.clear();
@@ -1041,7 +1382,7 @@ export namespace ra3::vulkan {
if (width == 0 || height == 0) return false; if (width == 0 || height == 0) return false;
vkDeviceWaitIdle(device_); vkDeviceWaitIdle(device_);
this->cleanup_swapchain(); this->cleanup_swapchain();
return this->create_swapchain(); return this->create_swapchain() && this->create_framebuffers();
} }
// ---- pipeline -------------------------------------------------------- // ---- pipeline --------------------------------------------------------
@@ -1058,22 +1399,36 @@ export namespace ra3::vulkan {
color.finalLayout = VK_IMAGE_LAYOUT_PRESENT_SRC_KHR; color.finalLayout = VK_IMAGE_LAYOUT_PRESENT_SRC_KHR;
VkAttachmentReference color_ref{0U, VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL}; VkAttachmentReference color_ref{0U, VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL};
VkAttachmentDescription depth{};
depth.format = VK_FORMAT_D32_SFLOAT;
depth.samples = VK_SAMPLE_COUNT_1_BIT;
depth.loadOp = VK_ATTACHMENT_LOAD_OP_CLEAR;
depth.storeOp = VK_ATTACHMENT_STORE_OP_DONT_CARE;
depth.stencilLoadOp = VK_ATTACHMENT_LOAD_OP_DONT_CARE;
depth.stencilStoreOp = VK_ATTACHMENT_STORE_OP_DONT_CARE;
depth.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED;
depth.finalLayout = VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL;
VkAttachmentReference depth_ref{1U, VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL};
VkSubpassDescription subpass{}; VkSubpassDescription subpass{};
subpass.pipelineBindPoint = VK_PIPELINE_BIND_POINT_GRAPHICS; subpass.pipelineBindPoint = VK_PIPELINE_BIND_POINT_GRAPHICS;
subpass.colorAttachmentCount = 1U; subpass.colorAttachmentCount = 1U;
subpass.pColorAttachments = &color_ref; subpass.pColorAttachments = &color_ref;
subpass.pDepthStencilAttachment = &depth_ref;
VkSubpassDependency dependency{}; VkSubpassDependency dependency{};
dependency.srcSubpass = VK_SUBPASS_EXTERNAL; dependency.srcSubpass = VK_SUBPASS_EXTERNAL;
dependency.dstSubpass = 0U; dependency.dstSubpass = 0U;
dependency.srcStageMask = VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT; dependency.srcStageMask = VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT | VK_PIPELINE_STAGE_EARLY_FRAGMENT_TESTS_BIT;
dependency.dstStageMask = VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT; dependency.dstStageMask = VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT | VK_PIPELINE_STAGE_EARLY_FRAGMENT_TESTS_BIT;
dependency.dstAccessMask = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT; dependency.dstAccessMask = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT | VK_ACCESS_DEPTH_STENCIL_ATTACHMENT_WRITE_BIT;
const VkAttachmentDescription attachments[] = {color, depth};
VkRenderPassCreateInfo render_pass_info{}; VkRenderPassCreateInfo render_pass_info{};
render_pass_info.sType = VK_STRUCTURE_TYPE_RENDER_PASS_CREATE_INFO; render_pass_info.sType = VK_STRUCTURE_TYPE_RENDER_PASS_CREATE_INFO;
render_pass_info.attachmentCount = 1U; render_pass_info.attachmentCount = 2U;
render_pass_info.pAttachments = &color; render_pass_info.pAttachments = attachments;
render_pass_info.subpassCount = 1U; render_pass_info.subpassCount = 1U;
render_pass_info.pSubpasses = &subpass; render_pass_info.pSubpasses = &subpass;
render_pass_info.dependencyCount = 1U; render_pass_info.dependencyCount = 1U;
@@ -1211,10 +1566,10 @@ export namespace ra3::vulkan {
command_buffers_.resize(frames_in_flight); command_buffers_.resize(frames_in_flight);
if (!detail::check(vkAllocateCommandBuffers(device_, &allocate, command_buffers_.data()), "vkAllocateCommandBuffers")) return false; if (!detail::check(vkAllocateCommandBuffers(device_, &allocate, command_buffers_.data()), "vkAllocateCommandBuffers")) return false;
VkDescriptorPoolSize pool_size{VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER, 4U}; VkDescriptorPoolSize pool_size{VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER, 8U};
VkDescriptorPoolCreateInfo descriptor_pool_info{}; VkDescriptorPoolCreateInfo descriptor_pool_info{};
descriptor_pool_info.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_POOL_CREATE_INFO; descriptor_pool_info.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_POOL_CREATE_INFO;
descriptor_pool_info.maxSets = 4U; descriptor_pool_info.maxSets = 8U;
descriptor_pool_info.poolSizeCount = 1U; descriptor_pool_info.poolSizeCount = 1U;
descriptor_pool_info.pPoolSizes = &pool_size; descriptor_pool_info.pPoolSizes = &pool_size;
if (!detail::check(vkCreateDescriptorPool(device_, &descriptor_pool_info, nullptr, &descriptor_pool_), "vkCreateDescriptorPool")) return false; if (!detail::check(vkCreateDescriptorPool(device_, &descriptor_pool_info, nullptr, &descriptor_pool_), "vkCreateDescriptorPool")) return false;
@@ -1307,13 +1662,16 @@ export namespace ra3::vulkan {
VkClearValue clear{}; VkClearValue clear{};
clear.color = {{0.05F, 0.06F, 0.08F, 1.0F}}; clear.color = {{0.05F, 0.06F, 0.08F, 1.0F}};
VkClearValue depth_clear{};
depth_clear.depthStencil = {1.0F, 0U};
const VkClearValue clears[2] = {clear, depth_clear};
VkRenderPassBeginInfo render_pass_begin{}; VkRenderPassBeginInfo render_pass_begin{};
render_pass_begin.sType = VK_STRUCTURE_TYPE_RENDER_PASS_BEGIN_INFO; render_pass_begin.sType = VK_STRUCTURE_TYPE_RENDER_PASS_BEGIN_INFO;
render_pass_begin.renderPass = render_pass_; render_pass_begin.renderPass = render_pass_;
render_pass_begin.framebuffer = framebuffers_[image_index]; render_pass_begin.framebuffer = framebuffers_[image_index];
render_pass_begin.renderArea = {{0U, 0U}, swapchain_extent_}; render_pass_begin.renderArea = {{0U, 0U}, swapchain_extent_};
render_pass_begin.clearValueCount = 1U; render_pass_begin.clearValueCount = 2U;
render_pass_begin.pClearValues = &clear; render_pass_begin.pClearValues = clears;
vkCmdBeginRenderPass(cmd, &render_pass_begin, VK_SUBPASS_CONTENTS_INLINE); vkCmdBeginRenderPass(cmd, &render_pass_begin, VK_SUBPASS_CONTENTS_INLINE);
VkViewport viewport{0.0F, 0.0F, static_cast<float>(swapchain_extent_.width), static_cast<float>(swapchain_extent_.height), 0.0F, 1.0F}; VkViewport viewport{0.0F, 0.0F, static_cast<float>(swapchain_extent_.width), static_cast<float>(swapchain_extent_.height), 0.0F, 1.0F};
@@ -1433,6 +1791,9 @@ export namespace ra3::vulkan {
std::vector<VkImage> swapchain_images_; std::vector<VkImage> swapchain_images_;
std::vector<VkImageView> swapchain_views_; std::vector<VkImageView> swapchain_views_;
std::vector<VkFramebuffer> framebuffers_; std::vector<VkFramebuffer> framebuffers_;
std::vector<VkImage> depth_images_;
std::vector<VkDeviceMemory> depth_memories_;
std::vector<VkImageView> depth_views_;
VkRenderPass render_pass_ = VK_NULL_HANDLE; VkRenderPass render_pass_ = VK_NULL_HANDLE;
VkDescriptorSetLayout descriptor_layout_ = VK_NULL_HANDLE; VkDescriptorSetLayout descriptor_layout_ = VK_NULL_HANDLE;
VkPipelineLayout pipeline_layout_ = VK_NULL_HANDLE; VkPipelineLayout pipeline_layout_ = VK_NULL_HANDLE;
@@ -1464,6 +1825,16 @@ export namespace ra3::vulkan {
VkPipeline terrain_pipeline_ = VK_NULL_HANDLE; VkPipeline terrain_pipeline_ = VK_NULL_HANDLE;
VkDescriptorPool terrain_pool_ = VK_NULL_HANDLE; VkDescriptorPool terrain_pool_ = VK_NULL_HANDLE;
VkDescriptorSet terrain_set_ = VK_NULL_HANDLE; VkDescriptorSet terrain_set_ = VK_NULL_HANDLE;
VkBuffer object_vertex_buffer_ = VK_NULL_HANDLE;
VkDeviceMemory object_vertex_memory_ = VK_NULL_HANDLE;
VkBuffer object_index_buffer_ = VK_NULL_HANDLE;
VkDeviceMemory object_index_memory_ = VK_NULL_HANDLE;
uint32_t object_index_count_ = 0U;
gpu_image object_texture_;
VkPipelineLayout object_pipeline_layout_ = VK_NULL_HANDLE;
VkPipeline object_pipeline_ = VK_NULL_HANDLE;
VkDescriptorSet object_set_ = VK_NULL_HANDLE;
bool objects_ready_ = false;
std::vector<VkSemaphore> image_available_; std::vector<VkSemaphore> image_available_;
std::vector<VkSemaphore> render_finished_; std::vector<VkSemaphore> render_finished_;
std::vector<VkFence> in_flight_; std::vector<VkFence> in_flight_;
+661
View File
@@ -0,0 +1,661 @@
module;
// Emscripten/GLES3 headers in the global module fragment (C headers). No SDL:
// this backend runs the engine on a Web Worker and renders into an
// OffscreenCanvas transferred from the page, so it must not touch the
// main-thread-only SDL Emscripten port.
#include <GLES3/gl3.h>
#include <emscripten/em_asm.h>
#include <emscripten/emscripten.h>
#include <emscripten/html5.h>
// Sized normalized texture formats used by the terrain textures; core in
// WebGL2/GLES3 but missing from some GLES3 headers.
#ifndef GL_R16
#define GL_R16 0x822A
#endif
#ifndef GL_RGBA16
#define GL_RGBA16 0x805B
#endif
#include "wasmgl_glsl_embedded.hpp"
export module ra3.wasmgl;
import std;
export import ra3.core;
export import ra3.render;
export import ra3.terrain;
export import ra3.client;
import ender.log;
/**
* An SDL-free WebGL 2 presentation backend for the wasm build.
*
* The whole runtime runs on a plain Web Worker (started by
* `apps/web/openra3.worker.js`), so `main()` lives there and its virtual
* filesystem is local to the worker: `FS.createLazyFile` (synchronous XHR) is
* legal and the browser fetches each asset only when the engine opens it.
* Note that Emscripten's `PROXY_TO_PTHREAD` cannot be used for this: in a
* pthread every filesystem syscall is proxied to the main browser thread, so
* the engine would read the main thread's FS, not the worker's lazy one.
*
* The page transfers its `#canvas` to the worker as an OffscreenCanvas; this
* module creates the WebGL2 context on it with `emscripten_webgl_create_context`
* and draws the same 2D menu / GPU terrain as the other backends. Input arrives
* over the worker message channel (the page owns the DOM listeners) into the
* `openra3_*` C functions below, which queue `ra3::render::ui_event`s.
*
* It reuses the same GL pipeline/texture code as the desktop GPU backends; only
* the windowing/context/event layer differs.
*/
export namespace ra3::wasmgl {
class wasmgl_display;
/**
* Receives the input events the worker forwards from the page. Each wasm
* backend (WebGL here, WebGPU in `ra3.webgpu`) implements it and registers
* itself on init, so the `openra3_*` C exports live in exactly one module.
*/
class event_sink {
public:
virtual ~event_sink() = default;
virtual void on_key(std::string_view code, bool down) = 0;
virtual void on_text(int codepoint) = 0;
virtual void on_mouse_button(double x, double y, bool left) = 0;
virtual void on_mouse_move(double x, double y, double dx, double dy, bool left) = 0;
virtual void on_wheel(double delta_y) = 0;
virtual void on_resize(int width, int height) = 0;
virtual void on_quit() = 0;
};
namespace detail {
/** The backend that currently receives worker input. */
[[nodiscard]] inline auto sink_slot() -> event_sink *& {
static event_sink *sink = nullptr;
return sink;
}
} // namespace detail
/** Register (or, with null, clear) the backend that receives input events. */
inline auto set_event_sink(event_sink *sink) -> void { detail::sink_slot() = sink; }
namespace detail {
/** Movement keys tracked as "held" for panning (physical codes). */
enum class move_key : std::uint8_t { w, s, a, d, arrow_up, arrow_down, arrow_left, arrow_right, count };
[[nodiscard]] inline auto move_key_index(std::string_view code) -> int {
if (code == "KeyW") return static_cast<int>(move_key::w);
if (code == "KeyS") return static_cast<int>(move_key::s);
if (code == "KeyA") return static_cast<int>(move_key::a);
if (code == "KeyD") return static_cast<int>(move_key::d);
if (code == "ArrowUp") return static_cast<int>(move_key::arrow_up);
if (code == "ArrowDown") return static_cast<int>(move_key::arrow_down);
if (code == "ArrowLeft") return static_cast<int>(move_key::arrow_left);
if (code == "ArrowRight") return static_cast<int>(move_key::arrow_right);
return -1;
}
/** The `ui_key` for an action key, or `none`. */
[[nodiscard]] inline auto key_from_code(std::string_view code) -> ra3::render::ui_key {
using ra3::render::ui_key;
if (code == "ArrowUp") return ui_key::up;
if (code == "ArrowDown") return ui_key::down;
if (code == "ArrowLeft") return ui_key::left;
if (code == "ArrowRight") return ui_key::right;
if (code == "PageUp") return ui_key::page_up;
if (code == "PageDown") return ui_key::page_down;
if (code == "Enter" || code == "NumpadEnter") return ui_key::confirm;
if (code == "Escape") return ui_key::cancel;
if (code == "Tab") return ui_key::tab;
if (code == "Backspace") return ui_key::backspace;
return ui_key::none;
}
/** The single live display (there is only ever one). */
[[nodiscard]] inline auto active_slot() -> wasmgl_display *& {
static wasmgl_display *instance = nullptr;
return instance;
}
/** Camera + terrain uniforms (5 x vec4), matching the other backends. */
[[nodiscard]] inline auto terrain_uniforms(const ra3::terrain::gpu_terrain &terrain, const ra3::render::camera3d &camera, float time_s,
float aspect) -> std::array<float, 20> {
return {camera.target_x,
camera.target_y,
camera.yaw,
camera.height,
camera.pitch,
camera.fov,
terrain.water_z,
terrain.has_water ? 1.0F : 0.0F,
0.45F,
0.35F,
0.82F,
0.38F,
static_cast<float>(terrain.width),
static_cast<float>(terrain.height),
0.0F,
terrain.z_scale,
time_s,
0.0F,
terrain.cell_span,
aspect};
}
/**
* Read `assets.manifest.json` and register every listed file as an
* Emscripten lazy file, so the browser fetches a file only when the
* engine first opens it. Runs on the worker, where synchronous XHR
* (used by `FS.createLazyFile`) is allowed. Returns the number of files
* registered, or -1 on failure. A missing manifest is not an error (the
* caller may have preloaded a full asset tree instead).
*/
[[nodiscard]] inline auto mount_lazy_assets(const std::string &manifest_path, const std::string &assets_root, const std::string &url_prefix)
-> int {
std::ifstream in(manifest_path, std::ios::binary);
if (!in) return 0;
const std::string json{std::istreambuf_iterator<char>(in), std::istreambuf_iterator<char>()};
if (json.empty()) return 0;
return EM_ASM_INT(
{
try {
var manifest = JSON.parse(UTF8ToString($0));
var root = UTF8ToString($1);
var prefix = UTF8ToString($2);
var files = manifest.files || [];
var n = 0;
for (var i = 0; i < files.length; ++i) {
var rel = files[i];
var full = root + '/' + rel;
var slash = full.lastIndexOf('/');
var dir = full.substring(0, slash);
var name = full.substring(slash + 1);
FS.mkdirTree(dir);
FS.createLazyFile(dir, name, prefix + rel, true, false);
++n;
}
console.log('ra3.wasmgl: mounted ' + n + ' lazy assets under ' + root);
return n;
} catch (e) {
console.error('ra3.wasmgl: lazy asset mount failed: ' + e);
return -1;
}
},
json.c_str(), assets_root.c_str(), url_prefix.c_str());
}
} // namespace detail
/**
* Mount the wasm asset tree lazily (call once, before any asset access).
* On non-wasm builds this does nothing.
*/
inline auto mount_assets(const std::string &manifest_path = "/assets.manifest.json", const std::string &assets_root = "/assets",
const std::string &url_prefix = "assets/") -> int {
#if defined(__EMSCRIPTEN__)
return detail::mount_lazy_assets(manifest_path, assets_root, url_prefix);
#else
(void) manifest_path;
(void) assets_root;
(void) url_prefix;
return 0;
#endif
}
class wasmgl_display final : public ra3::client::display, public event_sink {
public:
wasmgl_display() = default;
wasmgl_display(const wasmgl_display &) = delete;
auto operator=(const wasmgl_display &) -> wasmgl_display & = delete;
~wasmgl_display() override { this->shutdown(); }
[[nodiscard]] auto init(const ra3::client::display_options &options) -> bool override {
this->fps_limit_ = options.fps_limit;
// There is no DOM in the worker, so Emscripten's canvas lookup
// (`findEventTarget`, a CSS selector) cannot find #canvas. Register
// the transferred OffscreenCanvas (Module.canvas) as a special target
// so `emscripten_webgl_create_context`/`..._element_size` resolve it.
EM_ASM({
if (typeof specialHTMLTargets !== 'undefined' && Module['canvas']) {
specialHTMLTargets['#canvas'] = Module['canvas'];
}
});
EmscriptenWebGLContextAttributes attributes;
emscripten_webgl_init_context_attributes(&attributes);
attributes.majorVersion = 2;
attributes.minorVersion = 0;
attributes.alpha = false;
attributes.depth = true;
attributes.stencil = false;
attributes.antialias = false;
attributes.premultipliedAlpha = false;
attributes.preserveDrawingBuffer = false;
attributes.powerPreference = EM_WEBGL_POWER_PREFERENCE_HIGH_PERFORMANCE;
attributes.proxyContextToMainThread = EMSCRIPTEN_WEBGL_CONTEXT_PROXY_DISALLOW;
context_ = emscripten_webgl_create_context("#canvas", &attributes);
if (context_ == 0) {
ender::log::error("ra3.wasmgl: emscripten_webgl_create_context failed (is #canvas an OffscreenCanvas in this worker?)");
return false;
}
if (emscripten_webgl_make_context_current(context_) != EMSCRIPTEN_RESULT_SUCCESS) {
ender::log::error("ra3.wasmgl: emscripten_webgl_make_context_current failed");
this->shutdown();
return false;
}
// The page sized the canvas before transfer; fall back to the
// requested size if it is missing.
const auto [width, height] = this->window_size();
if (width <= 0 || height <= 0) {
emscripten_set_canvas_element_size("#canvas", options.width, options.height);
}
if (!this->create_scene_pipeline()) {
this->shutdown();
return false;
}
glGenVertexArrays(1, &vao_);
glBindVertexArray(vao_);
detail::active_slot() = this;
detail::sink_slot() = this;
ender::log::info(std::format("ra3.wasmgl: WebGL worker backend initialized ({})",
reinterpret_cast<const char *>(glGetString(GL_VERSION))));
return true;
}
[[nodiscard]] auto present(const ra3::render::image &frame, const ra3::render::view_rect &dest, bool changed) -> bool override {
if (frame.empty()) return false;
if (changed || scene_tex_ == 0U || frame.width() != scene_w_ || frame.height() != scene_h_) {
this->upload_scene(frame);
}
const auto [window_w, window_h] = this->window_size();
if (window_w <= 0 || window_h <= 0) return true;
glViewport(0, 0, window_w, window_h);
glDisable(GL_DEPTH_TEST);
glEnable(GL_BLEND);
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
glClearColor(0.05F, 0.06F, 0.08F, 1.0F);
glClear(GL_COLOR_BUFFER_BIT);
this->draw_fullscreen(scene_program_, scene_rect_loc_, scene_tex_, dest, window_w, window_h);
this->mark_frame();
return true;
}
[[nodiscard]] auto poll_event(ra3::render::ui_event &out) -> bool override {
if (events_.empty()) return false;
out = events_.front();
events_.pop_front();
return true;
}
[[nodiscard]] auto window_size() const -> std::pair<int, int> override {
int w = 0;
int h = 0;
if (emscripten_get_canvas_element_size("#canvas", &w, &h) != EMSCRIPTEN_RESULT_SUCCESS) return {0, 0};
return {w, h};
}
[[nodiscard]] auto key_down(ra3::render::ui_key key) const -> bool override {
const auto held = [this](detail::move_key which) { return held_[static_cast<std::size_t>(which)]; };
switch (key) {
case ra3::render::ui_key::up: return held(detail::move_key::w) || held(detail::move_key::arrow_up);
case ra3::render::ui_key::down: return held(detail::move_key::s) || held(detail::move_key::arrow_down);
case ra3::render::ui_key::left: return held(detail::move_key::a) || held(detail::move_key::arrow_left);
case ra3::render::ui_key::right: return held(detail::move_key::d) || held(detail::move_key::arrow_right);
default: return false;
}
}
[[nodiscard]] auto supports_terrain() const -> bool override { return true; }
[[nodiscard]] auto present_terrain(const ra3::terrain::gpu_terrain &terrain, const ra3::render::camera3d &camera, float time_s,
const ra3::client::terrain_overlay &overlay) -> bool override {
if (!terrain_ready_) {
if (!this->create_terrain_pipeline()) return false;
if (!this->create_terrain_textures(terrain)) return false;
terrain_ready_ = true;
ender::log::info(std::format("ra3.wasmgl: terrain ready ({}x{}, {} layers)", terrain.width, terrain.height, terrain.layer_count));
}
if (overlay.label_changed) this->upload_overlay(label_tex_, overlay.label);
if (overlay.minimap_changed) this->upload_overlay(minimap_tex_, overlay.minimap);
const auto [window_w, window_h] = this->window_size();
if (window_w <= 0 || window_h <= 0) return true;
const auto aspect = static_cast<float>(window_w) / static_cast<float>(std::max(1, window_h));
glViewport(0, 0, window_w, window_h);
glDisable(GL_DEPTH_TEST);
glEnable(GL_BLEND);
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
glClearColor(0.45F, 0.55F, 0.70F, 1.0F);
glClear(GL_COLOR_BUFFER_BIT);
glBindVertexArray(vao_);
glUseProgram(terrain_program_);
const auto uniforms = detail::terrain_uniforms(terrain, camera, time_s, aspect);
glUniform4fv(terrain_cam_loc_, 5, uniforms.data());
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, height_tex_);
glActiveTexture(GL_TEXTURE1);
glBindTexture(GL_TEXTURE_2D, cell_tex_);
glActiveTexture(GL_TEXTURE2);
glBindTexture(GL_TEXTURE_2D_ARRAY, atlas_tex_);
glUniform1i(terrain_height_loc_, 0);
glUniform1i(terrain_cell_loc_, 1);
glUniform1i(terrain_atlas_loc_, 2);
glDrawArrays(GL_TRIANGLES, 0, 3);
// Overlays (FPS label, corner minimap) use the image program.
const float margin = 12.0F;
this->draw_overlay(label_tex_, overlay.label.width(), overlay.label.height(), margin, margin, window_w, window_h);
this->draw_overlay(minimap_tex_, overlay.minimap.width(), overlay.minimap.height(),
static_cast<float>(window_w) - static_cast<float>(overlay.minimap.width()) - margin, margin, window_w, window_h);
this->mark_frame();
return true;
}
auto shutdown() -> void override {
if (detail::active_slot() == this) detail::active_slot() = nullptr;
if (detail::sink_slot() == this) detail::sink_slot() = nullptr;
if (context_ != 0) {
this->destroy_textures();
if (scene_program_ != 0U) glDeleteProgram(scene_program_);
if (terrain_program_ != 0U) glDeleteProgram(terrain_program_);
if (vao_ != 0U) glDeleteVertexArrays(1, &vao_);
emscripten_webgl_destroy_context(context_);
}
context_ = 0;
vao_ = 0U;
scene_program_ = terrain_program_ = 0U;
scene_rect_loc_ = terrain_cam_loc_ = terrain_height_loc_ = terrain_cell_loc_ = terrain_atlas_loc_ = -1;
terrain_ready_ = false;
scene_w_ = scene_h_ = 0U;
events_.clear();
held_.fill(false);
}
[[nodiscard]] auto name() const -> std::string_view override { return "wasmgl"; }
// ---- worker message entry points (see apps/web/openra3.worker.js) ----
/** A key down/up; `code` is a DOM `KeyboardEvent.code`. */
auto on_key(std::string_view code, bool down) -> void override {
if (const int index = detail::move_key_index(code); index >= 0) {
held_[static_cast<std::size_t>(index)] = down;
}
if (!down) return;
if (const auto key = detail::key_from_code(code); key != ra3::render::ui_key::none) {
ra3::render::ui_event out;
out.type = ra3::render::ui_event_type::key;
out.key = key;
events_.push_back(out);
}
}
/** A printable character (DOM `KeyboardEvent.charCode`). */
auto on_text(int codepoint) -> void override {
if (codepoint <= 0 || codepoint > 0x7F) return;
ra3::render::ui_event out;
out.type = ra3::render::ui_event_type::text;
out.character = static_cast<char>(codepoint);
events_.push_back(out);
}
auto on_mouse_button(double x, double y, bool left) -> void override {
ra3::render::ui_event out;
out.type = ra3::render::ui_event_type::mouse_button;
out.x = static_cast<float>(x);
out.y = static_cast<float>(y);
out.left = left;
events_.push_back(out);
}
auto on_mouse_move(double x, double y, double dx, double dy, bool left) -> void override {
ra3::render::ui_event out;
out.type = ra3::render::ui_event_type::mouse_move;
out.x = static_cast<float>(x);
out.y = static_cast<float>(y);
out.dx = static_cast<float>(dx);
out.dy = static_cast<float>(dy);
out.left = left;
events_.push_back(out);
}
auto on_wheel(double delta_y) -> void override {
ra3::render::ui_event out;
out.type = ra3::render::ui_event_type::wheel;
out.wheel = static_cast<float>(-delta_y); // browser +Y is scroll-down; SDL +Y is scroll-up.
events_.push_back(out);
}
auto on_quit() -> void override {
ra3::render::ui_event out;
out.type = ra3::render::ui_event_type::quit;
events_.push_back(out);
}
/** Resize the drawing buffer (backing pixels, already DPR-scaled). */
auto on_resize(int width, int height) -> void override {
if (width > 0 && height > 0) emscripten_set_canvas_element_size("#canvas", width, height);
}
private:
// ---- GL helpers ------------------------------------------------------
[[nodiscard]] auto compile(unsigned type, const char *source, std::string_view what) -> GLuint {
const GLuint shader = glCreateShader(type);
glShaderSource(shader, 1, &source, nullptr);
glCompileShader(shader);
GLint ok = GL_FALSE;
glGetShaderiv(shader, GL_COMPILE_STATUS, &ok);
if (ok == GL_FALSE) {
std::array<char, 1024> log{};
glGetShaderInfoLog(shader, static_cast<GLsizei>(log.size()), nullptr, log.data());
ender::log::error(std::format("ra3.wasmgl: {} shader compile failed: {}", what, log.data()));
glDeleteShader(shader);
return 0U;
}
return shader;
}
[[nodiscard]] auto link(const char *vertex, const char *fragment, std::string_view what) -> GLuint {
const GLuint vs = this->compile(GL_VERTEX_SHADER, vertex, what);
const GLuint fs = this->compile(GL_FRAGMENT_SHADER, fragment, what);
if (vs == 0U || fs == 0U) {
if (vs != 0U) glDeleteShader(vs);
if (fs != 0U) glDeleteShader(fs);
return 0U;
}
const GLuint program = glCreateProgram();
glAttachShader(program, vs);
glAttachShader(program, fs);
glLinkProgram(program);
glDeleteShader(vs);
glDeleteShader(fs);
GLint ok = GL_FALSE;
glGetProgramiv(program, GL_LINK_STATUS, &ok);
if (ok == GL_FALSE) {
std::array<char, 1024> log{};
glGetProgramInfoLog(program, static_cast<GLsizei>(log.size()), nullptr, log.data());
ender::log::error(std::format("ra3.wasmgl: {} program link failed: {}", what, log.data()));
glDeleteProgram(program);
return 0U;
}
return program;
}
[[nodiscard]] auto create_scene_pipeline() -> bool {
scene_program_ = this->link(ra3_shaders::webgl_scene_vert_glsl, ra3_shaders::webgl_scene_frag_glsl, "scene");
if (scene_program_ == 0U) return false;
scene_rect_loc_ = glGetUniformLocation(scene_program_, "u_rect");
glUseProgram(scene_program_);
glUniform1i(glGetUniformLocation(scene_program_, "u_scene"), 0);
return true;
}
[[nodiscard]] auto create_terrain_pipeline() -> bool {
terrain_program_ = this->link(ra3_shaders::webgl_terrain_vert_glsl, ra3_shaders::webgl_terrain_frag_glsl, "terrain");
if (terrain_program_ == 0U) return false;
terrain_cam_loc_ = glGetUniformLocation(terrain_program_, "u_data");
terrain_height_loc_ = glGetUniformLocation(terrain_program_, "u_heightmap");
terrain_cell_loc_ = glGetUniformLocation(terrain_program_, "u_celldata");
terrain_atlas_loc_ = glGetUniformLocation(terrain_program_, "u_atlas");
return true;
}
/** (Re)create the 2D texture used by the scene/overlay path. */
auto make_texture2d(GLuint &texture, GLint internal, GLenum format, GLenum type, int w, int h, const void *data, bool linear,
bool wrap) -> void {
if (texture == 0U) glGenTextures(1, &texture);
glBindTexture(GL_TEXTURE_2D, texture);
glPixelStorei(GL_UNPACK_ALIGNMENT, 1);
glTexImage2D(GL_TEXTURE_2D, 0, internal, w, h, 0, format, type, data);
if (const GLenum error = glGetError(); error != GL_NO_ERROR) {
ender::log::error(std::format("ra3.wasmgl: texImage2D failed: internal=0x{:X} format=0x{:X} type=0x{:X} size={}x{} err=0x{:X}",
static_cast<unsigned>(internal), static_cast<unsigned>(format), static_cast<unsigned>(type), w, h,
static_cast<unsigned>(error)));
}
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, linear ? GL_LINEAR : GL_NEAREST);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, linear ? GL_LINEAR : GL_NEAREST);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, wrap ? GL_REPEAT : GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, wrap ? GL_REPEAT : GL_CLAMP_TO_EDGE);
}
auto upload_scene(const ra3::render::image &frame) -> void {
// The image is 0xAARRGGBB (BGRA in memory); store it as RGBA and let
// the shader swizzle `.bgra`.
this->make_texture2d(scene_tex_, GL_RGBA8, GL_RGBA, GL_UNSIGNED_BYTE, static_cast<int>(frame.width()),
static_cast<int>(frame.height()), frame.data(), true, false);
scene_w_ = frame.width();
scene_h_ = frame.height();
}
auto upload_overlay(GLuint &texture, const ra3::render::image &source) -> void {
if (source.empty()) return;
this->make_texture2d(texture, GL_RGBA8, GL_RGBA, GL_UNSIGNED_BYTE, static_cast<int>(source.width()),
static_cast<int>(source.height()), source.data(), true, false);
}
[[nodiscard]] auto create_terrain_textures(const ra3::terrain::gpu_terrain &terrain) -> bool {
this->make_texture2d(height_tex_, GL_R16, GL_RED, GL_UNSIGNED_SHORT, static_cast<int>(terrain.width),
static_cast<int>(terrain.height), terrain.heights.data(), false, false);
this->make_texture2d(cell_tex_, GL_RGBA16, GL_RGBA, GL_UNSIGNED_SHORT, static_cast<int>(terrain.width),
static_cast<int>(terrain.height), terrain.cell_data.data(), false, false);
if (atlas_tex_ == 0U) glGenTextures(1, &atlas_tex_);
glBindTexture(GL_TEXTURE_2D_ARRAY, atlas_tex_);
glPixelStorei(GL_UNPACK_ALIGNMENT, 1);
glTexImage3D(GL_TEXTURE_2D_ARRAY, 0, GL_RGBA8, static_cast<GLsizei>(terrain.layer_size), static_cast<GLsizei>(terrain.layer_size),
static_cast<GLsizei>(terrain.layer_count), 0, GL_RGBA, GL_UNSIGNED_BYTE, terrain.layers.data());
glTexParameteri(GL_TEXTURE_2D_ARRAY, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D_ARRAY, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D_ARRAY, GL_TEXTURE_WRAP_S, GL_REPEAT);
glTexParameteri(GL_TEXTURE_2D_ARRAY, GL_TEXTURE_WRAP_T, GL_REPEAT);
return true;
}
auto destroy_textures() -> void {
for (GLuint *texture: {&scene_tex_, &height_tex_, &cell_tex_, &atlas_tex_, &label_tex_, &minimap_tex_}) {
if (*texture != 0U) glDeleteTextures(1, texture);
*texture = 0U;
}
}
/** Publish a coarse frame counter to the page for the fps probe. */
auto mark_frame() const -> void {
++frames_;
if (frames_ % 15U == 0U) {
EM_ASM({ self.postMessage({ openra3Frames: $0 }); }, static_cast<int>(frames_));
}
}
auto draw_fullscreen(GLuint program, GLint rect_location, GLuint texture, const ra3::render::view_rect &dest, int window_w, int window_h)
-> void {
glUseProgram(program);
glBindVertexArray(vao_);
const float rect[4] = {dest.x / static_cast<float>(window_w), dest.y / static_cast<float>(window_h),
dest.w / static_cast<float>(window_w), dest.h / static_cast<float>(window_h)};
glUniform4fv(rect_location, 1, rect);
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, texture);
glDrawArrays(GL_TRIANGLES, 0, 3);
}
auto draw_overlay(GLuint texture, core::uint32 w, core::uint32 h, float x, float y, int window_w, int window_h) -> void {
if (texture == 0U || w == 0U || h == 0U) return;
const ra3::render::view_rect rect{x, y, static_cast<float>(w), static_cast<float>(h)};
this->draw_fullscreen(scene_program_, scene_rect_loc_, texture, rect, window_w, window_h);
}
EMSCRIPTEN_WEBGL_CONTEXT_HANDLE context_ = 0;
GLuint vao_ = 0U;
GLuint scene_program_ = 0U;
GLint scene_rect_loc_ = -1;
GLuint terrain_program_ = 0U;
GLint terrain_cam_loc_ = -1;
GLint terrain_height_loc_ = -1;
GLint terrain_cell_loc_ = -1;
GLint terrain_atlas_loc_ = -1;
GLuint scene_tex_ = 0U;
core::uint32 scene_w_ = 0U;
core::uint32 scene_h_ = 0U;
bool terrain_ready_ = false;
GLuint height_tex_ = 0U;
GLuint cell_tex_ = 0U;
GLuint atlas_tex_ = 0U;
GLuint label_tex_ = 0U;
GLuint minimap_tex_ = 0U;
std::deque<ra3::render::ui_event> events_;
std::array<bool, static_cast<std::size_t>(detail::move_key::count)> held_{};
mutable unsigned frames_ = 0U;
};
// ---- C API called by apps/web/openra3.worker.js --------------------------
/** Key down/up: `code` is a DOM KeyboardEvent.code. */
inline auto post_key(const char *code, int down) -> void {
if (auto *sink = detail::sink_slot(); sink != nullptr) sink->on_key(code != nullptr ? code : "", down != 0);
}
/** Printable character (DOM charCode). */
inline auto post_text(int codepoint) -> void {
if (auto *sink = detail::sink_slot(); sink != nullptr) sink->on_text(codepoint);
}
inline auto post_mouse_button(double x, double y, int left) -> void {
if (auto *sink = detail::sink_slot(); sink != nullptr) sink->on_mouse_button(x, y, left != 0);
}
inline auto post_mouse_move(double x, double y, double dx, double dy, int left) -> void {
if (auto *sink = detail::sink_slot(); sink != nullptr) sink->on_mouse_move(x, y, dx, dy, left != 0);
}
inline auto post_wheel(double delta_y) -> void {
if (auto *sink = detail::sink_slot(); sink != nullptr) sink->on_wheel(delta_y);
}
inline auto post_resize(int width, int height) -> void {
if (auto *sink = detail::sink_slot(); sink != nullptr) sink->on_resize(width, height);
}
inline auto post_quit() -> void {
if (auto *sink = detail::sink_slot(); sink != nullptr) sink->on_quit();
}
} // namespace ra3::wasmgl
extern "C" {
EMSCRIPTEN_KEEPALIVE inline auto openra3_key(const char *code, int down) -> void { ra3::wasmgl::post_key(code, down); }
EMSCRIPTEN_KEEPALIVE inline auto openra3_text(int codepoint) -> void { ra3::wasmgl::post_text(codepoint); }
EMSCRIPTEN_KEEPALIVE inline auto openra3_mouse_button(double x, double y, int left) -> void { ra3::wasmgl::post_mouse_button(x, y, left); }
EMSCRIPTEN_KEEPALIVE inline auto openra3_mouse_move(double x, double y, double dx, double dy, int left) -> void {
ra3::wasmgl::post_mouse_move(x, y, dx, dy, left);
}
EMSCRIPTEN_KEEPALIVE inline auto openra3_wheel(double delta_y) -> void { ra3::wasmgl::post_wheel(delta_y); }
EMSCRIPTEN_KEEPALIVE inline auto openra3_resize(int width, int height) -> void { ra3::wasmgl::post_resize(width, height); }
EMSCRIPTEN_KEEPALIVE inline auto openra3_quit() -> void { ra3::wasmgl::post_quit(); }
}
+32
View File
@@ -0,0 +1,32 @@
export module ra3.wasmgl;
import std;
export import ra3.core;
import ra3.render;
import ra3.client;
/**
* Fallback wasm-worker WebGL backend for builds without Emscripten. `init` fails
* so the caller can select another display; the class name matches the real
* `ra3.wasmgl` module so the backend factory in `ra3.display` compiles
* unchanged.
*/
export namespace ra3::wasmgl {
/** No-op off Emscripten (there is no Emscripten virtual filesystem). */
inline auto mount_assets(const std::string & = "/assets.manifest.json", const std::string & = "/assets",
const std::string & = "assets/") -> int {
return 0;
}
class wasmgl_display final : public ra3::client::display {
public:
[[nodiscard]] auto init(const ra3::client::display_options &) -> bool override { return false; }
[[nodiscard]] auto present(const ra3::render::image &, const ra3::render::view_rect &, bool) -> bool override { return false; }
[[nodiscard]] auto poll_event(ra3::render::ui_event &) -> bool override { return false; }
[[nodiscard]] auto window_size() const -> std::pair<int, int> override { return {0, 0}; }
[[nodiscard]] auto key_down(ra3::render::ui_key) const -> bool override { return false; }
auto shutdown() -> void override {}
[[nodiscard]] auto name() const -> std::string_view override { return "wasmgl(null)"; }
};
}
+610
View File
@@ -0,0 +1,610 @@
module;
// Dawn's header-only C++ wrapper for the WebGPU C API, provided by Emscripten's
// emdawnwebgpu port (`--use-port=emdawnwebgpu`). This backend is wasm-only: on
// the web WebGPU accepts WGSL only (SPIR-V is not available), so the shaders are
// WGSL ports of the Vulkan/WebGL ones rather than a reuse of the SPIR-V blobs.
#include <webgpu/webgpu_cpp.h>
#include <emscripten/em_asm.h>
#include <emscripten/emscripten.h>
#include <emscripten/html5.h>
#include "webgpu_wgsl_embedded.hpp"
export module ra3.webgpu;
import std;
export import ra3.core;
export import ra3.render;
export import ra3.terrain;
export import ra3.client;
import ra3.wasmgl;
import ender.log;
/**
* A WebGPU presentation backend for the wasm build, a peer of `ra3.wasmgl`.
*
* It runs in the same plain Web Worker and draws into the same transferred
* OffscreenCanvas, but through WebGPU: a surface is created on `#canvas`, a
* swapchain image is acquired each frame and presented. The uniform layout and
* the terrain data plumbing are shared with the Vulkan/WebGL backends
* (`terrain_uniforms`); only the API calls and the WGSL shaders differ.
*
* It reuses `ra3.wasmgl`'s `event_sink` so the page/worker input protocol is the
* same for both wasm backends.
*/
export namespace ra3::webgpu {
class webgpu_display final : public ra3::client::display, public ra3::wasmgl::event_sink {
public:
webgpu_display() = default;
webgpu_display(const webgpu_display &) = delete;
auto operator=(const webgpu_display &) -> webgpu_display & = delete;
~webgpu_display() override { this->shutdown(); }
[[nodiscard]] auto init(const ra3::client::display_options &options) -> bool override {
this->fps_limit_ = options.fps_limit;
wgpu::Device device(emscripten_webgpu_get_device());
if (device == nullptr) {
ender::log::warn("ra3.webgpu: no preinitialized WebGPU device");
return false;
}
device_ = std::move(device);
queue_ = device_.GetQueue();
instance_ = wgpu::CreateInstance();
if (instance_ == nullptr) {
ender::log::warn("ra3.webgpu: wgpuCreateInstance failed");
return false;
}
wgpu::EmscriptenSurfaceSourceCanvasHTMLSelector selector;
selector.selector = "#canvas";
// There is no DOM in the worker, so Emscripten's canvas lookup cannot
// find #canvas by CSS selector; register the transferred OffscreenCanvas
// (Module.canvas) as a special target for `wgpuInstanceCreateSurface`.
EM_ASM({
if (typeof specialHTMLTargets !== 'undefined' && Module['canvas']) {
specialHTMLTargets['#canvas'] = Module['canvas'];
}
});
wgpu::SurfaceDescriptor surface_descriptor;
surface_descriptor.nextInChain = &selector;
surface_ = instance_.CreateSurface(&surface_descriptor);
if (surface_ == nullptr) {
ender::log::warn("ra3.webgpu: CreateSurface failed (is #canvas an OffscreenCanvas?)");
return false;
}
if (!this->create_pipelines()) {
ender::log::warn("ra3.webgpu: pipeline creation failed");
this->shutdown();
return false;
}
const auto [width, height] = this->window_size();
this->configure(width > 0 ? width : options.width, height > 0 ? height : options.height);
ra3::wasmgl::set_event_sink(this);
ender::log::info("ra3.webgpu: WebGPU backend initialized");
return true;
}
[[nodiscard]] auto present(const ra3::render::image &frame, const ra3::render::view_rect &dest, bool changed) -> bool override {
if (frame.empty()) return false;
if (changed || !scene_.texture || static_cast<int>(frame.width()) != scene_.width || static_cast<int>(frame.height()) != scene_.height) {
this->upload_image(scene_, frame);
}
const auto [window_w, window_h] = this->window_size();
if (window_w <= 0 || window_h <= 0) return true;
return this->draw({0.05F, 0.06F, 0.08F, 1.0F}, [&](wgpu::RenderPassEncoder &pass, int w, int h) {
this->draw_image(pass, scene_, dest, w, h);
});
}
[[nodiscard]] auto poll_event(ra3::render::ui_event &out) -> bool override {
if (events_.empty()) return false;
out = events_.front();
events_.pop_front();
return true;
}
[[nodiscard]] auto window_size() const -> std::pair<int, int> override {
int w = 0;
int h = 0;
if (emscripten_get_canvas_element_size("#canvas", &w, &h) != EMSCRIPTEN_RESULT_SUCCESS) return {0, 0};
return {w, h};
}
[[nodiscard]] auto key_down(ra3::render::ui_key key) const -> bool override {
const auto held = [this](ra3::wasmgl::detail::move_key which) { return held_[static_cast<std::size_t>(which)]; };
switch (key) {
case ra3::render::ui_key::up: return held(ra3::wasmgl::detail::move_key::w) || held(ra3::wasmgl::detail::move_key::arrow_up);
case ra3::render::ui_key::down: return held(ra3::wasmgl::detail::move_key::s) || held(ra3::wasmgl::detail::move_key::arrow_down);
case ra3::render::ui_key::left: return held(ra3::wasmgl::detail::move_key::a) || held(ra3::wasmgl::detail::move_key::arrow_left);
case ra3::render::ui_key::right: return held(ra3::wasmgl::detail::move_key::d) || held(ra3::wasmgl::detail::move_key::arrow_right);
default: return false;
}
}
[[nodiscard]] auto supports_terrain() const -> bool override { return true; }
[[nodiscard]] auto present_terrain(const ra3::terrain::gpu_terrain &terrain, const ra3::render::camera3d &camera, float time_s,
const ra3::client::terrain_overlay &overlay) -> bool override {
if (!terrain_ready_) {
if (!this->create_terrain(terrain)) return false;
terrain_ready_ = true;
ender::log::info(std::format("ra3.webgpu: terrain ready ({}x{}, {} layers)", terrain.width, terrain.height, terrain.layer_count));
}
if (overlay.label_changed) this->upload_image(label_, overlay.label);
if (overlay.minimap_changed) this->upload_image(minimap_, overlay.minimap);
const auto [window_w, window_h] = this->window_size();
if (window_w <= 0 || window_h <= 0) return true;
const auto aspect = static_cast<float>(window_w) / static_cast<float>(std::max(1, window_h));
const auto uniforms = detail_uniforms(terrain, camera, time_s, aspect);
queue_.WriteBuffer(terrain_.ubo, 0, uniforms.data(), uniforms.size() * sizeof(float));
const float margin = 12.0F;
return this->draw({0.45F, 0.55F, 0.70F, 1.0F}, [&](wgpu::RenderPassEncoder &pass, int w, int h) {
pass.SetPipeline(terrain_.pipeline);
pass.SetBindGroup(0, terrain_.bind_group);
pass.Draw(3);
this->draw_image(pass, label_, {margin, margin, static_cast<float>(label_.width), static_cast<float>(label_.height)}, w, h);
this->draw_image(pass, minimap_, {static_cast<float>(w) - static_cast<float>(minimap_.width) - margin, margin,
static_cast<float>(minimap_.width), static_cast<float>(minimap_.height)},
w, h);
});
}
auto shutdown() -> void override {
if (ra3::wasmgl::detail::sink_slot() == this) ra3::wasmgl::set_event_sink(nullptr);
terrain_ready_ = false;
scene_ = {};
label_ = {};
minimap_ = {};
terrain_ = {};
surface_ = nullptr;
queue_ = nullptr;
device_ = nullptr;
instance_ = nullptr;
events_.clear();
held_.fill(false);
}
[[nodiscard]] auto name() const -> std::string_view override { return "webgpu"; }
// ---- worker input (ra3.wasmgl::event_sink) ---------------------------
auto on_key(std::string_view code, bool down) -> void override {
if (const int index = ra3::wasmgl::detail::move_key_index(code); index >= 0) {
held_[static_cast<std::size_t>(index)] = down;
}
if (!down) return;
if (const auto key = ra3::wasmgl::detail::key_from_code(code); key != ra3::render::ui_key::none) {
push_key(key);
}
}
auto on_text(int codepoint) -> void override {
if (codepoint <= 0 || codepoint > 0x7F) return;
ra3::render::ui_event out;
out.type = ra3::render::ui_event_type::text;
out.character = static_cast<char>(codepoint);
events_.push_back(out);
}
auto on_mouse_button(double x, double y, bool left) -> void override {
ra3::render::ui_event out;
out.type = ra3::render::ui_event_type::mouse_button;
out.x = static_cast<float>(x);
out.y = static_cast<float>(y);
out.left = left;
events_.push_back(out);
}
auto on_mouse_move(double x, double y, double dx, double dy, bool left) -> void override {
ra3::render::ui_event out;
out.type = ra3::render::ui_event_type::mouse_move;
out.x = static_cast<float>(x);
out.y = static_cast<float>(y);
out.dx = static_cast<float>(dx);
out.dy = static_cast<float>(dy);
out.left = left;
events_.push_back(out);
}
auto on_wheel(double delta_y) -> void override {
ra3::render::ui_event out;
out.type = ra3::render::ui_event_type::wheel;
out.wheel = static_cast<float>(-delta_y);
events_.push_back(out);
}
auto on_quit() -> void override {
ra3::render::ui_event out;
out.type = ra3::render::ui_event_type::quit;
events_.push_back(out);
}
auto on_resize(int width, int height) -> void override {
if (width > 0 && height > 0) emscripten_set_canvas_element_size("#canvas", width, height);
}
private:
/** A drawn image: its texture, uniform rect buffer and bind group. */
struct image_target {
wgpu::Texture texture;
wgpu::TextureView view;
wgpu::Buffer ubo;
wgpu::BindGroup bind_group;
int width = 0;
int height = 0;
int tex_width = 0;
int tex_height = 0;
};
struct terrain_target {
wgpu::RenderPipeline pipeline;
wgpu::Buffer ubo;
wgpu::Texture height;
wgpu::Texture cell;
wgpu::Texture atlas;
wgpu::TextureView height_view;
wgpu::TextureView cell_view;
wgpu::TextureView atlas_view;
wgpu::BindGroup bind_group;
};
static auto detail_uniforms(const ra3::terrain::gpu_terrain &terrain, const ra3::render::camera3d &camera, float time_s, float aspect)
-> std::array<float, 20> {
return {camera.target_x,
camera.target_y,
camera.yaw,
camera.height,
camera.pitch,
camera.fov,
terrain.water_z,
terrain.has_water ? 1.0F : 0.0F,
0.45F,
0.35F,
0.82F,
0.38F,
static_cast<float>(terrain.width),
static_cast<float>(terrain.height),
0.0F,
terrain.z_scale,
time_s,
0.0F,
terrain.cell_span,
aspect};
}
auto push_key(ra3::render::ui_key key) -> void {
ra3::render::ui_event out;
out.type = ra3::render::ui_event_type::key;
out.key = key;
events_.push_back(out);
}
[[nodiscard]] auto make_shader(const char *wgsl) -> wgpu::ShaderModule {
wgpu::ShaderSourceWGSL source;
source.code = wgsl;
wgpu::ShaderModuleDescriptor descriptor;
descriptor.nextInChain = &source;
return device_.CreateShaderModule(&descriptor);
}
[[nodiscard]] auto create_pipelines() -> bool {
scene_shader_ = this->make_shader(ra3_shaders::webgpu_scene_wgsl);
terrain_shader_ = this->make_shader(ra3_shaders::webgpu_terrain_wgsl);
if (scene_shader_ == nullptr || terrain_shader_ == nullptr) {
ender::log::warn(std::format("ra3.webgpu: shader module creation failed (scene={} terrain={})",
scene_shader_ != nullptr, terrain_shader_ != nullptr));
return false;
}
wgpu::SamplerDescriptor clamp_desc;
clamp_desc.addressModeU = wgpu::AddressMode::ClampToEdge;
clamp_desc.addressModeV = wgpu::AddressMode::ClampToEdge;
clamp_desc.magFilter = wgpu::FilterMode::Linear;
clamp_desc.minFilter = wgpu::FilterMode::Linear;
clamp_desc.lodMaxClamp = 32.0F;
clamp_sampler_ = device_.CreateSampler(&clamp_desc);
wgpu::SamplerDescriptor repeat_desc = clamp_desc;
repeat_desc.addressModeU = wgpu::AddressMode::Repeat;
repeat_desc.addressModeV = wgpu::AddressMode::Repeat;
repeat_sampler_ = device_.CreateSampler(&repeat_desc);
// Scene: uniform(rect) + texture + sampler.
wgpu::BindGroupLayoutEntry scene_entries[3] = {};
scene_entries[0].binding = 0;
scene_entries[0].visibility = wgpu::ShaderStage::Vertex | wgpu::ShaderStage::Fragment;
scene_entries[0].buffer.type = wgpu::BufferBindingType::Uniform;
scene_entries[0].buffer.minBindingSize = 16;
scene_entries[1].binding = 1;
scene_entries[1].visibility = wgpu::ShaderStage::Fragment;
scene_entries[1].texture.sampleType = wgpu::TextureSampleType::Float;
scene_entries[1].texture.viewDimension = wgpu::TextureViewDimension::e2D;
scene_entries[2].binding = 2;
scene_entries[2].visibility = wgpu::ShaderStage::Fragment;
scene_entries[2].sampler.type = wgpu::SamplerBindingType::Filtering;
wgpu::BindGroupLayoutDescriptor scene_layout_desc;
scene_layout_desc.entryCount = 3;
scene_layout_desc.entries = scene_entries;
scene_bind_group_layout_ = device_.CreateBindGroupLayout(&scene_layout_desc);
wgpu::PipelineLayoutDescriptor scene_pipeline_layout_desc;
scene_pipeline_layout_desc.bindGroupLayoutCount = 1;
scene_pipeline_layout_desc.bindGroupLayouts = &scene_bind_group_layout_;
scene_pipeline_layout_ = device_.CreatePipelineLayout(&scene_pipeline_layout_desc);
scene_pipeline_ = this->create_pipeline(scene_pipeline_layout_, scene_shader_, ra3_shaders::webgpu_scene_wgsl);
if (scene_pipeline_ == nullptr) {
ender::log::warn("ra3.webgpu: scene render pipeline creation failed");
}
// Terrain: uniform(data[5]) + R16Uint + RGBA16Uint + RGBA8 array + sampler.
wgpu::BindGroupLayoutEntry terrain_entries[5] = {};
terrain_entries[0].binding = 0;
terrain_entries[0].visibility = wgpu::ShaderStage::Fragment;
terrain_entries[0].buffer.type = wgpu::BufferBindingType::Uniform;
terrain_entries[0].buffer.minBindingSize = 80;
terrain_entries[1].binding = 1;
terrain_entries[1].visibility = wgpu::ShaderStage::Fragment;
terrain_entries[1].texture.sampleType = wgpu::TextureSampleType::Uint;
terrain_entries[1].texture.viewDimension = wgpu::TextureViewDimension::e2D;
terrain_entries[2].binding = 2;
terrain_entries[2].visibility = wgpu::ShaderStage::Fragment;
terrain_entries[2].texture.sampleType = wgpu::TextureSampleType::Uint;
terrain_entries[2].texture.viewDimension = wgpu::TextureViewDimension::e2D;
terrain_entries[3].binding = 3;
terrain_entries[3].visibility = wgpu::ShaderStage::Fragment;
terrain_entries[3].texture.sampleType = wgpu::TextureSampleType::Float;
terrain_entries[3].texture.viewDimension = wgpu::TextureViewDimension::e2DArray;
terrain_entries[4].binding = 4;
terrain_entries[4].visibility = wgpu::ShaderStage::Fragment;
terrain_entries[4].sampler.type = wgpu::SamplerBindingType::Filtering;
wgpu::BindGroupLayoutDescriptor terrain_layout_desc;
terrain_layout_desc.entryCount = 5;
terrain_layout_desc.entries = terrain_entries;
terrain_bind_group_layout_ = device_.CreateBindGroupLayout(&terrain_layout_desc);
return scene_pipeline_ != nullptr;
}
[[nodiscard]] auto create_pipeline(const wgpu::PipelineLayout &layout, const wgpu::ShaderModule &shader, const char *wgsl) -> wgpu::RenderPipeline {
(void) wgsl;
wgpu::ColorTargetState target;
target.format = surface_format_;
target.writeMask = wgpu::ColorWriteMask::All;
wgpu::FragmentState fragment;
fragment.module = shader;
fragment.entryPoint = "fs_main";
fragment.targetCount = 1;
fragment.targets = &target;
wgpu::RenderPipelineDescriptor descriptor;
descriptor.layout = layout;
descriptor.vertex.module = shader;
descriptor.vertex.entryPoint = "vs_main";
descriptor.primitive.topology = wgpu::PrimitiveTopology::TriangleList;
descriptor.fragment = &fragment;
descriptor.multisample.count = 1;
return device_.CreateRenderPipeline(&descriptor);
}
auto configure(int width, int height) -> void {
if (surface_ == nullptr || width <= 0 || height <= 0) return;
if (width == surface_w_ && height == surface_h_) return;
wgpu::SurfaceConfiguration config;
config.device = device_;
config.format = surface_format_;
config.usage = wgpu::TextureUsage::RenderAttachment;
config.width = static_cast<uint32_t>(width);
config.height = static_cast<uint32_t>(height);
config.presentMode = wgpu::PresentMode::Fifo;
surface_.Configure(&config);
surface_w_ = width;
surface_h_ = height;
}
auto make_target(image_target &target) -> void {
wgpu::BufferDescriptor ubo_desc;
ubo_desc.size = 16;
ubo_desc.usage = wgpu::BufferUsage::Uniform | wgpu::BufferUsage::CopyDst;
target.ubo = device_.CreateBuffer(&ubo_desc);
}
auto upload_image(image_target &target, const ra3::render::image &frame) -> void {
if (frame.empty()) return;
target.width = static_cast<int>(frame.width());
target.height = static_cast<int>(frame.height());
if (target.texture == nullptr || target.tex_width != target.width || target.tex_height != target.height) {
wgpu::TextureDescriptor descriptor;
descriptor.size = {frame.width(), frame.height(), 1};
descriptor.format = wgpu::TextureFormat::RGBA8Unorm;
descriptor.usage = wgpu::TextureUsage::TextureBinding | wgpu::TextureUsage::CopyDst;
descriptor.dimension = wgpu::TextureDimension::e2D;
descriptor.mipLevelCount = 1;
descriptor.sampleCount = 1;
target.texture = device_.CreateTexture(&descriptor);
target.view = target.texture.CreateView();
target.tex_width = target.width;
target.tex_height = target.height;
if (!target.ubo) this->make_target(target);
target.bind_group = this->make_image_bind_group(target);
}
wgpu::TexelCopyTextureInfo destination;
destination.texture = target.texture;
wgpu::TexelCopyBufferLayout layout;
layout.bytesPerRow = frame.width() * 4;
layout.rowsPerImage = frame.height();
wgpu::Extent3D size = {frame.width(), frame.height(), 1};
queue_.WriteTexture(&destination, frame.data(), static_cast<size_t>(frame.width()) * frame.height() * 4, &layout, &size);
}
[[nodiscard]] auto make_image_bind_group(image_target &target) -> wgpu::BindGroup {
wgpu::BindGroupEntry entries[3] = {};
entries[0].binding = 0;
entries[0].buffer = target.ubo;
entries[0].size = 16;
entries[1].binding = 1;
entries[1].textureView = target.view;
entries[2].binding = 2;
entries[2].sampler = clamp_sampler_;
wgpu::BindGroupDescriptor descriptor;
descriptor.layout = scene_bind_group_layout_;
descriptor.entryCount = 3;
descriptor.entries = entries;
return device_.CreateBindGroup(&descriptor);
}
auto draw_image(wgpu::RenderPassEncoder &pass, image_target &target, const ra3::render::view_rect &dest, int window_w, int window_h) -> void {
if (target.bind_group == nullptr || window_w <= 0 || window_h <= 0) return;
const float rect[4] = {dest.x / static_cast<float>(window_w), dest.y / static_cast<float>(window_h),
dest.w / static_cast<float>(window_w), dest.h / static_cast<float>(window_h)};
queue_.WriteBuffer(target.ubo, 0, rect, sizeof(rect));
pass.SetPipeline(scene_pipeline_);
pass.SetBindGroup(0, target.bind_group);
pass.Draw(3);
}
[[nodiscard]] auto create_terrain(const ra3::terrain::gpu_terrain &terrain) -> bool {
{
wgpu::BufferDescriptor descriptor;
descriptor.size = 80;
descriptor.usage = wgpu::BufferUsage::Uniform | wgpu::BufferUsage::CopyDst;
terrain_.ubo = device_.CreateBuffer(&descriptor);
}
terrain_.height = this->make_texture(wgpu::TextureFormat::R16Uint, terrain.width, terrain.height, 1);
terrain_.cell = this->make_texture(wgpu::TextureFormat::RGBA16Uint, terrain.width, terrain.height, 1);
terrain_.atlas = this->make_texture(wgpu::TextureFormat::RGBA8Unorm, terrain.layer_size, terrain.layer_size, terrain.layer_count);
terrain_.height_view = terrain_.height.CreateView();
terrain_.cell_view = terrain_.cell.CreateView();
terrain_.atlas_view = terrain_.atlas.CreateView();
this->write_texture(terrain_.height, terrain.heights.data(), terrain.width, terrain.height, 1, 2);
this->write_texture(terrain_.cell, terrain.cell_data.data(), terrain.width, terrain.height, 1, 8);
this->write_texture(terrain_.atlas, terrain.layers.data(), terrain.layer_size, terrain.layer_size, terrain.layer_count, 4);
wgpu::BindGroupEntry entries[5] = {};
entries[0].binding = 0;
entries[0].buffer = terrain_.ubo;
entries[0].size = 80;
entries[1].binding = 1;
entries[1].textureView = terrain_.height_view;
entries[2].binding = 2;
entries[2].textureView = terrain_.cell_view;
entries[3].binding = 3;
entries[3].textureView = terrain_.atlas_view;
entries[4].binding = 4;
entries[4].sampler = repeat_sampler_;
wgpu::BindGroupDescriptor bind_desc;
bind_desc.layout = terrain_bind_group_layout_;
bind_desc.entryCount = 5;
bind_desc.entries = entries;
terrain_.bind_group = device_.CreateBindGroup(&bind_desc);
terrain_.pipeline = this->create_pipeline(terrain_pipeline_layout(), terrain_shader_, ra3_shaders::webgpu_terrain_wgsl);
return terrain_.pipeline != nullptr && terrain_.bind_group != nullptr;
}
[[nodiscard]] auto terrain_pipeline_layout() -> wgpu::PipelineLayout {
if (terrain_pipeline_layout_ == nullptr) {
wgpu::PipelineLayoutDescriptor descriptor;
descriptor.bindGroupLayoutCount = 1;
descriptor.bindGroupLayouts = &terrain_bind_group_layout_;
terrain_pipeline_layout_ = device_.CreatePipelineLayout(&descriptor);
}
return terrain_pipeline_layout_;
}
[[nodiscard]] auto make_texture(wgpu::TextureFormat format, uint32_t width, uint32_t height, uint32_t layers) -> wgpu::Texture {
wgpu::TextureDescriptor descriptor;
descriptor.size = {width, height, layers};
descriptor.format = format;
descriptor.usage = wgpu::TextureUsage::TextureBinding | wgpu::TextureUsage::CopyDst;
descriptor.dimension = wgpu::TextureDimension::e2D;
descriptor.mipLevelCount = 1;
descriptor.sampleCount = 1;
return device_.CreateTexture(&descriptor);
}
auto write_texture(const wgpu::Texture &texture, const void *data, uint32_t width, uint32_t height, uint32_t layers, uint32_t bytes_per_pixel)
-> void {
wgpu::TexelCopyTextureInfo destination;
destination.texture = texture;
wgpu::TexelCopyBufferLayout layout;
layout.bytesPerRow = width * bytes_per_pixel;
layout.rowsPerImage = height;
wgpu::Extent3D size = {width, height, layers};
queue_.WriteTexture(&destination, data, static_cast<size_t>(width) * height * layers * bytes_per_pixel, &layout, &size);
}
/** Clear, run `record`, submit and present. */
template<typename Record>
[[nodiscard]] auto draw(std::array<float, 4> clear, Record &&record) -> bool {
const auto [window_w, window_h] = this->window_size();
this->configure(window_w, window_h);
wgpu::SurfaceTexture surface_texture;
surface_.GetCurrentTexture(&surface_texture);
if (surface_texture.status != wgpu::SurfaceGetCurrentTextureStatus::SuccessOptimal &&
surface_texture.status != wgpu::SurfaceGetCurrentTextureStatus::SuccessSuboptimal) {
return false;
}
wgpu::TextureView view = surface_texture.texture.CreateView();
wgpu::RenderPassColorAttachment color;
color.view = view;
color.depthSlice = 0xFFFFFFFFu; // kWGPU_DEPTH_SLICE_UNDEFINED
color.loadOp = wgpu::LoadOp::Clear;
color.storeOp = wgpu::StoreOp::Store;
color.clearValue = {clear[0], clear[1], clear[2], clear[3]};
wgpu::RenderPassDescriptor pass_desc;
pass_desc.colorAttachmentCount = 1;
pass_desc.colorAttachments = &color;
wgpu::CommandEncoder encoder = device_.CreateCommandEncoder();
wgpu::RenderPassEncoder pass = encoder.BeginRenderPass(&pass_desc);
record(pass, window_w, window_h);
pass.End();
wgpu::CommandBuffer commands = encoder.Finish();
queue_.Submit(1, &commands);
// WebGPU presents implicitly at the end of the frame; the port's
// wgpuSurfacePresent is intentionally unsupported.
this->mark_frame();
return true;
}
/** Publish a coarse frame counter to the page for the fps probe. */
auto mark_frame() const -> void {
++frames_;
if (frames_ % 15U == 0U) {
EM_ASM({ self.postMessage({ openra3Frames: $0 }); }, static_cast<int>(frames_));
}
}
wgpu::Instance instance_;
wgpu::Device device_;
wgpu::Queue queue_;
wgpu::Surface surface_;
wgpu::TextureFormat surface_format_ = wgpu::TextureFormat::BGRA8Unorm;
int surface_w_ = 0;
int surface_h_ = 0;
wgpu::ShaderModule scene_shader_;
wgpu::ShaderModule terrain_shader_;
wgpu::BindGroupLayout scene_bind_group_layout_;
wgpu::BindGroupLayout terrain_bind_group_layout_;
wgpu::PipelineLayout scene_pipeline_layout_;
wgpu::PipelineLayout terrain_pipeline_layout_;
wgpu::RenderPipeline scene_pipeline_;
wgpu::Sampler clamp_sampler_;
wgpu::Sampler repeat_sampler_;
image_target scene_;
image_target label_;
image_target minimap_;
terrain_target terrain_;
bool terrain_ready_ = false;
std::deque<ra3::render::ui_event> events_;
std::array<bool, static_cast<std::size_t>(ra3::wasmgl::detail::move_key::count)> held_{};
mutable unsigned frames_ = 0U;
};
}
+26
View File
@@ -0,0 +1,26 @@
export module ra3.webgpu;
import std;
export import ra3.core;
export import ra3.render;
export import ra3.client;
/**
* Fallback WebGPU backend for builds without Emscripten. `init` fails so the
* caller can select another display; the class name matches the real
* `ra3.webgpu` module so the backend factory in `ra3.display` compiles
* unchanged.
*/
export namespace ra3::webgpu {
class webgpu_display final : public ra3::client::display {
public:
[[nodiscard]] auto init(const ra3::client::display_options &) -> bool override { return false; }
[[nodiscard]] auto present(const ra3::render::image &, const ra3::render::view_rect &, bool) -> bool override { return false; }
[[nodiscard]] auto poll_event(ra3::render::ui_event &) -> bool override { return false; }
[[nodiscard]] auto window_size() const -> std::pair<int, int> override { return {0, 0}; }
[[nodiscard]] auto key_down(ra3::render::ui_key) const -> bool override { return false; }
auto shutdown() -> void override {}
[[nodiscard]] auto name() const -> std::string_view override { return "webgpu(null)"; }
};
}
+179
View File
@@ -1,5 +1,6 @@
import std; import std;
import ra3; import ra3;
import ra3.assets;
namespace { namespace {
int failures = 0; int failures = 0;
@@ -193,6 +194,12 @@ auto main() -> int {
} }
check(!space_lit, "a space paints nothing"); check(!space_lit, "a space paints nothing");
// FPS label: the active backend name is appended (e.g. `[vulkan]`).
const auto fps_label = render::compose_fps_label(155U, 0, "vulkan");
const auto fps_plain = render::compose_fps_label(155U, 0);
check(fps_label.width() > fps_plain.width(), "FPS label grows when the backend is shown");
check(fps_label.width() == render::text_width("FPS: 155/vsync [vulkan]") + 8U, "FPS label includes the backend name");
const auto box = render::fit_rect(1280.0F, 720.0F, 1920.0F, 1080.0F); const auto box = render::fit_rect(1280.0F, 720.0F, 1920.0F, 1080.0F);
check(box.x == 0.0F && box.y == 0.0F && box.w == 1920.0F && box.h == 1080.0F, "fit_rect fills a same-aspect window"); check(box.x == 0.0F && box.y == 0.0F && box.w == 1920.0F && box.h == 1080.0F, "fit_rect fills a same-aspect window");
@@ -222,6 +229,178 @@ auto main() -> int {
check(names.lookup("map_mp_2_feasel4") == "Battlebase Beta", "CSF map names decode (byte-XOR 0xFF)"); check(names.lookup("map_mp_2_feasel4") == "Battlebase Beta", "CSF map names decode (byte-XOR 0xFF)");
check(names.lookup("unknown_map") == "unknown_map", "an unknown id falls back to itself"); check(names.lookup("unknown_map") == "unknown_map", "an unknown id falls back to itself");
// Map objects (`ObjectsList`): decode a synthetic one-object chunk.
const auto put16 = [](std::vector<core::uint8> &out, std::uint16_t v) {
out.push_back(static_cast<core::uint8>(v));
out.push_back(static_cast<core::uint8>(v >> 8U));
};
const auto put32 = [](std::vector<core::uint8> &out, std::uint32_t v) {
for (int i = 0; i < 4; ++i) out.push_back(static_cast<core::uint8>(v >> (8 * i)));
};
const auto putf = [&put32](std::vector<core::uint8> &out, float f) {
std::uint32_t bits = 0;
std::memcpy(&bits, &f, sizeof(bits));
put32(out, bits);
};
std::vector<core::uint8> object_data;
putf(object_data, 100.0F);
putf(object_data, 200.0F);
putf(object_data, 0.0F);
putf(object_data, 0.5F);
put32(object_data, 0U); // road type
const std::string object_type = "BB_TEST";
put16(object_data, static_cast<std::uint16_t>(object_type.size()));
object_data.insert(object_data.end(), object_type.begin(), object_type.end());
put16(object_data, 0U); // no properties
std::vector<core::uint8> object_asset;
put32(object_asset, 1U); // asset index
put16(object_asset, 1U); // version
put32(object_asset, static_cast<std::uint32_t>(object_data.size()));
object_asset.insert(object_asset.end(), object_data.begin(), object_data.end());
std::vector<core::uint8> ckmp;
ckmp.insert(ckmp.end(), {'C', 'k', 'M', 'p'});
put32(ckmp, 1U); // one asset name
const std::string chunk_name = "ObjectsList";
ckmp.push_back(static_cast<core::uint8>(chunk_name.size()));
ckmp.insert(ckmp.end(), chunk_name.begin(), chunk_name.end());
put32(ckmp, 1U); // name index
put32(ckmp, 1U); // chunk index
put16(ckmp, 3U); // chunk version
put32(ckmp, static_cast<std::uint32_t>(object_asset.size()));
ckmp.insert(ckmp.end(), object_asset.begin(), object_asset.end());
const auto objects = map::parse_objects(ckmp);
check(objects.size() == 1U, "ObjectsList parses one object");
check(!objects.empty() && objects[0].type == "BB_TEST", "object type-name decodes");
check(!objects.empty() && objects[0].x == 100.0F && objects[0].y == 200.0F && objects[0].angle == 0.5F, "object position/angle decode");
// DDS: a 1x1 uncompressed RGB32 image decodes with correct channels.
std::vector<core::uint8> dds(128U, 0U);
dds[0] = 'D';
dds[1] = 'D';
dds[2] = 'S';
dds[3] = ' ';
const auto put_dds = [&dds](std::size_t off, std::uint32_t v) {
for (int i = 0; i < 4; ++i) dds[off + static_cast<std::size_t>(i)] = static_cast<core::uint8>(v >> (8 * i));
};
put_dds(4U, 124U);
put_dds(8U, 0x1U); // flags: DDSD_CAPS|... (unused)
put_dds(12U, 1U); // height
put_dds(16U, 1U); // width
put_dds(76U, 32U); // pixel format size
put_dds(80U, 0x40U); // DDPF_RGB
put_dds(84U, 0U); // no fourcc
put_dds(88U, 32U); // bits per pixel
put_dds(92U, 0x00FF0000U);
put_dds(96U, 0x0000FF00U);
put_dds(100U, 0x000000FFU);
put_dds(104U, 0xFF000000U);
dds.push_back(0x00U); // B
dds.push_back(0x00U); // G
dds.push_back(0xFFU); // R
dds.push_back(0xFFU); // A
const auto decoded_dds = models::decode_dds(dds);
check(decoded_dds.width() == 1U && decoded_dds.height() == 1U, "DDS dimensions decode");
check(!decoded_dds.empty() && decoded_dds.data()[0] == render::argb(255, 0, 0), "DDS RGB32 channels map to ARGB");
check(models::decode_dds(std::vector<core::uint8>{1U, 2U, 3U}).empty(), "a non-DDS payload yields no image");
// Real-asset checks (opt-in): set OPENRA3_TEST_ASSETS to an extracted assets
// directory. They run the libra3assets-backed readers over the retail data and
// compare the map names against the golden `maps/map_names.tsv` written by the
// previous hand-rolled parser.
if (const char *assets_env = std::getenv("OPENRA3_TEST_ASSETS"); assets_env != nullptr && *assets_env != '\0') {
const std::filesystem::path assets{assets_env};
const auto read_all = [](const std::filesystem::path &path) {
std::ifstream in(path, std::ios::binary);
return std::vector<core::uint8>{std::istreambuf_iterator<char>(in), std::istreambuf_iterator<char>()};
};
const auto golden = assets / "maps" / "map_names.tsv";
// The golden table came from the newest `Lang-English<N>.big`; mirror that.
const auto pick_english_csf = [](const std::filesystem::path &raw) -> std::filesystem::path {
std::filesystem::path best;
int best_version = -1;
std::error_code ec;
for (const auto &entry: std::filesystem::directory_iterator(raw, ec)) {
if (!entry.is_directory()) continue;
const auto name = entry.path().filename().string();
if (name.rfind("Lang-English", 0) != 0) continue;
int version = 0;
try {
version = std::stoi(name.substr(12));
} catch (const std::exception &) {
continue;
}
if (version > best_version) {
best_version = version;
best = entry.path() / "data" / "gamestrings.csf";
}
}
if (!best.empty() && std::filesystem::exists(best)) return best;
return raw / "English" / "data" / "gamestrings.csf";
};
const auto csf_path = pick_english_csf(assets / "raw");
if (std::filesystem::exists(golden) && std::filesystem::exists(csf_path)) {
const auto table = map::parse_map_names(read_all(csf_path));
std::ifstream in(golden, std::ios::binary);
std::string line;
std::size_t matched = 0;
std::size_t mismatched = 0;
while (std::getline(in, line)) {
const auto tab = line.find('\t');
if (tab == std::string::npos) continue;
const auto id = line.substr(0, tab);
const auto expected = line.substr(tab + 1U);
if (table.lookup(id) == expected) {
++matched;
} else {
++mismatched;
if (mismatched <= 5U) std::printf(" name mismatch %s: golden '%s' new '%s'\n", id.c_str(), expected.c_str(), table.lookup(id).c_str());
}
}
std::printf(" real CSF: %zu names, golden %zu matched, %zu mismatched\n", table.names.size(), matched, mismatched);
check(mismatched == 0U, "map names reproduce the golden map_names.tsv");
}
std::size_t maps_ok = 0;
std::size_t maps_bad = 0;
std::size_t library_starts = 0;
std::size_t no_starts = 0;
std::size_t roundtrip_bad = 0;
const auto to_bytes = [](std::span<const core::uint8> data) {
return std::span<const std::byte>{reinterpret_cast<const std::byte *>(data.data()), data.size()};
};
for (const auto &entry: std::filesystem::directory_iterator(assets / "maps")) {
if (!entry.is_regular_file() || entry.path().extension() != ".map") continue;
try {
const auto ckmp = map::to_ckmp(read_all(entry.path()));
const auto parsed = terrain::parse_map(ckmp);
const auto starts = map::starts_from_ckmp(ckmp);
const auto document = ra3::assets::map_document::parse(to_bytes(ckmp));
if (document.player_starts().size() >= 2U) ++library_starts; else ++no_starts;
if (!std::ranges::equal(document.to_ckmp(), to_bytes(ckmp))) ++roundtrip_bad;
if (parsed.width > 0U && parsed.height > 0U && parsed.elevations.size() == static_cast<std::size_t>(parsed.width) * parsed.height && starts.size() >= 2U) {
++maps_ok;
} else {
++maps_bad;
if (maps_bad <= 5U)
std::printf(" map oddity %s: %ux%u starts %zu\n", entry.path().filename().string().c_str(), parsed.width, parsed.height, starts.size());
}
} catch (const std::exception &error) {
++maps_bad;
if (maps_bad <= 5U) std::printf(" map FAIL %s: %s\n", entry.path().filename().string().c_str(), error.what());
}
}
std::printf(" real maps: %zu ok, %zu bad; player_starts resolved %zu, unresolved %zu; round-trip mismatched %zu\n", maps_ok, maps_bad, library_starts,
no_starts, roundtrip_bad);
check(maps_bad == 0U, "every extracted map parses terrain + starts through libra3assets");
check(no_starts == 0U, "libra3assets player_starts() resolves every retail map's start waypoints");
check(roundtrip_bad == 0U, "libra3assets re-serialises every retail map byte-for-byte");
}
if (failures == 0) { if (failures == 0) {
std::puts("ra3_tests: OK"); std::puts("ra3_tests: OK");
} }
+14 -7
View File
@@ -23,8 +23,8 @@ engine, not on a logging framework.
(for tests and in-game consoles) ship; `sink` is a small interface. (for tests and in-game consoles) ship; `sink` is a small interface.
- **File output with archiving.** `file_sink` writes to a file and, on open, - **File output with archiving.** `file_sink` writes to a file and, on open,
moves an existing log aside to a timestamped archive, so a run never appends moves an existing log aside to a timestamped archive, so a run never appends
onto a previous run's log. It can also rotate by size and bound how many onto a previous run's log. It can also rotate by size, bound how many archives
archives are kept. are kept, and format both the record timestamp and the archive names.
- **No stacktrace? No problem.** Where `<stacktrace>` is missing (libc++, and - **No stacktrace? No problem.** Where `<stacktrace>` is missing (libc++, and
therefore every cross target), the module still compiles and records still therefore every cross target), the module still compiles and records still
carry their call site — they simply have no stack. carry their call site — they simply have no stack.
@@ -104,7 +104,7 @@ auto main() -> int {
Example output: Example output:
``` ```
[11:32:18] ERROR example: a body left the world (examples/main.cpp:8) [2026-10-01 11:32:18] ERROR example: a body left the world (examples/main.cpp:8)
#0 simulate_one_step (examples/main.cpp:8) #0 simulate_one_step (examples/main.cpp:8)
#1 main (examples/main.cpp:20) #1 main (examples/main.cpp:20)
#2 <unknown> #2 <unknown>
@@ -131,20 +131,27 @@ and `log::set_sinks({...})` replaces them.
```cpp ```cpp
namespace log = ender::log; namespace log = ender::log;
// Archive any existing enderlog.log to enderlog.log.<timestamp>, then start a // Archive any existing enderlog.log to enderlog.<timestamp>.log, then start a
// fresh file for this run. Rotate at 64 KiB and keep the last 5 archives. // fresh file for this run. Rotate at 64 KiB and keep the last 5 archives.
auto sink = log::add_file_sink("enderlog.log", {.max_file_size = 64 * 1024, .max_archives = 5}); auto sink = log::add_file_sink("enderlog.log", {.max_file_size = 64 * 1024, .max_archives = 5});
``` ```
- **No appending onto a previous run.** On open, an existing non-empty - **No appending onto a previous run.** On open, an existing non-empty
`enderlog.log` is renamed to `enderlog.log.<YYYYmmdd-HHMMSS>` before the new `enderlog.log` is renamed to `enderlog.<timestamp>.log` before the new file is
file is created, so every run gets its own file and the previous run's log is created, so every run gets its own file and the previous run's log is
preserved. A leftover empty file is simply replaced. preserved. The timestamp is inserted before the extension, which stays last
(`.log` when the active file has none). A leftover empty file is simply
replaced.
- `file_options::max_file_size` (0 disables) rotates the active file mid-run the - `file_options::max_file_size` (0 disables) rotates the active file mid-run the
same way, and never archives an empty file. `file_options::max_archives` same way, and never archives an empty file. `file_options::max_archives`
(0 keeps all) deletes the oldest archives beyond the limit. (0 keeps all) deletes the oldest archives beyond the limit.
- `file_options::flush_each_record` (on by default) flushes after every record so - `file_options::flush_each_record` (on by default) flushes after every record so
a crash keeps the tail. a crash keeps the tail.
- `file_options::timestamp_format` (chrono syntax, default `%Y-%m-%d %H:%M:%S`)
controls the timestamp on each record's header line;
`file_options::archive_time_format` (chrono syntax, default `%Y%m%d-%H%M%S`)
controls the timestamp inserted into archive names. A chrono format string must
begin with `%` (e.g. `%Y-%m-%d_%H%M%S`).
- `add_file_sink` adds the sink to the global logger and returns it; `path()` and - `add_file_sink` adds the sink to the global logger and returns it; `path()` and
`archives()` expose what it wrote. The `file_sink` class can also be used `archives()` expose what it wrote. The `file_sink` class can also be used
directly and installed with `set_sinks`. directly and installed with `set_sinks`.
+48 -17
View File
@@ -22,7 +22,7 @@ module;
#define WIN32_LEAN_AND_MEAN #define WIN32_LEAN_AND_MEAN
#define NOMINMAX #define NOMINMAX
#include <windows.h> #include <windows.h>
#elif defined(__unix__) || defined(__APPLE__) #elif (defined(__unix__) || defined(__APPLE__)) && !defined(__EMSCRIPTEN__)
#include <execinfo.h> #include <execinfo.h>
#endif #endif
@@ -54,6 +54,9 @@ export namespace ender::log {
return "?"; return "?";
} }
/** Default timestamp rendered on a record, chrono format syntax. */
inline constexpr std::string_view default_time_format{"%Y-%m-%d %H:%M:%S"};
/** Logger configuration. */ /** Logger configuration. */
struct options { struct options {
/** Records below this level are dropped before anything is built. */ /** Records below this level are dropped before anything is built. */
@@ -90,15 +93,32 @@ export namespace ender::log {
}; };
namespace detail { namespace detail {
/**
* Render a time point with a runtime chrono format string.
*
* `std::format`'s format string is compile-time only, so the spec is
* wrapped in a replacement field and fed to `std::vformat`; a bare spec
* would be read as literal text rather than a chrono conversion.
*/
[[nodiscard]] inline auto format_time(const std::chrono::system_clock::time_point time,
const std::string_view time_format) -> std::string {
const auto moment = std::chrono::floor<std::chrono::seconds>(time);
const auto pattern = std::string{"{:"}.append(time_format).append("}");
return std::vformat(pattern, std::make_format_args(moment));
}
/** /**
* Render one record as a human-readable block: a header line and, when * Render one record as a human-readable block: a header line and, when
* present, the indented stack frames. Shared by the stream sinks. * present, the indented stack frames. Shared by the stream sinks.
*
* @param time_format A chrono format string applied to the record's
* timestamp; defaults to the date and time of day.
*/ */
[[nodiscard]] inline auto format_record(const record &entry) -> std::string { [[nodiscard]] inline auto format_record(const record &entry,
auto text = std::format("[{:%H:%M:%S}] {:<8} {}", const std::string_view time_format = default_time_format)
std::chrono::floor<std::chrono::seconds>(entry.time), -> std::string {
to_string(entry.severity), const auto stamp = format_time(entry.time, time_format);
entry.message); auto text = std::format("[{}] {:<8} {}", stamp, to_string(entry.severity), entry.message);
if (!entry.file.empty()) { if (!entry.file.empty()) {
text += std::format(" ({}:{})", entry.file, entry.line); text += std::format(" ({}:{})", entry.file, entry.line);
} }
@@ -125,15 +145,18 @@ export namespace ender::log {
/** Writes a human-readable line per record to a stream (stderr by default). */ /** Writes a human-readable line per record to a stream (stderr by default). */
class console_sink final: public sink { class console_sink final: public sink {
public: public:
explicit console_sink(std::ostream &stream = std::cerr): stream_(&stream) {} explicit console_sink(std::ostream &stream = std::cerr,
std::string time_format = std::string{default_time_format})
: stream_(&stream), time_format_(std::move(time_format)) {}
auto write(const record &entry) -> void override { auto write(const record &entry) -> void override {
*stream_ << detail::format_record(entry); *stream_ << detail::format_record(entry, time_format_);
stream_->flush(); stream_->flush();
} }
private: private:
std::ostream *stream_; std::ostream *stream_;
std::string time_format_;
}; };
/** Keeps every record in memory; useful for tests and in-game consoles. */ /** Keeps every record in memory; useful for tests and in-game consoles. */
@@ -174,6 +197,10 @@ export namespace ender::log {
std::size_t max_file_size{0}; std::size_t max_file_size{0};
/** Keep at most this many archives, dropping the oldest first; 0 keeps them all. */ /** Keep at most this many archives, dropping the oldest first; 0 keeps them all. */
std::size_t max_archives{0}; std::size_t max_archives{0};
/** Timestamp format used on each record's header line (chrono syntax). */
std::string timestamp_format{std::string{default_time_format}};
/** Chrono format for the timestamp inserted into archive names. */
std::string archive_time_format{"%Y%m%d-%H%M%S"};
}; };
/** /**
@@ -182,7 +209,9 @@ export namespace ender::log {
* `path` is the active file. When the sink opens it and the file already * `path` is the active file. When the sink opens it and the file already
* holds data, that file is renamed to a timestamped archive first, so a run * holds data, that file is renamed to a timestamped archive first, so a run
* never appends onto a previous run's log: every start begins a fresh file * never appends onto a previous run's log: every start begins a fresh file
* and the old one is preserved as `<path>.<YYYYmmdd-HHMMSS>`. The same * and the old one is preserved as `<stem>.<timestamp><extension>` (`.log`
* when the active file has no extension), where the timestamp is rendered by
* `file_options::archive_time_format` (default `<YYYYmmdd-HHMMSS>`). The same
* happens mid-run once the active file passes `file_options::max_file_size`. * happens mid-run once the active file passes `file_options::max_file_size`.
* `file_options::max_archives` bounds how many archives are kept. * `file_options::max_archives` bounds how many archives are kept.
* *
@@ -205,7 +234,7 @@ export namespace ender::log {
} }
auto write(const record &entry) -> void override { auto write(const record &entry) -> void override {
const auto block = detail::format_record(entry); const auto block = detail::format_record(entry, options_.timestamp_format);
// Rotate before writing, but never rotate an empty file: that would // Rotate before writing, but never rotate an empty file: that would
// archive nothing and lose the record that is about to be written. // archive nothing and lose the record that is about to be written.
if (options_.max_file_size > 0 && size_ > 0 && size_ + block.size() > options_.max_file_size) { if (options_.max_file_size > 0 && size_ > 0 && size_ + block.size() > options_.max_file_size) {
@@ -236,15 +265,17 @@ export namespace ender::log {
auto archive_current() -> void { auto archive_current() -> void {
if (stream_.is_open()) stream_.close(); if (stream_.is_open()) stream_.close();
const auto stamp = std::format("{:%Y%m%d-%H%M%S}", const auto stamp = detail::format_time(std::chrono::system_clock::now(), options_.archive_time_format);
std::chrono::floor<std::chrono::seconds>(std::chrono::system_clock::now())); // Keep the extension last: `<stem>.<timestamp><extension>`, falling
auto archive = path_; // back to `.log` when the active file has none.
archive += "." + stamp; const auto stem = path_.stem().string();
const auto extension = path_.has_extension() ? path_.extension().string() : std::string{".log"};
auto archive = path_.parent_path() / (stem + "." + stamp + extension);
// Two rotations can land in the same second; disambiguate with a // Two rotations can land in the same second; disambiguate with a
// counter rather than overwrite the earlier archive. // counter rather than overwrite the earlier archive.
for (auto counter = 1; std::filesystem::exists(archive); ++counter) { for (auto counter = 1; std::filesystem::exists(archive); ++counter) {
archive = path_; archive = path_.parent_path() /
archive += std::format(".{}.{}", stamp, counter); (stem + "." + stamp + "." + std::to_string(counter) + extension);
} }
std::filesystem::rename(path_, archive); std::filesystem::rename(path_, archive);
archives_.push_back(archive); archives_.push_back(archive);
@@ -367,7 +398,7 @@ export namespace ender::log {
text += std::format(" #{:<3}{}\n", index, symbolicate_frame(frames[index])); text += std::format(" #{:<3}{}\n", index, symbolicate_frame(frames[index]));
} }
return text; return text;
#elif defined(__unix__) || defined(__APPLE__) #elif (defined(__unix__) || defined(__APPLE__)) && !defined(__EMSCRIPTEN__)
void *frames[max_frames] = {}; void *frames[max_frames] = {};
const int count = ::backtrace(frames, static_cast<int>(std::min(max_frames, skip + std::max<std::size_t>(depth, 1U)))); const int count = ::backtrace(frames, static_cast<int>(std::min(max_frames, skip + std::max<std::size_t>(depth, 1U))));
char **symbols = ::backtrace_symbols(frames, count); char **symbols = ::backtrace_symbols(frames, count);
+134
View File
@@ -0,0 +1,134 @@
# libra3assets
A dependency-free C++26 library that reads **and writes** the resource files
Red Alert 3 ships, so tooling (map editors in particular) can work with the
retail assets directly. No engine, no game install required to build - only to
feed it data.
It is a sub-project of this repository, a sibling of
[`libra3replay`](../libra3replay/README.md), and follows the same build
conventions (C++26 modules, `import std;`, GCC 16, CMake 4 + Ninja).
## What it handles
| Module | Format | Read | Write |
| ------------------ | ------------------------------------------------------------------- | :--: | :---: |
| `ra3.assets:big` | `BIG4` archives (`Data\*.big`) | yes | yes |
| `ra3.assets:refpack` | EA RefPack codec (`10 FB`) | yes | yes |
| `ra3.assets:binary` | compiled `BinaryAsset` streams (`.bin` + `.manifest`, `cdata`) | yes | - |
| `ra3.assets:csf` | SAGE `.csf` string tables (`gamestrings.csf`) | yes | yes |
| `ra3.assets:map` | SAGE `.map` containers (`CkMp`) incl. `HeightMapData`/objects | yes | yes |
| `ra3.assets:bytes` | bounds-checked little-/big-endian readers and writers | yes | yes |
The **map** module is the centrepiece for a map editor. A `.map` is modelled as
an ordered list of named, versioned `CkMp` chunks; chunks the library does not
type (e.g. `BlendTileData`, `SidesList`) are preserved byte-for-byte and the
asset-name table keeps its original indices, so an edit/ serialise cycle is
lossless. Typed accessors cover the terrain grid (`HeightMapData`), every placed
object (`ObjectsList`, including the `*Waypoints/Waypoint` objects that carry
`Player_N_Start`), `MPPositionList`, `WorldInfo` and `WaypointsList`.
The **binary** module parses the compiled `BinaryAsset` streams that
BinaryAssetBuilder produces (`data\static.bin`, `data\global.bin`, ...): the
manifest index, each asset's instance slice, its relocation/import sidecars and
its `cdata` blob, plus the hash used to name assets.
## Build
```bash
cmake -S libra3assets -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=g++-16
cmake --build build
ctest --test-dir build --output-on-failure
```
The library is a set of C++20/26 modules (`ra3.assets` plus the `:error`,
`:bytes`, `:refpack`, `:big`, `:binary`, `:csf` and `:map` partitions) that
imports the standard library (`import std;`). That needs **CMake 4.0+ with the
Ninja generator** and a compiler whose standard library ships a `std` module -
**GCC 16** in practice. The tests need nothing but the library itself.
> Building with the distro GCC 16: pass `-DCMAKE_CXX_COMPILER=g++-16`.
## Use
```cpp
import std;
import ra3.assets;
using namespace ra3::assets;
// --- a map, for a map editor -------------------------------------------------
auto document = map_document::open("map_mp_2_black1b.map");
if (const auto height = document.height_map()) {
std::println("terrain {}x{}", height->width, height->height);
auto grid = *height; // copy, then edit
grid.set(10, 10, grid.at(10, 10) + 1); // raise a cell
document.set_height_map(std::move(grid)); // re-encode the chunk
}
for (const auto &start: document.player_starts())
std::println("Player {} at ({}, {})", start.index, start.position.x, start.position.y);
for (const auto &object: document.objects())
if (object.type_name == "*Waypoints/Waypoint")
std::println("{} -> {}", object.property("waypointName")->as_ascii(), object.position.y);
// Lossless: unknown chunks and the name table survive the round-trip.
write_file("edited.map", document.serialize(/* compress = */ true));
// --- a BIG4 archive ----------------------------------------------------------
const auto archive = big_archive::open("Data/GlobalStream.big");
std::println("{} entries", archive.size());
for (const auto *entry: archive.find("audio"))
std::println("{} ({} bytes)", entry->name, entry->size);
// --- a compiled BinaryAsset stream ------------------------------------------
const auto stream = binary_stream_from_big(archive, binary_stream::global, /* need_data = */ false);
for (const auto &[type, count]: stream.type_counts())
std::println("{:6} {}", count, type);
```
The API deliberately avoids integer IDs for anything selectable: chunk kinds are
`chunk_kind` (e.g. `chunk_kind::height_map_data`), streams are `binary_stream`
(`binary_stream::global`), property types are `property_type`, and an
`asset_property` carries a `std::variant<bool, std::int32_t, float, std::string,
std::u16string>`.
## Command-line tool
`ra3assets-cli` is built alongside the library:
```
ra3assets-cli big list <archive.big> [match]
ra3assets-cli big extract <archive.big> <out-dir> [match]
ra3assets-cli binary list <path> [type]
ra3assets-cli binary types <path> [static|global|locale|static_l|static_m]
ra3assets-cli binary cat <path> <Type:Instance|#index> <out-file>
ra3assets-cli map info <map-file>
ra3assets-cli map starts <map-file>
ra3assets-cli map repack <map-file> <out-file> [--compress]
ra3assets-cli csf <csf-or-game-dir-or-big> [label]
ra3assets-cli hash <text>...
```
## Notes and limits
- A `BIG4` archive opened from disk keeps only its index in memory and reads
payloads on demand, so enumerating the ~700 MB retail `StaticStream.big` costs
a few megabytes; pass `need_data = false` to `binary_stream_from_big` when you
only need the manifest. A parsed `.map` document is held in full - that is what
its typed accessors edit.
- `set_height_map` keeps the on-disk `HeightMapData` version (RA3 uses 6, i.e.
16-bit elevations). Older 8-bit maps serialise back as 8-bit.
- RefPack compression is a plain greedy LZ77 matcher. It is lossless against the
bundled decoder and produces streams the game's decoder accepts, but it is
slightly less dense than the retail compressor (≈3% on a typical map).
- `BlendTileData` (terrain texture blending) is preserved but not yet
type-modelled; a map editor should treat the chunk as opaque for now. The
planned follow-up covers `BlendTileData` and `SidesList`.
- Only `BIG4` is supported (RA3's format); older `BIGF` archives are not.
The on-disk layouts were reverse engineered from the shipped files and
cross-checked against [`ra3tools`](../ra3tools/), OpenRA3's `ra3.fs`/`ra3.map`
modules and the OpenSAGE re-implementation (`reference/OpenSAGE`).
+65
View File
@@ -0,0 +1,65 @@
/**
* libra3assets - read (and write) Red Alert 3's built-in resource files.
*
* The primary module interface re-exports every partition:
*
* - `:error` - the `asset_error` exception family
* - `:bytes` - bounds-checked little-/big-endian readers and writers
* - `:refpack` - EA's RefPack codec (decompress and compress)
* - `:big` - `BIG4` archives (`Data\*.big`), read and write
* - `:binary` - compiled `BinaryAsset` streams (`*.bin` + `.manifest`)
* - `:csf` - SAGE `.csf` string tables
* - `:map` - SAGE `.map` containers (`CkMp`), read and write
*
* Nothing here reads game data on its own: callers point the library at their
* own installation (`RA3_GAME_DIR` / `C:\Red Alert 3`).
*/
export module ra3.assets;
export import :error;
export import :bytes;
export import :refpack;
export import :big;
export import :binary;
export import :csf;
export import :map;
import std;
export namespace ra3::assets {
/**
* Locate a Red Alert 3 installation.
*
* Resolution order: the explicit argument, then `$RA3_GAME_DIR`, then the
* default `C:\Red Alert 3`. A directory qualifies only if it has a `Data`
* subdirectory.
*/
[[nodiscard]] inline auto find_game_dir(const std::optional<std::filesystem::path> &explicit_dir = std::nullopt) -> std::optional<std::filesystem::path> {
const auto qualifies = [](const std::filesystem::path &candidate) {
std::error_code ec;
return !candidate.empty() && std::filesystem::is_directory(candidate / "Data", ec);
};
if (explicit_dir && qualifies(*explicit_dir)) return explicit_dir;
if (const auto *env = std::getenv("RA3_GAME_DIR"); env != nullptr && *env != '\0') {
const std::filesystem::path candidate{env};
if (qualifies(candidate)) return candidate;
}
const std::filesystem::path default_dir{"C:/Red Alert 3"};
if (qualifies(default_dir)) return default_dir;
return std::nullopt;
}
/** Every `Data\*.big` archive in an installation, sorted by name. */
[[nodiscard]] inline auto list_archives(const std::filesystem::path &game_dir) -> std::vector<std::filesystem::path> {
std::vector<std::filesystem::path> archives;
std::error_code ec;
for (const auto &entry: std::filesystem::directory_iterator(game_dir / "Data", ec)) {
if (entry.is_regular_file() && entry.path().extension() == ".big") archives.push_back(entry.path());
}
std::ranges::sort(archives);
return archives;
}
} // namespace ra3::assets
+274
View File
@@ -0,0 +1,274 @@
/**
* `BIG4` archives - Red Alert 3's on-disk asset container.
*
* Retail RA3 ships its data as `Data\*.big`: a 16-byte header, a variable-
* length entry index and the 64-byte-aligned payloads. The count and the entry
* offsets/sizes are **big-endian** (the header size is little-endian, matching
* the retail files); each payload may itself be RefPack-compressed, which is
* detected from its own magic rather than flagged in the index.
*
* Both directions are implemented so a map editor can repack an archive.
*/
export module ra3.assets:big;
import std;
import :bytes;
import :error;
import :refpack;
export namespace ra3::assets {
/** One file inside a `BIG4` archive. */
struct big_entry {
std::string name;
std::uint32_t offset = 0;
std::uint32_t size = 0;
};
namespace detail {
[[nodiscard]] inline auto align_up(std::uint64_t value, std::uint64_t alignment) -> std::uint64_t {
return (value + alignment - 1U) / alignment * alignment;
}
[[nodiscard]] inline constexpr auto bswap32(std::uint32_t value) -> std::uint32_t {
return ((value & 0x000000FFU) << 24U) | ((value & 0x0000FF00U) << 8U) | ((value & 0x00FF0000U) >> 8U) | ((value & 0xFF000000U) >> 24U);
}
}
/**
* A parsed `BIG4` archive.
*
* Opening from disk parses only the index; payloads are read (and
* RefPack-decompressed) on demand, so a multi-hundred-megabyte archive costs
* a few megabytes to enumerate. An archive built from an in-memory image
* (`from_bytes`) slices its payloads out of that buffer instead.
*/
class big_archive {
public:
/** Read and parse an archive from disk, keeping only the index in memory. */
[[nodiscard]] static auto open(const std::filesystem::path &path) -> big_archive {
big_archive archive;
archive.path_ = path;
std::error_code ec;
archive.file_size_ = static_cast<std::uint64_t>(std::filesystem::file_size(path, ec));
if (ec) throw big_error("cannot stat archive: " + path.string());
std::ifstream in(path, std::ios::binary);
if (!in) throw big_error("cannot open archive: " + path.string());
std::array<std::byte, 16> header{};
in.read(reinterpret_cast<char *>(header.data()), static_cast<std::streamsize>(header.size()));
if (!in || std::string_view{reinterpret_cast<const char *>(header.data()), 4} != "BIG4") throw big_error("not a BIG4 archive: " + path.string());
// The index is a dense variable-length table from offset 16; read a
// window and grow it until every entry parses. Payloads are never
// touched, so the memory cost stays at the index size.
const auto size = archive.file_size_;
std::uint64_t window = std::min<std::uint64_t>(size, 1U << 20U);
std::vector<std::byte> buffer;
for (;;) {
buffer.resize(static_cast<std::size_t>(window));
in.clear();
in.seekg(0);
in.read(reinterpret_cast<char *>(buffer.data()), static_cast<std::streamsize>(window));
const auto got = in.gcount();
buffer.resize(got > 0 ? static_cast<std::size_t>(got) : 0U);
std::vector<big_entry> entries;
if (big_archive::parse_index(buffer, entries)) {
archive.entries_ = std::move(entries);
break;
}
if (window >= size) throw big_error("truncated BIG4 index: " + path.string());
window = std::min<std::uint64_t>(size, window * 2U);
}
archive.reindex();
return archive;
}
/** Parse an in-memory archive image; payloads are sliced from it. */
[[nodiscard]] static auto from_bytes(std::vector<std::byte> bytes) -> big_archive {
big_archive archive;
archive.bytes_ = std::move(bytes);
archive.in_memory_ = true;
if (!big_archive::parse_index(archive.bytes_, archive.entries_)) throw big_error("not a BIG4 archive");
archive.reindex();
return archive;
}
[[nodiscard]] auto path() const -> const std::filesystem::path & { return this->path_; }
[[nodiscard]] auto entries() const -> const std::vector<big_entry> & { return this->entries_; }
[[nodiscard]] auto size() const -> std::size_t { return this->entries_.size(); }
[[nodiscard]] auto contains(std::string_view name) const -> bool { return this->index_.contains(std::string{name}); }
/** The entry with exactly this name, or `nullptr`. */
[[nodiscard]] auto find_exact(std::string_view name) const -> const big_entry * {
const auto it = this->index_.find(std::string{name});
return it == this->index_.end() ? nullptr : &this->entries_[it->second];
}
/** Every entry whose name contains `needle`, in index order. */
[[nodiscard]] auto find(std::string_view needle) const -> std::vector<const big_entry *> {
std::vector<const big_entry *> matches;
for (const auto &entry: this->entries_) {
if (entry.name.find(needle) != std::string::npos) matches.push_back(&entry);
}
return matches;
}
/** Read an entry's payload; RefPack-decompress it when `decompress`. */
[[nodiscard]] auto read(std::string_view name, bool decompress = true) const -> std::vector<std::byte> {
const auto *entry = this->find_exact(name);
if (entry == nullptr) throw big_error("no such entry: " + std::string{name});
return this->read(*entry, decompress);
}
[[nodiscard]] auto read(const big_entry &entry, bool decompress = true) const -> std::vector<std::byte> {
const auto raw = this->stored(entry.offset, entry.size);
return decompress ? maybe_decompress(raw) : raw;
}
/** The first `count` stored bytes of an entry (no decompression). */
[[nodiscard]] auto read_prefix(std::string_view name, std::size_t count) const -> std::vector<std::byte> {
const auto *entry = this->find_exact(name);
if (entry == nullptr) throw big_error("no such entry: " + std::string{name});
return this->stored(entry->offset, static_cast<std::uint32_t>(std::min<std::size_t>(count, entry->size)));
}
private:
big_archive() = default;
/** Parse a `BIG4` index from `data`; false when it is not fully present. */
[[nodiscard]] static auto parse_index(std::span<const std::byte> data, std::vector<big_entry> &out) -> bool {
if (data.size() < 16U) return false;
byte_reader reader{data};
if (reader.read_ascii(4) != "BIG4") return false;
(void) reader.read_u32(); // total file size (little-endian); implied by the buffer
const auto count = reader.read_be_u32();
(void) reader.read_be_u32(); // index size
// Each entry is at least `u32 offset + u32 size + NUL name` (9 bytes),
// so a count that cannot fit bounds both the reserve and the window.
if (count > (data.size() - 16U) / 9U) return false;
out.clear();
out.reserve(count);
for (std::uint32_t i = 0; i < count; ++i) {
if (reader.remaining() < 8U) return false;
big_entry entry;
entry.offset = reader.read_be_u32();
entry.size = reader.read_be_u32();
// The NUL-terminated name may run past the current window.
std::string name;
bool terminated = false;
while (reader.remaining() > 0U) {
const auto ch = reader.read_u8();
if (ch == 0U) {
terminated = true;
break;
}
name.push_back(static_cast<char>(ch));
}
if (!terminated) return false;
entry.name = std::move(name);
out.push_back(std::move(entry));
}
return true;
}
/** Read `count` stored bytes at `offset`, from memory or from disk. */
[[nodiscard]] auto stored(std::uint32_t offset, std::uint32_t count) const -> std::vector<std::byte> {
if (this->in_memory_) {
if (static_cast<std::uint64_t>(offset) + count > this->bytes_.size()) throw big_error("entry extends past end of archive");
const auto *begin = this->bytes_.data() + offset;
return std::vector<std::byte>{begin, begin + count};
}
if (static_cast<std::uint64_t>(offset) + count > this->file_size_) throw big_error("entry extends past end of archive");
std::ifstream in(this->path_, std::ios::binary);
if (!in) throw big_error("cannot open archive: " + this->path_.string());
in.seekg(static_cast<std::streamoff>(offset));
std::vector<std::byte> raw(count);
if (count > 0U) in.read(reinterpret_cast<char *>(raw.data()), static_cast<std::streamsize>(count));
if (!in) throw big_error("short read from archive: " + this->path_.string());
return raw;
}
auto reindex() -> void {
this->index_.clear();
for (std::size_t i = 0; i < this->entries_.size(); ++i) this->index_.emplace(this->entries_[i].name, i);
}
std::filesystem::path path_;
std::vector<std::byte> bytes_; ///< In-memory image (from_bytes); empty when opened from disk.
bool in_memory_ = false;
std::uint64_t file_size_ = 0;
std::vector<big_entry> entries_;
std::unordered_map<std::string, std::size_t> index_;
};
/**
* Builds a `BIG4` archive.
*
* Entry payloads are written 64-byte aligned (as the retail archives are),
* and the header's index size reproduces the retail convention
* (`last-entry end + 8`). `add(..., compress = true)` RefPack-encodes the
* payload when that is smaller, otherwise stores it verbatim.
*/
class big_writer {
public:
auto add(std::string name, std::vector<std::byte> payload, bool compress = false) -> void {
if (compress) {
auto packed = refpack_compress(payload);
if (packed.size() < payload.size()) payload = std::move(packed);
}
this->items_.push_back({std::move(name), std::move(payload)});
}
/** Serialise the archive into a fresh byte image. */
[[nodiscard]] auto write() const -> std::vector<std::byte> {
std::uint64_t table_end = 16U;
for (const auto &item: this->items_) table_end += 8U + item.name.size() + 1U;
const auto index_size = table_end + 8U;
const auto data_start = detail::align_up(index_size, 64U);
std::vector<std::uint64_t> offsets;
offsets.reserve(this->items_.size());
std::uint64_t cursor = data_start;
for (const auto &item: this->items_) {
offsets.push_back(cursor);
cursor = detail::align_up(cursor + item.payload.size(), 64U);
}
byte_writer writer;
writer.write_ascii("BIG4");
writer.write_u32(static_cast<std::uint32_t>(cursor)); // total file size (little-endian)
writer.write_u32(detail::bswap32(static_cast<std::uint32_t>(this->items_.size())));
writer.write_u32(detail::bswap32(static_cast<std::uint32_t>(index_size)));
for (std::size_t i = 0; i < this->items_.size(); ++i) {
writer.write_u32(detail::bswap32(static_cast<std::uint32_t>(offsets[i])));
writer.write_u32(detail::bswap32(static_cast<std::uint32_t>(this->items_[i].payload.size())));
writer.write_ascii(this->items_[i].name);
writer.write_u8(0);
}
while (writer.size() < data_start) writer.write_u8(0);
for (std::size_t i = 0; i < this->items_.size(); ++i) {
writer.write_bytes(this->items_[i].payload);
if (i + 1U < this->items_.size()) {
while (writer.size() < offsets[i + 1U]) writer.write_u8(0);
}
}
return writer.take();
}
private:
struct pending_item {
std::string name;
std::vector<std::byte> payload;
};
std::vector<pending_item> items_;
};
} // namespace ra3::assets
+396
View File
@@ -0,0 +1,396 @@
/**
* Compiled `BinaryAsset` streams (`Data\*.bin` + `.manifest`).
*
* Retail RA3 does not ship its gameplay/art assets as source files: the
* BinaryAssetBuilder compiles every XML/`.w3x` into a `.bin` *instance stream*
* plus a `.manifest` index (`data\static.bin`, `data\global.bin`, ...). This
* partition parses that index, exposes each asset's instance slice, its
* relocation/import sidecars and its custom-data (`cdata`) blob, and implements
* the Bob-Jenkins-style hash the builder names assets with.
*
* Asset names are `"Type:Instance"` (e.g. `W3DMesh:ABAIRFIELD`); `Type` is
* case-sensitive and `Instance` is lowercased before hashing.
*/
export module ra3.assets:binary;
import std;
import :big;
import :bytes;
import :error;
import :refpack;
export namespace ra3::assets {
/** The compiled instance streams a retail install ships. */
enum class binary_stream {
static_data, ///< `data\static.bin` - the main art/gameplay stream
global, ///< `data\global.bin` - sounds, AI, UI
locale, ///< `data\locale.bin` - localized on-demand textures
static_low, ///< `data\static_l.bin` - low-detail model LODs
static_medium, ///< `data\static_m.bin` - medium-detail model LODs
};
/** The `data\<name>.manifest` stem for a stream. */
[[nodiscard]] inline auto to_string(binary_stream stream) -> std::string_view {
switch (stream) {
case binary_stream::static_data: return "static";
case binary_stream::global: return "global";
case binary_stream::locale: return "locale";
case binary_stream::static_low: return "static_l";
case binary_stream::static_medium: return "static_m";
}
return "static";
}
/** Resolve a stream name (e.g. `"global"`), or `std::nullopt`. */
[[nodiscard]] inline auto binary_stream_from_name(std::string_view name) -> std::optional<binary_stream> {
for (const auto candidate: {binary_stream::static_data, binary_stream::global, binary_stream::locale, binary_stream::static_low, binary_stream::static_medium}) {
if (to_string(candidate) == name) return candidate;
}
return std::nullopt;
}
/** Streams in the order `binary_stream_from_big` prefers when none is given. */
inline constexpr std::array<binary_stream, 5> binary_stream_order{binary_stream::static_data, binary_stream::global, binary_stream::locale, binary_stream::static_low,
binary_stream::static_medium};
/** Types compiled from a `.w3x` model source. */
inline constexpr std::array<std::string_view, 5> w3d_types{"W3DMesh", "W3DHierarchy", "W3DAnimation", "W3DContainer", "W3DCollisionBox"};
/**
* The hash BinaryAssetBuilder names assets with.
*
* A Bob-Jenkins `lookup3`-style mix. `seed` is the running value; the public
* overloads seed it with the length (`hash_string`).
*/
[[nodiscard]] inline auto fast_hash(std::span<const std::byte> data, std::uint32_t seed = 0) -> std::uint32_t {
const auto length = data.size();
if (length == 0U) return 0x1337C0DEU;
const auto at16 = [&](std::size_t pos) -> std::uint32_t {
return static_cast<std::uint32_t>(std::to_integer<std::uint8_t>(data[pos])) |
(static_cast<std::uint32_t>(std::to_integer<std::uint8_t>(data[pos + 1U])) << 8U);
};
std::uint32_t h = seed;
std::size_t pos = 0;
const auto extra = length & 3U;
for (std::size_t i = 0; i < (length >> 2U); ++i) {
h += at16(pos);
h ^= (at16(pos + 2U) ^ (h << 5U)) << 11U;
h += h >> 11U;
pos += 4U;
}
if (extra == 1U) {
h += std::to_integer<std::uint8_t>(data[pos]);
h ^= h << 10U;
h += h >> 1U;
} else if (extra == 2U) {
h += at16(pos);
h ^= h << 11U;
h += h >> 17U;
} else if (extra == 3U) {
h += at16(pos);
h ^= h << 16U;
h ^= static_cast<std::uint32_t>(std::to_integer<std::uint8_t>(data[pos + 2U])) << 18U;
h += h >> 11U;
}
h ^= h << 3U;
h += h >> 5U;
h ^= h << 2U;
h += h >> 15U;
h ^= h << 10U;
return h;
}
/**
* Hash an asset type (case-sensitive, the `TypeId`) or an instance name
* (`case_sensitive = false`, the `InstanceId`).
*/
[[nodiscard]] inline auto hash_string(std::string_view text, bool case_sensitive = true) -> std::uint32_t {
std::string buffer{text};
if (!case_sensitive) {
std::ranges::transform(buffer, buffer.begin(), [](unsigned char ch) { return static_cast<char>(std::tolower(ch)); });
}
const auto *bytes = reinterpret_cast<const std::byte *>(buffer.data());
return fast_hash(std::span<const std::byte>{bytes, buffer.size()}, static_cast<std::uint32_t>(buffer.size()));
}
/** One asset in a compiled stream. */
struct binary_asset {
std::uint32_t index = 0;
std::uint32_t type_id = 0;
std::uint32_t instance_id = 0;
std::uint32_t type_hash = 0;
std::uint32_t instance_hash = 0;
std::vector<std::pair<std::uint32_t, std::uint32_t>> references; ///< (typeId, instanceId) pairs
std::string name; ///< "Type:Instance"
std::string source; ///< the `.w3x`/XML it was compiled from
std::uint32_t instance_size = 0;
std::uint32_t relocation_size = 0;
std::uint32_t imports_size = 0;
bool tokenized = false;
std::uint32_t instance_offset = 4; ///< into the decompressed `.bin` (after its 4-byte checksum)
std::uint32_t relocation_offset = 4; ///< into the `.relo`
std::uint32_t imports_offset = 4; ///< into the `.imp`
[[nodiscard]] auto type_name() const -> std::string_view {
const auto colon = this->name.find(':');
return std::string_view{this->name}.substr(0, colon);
}
[[nodiscard]] auto instance_name() const -> std::string_view {
const auto colon = this->name.find(':');
return colon == std::string::npos ? std::string_view{this->name} : std::string_view{this->name}.substr(colon + 1U);
}
/**
* The `cdata` blob name for a custom-data asset (`AudioFile`,
* `OnDemandTexture`, ...):
* `data\<stream>\cdata\<typeId>.<typeHash>.<instanceId>.<instanceHash>.cdata`.
*/
[[nodiscard]] auto cdata_name(std::string_view stream) const -> std::string {
return std::format("data\\{}\\cdata\\{:08x}.{:08x}.{:08x}.{:08x}.cdata", stream, this->type_id, this->type_hash, this->instance_id,
this->instance_hash);
}
};
/** A callback that resolves a `cdata` blob name to its decompressed bytes. */
using cdata_source = std::function<std::optional<std::vector<std::byte>>(const std::string &name)>;
/**
* A parsed `.manifest` (+ optional `.bin`/`.relo`/`.imp`).
*
* Only the manifest is mandatory; `need_data = false` when constructing
* skips the (up to hundreds of megabytes) instance stream, which is all
* listing the assets requires.
*/
class binary_container {
public:
/** Parse a manifest, its instance stream and its optional fixup streams. */
[[nodiscard]] static auto from_bytes(std::vector<std::byte> manifest, std::vector<std::byte> data = {}, std::vector<std::byte> relocation = {},
std::vector<std::byte> imports = {}, std::string stream = "static") -> binary_container {
binary_container container;
container.stream_ = std::move(stream);
container.manifest_ = std::move(manifest);
container.data_ = std::move(data);
container.relocation_ = std::move(relocation);
container.imports_ = std::move(imports);
container.parse();
return container;
}
[[nodiscard]] auto stream() const -> const std::string & { return this->stream_; }
[[nodiscard]] auto version() const -> std::uint16_t { return this->version_; }
[[nodiscard]] auto is_linked() const -> bool { return this->is_linked_; }
[[nodiscard]] auto assets() const -> const std::vector<binary_asset> & { return this->assets_; }
[[nodiscard]] auto size() const -> std::size_t { return this->assets_.size(); }
/** Count of assets per `Type` (the part before `:`). */
[[nodiscard]] auto type_counts() const -> std::map<std::string, std::size_t> {
std::map<std::string, std::size_t> counts;
for (const auto &asset: this->assets_) ++counts[std::string{asset.type_name()}];
return counts;
}
/** Map `(typeId << 32 | instanceId)` to the owning asset. */
[[nodiscard]] auto asset_index() const -> const std::unordered_map<std::uint64_t, std::size_t> & { return this->asset_index_; }
/** Every asset, in manifest order, whose `Type` matches (case-insensitive). */
[[nodiscard]] auto of_type(std::string_view type) const -> std::vector<const binary_asset *> {
std::vector<const binary_asset *> matches;
for (const auto &asset: this->assets_) {
if (std::ranges::equal(asset.type_name(), type, [](char a, char b) { return std::tolower(static_cast<unsigned char>(a)) == std::tolower(static_cast<unsigned char>(b)); }))
matches.push_back(&asset);
}
return matches;
}
/** Resolve `"Type:Instance"` (case-sensitive, then insensitive) or `"#index"`. */
[[nodiscard]] auto find(std::string_view selector) const -> const binary_asset & {
if (selector.starts_with('#')) {
const auto index = static_cast<std::size_t>(std::stoul(std::string{selector.substr(1)}));
if (index >= this->assets_.size()) throw binary_error("asset index out of range: " + std::string{selector});
return this->assets_[index];
}
for (const auto &asset: this->assets_) {
if (asset.name == selector) return asset;
}
for (const auto &asset: this->assets_) {
if (std::ranges::equal(asset.name, selector, [](char a, char b) { return std::tolower(static_cast<unsigned char>(a)) == std::tolower(static_cast<unsigned char>(b)); }))
return asset;
}
throw binary_error("no such asset: " + std::string{selector});
}
auto set_cdata_source(cdata_source source) -> void { this->cdata_ = std::move(source); }
[[nodiscard]] auto read_instance(const binary_asset &asset) const -> std::span<const std::byte> {
if (asset.instance_size == 0U) return {};
if (static_cast<std::uint64_t>(asset.instance_offset) + asset.instance_size > this->data_.size()) throw binary_error("instance extends past stream: " + asset.name);
return std::span<const std::byte>{this->data_}.subspan(asset.instance_offset, asset.instance_size);
}
[[nodiscard]] auto read_relocation(const binary_asset &asset) const -> std::span<const std::byte> {
if (asset.relocation_size == 0U) return {};
if (static_cast<std::uint64_t>(asset.relocation_offset) + asset.relocation_size > this->relocation_.size()) throw binary_error("relocation extends past stream: " + asset.name);
return std::span<const std::byte>{this->relocation_}.subspan(asset.relocation_offset, asset.relocation_size);
}
[[nodiscard]] auto read_imports(const binary_asset &asset) const -> std::span<const std::byte> {
if (asset.imports_size == 0U) return {};
if (static_cast<std::uint64_t>(asset.imports_offset) + asset.imports_size > this->imports_.size()) throw binary_error("imports extend past stream: " + asset.name);
return std::span<const std::byte>{this->imports_}.subspan(asset.imports_offset, asset.imports_size);
}
/** The asset's `cdata` blob, or `std::nullopt` when it has none. */
[[nodiscard]] auto read_cdata(const binary_asset &asset) const -> std::optional<std::vector<std::byte>> {
if (!this->cdata_) return std::nullopt;
return this->cdata_(asset.cdata_name(this->stream_));
}
/** The asset's real payload: its `cdata` blob when present, else its instance. */
[[nodiscard]] auto read_payload(const binary_asset &asset) const -> std::vector<std::byte> {
if (auto cdata = this->read_cdata(asset)) return *cdata;
const auto instance = this->read_instance(asset);
return {instance.begin(), instance.end()};
}
private:
binary_container() = default;
auto parse() -> void {
byte_reader reader{this->manifest_};
const auto is_big_endian = reader.read_u8();
this->is_linked_ = reader.read_bool();
this->version_ = reader.read_u16();
if (is_big_endian != 0U) throw binary_error("big-endian manifests are not supported");
if (this->version_ != 5U && this->version_ != 6U) throw binary_error("unsupported manifest version " + std::to_string(this->version_));
(void) reader.read_u32(); // stream checksum
(void) reader.read_u32(); // all-types hash
const auto count = reader.read_u32();
(void) reader.read_u32(); // total instance data size
(void) reader.read_u32(); // max instance chunk size
(void) reader.read_u32(); // max relocation chunk size
(void) reader.read_u32(); // max imports chunk size
const auto reference_buffer_size = reader.read_u32();
const auto reference_name_buffer_size = reader.read_u32();
const auto name_buffer_size = reader.read_u32();
const auto source_buffer_size = reader.read_u32();
constexpr std::size_t header_size = 48U;
constexpr std::size_t entry_size = 48U;
if (this->manifest_.size() < header_size + static_cast<std::size_t>(count) * entry_size) throw binary_error("manifest entry table is truncated");
const auto entries_off = header_size;
const auto references_off = entries_off + static_cast<std::size_t>(count) * entry_size;
const auto reference_names_off = references_off + reference_buffer_size;
const auto names_off = reference_names_off + reference_name_buffer_size;
const auto sources_off = names_off + name_buffer_size;
if (sources_off + source_buffer_size > this->manifest_.size()) throw binary_error("manifest string buffers are truncated");
const auto cstr = [&](std::size_t base, std::int32_t offset) -> std::string {
if (offset < 0) return {};
const auto start = base + static_cast<std::size_t>(offset);
if (start >= this->manifest_.size()) return {};
std::size_t end = start;
while (end < this->manifest_.size() && std::to_integer<std::uint8_t>(this->manifest_[end]) != 0U) ++end;
return std::string{reinterpret_cast<const char *>(this->manifest_.data() + start), end - start};
};
this->assets_.reserve(count);
std::uint32_t instance_offset = 4;
std::uint32_t relocation_offset = 4;
std::uint32_t imports_offset = 4;
for (std::uint32_t i = 0; i < count; ++i) {
byte_reader entry{std::span<const std::byte>{this->manifest_}.subspan(entries_off + static_cast<std::size_t>(i) * entry_size, entry_size)};
binary_asset asset;
asset.index = i;
asset.type_id = entry.read_u32();
asset.instance_id = entry.read_u32();
asset.type_hash = entry.read_u32();
asset.instance_hash = entry.read_u32();
const auto reference_offset = entry.read_i32();
const auto reference_count = entry.read_i32();
const auto name_offset = entry.read_i32();
const auto source_offset = entry.read_i32();
asset.instance_size = entry.read_u32();
asset.relocation_size = entry.read_u32();
asset.imports_size = entry.read_u32();
asset.tokenized = entry.read_u32() != 0U;
asset.name = cstr(names_off, name_offset);
asset.source = cstr(sources_off, source_offset);
asset.instance_offset = instance_offset;
asset.relocation_offset = relocation_offset;
asset.imports_offset = imports_offset;
instance_offset += asset.instance_size;
relocation_offset += asset.relocation_size;
imports_offset += asset.imports_size;
for (std::int32_t j = 0; j < reference_count; ++j) {
byte_reader ref{std::span<const std::byte>{this->manifest_}.subspan(references_off + static_cast<std::size_t>(reference_offset) +
static_cast<std::size_t>(j) * 8U,
8U)};
asset.references.emplace_back(ref.read_u32(), ref.read_u32());
}
this->asset_index_.emplace((static_cast<std::uint64_t>(asset.type_id) << 32U) | asset.instance_id, this->assets_.size());
this->assets_.push_back(std::move(asset));
}
}
std::string stream_ = "static";
std::vector<std::byte> manifest_;
std::vector<std::byte> data_;
std::vector<std::byte> relocation_;
std::vector<std::byte> imports_;
std::uint16_t version_ = 0;
bool is_linked_ = false;
std::vector<binary_asset> assets_;
std::unordered_map<std::uint64_t, std::size_t> asset_index_;
cdata_source cdata_;
};
namespace detail {
[[nodiscard]] inline auto pick_manifest(std::string_view stream) -> std::string { return std::format("data\\{}.manifest", stream); }
}
/**
* Load one BinaryAsset stream out of a `BIG4` archive.
*
* `stream` selects the `.manifest` (default: the first of
* `binary_stream_order` found). When `need_data` is false only the manifest
* is read, which is enough to enumerate assets without decompressing the
* multi-hundred-megabyte instance stream.
*
* The returned container keeps a `cdata` reader that borrows `archive`; keep
* the archive alive for as long as the container is used.
*/
[[nodiscard]] inline auto binary_stream_from_big(const big_archive &archive, std::optional<binary_stream> stream = std::nullopt, bool need_data = true) -> binary_container {
std::optional<binary_stream> chosen = stream;
if (!chosen) {
for (const auto candidate: binary_stream_order) {
if (archive.contains(detail::pick_manifest(to_string(candidate)))) {
chosen = candidate;
break;
}
}
}
if (!chosen) throw binary_error("archive has no known .manifest entry");
const auto manifest_name = detail::pick_manifest(to_string(*chosen));
const auto base = manifest_name.substr(0, manifest_name.size() - std::string_view{".manifest"}.size());
const auto manifest = maybe_decompress(archive.read(manifest_name, false));
std::vector<std::byte> data;
if (need_data) data = maybe_decompress(archive.read(base + ".bin", false));
const auto relocation = maybe_decompress(archive.read(base + ".relo", false));
const auto imports = maybe_decompress(archive.read(base + ".imp", false));
auto container = binary_container::from_bytes(std::move(manifest), std::move(data), std::move(relocation), std::move(imports), std::string{to_string(*chosen)});
container.set_cdata_source([&archive](const std::string &name) -> std::optional<std::vector<std::byte>> {
const auto *entry = archive.find_exact(name);
if (entry == nullptr) return std::nullopt;
return maybe_decompress(archive.read(*entry, false));
});
return container;
}
} // namespace ra3::assets
+220
View File
@@ -0,0 +1,220 @@
/**
* Bounds-checked little-/big-endian byte readers and writers.
*
* Red Alert 3's asset formats are little-endian, with one notable exception:
* the `BIG4` index stores its counts, offsets and sizes big-endian. Both byte
* orders are provided here. Every read is bounds-checked and throws
* `format_error` rather than reading past the end of the buffer.
*/
export module ra3.assets:bytes;
import std;
import :error;
export namespace ra3::assets {
/** A cursor over a read-only byte range with checked accessors. */
class byte_reader {
public:
explicit byte_reader(std::span<const std::byte> data)
: data_(data) {
}
[[nodiscard]] auto size() const -> std::size_t { return this->data_.size(); }
[[nodiscard]] auto position() const -> std::size_t { return this->pos_; }
[[nodiscard]] auto remaining() const -> std::size_t { return this->data_.size() - this->pos_; }
[[nodiscard]] auto empty() const -> bool { return this->pos_ >= this->data_.size(); }
auto seek(std::size_t at) -> void {
if (at > this->data_.size()) throw format_error("seek past end of buffer");
this->pos_ = at;
}
auto skip(std::size_t count) -> void {
this->require(count);
this->pos_ += count;
}
[[nodiscard]] auto read_u8() -> std::uint8_t {
this->require(1);
return std::to_integer<std::uint8_t>(this->data_[this->pos_++]);
}
[[nodiscard]] auto read_bool() -> bool { return this->read_u8() != 0U; }
[[nodiscard]] auto read_u16() -> std::uint16_t {
return static_cast<std::uint16_t>(this->read_le(2));
}
[[nodiscard]] auto read_u24() -> std::uint32_t { return this->read_le(3); }
[[nodiscard]] auto read_u32() -> std::uint32_t { return this->read_le(4); }
[[nodiscard]] auto read_u64() -> std::uint64_t { return this->read_le(8); }
[[nodiscard]] auto read_i16() -> std::int16_t { return static_cast<std::int16_t>(this->read_u16()); }
[[nodiscard]] auto read_i32() -> std::int32_t { return static_cast<std::int32_t>(this->read_u32()); }
[[nodiscard]] auto read_f32() -> float { return std::bit_cast<float>(this->read_u32()); }
[[nodiscard]] auto read_be_u16() -> std::uint16_t { return static_cast<std::uint16_t>(this->read_be(2)); }
[[nodiscard]] auto read_be_u32() -> std::uint32_t { return this->read_be(4); }
/** A view of the next `count` bytes; the cursor advances past them. */
[[nodiscard]] auto read_bytes(std::size_t count) -> std::span<const std::byte> {
this->require(count);
const auto view = this->data_.subspan(this->pos_, count);
this->pos_ += count;
return view;
}
/** `count` bytes decoded as Latin-1 (one byte per character). */
[[nodiscard]] auto read_ascii(std::size_t count) -> std::string {
const auto view = this->read_bytes(count);
return std::string{reinterpret_cast<const char *>(view.data()), view.size()};
}
/** A `u16`-length-prefixed ASCII string. */
[[nodiscard]] auto read_u16_prefixed_ascii() -> std::string { return this->read_ascii(this->read_u16()); }
/** A `u16`-length-prefixed string, decoded as UTF-8 (ASCII-compatible). */
[[nodiscard]] auto read_u16_prefixed_ascii_as_utf8() -> std::string { return this->read_u16_prefixed_ascii(); }
/** A `u16`-length-prefixed UTF-16LE string (length counted in code units). */
[[nodiscard]] auto read_u16_prefixed_utf16() -> std::u16string {
const auto count = this->read_u16();
this->require(static_cast<std::size_t>(count) * 2U);
std::u16string text(count, u'\0');
for (std::uint16_t i = 0; i < count; ++i) text[i] = static_cast<char16_t>(this->read_u16());
return text;
}
/** A NUL-terminated ASCII string; the cursor stops just past the NUL. */
[[nodiscard]] auto read_cstring() -> std::string {
std::string text;
while (this->pos_ < this->data_.size()) {
const auto ch = static_cast<char>(this->read_u8());
if (ch == '\0') break;
text.push_back(ch);
}
return text;
}
private:
auto require(std::size_t count) const -> void {
if (this->pos_ + count > this->data_.size()) throw format_error("unexpected end of buffer");
}
[[nodiscard]] auto read_le(std::size_t width) -> std::uint64_t {
this->require(width);
std::uint64_t value = 0;
for (std::size_t i = 0; i < width; ++i) value |= static_cast<std::uint64_t>(std::to_integer<std::uint8_t>(this->data_[this->pos_ + i])) << (8U * i);
this->pos_ += width;
return value;
}
[[nodiscard]] auto read_be(std::size_t width) -> std::uint64_t {
this->require(width);
std::uint64_t value = 0;
for (std::size_t i = 0; i < width; ++i) value = (value << 8U) | static_cast<std::uint64_t>(std::to_integer<std::uint8_t>(this->data_[this->pos_ + i]));
this->pos_ += width;
return value;
}
std::span<const std::byte> data_;
std::size_t pos_ = 0;
};
/** An append-only little-endian byte buffer. */
class byte_writer {
public:
[[nodiscard]] auto size() const -> std::size_t { return this->buffer_.size(); }
[[nodiscard]] auto data() const -> std::span<const std::byte> { return this->buffer_; }
[[nodiscard]] auto take() -> std::vector<std::byte> { return std::move(this->buffer_); }
auto write_u8(std::uint8_t value) -> void { this->buffer_.push_back(static_cast<std::byte>(value)); }
auto write_bool(bool value) -> void { this->write_u8(value ? 1U : 0U); }
auto write_u16(std::uint16_t value) -> void { this->write_le(value, 2); }
auto write_u24(std::uint32_t value) -> void { this->write_le(value, 3); }
auto write_u32(std::uint32_t value) -> void { this->write_le(value, 4); }
auto write_u64(std::uint64_t value) -> void { this->write_le(value, 8); }
auto write_i16(std::int16_t value) -> void { this->write_u16(static_cast<std::uint16_t>(value)); }
auto write_i32(std::int32_t value) -> void { this->write_u32(static_cast<std::uint32_t>(value)); }
auto write_f32(float value) -> void { this->write_u32(std::bit_cast<std::uint32_t>(value)); }
auto write_bytes(std::span<const std::byte> bytes) -> void { this->buffer_.insert(this->buffer_.end(), bytes.begin(), bytes.end()); }
auto write_ascii(std::string_view text) -> void {
for (const auto ch: text) this->buffer_.push_back(static_cast<std::byte>(static_cast<unsigned char>(ch)));
}
auto write_u16_prefixed_ascii(std::string_view text) -> void {
this->write_u16(static_cast<std::uint16_t>(text.size()));
this->write_ascii(text);
}
auto write_u16_prefixed_utf16(std::u16string_view text) -> void {
this->write_u16(static_cast<std::uint16_t>(text.size()));
for (const auto unit: text) {
this->write_u16(static_cast<std::uint16_t>(unit));
}
}
/** Overwrite a previously written `u32` (used to backpatch sizes). */
auto patch_u32(std::size_t offset, std::uint32_t value) -> void {
if (offset + 4U > this->buffer_.size()) throw format_error("patch offset past end of buffer");
for (std::size_t i = 0; i < 4U; ++i) this->buffer_[offset + i] = static_cast<std::byte>((value >> (8U * i)) & 0xFFU);
}
private:
auto write_le(std::uint64_t value, std::size_t width) -> void {
for (std::size_t i = 0; i < width; ++i) this->buffer_.push_back(static_cast<std::byte>((value >> (8U * i)) & 0xFFU));
}
std::vector<std::byte> buffer_;
};
/** Convenience: encode a UTF-8 string (ASCII subset) as UTF-16LE code units. */
[[nodiscard]] inline auto to_utf16(std::string_view text) -> std::u16string {
std::u16string out;
out.reserve(text.size());
for (const auto ch: text) out.push_back(static_cast<char16_t>(static_cast<unsigned char>(ch)));
return out;
}
/** Convenience: narrow a UTF-16 string that is known to be within Latin-1. */
[[nodiscard]] inline auto to_ascii(std::u16string_view text) -> std::string {
std::string out;
out.reserve(text.size());
for (const auto unit: text) out.push_back(static_cast<char>(unit & 0xFFU));
return out;
}
/** Read a whole file as bytes. */
[[nodiscard]] inline auto read_file(const std::filesystem::path &path) -> std::vector<std::byte> {
std::ifstream in(path, std::ios::binary);
if (!in) throw format_error("cannot open file: " + path.string());
in.seekg(0, std::ios::end);
const auto end = in.tellg();
if (end < 0) throw format_error("cannot size file: " + path.string());
std::vector<std::byte> bytes(static_cast<std::size_t>(end));
in.seekg(0, std::ios::beg);
if (!bytes.empty()) in.read(reinterpret_cast<char *>(bytes.data()), static_cast<std::streamsize>(bytes.size()));
return bytes;
}
/** Write bytes to a file, creating parent directories as needed. */
inline auto write_file(const std::filesystem::path &path, std::span<const std::byte> bytes) -> void {
std::error_code ec;
if (!path.parent_path().empty()) std::filesystem::create_directories(path.parent_path(), ec);
std::ofstream out(path, std::ios::binary);
if (!out) throw format_error("cannot write file: " + path.string());
out.write(reinterpret_cast<const char *>(bytes.data()), static_cast<std::streamsize>(bytes.size()));
if (!out) throw format_error("cannot write file: " + path.string());
}
} // namespace ra3::assets
+180
View File
@@ -0,0 +1,180 @@
/**
* SAGE `.csf` string tables (`data\gamestrings.csf`).
*
* The compressed unit/label localization table Red Alert 3 ships per language
* in `Lang-*.big` / `English.big`. Each value is a UTF-16 string whose every
* code unit is bit-inverted (`code ^ 0xFFFF`); labels are plain ASCII and are
* matched case-insensitively.
*/
export module ra3.assets:csf;
import std;
import :big;
import :bytes;
import :error;
import :refpack;
export namespace ra3::assets {
/** Magic of the label block (`" LBL"` little-endian). */
inline constexpr std::uint32_t csf_label_flag = 0x4C424C20U;
/** Magic of a value block (`" RTS"` little-endian). */
inline constexpr std::uint32_t csf_value_flag = 0x53545220U;
/** One label and its (usually single) string value. */
struct csf_entry {
std::string label;
std::vector<std::u16string> values;
/** The concatenation of every value block, the way the game renders it. */
[[nodiscard]] auto value() const -> std::u16string {
std::u16string joined;
for (const auto &chunk: this->values) joined += chunk;
return joined;
}
};
/** Decode a UTF-16 string (with surrogate pairs) to UTF-8. */
[[nodiscard]] inline auto utf16_to_utf8(std::u16string_view text) -> std::string {
std::string out;
for (std::size_t i = 0; i < text.size(); ++i) {
auto code = static_cast<std::uint32_t>(text[i]);
if (code >= 0xD800U && code <= 0xDBFFU && i + 1U < text.size()) {
const auto low = static_cast<std::uint32_t>(text[i + 1U]);
if (low >= 0xDC00U && low <= 0xDFFFU) {
code = 0x10000U + ((code - 0xD800U) << 10U) + (low - 0xDC00U);
++i;
}
}
if (code < 0x80U) {
out.push_back(static_cast<char>(code));
} else if (code < 0x800U) {
out.push_back(static_cast<char>(0xC0U | (code >> 6U)));
out.push_back(static_cast<char>(0x80U | (code & 0x3FU)));
} else if (code < 0x10000U) {
out.push_back(static_cast<char>(0xE0U | (code >> 12U)));
out.push_back(static_cast<char>(0x80U | ((code >> 6U) & 0x3FU)));
out.push_back(static_cast<char>(0x80U | (code & 0x3FU)));
} else {
out.push_back(static_cast<char>(0xF0U | (code >> 18U)));
out.push_back(static_cast<char>(0x80U | ((code >> 12U) & 0x3FU)));
out.push_back(static_cast<char>(0x80U | ((code >> 6U) & 0x3FU)));
out.push_back(static_cast<char>(0x80U | (code & 0x3FU)));
}
}
return out;
}
/**
* A parsed `.csf` table.
*
* Labels are stored verbatim; lookup upper-cases the key because the engine
* treats them case-insensitively.
*/
class csf_table {
public:
/** Parse a `.csf` image. */
[[nodiscard]] static auto parse(std::span<const std::byte> bytes) -> csf_table {
byte_reader reader{bytes};
const auto magic = reader.read_ascii(4);
if (magic != "CSF " && magic != " FSC") throw csf_error("not a CSF file");
csf_table table;
table.version_ = reader.read_u32();
const auto num_labels = reader.read_u32();
(void) reader.read_u32(); // number of value blocks
(void) reader.read_u32(); // reserved
(void) reader.read_u32(); // reserved
table.entries_.reserve(num_labels);
for (std::uint32_t i = 0; i < num_labels; ++i) {
(void) reader.read_u32(); // label flag (" LBL")
const auto value_count = reader.read_u32();
const auto label_length = reader.read_u32();
csf_entry entry;
entry.label = reader.read_ascii(label_length);
entry.values.reserve(value_count);
for (std::uint32_t j = 0; j < value_count; ++j) {
(void) reader.read_u32(); // value flag (" RTS")
const auto char_count = reader.read_u32();
std::u16string value(char_count, u'\0');
for (std::uint32_t k = 0; k < char_count; ++k) value[k] = static_cast<char16_t>(reader.read_u16() ^ 0xFFFFU);
entry.values.push_back(std::move(value));
}
table.entries_.push_back(std::move(entry));
}
table.reindex();
return table;
}
/** Read and parse a `.csf` file from disk. */
[[nodiscard]] static auto open(const std::filesystem::path &path) -> csf_table {
return csf_table::parse(read_file(path));
}
/** Load `data\gamestrings.csf` out of an archive. */
[[nodiscard]] static auto from_big(const big_archive &archive, std::string_view entry = "data\\gamestrings.csf") -> csf_table {
return csf_table::parse(maybe_decompress(archive.read(entry, false)));
}
/** Serialise the table back to a `.csf` image. */
[[nodiscard]] auto write() const -> std::vector<std::byte> {
byte_writer writer;
writer.write_ascii(" FSC");
writer.write_u32(this->version_);
writer.write_u32(static_cast<std::uint32_t>(this->entries_.size()));
std::uint32_t value_count = 0;
for (const auto &entry: this->entries_) value_count += static_cast<std::uint32_t>(entry.values.size());
writer.write_u32(value_count);
writer.write_u32(0);
writer.write_u32(0);
for (const auto &entry: this->entries_) {
writer.write_u32(csf_label_flag);
writer.write_u32(static_cast<std::uint32_t>(entry.values.size()));
writer.write_u32(static_cast<std::uint32_t>(entry.label.size()));
writer.write_ascii(entry.label);
for (const auto &value: entry.values) {
writer.write_u32(csf_value_flag);
writer.write_u32(static_cast<std::uint32_t>(value.size()));
for (const auto unit: value) writer.write_u16(static_cast<std::uint16_t>(unit) ^ 0xFFFFU);
}
}
return writer.take();
}
[[nodiscard]] auto version() const -> std::uint32_t { return this->version_; }
[[nodiscard]] auto entries() const -> const std::vector<csf_entry> & { return this->entries_; }
[[nodiscard]] auto size() const -> std::size_t { return this->entries_.size(); }
/** The entry for `label`, or `nullptr`; the match is case-insensitive. */
[[nodiscard]] auto find(std::string_view label) const -> const csf_entry * {
auto key = std::string{label};
std::ranges::transform(key, key.begin(), [](unsigned char ch) { return static_cast<char>(std::toupper(ch)); });
const auto it = this->index_.find(key);
return it == this->index_.end() ? nullptr : &this->entries_[it->second];
}
/** The UTF-8 value for `label`, or `fallback` when absent. */
[[nodiscard]] auto lookup(std::string_view label, std::string fallback = {}) const -> std::string {
const auto *entry = this->find(label);
if (entry == nullptr) return fallback;
auto text = utf16_to_utf8(entry->value());
return text.empty() ? std::move(fallback) : text;
}
private:
auto reindex() -> void {
this->index_.clear();
for (std::size_t i = 0; i < this->entries_.size(); ++i) {
auto key = this->entries_[i].label;
std::ranges::transform(key, key.begin(), [](unsigned char ch) { return static_cast<char>(std::toupper(ch)); });
this->index_.try_emplace(std::move(key), i);
}
}
std::uint32_t version_ = 3;
std::vector<csf_entry> entries_;
std::unordered_map<std::string, std::size_t> index_;
};
} // namespace ra3::assets
+56
View File
@@ -0,0 +1,56 @@
/**
* Exception hierarchy of libra3assets.
*
* Every failure the library reports derives from `asset_error` (itself a
* `std::runtime_error`), so a caller can catch the whole family with one
* handler and still discriminate by the concrete type when it matters.
*/
export module ra3.assets:error;
import std;
export namespace ra3::assets {
/** Base class of every failure libra3assets reports. */
class asset_error: public std::runtime_error {
public:
using std::runtime_error::runtime_error;
};
/** A byte range is not a well-formed Red Alert 3 asset. */
class format_error: public asset_error {
public:
using asset_error::asset_error;
};
/** An EA RefPack stream is malformed or its length disagrees. */
class refpack_error: public asset_error {
public:
using asset_error::asset_error;
};
/** A `BIG4` archive is malformed, truncated or missing an entry. */
class big_error: public asset_error {
public:
using asset_error::asset_error;
};
/** A compiled `BinaryAsset` manifest/stream is malformed. */
class binary_error: public asset_error {
public:
using asset_error::asset_error;
};
/** A SAGE `.csf` string table is malformed. */
class csf_error: public asset_error {
public:
using asset_error::asset_error;
};
/** A SAGE `.map` container or one of its chunks is malformed. */
class map_error: public asset_error {
public:
using asset_error::asset_error;
};
} // namespace ra3::assets
+778
View File
@@ -0,0 +1,778 @@
/**
* SAGE `.map` containers - the core of the map editor.
*
* A `.map` is a `CkMp` chunk tree. On disk it may be wrapped twice: inside a
* `BIG4` the payload is one RefPack stream; the `.map` file it yields is either
* a bare `CkMp` tree or an `EAR\0`-wrapped RefPack stream around one. This
* partition unwraps all three shapes and models the tree as an ordered list of
* named, versioned chunks.
*
* The container round-trips losslessly: chunks this library does not model are
* preserved byte-for-byte, and the asset-name table keeps its original indices
* so unparsed chunks (which embed name indices) stay valid. On top of the
* container sit typed accessors for the chunks a map editor needs:
* `HeightMapData` (the terrain grid), `ObjectsList` (every placed prop, building
* and `*Waypoints/Waypoint`), `MPPositionList`, `WorldInfo` and `WaypointsList`.
*/
export module ra3.assets:map;
import std;
import :big;
import :bytes;
import :error;
import :refpack;
export namespace ra3::assets {
/** A world-space coordinate. */
struct coord3d {
float x = 0.0F;
float y = 0.0F;
float z = 0.0F;
};
/**
* A map object's `RoadType` bit flags.
*
* Zero for most objects; set on objects the terrain road system connects
* (`Start`/`End`/`BridgeStart`/`BridgeEnd`/`Angled`/`TightCurve`/`EndCap`).
*/
enum class road_type : std::uint32_t {
none = 0,
start = 2,
end = 4,
angled = 8,
bridge_start = 16,
bridge_end = 32,
tight_curve = 64,
end_cap = 128,
unknown3 = 256,
unknown4 = 512,
};
[[nodiscard]] constexpr auto operator|(road_type a, road_type b) -> road_type {
return static_cast<road_type>(static_cast<std::uint32_t>(a) | static_cast<std::uint32_t>(b));
}
[[nodiscard]] constexpr auto operator&(road_type a, road_type b) -> road_type {
return static_cast<road_type>(static_cast<std::uint32_t>(a) & static_cast<std::uint32_t>(b));
}
/** True when `value` has every bit of `flag` set. */
[[nodiscard]] constexpr auto has_flag(road_type value, road_type flag) -> bool {
return (static_cast<std::uint32_t>(value) & static_cast<std::uint32_t>(flag)) != 0U;
}
/** The `CkMp` chunk types SAGE defines, plus `unknown` for anything else. */
enum class chunk_kind {
asset_list,
global_version,
height_map_data,
blend_tile_data,
world_info,
mp_position_list,
sides_list,
library_map_lists,
teams,
player_scripts_list,
build_lists,
objects_list,
polygon_triggers,
trigger_areas,
global_water_settings,
fog_settings,
mission_hot_spots,
mission_objectives,
standing_water_areas,
river_areas,
standing_wave_areas,
global_lighting,
post_effects_chunk,
environment_data,
named_cameras,
camera_animation_list,
castle_templates,
waypoints_list,
skybox_settings,
unknown,
};
/** The canonical chunk name (the `unknown` kind has no name). */
[[nodiscard]] inline auto to_string(chunk_kind kind) -> std::string_view {
switch (kind) {
case chunk_kind::asset_list: return "AssetList";
case chunk_kind::global_version: return "GlobalVersion";
case chunk_kind::height_map_data: return "HeightMapData";
case chunk_kind::blend_tile_data: return "BlendTileData";
case chunk_kind::world_info: return "WorldInfo";
case chunk_kind::mp_position_list: return "MPPositionList";
case chunk_kind::sides_list: return "SidesList";
case chunk_kind::library_map_lists: return "LibraryMapLists";
case chunk_kind::teams: return "Teams";
case chunk_kind::player_scripts_list: return "PlayerScriptsList";
case chunk_kind::build_lists: return "BuildLists";
case chunk_kind::objects_list: return "ObjectsList";
case chunk_kind::polygon_triggers: return "PolygonTriggers";
case chunk_kind::trigger_areas: return "TriggerAreas";
case chunk_kind::global_water_settings: return "GlobalWaterSettings";
case chunk_kind::fog_settings: return "FogSettings";
case chunk_kind::mission_hot_spots: return "MissionHotSpots";
case chunk_kind::mission_objectives: return "MissionObjectives";
case chunk_kind::standing_water_areas: return "StandingWaterAreas";
case chunk_kind::river_areas: return "RiverAreas";
case chunk_kind::standing_wave_areas: return "StandingWaveAreas";
case chunk_kind::global_lighting: return "GlobalLighting";
case chunk_kind::post_effects_chunk: return "PostEffectsChunk";
case chunk_kind::environment_data: return "EnvironmentData";
case chunk_kind::named_cameras: return "NamedCameras";
case chunk_kind::camera_animation_list: return "CameraAnimationList";
case chunk_kind::castle_templates: return "CastleTemplates";
case chunk_kind::waypoints_list: return "WaypointsList";
case chunk_kind::skybox_settings: return "SkyboxSettings";
case chunk_kind::unknown: break;
}
return {};
}
/** Classify a chunk name. */
[[nodiscard]] inline auto chunk_kind_of(std::string_view name) -> chunk_kind {
for (int i = 0; i < static_cast<int>(chunk_kind::unknown); ++i) {
const auto kind = static_cast<chunk_kind>(i);
if (to_string(kind) == name) return kind;
}
return chunk_kind::unknown;
}
/** The map's asset-name table: `index -> name`, indices preserved on write. */
class name_table {
public:
[[nodiscard]] auto name(std::uint32_t index) const -> std::string_view {
if (index >= this->names_.size()) throw map_error("asset name index out of range: " + std::to_string(index));
return this->names_[index];
}
/** The existing index of `name`, or a freshly appended one. */
[[nodiscard]] auto get_or_create(std::string_view name) -> std::uint32_t {
for (std::uint32_t i = 1; i < this->names_.size(); ++i) {
if (this->names_[i] == name) return i;
}
this->names_.emplace_back(name);
return static_cast<std::uint32_t>(this->names_.size() - 1U);
}
[[nodiscard]] auto entries() const -> const std::vector<std::string> & { return this->names_; }
[[nodiscard]] auto size() const -> std::size_t { return this->names_.empty() ? 0U : this->names_.size() - 1U; }
/** Resize the table (used while parsing); index 0 stays the unused slot. */
auto resize(std::size_t count) -> void { this->names_.resize(count); }
/** Assign the name at `index` (used while parsing). */
auto set_name(std::uint32_t index, std::string name) -> void { this->names_.at(index) = std::move(name); }
private:
std::vector<std::string> names_{std::string{}}; // index 0 is unused
};
/** One `CkMp` chunk: a named, versioned, opaque payload. */
struct map_chunk {
std::string name;
chunk_kind kind = chunk_kind::unknown;
std::uint32_t asset_index = 0;
std::uint16_t version = 0;
std::vector<std::byte> payload;
};
/** A `HeightMapData` border rectangle. */
struct height_map_border {
std::uint32_t corner1_x = 0;
std::uint32_t corner1_y = 0;
std::uint32_t x = 0;
std::uint32_t y = 0;
};
/** The parsed `HeightMapData` chunk: the terrain elevation grid. */
struct height_map_data {
std::uint32_t width = 0;
std::uint32_t height = 0;
std::uint32_t border_width = 0;
std::vector<height_map_border> borders;
std::vector<std::uint16_t> elevations; ///< `width * height`, row-major (y outer, x inner)
std::uint16_t version = 6;
/** Metres per elevation unit (uint16 grids scale by 0.0390625). */
[[nodiscard]] auto vertical_scale() const -> float { return this->version >= 5U ? 0.0390625F : 0.625F; }
[[nodiscard]] auto at(std::uint32_t x, std::uint32_t y) const -> std::uint16_t { return this->elevations.at(static_cast<std::size_t>(y) * this->width + x); }
auto set(std::uint32_t x, std::uint32_t y, std::uint16_t value) -> void { this->elevations.at(static_cast<std::size_t>(y) * this->width + x) = value; }
/** Parse a `HeightMapData` chunk payload. */
[[nodiscard]] static auto parse(std::uint16_t version, std::span<const std::byte> payload) -> height_map_data {
byte_reader reader{payload};
height_map_data data;
data.version = version;
data.width = reader.read_u32();
data.height = reader.read_u32();
data.border_width = reader.read_u32();
const auto border_count = reader.read_u32();
data.borders.reserve(border_count);
for (std::uint32_t i = 0; i < border_count; ++i) {
height_map_border border;
if (version >= 6U) {
border.corner1_x = reader.read_u32();
border.corner1_y = reader.read_u32();
}
border.x = reader.read_u32();
border.y = reader.read_u32();
data.borders.push_back(border);
}
const auto area = reader.read_u32();
const auto expected = static_cast<std::uint64_t>(data.width) * data.height;
if (area != expected) throw map_error("HeightMapData area does not match width * height");
data.elevations.resize(static_cast<std::size_t>(expected));
for (std::uint32_t y = 0; y < data.height; ++y) {
for (std::uint32_t x = 0; x < data.width; ++x) {
data.elevations[static_cast<std::size_t>(y) * data.width + x] = version >= 5U ? reader.read_u16() : reader.read_u8();
}
}
return data;
}
/** Serialise the chunk payload (without the 10-byte chunk header). */
[[nodiscard]] auto serialize() const -> std::vector<std::byte> {
byte_writer writer;
writer.write_u32(this->width);
writer.write_u32(this->height);
writer.write_u32(this->border_width);
writer.write_u32(static_cast<std::uint32_t>(this->borders.size()));
for (const auto &border: this->borders) {
if (this->version >= 6U) {
writer.write_u32(border.corner1_x);
writer.write_u32(border.corner1_y);
}
writer.write_u32(border.x);
writer.write_u32(border.y);
}
writer.write_u32(this->width * this->height);
for (const auto elevation: this->elevations) {
if (this->version >= 5U) {
writer.write_u16(elevation);
} else {
writer.write_u8(static_cast<std::uint8_t>(elevation));
}
}
return writer.take();
}
};
/** The value type of an asset property. */
enum class property_type : std::uint8_t {
boolean = 0,
integer = 1,
real_number = 2,
ascii_string = 3,
unicode_string = 4,
unknown = 5,
};
/** The payload of an `asset_property`; the active alternative tracks `property_type`. */
using property_value = std::variant<bool, std::int32_t, float, std::string, std::u16string>;
/** One key/value pair carried by a map object or the world info. */
struct asset_property {
std::string name;
property_type type = property_type::integer;
property_value value{}; ///< bool, int32, float, ASCII/UTF-8 string, or UTF-16 string
[[nodiscard]] static auto boolean(std::string name, bool value) -> asset_property {
return {std::move(name), property_type::boolean, value};
}
[[nodiscard]] static auto integer(std::string name, std::int32_t value) -> asset_property {
return {std::move(name), property_type::integer, value};
}
[[nodiscard]] static auto real(std::string name, float value) -> asset_property {
return {std::move(name), property_type::real_number, value};
}
[[nodiscard]] static auto text(std::string name, std::string value) -> asset_property {
return {std::move(name), property_type::ascii_string, std::move(value)};
}
[[nodiscard]] static auto wide_text(std::string name, std::u16string value) -> asset_property {
return {std::move(name), property_type::unicode_string, std::move(value)};
}
[[nodiscard]] auto as_bool() const -> bool {
const auto *held = std::get_if<bool>(&this->value);
return held != nullptr && *held;
}
[[nodiscard]] auto as_int() const -> std::int32_t {
const auto *held = std::get_if<std::int32_t>(&this->value);
return held != nullptr ? *held : 0;
}
[[nodiscard]] auto as_real() const -> float {
const auto *held = std::get_if<float>(&this->value);
return held != nullptr ? *held : 0.0F;
}
[[nodiscard]] auto as_ascii() const -> std::string_view {
const auto *held = std::get_if<std::string>(&this->value);
return held != nullptr ? std::string_view{*held} : std::string_view{};
}
[[nodiscard]] auto as_unicode() const -> const std::u16string * { return std::get_if<std::u16string>(&this->value); }
};
/** One object placed on the map (building, prop, waypoint, ...). */
struct map_object {
std::string type_name; ///< the SAGE `ThingTemplate` name, e.g. `*Waypoints/Waypoint`
coord3d position;
float angle = 0.0F;
road_type road = road_type::none; ///< road connectivity flags (0 for most objects)
std::uint16_t version = 1;
std::vector<asset_property> properties;
[[nodiscard]] auto property(std::string_view key) const -> const asset_property * {
for (const auto &entry: this->properties) {
if (entry.name == key) return &entry;
}
return nullptr;
}
};
/** A player start position recovered from a `Player_N_Start` waypoint. */
struct player_start {
int index = 0; ///< the `N` in `Player_N_Start` (1-based)
coord3d position;
};
/** One `MPPositionInfo` entry. */
struct mp_position {
bool is_human = false;
bool is_computer = false;
bool load_ai_script = false;
std::uint32_t team = 0;
std::vector<std::string> side_restrictions;
std::uint16_t version = 1;
};
/** One `WaypointPath` (a link between two waypoints by id). */
struct waypoint_path {
std::int32_t start_id = 0;
std::int32_t end_id = 0;
};
namespace detail {
/** True when `data` begins with the ASCII tag (avoids `memcmp` with a literal). */
[[nodiscard]] inline auto starts_with(std::span<const std::byte> data, std::string_view tag) -> bool {
if (data.size() < tag.size()) return false;
for (std::size_t i = 0; i < tag.size(); ++i) {
if (std::to_integer<std::uint8_t>(data[i]) != static_cast<std::uint8_t>(tag[i])) return false;
}
return true;
}
[[nodiscard]] inline auto read_property(byte_reader &reader, const name_table &names) -> asset_property {
asset_property property;
property.type = static_cast<property_type>(reader.read_u8());
property.name = names.name(reader.read_u24());
switch (property.type) {
case property_type::boolean: property.value = reader.read_bool(); break;
case property_type::integer: property.value = reader.read_i32(); break;
case property_type::real_number: property.value = reader.read_f32(); break;
case property_type::ascii_string:
case property_type::unknown: property.value = reader.read_u16_prefixed_ascii(); break;
case property_type::unicode_string: property.value = reader.read_u16_prefixed_utf16(); break;
}
return property;
}
inline auto write_property(byte_writer &writer, const asset_property &property, name_table &names) -> void {
writer.write_u8(static_cast<std::uint8_t>(property.type));
writer.write_u24(names.get_or_create(property.name));
switch (property.type) {
case property_type::boolean: writer.write_bool(property.as_bool()); break;
case property_type::integer: writer.write_i32(property.as_int()); break;
case property_type::real_number: writer.write_f32(property.as_real()); break;
case property_type::ascii_string:
case property_type::unknown: writer.write_u16_prefixed_ascii(property.as_ascii()); break;
case property_type::unicode_string: {
const auto *text = property.as_unicode();
writer.write_u16_prefixed_utf16(text != nullptr ? *text : std::u16string{});
break;
}
}
}
[[nodiscard]] inline auto parse_property_list(byte_reader &reader, const name_table &names) -> std::vector<asset_property> {
const auto count = reader.read_u16();
std::vector<asset_property> properties;
properties.reserve(count);
for (std::uint16_t i = 0; i < count; ++i) properties.push_back(read_property(reader, names));
return properties;
}
inline auto write_property_list(byte_writer &writer, const std::vector<asset_property> &properties, name_table &names) -> void {
writer.write_u16(static_cast<std::uint16_t>(properties.size()));
for (const auto &property: properties) write_property(writer, property, names);
}
/** Split `Player_<n>_Start` into `n`, or 0 when the name does not match. */
[[nodiscard]] inline auto parse_player_start_name(std::string_view name) -> int {
constexpr std::string_view prefix = "Player_";
constexpr std::string_view suffix = "_Start";
if (!name.starts_with(prefix) || !name.ends_with(suffix)) return 0;
const auto digits = name.substr(prefix.size(), name.size() - prefix.size() - suffix.size());
if (digits.empty() || !std::ranges::all_of(digits, [](char ch) { return ch >= '0' && ch <= '9'; })) return 0;
return std::stoi(std::string{digits});
}
}
/**
* A parsed `.map`: the asset-name table plus the ordered chunk list.
*/
class map_document {
public:
/** Unwrap (BIG payload / `EAR\0` / bare RefPack / bare `CkMp`) and parse. */
[[nodiscard]] static auto parse(std::span<const std::byte> file) -> map_document {
auto data = std::vector<std::byte>{file.begin(), file.end()};
if (is_refpack(data)) data = refpack_decompress(data); // a BIG payload
if (detail::starts_with(data, std::string_view{"EAR\0", 4})) {
const auto *body = reinterpret_cast<const std::byte *>(data.data()) + 8;
std::span<const std::byte> payload{body, data.size() - 8U};
data = maybe_decompress(payload);
} else if (is_refpack(data)) {
data = refpack_decompress(data);
}
return map_document::from_ckmp(data);
}
[[nodiscard]] static auto open(const std::filesystem::path &path) -> map_document {
return map_document::parse(read_file(path));
}
[[nodiscard]] auto names() const -> const name_table & { return this->names_; }
[[nodiscard]] auto names() -> name_table & { return this->names_; }
[[nodiscard]] auto chunks() const -> const std::vector<map_chunk> & { return this->chunks_; }
[[nodiscard]] auto chunks() -> std::vector<map_chunk> & { return this->chunks_; }
[[nodiscard]] auto find_chunk(std::string_view name) const -> const map_chunk * {
for (const auto &chunk: this->chunks_) {
if (chunk.name == name) return &chunk;
}
return nullptr;
}
[[nodiscard]] auto find_chunk(std::string_view name) -> map_chunk * {
for (auto &chunk: this->chunks_) {
if (chunk.name == name) return &chunk;
}
return nullptr;
}
[[nodiscard]] auto has_chunk(std::string_view name) const -> bool { return this->find_chunk(name) != nullptr; }
/** The chunk of a known kind, or `nullptr`. */
[[nodiscard]] auto find_chunk(chunk_kind kind) const -> const map_chunk * {
for (const auto &chunk: this->chunks_) {
if (chunk.kind == kind) return &chunk;
}
return nullptr;
}
[[nodiscard]] auto find_chunk(chunk_kind kind) -> map_chunk * {
for (auto &chunk: this->chunks_) {
if (chunk.kind == kind) return &chunk;
}
return nullptr;
}
[[nodiscard]] auto has_chunk(chunk_kind kind) const -> bool { return this->find_chunk(kind) != nullptr; }
// -- typed chunks ---------------------------------------------------
/** The terrain grid, or `std::nullopt` when the map has no `HeightMapData`. */
[[nodiscard]] auto height_map() const -> std::optional<height_map_data> {
const auto *chunk = this->find_chunk("HeightMapData");
if (chunk == nullptr) return std::nullopt;
return height_map_data::parse(chunk->version, chunk->payload);
}
/** Replace (or create) the `HeightMapData` chunk from a grid. */
auto set_height_map(height_map_data data) -> void {
auto *chunk = this->find_chunk("HeightMapData");
if (chunk == nullptr) {
chunk = &this->chunks_.emplace_back();
chunk->name = "HeightMapData";
chunk->kind = chunk_kind::height_map_data;
chunk->asset_index = this->names_.get_or_create("HeightMapData");
chunk->version = data.version;
} else {
data.version = chunk->version; // keep the file's on-disk version
}
chunk->payload = data.serialize();
}
/** Every object the map places. */
[[nodiscard]] auto objects() const -> std::vector<map_object> {
const auto *chunk = this->find_chunk("ObjectsList");
if (chunk == nullptr) return {};
return map_document::decode_objects(this->names_, chunk->payload);
}
/** Replace (or create) the `ObjectsList` chunk. */
auto set_objects(const std::vector<map_object> &objects) -> void {
auto *chunk = this->find_chunk("ObjectsList");
if (chunk == nullptr) {
chunk = &this->chunks_.emplace_back();
chunk->name = "ObjectsList";
chunk->kind = chunk_kind::objects_list;
chunk->asset_index = this->names_.get_or_create("ObjectsList");
chunk->version = 1;
}
chunk->payload = map_document::encode_objects(this->names_, objects);
}
/** The `Player_N_Start` waypoints, ordered by index. */
[[nodiscard]] auto player_starts() const -> std::vector<player_start> {
std::vector<player_start> starts;
for (const auto &object: this->objects()) {
if (object.type_name != "*Waypoints/Waypoint") continue;
const auto *name_property = object.property("waypointName");
if (name_property == nullptr) continue;
const auto index = detail::parse_player_start_name(name_property->as_ascii());
if (index == 0) continue;
starts.push_back({index, object.position});
}
std::ranges::sort(starts, {}, &player_start::index);
return starts;
}
[[nodiscard]] auto mp_positions() const -> std::vector<mp_position> {
const auto *chunk = this->find_chunk("MPPositionList");
if (chunk == nullptr) return {};
return map_document::decode_mp_positions(chunk->payload);
}
auto set_mp_positions(const std::vector<mp_position> &positions) -> void {
auto *chunk = this->find_chunk("MPPositionList");
if (chunk == nullptr) {
chunk = &this->chunks_.emplace_back();
chunk->name = "MPPositionList";
chunk->kind = chunk_kind::mp_position_list;
chunk->asset_index = this->names_.get_or_create("MPPositionList");
chunk->version = 1;
}
chunk->payload = map_document::encode_mp_positions(this->names_, positions);
}
/** The `WorldInfo` property list. */
[[nodiscard]] auto world_info() const -> std::vector<asset_property> {
const auto *chunk = this->find_chunk("WorldInfo");
if (chunk == nullptr) return {};
byte_reader reader{chunk->payload};
return detail::parse_property_list(reader, this->names_);
}
auto set_world_info(const std::vector<asset_property> &properties) -> void {
auto *chunk = this->find_chunk("WorldInfo");
if (chunk == nullptr) {
chunk = &this->chunks_.emplace_back();
chunk->name = "WorldInfo";
chunk->kind = chunk_kind::world_info;
chunk->asset_index = this->names_.get_or_create("WorldInfo");
chunk->version = 1;
}
byte_writer writer;
detail::write_property_list(writer, properties, this->names_);
chunk->payload = writer.take();
}
/** The waypoint paths (`WaypointsList`). */
[[nodiscard]] auto waypoint_paths() const -> std::vector<waypoint_path> {
const auto *chunk = this->find_chunk("WaypointsList");
if (chunk == nullptr) return {};
byte_reader reader{chunk->payload};
const auto count = reader.read_u32();
std::vector<waypoint_path> paths;
paths.reserve(count);
for (std::uint32_t i = 0; i < count; ++i) paths.push_back({reader.read_i32(), reader.read_i32()});
return paths;
}
// -- serialisation --------------------------------------------------
/** The bare `CkMp` byte image. */
[[nodiscard]] auto to_ckmp() const -> std::vector<std::byte> {
byte_writer writer;
writer.write_ascii("CkMp");
const auto count = static_cast<std::uint32_t>(this->names_.size());
writer.write_u32(count);
for (std::uint32_t i = count; i >= 1U; --i) {
const auto &name = this->names_.entries()[i];
writer.write_u8(static_cast<std::uint8_t>(name.size()));
writer.write_ascii(name);
writer.write_u32(i);
}
for (const auto &chunk: this->chunks_) {
writer.write_u32(chunk.asset_index);
writer.write_u16(chunk.version);
writer.write_u32(static_cast<std::uint32_t>(chunk.payload.size()));
writer.write_bytes(chunk.payload);
}
return writer.take();
}
/**
* A standalone `.map` image.
*
* When `compress` is true the result is `EAR\0` + `u32` decompressed size
* + a RefPack stream (the retail compression). When false it is a bare
* `CkMp` tree. To store a map inside a `BIG4`, pass the *uncompressed*
* image and let `big_writer::add(..., compress = true)` wrap it once.
*/
[[nodiscard]] auto serialize(bool compress = false) const -> std::vector<std::byte> {
auto ckmp = this->to_ckmp();
if (!compress) return ckmp;
byte_writer writer;
writer.write_ascii(std::string_view{"EAR\0", 4});
writer.write_u32(static_cast<std::uint32_t>(ckmp.size()));
const auto packed = refpack_compress(ckmp);
writer.write_bytes(packed);
return writer.take();
}
/** True when the name table carries `name` (top-level chunk, property, ...). */
[[nodiscard]] auto has_name(std::string_view name) const -> bool {
for (std::uint32_t i = 1; i < this->names_.entries().size(); ++i) {
if (this->names_.entries()[i] == name) return true;
}
return false;
}
private:
map_document() = default;
[[nodiscard]] static auto from_ckmp(std::span<const std::byte> data) -> map_document {
if (data.size() < 8U || !detail::starts_with(data, "CkMp")) throw map_error("not a CkMp map");
byte_reader reader{data};
(void) reader.read_ascii(4);
const auto count = reader.read_u32();
map_document document;
document.names_.resize(count + 1U);
for (std::uint32_t i = count; i >= 1U; --i) {
const auto length = reader.read_u8();
auto name = reader.read_ascii(length);
const auto index = reader.read_u32();
if (index != i) throw map_error("asset name table index mismatch");
document.names_.set_name(i, std::move(name));
}
while (reader.remaining() >= 10U) {
const auto index = reader.read_u32();
const auto version = reader.read_u16();
const auto size = reader.read_u32();
if (index >= document.names_.entries().size() || reader.remaining() < size) throw map_error("truncated CkMp chunk");
map_chunk chunk;
chunk.asset_index = index;
chunk.name = std::string{document.names_.entries()[index]};
chunk.kind = chunk_kind_of(chunk.name);
chunk.version = version;
const auto payload = reader.read_bytes(size);
chunk.payload.assign(payload.begin(), payload.end());
document.chunks_.push_back(std::move(chunk));
}
return document;
}
[[nodiscard]] static auto decode_objects(const name_table &names, std::span<const std::byte> payload) -> std::vector<map_object> {
byte_reader reader{payload};
std::vector<map_object> objects;
while (reader.remaining() >= 10U) {
(void) reader.read_u32(); // nested asset index (always "Object")
const auto version = reader.read_u16();
const auto size = reader.read_u32();
if (reader.remaining() < size) throw map_error("truncated Object asset");
const auto body = reader.read_bytes(size);
byte_reader body_reader{body};
map_object object;
object.version = version;
object.position.x = body_reader.read_f32();
object.position.y = body_reader.read_f32();
object.position.z = body_reader.read_f32();
object.angle = body_reader.read_f32();
object.road = static_cast<road_type>(body_reader.read_u32());
object.type_name = body_reader.read_ascii(body_reader.read_u16());
object.properties = detail::parse_property_list(body_reader, names);
objects.push_back(std::move(object));
}
return objects;
}
[[nodiscard]] static auto encode_objects(name_table &names, const std::vector<map_object> &objects) -> std::vector<std::byte> {
const auto object_index = names.get_or_create("Object");
byte_writer writer;
for (const auto &object: objects) {
byte_writer body;
body.write_f32(object.position.x);
body.write_f32(object.position.y);
body.write_f32(object.position.z);
body.write_f32(object.angle);
body.write_u32(static_cast<std::uint32_t>(object.road));
body.write_u16(static_cast<std::uint16_t>(object.type_name.size()));
body.write_ascii(object.type_name);
detail::write_property_list(body, object.properties, names);
writer.write_u32(object_index);
writer.write_u16(object.version);
writer.write_u32(static_cast<std::uint32_t>(body.size()));
writer.write_bytes(body.data());
}
return writer.take();
}
[[nodiscard]] static auto decode_mp_positions(std::span<const std::byte> payload) -> std::vector<mp_position> {
byte_reader reader{payload};
std::vector<mp_position> positions;
while (reader.remaining() >= 10U) {
(void) reader.read_u32(); // nested asset index (always "MPPositionInfo")
const auto version = reader.read_u16();
const auto size = reader.read_u32();
if (reader.remaining() < size) throw map_error("truncated MPPositionInfo asset");
const auto body = reader.read_bytes(size);
byte_reader body_reader{body};
mp_position position;
position.version = version;
position.is_human = body_reader.read_bool();
position.is_computer = body_reader.read_bool();
if (version > 0U) position.load_ai_script = body_reader.read_bool();
position.team = body_reader.read_u32();
if (version > 0U) {
const auto count = body_reader.read_u32();
position.side_restrictions.reserve(count);
for (std::uint32_t i = 0; i < count; ++i) position.side_restrictions.push_back(body_reader.read_u16_prefixed_ascii());
}
positions.push_back(std::move(position));
}
return positions;
}
[[nodiscard]] static auto encode_mp_positions(name_table &names, const std::vector<mp_position> &positions) -> std::vector<std::byte> {
const auto info_index = names.get_or_create("MPPositionInfo");
byte_writer writer;
for (const auto &position: positions) {
byte_writer body;
body.write_bool(position.is_human);
body.write_bool(position.is_computer);
if (position.version > 0U) body.write_bool(position.load_ai_script);
body.write_u32(position.team);
if (position.version > 0U) {
body.write_u32(static_cast<std::uint32_t>(position.side_restrictions.size()));
for (const auto &side: position.side_restrictions) body.write_u16_prefixed_ascii(side);
}
writer.write_u32(info_index);
writer.write_u16(position.version);
writer.write_u32(static_cast<std::uint32_t>(body.size()));
writer.write_bytes(body.data());
}
return writer.take();
}
name_table names_;
std::vector<map_chunk> chunks_;
};
} // namespace ra3::assets
+284
View File
@@ -0,0 +1,284 @@
/**
* EA's RefPack compression codec.
*
* Red Alert 3 compresses individual `BIG4` payloads and every SAGE `.map` with
* RefPack (the `10 FB` stream shared across EA titles). This partition decodes
* and encodes that stream. The decoder is a port of the reference used across
* the project (`ra3tools/ra3_big.py`, `OpenRA3`'s `ra3.fs`); the encoder is a
* greedy LZ77 matcher that emits only canonical tokens, so anything it produces
* is readable by the same decoder (and by the game).
*/
export module ra3.assets:refpack;
import std;
import :bytes;
import :error;
export namespace ra3::assets {
/** Max back-reference distance (the 17-bit RefPack window). */
inline constexpr std::size_t refpack_window = 1U << 17U;
/** True when `data` starts with a RefPack header (`0b??010000`, `0xFB`). */
[[nodiscard]] inline auto is_refpack(std::span<const std::byte> data) -> bool {
return data.size() >= 2U && (std::to_integer<std::uint8_t>(data[0]) & 0x3EU) == 0x10U && std::to_integer<std::uint8_t>(data[1]) == 0xFBU;
}
/** Read the declared output size from a RefPack header without decompressing. */
[[nodiscard]] inline auto refpack_output_size(std::span<const std::byte> data) -> std::uint32_t {
if (!is_refpack(data)) throw refpack_error("not a RefPack stream");
std::size_t pos = 0;
const auto header = std::to_integer<std::uint8_t>(data[pos++]);
const bool large_files = (header & 0x80U) != 0U;
const bool compressed_size_present = (header & 0x01U) != 0U;
pos++; // 0xFB
const std::size_t size_bytes = large_files ? 4U : 3U;
const auto read_size = [&]() -> std::uint32_t {
std::uint32_t value = 0;
for (std::size_t i = 0; i < size_bytes; ++i) {
if (pos >= data.size()) throw refpack_error("truncated RefPack size field");
value = (value << 8U) | std::to_integer<std::uint8_t>(data[pos++]);
}
return value;
};
if (compressed_size_present) (void) read_size();
return read_size();
}
/**
* Decompress an EA RefPack stream.
*
* @throws refpack_error if the stream is malformed or the length disagrees.
*/
[[nodiscard]] inline auto refpack_decompress(std::span<const std::byte> data) -> std::vector<std::byte> {
if (!is_refpack(data)) throw refpack_error("not a RefPack stream");
std::size_t pos = 0;
const auto header = std::to_integer<std::uint8_t>(data[pos++]);
const bool large_files = (header & 0x80U) != 0U;
const bool compressed_size_present = (header & 0x01U) != 0U;
pos++; // 0xFB
const std::size_t size_bytes = large_files ? 4U : 3U;
const auto read_size = [&]() -> std::uint32_t {
std::uint32_t value = 0;
for (std::size_t i = 0; i < size_bytes; ++i) {
if (pos >= data.size()) throw refpack_error("truncated RefPack size field");
value = (value << 8U) | std::to_integer<std::uint8_t>(data[pos++]);
}
return value;
};
if (compressed_size_present) (void) read_size();
const auto out_len = read_size();
std::vector<std::byte> out;
out.reserve(out_len); // guarantees no reallocation, so overlapping reads stay valid
const auto copy_literals = [&](std::size_t count) {
if (pos + count > data.size()) throw refpack_error("truncated RefPack literals");
out.insert(out.end(), data.begin() + static_cast<std::ptrdiff_t>(pos), data.begin() + static_cast<std::ptrdiff_t>(pos + count));
pos += count;
};
// A back-reference may overlap its own output (an RLE run): out[start + i]
// is read one byte at a time, so the pattern repeats correctly.
const auto copy_reference = [&](std::size_t length, std::size_t distance) {
if (distance == 0U || distance > out.size()) throw refpack_error("RefPack back-reference out of range");
const auto start = out.size() - distance;
if (out.size() + length > out_len) throw refpack_error("RefPack output overrun");
for (std::size_t i = 0; i < length; ++i) out.push_back(out[start + i]);
};
while (pos < data.size()) {
const auto cmd = std::to_integer<std::uint8_t>(data[pos++]);
if ((cmd & 0x80U) == 0U) { // 2-byte command
if (pos >= data.size()) throw refpack_error("truncated 2-byte command");
const auto b2 = std::to_integer<std::uint8_t>(data[pos++]);
copy_literals(cmd & 0x03U);
copy_reference(((cmd & 0x1CU) >> 2U) + 3U, ((cmd & 0x60U) << 3U) + b2 + 1U);
} else if ((cmd & 0x40U) == 0U) { // 3-byte command
if (pos + 1U >= data.size()) throw refpack_error("truncated 3-byte command");
const auto b2 = std::to_integer<std::uint8_t>(data[pos]);
const auto b3 = std::to_integer<std::uint8_t>(data[pos + 1U]);
pos += 2U;
copy_literals((b2 & 0xC0U) >> 6U);
copy_reference((cmd & 0x3FU) + 4U, ((b2 & 0x3FU) << 8U) + b3 + 1U);
} else if ((cmd & 0x20U) == 0U) { // 4-byte command
if (pos + 2U >= data.size()) throw refpack_error("truncated 4-byte command");
const auto b2 = std::to_integer<std::uint8_t>(data[pos]);
const auto b3 = std::to_integer<std::uint8_t>(data[pos + 1U]);
const auto b4 = std::to_integer<std::uint8_t>(data[pos + 2U]);
pos += 3U;
copy_literals(cmd & 0x03U);
copy_reference(((cmd & 0x0CU) << 6U) + b4 + 5U, ((cmd & 0x10U) << 12U) + (static_cast<std::size_t>(b2) << 8U) + b3 + 1U);
} else if (cmd < 0xFCU) { // long literal run
copy_literals((static_cast<std::size_t>(cmd & 0x1FU) + 1U) << 2U);
} else { // stop
copy_literals(cmd & 0x03U);
break;
}
}
if (out.size() != out_len) throw refpack_error("RefPack length mismatch");
return out;
}
/** Decompress `data` when it is RefPack, otherwise copy it unchanged. */
[[nodiscard]] inline auto maybe_decompress(std::span<const std::byte> data) -> std::vector<std::byte> {
if (is_refpack(data)) return refpack_decompress(data);
return {data.begin(), data.end()};
}
namespace detail {
/** True when `(length, distance)` maps to one of the three canonical tokens. */
[[nodiscard]] inline constexpr auto refpack_match_encodable(std::size_t length, std::size_t distance) -> bool {
if (length >= 3U && length <= 10U && distance >= 1U && distance <= 1024U) return true;
if (length >= 4U && length <= 67U && distance >= 1U && distance <= 16384U) return true;
if (length >= 5U && length <= 1028U && distance >= 1U && distance <= 131072U) return true;
return false;
}
/** Greedy RefPack encoder state. */
class refpack_encoder {
public:
explicit refpack_encoder(std::span<const std::byte> input)
: input_(input), chain_(input.size(), -1) {
}
[[nodiscard]] auto run() -> std::vector<std::byte> {
const auto n = this->input_.size();
const bool large = n >= (1U << 24U);
this->out_.write_u8(static_cast<std::uint8_t>(0x10U | (large ? 0x80U : 0x00U)));
this->out_.write_u8(0xFBU);
const std::size_t size_bytes = large ? 4U : 3U;
for (std::size_t i = size_bytes; i-- > 0U;) this->out_.write_u8(static_cast<std::uint8_t>((n >> (8U * i)) & 0xFFU));
std::size_t pos = 0;
std::size_t literals_start = 0;
while (pos < n) {
std::size_t best_len = 0;
std::size_t best_dist = 0;
if (pos + 2U < n) this->find_match(pos, best_len, best_dist);
if (best_len >= 3U) {
const auto pending = pos - literals_start;
const auto carry = pending % 4U; // 0..3 literals ride with the token
const auto run = pending - carry; // always a multiple of 4
this->emit_literal_run(literals_start, run);
this->emit_match(literals_start + run, carry, best_len, best_dist);
for (std::size_t i = pos; i < pos + best_len; ++i) this->insert(i);
pos += best_len;
literals_start = pos;
} else {
this->insert(pos);
++pos;
}
}
const auto pending = n - literals_start;
const auto carry = pending % 4U;
this->emit_literal_run(literals_start, pending - carry);
this->out_.write_u8(static_cast<std::uint8_t>(0xFCU | carry));
this->write_literals(literals_start + (pending - carry), carry);
return this->out_.take();
}
private:
void insert(std::size_t pos) {
if (pos + 2U >= this->input_.size()) return;
const auto bucket = this->hash3(pos);
this->chain_[pos] = this->head_[bucket];
this->head_[bucket] = static_cast<std::int32_t>(pos);
}
void find_match(std::size_t pos, std::size_t &best_len, std::size_t &best_dist) const {
const auto n = this->input_.size();
const auto max_len = std::min<std::size_t>(1028U, n - pos);
auto candidate = this->head_[this->hash3(pos)];
int depth = 0;
while (candidate >= 0 && depth < 64) {
const auto c = static_cast<std::size_t>(candidate);
const auto distance = pos - c;
if (distance > refpack_window) break; // the chain only walks backwards
std::size_t length = 0;
while (length < max_len && this->input_[c + length] == this->input_[pos + length]) ++length;
if (length > best_len && refpack_match_encodable(length, distance)) {
best_len = length;
best_dist = distance;
if (length == max_len) break;
}
candidate = this->chain_[c];
++depth;
}
}
void emit_literal_run(std::size_t offset, std::size_t count) {
std::size_t remaining = count;
std::size_t at = offset;
while (remaining >= 4U) {
std::size_t step = std::min<std::size_t>(112U, remaining);
step -= step % 4U;
if (step < 4U) step = 4U;
this->out_.write_u8(static_cast<std::uint8_t>(0xE0U | ((step / 4U) - 1U)));
this->write_literals(at, step);
at += step;
remaining -= step;
}
}
void emit_match(std::size_t literal_offset, std::size_t literal_count, std::size_t length, std::size_t distance) {
const auto d = static_cast<std::uint32_t>(distance - 1U);
if (length >= 3U && length <= 10U && distance <= 1024U) { // 2-byte token
const auto cmd = static_cast<std::uint8_t>((((d >> 8U) & 0x03U) << 5U) | (static_cast<std::uint32_t>(length - 3U) << 2U) |
static_cast<std::uint32_t>(literal_count));
this->out_.write_u8(cmd);
this->out_.write_u8(static_cast<std::uint8_t>(d & 0xFFU));
} else if (length >= 4U && length <= 67U && distance <= 16384U) { // 3-byte token
this->out_.write_u8(static_cast<std::uint8_t>(0x80U | (length - 4U)));
this->out_.write_u8(static_cast<std::uint8_t>((literal_count << 6U) | ((d >> 8U) & 0x3FU)));
this->out_.write_u8(static_cast<std::uint8_t>(d & 0xFFU));
} else if (length >= 5U && length <= 1028U && distance <= 131072U) { // 4-byte token
const auto l = static_cast<std::uint32_t>(length - 5U);
const auto cmd = static_cast<std::uint8_t>(0xC0U | (((l >> 8U) & 0x03U) << 2U) | ((d >> 12U) & 0x10U) |
static_cast<std::uint32_t>(literal_count));
this->out_.write_u8(cmd);
this->out_.write_u8(static_cast<std::uint8_t>((d >> 8U) & 0xFFU));
this->out_.write_u8(static_cast<std::uint8_t>(d & 0xFFU));
this->out_.write_u8(static_cast<std::uint8_t>(l & 0xFFU));
} else {
throw refpack_error("internal error: unencodable RefPack match");
}
this->write_literals(literal_offset, literal_count);
}
void write_literals(std::size_t offset, std::size_t count) {
for (std::size_t i = 0; i < count; ++i) this->out_.write_u8(std::to_integer<std::uint8_t>(this->input_[offset + i]));
}
[[nodiscard]] auto hash3(std::size_t pos) const -> std::size_t {
const auto a = static_cast<std::uint32_t>(std::to_integer<std::uint8_t>(this->input_[pos]));
const auto b = static_cast<std::uint32_t>(std::to_integer<std::uint8_t>(this->input_[pos + 1U]));
const auto c = static_cast<std::uint32_t>(std::to_integer<std::uint8_t>(this->input_[pos + 2U]));
return ((a | (b << 8U) | (c << 16U)) * 2654435761U) >> (32U - 16U);
}
std::span<const std::byte> input_;
std::vector<std::int32_t> chain_;
std::vector<std::int32_t> head_ = std::vector<std::int32_t>(1U << 16U, -1);
byte_writer out_;
};
}
/**
* Compress a byte range into a canonical RefPack stream.
*
* The encoder is a plain greedy LZ77: it never emits a token form the
* decoder above cannot read, so `refpack_decompress(refpack_compress(x))`
* is lossless for every input.
*/
[[nodiscard]] inline auto refpack_compress(std::span<const std::byte> input) -> std::vector<std::byte> {
return detail::refpack_encoder{input}.run();
}
} // namespace ra3::assets
+310
View File
@@ -0,0 +1,310 @@
#include "test_main.hpp"
import std;
import ra3.assets;
using namespace ra3::assets;
namespace {
auto bytes_of(std::string_view text) -> std::vector<std::byte> {
std::vector<std::byte> out;
out.reserve(text.size());
for (const auto ch: text) out.push_back(static_cast<std::byte>(static_cast<unsigned char>(ch)));
return out;
}
auto text_of(std::span<const std::byte> bytes) -> std::string {
return std::string{reinterpret_cast<const char *>(bytes.data()), bytes.size()};
}
/** A deterministic pseudo-random buffer (no external RNG needed). */
auto pseudo_random(std::size_t size, std::uint32_t seed) -> std::vector<std::byte> {
std::vector<std::byte> out;
out.reserve(size);
std::uint32_t state = seed;
for (std::size_t i = 0; i < size; ++i) {
state = state * 1664525U + 1013904223U;
out.push_back(static_cast<std::byte>((state >> 16U) & 0xFFU));
}
return out;
}
auto make_empty_ckmp(const std::vector<std::string> &names) -> std::vector<std::byte> {
byte_writer writer;
writer.write_ascii("CkMp");
writer.write_u32(static_cast<std::uint32_t>(names.size()));
for (std::uint32_t i = static_cast<std::uint32_t>(names.size()); i >= 1U; --i) {
writer.write_u8(static_cast<std::uint8_t>(names[i - 1].size()));
writer.write_ascii(names[i - 1]);
writer.write_u32(i);
}
return writer.take();
}
auto make_csf() -> std::vector<std::byte> {
byte_writer writer;
writer.write_ascii(" FSC");
writer.write_u32(3);
writer.write_u32(2); // labels
writer.write_u32(2); // value blocks
writer.write_u32(0);
writer.write_u32(0);
writer.write_u32(csf_label_flag);
writer.write_u32(1);
writer.write_u32(3);
writer.write_ascii("ABC");
writer.write_u32(csf_value_flag);
writer.write_u32(5);
for (const auto ch: std::string_view{"Hello"}) writer.write_u16(static_cast<std::uint16_t>(ch) ^ 0xFFFFU);
writer.write_u32(csf_label_flag);
writer.write_u32(1);
writer.write_u32(3);
writer.write_ascii("UNI");
writer.write_u32(csf_value_flag);
writer.write_u32(1);
writer.write_u16(0x4E2DU ^ 0xFFFFU); // U+4E2D (CJK)
return writer.take();
}
} // namespace
TEST(refpack_roundtrip_small) {
const std::vector<std::string> samples{"", "A", "AB", "ABC", "ABCD", "abcabcabc", std::string(300, 'z')};
for (const auto &sample: samples) {
const auto input = bytes_of(sample);
const auto packed = refpack_compress(input);
const auto restored = refpack_decompress(packed);
CHECK_EQ(restored.size(), input.size());
CHECK(std::ranges::equal(restored, input));
}
}
TEST(refpack_roundtrip_large_and_repetitive) {
std::vector<std::byte> input;
for (int i = 0; i < 2000; ++i) {
for (const auto ch: std::string_view{"lorem ipsum dolor sit amet "}) input.push_back(static_cast<std::byte>(ch));
}
const auto random = pseudo_random(50000, 0x1234ABCDU);
input.insert(input.end(), random.begin(), random.end());
const auto packed = refpack_compress(input);
CHECK(packed.size() < input.size()); // the repeated prefix must compress
CHECK(std::ranges::equal(refpack_decompress(packed), input));
}
TEST(refpack_decode_known_stream) {
// "abcd" literals, then a back-reference (distance 4, length 3) -> "abcdabc".
const std::vector<std::byte> stream{
std::byte{0x10}, std::byte{0xFB}, std::byte{0x00}, std::byte{0x00}, std::byte{0x07}, std::byte{0xE0},
std::byte{'a'}, std::byte{'b'}, std::byte{'c'}, std::byte{'d'}, std::byte{0x00}, std::byte{0x03},
std::byte{0xFC},
};
CHECK(is_refpack(stream));
CHECK_EQ(text_of(refpack_decompress(stream)), "abcdabc");
CHECK_EQ(refpack_output_size(stream), 7U);
}
TEST(big_roundtrip) {
big_writer writer;
writer.add("data\\a.txt", bytes_of("hello world"));
writer.add("data\\b.bin", bytes_of(std::string(1000, 'x')), true);
const auto image = writer.write();
CHECK(image.size() >= 16U);
const std::string magic{reinterpret_cast<const char *>(image.data()), 4};
CHECK_EQ(magic, "BIG4");
const auto archive = big_archive::from_bytes(image);
CHECK_EQ(archive.size(), 2U);
CHECK(archive.contains("data\\a.txt"));
CHECK_EQ(text_of(archive.read("data\\a.txt")), "hello world");
CHECK_EQ(archive.read("data\\b.bin").size(), 1000U);
CHECK_EQ(archive.find("data\\").size(), 2U);
// The first payload starts 64-byte aligned, as the retail archives do.
CHECK_EQ(archive.entries()[0].offset % 64U, 0U);
}
TEST(big_open_from_disk_is_lazy) {
big_writer writer;
writer.add("data\\a.txt", bytes_of("hello disk"));
writer.add("data\\b.bin", bytes_of(std::string(4096, 'q')), true);
const auto image = writer.write();
const auto path = std::filesystem::temp_directory_path() / "libra3assets_big_lazy.big";
write_file(path, image);
const auto archive = big_archive::open(path);
CHECK_EQ(archive.size(), 2U);
CHECK(archive.path() == path);
CHECK(archive.contains("data\\a.txt"));
CHECK_EQ(archive.entries()[0].offset % 64U, 0U);
CHECK_EQ(text_of(archive.read("data\\a.txt")), "hello disk");
CHECK_EQ(archive.read("data\\b.bin").size(), 4096U);
CHECK_EQ(text_of(archive.read_prefix("data\\a.txt", 5)), "hello");
CHECK(archive.read_prefix("data\\b.bin", 8U).size() == 8U);
std::error_code ec;
std::filesystem::remove(path, ec);
}
TEST(binary_manifest_and_hash) {
CHECK_EQ(hash_string("W3DMesh"), 0xC2B1A262U);
CHECK_EQ(hash_string("ABAIRFIELD", false), 0x2B479BD3U);
CHECK(hash_string("Texture") != hash_string("Texture", false));
const std::string name = "W3DMesh:TEST";
const std::string source = "art:test.w3x";
byte_writer manifest;
manifest.write_u8(0); // not big-endian
manifest.write_u8(0); // not linked
manifest.write_u16(5); // version
manifest.write_u32(0); // checksum
manifest.write_u32(0); // all-types hash
manifest.write_u32(1); // count
manifest.write_u32(12);
manifest.write_u32(0);
manifest.write_u32(0);
manifest.write_u32(0);
manifest.write_u32(0); // reference buffer
manifest.write_u32(0); // reference-name buffer
manifest.write_u32(static_cast<std::uint32_t>(name.size() + 1));
manifest.write_u32(static_cast<std::uint32_t>(source.size() + 1));
manifest.write_u32(hash_string("W3DMesh"));
manifest.write_u32(hash_string("TEST", false));
manifest.write_u32(hash_string("W3DMesh"));
manifest.write_u32(hash_string("TEST", false));
manifest.write_i32(0); // reference offset
manifest.write_i32(0); // reference count
manifest.write_i32(0); // name offset
manifest.write_i32(0); // source offset
manifest.write_u32(8); // instance size
manifest.write_u32(0); // relocation size
manifest.write_u32(0); // imports size
manifest.write_u32(0); // tokenized
manifest.write_ascii(name);
manifest.write_u8(0);
manifest.write_ascii(source);
manifest.write_u8(0);
byte_writer data;
data.write_u32(0); // stream checksum
data.write_ascii("PAYLOAD!");
auto container = binary_container::from_bytes(manifest.take(), data.take());
CHECK_EQ(container.size(), 1U);
const auto &asset = container.assets()[0];
CHECK_EQ(asset.name, "W3DMesh:TEST");
CHECK_EQ(asset.type_name(), "W3DMesh");
CHECK_EQ(asset.instance_name(), "TEST");
CHECK_EQ(asset.source, "art:test.w3x");
CHECK_EQ(text_of(container.read_instance(asset)), "PAYLOAD!");
CHECK_EQ(container.find("#0").name, "W3DMesh:TEST");
CHECK_EQ(container.of_type("w3dmesh").size(), 1U);
CHECK_EQ(container.type_counts().at("W3DMesh"), 1U);
CHECK(container.read_relocation(asset).empty());
container.set_cdata_source([&asset](const std::string &cdata_name) -> std::optional<std::vector<std::byte>> {
if (cdata_name == asset.cdata_name("static")) return bytes_of("CDATA");
return std::nullopt;
});
CHECK(container.read_cdata(asset).has_value());
CHECK_EQ(text_of(container.read_payload(asset)), "CDATA");
}
TEST(csf_roundtrip) {
auto table = csf_table::parse(make_csf());
CHECK_EQ(table.size(), 2U);
CHECK_EQ(table.version(), 3U);
CHECK_EQ(table.lookup("abc"), "Hello"); // lookup is case-insensitive
CHECK_EQ(table.lookup("ABC"), "Hello");
CHECK_EQ(table.lookup("uni"), "\xE4\xB8\xAD");
CHECK(table.find("missing") == nullptr);
const auto rewritten = table.write();
const auto reparsed = csf_table::parse(rewritten);
CHECK_EQ(reparsed.size(), 2U);
CHECK_EQ(reparsed.lookup("abc"), "Hello");
CHECK_EQ(reparsed.lookup("uni"), "\xE4\xB8\xAD");
}
TEST(csf_decode_utf16_surrogate) {
const std::u16string emoji = u"\U0001F600";
CHECK_EQ(utf16_to_utf8(emoji), "\xF0\x9F\x98\x80");
}
TEST(map_roundtrip) {
const std::vector<std::string> names{"HeightMapData", "ObjectsList", "Object", "waypointName", "MPPositionList", "MPPositionInfo", "WorldInfo", "waypointID"};
auto document = map_document::parse(make_empty_ckmp(names));
CHECK_EQ(document.names().size(), names.size());
height_map_data height;
height.width = 2;
height.height = 2;
height.version = 6;
height.elevations = {1, 2, 3, 4};
document.set_height_map(height);
map_object waypoint;
waypoint.type_name = "*Waypoints/Waypoint";
waypoint.position = {100.0F, 200.0F, 0.0F};
waypoint.properties.push_back(asset_property::text("waypointName", "Player_1_Start"));
waypoint.properties.push_back(asset_property::integer("waypointID", 0));
document.set_objects({waypoint});
mp_position position;
position.is_human = true;
position.team = 1;
document.set_mp_positions({position});
const auto ckmp = document.to_ckmp();
const auto reparsed = map_document::parse(ckmp);
CHECK(reparsed.has_chunk(chunk_kind::height_map_data));
CHECK(reparsed.has_chunk(chunk_kind::objects_list));
CHECK(reparsed.has_chunk(chunk_kind::mp_position_list));
CHECK_EQ(reparsed.find_chunk("HeightMapData")->kind, chunk_kind::height_map_data);
CHECK_EQ(chunk_kind_of("BlendTileData"), chunk_kind::blend_tile_data);
CHECK_EQ(chunk_kind_of("NotAChunk"), chunk_kind::unknown);
const auto restored = reparsed.height_map();
CHECK(restored.has_value());
CHECK_EQ(restored->width, 2U);
CHECK_EQ(restored->at(1, 1), 4U);
CHECK_EQ(restored->vertical_scale(), 0.0390625F);
const auto objects = reparsed.objects();
CHECK_EQ(objects.size(), 1U);
CHECK_EQ(objects[0].type_name, "*Waypoints/Waypoint");
CHECK_EQ(objects[0].property("waypointID")->as_int(), 0);
const auto starts = reparsed.player_starts();
CHECK_EQ(starts.size(), 1U);
CHECK_EQ(starts[0].index, 1);
CHECK_EQ(starts[0].position.x, 100.0F);
const auto positions = reparsed.mp_positions();
CHECK_EQ(positions.size(), 1U);
CHECK(positions[0].is_human);
CHECK_EQ(positions[0].team, 1U);
}
TEST(map_compressed_and_big_payload) {
const std::vector<std::string> names{"HeightMapData"};
auto document = map_document::parse(make_empty_ckmp(names));
height_map_data height;
height.width = 1;
height.height = 1;
height.version = 6;
height.elevations = {7};
document.set_height_map(height);
// EAR\0 + RefPack round-trip.
const auto compressed = document.serialize(true);
const auto from_compressed = map_document::parse(compressed);
CHECK(from_compressed.height_map().has_value());
CHECK_EQ(from_compressed.height_map()->at(0, 0), 7U);
// A BIG payload is one more RefPack layer over the map file.
const auto big_payload = refpack_compress(document.to_ckmp());
const auto from_big = map_document::parse(big_payload);
CHECK_EQ(from_big.height_map()->at(0, 0), 7U);
}
+5
View File
@@ -0,0 +1,5 @@
#include "test_main.hpp"
int main() {
return ra3test::run_all();
}
+83
View File
@@ -0,0 +1,83 @@
#ifndef RA3TEST_HPP
#define RA3TEST_HPP
// A tiny dependency-free test harness (registry + CHECK macros). Each unit-test
// executable compiles test_main.cpp and links the sources under test.
#include <functional>
#include <iostream>
#include <sstream>
#include <string>
#include <vector>
namespace ra3test {
struct test_case {
std::string name;
std::function<void()> fn;
};
inline auto registry() -> std::vector<test_case> & {
static std::vector<test_case> tests;
return tests;
}
inline auto failure_count() -> int & {
static int count = 0;
return count;
}
struct registrar {
registrar(std::string name, std::function<void()> fn) { registry().push_back({std::move(name), std::move(fn)}); }
};
inline auto report_failure(const std::string &expr, const std::string &file, int line) -> void {
++failure_count();
std::cerr << " FAIL " << file << ':' << line << " " << expr << '\n';
}
inline auto check(bool condition, const std::string &expr, const std::string &file, int line) -> void {
if (!condition) report_failure(expr, file, line);
}
inline auto run_all() -> int {
int passed = 0;
for (auto &test: registry()) {
const int before = failure_count();
std::cout << "[ RUN ] " << test.name << '\n';
try {
test.fn();
} catch (const std::exception &exc) {
report_failure(std::string("uncaught exception: ") + exc.what(), __FILE__, __LINE__);
}
if (failure_count() == before) {
++passed;
std::cout << "[ OK ] " << test.name << '\n';
}
}
std::cout << "\n" << passed << '/' << registry().size() << " tests passed, " << failure_count() << " failure(s)\n";
return failure_count() == 0 ? 0 : 1;
}
} // namespace ra3test
#define TEST(name) \
static void name(); \
static ::ra3test::registrar ra3test_reg_##name(#name, name); \
static void name()
#define CHECK(cond) ::ra3test::check((cond), #cond, __FILE__, __LINE__)
#define CHECK_EQ(a, b) ::ra3test::check_eq((a), (b), #a, #b, __FILE__, __LINE__)
namespace ra3test {
template<typename A, typename B>
auto check_eq(const A &a, const B &b, const std::string &ea, const std::string &eb, const std::string &file, int line) -> void {
if (!(a == b)) {
std::ostringstream os;
os << ea << " == " << eb;
report_failure(os.str(), file, line);
}
}
}
#endif // RA3TEST_HPP