0.1.9
This commit is contained in:
@@ -16,6 +16,12 @@
|
||||
- `<Include source>` 补全可解析的 `DATA:` / `ART:` / `AUDIO:` 与项目相对路径。
|
||||
- **悬停提示**：元素/属性显示 XSD 文档、类型、必填/默认值；引用值显示定义位置；`$DEFINE` 显示值与定义位置；`Include source` / `xi:include href` 显示解析后的目标文件；`xi:include` 元素与属性给出 XInclude 说明。
|
||||
- **引用导航**：从引用值（`CommandSet="..."`、`Weapon="..."`、`inheritFrom`）跳转到定义（严格按引用类型过滤，候选由 `ra3modxml.definitionMode` 控制：`all` 列出 mod + 原版、`project-only` 优先项目内定义）；`Ctrl+点击` Include / `xi:include href` 打开目标文件；Find All References 搜索整个工作区；文档大纲列出顶层资产与 `$DEFINE`。
|
||||
- **当前文档局部作用域（T1）**：即使一个文件不在任何全局流里（没有从
|
||||
`Data/Mod.xml` / `additionalmaps` 可达），插件也会按当前文件自身的资产、
|
||||
`$DEFINE` 及其 include 链建立局部索引。`xi:include` 会在逻辑树中展开，
|
||||
使 include 进来的内容获得正确的父上下文；`AttachModuleId` / `ModuleId` /
|
||||
`AutoResolveBody` 等管线局部（Poid）引用可以补全、悬停与跳转到同一
|
||||
GameObject 内的模块（含通过 `xi:include` 拼入的兄弟模块）。
|
||||
- **错误检查**：XML 格式错误、未知元素/属性（`xi:` 等外来命名空间不误报）、顶层资产缺 `id`、重复 ID、未解析引用（含类型不匹配）、Include / 嵌套 `xi:include` 目标找不到、`$DEFINE` 未定义。
|
||||
- **manifest 支持**：`<Include type="reference">` 指向的 `static/global/audio.manifest`（SDK `builtmods`）会被解析，manifest 中的原版资产 ID 可用于补全/悬停/导航/诊断。
|
||||
- **美术资产（`.w3x`）**：`W3X.xml` / `ART:` include 链中的 `.w3x` 模型文件会被
|
||||
@@ -23,14 +29,24 @@
|
||||
`Model@Name`、`Hierarchy`、`Mesh` 等引用可以解析、悬停与跳转。超大模型
|
||||
（几十 MB 的顶点/三角形数据）采用浅扫描——只提取顶层资产记录、不建 DOM 树，
|
||||
结果在 workspace 级缓存并跨重建复用，保存文件触发的重建不会重读未变化的模型文件。
|
||||
- **索引分阶段与部分可用性**：先建立 XML + manifest 索引（首建早期即可用），
|
||||
w3x 美术资产随后台扫描补齐。索引完成前，语法/模型诊断、枚举与子元素补全、
|
||||
Include 跳转/悬停照常工作；引用类诊断会“显示但标注”
|
||||
（`unresolved-reference-indexing` + `(index incomplete)` 说明），不会把
|
||||
未完成的索引误当成最终结论。
|
||||
- **大项目性能**：索引记录（资产 / Define / Include / 行号）与 include 解析结果
|
||||
跨重建缓存，保存触发的重建零 stat、零重读（Corona 实测约 2 秒）；DOM 树只按需
|
||||
保留并设元素预算，避免内存膨胀。
|
||||
保留并设元素预算，避免内存膨胀。编辑器外的文件改动（git pull、导出工具）会
|
||||
触发防抖重建；构建期间文件再次被修改时，已发布索引会标记 `(stale)` 并自动重跑。
|
||||
include 路径解析使用目录枚举建立的文件集快照（无 statSync 风暴）；records
|
||||
缓存会持久化到磁盘（gzip + 多信号 stat 校验 + 原子写），重启 VS Code 后冷启动
|
||||
只需秒级校验，Corona 实测约 11 秒（首次全量约 2 分钟）。
|
||||
|
||||
## 使用
|
||||
|
||||
1. 用 VS Code 打开 RA3 Mod 项目文件夹（含 `Data/Mod.xml` 或 `mod.babproj`）。
|
||||
2. 插件自动激活并开始后台索引（状态栏显示资产数量）。
|
||||
2. 插件自动激活并开始后台索引（状态栏显示阶段与资产数量；扩展也会在打开
|
||||
任意 XML 文件时激活，非 RA3 工作区不会显示 RA3 专属功能）。
|
||||
3. 编辑任意 `*.xml` 即可获得补全、跳转与诊断。
|
||||
|
||||
### 设置（`settings.json`）
|
||||
@@ -48,6 +64,8 @@
|
||||
|
||||
- `RA3 Mod XML: Re-index workspace`：手动重建索引。
|
||||
- `RA3 Mod XML: Show index report`：查看索引统计。
|
||||
- `RA3 Mod XML: Clear caches and rebuild`：清空内存/磁盘缓存并强制全量重建。
|
||||
- `RA3 Mod XML: Show cache report`：查看磁盘缓存路径、大小、校验统计与命中数。
|
||||
|
||||
## 开发
|
||||
|
||||
@@ -77,14 +95,19 @@ src/
|
||||
schemaModel.ts XSD 模型运行时（schema-model.json / asset-types.json 由 tools 生成）
|
||||
indexer/
|
||||
includeResolver.ts Include 路径解析（纯 TS，移植 check_duplicate_ids.py）
|
||||
existence.ts 文件集存在性快照（目录枚举 Set，替代逐路径 statSync）
|
||||
manifestParser.ts .manifest 二进制解析（移植 OpenSAGE ManifestFile.cs）
|
||||
fileScanner.ts 目录扫描与 Include source 候选
|
||||
refs.ts 引用目标解析（按引用类型过滤）
|
||||
xpointer.ts xi:include xpointer 子集解析（纯 TS）
|
||||
logicalTree.ts 当前文档逻辑树（xi:include 拼接、局部作用域）
|
||||
localScope.ts 文档局部索引 overlay（自身链 + include 链）
|
||||
shallowScan.ts .w3x 等大体积美术资产顶层浅扫描（纯 TS，不建 DOM）
|
||||
records.ts 每文件紧凑索引记录（资产/Define/Include/xi + 行号）
|
||||
caches.ts 跨重建持久缓存（DocumentCache / IndexRecordsCache /
|
||||
IncludeResolveCache）
|
||||
indexer.ts 工作区索引器（后台、缓存、记录驱动重建）
|
||||
IncludeResolveCache）+ 失效纪元 InvalidationsEpoch
|
||||
diskCache.ts 跨会话磁盘缓存（gzip JSON、原子写、多信号 stat 校验）
|
||||
indexer.ts 工作区索引器（后台、缓存、记录驱动重建、分阶段发布）
|
||||
features/ completion / hover / navigation / diagnostics / semanticTokens
|
||||
syntaxes/ TextMate 注入语法
|
||||
tools/ XSD → 模型、AssetType 枚举提取
|
||||
|
||||
+305
-3
@@ -607,7 +607,8 @@ AttachTest `Harbinger Gunship\GameObject.xml` 中 `<Model Name="AUGunship_SKN"/>
|
||||
|
||||
两条是同一缺失定义的两种呈现（诊断 + hover），不是两个独立 bug。
|
||||
`W3DContainer:AUGunship_SKN` 确实由 mod 定义——但定义在
|
||||
`Harbinger Gunship\W3X\AUGUNSHIP_SKN.w3x` 里，而索引器只解析 `.xml` / `.manifestxml`。
|
||||
`Harbinger Gunship\W3X\AUGUNSHIP_SKN.w3x` 里，而索引器只解析 `.xml`
|
||||
（manifest 是 `*.manifest` 二进制，不存在 `.manifestxml` 源码格式）。
|
||||
|
||||
### 关键事实（实测）
|
||||
|
||||
@@ -644,7 +645,7 @@ AttachTest `Harbinger Gunship\GameObject.xml` 中 `<Model Name="AUGunship_SKN"/>
|
||||
单次线性扫描，只提取顶层元素 `name + id`（含精确 offset）、顶层 `<Includes>`
|
||||
的 `Include@type/source`、任意层级 `<xi:include>` 的 `href/xpointer`、`<Defines>`
|
||||
常量；注释 / CDATA / DOCTYPE / PI 整体跳过；属性解析兼容引号内 `>` 与 `/`。
|
||||
2. **索引模式三分**：`.xml`/`.manifestxml` 全量解析（4 MB 上限不变）；
|
||||
2. **索引模式三分**：`.xml` 全量解析（4 MB 上限不变）；
|
||||
`.w3x` 一律浅扫描；未知扩展名先嗅探文件头（512 字节，BOM/空白后以 `<` 开头且无
|
||||
NUL 字节 → 按 XML 浅扫描，否则按二进制仅登记）。w3x 自身的 `<Includes>` 与
|
||||
嵌套 `xi:include` 会继续被 walk（BAB 语义）。
|
||||
@@ -661,7 +662,8 @@ AttachTest `Harbinger Gunship\GameObject.xml` 中 `<Model Name="AUGunship_SKN"/>
|
||||
|
||||
- 单元测试 **53 → 63 全绿**：新增 `test/shallowScan.test.mjs`（顶层资产 / Includes /
|
||||
xi:include / Defines / 引号内 `>` / CDATA / 未闭合标签 / 数值载荷不产生记录）；
|
||||
indexer 新增 w3x 链、`.w3d` 嗅探、二进制 `.dds` 跳过、跨重建缓存命中、
|
||||
indexer 新增 w3x 链、未知扩展名 XML 嗅探（fixture 用 `.dat`）、二进制 `.dds`
|
||||
跳过、跨重建缓存命中、
|
||||
BOM 偏移断言；xmlParser 新增 `stripBom` 用例。
|
||||
- AttachTest 实机：
|
||||
- 首次构建 1.2 s，浅扫 62 个文件；`Model@Name=AUGunship_SKN` →
|
||||
@@ -762,3 +764,303 @@ resolve 缓存命中与"信任重建 0 次重解析"断言。
|
||||
并行浅扫描（worker）或把 w3x 顶层记录持久化到磁盘缓存。
|
||||
- `ra3modxml.reindex` 出于正确性会清空解析缓存（可能 ~15-25s）；如接受 watcher
|
||||
可靠性可改为保留。
|
||||
|
||||
---
|
||||
|
||||
## 十五、问题分析（第十轮，2026-08-03）：索引未完成时的部分可用性与分阶段索引
|
||||
|
||||
### 目标
|
||||
|
||||
第九轮遗留：Corona 首次全量建索引约 4 分钟（2.6GB w3x 读取 + 冷 statSync），
|
||||
期间插件所有功能被整体关闭（`ws.index == null`）。本轮让插件在索引完成前
|
||||
分层可用，并把最耗时的 w3x 扫描放到后台阶段。
|
||||
|
||||
### 现状证据
|
||||
|
||||
- `diagnostics.update` 无索引时直接清空诊断：连 XML 语法错误、未知元素、缺 id
|
||||
这类不依赖索引的检查也被关闭；
|
||||
- 补全的 `attribute-value` / `content` 分支无索引直接返回空，但枚举 / 布尔 /
|
||||
Include type / 子元素补全并不需要索引（`contentItems` 甚至没用 idx）；
|
||||
- hover / 文档链接的 Include 解析只依赖搜索路径（settings），不需要索引；
|
||||
- Find All References 完全不使用索引。
|
||||
|
||||
### 设计
|
||||
|
||||
1. **索引状态模型**：`ModIndex` 增加 `complete` / `phase`（`"xml" | "art"`）/
|
||||
`stale`；`stats` 增加 `phase` / `complete` / `deferredArtFiles` /
|
||||
`artScanMs`。
|
||||
2. **分阶段索引**：
|
||||
- 阶段 A（xml）：walk include 链时只**登记** w3x / 嗅探 XML 文件
|
||||
（`readDocument` 的 `deferArt` 模式），不读内容；manifest、项目 XML、
|
||||
mapmetadata 全部正常索引；结束后发布不可变快照；
|
||||
- 阶段 B（art）：按队列浅扫描 w3x、应用资产记录，并继续走 w3x 内的
|
||||
Include / xi:include（此时新遇到的美术文件立即扫描，不再入队）；
|
||||
- 快照不可变性的关键：`addAsset` 在阶段 B 仍会向数组 push，所以阶段 A
|
||||
发布时必须复制 `assets` / `assetsById` / `defines` / `streams.files` /
|
||||
`diagnostics`（`snapshotIndex` 深拷贝嵌套 Map/数组）。
|
||||
3. **部分可用性（T0 解耦）**：
|
||||
- 诊断：语法错误、未知元素/属性、缺 id、同文件重复 ID、Include 目标存在性
|
||||
不再依赖索引；引用 / `$DEFINE` / 跨文件重复在索引未完成或 stale 时
|
||||
“显示但标注”：code 变为 `unresolved-reference-indexing` /
|
||||
`undefined-define-indexing`，消息追加 `(index incomplete — may be a
|
||||
false positive)`；跨文件重复追加 `(based on a partial index)`；
|
||||
- 补全：枚举 / 布尔 / Include type / `xai:joinAction` / 子元素（content）
|
||||
在无索引时可用；资产 ID / define / include source 仍需索引；
|
||||
- hover / 文档链接：Include source / `xi:include href` 用 settings 搜索路径
|
||||
即可解析；无索引时引用值 hover 提示 “Index is still building”。
|
||||
4. **构建中文件被修改**：
|
||||
- watcher 的 change / create / delete 现在都会 `scheduleRebuild()`（此前
|
||||
只 `invalidate`，外部修改后没有任何东西触发重建）；
|
||||
- 新增 `InvalidationsEpoch`：`invalidate` / `invalidateExistence` 递增；
|
||||
构建开始时记录 epoch，发布每个快照（含最终）时若 epoch 变化则标记
|
||||
`stale`（状态栏显示 `(stale)`）；dirty 机制保证构建结束后立即再重建收敛；
|
||||
- 构建失败时保留上一个可用快照并标记 stale，不再清空索引。
|
||||
5. **stat 结构扩展**：`IndexedFile.stat` 从 `{ mtimeMs, size }` 变为
|
||||
`{ mtimeMs, size, birthtimeMs, ctimeMs }`，为磁盘持久化缓存铺路
|
||||
（FAT32 的 mtime 只有 2 秒粒度，多信号可捕捉“保留 mtime 的整体替换写入”）。
|
||||
|
||||
### 验证
|
||||
|
||||
- 单元测试 **73 → 79 全绿**：
|
||||
- indexer：phase-A 快照发布（XML/manifest 资产可用、美术资产缺席）、快照
|
||||
不可变性、`deferredArtFiles` 统计、`artScanMs`、mtime 变更强制重读；
|
||||
- caches：`InvalidationsEpoch` 递增 / 快照语义；
|
||||
- completion：无索引时枚举 / 元素名 / 属性名 / 子元素补全仍工作。
|
||||
- `tsc` + esbuild 构建通过。
|
||||
|
||||
### 实机复现与补充修复（2026-08-03）
|
||||
|
||||
在 Corona 上实测新版索引器：
|
||||
|
||||
- 阶段 A（xml）**27.0s** 发布：54,283 资产（含 manifest 35,322）、8,399 文件、
|
||||
4,797 个 w3x 待扫；最终 **118.0s**：64,868 资产、8,976 文件、4,829 浅扫
|
||||
（artScanMs ≈ 91s，walkMs ≈ 26s）。
|
||||
- `OnSeaUnitCrate.xml` 中的引用在阶段 A 即可解析（`Locomotor=JapanEggLocomotor`
|
||||
、`CommandSet=EmptyCommandSet` 均有候选）；`LocomotorSet@Condition` 是
|
||||
19 个枚举值的模型属性，无索引时即可补全。
|
||||
|
||||
由此确认两个真实体验问题并修复：
|
||||
|
||||
1. **状态栏初始不显示 indexing**：`workspaceContains` 激活事件在大目录上扫描
|
||||
较慢，打开工作区后扩展尚未激活，看起来“没有任何功能”。`activationEvents`
|
||||
增加 `onLanguage:xml`，打开任意 XML 文件即激活；同时各 provider 增加
|
||||
`isRa3Workspace()` 守卫，避免在非 RA3 工作区误补全/误诊断。
|
||||
2. 索引构建中执行 “Show index report” 提示 “no index available” 有误导 →
|
||||
新增 `ws.isBuilding`，构建中改为提示 “index is still building”。
|
||||
|
||||
版本 **0.1.1 → 0.1.2**（重新打包 `ra3-mod-xml-0.1.2.vsix`）。
|
||||
|
||||
---
|
||||
|
||||
## 十六、问题分析（第十一轮，2026-08-03）：冷启动磁盘缓存与首建 statSync 消除
|
||||
|
||||
### 目标
|
||||
|
||||
第十轮后 Corona 首次建索引仍需 ~2 分钟，且每次新会话（重启 VS Code）都要
|
||||
重来一遍。本轮做两件事：
|
||||
|
||||
1. **文件集快照替代 statSync**：`resolveSource` 的逐 base `statSync` 存在性
|
||||
检查（Corona 首建约 11 万次）改为目录枚举建立的 `Set` 查询；
|
||||
2. **磁盘持久化缓存**：records 缓存（含 w3x 浅扫记录）跨会话落盘，冷启动
|
||||
只做 stat 校验，不再重读 2.6GB 美术资产。
|
||||
|
||||
### 设计
|
||||
|
||||
1. **ExistenceSnapshot**（`src/indexer/existence.ts`）：
|
||||
- **惰性按目录 readdir**：只在实际查询某个候选路径时读取它的父目录
|
||||
（`readdirSync` + `withFileTypes`，不 stat 单个文件），结果按目录缓存；
|
||||
不做首建前的全量递归枚举（该方案曾让 XML 阶段从 27s 涨到 45s）；
|
||||
- 覆盖根判定：候选父目录落在根内 → 目录条目 Set 查询（`hits`）；落在
|
||||
根外 → `statSync` 回退（`fallbacks`）；
|
||||
- 盘符根（`C:\` 等）与不存在的根不枚举，避免遍历整个磁盘；
|
||||
- 子根被更宽的根覆盖时跳过（如 `sdkDir` 覆盖 `sdkDir/SageXml`）。
|
||||
2. **DiskRecordsCache**（`src/indexer/diskCache.ts`）：
|
||||
- gzip JSON，原子写（tmp + rename），版本号 + identity key
|
||||
（项目/SDK/设置哈希，配置变化自动忽略旧缓存）；
|
||||
- 每条记录保存多信号 stamp `{ size, mtimeMs, birthtimeMs, ctimeMs }`；
|
||||
- 加载时并发（32）stat 校验，不匹配/缺失丢弃，构建时重读；
|
||||
- 缺失/损坏/身份不符 → 空结果，不报错。
|
||||
3. **workspace 集成**：
|
||||
- 构建前若内存 records 缓存为空，从磁盘加载并校验（状态栏显示
|
||||
“validating cache…”）；构建成功后异步回写（entries 先快照，避免与
|
||||
下一次构建竞争）；
|
||||
- 新命令 `ra3modxml.clearCache`（清内存 + 删磁盘文件 + 强制重建）与
|
||||
`ra3modxml.showCacheReport`（缓存路径/大小/加载校验统计/命中数）。
|
||||
|
||||
### 实测（Corona）
|
||||
|
||||
| 场景 | 结果 |
|
||||
|---|---|
|
||||
| 首次构建 | 118.7s（phaseA **24.0s**）；snapshotHits **75,926**、fallbacks **0**、resolveCalls 11,061 |
|
||||
| 保存缓存 | 0.1s；gzip 后 **651 KB** |
|
||||
| 加载 + stat 校验 | 0.2s（8,976 条全部校验通过） |
|
||||
| 新会话二次构建 | **10.9s**（w3x 重扫 0、shallowCacheHits 4,829、recordsCacheHits 4,166） |
|
||||
|
||||
冷启动（加载 + 校验 + 构建）约 **11s**，对比之前每次 ~2.5 分钟。
|
||||
|
||||
### 测试与验证
|
||||
|
||||
- 单元测试 **79 → 90 全绿**：
|
||||
- `existence.test.mjs`：覆盖/未覆盖判定、hits/fallbacks、盘符根识别、
|
||||
搜索 base 枚举、`resolveSource` 快照命中与 statSync 回退；
|
||||
- `diskCache.test.mjs`：roundtrip、原子写无残留 tmp、stat 变更丢弃、
|
||||
identity 不符忽略、损坏文件空结果、clear、key 稳定性；
|
||||
- `caches.test.mjs` 增加 `entries()`；indexer 统计断言 snapshotHits。
|
||||
- `tsc` + esbuild + `vsce package` 通过。
|
||||
|
||||
### 遗留
|
||||
|
||||
- 首次构建 phase A 现在包含快照目录枚举成本（Corona 实测约 +10-17s，后续
|
||||
构建由 walker 缓存吸收）；如 SDK 根目录特别大可再优化根覆盖策略。
|
||||
- w3x 并行/流水线浅扫描仍未做（HDD 收益存疑，SSD 再做）。
|
||||
|
||||
### 补充（用户实测反馈，2026-08-03）
|
||||
|
||||
用户在 0.1.3 上删除 workspace storage 缓存后实测：
|
||||
|
||||
- XML 阶段约 45s（比之前 27s 多）→ 根因是磁盘缓存轮实现的**全量目录枚举
|
||||
快照**在首建前递归枚举 SDK 根等搜索根。已改为惰性按目录 readdir，复测
|
||||
phase A **24.0s**，statSync 消除效果不变（75,926 次快照命中、0 回退）。
|
||||
- “show index report 显示 Indexed in 1.1s、0 浅扫、8995 缓存命中”与观察
|
||||
不一致 → 首建完成后又发生了一次信任重建（follow-up）。为定位触发源，
|
||||
新增**重建插桩**：`buildCount` / `lastBuildTrigger`（initial、save、
|
||||
watcher-create/change/delete、config、reindex-command、clear-cache、
|
||||
dirty-followup）+ “RA3 Mod XML” 输出通道（每次构建记录 trigger、phase A
|
||||
发布时间、完成耗时）；索引报告与缓存报告均显示构建序号与触发原因。
|
||||
|
||||
### 补充 2（0.1.4 日志复现，2026-08-03）
|
||||
|
||||
0.1.4 输出通道日志：
|
||||
|
||||
- build #1：phase A 31.4s、done 120.1s、**stale=true**；
|
||||
- build #2：`dirty-followup (initial)`，0.6s（报告计时不一致的直接来源）；
|
||||
- build #3/#4/#5：每 ~35s 一次 `watcher-change`，每次 0.5s 重建。
|
||||
|
||||
build #1 的 stale=true 与周期性 watcher-change 均不符合预期：说明有后台进程
|
||||
在周期性触碰被监视目录（最可疑是 `.git` 内部文件，如后台 fetch/maintenance）。
|
||||
处理：
|
||||
|
||||
1. watcher 事件现在把**触发 URI 写入输出通道**（`[watcher-change] <path>`），
|
||||
可直接定位是哪个文件/目录在变化；
|
||||
2. 新增 `isWatcherNoisePath`：路径含 `.git` 段的事件直接忽略（不 invalidate、
|
||||
不标记 stale、不触发重建）；单元测试 90 → 91。
|
||||
|
||||
版本 **0.1.4 → 0.1.5**。
|
||||
|
||||
### 补充 3（0.1.5 日志复现，2026-08-03）
|
||||
|
||||
0.1.5 日志中周期性重建已消失，但首建期间仍有一次：
|
||||
|
||||
```
|
||||
[watcher-change] d:\...\corona\Data\Neutral\Crate\UnitCrate.xml.git
|
||||
```
|
||||
|
||||
该文件**并不存在**（疑似其他 VSCode 扩展产生的瞬时临时文件），且它是
|
||||
`*.xml.git` 文件名，不是 `.git` 目录段，绕过了上一轮过滤；它也导致 build #1
|
||||
stale=true 并触发 follow-up。处理（按用户建议的扩展名白名单思路）：
|
||||
|
||||
1. `isWatcherNoisePath` 增加临时文件命名模式：`.git` / `.tmp` / `.lock` /
|
||||
`~` / `.swp` / `.bak` / `.orig` 后缀，以及 `.#` / `.~` 前缀；
|
||||
2. `onDidChange` 只响应**内容相关**文件：扩展名白名单（`.xml` / `.w3x`；
|
||||
领域修正：RA3 合理文本格式为 xml/w3x/lua，lua 暂未索引，manifest 为
|
||||
`*.manifest` 二进制，不存在 `.manifestxml`）或已在当前索引中的文件
|
||||
（`ModIndexer.isIndexedFile`，覆盖被嗅探为 XML 的未知扩展名）；纹理等
|
||||
二进制内容变更不触发重建；
|
||||
3. 创建/删除仍对所有真实文件响应（影响 include 存在性），临时文件模式除外。
|
||||
|
||||
测试 91 → 92；版本 **0.1.5 → 0.1.6**。
|
||||
|
||||
### 遗留（此处的磁盘缓存与文件集快照已在第十一轮完成，T1 已在第十二轮完成）
|
||||
|
||||
- w3x 文件名启发式定向扫描（按约定后续再做）。
|
||||
- `AssetIdList` 等“任意资产 ID 列表”的引用语义建模。
|
||||
- Find All References 目前仍是全文搜索，未走索引（对应需求 P1 高效搜索）。
|
||||
|
||||
---
|
||||
|
||||
## 十七、问题分析（第十二轮，2026-08-03）：当前文档局部链与 include 逻辑树展开（T1）
|
||||
|
||||
### 目标
|
||||
|
||||
上一轮遗留的 T1：让**不在任何全局流里的文件**也能解析自身引用，并为 GameObject
|
||||
内模块 `id` 的局部作用域（`AttachModuleId` / `ModuleId` / `AutoResolveBody` 等）
|
||||
铺路。两件事一起做：
|
||||
|
||||
1. **T1a 文档局部 overlay**：当前打开的文档（含未保存文本）自身资产 / `$DEFINE`
|
||||
及其 include 链进入一个轻量局部索引，与全局索引叠加使用；
|
||||
2. **T1b 逻辑树展开**：`xi:include` 按现有 `xpointer` 子集拼入当前文档的逻辑树，
|
||||
使 include 进来的内容获得正确的父上下文，并让 Poid 引用能在同一 GameObject
|
||||
子树内解析。
|
||||
|
||||
### 现状证据
|
||||
|
||||
- 全局索引只从 `Data/Mod.xml` 与 `additionalmaps/mapmetadata_*.xml` 出发；
|
||||
`Data/Standalone.xml` 这类未进流的文件，其自身 `GameObject` / `$DEFINE` 不会
|
||||
出现在 `ws.index`，`inheritFrom`、`CommandSet` 全部无法解析；
|
||||
- `xi:include` 此前只做到“目标文件可索引 / 缺失可诊断”，没有拼入当前文档树；
|
||||
include 进来的 `TruckDraw` 等模块拿不到 `Draws` 子元素的上下文类型；
|
||||
- `isLocalReferenceAttribute` 对所有 Poid 属性一律“不检查、不解析”，导致
|
||||
同一 GameObject 内完全可静态判断的模块引用也没有 hover / 跳转 / 补全。
|
||||
|
||||
### 设计
|
||||
|
||||
1. **共享 `xpointer.ts`**：`localName` / `findXPointerContainer` 从 indexer 迁出，
|
||||
全局索引与局部展开共用同一份 xpointer 子集实现。
|
||||
2. **`logicalTree.ts`**：
|
||||
- 按解析器扁平元素表预建逻辑节点壳（保留原始 `sourceFile` 与偏移），再按
|
||||
真实根 + 容错孤儿根遍历，重建 parent/child 链；
|
||||
- `xi:include` 解析 href → `readDom` 目标 → 按 `xpointer` 选择容器子节点 →
|
||||
拼入逻辑父元素；目标缺失 / 环 / 超深时跳过（环与深度用 visited + 64）；
|
||||
- 保留 xi 节点本身在 `elements` 中，hover 仍可解释 XInclude。
|
||||
3. **`localScope.ts`**：
|
||||
- `buildDocumentScope` 返回原始 parse、逻辑树、per-source LineMap、局部
|
||||
overlay 与 overlay-aware merged index；
|
||||
- overlay 沿 `<Include type="all/instance">` 与 `xi:include` 递归收集资产 /
|
||||
Define；`reference` 指向 manifest，资产由全局索引提供，不重复解析；
|
||||
- `withLocalOverlay` 不复制全局 Map（Corona ~65k 资产），只把 overlay 挂到
|
||||
`ModIndex.local`，查询函数“局部优先、全局兜底”。
|
||||
4. **workspace 集成**：`getScope(document)` 按 URI + 文档 version + 全局索引
|
||||
代次缓存；全局索引发布 / 文件关闭时失效；首个 indexer 创建前的兜底读取走
|
||||
`fallbackRead`（小 XML 直接解析）。
|
||||
5. **features 接入**：
|
||||
- diagnostics / hover / navigation / completion 改从 `getScope` 取逻辑树与
|
||||
merged index；诊断只上报 `sourceFile === 当前文件` 的节点；
|
||||
- 引用解析 / 补全 / Define 查询支持 `idx.local` 优先；
|
||||
- Poid 属性：`AttachModuleId` / `ModuleId` / `UpdateModuleId` 等可在最近
|
||||
GameObject 子树内补全、hover、跳转；未命中**不新增诊断**（保守，避免
|
||||
“武器引用另一 GameObject 模块”这类跨文件语义误报）；
|
||||
- 导航对当前未保存文件优先用编辑器文本定位，再回退磁盘 DOM。
|
||||
|
||||
### 验证
|
||||
|
||||
- 单元测试 **92 → 98 全绿**：
|
||||
- `localScope.test.mjs`：未进流文件 overlay（自身资产 / Define / instance
|
||||
include 链）、inheritFrom 局部解析、xi:include 展开后模块上下文类型、
|
||||
Poid 引用找到 include 进来的兄弟模块、局部定义优先于全局同名定义、
|
||||
xi:include 环终止；
|
||||
- `completion.test.mjs`：Poid 属性只补全所在 GameObject 子树内的 id。
|
||||
- `tsc` / `esbuild` 构建通过（打包验证见版本发布步骤）。
|
||||
|
||||
### 边界与后续
|
||||
|
||||
- 顶层 `<Include type="all">` 仍**不**并入逻辑树：它通常是独立整文件或大体积
|
||||
w3x，展开收益与风险不成比例；如需要“当前文档视角的全量合并诊断”再单独做。
|
||||
- `AttachModuleId` 若出现在独立 `WeaponTemplate`（不在 GameObject 子树内），
|
||||
本轮仍不解析；等真实样本确认跨文件语义后再扩展。
|
||||
- 未命中 Poid 不报诊断是刻意保守，后续可加配置项开启。
|
||||
|
||||
版本 **0.1.7 → 0.1.8**。
|
||||
|
||||
### 补充：构建期局部作用域闸门（2026-08-04）
|
||||
|
||||
用户清空缓存后在 Corona 上测得 phase A **101.2s** / 完成 **254.6s**，明显高于
|
||||
第十一轮冷建基线（phase A 约 24s / 总约 118s）。代码审查确认 T1 没有改动 indexer
|
||||
的构建路径（`localScope` / `logicalTree` 不参与 `build()`），但自动诊断在构建中
|
||||
会触发 `getScope()`，可能和 indexer 抢磁盘 / CPU。
|
||||
|
||||
修复：`getScope()` 在 `building === true` 时返回 **parse-only 轻量 scope**
|
||||
（只解析当前文件，不沿 include 链读盘、不展开逻辑树）；构建完全结束后再刷新
|
||||
全量局部 scope。这样首建期间诊断 / 补全仍可用，但不会拖慢索引。索引报告与
|
||||
输出通道同步增加 `walk / candidates / art` 耗时分解，便于下次直接定位慢在哪一段。
|
||||
|
||||
版本 **0.1.8 → 0.1.9**。
|
||||
|
||||
+77
-14
@@ -82,14 +82,17 @@ src/
|
||||
asset-types.json 由 tools/extract-asset-types.mjs 生成（TypeId 哈希→类型名）
|
||||
indexer/
|
||||
includeResolver.ts Include 路径解析（纯 TS，BAB /data /art /audio 顺序）
|
||||
existence.ts 文件集存在性快照（目录枚举 Set，替代逐路径 statSync）
|
||||
manifestParser.ts .manifest 二进制解析 + 类型/ID 推导（纯 TS）
|
||||
fileScanner.ts 目录遍历缓存 + Include source 候选收集
|
||||
refs.ts 引用目标解析（按 refType / isRef / inheritFrom 过滤，纯 TS）
|
||||
shallowScan.ts 大体积美术资产（.w3x 等）顶层浅扫描（纯 TS，不建 DOM）
|
||||
records.ts 每文件紧凑索引记录（资产/Define/Include/xi + 行号）
|
||||
caches.ts 跨重建持久缓存（DocumentCache / IndexRecordsCache /
|
||||
IncludeResolveCache）
|
||||
indexer.ts 工作区索引器（资产/Define/流/manifest/w3x 合并）
|
||||
IncludeResolveCache）+ 失效纪元 InvalidationsEpoch
|
||||
diskCache.ts 跨会话磁盘缓存（gzip JSON、原子写、多信号 stat 校验）
|
||||
indexer.ts 工作区索引器（资产/Define/流/manifest/w3x 合并，
|
||||
分阶段：XML → art，中间快照可发布）
|
||||
types.ts 共享类型
|
||||
features/
|
||||
completion.ts 补全 provider（元素/属性/值，上下文感知；xs:list 多值按当前段过滤）
|
||||
@@ -144,8 +147,9 @@ test/
|
||||
- 全量解析内存放大约 17 倍（6.3 MB 文本 → +109 MB DOM），不可接受；`scanXmlShallow`
|
||||
单次线性扫描只提取顶层 `name+id`、`<Includes>`、`<xi:include>`、`<Defines>`，
|
||||
22 MB 文件 ~600 ms、保留内存≈0。
|
||||
- 索引按扩展名三分：`.xml`/`.manifestxml` 全量解析（4 MB 上限不变）；`.w3x` 浅扫描；
|
||||
未知扩展名嗅探文件头（`<` 开头、无 NUL）决定按 XML 浅扫描或二进制登记。
|
||||
- 索引按扩展名三分：`.xml` 全量解析（4 MB 上限不变）；`.w3x` 浅扫描；
|
||||
未知扩展名嗅探文件头（`<` 开头、无 NUL）决定按 XML 浅扫描或二进制登记
|
||||
（manifest 是 `*.manifest` 二进制，不存在 `.manifestxml` 源码格式）。
|
||||
- `DocumentCache` / `ShallowScanCache` 由 `ModWorkspace` 持有，每次重建传入新的
|
||||
`ModIndexer`；按 `mtimeMs + size` 校验，未变化不重读。Corona 第二次构建
|
||||
w3x 重扫数为 0（4,829 次缓存命中）。
|
||||
@@ -164,6 +168,52 @@ test/
|
||||
堆保留确认为构建期可回收垃圾，常驻 ~100MB；强制 reindex ~5-25s。
|
||||
- 候选目录扫描并行化；`stats` 新增 `candidatesMs` / `walkMs` / `resolveCalls` /
|
||||
`resolveCacheHits` 供索引报告定位耗时。
|
||||
16. **分阶段索引与部分可用性**（第十轮，2026-08-03）：索引分两阶段发布——
|
||||
阶段 A（xml）只走 XML + manifest include 链，w3x 只登记进待扫队列；
|
||||
阶段 B（art）浅扫描队列并继续走 w3x 内的 include。阶段 A 结束即发布
|
||||
不可变快照（`snapshotIndex` 深拷贝嵌套 Map/数组），XML/枚举/语法类功能
|
||||
在首建早期即可用；引用类诊断在 `!complete || stale` 时“显示但标注”
|
||||
（code 为 `*-indexing`，消息注明 index incomplete）。快照携带
|
||||
`complete` / `phase` / `stale`，状态栏显示阶段。
|
||||
17. **构建中失效与 stale 标记**（第十轮）：watcher 的 change / create /
|
||||
delete 均触发防抖重建；`InvalidationsEpoch` 记录失效次数，快照发布时若
|
||||
期间有失效则标记 stale，由 dirty 机制随后重建收敛；构建失败保留上一个
|
||||
快照而非清空索引。
|
||||
18. **多信号文件 stamp**（第十轮）：`IndexedFile.stat` 扩展为
|
||||
`{ size, mtimeMs, birthtimeMs, ctimeMs }`，任一不匹配即重读，为磁盘
|
||||
持久化缓存铺路（FAT32 mtime 2s 粒度、工具保留 mtime 等场景）。
|
||||
19. **文件集快照替代 statSync**（第十一轮，2026-08-03）：`ExistenceSnapshot`
|
||||
用**惰性按目录 readdir**（只读查询到的父目录，按目录缓存）回答 include
|
||||
存在性，覆盖根之外才 statSync 回退；盘符根不枚举。首版全量递归枚举使
|
||||
XML 阶段从 27s 涨到 45s，已改为惰性模式（复测 24.0s）。
|
||||
`stats.snapshotHits / snapshotFallbacks` 入报告；Corona 首建 75,926 次
|
||||
查询全部由快照回答、0 回退。
|
||||
20. **磁盘持久化缓存**（第十一轮）：`DiskRecordsCache` 持久化 records 缓存
|
||||
（gzip JSON、原子写、identity key）；启动时并发 stat 多信号校验，不匹配
|
||||
丢弃重读；构建后异步回写。Corona 缓存仅 651 KB，冷启动约 11s（原
|
||||
~2.5 分钟）。
|
||||
21. **缓存命令**（第十一轮）：`ra3modxml.clearCache`（清内存 + 磁盘 +
|
||||
强制重建）、`ra3modxml.showCacheReport`（路径/大小/校验统计/命中数）。
|
||||
22. **重建插桩**（第十一轮补充）：`buildCount` / `lastBuildTrigger`
|
||||
（initial、save、watcher-*、config、reindex、clear-cache、
|
||||
dirty-followup）进入索引/缓存报告；“RA3 Mod XML” 输出通道记录每次构建
|
||||
的触发原因、phase A 发布时间与完成耗时，用于定位“首建后又重建一次”
|
||||
之类的现象。
|
||||
23. **watcher 噪声过滤与 URI 日志**（第十一轮补充）：watcher 事件把触发
|
||||
URI 写入输出通道；路径含 `.git` 段的事件直接忽略（后台 fetch /
|
||||
maintenance 会周期性触碰 `.git`，不应触发重建或 stale 标记）。
|
||||
24. **watcher 内容白名单**（第十一轮补充）：临时文件命名模式
|
||||
（`.git`/`.tmp`/`.lock`/`~`/`.swp`/`.bak`/`.orig` 后缀、`.#`/`.~`
|
||||
前缀）全部忽略；`onDidChange` 只响应扩展名白名单（`.xml` / `.w3x`，
|
||||
RA3 合理文本格式为 xml/w3x/lua，lua 暂未索引、manifest 为二进制）或已在
|
||||
索引中的文件；创建/删除仍响应所有真实文件（影响 include 存在性）。
|
||||
25. **当前文档局部链 + 逻辑树展开（第十二轮，T1）**：新增 `xpointer.ts` /
|
||||
`logicalTree.ts` / `localScope.ts`。打开文件时按当前文本建立局部 overlay
|
||||
（自身资产 / `$DEFINE` / include 链），并生成展开 `xi:include` 的逻辑树；
|
||||
features 经 `ws.getScope(document)` 拿到 overlay-aware 索引。Poid 引用
|
||||
（`AttachModuleId` / `ModuleId` / `AutoResolveBody` 等）在最近 GameObject
|
||||
子树内解析；未命中不新增诊断（保守策略，避免跨文件误报）。
|
||||
顶层 `<Include type="all">` 暂不并入逻辑树（保留为后续扩展）。
|
||||
|
||||
## 三、实施步骤
|
||||
|
||||
@@ -182,6 +232,24 @@ test/
|
||||
9. [x] Corona 性能与内存优化（第九轮，v0.1.1）：`records.ts` 记录驱动索引、
|
||||
`IncludeResolveCache` 零 stat 重建、DOM 元素预算淘汰、w3x LineMap 移除、
|
||||
候选并行扫描、阶段计时；Corona 信任重建 2.0s；确认首建 2.5GB 为可回收垃圾。
|
||||
10. [x] 部分可用性 + 分阶段索引（第十轮，2026-08-03）：T0 解耦（语法/模型
|
||||
诊断、枚举/子元素补全、Include 链接无索引可用）、A/B 分阶段发布、watcher
|
||||
触发重建 + 失效纪元 stale 标记、stat 多信号扩展；测试 73 → 79。
|
||||
11. [x] 冷启动提速（第十一轮，2026-08-03）：文件集快照替代 statSync、
|
||||
磁盘持久化缓存（多信号校验、原子写）、clearCache/showCacheReport 命令；
|
||||
测试 79 → 90；Corona 冷启动 ~11s。
|
||||
12. [x] 惰性存在性快照 + 重建插桩（第十一轮补充，2026-08-03）：全量目录枚举
|
||||
改为惰性按目录 readdir（phase A 45s → 24.0s）；新增 buildCount /
|
||||
lastBuildTrigger 与输出通道日志；索引/缓存报告显示构建序号与触发原因。
|
||||
13. [x] watcher 噪声过滤 + URI 日志（第十一轮补充，2026-08-03）：输出通道
|
||||
记录 `[watcher-*] <path>`；`.git` 段路径忽略；测试 90 → 91。
|
||||
14. [x] watcher 内容白名单（第十一轮补充，2026-08-03）：临时文件命名模式
|
||||
过滤 + 内容变更扩展名白名单 + `isIndexedFile` 兜底；测试 91 → 92。
|
||||
15. [x] T1 当前文档局部链 + include 展开（第十二轮，2026-08-03）：局部
|
||||
overlay（不在任何流里的文件也能解析自身引用）+ `xi:include` 逻辑树展开
|
||||
+ Poid 局部作用域补全/悬停/跳转；测试 92 → 98。2026-08-04 补充构建期
|
||||
闸门：`getScope` 在重建进行中只返回 parse-only scope，避免与 indexer
|
||||
抢盘（版本 0.1.9）。
|
||||
|
||||
## 四、验证结果（实测）
|
||||
|
||||
@@ -200,16 +268,11 @@ test/
|
||||
- 假设 SDK 路径默认 `C:\Apps\RA3-MODSDK-X`（与 prompts 一致），可在设置中修改。
|
||||
- 假设补全/导航以“文本语义分析”为主，不做完整 XSD 校验（BAB 才是权威校验器）。
|
||||
- 开放：是否发布到 VS Code Marketplace（需要 publisher）——本期先保证本地 `vsce package` 可安装。
|
||||
- 开放：**“宏展开”式虚拟合并**（用户提议，方向已确认）：解析前先把 `xi:include`
|
||||
（以及顶层 `<Include type="all">`、`inheritFrom` 继承合并）展开成不含 include 的
|
||||
文档树，再对展开后的树做 XSD 校验、补全与诊断——与 BAB 编译时把整个 Mod 合并成
|
||||
一份大 XML 的行为一致。展开树需携带**来源追溯**（错误/跳转仍定位到原始文件），
|
||||
并处理 xpointer 子集解析与 include 循环。当前仅做到：目标文件可索引、可导航、
|
||||
缺失可诊断；`xi:include` 本身不参与 XSD 校验（第五轮）。
|
||||
该功能也是“GameObject 内模块 id 局部作用域解析”（第四轮遗留）的地基。
|
||||
详细设计备忘见下一节。
|
||||
- 开放：**“宏展开”式虚拟合并**（用户提议，方向已确认）：`xi:include` 已按逻辑树
|
||||
展开（第十二轮，见第六节）；顶层 `<Include type="all">` 与 `inheritFrom` +
|
||||
`xai:joinAction` 的深合并仍未实现，后续如需要“当前文档视角的全量合并诊断”再继续。
|
||||
|
||||
## 六、include 展开设计备忘（2026-08-01，待实施）
|
||||
## 六、include 展开设计备忘（2026-08-01；xi:include 部分已实施于第十二轮）
|
||||
|
||||
> 目的：集中记录 include 处理相关的现状、结论与设计，下次遇到 include 问题时从这里继续，
|
||||
> 并在实施后把结果回写本节。
|
||||
@@ -222,7 +285,7 @@ test/
|
||||
| `reference` → builtmods manifest 解析 / 缺失回退占位 XML | 已实现 |
|
||||
| 嵌套 `xi:include`（任意层级）：目标可索引、缺失报 `include-not-found`、Ctrl+点击跳转、`href` hover 解析目标 | 已实现（第二轮 + 第五轮） |
|
||||
| `xi:include` 及其属性不参与 XSD 校验（外来命名空间守卫 `isXsdElementName` / `isXsdAttributeName`） | 已实现（第五轮） |
|
||||
| include 目标内容“虚拟合并”进父文档的逻辑树 | **未实现**（本文档主题） |
|
||||
| include 目标内容“虚拟合并”进父文档的逻辑树 | 已实现 `xi:include`（第十二轮）；顶层 `<Include type="all">` 仍不展开 |
|
||||
|
||||
### 2. 已确认的方向
|
||||
|
||||
|
||||
@@ -90,7 +90,9 @@ XML 之间的组织靠 `<Include>` 标签，共有三种语义：
|
||||
- 中小项目：`C:\Apps\RA3-MODSDK-X\Mods\AttachTest`、`D:\Mods\CoronaMod\mods\mods\GenEvoTest`。
|
||||
- 大型项目：`D:\Mods\CoronaMod\mods\mods\corona`（自带 `xsd/`）。
|
||||
- 现有工具：工作区 `check_duplicate_ids.py`（include 解析与重复 ID 检测的参考实现）。
|
||||
- manifest 格式参考：OpenSAGE `src/OpenSage.Game/Data/StreamFS/ManifestFile.cs`（commit `d45d361`，最新分支已移除该文件）。本机网络受限未能拉取，且 manifest 为压缩/哈希的二进制，本期不实现其解析。
|
||||
- manifest 格式参考：OpenSAGE `src/OpenSage.Game/Data/StreamFS/ManifestFile.cs`
|
||||
（commit `d45d361`，本仓库 `OpenSAGE/` 子目录）。插件已实现 v5/v6/v7 二进制
|
||||
解析（`src/indexer/manifestParser.ts`），未知 TypeId 哈希时按资产名前缀推导类型。
|
||||
|
||||
## 四、验收标准
|
||||
|
||||
|
||||
Generated
+2
-2
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "ra3-mod-xml",
|
||||
"version": "0.1.1",
|
||||
"version": "0.1.9",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "ra3-mod-xml",
|
||||
"version": "0.1.1",
|
||||
"version": "0.1.9",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"fast-xml-parser": "^4.5.0"
|
||||
|
||||
+10
-1
@@ -2,7 +2,7 @@
|
||||
"name": "ra3-mod-xml",
|
||||
"displayName": "RA3 Mod XML",
|
||||
"description": "Red Alert 3 Mod XML tooling: syntax highlighting, completions, reference navigation and diagnostics for SAGE/BinaryAssetBuilder XML.",
|
||||
"version": "0.1.1",
|
||||
"version": "0.1.9",
|
||||
"publisher": "ra3-mod-xml",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
@@ -22,6 +22,7 @@
|
||||
],
|
||||
"main": "./dist/extension.js",
|
||||
"activationEvents": [
|
||||
"onLanguage:xml",
|
||||
"workspaceContains:**/Data/Mod.xml",
|
||||
"workspaceContains:**/mod.babproj",
|
||||
"workspaceContains:**/*.babproj"
|
||||
@@ -91,6 +92,14 @@
|
||||
{
|
||||
"command": "ra3modxml.openIndexReport",
|
||||
"title": "RA3 Mod XML: Show index report"
|
||||
},
|
||||
{
|
||||
"command": "ra3modxml.clearCache",
|
||||
"title": "RA3 Mod XML: Clear caches and rebuild"
|
||||
},
|
||||
{
|
||||
"command": "ra3modxml.showCacheReport",
|
||||
"title": "RA3 Mod XML: Show cache report"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
+43
-7
@@ -44,7 +44,7 @@ export function activate(context: vscode.ExtensionContext): void {
|
||||
context.subscriptions.push(
|
||||
vscode.languages.registerReferenceProvider(
|
||||
XML_SELECTOR,
|
||||
new Ra3ReferenceProvider(),
|
||||
new Ra3ReferenceProvider(ws),
|
||||
),
|
||||
);
|
||||
context.subscriptions.push(
|
||||
@@ -56,19 +56,26 @@ export function activate(context: vscode.ExtensionContext): void {
|
||||
context.subscriptions.push(
|
||||
vscode.languages.registerDocumentSymbolProvider(
|
||||
XML_SELECTOR,
|
||||
new Ra3DocumentSymbolProvider(),
|
||||
new Ra3DocumentSymbolProvider(ws),
|
||||
),
|
||||
);
|
||||
context.subscriptions.push(
|
||||
vscode.languages.registerDocumentSemanticTokensProvider(
|
||||
XML_SELECTOR,
|
||||
new Ra3SemanticTokensProvider(),
|
||||
new Ra3SemanticTokensProvider(ws),
|
||||
RA3_SEMANTIC_TOKENS_LEGEND,
|
||||
),
|
||||
);
|
||||
|
||||
const diagnostics = new Ra3Diagnostics(ws);
|
||||
context.subscriptions.push(diagnostics);
|
||||
// Refresh diagnostics for every open XML document whenever a new index
|
||||
// snapshot is published (XML phase, art phase, stale/final rebuild).
|
||||
ws.onIndexUpdate = () => {
|
||||
for (const doc of vscode.workspace.textDocuments) {
|
||||
if (doc.languageId === "xml") void diagnostics.update(doc);
|
||||
}
|
||||
};
|
||||
|
||||
const diagnosticTimers = new Map<string, ReturnType<typeof setTimeout>>();
|
||||
const scheduleDiagnostics = (doc: vscode.TextDocument) => {
|
||||
@@ -110,7 +117,7 @@ export function activate(context: vscode.ExtensionContext): void {
|
||||
vscode.workspace.onDidSaveTextDocument((doc) => {
|
||||
if (doc.languageId !== "xml") return;
|
||||
ws.invalidate(doc.uri.fsPath);
|
||||
ws.scheduleRebuild();
|
||||
ws.scheduleRebuild("save");
|
||||
void diagnostics.update(doc);
|
||||
}),
|
||||
);
|
||||
@@ -120,31 +127,60 @@ export function activate(context: vscode.ExtensionContext): void {
|
||||
// Search paths / builtmods locations may have changed: cached include
|
||||
// resolutions and manifest lookups are no longer valid.
|
||||
ws.invalidateExistence();
|
||||
ws.scheduleRebuild();
|
||||
ws.scheduleRebuild("config");
|
||||
}
|
||||
}),
|
||||
);
|
||||
|
||||
context.subscriptions.push(
|
||||
vscode.commands.registerCommand("ra3modxml.reindex", () => ws.rebuild(true)),
|
||||
vscode.commands.registerCommand(
|
||||
"ra3modxml.reindex",
|
||||
() => void ws.rebuild(true, "reindex-command"),
|
||||
),
|
||||
);
|
||||
context.subscriptions.push(
|
||||
vscode.commands.registerCommand("ra3modxml.clearCache", () => {
|
||||
ws.clearCaches();
|
||||
void vscode.window.showInformationMessage(
|
||||
"RA3 Mod XML: caches cleared; rebuilding from scratch…",
|
||||
);
|
||||
}),
|
||||
);
|
||||
context.subscriptions.push(
|
||||
vscode.commands.registerCommand("ra3modxml.showCacheReport", async () => {
|
||||
void vscode.window.showInformationMessage(await ws.cacheReport(), {
|
||||
modal: false,
|
||||
});
|
||||
}),
|
||||
);
|
||||
context.subscriptions.push(
|
||||
vscode.commands.registerCommand("ra3modxml.openIndexReport", () => {
|
||||
const idx = ws.index;
|
||||
if (!idx) {
|
||||
if (ws.isBuilding) {
|
||||
void vscode.window.showInformationMessage(
|
||||
"RA3 Mod XML: index is still building — check the status bar. " +
|
||||
"Most features become available after the XML phase.",
|
||||
);
|
||||
return;
|
||||
}
|
||||
void vscode.window.showInformationMessage(
|
||||
"RA3 Mod XML: no index available. Open a workspace that contains Data/Mod.xml.",
|
||||
);
|
||||
return;
|
||||
}
|
||||
const s = idx.stats;
|
||||
const stale = idx.stale ? " (stale)" : "";
|
||||
void vscode.window.showInformationMessage(
|
||||
`RA3 Mod XML index\n` +
|
||||
`Project: ${s.projectDir}\n` +
|
||||
`Files: ${s.indexedFiles} (${s.parsedFiles} parsed, ${s.shallowScannedFiles} shallow-scanned, ${s.shallowCacheHits + s.recordsCacheHits} cache hits)\n` +
|
||||
`Assets: ${s.assetCount} (${s.manifestAssetCount} from ${s.manifestFiles} manifests)\n` +
|
||||
`Defines: ${s.defineCount} · Streams: ${s.streams} · Candidates: ${s.sourceCandidates}\n` +
|
||||
`Indexed in ${(s.elapsedMs / 1000).toFixed(1)}s`,
|
||||
`Phase: ${s.phase} · Complete: ${s.complete}${stale}\n` +
|
||||
`Build #${ws.buildCount} (trigger: ${ws.lastTrigger})\n` +
|
||||
`Indexed in ${(s.elapsedMs / 1000).toFixed(1)}s\n` +
|
||||
`XML walk: ${(s.walkMs / 1000).toFixed(1)}s · Candidates: ${(s.candidatesMs / 1000).toFixed(1)}s · Art scan: ${(s.artScanMs / 1000).toFixed(1)}s`,
|
||||
{ modal: false },
|
||||
);
|
||||
}),
|
||||
|
||||
+73
-16
@@ -1,5 +1,5 @@
|
||||
import * as vscode from "vscode";
|
||||
import { parseXml, type XmlElement } from "../language/xmlParser";
|
||||
import type { XmlElement } from "../language/xmlParser";
|
||||
import {
|
||||
analyzeContext,
|
||||
splitListValuePrefix,
|
||||
@@ -8,6 +8,11 @@ import {
|
||||
import { resolveElementType } from "../language/typeContext";
|
||||
import * as model from "../model/schemaModel";
|
||||
import { isLocalReferenceAttribute } from "../indexer/refs";
|
||||
import {
|
||||
findContainingGameObject,
|
||||
collectLocalIds,
|
||||
type LogicalElement,
|
||||
} from "../indexer/logicalTree";
|
||||
import type { ModWorkspace } from "../workspace";
|
||||
import type { ModIndex, AssetDef } from "../indexer/types";
|
||||
|
||||
@@ -21,11 +26,13 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
|
||||
position: vscode.Position,
|
||||
_token: vscode.CancellationToken,
|
||||
): Promise<vscode.CompletionItem[]> {
|
||||
if (!this.ws.isRa3Workspace()) return [];
|
||||
const text = document.getText();
|
||||
const offset = document.offsetAt(position);
|
||||
const doc = parseXml(text);
|
||||
const scope = await this.ws.getScope(document);
|
||||
const doc = scope.expanded;
|
||||
const ctx = analyzeContext(doc, text, offset);
|
||||
const idx = this.ws.index;
|
||||
const idx = scope.merged;
|
||||
|
||||
switch (ctx.kind) {
|
||||
case "element-name":
|
||||
@@ -33,9 +40,9 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
|
||||
case "attribute-name":
|
||||
return this.attributeNameItems(ctx, document, position);
|
||||
case "attribute-value":
|
||||
return idx ? this.valueItems(ctx, document, position, idx) : [];
|
||||
return this.valueItems(ctx, document, position, idx);
|
||||
case "content":
|
||||
return idx ? this.contentItems(ctx, document, position, idx) : [];
|
||||
return this.contentItems(ctx, document, position, idx);
|
||||
default:
|
||||
return [];
|
||||
}
|
||||
@@ -177,7 +184,7 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
|
||||
ctx: CompletionContext,
|
||||
document: vscode.TextDocument,
|
||||
position: vscode.Position,
|
||||
idx: ModIndex,
|
||||
idx: ModIndex | null,
|
||||
): vscode.CompletionItem[] {
|
||||
const el = ctx.element;
|
||||
const attr = ctx.attr;
|
||||
@@ -232,6 +239,7 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
|
||||
);
|
||||
}
|
||||
if (isInclude && attrName === "source") {
|
||||
if (!idx) return [];
|
||||
return this.includeSourceItems(idx, prefix, make);
|
||||
}
|
||||
if (attrName === "xai:joinaction" || attrName === "joinaction") {
|
||||
@@ -242,16 +250,19 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
|
||||
|
||||
// inheritFrom: same element type first, then everything.
|
||||
if (attrName === "inheritfrom") {
|
||||
if (!idx) return [];
|
||||
return this.assetIdItems(idx, el.name, null, prefix, make);
|
||||
}
|
||||
|
||||
// `id` attributes are definitions and Poid attributes are pipeline-local
|
||||
// references; offering global asset ids for them would be wrong.
|
||||
if (attrInfo && isLocalReferenceAttribute(elType, attr.name)) {
|
||||
return [];
|
||||
if (attrName === "id") return [];
|
||||
return this.localIdItems(el as LogicalElement, prefix, make);
|
||||
}
|
||||
|
||||
if (attrInfo?.refType) {
|
||||
if (!idx) return [];
|
||||
return this.assetIdItems(idx, null, attrInfo.refType, prefix, make);
|
||||
}
|
||||
if (attrInfo?.enumValues?.length) {
|
||||
@@ -264,7 +275,7 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
|
||||
.filter((v) => v.startsWith(prefix.toLowerCase()))
|
||||
.map((v) => make(v, vscode.CompletionItemKind.Value, "boolean"));
|
||||
}
|
||||
if (attrInfo?.allowsDefine) {
|
||||
if (idx && attrInfo?.allowsDefine) {
|
||||
return this.defineItems(idx, prefix, make);
|
||||
}
|
||||
return [];
|
||||
@@ -304,8 +315,12 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
|
||||
): vscode.CompletionItem[] {
|
||||
const lower = prefix.toLowerCase();
|
||||
const scored: { def: AssetDef; score: number }[] = [];
|
||||
const seen = new Set<string>();
|
||||
|
||||
const consider = (def: AssetDef) => {
|
||||
const key = `${def.type}:${def.id.toLowerCase()}:${def.file}:${def.line}`;
|
||||
if (seen.has(key)) return;
|
||||
seen.add(key);
|
||||
if (!def.id.toLowerCase().startsWith(lower)) return;
|
||||
let score = 3;
|
||||
if (refType && model.isAssignableTo(def.type, refType)) score = 1;
|
||||
@@ -315,6 +330,18 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
|
||||
};
|
||||
|
||||
const targetType = selfType ?? refType;
|
||||
const localAssets = idx.local?.assets;
|
||||
const localById = idx.local?.assetsById;
|
||||
if (localAssets || localById) {
|
||||
if (!targetType) {
|
||||
for (const list of localById!.values()) for (const d of list) consider(d);
|
||||
} else {
|
||||
for (const [typeName, byId] of localAssets!) {
|
||||
if (!model.isAssignableTo(typeName, targetType)) continue;
|
||||
for (const list of byId.values()) for (const d of list) consider(d);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (!targetType) {
|
||||
for (const list of idx.assetsById.values()) for (const d of list) consider(d);
|
||||
} else {
|
||||
@@ -343,24 +370,54 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
|
||||
): vscode.CompletionItem[] {
|
||||
const lower = prefix.replace(/^[=$]*/, "").toLowerCase();
|
||||
const items: vscode.CompletionItem[] = [];
|
||||
for (const [key, defs] of idx.defines) {
|
||||
if (!key.includes(lower)) continue;
|
||||
const def = defs[0];
|
||||
const label = `$${def.name}`;
|
||||
const item = make(label, vscode.CompletionItemKind.Constant, "Define", def.value);
|
||||
item.insertText = label;
|
||||
items.push(item);
|
||||
const seen = new Set<string>();
|
||||
for (const defines of [idx.local?.defines, idx.defines]) {
|
||||
if (!defines) continue;
|
||||
for (const [key, defs] of defines) {
|
||||
if (!key.includes(lower)) continue;
|
||||
const def = defs[0];
|
||||
const dedupe = `${def.name.toLowerCase()}:${def.file}:${def.line}`;
|
||||
if (seen.has(dedupe)) continue;
|
||||
seen.add(dedupe);
|
||||
const label = `$${def.name}`;
|
||||
const item = make(label, vscode.CompletionItemKind.Constant, "Define", def.value);
|
||||
item.insertText = label;
|
||||
items.push(item);
|
||||
}
|
||||
}
|
||||
return items.slice(0, MAX_VALUE_ITEMS);
|
||||
}
|
||||
|
||||
private localIdItems(
|
||||
el: LogicalElement,
|
||||
prefix: string,
|
||||
make: (label: string, kind: vscode.CompletionItemKind, detail: string, doc?: string) => vscode.CompletionItem,
|
||||
): vscode.CompletionItem[] {
|
||||
const root = findContainingGameObject(el);
|
||||
if (!root) return [];
|
||||
const lower = prefix.toLowerCase();
|
||||
const items: vscode.CompletionItem[] = [];
|
||||
for (const { id } of collectLocalIds(root)) {
|
||||
if (!id.toLowerCase().startsWith(lower)) continue;
|
||||
items.push(
|
||||
make(
|
||||
id,
|
||||
vscode.CompletionItemKind.Value,
|
||||
"local module",
|
||||
"Pipeline-local id in the enclosing GameObject (includes xi:include targets).",
|
||||
),
|
||||
);
|
||||
}
|
||||
return items;
|
||||
}
|
||||
|
||||
// ── Element content ───────────────────────────────────────────────
|
||||
|
||||
private contentItems(
|
||||
ctx: CompletionContext,
|
||||
_document: vscode.TextDocument,
|
||||
_position: vscode.Position,
|
||||
_idx: ModIndex,
|
||||
_idx: ModIndex | null,
|
||||
): vscode.CompletionItem[] {
|
||||
const el = ctx.element;
|
||||
if (!el) return [];
|
||||
|
||||
+78
-25
@@ -1,6 +1,6 @@
|
||||
import * as vscode from "vscode";
|
||||
import { dirname } from "node:path";
|
||||
import { LineMap, parseXml, type XmlElement } from "../language/xmlParser";
|
||||
import { LineMap, type XmlElement } from "../language/xmlParser";
|
||||
import { resolveElementType } from "../language/typeContext";
|
||||
import { resolveSource, buildSearchPaths } from "../indexer/includeResolver";
|
||||
import * as model from "../model/schemaModel";
|
||||
@@ -8,8 +8,11 @@ import type { ModWorkspace } from "../workspace";
|
||||
import type { ModIndex } from "../indexer/types";
|
||||
import {
|
||||
isReferenceAttributeOfType,
|
||||
mergeLocalAndGlobalDefs,
|
||||
resolveReferenceTargetsForType,
|
||||
} from "../indexer/refs";
|
||||
import type { LogicalElement } from "../indexer/logicalTree";
|
||||
import { scopePathKey } from "../indexer/localScope";
|
||||
|
||||
export class Ra3Diagnostics {
|
||||
private collection: vscode.DiagnosticCollection;
|
||||
@@ -19,15 +22,19 @@ export class Ra3Diagnostics {
|
||||
}
|
||||
|
||||
async update(document: vscode.TextDocument): Promise<void> {
|
||||
const idx = this.ws.index;
|
||||
if (!idx) {
|
||||
if (!this.ws.isRa3Workspace()) {
|
||||
this.collection.set(document.uri, []);
|
||||
return;
|
||||
}
|
||||
const scope = await this.ws.getScope(document);
|
||||
const idx = scope.merged;
|
||||
const text = document.getText();
|
||||
const lineMap = new LineMap(text);
|
||||
const doc = parseXml(text);
|
||||
const doc = scope.parse;
|
||||
const diags: vscode.Diagnostic[] = [];
|
||||
// Reference/duplicate checks are provisional while the index is
|
||||
// incomplete or stale: "not found" may be a false positive.
|
||||
const provisional = idx ? !idx.complete || idx.stale === true : false;
|
||||
|
||||
for (const err of doc.errors) {
|
||||
diags.push(
|
||||
@@ -44,7 +51,15 @@ export class Ra3Diagnostics {
|
||||
}
|
||||
|
||||
if (doc.root) {
|
||||
this.checkElements(doc.root, doc, lineMap, idx, document, diags);
|
||||
this.checkElements(
|
||||
scope.expanded.root,
|
||||
scope.expanded,
|
||||
lineMap,
|
||||
idx,
|
||||
document,
|
||||
diags,
|
||||
provisional,
|
||||
);
|
||||
}
|
||||
|
||||
this.collection.set(document.uri, diags);
|
||||
@@ -59,19 +74,29 @@ export class Ra3Diagnostics {
|
||||
}
|
||||
|
||||
private checkElements(
|
||||
root: XmlElement,
|
||||
doc: { elements: XmlElement[] },
|
||||
root: LogicalElement | null,
|
||||
doc: { elements: LogicalElement[] },
|
||||
lineMap: LineMap,
|
||||
idx: ModIndex,
|
||||
idx: ModIndex | null,
|
||||
document: vscode.TextDocument,
|
||||
diags: vscode.Diagnostic[],
|
||||
provisional: boolean,
|
||||
): void {
|
||||
const settings = this.ws.settings;
|
||||
const fileDuplicates = new Map<string, { line: number }>();
|
||||
|
||||
for (const el of doc.elements) {
|
||||
// Only report diagnostics for nodes that belong to the document being
|
||||
// edited. Nodes spliced in through xi:include keep their own source
|
||||
// file and are diagnosed when that file is opened.
|
||||
if (scopePathKey(el.sourceFile) !== scopePathKey(document.uri.fsPath)) {
|
||||
continue;
|
||||
}
|
||||
const local = localName(el.name);
|
||||
const isTopLevel = el.parent === root && !["Tags", "Includes", "Defines"].includes(local);
|
||||
const isTopLevel =
|
||||
root !== null &&
|
||||
el.parent === root &&
|
||||
!["Tags", "Includes", "Defines"].includes(local);
|
||||
const range = tagRange(document, el);
|
||||
|
||||
// Top-level assets must have an id.
|
||||
@@ -111,6 +136,7 @@ export class Ra3Diagnostics {
|
||||
document,
|
||||
idx,
|
||||
diags,
|
||||
provisional,
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -169,6 +195,7 @@ export class Ra3Diagnostics {
|
||||
document,
|
||||
idx,
|
||||
diags,
|
||||
provisional,
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -184,12 +211,17 @@ export class Ra3Diagnostics {
|
||||
type: string,
|
||||
id: string,
|
||||
document: vscode.TextDocument,
|
||||
idx: ModIndex,
|
||||
idx: ModIndex | null,
|
||||
diags: vscode.Diagnostic[],
|
||||
provisional: boolean,
|
||||
): void {
|
||||
if (!idx) return;
|
||||
const byType = idx.assets.get(type);
|
||||
const defs = byType?.get(id.toLowerCase());
|
||||
if (!defs || defs.length < 2) return;
|
||||
const defs = mergeLocalAndGlobalDefs(
|
||||
idx.local?.assets.get(type)?.get(id.toLowerCase()),
|
||||
byType?.get(id.toLowerCase()),
|
||||
);
|
||||
if (defs.length < 2) return;
|
||||
const self = defs.filter(
|
||||
(d) =>
|
||||
d.origin === "project" &&
|
||||
@@ -211,7 +243,8 @@ export class Ra3Diagnostics {
|
||||
diags.push(
|
||||
this.diag(
|
||||
range,
|
||||
`Duplicate id "${id}" for <${type}> (also defined in ${other.file})`,
|
||||
`Duplicate id "${id}" for <${type}> (also defined in ${other.file})` +
|
||||
(provisional ? " (based on a partial index)" : ""),
|
||||
vscode.DiagnosticSeverity.Error,
|
||||
"duplicate-id",
|
||||
),
|
||||
@@ -225,8 +258,9 @@ export class Ra3Diagnostics {
|
||||
value: string,
|
||||
attr: { valueStart: number; valueEnd: number },
|
||||
document: vscode.TextDocument,
|
||||
idx: ModIndex,
|
||||
idx: ModIndex | null,
|
||||
diags: vscode.Diagnostic[],
|
||||
provisional: boolean,
|
||||
): void {
|
||||
if (!value) return;
|
||||
const range = new vscode.Range(
|
||||
@@ -238,13 +272,19 @@ export class Ra3Diagnostics {
|
||||
const defineRe = /\$([A-Za-z_][A-Za-z0-9_]*)/g;
|
||||
let m: RegExpExecArray | null;
|
||||
while ((m = defineRe.exec(value)) !== null) {
|
||||
if (!idx.defines.has(m[1].toLowerCase())) {
|
||||
if (
|
||||
idx &&
|
||||
!(idx.local?.defines.has(m[1].toLowerCase()) ??
|
||||
idx.defines.has(m[1].toLowerCase()))
|
||||
) {
|
||||
const code = provisional ? "undefined-define-indexing" : "undefined-define";
|
||||
diags.push(
|
||||
this.diag(
|
||||
range,
|
||||
`Undefined define "$${m[1]}"`,
|
||||
`Undefined define "$${m[1]}"` +
|
||||
(provisional ? " (index incomplete — may be a false positive)" : ""),
|
||||
vscode.DiagnosticSeverity.Warning,
|
||||
"undefined-define",
|
||||
code,
|
||||
),
|
||||
);
|
||||
}
|
||||
@@ -253,10 +293,13 @@ export class Ra3Diagnostics {
|
||||
if (value.startsWith("$") || value.startsWith("=")) return;
|
||||
const severity = this.ws.settings.reportUnresolvedReferences;
|
||||
if (severity === "none") return;
|
||||
if (!idx) return;
|
||||
if (!isReferenceAttributeOfType(elType, attrName)) return;
|
||||
const targets = resolveReferenceTargetsForType(idx, elType, attrName, value);
|
||||
if (targets.length) return;
|
||||
const anyDef = idx.assetsById.has(value.toLowerCase());
|
||||
const anyDef =
|
||||
(idx.local?.assetsById.has(value.toLowerCase()) ?? false) ||
|
||||
idx.assetsById.has(value.toLowerCase());
|
||||
const attrRef = model
|
||||
.attributesOfType(elType)
|
||||
.find((a) => a.name === attrName);
|
||||
@@ -265,16 +308,20 @@ export class Ra3Diagnostics {
|
||||
: attrRef?.isRef
|
||||
? "of the expected declared type"
|
||||
: "matching";
|
||||
const code = provisional ? "unresolved-reference-indexing" : "unresolved-reference";
|
||||
const baseMessage = anyDef
|
||||
? `Reference "${value}" has no definition ${expected} (ids with the same name exist for other types)`
|
||||
: `Unresolved reference "${value}" (not found in the current index)`;
|
||||
diags.push(
|
||||
this.diag(
|
||||
range,
|
||||
anyDef
|
||||
? `Reference "${value}" has no definition ${expected} (ids with the same name exist for other types)`
|
||||
: `Unresolved reference "${value}" (not found in the current index)`,
|
||||
provisional
|
||||
? `${baseMessage} (index incomplete — may be a false positive)`
|
||||
: baseMessage,
|
||||
severity === "warning"
|
||||
? vscode.DiagnosticSeverity.Warning
|
||||
: vscode.DiagnosticSeverity.Information,
|
||||
"unresolved-reference",
|
||||
code,
|
||||
),
|
||||
);
|
||||
}
|
||||
@@ -282,7 +329,7 @@ export class Ra3Diagnostics {
|
||||
private checkInclude(
|
||||
el: XmlElement,
|
||||
document: vscode.TextDocument,
|
||||
idx: ModIndex,
|
||||
idx: ModIndex | null,
|
||||
diags: vscode.Diagnostic[],
|
||||
): void {
|
||||
const typeAttr = el.attrs.find((a) => a.name === "type");
|
||||
@@ -301,12 +348,18 @@ export class Ra3Diagnostics {
|
||||
);
|
||||
}
|
||||
if (!sourceAttr?.hasValue) return;
|
||||
const searchPaths = idx
|
||||
? buildSearchPaths(idx.sdkDir, idx.projectDir)
|
||||
: this.ws.searchPaths();
|
||||
if (!searchPaths) return;
|
||||
const resolved = resolveSource(
|
||||
sourceAttr.value,
|
||||
dirname(document.uri.fsPath),
|
||||
buildSearchPaths(idx.sdkDir, idx.projectDir),
|
||||
searchPaths,
|
||||
);
|
||||
if (!resolved.path && !idx.sourceCandidates.some((c) => c.source === sourceAttr.value)) {
|
||||
const candidateHit =
|
||||
idx?.sourceCandidates.some((c) => c.source === sourceAttr.value) ?? false;
|
||||
if (!resolved.path && !candidateHit) {
|
||||
diags.push(
|
||||
this.diag(
|
||||
new vscode.Range(
|
||||
|
||||
+62
-11
@@ -1,13 +1,19 @@
|
||||
import * as vscode from "vscode";
|
||||
import { findElementAt, parseXml } from "../language/xmlParser";
|
||||
import { findElementAt } from "../language/xmlParser";
|
||||
import { resolveElementType } from "../language/typeContext";
|
||||
import * as model from "../model/schemaModel";
|
||||
import type { ModWorkspace } from "../workspace";
|
||||
import type { ModIndex } from "../indexer/types";
|
||||
import {
|
||||
isLocalReferenceAttribute,
|
||||
isReferenceAttributeOfType,
|
||||
resolveReferenceTargetsForType,
|
||||
} from "../indexer/refs";
|
||||
import {
|
||||
findContainingGameObject,
|
||||
findLocalId,
|
||||
type LogicalElement,
|
||||
} from "../indexer/logicalTree";
|
||||
import { scopePathKey, type DocumentScope } from "../indexer/localScope";
|
||||
import { dirname } from "node:path";
|
||||
import { buildSearchPaths, resolveSource } from "../indexer/includeResolver";
|
||||
|
||||
@@ -19,9 +25,10 @@ export class Ra3HoverProvider implements vscode.HoverProvider {
|
||||
position: vscode.Position,
|
||||
_token: vscode.CancellationToken,
|
||||
): Promise<vscode.Hover | null> {
|
||||
const text = document.getText();
|
||||
if (!this.ws.isRa3Workspace()) return null;
|
||||
const offset = document.offsetAt(position);
|
||||
const doc = parseXml(text);
|
||||
const scope = await this.ws.getScope(document);
|
||||
const doc = scope.expanded;
|
||||
const el = findElementAt(doc, offset);
|
||||
if (!el) return null;
|
||||
const elType = resolveElementType(el);
|
||||
@@ -35,7 +42,7 @@ export class Ra3HoverProvider implements vscode.HoverProvider {
|
||||
// Attribute value.
|
||||
for (const attr of el.attrs) {
|
||||
if (attr.hasValue && offset >= attr.valueStart && offset <= attr.valueEnd) {
|
||||
return this.valueHover(el, elType, attr.name, attr.value, document, this.ws.index);
|
||||
return this.valueHover(el, elType, attr.name, attr.value, document, scope);
|
||||
}
|
||||
}
|
||||
// Element name.
|
||||
@@ -116,14 +123,17 @@ export class Ra3HoverProvider implements vscode.HoverProvider {
|
||||
attrName: string,
|
||||
value: string,
|
||||
document: vscode.TextDocument,
|
||||
idx: ModIndex | null,
|
||||
scope: DocumentScope,
|
||||
): vscode.Hover | null {
|
||||
const idx = scope.merged;
|
||||
const md = new vscode.MarkdownString();
|
||||
|
||||
// $DEFINE reference.
|
||||
const defineMatch = /\$([A-Za-z_][A-Za-z0-9_]*)/.exec(value);
|
||||
if (defineMatch && idx) {
|
||||
const defs = idx.defines.get(defineMatch[1].toLowerCase());
|
||||
const defs =
|
||||
idx.local?.defines.get(defineMatch[1].toLowerCase()) ??
|
||||
idx.defines.get(defineMatch[1].toLowerCase());
|
||||
if (defs?.length) {
|
||||
const d = defs[0];
|
||||
md.appendMarkdown(`**Define** \`$${d.name}\` \n`);
|
||||
@@ -139,11 +149,14 @@ export class Ra3HoverProvider implements vscode.HoverProvider {
|
||||
(el.name === "Include" && attrName === "source") ||
|
||||
(el.name === "xi:include" && attrName === "href")
|
||||
) {
|
||||
const resolved = idx
|
||||
const searchPaths = idx
|
||||
? buildSearchPaths(idx.sdkDir, idx.projectDir)
|
||||
: this.ws.searchPaths();
|
||||
const resolved = searchPaths
|
||||
? resolveSource(
|
||||
value,
|
||||
dirname(document.uri.fsPath),
|
||||
buildSearchPaths(idx.sdkDir, idx.projectDir),
|
||||
searchPaths,
|
||||
).path
|
||||
: null;
|
||||
if (resolved) {
|
||||
@@ -161,8 +174,26 @@ export class Ra3HoverProvider implements vscode.HoverProvider {
|
||||
return new vscode.Hover(md);
|
||||
}
|
||||
|
||||
// Pipeline-local (Poid) references: resolve inside the enclosing
|
||||
// GameObject's logical subtree (including xi:include targets).
|
||||
if (
|
||||
isLocalReferenceAttribute(elType, attrName) &&
|
||||
attrName.toLowerCase() !== "id"
|
||||
) {
|
||||
return this.localIdHover(scope, el as LogicalElement, value, document);
|
||||
}
|
||||
|
||||
// Asset reference / inheritFrom.
|
||||
if (idx) {
|
||||
if (!idx) {
|
||||
if (isReferenceAttributeOfType(elType, attrName)) {
|
||||
md.appendMarkdown(
|
||||
"Index is still building — references cannot be resolved yet.",
|
||||
);
|
||||
return new vscode.Hover(md);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
{
|
||||
if (!isReferenceAttributeOfType(elType, attrName)) return null;
|
||||
const targets = resolveReferenceTargetsForType(idx, elType, attrName, value);
|
||||
if (targets.length) {
|
||||
@@ -191,8 +222,28 @@ export class Ra3HoverProvider implements vscode.HoverProvider {
|
||||
);
|
||||
return new vscode.Hover(md);
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
private localIdHover(
|
||||
scope: DocumentScope,
|
||||
el: LogicalElement,
|
||||
value: string,
|
||||
document: vscode.TextDocument,
|
||||
): vscode.Hover | null {
|
||||
const root = findContainingGameObject(el);
|
||||
if (!root) return null;
|
||||
const target = findLocalId(root, value);
|
||||
if (!target) return null;
|
||||
const idAttr = target.attrs.find((a) => a.name === "id");
|
||||
if (!idAttr?.hasValue) return null;
|
||||
const lineMap = scope.lineMaps.get(scopePathKey(target.sourceFile));
|
||||
const line = lineMap ? lineMap.positionAt(idAttr.valueStart).line + 1 : 0;
|
||||
const md = new vscode.MarkdownString();
|
||||
md.appendMarkdown(`**Local pipeline id** \`${value}\` \n`);
|
||||
md.appendCodeblock(`<${target.name}>`);
|
||||
const rel = relativePath(document, target.sourceFile);
|
||||
md.appendMarkdown(`Defined in \`${rel}:${line}\``);
|
||||
return new vscode.Hover(md);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+113
-16
@@ -7,7 +7,16 @@ import {
|
||||
resolveSource,
|
||||
type SearchPaths,
|
||||
} from "../indexer/includeResolver";
|
||||
import { resolveReferenceTargetsForType } from "../indexer/refs";
|
||||
import {
|
||||
isLocalReferenceAttribute,
|
||||
resolveReferenceTargetsForType,
|
||||
} from "../indexer/refs";
|
||||
import {
|
||||
findContainingGameObject,
|
||||
findLocalId,
|
||||
type LogicalElement,
|
||||
} from "../indexer/logicalTree";
|
||||
import { scopePathKey, type DocumentScope } from "../indexer/localScope";
|
||||
import type { ModWorkspace } from "../workspace";
|
||||
import type { AssetDef, ModIndex } from "../indexer/types";
|
||||
|
||||
@@ -25,11 +34,11 @@ export class Ra3DefinitionProvider implements vscode.DefinitionProvider {
|
||||
position: vscode.Position,
|
||||
_token: vscode.CancellationToken,
|
||||
): Promise<vscode.Location | vscode.Location[] | null> {
|
||||
const idx = this.ws.index;
|
||||
if (!idx) return null;
|
||||
const text = document.getText();
|
||||
if (!this.ws.isRa3Workspace()) return null;
|
||||
const scope = await this.ws.getScope(document);
|
||||
const idx = scope.merged;
|
||||
const offset = document.offsetAt(position);
|
||||
const doc = parseXml(text);
|
||||
const doc = scope.expanded;
|
||||
const el = findElementAt(doc, offset);
|
||||
if (!el) return null;
|
||||
const elType = resolveElementType(el);
|
||||
@@ -46,17 +55,31 @@ export class Ra3DefinitionProvider implements vscode.DefinitionProvider {
|
||||
(el.name === "Include" && nameLower === "source") ||
|
||||
(el.name === "include" && nameLower === "href")
|
||||
) {
|
||||
const resolved =
|
||||
resolveSource(value, dirname(document.uri.fsPath), searchPathsFor(idx)).path ??
|
||||
idx.sourceCandidates.find((c) => c.source === value)?.path ??
|
||||
null;
|
||||
return resolved
|
||||
? new vscode.Location(vscode.Uri.file(resolved), new vscode.Position(0, 0))
|
||||
const searchPaths = idx ? searchPathsFor(idx) : this.ws.searchPaths();
|
||||
const resolved = searchPaths
|
||||
? resolveSource(value, dirname(document.uri.fsPath), searchPaths).path
|
||||
: null;
|
||||
const fallback = idx?.sourceCandidates.find((c) => c.source === value)?.path;
|
||||
const target = resolved ?? fallback ?? null;
|
||||
return target
|
||||
? new vscode.Location(vscode.Uri.file(target), new vscode.Position(0, 0))
|
||||
: null;
|
||||
}
|
||||
|
||||
// Asset reference / inheritFrom (filtered by the attribute's ref type).
|
||||
if (!idx) return null;
|
||||
if (value && !value.startsWith("$")) {
|
||||
if (
|
||||
isLocalReferenceAttribute(elType, attr.name) &&
|
||||
nameLower !== "id"
|
||||
) {
|
||||
const local = this.localIdLocation(
|
||||
scope,
|
||||
el as LogicalElement,
|
||||
value,
|
||||
);
|
||||
if (local) return local;
|
||||
}
|
||||
let targets = resolveReferenceTargetsForType(idx, elType, attr.name, value);
|
||||
if (!targets.length) return null;
|
||||
if (
|
||||
@@ -67,13 +90,40 @@ export class Ra3DefinitionProvider implements vscode.DefinitionProvider {
|
||||
}
|
||||
const locations: vscode.Location[] = [];
|
||||
for (const { def } of targets.slice(0, 8)) {
|
||||
const loc = await assetDefLocation(this.ws, def, idx);
|
||||
const loc = await assetDefLocation(this.ws, def, idx, scope, document);
|
||||
if (loc) locations.push(loc);
|
||||
}
|
||||
return locations.length ? locations : null;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
private localIdLocation(
|
||||
scope: DocumentScope,
|
||||
el: LogicalElement,
|
||||
value: string,
|
||||
): vscode.Location | null {
|
||||
const root = findContainingGameObject(el);
|
||||
if (!root) return null;
|
||||
const target = findLocalId(root, value);
|
||||
if (!target) return null;
|
||||
const idAttr = target.attrs.find((a) => a.name === "id");
|
||||
if (!idAttr?.hasValue) return null;
|
||||
const lineMap = scope.lineMaps.get(scopePathKey(target.sourceFile));
|
||||
if (!lineMap) {
|
||||
return new vscode.Location(
|
||||
vscode.Uri.file(target.sourceFile),
|
||||
new vscode.Position(0, 0),
|
||||
);
|
||||
}
|
||||
return new vscode.Location(
|
||||
vscode.Uri.file(target.sourceFile),
|
||||
new vscode.Range(
|
||||
toVscodePosition(lineMap.positionAt(idAttr.valueStart)),
|
||||
toVscodePosition(lineMap.positionAt(idAttr.valueEnd)),
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -85,6 +135,8 @@ async function assetDefLocation(
|
||||
ws: ModWorkspace,
|
||||
def: AssetDef,
|
||||
idx: ModIndex,
|
||||
scope: DocumentScope,
|
||||
currentDocument: vscode.TextDocument,
|
||||
): Promise<vscode.Location | null> {
|
||||
if (def.origin === "manifest") {
|
||||
const src = def.manifestSource;
|
||||
@@ -100,6 +152,11 @@ async function assetDefLocation(
|
||||
return null;
|
||||
}
|
||||
|
||||
if (scopePathKey(def.file) === scopePathKey(currentDocument.uri.fsPath)) {
|
||||
const precise = locationInCurrentDocument(scope, def.id, currentDocument);
|
||||
if (precise) return precise;
|
||||
}
|
||||
|
||||
return (
|
||||
(await locationInDocument(ws, def.file, def.id)) ??
|
||||
new vscode.Location(
|
||||
@@ -109,6 +166,36 @@ async function assetDefLocation(
|
||||
);
|
||||
}
|
||||
|
||||
function locationInCurrentDocument(
|
||||
scope: DocumentScope,
|
||||
id: string,
|
||||
document: vscode.TextDocument,
|
||||
): vscode.Location | null {
|
||||
const el = scope.parse.elements.find((e) =>
|
||||
e.attrs.some(
|
||||
(a) => a.name === "id" && a.value.toLowerCase() === id.toLowerCase(),
|
||||
),
|
||||
);
|
||||
if (!el) return null;
|
||||
const idAttr = el.attrs.find((a) => a.name === "id");
|
||||
if (idAttr?.hasValue) {
|
||||
return new vscode.Location(
|
||||
document.uri,
|
||||
new vscode.Range(
|
||||
document.positionAt(idAttr.valueStart),
|
||||
document.positionAt(idAttr.valueEnd),
|
||||
),
|
||||
);
|
||||
}
|
||||
return new vscode.Location(
|
||||
document.uri,
|
||||
new vscode.Range(
|
||||
document.positionAt(el.start),
|
||||
document.positionAt(el.startTagEnd),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Finds the precise range of an asset definition inside an XML file: the id
|
||||
* attribute value when present, otherwise the element start tag.
|
||||
@@ -118,7 +205,9 @@ async function locationInDocument(
|
||||
file: string,
|
||||
id: string,
|
||||
): Promise<vscode.Location | null> {
|
||||
const parsed = await ws.indexer?.readDocument(file);
|
||||
// readDom (not readDocument) guarantees a DOM even when the compact
|
||||
// records cache already has an entry for the file.
|
||||
const parsed = await ws.indexer?.readDom(file);
|
||||
if (parsed?.parse && parsed.lineMap) {
|
||||
const el = parsed.parse.elements.find(
|
||||
(e) =>
|
||||
@@ -156,12 +245,15 @@ function toVscodePosition(p: { line: number; character: number }): vscode.Positi
|
||||
// ── Find all references ─────────────────────────────────────────────
|
||||
|
||||
export class Ra3ReferenceProvider implements vscode.ReferenceProvider {
|
||||
constructor(private ws: ModWorkspace) {}
|
||||
|
||||
async provideReferences(
|
||||
document: vscode.TextDocument,
|
||||
position: vscode.Position,
|
||||
_context: vscode.ReferenceContext,
|
||||
_token: vscode.CancellationToken,
|
||||
): Promise<vscode.Location[] | null> {
|
||||
if (!this.ws.isRa3Workspace()) return null;
|
||||
const text = document.getText();
|
||||
const offset = document.offsetAt(position);
|
||||
const doc = parseXml(text);
|
||||
@@ -201,8 +293,10 @@ export class Ra3DocumentLinkProvider implements vscode.DocumentLinkProvider {
|
||||
document: vscode.TextDocument,
|
||||
_token: vscode.CancellationToken,
|
||||
): Promise<vscode.DocumentLink[]> {
|
||||
if (!this.ws.isRa3Workspace()) return [];
|
||||
const idx = this.ws.index;
|
||||
if (!idx) return [];
|
||||
const searchPaths = idx ? searchPathsFor(idx) : this.ws.searchPaths();
|
||||
if (!searchPaths) return [];
|
||||
const text = document.getText();
|
||||
const doc = parseXml(text);
|
||||
const links: vscode.DocumentLink[] = [];
|
||||
@@ -214,9 +308,9 @@ export class Ra3DocumentLinkProvider implements vscode.DocumentLinkProvider {
|
||||
resolveSource(
|
||||
srcAttr.value,
|
||||
dirname(document.uri.fsPath),
|
||||
searchPathsFor(idx),
|
||||
searchPaths,
|
||||
).path ??
|
||||
idx.sourceCandidates.find((c) => c.source === srcAttr.value)?.path ??
|
||||
idx?.sourceCandidates.find((c) => c.source === srcAttr.value)?.path ??
|
||||
null;
|
||||
if (!target) continue;
|
||||
links.push(
|
||||
@@ -236,10 +330,13 @@ export class Ra3DocumentLinkProvider implements vscode.DocumentLinkProvider {
|
||||
// ── Document symbols (outline) ──────────────────────────────────────
|
||||
|
||||
export class Ra3DocumentSymbolProvider implements vscode.DocumentSymbolProvider {
|
||||
constructor(private ws: ModWorkspace) {}
|
||||
|
||||
async provideDocumentSymbols(
|
||||
document: vscode.TextDocument,
|
||||
_token: vscode.CancellationToken,
|
||||
): Promise<vscode.DocumentSymbol[]> {
|
||||
if (!this.ws.isRa3Workspace()) return [];
|
||||
const text = document.getText();
|
||||
const doc = parseXml(text);
|
||||
const root = doc.root;
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import * as vscode from "vscode";
|
||||
import { parseXml } from "../language/xmlParser";
|
||||
import { buildSemanticTokenRanges } from "../language/semanticTokens";
|
||||
import type { ModWorkspace } from "../workspace";
|
||||
|
||||
const TOKEN_TYPES = ["type", "property", "string"] as const;
|
||||
|
||||
@@ -20,10 +21,15 @@ export const RA3_SEMANTIC_TOKENS_LEGEND = new vscode.SemanticTokensLegend([
|
||||
export class Ra3SemanticTokensProvider
|
||||
implements vscode.DocumentSemanticTokensProvider
|
||||
{
|
||||
constructor(private ws: ModWorkspace) {}
|
||||
|
||||
async provideDocumentSemanticTokens(
|
||||
document: vscode.TextDocument,
|
||||
_token: vscode.CancellationToken,
|
||||
): Promise<vscode.SemanticTokens> {
|
||||
if (!this.ws.isRa3Workspace()) {
|
||||
return new vscode.SemanticTokens(new Uint32Array(0));
|
||||
}
|
||||
const text = document.getText();
|
||||
const doc = parseXml(text);
|
||||
if (doc.errors.length === 0) {
|
||||
|
||||
@@ -146,6 +146,11 @@ export class IndexRecordsCache {
|
||||
return this.map.size;
|
||||
}
|
||||
|
||||
/** Iterates [normalized key, entry] pairs (used by disk persistence). */
|
||||
entries(): IterableIterator<[string, IndexRecordsCacheEntry]> {
|
||||
return this.map.entries();
|
||||
}
|
||||
|
||||
set(path: string, entry: IndexRecordsCacheEntry): void {
|
||||
const key = normKey(path);
|
||||
this.map.delete(key);
|
||||
@@ -224,3 +229,35 @@ export class IncludeResolveCache {
|
||||
return this.map.size;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Monotonic counter for workspace-level invalidations.
|
||||
*
|
||||
* A build captures `snapshot()` when it starts; if `changedSince()` is true
|
||||
* when a phase snapshot is about to be published, files may have changed
|
||||
* mid-build, so the published index is marked stale (and the workspace's
|
||||
* dirty/rebuild mechanism converges to fresh data shortly after).
|
||||
*/
|
||||
export class InvalidationsEpoch {
|
||||
private value = 0;
|
||||
|
||||
/** Records a new invalidation (content, creation or deletion). */
|
||||
mark(): void {
|
||||
this.value++;
|
||||
}
|
||||
|
||||
/** Returns the current epoch value. */
|
||||
snapshot(): number {
|
||||
return this.value;
|
||||
}
|
||||
|
||||
/** True when at least one invalidation happened since `epoch`. */
|
||||
changedSince(epoch: number): boolean {
|
||||
return this.value !== epoch;
|
||||
}
|
||||
|
||||
/** Current epoch value (read-only accessor). */
|
||||
get current(): number {
|
||||
return this.value;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,208 @@
|
||||
/**
|
||||
* On-disk persistence for per-file index records.
|
||||
*
|
||||
* The records cache (top-level assets / defines / includes / xi:include with
|
||||
* line numbers) is tiny compared to the source corpus (Corona: ~9k files,
|
||||
* ~10 MB in memory), but rebuilding it from scratch means reading ~2.6 GB of
|
||||
* art assets again. Persisting it makes a cold start cost a stat validation
|
||||
* pass (~seconds on SSD, a few to tens of seconds on a mechanical drive)
|
||||
* instead of a full rebuild.
|
||||
*
|
||||
* Correctness model (layered):
|
||||
* - every cached record stores a multi-signal stamp
|
||||
* `{ size, mtimeMs, birthtimeMs, ctimeMs }`;
|
||||
* - on load, each file is stat-validated (no content reads); mismatches and
|
||||
* missing files are dropped and re-read during the build;
|
||||
* - during a session the file watcher invalidates entries precisely;
|
||||
* - `ra3modxml.reindex` / `ra3modxml.clearCache` remain the final authority.
|
||||
*
|
||||
* The file is gzip-compressed JSON written atomically (temp + rename), keyed
|
||||
* by project identity + settings so stale caches are ignored automatically.
|
||||
*
|
||||
* Pure TypeScript: no vscode dependency.
|
||||
*/
|
||||
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdir, readFile, rename, rm, stat, writeFile } from "node:fs/promises";
|
||||
import { dirname } from "node:path";
|
||||
import { gzip, gunzip } from "node:zlib";
|
||||
import { promisify } from "node:util";
|
||||
import type { IndexRecordsCacheEntry } from "./caches";
|
||||
import type { IndexRecords } from "./records";
|
||||
import type { IndexedFile } from "./types";
|
||||
|
||||
const gzipAsync = promisify(gzip);
|
||||
const gunzipAsync = promisify(gunzip);
|
||||
|
||||
export const DISK_CACHE_VERSION = 1;
|
||||
/** How many stat validations run concurrently on load. */
|
||||
const VALIDATE_CONCURRENCY = 32;
|
||||
|
||||
/** Settings that change what the index contains; a mismatch ignores the cache. */
|
||||
export interface DiskCacheIdentity {
|
||||
projectDir: string;
|
||||
sdkDir: string;
|
||||
indexSageXml: boolean;
|
||||
additionalDataSearchPaths: string[];
|
||||
builtmodsDirs: string[];
|
||||
}
|
||||
|
||||
export interface DiskCacheRecord {
|
||||
/** Normalized cache key (see `normKey`). */
|
||||
key: string;
|
||||
stat: NonNullable<IndexedFile["stat"]>;
|
||||
records: IndexRecords;
|
||||
kind: "full" | "shallow";
|
||||
}
|
||||
|
||||
interface DiskCacheFile {
|
||||
version: number;
|
||||
key: string;
|
||||
savedAt: string;
|
||||
records: DiskCacheRecord[];
|
||||
}
|
||||
|
||||
export interface DiskCacheLoadStats {
|
||||
fileExists: boolean;
|
||||
keyMatched: boolean;
|
||||
/** Records stored in the file. */
|
||||
loaded: number;
|
||||
/** Records whose stat still matches (kept). */
|
||||
validated: number;
|
||||
/** Records dropped because the file changed, moved or was deleted. */
|
||||
dropped: number;
|
||||
}
|
||||
|
||||
export function diskCacheKey(identity: DiskCacheIdentity): string {
|
||||
return createHash("sha256")
|
||||
.update(JSON.stringify(identity))
|
||||
.digest("hex")
|
||||
.slice(0, 16);
|
||||
}
|
||||
|
||||
export class DiskRecordsCache {
|
||||
constructor(
|
||||
private readonly filePath: string,
|
||||
private readonly identity: DiskCacheIdentity,
|
||||
) {}
|
||||
|
||||
get path(): string {
|
||||
return this.filePath;
|
||||
}
|
||||
|
||||
/**
|
||||
* Loads and stat-validates the cache. Returns the kept records plus load
|
||||
* statistics; missing/corrupt/key-mismatched caches yield an empty result
|
||||
* instead of an error.
|
||||
*/
|
||||
async loadValidated(): Promise<{
|
||||
records: DiskCacheRecord[];
|
||||
stats: DiskCacheLoadStats;
|
||||
}> {
|
||||
const stats: DiskCacheLoadStats = {
|
||||
fileExists: false,
|
||||
keyMatched: false,
|
||||
loaded: 0,
|
||||
validated: 0,
|
||||
dropped: 0,
|
||||
};
|
||||
let raw: DiskCacheFile | null = null;
|
||||
try {
|
||||
const buf = await readFile(this.filePath);
|
||||
stats.fileExists = true;
|
||||
const text = (await gunzipAsync(buf)).toString("utf8");
|
||||
const parsed = JSON.parse(text);
|
||||
if (
|
||||
parsed &&
|
||||
parsed.version === DISK_CACHE_VERSION &&
|
||||
parsed.key === diskCacheKey(this.identity) &&
|
||||
Array.isArray(parsed.records)
|
||||
) {
|
||||
raw = parsed as DiskCacheFile;
|
||||
}
|
||||
} catch {
|
||||
// Missing or corrupt cache: fall through with an empty result.
|
||||
}
|
||||
if (!raw) return { records: [], stats };
|
||||
|
||||
stats.keyMatched = true;
|
||||
stats.loaded = raw.records.length;
|
||||
const kept: DiskCacheRecord[] = [];
|
||||
for (let i = 0; i < raw.records.length; i += VALIDATE_CONCURRENCY) {
|
||||
const chunk = raw.records.slice(i, i + VALIDATE_CONCURRENCY);
|
||||
const results = await Promise.all(
|
||||
chunk.map(async (rec): Promise<DiskCacheRecord | null> => {
|
||||
try {
|
||||
const s = await stat(rec.key);
|
||||
if (
|
||||
s.isFile() &&
|
||||
s.size === rec.stat.size &&
|
||||
s.mtimeMs === rec.stat.mtimeMs &&
|
||||
s.birthtimeMs === rec.stat.birthtimeMs &&
|
||||
s.ctimeMs === rec.stat.ctimeMs
|
||||
) {
|
||||
return rec;
|
||||
}
|
||||
} catch {
|
||||
// File missing or inaccessible.
|
||||
}
|
||||
return null;
|
||||
}),
|
||||
);
|
||||
for (const r of results) {
|
||||
if (r) {
|
||||
kept.push(r);
|
||||
stats.validated++;
|
||||
} else {
|
||||
stats.dropped++;
|
||||
}
|
||||
}
|
||||
}
|
||||
return { records: kept, stats };
|
||||
}
|
||||
|
||||
/** Writes the current records cache atomically (temp file + rename). */
|
||||
async save(
|
||||
entries: Iterable<[string, IndexRecordsCacheEntry]>,
|
||||
): Promise<void> {
|
||||
const records: DiskCacheRecord[] = [];
|
||||
for (const [key, entry] of entries) {
|
||||
if (!entry.stat) continue;
|
||||
records.push({
|
||||
key,
|
||||
stat: entry.stat,
|
||||
records: entry.records,
|
||||
kind: entry.kind,
|
||||
});
|
||||
}
|
||||
const payload: DiskCacheFile = {
|
||||
version: DISK_CACHE_VERSION,
|
||||
key: diskCacheKey(this.identity),
|
||||
savedAt: new Date().toISOString(),
|
||||
records,
|
||||
};
|
||||
const buf = await gzipAsync(Buffer.from(JSON.stringify(payload), "utf8"));
|
||||
await mkdir(dirname(this.filePath), { recursive: true });
|
||||
const tmp = `${this.filePath}.tmp`;
|
||||
await writeFile(tmp, buf);
|
||||
await rename(tmp, this.filePath);
|
||||
}
|
||||
|
||||
/** Deletes the cache file (used by the clear-cache command). */
|
||||
async clear(): Promise<void> {
|
||||
try {
|
||||
await rm(this.filePath, { force: true });
|
||||
} catch {
|
||||
// Best effort.
|
||||
}
|
||||
}
|
||||
|
||||
async status(): Promise<{ exists: boolean; sizeBytes: number }> {
|
||||
try {
|
||||
const s = await stat(this.filePath);
|
||||
return { exists: s.isFile(), sizeBytes: s.size };
|
||||
} catch {
|
||||
return { exists: false, sizeBytes: 0 };
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,117 @@
|
||||
/**
|
||||
* Lazy file-existence snapshot for include resolution.
|
||||
*
|
||||
* `resolveSource` performs synchronous `statSync` existence checks against
|
||||
* every search base; a cold Corona build does ~110k of them (tens of seconds
|
||||
* on a mechanical drive). Instead, existence is answered by reading the
|
||||
* candidate's **parent directory** once (`readdir`, no per-file stat) and
|
||||
* caching the entry set for the rest of the build. Only directories that are
|
||||
* actually queried are ever listed, so a cold build pays a handful of
|
||||
* readdir calls instead of an upfront recursive enumeration of every search
|
||||
* root (which measurably slowed the XML phase).
|
||||
*
|
||||
* Correctness: the workspace clears `IncludeResolveCache` on file
|
||||
* create/delete; each rebuild creates a fresh snapshot, and the debounced
|
||||
* rebuild triggered by the watcher converges if anything changed mid-build.
|
||||
*
|
||||
* Pure TypeScript: no vscode dependency.
|
||||
*/
|
||||
|
||||
import { existsSync, readdirSync } from "node:fs";
|
||||
import { basename, dirname, parse, resolve, sep } from "node:path";
|
||||
import { normKey } from "./caches";
|
||||
import type { SearchPaths } from "./includeResolver";
|
||||
|
||||
/**
|
||||
* Answers file-existence questions from lazily read directory listings.
|
||||
* `has()` is authoritative for paths inside the roots and returns null for
|
||||
* paths the snapshot does not cover (the caller falls back to `statSync`).
|
||||
*/
|
||||
export class ExistenceSnapshot {
|
||||
private roots: string[] = [];
|
||||
/** parent dir (normalized) -> lowercased file names, or null (no dir). */
|
||||
private dirCache = new Map<string, Set<string> | null>();
|
||||
/** Existence answers served from cached directory listings. */
|
||||
hits = 0;
|
||||
/** Paths outside the snapshot that required a statSync fallback. */
|
||||
fallbacks = 0;
|
||||
|
||||
constructor(roots: string[]) {
|
||||
// Roots and lookups must use the same normalization (case-insensitive on
|
||||
// Windows), otherwise `startsWith` misses due to case differences.
|
||||
this.roots = roots.map((r) => {
|
||||
const n = normKey(r);
|
||||
return n.endsWith(sep) ? n : n + sep;
|
||||
});
|
||||
}
|
||||
|
||||
/** true/false when covered by the snapshot, null when not covered. */
|
||||
has(absPath: string): boolean | null {
|
||||
const parentKey = normKey(dirname(absPath));
|
||||
if (!this.isCovered(parentKey)) {
|
||||
this.fallbacks++;
|
||||
return null;
|
||||
}
|
||||
let entries = this.dirCache.get(parentKey);
|
||||
if (entries === undefined) {
|
||||
entries = listDirEntries(parentKey);
|
||||
this.dirCache.set(parentKey, entries);
|
||||
}
|
||||
this.hits++;
|
||||
return entries ? entries.has(basename(absPath).toLowerCase()) : false;
|
||||
}
|
||||
|
||||
private isCovered(dirKey: string): boolean {
|
||||
const key = dirKey.endsWith(sep) ? dirKey : dirKey + sep;
|
||||
for (const root of this.roots) {
|
||||
if (key.startsWith(root)) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
function listDirEntries(dir: string): Set<string> | null {
|
||||
try {
|
||||
const out = new Set<string>();
|
||||
for (const entry of readdirSync(dir, { withFileTypes: true })) {
|
||||
if (entry.isFile()) out.add(entry.name.toLowerCase());
|
||||
}
|
||||
return out;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* True when `dir` is a filesystem root (e.g. "C:\" or "/"). Such roots are
|
||||
* never treated as search bases (they may contain the whole disk).
|
||||
*/
|
||||
export function isDriveRoot(dir: string): boolean {
|
||||
const resolved = resolve(dir);
|
||||
return parse(resolved).root === resolved;
|
||||
}
|
||||
|
||||
/**
|
||||
* Builds the snapshot root list from the search bases: drive roots and
|
||||
* missing directories are skipped, and a root covered by a broader root is
|
||||
* dropped (e.g. `sdkDir` covers `sdkDir/SageXml`). No directory is listed
|
||||
* here; listings happen lazily per queried parent directory.
|
||||
*/
|
||||
export function buildExistenceSnapshot(searchPaths: SearchPaths): ExistenceSnapshot {
|
||||
const candidates = [
|
||||
...searchPaths.DATA,
|
||||
...searchPaths.ART,
|
||||
...searchPaths.AUDIO,
|
||||
].map((r) => resolve(r));
|
||||
candidates.sort((a, b) => a.length - b.length);
|
||||
|
||||
const roots: string[] = [];
|
||||
for (const root of candidates) {
|
||||
if (isDriveRoot(root)) continue;
|
||||
if (!existsSync(root)) continue;
|
||||
const normalized = normKey(root);
|
||||
if (roots.some((r) => normalized.startsWith(r + sep))) continue;
|
||||
roots.push(normalized);
|
||||
}
|
||||
return new ExistenceSnapshot(roots);
|
||||
}
|
||||
@@ -61,6 +61,38 @@ export class CachedDirectoryWalker implements FileWalker {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* True for paths whose changes can never affect the index: `.git` internals
|
||||
* touched by background fetch/maintenance, and transient temp/backup files
|
||||
* created by editors or other extensions (`UnitCrate.xml.git`, `*.tmp`,
|
||||
* `*.lock`, `file~`, `.#file`, ...). Such events are ignored by the file
|
||||
* watcher instead of triggering rebuilds.
|
||||
*/
|
||||
export function isWatcherNoisePath(fsPath: string): boolean {
|
||||
const segments = fsPath.split(/[\\/]/);
|
||||
if (segments.some((seg) => seg.toLowerCase() === ".git")) return true;
|
||||
const base = segments[segments.length - 1] ?? "";
|
||||
const lower = base.toLowerCase();
|
||||
const noiseSuffixes = [".git", ".tmp", ".lock", "~", ".swp", ".bak", ".orig"];
|
||||
if (noiseSuffixes.some((suffix) => lower.endsWith(suffix))) return true;
|
||||
return lower.startsWith(".#") || lower.startsWith(".~");
|
||||
}
|
||||
|
||||
/**
|
||||
* True for files whose *content* participates in the index: XML documents
|
||||
* and art-asset XML (.w3x). Reasonable text formats in RA3 mods are `.xml`,
|
||||
* `.w3x` and `.lua`; lua is not indexed yet, and compiled manifests are
|
||||
* binary `*.manifest` (there is no `.manifestxml` source format). Files
|
||||
* already in the index with other extensions (e.g. sniffed XML) are handled
|
||||
* separately via `ModIndexer.isIndexedFile`. Content changes to binary art
|
||||
* (e.g. textures, `.w3d`) cannot change index records, so they do not need
|
||||
* to trigger a rebuild.
|
||||
*/
|
||||
export function isContentRelevantPath(fsPath: string): boolean {
|
||||
const ext = extname(fsPath).toLowerCase();
|
||||
return ext === ".xml" || ext === ".w3x";
|
||||
}
|
||||
|
||||
/**
|
||||
* Builds the candidate list for Include/@source completion from a set of
|
||||
* search directories. For DATA directories only *.xml files are listed; for
|
||||
|
||||
@@ -8,6 +8,7 @@
|
||||
|
||||
import { join, resolve, normalize, isAbsolute } from "node:path";
|
||||
import { statSync } from "node:fs";
|
||||
import type { ExistenceSnapshot } from "./existence";
|
||||
|
||||
export type IncludeKind = "all" | "instance" | "reference";
|
||||
export type SourcePrefix = "DATA" | "ART" | "AUDIO" | null;
|
||||
@@ -99,41 +100,58 @@ export function resolveSource(
|
||||
source: string,
|
||||
currentDir: string | null,
|
||||
searchPaths: SearchPaths,
|
||||
existence?: ExistenceSnapshot,
|
||||
): ResolveResult {
|
||||
const raw = source.trim().replace(/\\/g, "/");
|
||||
const { prefix, rest } = splitPrefix(raw);
|
||||
|
||||
if (prefix) {
|
||||
const bases = searchPaths[prefix] ?? [];
|
||||
const direct = findInBases(rest, bases);
|
||||
const direct = findInBases(rest, bases, existence);
|
||||
if (direct) return { path: direct, prefix, raw };
|
||||
if (prefix === "ART" && !rest.includes("/")) {
|
||||
const two = rest.slice(0, 2).toLowerCase();
|
||||
const prefixed = findInBases(`${two}/${rest}`, bases);
|
||||
const prefixed = findInBases(`${two}/${rest}`, bases, existence);
|
||||
if (prefixed) return { path: prefixed, prefix, raw };
|
||||
}
|
||||
return { path: null, prefix, raw };
|
||||
}
|
||||
|
||||
if (currentDir && isAbsolute(rest)) {
|
||||
return { path: fileExists(rest) ? rest : null, prefix: null, raw };
|
||||
return {
|
||||
path: fileExists(rest, existence) ? rest : null,
|
||||
prefix: null,
|
||||
raw,
|
||||
};
|
||||
}
|
||||
if (currentDir) {
|
||||
const candidate = resolve(currentDir, rest);
|
||||
return { path: fileExists(candidate) ? candidate : null, prefix: null, raw };
|
||||
return {
|
||||
path: fileExists(candidate, existence) ? candidate : null,
|
||||
prefix: null,
|
||||
raw,
|
||||
};
|
||||
}
|
||||
return { path: null, prefix: null, raw };
|
||||
}
|
||||
|
||||
function findInBases(relPath: string, bases: string[]): string | null {
|
||||
function findInBases(
|
||||
relPath: string,
|
||||
bases: string[],
|
||||
existence?: ExistenceSnapshot,
|
||||
): string | null {
|
||||
for (const base of bases) {
|
||||
const candidate = normalize(resolve(base, relPath));
|
||||
if (fileExists(candidate)) return candidate;
|
||||
if (fileExists(candidate, existence)) return candidate;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function fileExists(path: string): boolean {
|
||||
function fileExists(path: string, existence?: ExistenceSnapshot): boolean {
|
||||
if (existence) {
|
||||
const known = existence.has(path);
|
||||
if (known !== null) return known;
|
||||
}
|
||||
try {
|
||||
return statSync(path).isFile();
|
||||
} catch {
|
||||
|
||||
+203
-52
@@ -18,7 +18,6 @@ import {
|
||||
LineMap,
|
||||
parseXml,
|
||||
stripBom,
|
||||
type XmlDocument,
|
||||
type XmlElement,
|
||||
} from "../language/xmlParser";
|
||||
import {
|
||||
@@ -28,6 +27,11 @@ import {
|
||||
type ResolveResult,
|
||||
type SearchPaths,
|
||||
} from "./includeResolver";
|
||||
import {
|
||||
buildExistenceSnapshot,
|
||||
type ExistenceSnapshot,
|
||||
} from "./existence";
|
||||
import { findXPointerContainer, localName } from "./xpointer";
|
||||
import {
|
||||
deriveAssetId,
|
||||
deriveAssetType,
|
||||
@@ -59,10 +63,11 @@ const MAX_DEPTH = 300;
|
||||
/** Files above this size are never parsed (safety against binary blobs). */
|
||||
const MAX_PARSE_BYTES = 4 * 1024 * 1024;
|
||||
/**
|
||||
* Fully parsed XML documents. `.xml` / `.manifestxml` files are small enough
|
||||
* that a full DOM is affordable.
|
||||
* Fully parsed XML documents. `.xml` files are small enough that a full DOM
|
||||
* is affordable. (Compiled manifests are binary `*.manifest` files parsed by
|
||||
* `manifestParser`; there is no `.manifestxml` source format.)
|
||||
*/
|
||||
const FULL_XML_EXTENSIONS = new Set([".xml", ".manifestxml"]);
|
||||
const FULL_XML_EXTENSIONS = new Set([".xml"]);
|
||||
/**
|
||||
* XML documents whose top-level structure is all the index needs (art-asset
|
||||
* files exported by modeling tools, e.g. .w3x). They are shallow-scanned so
|
||||
@@ -79,6 +84,8 @@ export class ModIndexer {
|
||||
private docs: DocumentCache;
|
||||
private recordsCache: IndexRecordsCache;
|
||||
private resolveCache: IncludeResolveCache;
|
||||
/** Directory-based file existence snapshot (avoids cold statSync storms). */
|
||||
private existence: ExistenceSnapshot | null = null;
|
||||
private scanCounters = {
|
||||
shallowScannedFiles: 0,
|
||||
shallowCacheHits: 0,
|
||||
@@ -86,7 +93,19 @@ export class ModIndexer {
|
||||
resolveCacheHits: 0,
|
||||
resolveCalls: 0,
|
||||
};
|
||||
private phase = { candidatesMs: 0, walkMs: 0 };
|
||||
private timings = { candidatesMs: 0, walkMs: 0, artScanMs: 0 };
|
||||
/**
|
||||
* Phase A ("xml") registers art-asset XML files (`.w3x` and sniffed XML)
|
||||
* without reading their content; the queue is drained by phase B ("art"),
|
||||
* which shallow-scans them and walks any includes they contain.
|
||||
*/
|
||||
private deferArtScan = false;
|
||||
private artQueue: {
|
||||
path: string;
|
||||
stream: StreamInfo;
|
||||
depth: number;
|
||||
viaInstance: boolean;
|
||||
}[] = [];
|
||||
private assets = new Map<string, Map<string, AssetDef[]>>();
|
||||
private assetsById = new Map<string, AssetDef[]>();
|
||||
private defines = new Map<string, DefineDef[]>();
|
||||
@@ -111,8 +130,7 @@ export class ModIndexer {
|
||||
|
||||
/**
|
||||
* Returns a document for indexing/navigation:
|
||||
* - `.xml` / `.manifestxml` files are fully parsed (bounded by
|
||||
* MAX_PARSE_BYTES);
|
||||
* - `.xml` files are fully parsed (bounded by MAX_PARSE_BYTES);
|
||||
* - `.w3x` (and unknown-extension files whose content looks like XML) are
|
||||
* shallow-scanned, so huge model files never become a DOM;
|
||||
* - everything else is registered as a file but never parsed.
|
||||
@@ -120,7 +138,10 @@ export class ModIndexer {
|
||||
* Cached entries are reused when the file stat is unchanged, which lets a
|
||||
* workspace-owned cache survive rebuilds.
|
||||
*/
|
||||
async readDocument(path: string): Promise<ParsedFile | null> {
|
||||
async readDocument(
|
||||
path: string,
|
||||
opts?: { deferArt?: boolean },
|
||||
): Promise<ParsedFile | null> {
|
||||
const key = normKey(path);
|
||||
const trust =
|
||||
this.opts.trustUnchanged === true && !this.opts.changedFiles?.has(key);
|
||||
@@ -143,29 +164,71 @@ export class ModIndexer {
|
||||
try {
|
||||
const st = await stat(path);
|
||||
const rec = this.recordsCache.get(key);
|
||||
if (rec?.stat && rec.stat.mtimeMs === st.mtimeMs && rec.stat.size === st.size) {
|
||||
if (
|
||||
rec?.stat &&
|
||||
rec.stat.mtimeMs === st.mtimeMs &&
|
||||
rec.stat.size === st.size &&
|
||||
rec.stat.birthtimeMs === st.birthtimeMs &&
|
||||
rec.stat.ctimeMs === st.ctimeMs
|
||||
) {
|
||||
return this.recordsParsed(path, rec);
|
||||
}
|
||||
const hit = this.docs.get(key);
|
||||
if (
|
||||
hit?.file.stat &&
|
||||
hit.file.stat.mtimeMs === st.mtimeMs &&
|
||||
hit.file.stat.size === st.size
|
||||
hit.file.stat.size === st.size &&
|
||||
hit.file.stat.birthtimeMs === st.birthtimeMs &&
|
||||
hit.file.stat.ctimeMs === st.ctimeMs
|
||||
) {
|
||||
this.files.set(key, hit.file);
|
||||
return hit;
|
||||
}
|
||||
const mode = await this.detectXmlMode(path);
|
||||
if (mode === "shallow") return this.scanShallow(path, st);
|
||||
if (mode === "shallow") {
|
||||
// Phase A: register the art file, defer the shallow scan to phase B.
|
||||
if (opts?.deferArt) {
|
||||
const file: IndexedFile = {
|
||||
path: resolve(path),
|
||||
stat: {
|
||||
mtimeMs: st.mtimeMs,
|
||||
size: st.size,
|
||||
birthtimeMs: st.birthtimeMs,
|
||||
ctimeMs: st.ctimeMs,
|
||||
},
|
||||
};
|
||||
// Deliberately NOT stored in the DocumentCache: a deferred entry
|
||||
// has no records, and a later phase-B read must re-scan it.
|
||||
this.files.set(key, file);
|
||||
return { file, parse: null, records: null, lineMap: null, deferredArt: true };
|
||||
}
|
||||
return this.scanShallow(path, st);
|
||||
}
|
||||
if (mode === "binary") {
|
||||
const file: IndexedFile = { path: resolve(path), stat: { mtimeMs: st.mtimeMs, size: st.size } };
|
||||
const file: IndexedFile = {
|
||||
path: resolve(path),
|
||||
stat: {
|
||||
mtimeMs: st.mtimeMs,
|
||||
size: st.size,
|
||||
birthtimeMs: st.birthtimeMs,
|
||||
ctimeMs: st.ctimeMs,
|
||||
},
|
||||
};
|
||||
const parsed: ParsedFile = { file, parse: null, records: null, lineMap: null };
|
||||
this.docs.set(parsed);
|
||||
this.files.set(key, file);
|
||||
return parsed;
|
||||
}
|
||||
if (st.size > MAX_PARSE_BYTES) {
|
||||
const file: IndexedFile = { path: resolve(path), stat: { mtimeMs: st.mtimeMs, size: st.size } };
|
||||
const file: IndexedFile = {
|
||||
path: resolve(path),
|
||||
stat: {
|
||||
mtimeMs: st.mtimeMs,
|
||||
size: st.size,
|
||||
birthtimeMs: st.birthtimeMs,
|
||||
ctimeMs: st.ctimeMs,
|
||||
},
|
||||
};
|
||||
const parsed: ParsedFile = { file, parse: null, records: null, lineMap: null };
|
||||
this.docs.set(parsed);
|
||||
this.files.set(key, file);
|
||||
@@ -176,7 +239,15 @@ export class ModIndexer {
|
||||
const parse = parseXml(text);
|
||||
const records = extractIndexRecords(parse, lineMap);
|
||||
const parsed: ParsedFile = {
|
||||
file: { path: resolve(path), stat: { mtimeMs: st.mtimeMs, size: st.size } },
|
||||
file: {
|
||||
path: resolve(path),
|
||||
stat: {
|
||||
mtimeMs: st.mtimeMs,
|
||||
size: st.size,
|
||||
birthtimeMs: st.birthtimeMs,
|
||||
ctimeMs: st.ctimeMs,
|
||||
},
|
||||
},
|
||||
parse,
|
||||
records,
|
||||
lineMap,
|
||||
@@ -211,7 +282,15 @@ export class ModIndexer {
|
||||
const lineMap = new LineMap(text);
|
||||
const records = recordsFromShallow(scanXmlShallow(text), lineMap);
|
||||
const parsed: ParsedFile = {
|
||||
file: { path: resolve(path), stat: { mtimeMs: st.mtimeMs, size: st.size } },
|
||||
file: {
|
||||
path: resolve(path),
|
||||
stat: {
|
||||
mtimeMs: st.mtimeMs,
|
||||
size: st.size,
|
||||
birthtimeMs: st.birthtimeMs,
|
||||
ctimeMs: st.ctimeMs,
|
||||
},
|
||||
},
|
||||
parse: null,
|
||||
records,
|
||||
lineMap: null,
|
||||
@@ -235,11 +314,11 @@ export class ModIndexer {
|
||||
}
|
||||
|
||||
/**
|
||||
* Reads a document and guarantees a DOM parse tree. Used only for
|
||||
* root-level <xi:include> xpointer selection (rare), where the target's
|
||||
* container children are needed.
|
||||
* Reads a document and guarantees a DOM parse tree. Used for root-level
|
||||
* <xi:include> xpointer selection (rare) and by the document-local scope
|
||||
* (logical include expansion + precise definition locations).
|
||||
*/
|
||||
private async readDom(path: string): Promise<ParsedFile | null> {
|
||||
async readDom(path: string): Promise<ParsedFile | null> {
|
||||
const key = normKey(path);
|
||||
const cached = this.docs.get(key);
|
||||
if (cached?.parse?.root) {
|
||||
@@ -253,7 +332,9 @@ export class ModIndexer {
|
||||
hit?.parse?.root &&
|
||||
hit.file.stat &&
|
||||
hit.file.stat.mtimeMs === st.mtimeMs &&
|
||||
hit.file.stat.size === st.size
|
||||
hit.file.stat.size === st.size &&
|
||||
hit.file.stat.birthtimeMs === st.birthtimeMs &&
|
||||
hit.file.stat.ctimeMs === st.ctimeMs
|
||||
) {
|
||||
this.files.set(key, hit.file);
|
||||
return hit;
|
||||
@@ -264,7 +345,15 @@ export class ModIndexer {
|
||||
const parse = parseXml(text);
|
||||
const records = extractIndexRecords(parse, lineMap);
|
||||
const parsed: ParsedFile = {
|
||||
file: { path: resolve(path), stat: { mtimeMs: st.mtimeMs, size: st.size } },
|
||||
file: {
|
||||
path: resolve(path),
|
||||
stat: {
|
||||
mtimeMs: st.mtimeMs,
|
||||
size: st.size,
|
||||
birthtimeMs: st.birthtimeMs,
|
||||
ctimeMs: st.ctimeMs,
|
||||
},
|
||||
},
|
||||
parse,
|
||||
records,
|
||||
lineMap,
|
||||
@@ -299,7 +388,12 @@ export class ModIndexer {
|
||||
return hit;
|
||||
}
|
||||
this.scanCounters.resolveCalls++;
|
||||
const result = resolveSource(source, currentDir, this.searchPaths);
|
||||
const result = resolveSource(
|
||||
source,
|
||||
currentDir,
|
||||
this.searchPaths,
|
||||
this.existence ?? undefined,
|
||||
);
|
||||
this.resolveCache.set(key, result);
|
||||
return result;
|
||||
}
|
||||
@@ -323,14 +417,26 @@ export class ModIndexer {
|
||||
return this.docs.get(path);
|
||||
}
|
||||
|
||||
async build(): Promise<ModIndex> {
|
||||
/** True when the file is part of the current build's index. */
|
||||
isIndexedFile(path: string): boolean {
|
||||
return this.files.has(normKey(path));
|
||||
}
|
||||
|
||||
async build(onPhase?: (index: ModIndex) => void | Promise<void>): Promise<ModIndex> {
|
||||
const start = Date.now();
|
||||
// Root list only; directories are listed lazily on first query, so the
|
||||
// XML phase does not pay an upfront recursive enumeration of the SDK.
|
||||
this.existence = buildExistenceSnapshot(this.searchPaths);
|
||||
const projectData = await findCaseInsensitiveDir(join(this.opts.projectDir, "Data"));
|
||||
const additionalMaps = projectData
|
||||
? await findCaseInsensitiveDir(join(projectData, "additionalmaps"))
|
||||
: null;
|
||||
|
||||
// ── Streams ──
|
||||
// Phase A: walk the include graph without reading art-asset content.
|
||||
// `.w3x` (and sniffed XML) files are registered and queued; their
|
||||
// top-level assets and nested includes are processed in phase B.
|
||||
this.deferArtScan = true;
|
||||
const walkStart = Date.now();
|
||||
const staticEntry = projectData ? join(projectData, "Mod.xml") : null;
|
||||
if (staticEntry) {
|
||||
@@ -360,7 +466,7 @@ export class ModIndexer {
|
||||
await this.walk(entry, "all", stream, 0);
|
||||
}
|
||||
}
|
||||
this.phase.walkMs = Date.now() - walkStart;
|
||||
this.timings.walkMs = Date.now() - walkStart;
|
||||
|
||||
// ── Source completion candidates ──
|
||||
const candidatesStart = Date.now();
|
||||
@@ -408,45 +514,94 @@ export class ModIndexer {
|
||||
...sdkRootCandidates,
|
||||
...this.sourceCandidates,
|
||||
]);
|
||||
this.phase.candidatesMs = Date.now() - candidatesStart;
|
||||
this.timings.candidatesMs = Date.now() - candidatesStart;
|
||||
|
||||
// Publish the XML phase as an immutable snapshot: features get usable
|
||||
// completions/navigation/diagnostics for XML + manifest data immediately,
|
||||
// while the slow art scan continues in the background.
|
||||
const xmlPhase = this.snapshotIndex("xml", false, start);
|
||||
if (onPhase) await onPhase(xmlPhase);
|
||||
|
||||
// ── Phase B: art assets ──
|
||||
this.deferArtScan = false;
|
||||
const artStart = Date.now();
|
||||
while (this.artQueue.length) {
|
||||
const entry = this.artQueue.shift()!;
|
||||
const parsed = await this.readDocument(entry.path);
|
||||
if (parsed?.records) {
|
||||
await this.applyRecords(parsed, entry.stream, entry.depth, entry.viaInstance);
|
||||
}
|
||||
}
|
||||
this.timings.artScanMs = Date.now() - artStart;
|
||||
|
||||
return this.snapshotIndex("art", true, start);
|
||||
}
|
||||
|
||||
/**
|
||||
* Produces a copy of the current index state. Phase-A snapshots must be
|
||||
* immutable: phase B keeps mutating the live maps after the snapshot has
|
||||
* been handed to features, so every nested map/array is cloned here.
|
||||
*/
|
||||
private snapshotIndex(
|
||||
phase: "xml" | "art",
|
||||
complete: boolean,
|
||||
startedAt: number,
|
||||
): ModIndex {
|
||||
const manifestAssetCount = [...this.manifests.values()].reduce(
|
||||
(sum, m) => sum + m.assets.length,
|
||||
0,
|
||||
);
|
||||
const assets = new Map<string, Map<string, AssetDef[]>>();
|
||||
for (const [type, byId] of this.assets) {
|
||||
const copied = new Map<string, AssetDef[]>();
|
||||
for (const [id, defs] of byId) copied.set(id, defs.slice());
|
||||
assets.set(type, copied);
|
||||
}
|
||||
const assetsById = new Map<string, AssetDef[]>();
|
||||
for (const [id, defs] of this.assetsById) assetsById.set(id, defs.slice());
|
||||
const defines = new Map<string, DefineDef[]>();
|
||||
for (const [name, defs] of this.defines) defines.set(name, defs.slice());
|
||||
|
||||
return {
|
||||
projectDir: resolve(this.opts.projectDir),
|
||||
sdkDir: resolve(this.opts.sdkDir),
|
||||
assets: this.assets,
|
||||
assetsById: this.assetsById,
|
||||
defines: this.defines,
|
||||
files: this.files,
|
||||
streams: this.streams,
|
||||
manifests: this.manifests,
|
||||
sourceCandidates: this.sourceCandidates,
|
||||
diagnostics: this.diagnostics,
|
||||
complete,
|
||||
phase,
|
||||
assets,
|
||||
assetsById,
|
||||
defines,
|
||||
files: new Map(this.files),
|
||||
streams: this.streams.map((s) => ({ ...s, files: new Set(s.files) })),
|
||||
manifests: new Map(this.manifests),
|
||||
sourceCandidates: this.sourceCandidates.slice(),
|
||||
diagnostics: this.diagnostics.slice(),
|
||||
stats: {
|
||||
projectDir: resolve(this.opts.projectDir),
|
||||
sdkDir: resolve(this.opts.sdkDir),
|
||||
phase,
|
||||
complete,
|
||||
indexedFiles: this.files.size,
|
||||
parsedFiles: [...this.files.values()].filter(
|
||||
(f) => f.stat != null && f.stat.size <= MAX_PARSE_BYTES,
|
||||
).length,
|
||||
shallowScannedFiles: this.scanCounters.shallowScannedFiles,
|
||||
deferredArtFiles: this.artQueue.length,
|
||||
shallowCacheHits: this.scanCounters.shallowCacheHits,
|
||||
recordsCacheHits: this.scanCounters.recordsCacheHits,
|
||||
resolveCacheHits: this.scanCounters.resolveCacheHits,
|
||||
resolveCalls: this.scanCounters.resolveCalls,
|
||||
candidatesMs: this.phase.candidatesMs,
|
||||
walkMs: this.phase.walkMs,
|
||||
snapshotHits: this.existence?.hits ?? 0,
|
||||
snapshotFallbacks: this.existence?.fallbacks ?? 0,
|
||||
candidatesMs: this.timings.candidatesMs,
|
||||
walkMs: this.timings.walkMs,
|
||||
artScanMs: this.timings.artScanMs,
|
||||
assetCount: [...this.assets.values()].reduce((sum, byId) => sum + byId.size, 0),
|
||||
defineCount: this.defines.size,
|
||||
manifestFiles: this.manifests.size,
|
||||
manifestAssetCount,
|
||||
streams: this.streams.length,
|
||||
sourceCandidates: this.sourceCandidates.length,
|
||||
elapsedMs: Date.now() - start,
|
||||
elapsedMs: Date.now() - startedAt,
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -483,9 +638,19 @@ export class ModIndexer {
|
||||
|
||||
// readDocument returns compact index records for every indexable XML
|
||||
// document (full parse or shallow scan), or a bare file registration
|
||||
// for binary / unparseable targets.
|
||||
const parsed = await this.readDocument(path);
|
||||
// for binary / unparseable targets. During phase A, art-asset XML files
|
||||
// are registered and queued instead of scanned (deferredArt).
|
||||
const parsed = await this.readDocument(path, this.deferArtScan ? { deferArt: true } : undefined);
|
||||
if (!parsed) return;
|
||||
if (parsed.deferredArt) {
|
||||
this.artQueue.push({
|
||||
path: parsed.file.path,
|
||||
stream,
|
||||
depth,
|
||||
viaInstance: mode === "instance",
|
||||
});
|
||||
return;
|
||||
}
|
||||
if (parsed.records) {
|
||||
await this.applyRecords(parsed, stream, depth, mode === "instance");
|
||||
return;
|
||||
@@ -722,11 +887,6 @@ export class ModIndexer {
|
||||
|
||||
// ── Module-level helpers ─────────────────────────────────────────────
|
||||
|
||||
function localName(tag: string): string {
|
||||
const idx = tag.lastIndexOf(":");
|
||||
return idx >= 0 ? tag.slice(idx + 1) : tag;
|
||||
}
|
||||
|
||||
function lineOf(parsed: ParsedFile, offset: number): number {
|
||||
if (!parsed.lineMap) return 0;
|
||||
return parsed.lineMap.positionAt(offset).line + 1;
|
||||
@@ -774,15 +934,6 @@ async function findCaseInsensitiveDir(dir: string): Promise<string | null> {
|
||||
}
|
||||
}
|
||||
|
||||
function findXPointerContainer(doc: XmlDocument, xpointer: string): XmlElement | null {
|
||||
// Supports the form used by the mods:
|
||||
// xmlns(n=uri:ea.com:eala:asset) xpointer(/n:ElementName/child::*)
|
||||
const m = /xpointer\(\/\w+:(\w+)\/child::\*\)/.exec(xpointer);
|
||||
if (!m) return null;
|
||||
const name = m[1];
|
||||
return doc.elements.find((el) => localName(el.name) === name) ?? null;
|
||||
}
|
||||
|
||||
/** Keeps the first candidate for each case-insensitive source string. */
|
||||
function dedupeSourceCandidates(candidates: SourceCandidate[]): SourceCandidate[] {
|
||||
const seen = new Set<string>();
|
||||
|
||||
@@ -0,0 +1,265 @@
|
||||
import { dirname, resolve } from "node:path";
|
||||
import { LineMap, parseXml, type XmlDocument } from "../language/xmlParser";
|
||||
import { extractIndexRecords } from "./records";
|
||||
import { resolveSource, type SearchPaths } from "./includeResolver";
|
||||
import { expandDocument, type LogicalDocument } from "./logicalTree";
|
||||
import type {
|
||||
AssetDef,
|
||||
DefineDef,
|
||||
LocalOverlay,
|
||||
ModIndex,
|
||||
ParsedFile,
|
||||
} from "./types";
|
||||
|
||||
export interface LocalScopeContext {
|
||||
projectDir: string;
|
||||
sdkDir: string;
|
||||
searchPaths: SearchPaths;
|
||||
/** Reads a file's compact index records (full parse or shallow scan). */
|
||||
readRecords(path: string): Promise<ParsedFile | null>;
|
||||
/** Reads a file and guarantees a DOM parse tree. */
|
||||
readDom(path: string): Promise<ParsedFile | null>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Everything the features need for the currently open document: the original
|
||||
* parse, the expanded logical tree, per-source line maps, the local overlay
|
||||
* and the overlay-aware index for lookups.
|
||||
*/
|
||||
export interface DocumentScope {
|
||||
uri: string;
|
||||
version: number;
|
||||
parse: XmlDocument;
|
||||
lineMap: LineMap;
|
||||
expanded: LogicalDocument;
|
||||
/** scopePathKey(file) -> line map (current file + expanded include targets). */
|
||||
lineMaps: Map<string, LineMap>;
|
||||
/** Local assets/defines from the document itself and its include chain. */
|
||||
overlay: LocalOverlay;
|
||||
/** Global index with `local` attached (or a minimal standalone index). */
|
||||
merged: ModIndex | null;
|
||||
}
|
||||
|
||||
/** Normalized key used for source-file identity / line-map lookup. */
|
||||
export function scopePathKey(path: string): string {
|
||||
return path.replace(/\\/g, "/").toLowerCase();
|
||||
}
|
||||
|
||||
/**
|
||||
* Builds the document scope for the current (possibly unsaved) text:
|
||||
* - a local overlay of assets/defines reachable from this file;
|
||||
* - a logical tree with supported xi:include targets spliced in place.
|
||||
*/
|
||||
export async function buildDocumentScope(
|
||||
uri: string,
|
||||
text: string,
|
||||
version: number,
|
||||
ctx: LocalScopeContext,
|
||||
): Promise<DocumentScope> {
|
||||
const lineMap = new LineMap(text);
|
||||
const parse = parseXml(text);
|
||||
const builder = new OverlayBuilder(ctx);
|
||||
await builder.addEntry(uri, parse, lineMap);
|
||||
|
||||
const expanded = await expandDocument(uri, parse, {
|
||||
resolve: (source, currentDir) =>
|
||||
resolveSource(source, currentDir, ctx.searchPaths).path,
|
||||
readDom: async (path) => {
|
||||
const parsed = await ctx.readDom(path);
|
||||
return parsed?.parse && parsed.lineMap
|
||||
? { parse: parsed.parse, lineMap: parsed.lineMap }
|
||||
: null;
|
||||
},
|
||||
});
|
||||
|
||||
const lineMaps = new Map<string, LineMap>();
|
||||
lineMaps.set(scopePathKey(uri), lineMap);
|
||||
for (const [path, lm] of builder.lineMaps) {
|
||||
if (!lineMaps.has(path)) lineMaps.set(path, lm);
|
||||
}
|
||||
|
||||
return {
|
||||
uri,
|
||||
version,
|
||||
parse,
|
||||
lineMap,
|
||||
expanded,
|
||||
lineMaps,
|
||||
overlay: builder.overlay,
|
||||
merged: null,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Attaches a document-local overlay to a global index without copying the
|
||||
* global maps. When there is no global index yet, returns a minimal standalone
|
||||
* index so the local chain alone can serve completions / references.
|
||||
*/
|
||||
export function withLocalOverlay(
|
||||
global: ModIndex | null,
|
||||
overlay: LocalOverlay,
|
||||
projectDir: string,
|
||||
sdkDir: string,
|
||||
): ModIndex {
|
||||
if (!global) {
|
||||
return {
|
||||
projectDir,
|
||||
sdkDir,
|
||||
complete: false,
|
||||
phase: "xml",
|
||||
assets: new Map(),
|
||||
assetsById: new Map(),
|
||||
defines: new Map(),
|
||||
files: new Map(),
|
||||
streams: [],
|
||||
manifests: new Map(),
|
||||
sourceCandidates: [],
|
||||
diagnostics: [],
|
||||
stats: {
|
||||
projectDir,
|
||||
sdkDir,
|
||||
phase: "xml",
|
||||
complete: false,
|
||||
indexedFiles: 0,
|
||||
parsedFiles: 0,
|
||||
shallowScannedFiles: 0,
|
||||
deferredArtFiles: 0,
|
||||
shallowCacheHits: 0,
|
||||
recordsCacheHits: 0,
|
||||
resolveCacheHits: 0,
|
||||
resolveCalls: 0,
|
||||
snapshotHits: 0,
|
||||
snapshotFallbacks: 0,
|
||||
candidatesMs: 0,
|
||||
walkMs: 0,
|
||||
artScanMs: 0,
|
||||
assetCount: 0,
|
||||
defineCount: 0,
|
||||
manifestFiles: 0,
|
||||
manifestAssetCount: 0,
|
||||
streams: 0,
|
||||
sourceCandidates: 0,
|
||||
elapsedMs: 0,
|
||||
},
|
||||
local: overlay,
|
||||
};
|
||||
}
|
||||
return { ...global, local: overlay };
|
||||
}
|
||||
|
||||
const MAX_LOCAL_DEPTH = 64;
|
||||
|
||||
class OverlayBuilder {
|
||||
readonly overlay: LocalOverlay = {
|
||||
assets: new Map(),
|
||||
assetsById: new Map(),
|
||||
defines: new Map(),
|
||||
};
|
||||
readonly lineMaps = new Map<string, LineMap>();
|
||||
private visited = new Set<string>();
|
||||
|
||||
constructor(private ctx: LocalScopeContext) {}
|
||||
|
||||
async addEntry(
|
||||
path: string,
|
||||
parse: XmlDocument,
|
||||
lineMap: LineMap,
|
||||
): Promise<void> {
|
||||
this.lineMaps.set(scopePathKey(path), lineMap);
|
||||
await this.addParsed({
|
||||
file: { path: resolve(path), stat: null },
|
||||
parse,
|
||||
records: extractIndexRecords(parse, lineMap),
|
||||
lineMap,
|
||||
}, 0);
|
||||
}
|
||||
|
||||
async addFile(path: string, depth: number): Promise<void> {
|
||||
if (depth > MAX_LOCAL_DEPTH) return;
|
||||
const parsed = await this.ctx.readRecords(path);
|
||||
if (parsed) await this.addParsed(parsed, depth);
|
||||
}
|
||||
|
||||
private async addParsed(parsed: ParsedFile, depth: number): Promise<void> {
|
||||
const path = parsed.file.path;
|
||||
const key = scopePathKey(path);
|
||||
if (!parsed.records || this.visited.has(key)) return;
|
||||
this.visited.add(key);
|
||||
if (parsed.lineMap) this.lineMaps.set(key, parsed.lineMap);
|
||||
|
||||
const origin = this.originOf(path);
|
||||
for (const asset of parsed.records.assets) {
|
||||
this.addAsset({
|
||||
type: asset.type,
|
||||
id: asset.id,
|
||||
file: path,
|
||||
line: asset.line,
|
||||
origin,
|
||||
stream: "local",
|
||||
});
|
||||
}
|
||||
for (const define of parsed.records.defines) {
|
||||
const entry: DefineDef = {
|
||||
name: define.name,
|
||||
value: define.value,
|
||||
file: path,
|
||||
line: define.line,
|
||||
origin,
|
||||
};
|
||||
const arr = this.overlay.defines.get(define.name.toLowerCase());
|
||||
if (arr) arr.push(entry);
|
||||
else this.overlay.defines.set(define.name.toLowerCase(), [entry]);
|
||||
}
|
||||
|
||||
for (const inc of parsed.records.includes) {
|
||||
const resolved = resolveSource(inc.source, dirname(path), this.ctx.searchPaths);
|
||||
if (!resolved.path) continue;
|
||||
if (inc.type === "all" || inc.type === "instance") {
|
||||
await this.addFile(resolved.path, depth + 1);
|
||||
}
|
||||
// type="reference" points at compiled manifests; their assets are
|
||||
// provided by the global index, so the local text overlay skips them.
|
||||
}
|
||||
|
||||
for (const xi of [
|
||||
...parsed.records.nestedXiIncludes,
|
||||
...parsed.records.rootXiIncludes,
|
||||
]) {
|
||||
const resolved = resolveSource(xi.href, dirname(path), this.ctx.searchPaths);
|
||||
if (resolved.path) await this.addFile(resolved.path, depth + 1);
|
||||
}
|
||||
}
|
||||
|
||||
private addAsset(def: AssetDef): void {
|
||||
const typeKey = def.type;
|
||||
const idKey = def.id.toLowerCase();
|
||||
let byId = this.overlay.assets.get(typeKey);
|
||||
if (!byId) {
|
||||
byId = new Map();
|
||||
this.overlay.assets.set(typeKey, byId);
|
||||
}
|
||||
const arr = byId.get(idKey);
|
||||
if (arr) {
|
||||
if (arr.some((a) => a.file === def.file && a.line === def.line)) return;
|
||||
arr.push(def);
|
||||
} else {
|
||||
byId.set(idKey, [def]);
|
||||
}
|
||||
const all = this.overlay.assetsById.get(idKey);
|
||||
if (all) {
|
||||
if (all.some((a) => a.file === def.file && a.line === def.line)) return;
|
||||
all.push(def);
|
||||
} else {
|
||||
this.overlay.assetsById.set(idKey, [def]);
|
||||
}
|
||||
}
|
||||
|
||||
private originOf(path: string): "project" | "sdk" | "manifest" {
|
||||
const p = resolve(path).toLowerCase();
|
||||
const project = resolve(this.ctx.projectDir).toLowerCase();
|
||||
const sdk = resolve(this.ctx.sdkDir).toLowerCase();
|
||||
if (p.startsWith(project + "\\")) return "project";
|
||||
if (sdk && p.startsWith(sdk + "\\")) return "sdk";
|
||||
return "project";
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,249 @@
|
||||
import { dirname } from "node:path";
|
||||
import type {
|
||||
LineMap,
|
||||
XmlDocument,
|
||||
XmlElement,
|
||||
XmlParseError,
|
||||
} from "../language/xmlParser";
|
||||
import { resolveElementType } from "../language/typeContext";
|
||||
import { isAssignableTo } from "../model/schemaModel";
|
||||
import { findXPointerContainer, localName } from "./xpointer";
|
||||
|
||||
/**
|
||||
* A logical document is the parsed tree of the currently open file with
|
||||
* supported `xi:include` targets spliced in place. Every node keeps its
|
||||
* original source file and offsets so diagnostics / hover / navigation can
|
||||
* map back to the real file.
|
||||
*/
|
||||
export interface LogicalElement extends Omit<XmlElement, "parent" | "children"> {
|
||||
parent: LogicalElement | null;
|
||||
children: LogicalElement[];
|
||||
sourceFile: string;
|
||||
}
|
||||
|
||||
export interface LogicalDocument {
|
||||
root: LogicalElement | null;
|
||||
elements: LogicalElement[];
|
||||
/** Mirrors XmlDocument so existing helpers (findElementAt etc.) work. */
|
||||
errors: XmlParseError[];
|
||||
declarationEnd: number;
|
||||
}
|
||||
|
||||
export interface ExpandContext {
|
||||
/** Resolves an xi:include href (BAB search order). */
|
||||
resolve(source: string, currentDir: string): string | null;
|
||||
/** Reads a target and guarantees a DOM parse tree. */
|
||||
readDom(path: string): Promise<{ parse: XmlDocument; lineMap: LineMap } | null>;
|
||||
/** Include-depth guard. Defaults to 64. */
|
||||
maxDepth?: number;
|
||||
}
|
||||
|
||||
const DEFAULT_MAX_DEPTH = 64;
|
||||
|
||||
/**
|
||||
* Builds the logical document for `entryPath` by replacing supported
|
||||
* `xi:include` elements with their selected target children.
|
||||
*
|
||||
* The original xi:include node stays in `elements` (so hover keeps working)
|
||||
* but is removed from the logical child list; its selected content is spliced
|
||||
* in as siblings. Nodes are shallow-cloned shells with a rebuilt parent/child
|
||||
* chain, so cached parse trees in the indexer are never mutated.
|
||||
*/
|
||||
export async function expandDocument(
|
||||
entryPath: string,
|
||||
parse: XmlDocument,
|
||||
ctx: ExpandContext,
|
||||
): Promise<LogicalDocument> {
|
||||
const elements: LogicalElement[] = [];
|
||||
const map = new Map<XmlElement, LogicalElement>();
|
||||
// Pre-create a logical shell for every original element (including orphan
|
||||
// nodes produced by the parser's unterminated-quote recovery). Parent
|
||||
// pointers follow the original tree so type context is preserved.
|
||||
for (const orig of parse.elements) {
|
||||
const parent = orig.parent ? map.get(orig.parent) ?? null : null;
|
||||
const clone = cloneNode(orig, parent, entryPath);
|
||||
map.set(orig, clone);
|
||||
elements.push(clone);
|
||||
}
|
||||
// Cycle guard is a recursion stack, not a global visited set: the same
|
||||
// fragment may legitimately be included under several parents.
|
||||
const stack = new Set<string>();
|
||||
const root = parse.root ? map.get(parse.root) ?? null : null;
|
||||
|
||||
// Traverse from the real root plus any parser-recovery orphans (elements
|
||||
// whose parent is null but are not the root).
|
||||
const roots = new Set<XmlElement>();
|
||||
if (parse.root) roots.add(parse.root);
|
||||
for (const orig of parse.elements) {
|
||||
if (orig !== parse.root && orig.parent === null) roots.add(orig);
|
||||
}
|
||||
for (const origRoot of roots) {
|
||||
const logicalRoot = map.get(origRoot)!;
|
||||
await expandChildren(
|
||||
origRoot,
|
||||
logicalRoot,
|
||||
entryPath,
|
||||
0,
|
||||
ctx,
|
||||
elements,
|
||||
stack,
|
||||
map,
|
||||
);
|
||||
}
|
||||
return { root, elements, errors: parse.errors, declarationEnd: parse.declarationEnd };
|
||||
}
|
||||
|
||||
function cloneNode(
|
||||
el: XmlElement,
|
||||
parent: LogicalElement | null,
|
||||
sourceFile: string,
|
||||
): LogicalElement {
|
||||
return { ...el, parent, children: [], sourceFile };
|
||||
}
|
||||
|
||||
async function expandChildren(
|
||||
origParent: XmlElement,
|
||||
logicalParent: LogicalElement,
|
||||
file: string,
|
||||
depth: number,
|
||||
ctx: ExpandContext,
|
||||
elements: LogicalElement[],
|
||||
stack: Set<string>,
|
||||
map?: Map<XmlElement, LogicalElement>,
|
||||
): Promise<void> {
|
||||
for (const child of origParent.children) {
|
||||
await handleChild(child, logicalParent, file, depth, ctx, elements, stack, map);
|
||||
}
|
||||
}
|
||||
|
||||
async function handleChild(
|
||||
orig: XmlElement,
|
||||
logicalParent: LogicalElement,
|
||||
file: string,
|
||||
depth: number,
|
||||
ctx: ExpandContext,
|
||||
elements: LogicalElement[],
|
||||
stack: Set<string>,
|
||||
map?: Map<XmlElement, LogicalElement>,
|
||||
): Promise<void> {
|
||||
const isXi = orig.name.toLowerCase().startsWith("xi:") &&
|
||||
localName(orig.name).toLowerCase() === "include";
|
||||
if (isXi) {
|
||||
// Keep the xi:include itself discoverable (hover), but let its selected
|
||||
// content replace it in the logical child list.
|
||||
if (!map?.has(orig)) elements.push(cloneNode(orig, logicalParent, file));
|
||||
await expandXi(orig, logicalParent, file, depth, ctx, elements, stack);
|
||||
return;
|
||||
}
|
||||
|
||||
let clone = map?.get(orig);
|
||||
if (!clone) {
|
||||
clone = cloneNode(orig, logicalParent, file);
|
||||
elements.push(clone);
|
||||
}
|
||||
logicalParent.children.push(clone);
|
||||
await expandChildren(orig, clone, file, depth + 1, ctx, elements, stack, map);
|
||||
}
|
||||
|
||||
async function expandXi(
|
||||
xi: XmlElement,
|
||||
logicalParent: LogicalElement,
|
||||
parentFile: string,
|
||||
depth: number,
|
||||
ctx: ExpandContext,
|
||||
elements: LogicalElement[],
|
||||
stack: Set<string>,
|
||||
): Promise<void> {
|
||||
const href = xi.attrs.find((a) => a.name === "href")?.value;
|
||||
if (!href) return;
|
||||
const resolved = ctx.resolve(href, dirname(parentFile));
|
||||
if (!resolved) return;
|
||||
|
||||
const key = normPath(resolved);
|
||||
if (stack.has(key)) return; // include cycle
|
||||
if (depth > (ctx.maxDepth ?? DEFAULT_MAX_DEPTH)) return;
|
||||
stack.add(key);
|
||||
|
||||
try {
|
||||
const target = await ctx.readDom(resolved);
|
||||
if (!target?.parse?.root) return;
|
||||
|
||||
const xpointer = xi.attrs.find((a) => a.name === "xpointer")?.value ?? "";
|
||||
const selected = xpointer
|
||||
? findXPointerContainer(target.parse, xpointer)?.children ?? []
|
||||
: target.parse.root.children;
|
||||
|
||||
for (const sel of selected) {
|
||||
await handleChild(sel, logicalParent, resolved, depth + 1, ctx, elements, stack);
|
||||
}
|
||||
} finally {
|
||||
stack.delete(key);
|
||||
}
|
||||
}
|
||||
|
||||
/** True when a logical element's resolved XSD type is a GameObject. */
|
||||
export function isGameObjectElement(el: LogicalElement): boolean {
|
||||
const type = resolveElementType(el);
|
||||
return type != null && isAssignableTo(type, "GameObject");
|
||||
}
|
||||
|
||||
/** Nearest ancestor whose resolved type is a GameObject (or subclass). */
|
||||
export function findContainingGameObject(
|
||||
el: LogicalElement,
|
||||
): LogicalElement | null {
|
||||
let cur = el.parent;
|
||||
while (cur) {
|
||||
if (isGameObjectElement(cur)) return cur;
|
||||
cur = cur.parent;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
export interface LocalIdInfo {
|
||||
id: string;
|
||||
el: LogicalElement;
|
||||
}
|
||||
|
||||
/**
|
||||
* Collects every `id` defined inside a GameObject subtree (including modules
|
||||
* spliced in through xi:include). These are the candidates for Poid-typed
|
||||
* pipeline-local references such as AttachModuleId / ModuleId.
|
||||
*/
|
||||
export function collectLocalIds(root: LogicalElement): LocalIdInfo[] {
|
||||
const out: LocalIdInfo[] = [];
|
||||
const seen = new Set<string>();
|
||||
const stack: LogicalElement[] = [root];
|
||||
while (stack.length) {
|
||||
const el = stack.pop()!;
|
||||
const idAttr = el.attrs.find((a) => a.name === "id");
|
||||
if (idAttr?.hasValue) {
|
||||
const key = idAttr.value.toLowerCase();
|
||||
if (!seen.has(key)) {
|
||||
seen.add(key);
|
||||
out.push({ id: idAttr.value, el });
|
||||
}
|
||||
}
|
||||
for (const child of el.children) stack.push(child);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/** Finds an id inside a GameObject subtree (case-insensitive). */
|
||||
export function findLocalId(
|
||||
root: LogicalElement,
|
||||
id: string,
|
||||
): LogicalElement | null {
|
||||
const wanted = id.toLowerCase();
|
||||
const stack: LogicalElement[] = [root];
|
||||
while (stack.length) {
|
||||
const el = stack.pop()!;
|
||||
const idAttr = el.attrs.find((a) => a.name === "id");
|
||||
if (idAttr?.hasValue && idAttr.value.toLowerCase() === wanted) return el;
|
||||
for (const child of el.children) stack.push(child);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function normPath(path: string): string {
|
||||
return path.replace(/\\/g, "/").toLowerCase();
|
||||
}
|
||||
+27
-2
@@ -96,8 +96,11 @@ export function resolveReferenceTargetsForType(
|
||||
attrName: string,
|
||||
id: string,
|
||||
): ReferenceTarget[] {
|
||||
const defs = idx.assetsById.get(id.toLowerCase());
|
||||
if (!defs?.length) return [];
|
||||
const defs = mergeLocalAndGlobalDefs(
|
||||
idx.local?.assetsById.get(id.toLowerCase()),
|
||||
idx.assetsById.get(id.toLowerCase()),
|
||||
);
|
||||
if (!defs.length) return [];
|
||||
|
||||
const nameLower = attrName.toLowerCase();
|
||||
let refType: string | null = null;
|
||||
@@ -127,3 +130,25 @@ export function resolveReferenceTargetsForType(
|
||||
targets.sort((a, b) => a.score - b.score || a.def.id.localeCompare(b.def.id));
|
||||
return targets;
|
||||
}
|
||||
|
||||
/**
|
||||
* Merges document-local definitions with the global index, keeping local
|
||||
* entries first and de-duplicating definitions that exist in both.
|
||||
*/
|
||||
export function mergeLocalAndGlobalDefs(
|
||||
local: readonly AssetDef[] | undefined,
|
||||
global: readonly AssetDef[] | undefined,
|
||||
): AssetDef[] {
|
||||
const seen = new Set<string>();
|
||||
const out: AssetDef[] = [];
|
||||
for (const list of [local, global]) {
|
||||
if (!list) continue;
|
||||
for (const def of list) {
|
||||
const key = `${def.type}\u0000${def.id.toLowerCase()}\u0000${def.file}\u0000${def.line}`;
|
||||
if (seen.has(key)) continue;
|
||||
seen.add(key);
|
||||
out.push(def);
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
+64
-1
@@ -32,9 +32,36 @@ export interface DefineDef {
|
||||
origin: AssetOrigin;
|
||||
}
|
||||
|
||||
/**
|
||||
* Document-local overlay produced by `localScope.ts`. It contains assets /
|
||||
* defines reachable from the currently open document (its own text plus its
|
||||
* include chain), even when that file is not part of any global stream.
|
||||
*
|
||||
* The overlay is attached to a `ModIndex` as `local` rather than merged into
|
||||
* the global maps, so large indexes (Corona: ~65k assets) are never copied
|
||||
* on every keystroke. Lookup helpers consult `local` first.
|
||||
*/
|
||||
export interface LocalOverlay {
|
||||
/** type -> id -> definitions. */
|
||||
assets: Map<string, Map<string, AssetDef[]>>;
|
||||
/** id -> definitions across all types. */
|
||||
assetsById: Map<string, AssetDef[]>;
|
||||
/** `$NAME` -> definitions. */
|
||||
defines: Map<string, DefineDef[]>;
|
||||
}
|
||||
|
||||
export interface IndexedFile {
|
||||
path: string;
|
||||
stat: { mtimeMs: number; size: number } | null;
|
||||
/**
|
||||
* Multi-signal file stamp used to validate cached entries without
|
||||
* re-reading content: size, last-write time, creation time and change
|
||||
* time. Creation/change time catch tools that rewrite a file while
|
||||
* preserving its mtime (e.g. temp-file + rename exporters) — important on
|
||||
* removable drives where mtime resolution can be coarse (FAT32: 2 s).
|
||||
*/
|
||||
stat:
|
||||
| { mtimeMs: number; size: number; birthtimeMs: number; ctimeMs: number }
|
||||
| null;
|
||||
}
|
||||
|
||||
export interface StreamInfo {
|
||||
@@ -65,10 +92,16 @@ export interface IndexerDiagnostic {
|
||||
export interface IndexStats {
|
||||
projectDir: string;
|
||||
sdkDir: string;
|
||||
/** Last finished phase ("xml" or "art"). */
|
||||
phase: "xml" | "art";
|
||||
/** Whether the index is fully complete (art assets included). */
|
||||
complete: boolean;
|
||||
indexedFiles: number;
|
||||
parsedFiles: number;
|
||||
/** Art-asset documents indexed via shallow scan (no DOM tree). */
|
||||
shallowScannedFiles: number;
|
||||
/** Art files registered during the XML phase, scanned later in phase B. */
|
||||
deferredArtFiles: number;
|
||||
/** Shallow scans served from the persistent cache (unchanged files). */
|
||||
shallowCacheHits: number;
|
||||
/** Parsed XML files served from the persistent records cache. */
|
||||
@@ -77,10 +110,16 @@ export interface IndexStats {
|
||||
resolveCacheHits: number;
|
||||
/** Include/xi:include resolutions performed during this build. */
|
||||
resolveCalls: number;
|
||||
/** Existence checks answered by the directory snapshot (no statSync). */
|
||||
snapshotHits: number;
|
||||
/** Existence checks outside the snapshot that fell back to statSync. */
|
||||
snapshotFallbacks: number;
|
||||
/** Time spent enumerating Include source candidates (ms). */
|
||||
candidatesMs: number;
|
||||
/** Time spent walking the include graph (ms). */
|
||||
walkMs: number;
|
||||
/** Time spent shallow-scanning deferred art assets (ms). */
|
||||
artScanMs: number;
|
||||
assetCount: number;
|
||||
defineCount: number;
|
||||
manifestFiles: number;
|
||||
@@ -93,6 +132,19 @@ export interface IndexStats {
|
||||
export interface ModIndex {
|
||||
projectDir: string;
|
||||
sdkDir: string;
|
||||
/**
|
||||
* True when every indexing phase (including the art-asset shallow scan)
|
||||
* has finished. Features use this to decide whether unresolved references
|
||||
* are final errors or provisional "may be a false positive" diagnostics.
|
||||
*/
|
||||
complete: boolean;
|
||||
/** Last finished phase: "xml" (XML + manifests) or "art" (final). */
|
||||
phase: "xml" | "art";
|
||||
/**
|
||||
* True when files changed while this snapshot was being built, so some
|
||||
* entries may be stale. A follow-up rebuild is scheduled by the workspace.
|
||||
*/
|
||||
stale?: boolean;
|
||||
/** type -> id -> definitions (project + sdk + manifest, deduplicated). */
|
||||
assets: Map<string, Map<string, AssetDef[]>>;
|
||||
/** id -> definitions across all types. */
|
||||
@@ -108,6 +160,12 @@ export interface ModIndex {
|
||||
/** Problems found while indexing (unresolved includes, cycles, ...). */
|
||||
diagnostics: IndexerDiagnostic[];
|
||||
stats: IndexStats;
|
||||
/**
|
||||
* Document-local overlay (when the index was obtained through the
|
||||
* workspace's `getScope` / `getIndex` path). Optional so plain indexer
|
||||
* snapshots remain overlay-free.
|
||||
*/
|
||||
local?: LocalOverlay;
|
||||
}
|
||||
|
||||
export interface IndexOptions {
|
||||
@@ -160,4 +218,9 @@ export interface ParsedFile {
|
||||
*/
|
||||
records: IndexRecords | null;
|
||||
lineMap: LineMap | null;
|
||||
/**
|
||||
* True when the file is an art-asset XML that was only registered during
|
||||
* the XML phase (its shallow scan is deferred to the art phase).
|
||||
*/
|
||||
deferredArt?: boolean;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,24 @@
|
||||
import type { XmlDocument, XmlElement } from "../language/xmlParser";
|
||||
|
||||
/** Lowercases nothing; returns the part after the last ":" in a tag name. */
|
||||
export function localName(tag: string): string {
|
||||
const idx = tag.lastIndexOf(":");
|
||||
return idx >= 0 ? tag.slice(idx + 1) : tag;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolves the xpointer subset used by real RA3 mods:
|
||||
* xmlns(n=uri:ea.com:eala:asset) xpointer(/n:ElementName/child::*)
|
||||
*
|
||||
* Returns the container element whose children are selected by the xpointer,
|
||||
* or null when the form is unsupported / the container is missing.
|
||||
*/
|
||||
export function findXPointerContainer(
|
||||
doc: XmlDocument,
|
||||
xpointer: string,
|
||||
): XmlElement | null {
|
||||
const m = /xpointer\(\/\w+:(\w+)\/child::\*\)/.exec(xpointer);
|
||||
if (!m) return null;
|
||||
const name = m[1];
|
||||
return doc.elements.find((el) => localName(el.name) === name) ?? null;
|
||||
}
|
||||
+435
-23
@@ -1,10 +1,35 @@
|
||||
import * as vscode from "vscode";
|
||||
import { existsSync } from "node:fs";
|
||||
import { join, dirname } from "node:path";
|
||||
import { CachedDirectoryWalker } from "./indexer/fileScanner";
|
||||
import { readFile, stat } from "node:fs/promises";
|
||||
import { join, dirname, resolve } from "node:path";
|
||||
import {
|
||||
CachedDirectoryWalker,
|
||||
isContentRelevantPath,
|
||||
isWatcherNoisePath,
|
||||
} from "./indexer/fileScanner";
|
||||
import { ModIndexer } from "./indexer/indexer";
|
||||
import { DocumentCache, IncludeResolveCache, IndexRecordsCache } from "./indexer/caches";
|
||||
import type { ModIndex } from "./indexer/types";
|
||||
import {
|
||||
DocumentCache,
|
||||
IncludeResolveCache,
|
||||
IndexRecordsCache,
|
||||
InvalidationsEpoch,
|
||||
normKey,
|
||||
} from "./indexer/caches";
|
||||
import {
|
||||
DiskRecordsCache,
|
||||
diskCacheKey,
|
||||
type DiskCacheIdentity,
|
||||
type DiskCacheLoadStats,
|
||||
} from "./indexer/diskCache";
|
||||
import { buildSearchPaths, type SearchPaths } from "./indexer/includeResolver";
|
||||
import { extractIndexRecords } from "./indexer/records";
|
||||
import { LineMap, parseXml, stripBom } from "./language/xmlParser";
|
||||
import {
|
||||
buildDocumentScope,
|
||||
withLocalOverlay,
|
||||
type DocumentScope,
|
||||
} from "./indexer/localScope";
|
||||
import type { ModIndex, ParsedFile } from "./indexer/types";
|
||||
import { readSettings, type ExtensionSettings } from "./settings";
|
||||
|
||||
const REBUILD_DEBOUNCE_MS = 1500;
|
||||
@@ -22,6 +47,39 @@ export class ModWorkspace {
|
||||
private documentCache = new DocumentCache();
|
||||
private recordsCache = new IndexRecordsCache();
|
||||
private resolveCache = new IncludeResolveCache();
|
||||
/**
|
||||
* Monotonic invalidation counter. A build captures the epoch when it
|
||||
* starts; snapshots published after any invalidation are marked stale so
|
||||
* features can tell users "this index may be slightly out of date" while
|
||||
* the follow-up rebuild converges.
|
||||
*/
|
||||
private epoch = new InvalidationsEpoch();
|
||||
/** On-disk records cache (cold-start acceleration). */
|
||||
private diskCachePath: string | null = null;
|
||||
private diskCache: DiskRecordsCache | null = null;
|
||||
private diskCacheStats: DiskCacheLoadStats = {
|
||||
fileExists: false,
|
||||
keyMatched: false,
|
||||
loaded: 0,
|
||||
validated: 0,
|
||||
dropped: 0,
|
||||
};
|
||||
private diskSaved = false;
|
||||
private saving: Promise<void> | null = null;
|
||||
/** Document-local scopes (parse + expanded tree + overlay), per open doc. */
|
||||
private localScopes = new Map<
|
||||
string,
|
||||
{ version: number; indexEpoch: number; scope: DocumentScope }
|
||||
>();
|
||||
private localScopeBuilds = new Map<string, Promise<DocumentScope>>();
|
||||
private indexEpochValue = 0;
|
||||
/** Diagnostics: how many builds ran and what triggered the last one. */
|
||||
private buildCountValue = 0;
|
||||
private lastBuildTrigger = "initial";
|
||||
private pendingTrigger: string | null = null;
|
||||
private output: vscode.OutputChannel;
|
||||
/** Called whenever a new index snapshot is published (phase or final). */
|
||||
onIndexUpdate?: () => void;
|
||||
private context: vscode.ExtensionContext;
|
||||
private watchers: vscode.FileSystemWatcher[] = [];
|
||||
private statusBar: vscode.StatusBarItem;
|
||||
@@ -32,6 +90,11 @@ export class ModWorkspace {
|
||||
constructor(context: vscode.ExtensionContext) {
|
||||
this.context = context;
|
||||
this.settings = readSettings();
|
||||
const storageUri = context.storageUri ?? context.globalStorageUri;
|
||||
if (storageUri) {
|
||||
this.diskCachePath = join(storageUri.fsPath, "index-records-v1.json.gz");
|
||||
}
|
||||
this.output = vscode.window.createOutputChannel("RA3 Mod XML");
|
||||
this.statusBar = vscode.window.createStatusBarItem(
|
||||
vscode.StatusBarAlignment.Left,
|
||||
100,
|
||||
@@ -39,12 +102,32 @@ export class ModWorkspace {
|
||||
this.statusBar.name = "RA3 Mod XML";
|
||||
this.statusBar.command = "ra3modxml.openIndexReport";
|
||||
context.subscriptions.push(this.statusBar);
|
||||
context.subscriptions.push(
|
||||
vscode.workspace.onDidCloseTextDocument((document) => {
|
||||
this.localScopes.delete(document.uri.toString());
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
isRa3Workspace(): boolean {
|
||||
return this.projectRoot != null;
|
||||
}
|
||||
|
||||
/** True while a rebuild is running (before any snapshot is published). */
|
||||
get isBuilding(): boolean {
|
||||
return this.building;
|
||||
}
|
||||
|
||||
/** Number of index builds performed in this session. */
|
||||
get buildCount(): number {
|
||||
return this.buildCountValue;
|
||||
}
|
||||
|
||||
/** Why the last build started ("initial", "save", "watcher-*", ...). */
|
||||
get lastTrigger(): string {
|
||||
return this.lastBuildTrigger;
|
||||
}
|
||||
|
||||
detectProjectRoot(): string | null {
|
||||
const folders = vscode.workspace.workspaceFolders;
|
||||
if (!folders?.length) return null;
|
||||
@@ -64,7 +147,7 @@ export class ModWorkspace {
|
||||
this.startWatching();
|
||||
this.statusBar.text = "$(sync~spin) RA3 XML: indexing…";
|
||||
this.statusBar.show();
|
||||
await this.rebuild();
|
||||
await this.rebuild(false, "initial");
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -74,6 +157,7 @@ export class ModWorkspace {
|
||||
*/
|
||||
invalidate(path: string): void {
|
||||
if (!path) return;
|
||||
this.epoch.mark();
|
||||
this.documentCache.invalidate(path);
|
||||
this.recordsCache.invalidate(path);
|
||||
}
|
||||
@@ -83,6 +167,7 @@ export class ModWorkspace {
|
||||
* (which encode file existence) are no longer trustworthy.
|
||||
*/
|
||||
invalidateExistence(): void {
|
||||
this.epoch.mark();
|
||||
this.resolveCache.clear();
|
||||
}
|
||||
|
||||
@@ -107,13 +192,32 @@ export class ModWorkspace {
|
||||
new vscode.RelativePattern(root, "**/*"),
|
||||
);
|
||||
watcher.onDidCreate((uri) => {
|
||||
if (isWatcherNoisePath(uri.fsPath)) return;
|
||||
this.output.appendLine(`[watcher-create] ${uri.fsPath}`);
|
||||
this.invalidate(uri.fsPath);
|
||||
this.invalidateExistence();
|
||||
this.scheduleRebuild("watcher-create");
|
||||
});
|
||||
watcher.onDidChange((uri) => {
|
||||
if (isWatcherNoisePath(uri.fsPath)) return;
|
||||
// Content changes only matter for files that can change index
|
||||
// records (XML-ish documents); textures/binary art changes do not.
|
||||
if (
|
||||
!isContentRelevantPath(uri.fsPath) &&
|
||||
!this.isIndexedPath(uri.fsPath)
|
||||
) {
|
||||
return;
|
||||
}
|
||||
this.output.appendLine(`[watcher-change] ${uri.fsPath}`);
|
||||
this.invalidate(uri.fsPath);
|
||||
this.scheduleRebuild("watcher-change");
|
||||
});
|
||||
watcher.onDidChange((uri) => this.invalidate(uri.fsPath));
|
||||
watcher.onDidDelete((uri) => {
|
||||
if (isWatcherNoisePath(uri.fsPath)) return;
|
||||
this.output.appendLine(`[watcher-delete] ${uri.fsPath}`);
|
||||
this.invalidate(uri.fsPath);
|
||||
this.invalidateExistence();
|
||||
this.scheduleRebuild("watcher-delete");
|
||||
});
|
||||
this.watchers.push(watcher);
|
||||
this.context.subscriptions.push(watcher);
|
||||
@@ -124,15 +228,22 @@ export class ModWorkspace {
|
||||
}
|
||||
}
|
||||
|
||||
scheduleRebuild(): void {
|
||||
/** True when the path is part of the current index (any build state). */
|
||||
private isIndexedPath(fsPath: string): boolean {
|
||||
if (this.indexer?.isIndexedFile(fsPath)) return true;
|
||||
return this.index?.files.has(normKey(fsPath)) ?? false;
|
||||
}
|
||||
|
||||
scheduleRebuild(reason = "unknown"): void {
|
||||
if (!this.projectRoot) return;
|
||||
this.pendingTrigger = reason;
|
||||
if (this.rebuildTimer) clearTimeout(this.rebuildTimer);
|
||||
this.rebuildTimer = setTimeout(() => {
|
||||
void this.rebuild();
|
||||
void this.rebuild(false, this.pendingTrigger ?? reason);
|
||||
}, REBUILD_DEBOUNCE_MS);
|
||||
}
|
||||
|
||||
async rebuild(force = false): Promise<void> {
|
||||
async rebuild(force = false, trigger = "unknown"): Promise<void> {
|
||||
if (!this.projectRoot) return;
|
||||
if (this.building) {
|
||||
this.dirty = true;
|
||||
@@ -140,8 +251,18 @@ export class ModWorkspace {
|
||||
}
|
||||
if (force) this.resolveCache.clear();
|
||||
this.building = true;
|
||||
this.buildCountValue++;
|
||||
this.lastBuildTrigger = trigger;
|
||||
this.output.appendLine(
|
||||
`[build #${this.buildCountValue}] trigger=${trigger} force=${force} start=${new Date().toISOString()}`,
|
||||
);
|
||||
this.settings = readSettings();
|
||||
const epochAtStart = this.epoch.snapshot();
|
||||
try {
|
||||
// Cold start: seed the records cache from disk (stat-validated) so a
|
||||
// fresh session does not re-read unchanged files (Corona: 2.6 GB of
|
||||
// art assets) just because the in-memory caches are empty.
|
||||
await this.seedRecordsFromDisk();
|
||||
this.statusBar.text = "$(sync~spin) RA3 XML: indexing…";
|
||||
const indexer = new ModIndexer({
|
||||
projectDir: this.projectRoot,
|
||||
@@ -157,30 +278,320 @@ export class ModWorkspace {
|
||||
// verification (ra3modxml.reindex).
|
||||
trustUnchanged: !force,
|
||||
});
|
||||
const started = Date.now();
|
||||
this.index = await indexer.build();
|
||||
this.indexer = indexer;
|
||||
const secs = ((Date.now() - started) / 1000).toFixed(1);
|
||||
const s = this.index.stats;
|
||||
this.statusBar.text = `$(symbol-misc) RA3 XML: ${formatCount(s.assetCount)} assets`;
|
||||
this.statusBar.tooltip =
|
||||
`${s.projectDir}\n` +
|
||||
`${s.indexedFiles} files indexed (${s.parsedFiles} parsed, ${s.shallowScannedFiles} art assets shallow-scanned, ${secs}s)\n` +
|
||||
`${s.assetCount} assets (${s.manifestAssetCount} from ${s.manifestFiles} manifests)\n` +
|
||||
`${s.defineCount} defines, ${s.streams} streams, ${s.sourceCandidates} include candidates`;
|
||||
// The XML phase is published as soon as it is ready, so completion /
|
||||
// navigation / diagnostics work while the art scan continues.
|
||||
const finalIndex = await indexer.build((phaseIndex) => {
|
||||
this.publishIndex(phaseIndex, epochAtStart);
|
||||
});
|
||||
this.publishIndex(finalIndex, epochAtStart);
|
||||
this.output.appendLine(
|
||||
`[build #${this.buildCountValue}] done in ${(finalIndex.stats.elapsedMs / 1000).toFixed(1)}s (phase=${finalIndex.phase}, assets=${finalIndex.stats.assetCount}, stale=${finalIndex.stale === true}, walk=${(finalIndex.stats.walkMs / 1000).toFixed(1)}s, candidates=${(finalIndex.stats.candidatesMs / 1000).toFixed(1)}s, art=${(finalIndex.stats.artScanMs / 1000).toFixed(1)}s)`,
|
||||
);
|
||||
this.saveRecordsToDisk();
|
||||
} catch (err) {
|
||||
this.index = null;
|
||||
this.statusBar.text = "$(error) RA3 XML: indexing failed";
|
||||
this.statusBar.tooltip = err instanceof Error ? err.message : String(err);
|
||||
if (this.index) {
|
||||
// Keep the last good snapshot (marked stale) instead of disabling the
|
||||
// extension entirely; a later rebuild can recover.
|
||||
this.index.stale = true;
|
||||
this.statusBar.text = "$(error) RA3 XML: indexing failed (stale index kept)";
|
||||
this.statusBar.tooltip = err instanceof Error ? err.message : String(err);
|
||||
this.onIndexUpdate?.();
|
||||
} else {
|
||||
this.statusBar.text = "$(error) RA3 XML: indexing failed";
|
||||
this.statusBar.tooltip = err instanceof Error ? err.message : String(err);
|
||||
}
|
||||
} finally {
|
||||
this.building = false;
|
||||
if (this.dirty) {
|
||||
this.dirty = false;
|
||||
void this.rebuild();
|
||||
void this.rebuild(false, `dirty-followup (${this.lastBuildTrigger})`);
|
||||
} else if (this.index) {
|
||||
// Build is fully over: refresh diagnostics with full local scopes
|
||||
// (the snapshot published while `building` was still true only got
|
||||
// cheap parse-only scopes).
|
||||
this.onIndexUpdate?.();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private diskCacheIdentity(): DiskCacheIdentity | null {
|
||||
if (!this.projectRoot) return null;
|
||||
return {
|
||||
projectDir: this.projectRoot,
|
||||
sdkDir: this.settings.sdkPath,
|
||||
indexSageXml: this.settings.indexSageXml,
|
||||
additionalDataSearchPaths: this.settings.additionalDataSearchPaths,
|
||||
builtmodsDirs: this.settings.builtmodsDirs,
|
||||
};
|
||||
}
|
||||
|
||||
/** Loads + stat-validates the disk cache into the records cache once. */
|
||||
private async seedRecordsFromDisk(): Promise<void> {
|
||||
if (this.recordsCache.size > 0) return;
|
||||
const identity = this.diskCacheIdentity();
|
||||
if (!this.diskCachePath || !identity) return;
|
||||
this.diskCache = new DiskRecordsCache(this.diskCachePath, identity);
|
||||
this.statusBar.text = "$(sync~spin) RA3 XML: validating cache…";
|
||||
const { records, stats } = await this.diskCache.loadValidated();
|
||||
this.diskCacheStats = stats;
|
||||
this.diskSaved = false;
|
||||
for (const rec of records) {
|
||||
this.recordsCache.set(rec.key, {
|
||||
stat: rec.stat,
|
||||
records: rec.records,
|
||||
kind: rec.kind,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/** Persists the records cache after a successful build (best-effort). */
|
||||
private saveRecordsToDisk(): void {
|
||||
if (!this.diskCache) return;
|
||||
// Snapshot the entries now: the save runs in the background while the
|
||||
// next rebuild may already be mutating the live cache.
|
||||
const entries = [...this.recordsCache.entries()];
|
||||
const prev = this.saving ?? Promise.resolve();
|
||||
this.saving = prev
|
||||
.then(async () => {
|
||||
await this.diskCache!.save(entries);
|
||||
this.diskSaved = true;
|
||||
})
|
||||
.catch(() => {
|
||||
// Disk persistence is best-effort; the in-memory cache still works.
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Clears every cache (in-memory + disk + directory walker) and starts a
|
||||
* full forced rebuild. Used by the `ra3modxml.clearCache` command.
|
||||
*/
|
||||
clearCaches(): void {
|
||||
this.localScopes.clear();
|
||||
this.documentCache.clear();
|
||||
this.recordsCache.clear();
|
||||
this.resolveCache.clear();
|
||||
this.walker.clear();
|
||||
this.diskCacheStats = {
|
||||
fileExists: false,
|
||||
keyMatched: false,
|
||||
loaded: 0,
|
||||
validated: 0,
|
||||
dropped: 0,
|
||||
};
|
||||
this.diskSaved = false;
|
||||
void this.diskCache?.clear();
|
||||
void this.rebuild(true, "clear-cache");
|
||||
}
|
||||
|
||||
/** Human-readable cache status for the `ra3modxml.showCacheReport` command. */
|
||||
async cacheReport(): Promise<string> {
|
||||
const lines: string[] = ["RA3 Mod XML cache report"];
|
||||
lines.push(`Disk cache: ${this.diskCachePath ?? "not available"}`);
|
||||
if (this.diskCachePath) {
|
||||
const status = await this.diskCache?.status();
|
||||
lines.push(
|
||||
` file: ${status?.exists ? `${(status.sizeBytes / 1024).toFixed(1)} KB` : "missing"}`,
|
||||
);
|
||||
const identity = this.diskCacheIdentity();
|
||||
lines.push(` identity key: ${identity ? diskCacheKey(identity) : "-"}`);
|
||||
lines.push(
|
||||
` last load: file=${this.diskCacheStats.fileExists} keyMatched=${this.diskCacheStats.keyMatched} loaded=${this.diskCacheStats.loaded} validated=${this.diskCacheStats.validated} dropped=${this.diskCacheStats.dropped}`,
|
||||
);
|
||||
lines.push(` saved after last build: ${this.diskSaved}`);
|
||||
}
|
||||
lines.push(
|
||||
`In-memory: ${this.recordsCache.size} record entries · ${this.documentCache.size} documents (${this.documentCache.elements} elements) · ${this.resolveCache.size} include resolutions`,
|
||||
);
|
||||
lines.push(`Builds: #${this.buildCount} (last trigger: ${this.lastBuildTrigger})`);
|
||||
if (this.index) {
|
||||
const s = this.index.stats;
|
||||
lines.push(
|
||||
`Last build: snapshotHits=${s.snapshotHits} snapshotFallbacks=${s.snapshotFallbacks} recordsCacheHits=${s.recordsCacheHits} shallowCacheHits=${s.shallowCacheHits}`,
|
||||
);
|
||||
}
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
/**
|
||||
* Publishes an index snapshot (intermediate phase or final). If any file
|
||||
* was invalidated while the snapshot was being built, it is marked stale;
|
||||
* the dirty/rebuild mechanism converges shortly after.
|
||||
*/
|
||||
private publishIndex(index: ModIndex, epochAtStart: number): void {
|
||||
if (this.epoch.changedSince(epochAtStart)) index.stale = true;
|
||||
this.index = index;
|
||||
this.indexEpochValue++;
|
||||
// The merged index attached to a document scope changes with every
|
||||
// published snapshot, so cached scopes are rebuilt lazily on next use.
|
||||
this.localScopes.clear();
|
||||
this.updateStatusBar(index);
|
||||
this.onIndexUpdate?.();
|
||||
if (!index.complete) {
|
||||
this.output.appendLine(
|
||||
`[build #${this.buildCountValue}] phase A published in ${(index.stats.elapsedMs / 1000).toFixed(1)}s (${index.stats.assetCount} assets, ${index.stats.deferredArtFiles} art files pending)`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
private updateStatusBar(idx: ModIndex): void {
|
||||
const s = idx.stats;
|
||||
const stale = idx.stale ? " (stale)" : "";
|
||||
if (!idx.complete) {
|
||||
this.statusBar.text = `$(sync~spin) RA3 XML: XML indexed, scanning art…${stale}`;
|
||||
} else {
|
||||
this.statusBar.text = `$(symbol-misc) RA3 XML: ${formatCount(s.assetCount)} assets${stale}`;
|
||||
}
|
||||
this.statusBar.tooltip =
|
||||
`${s.projectDir}\n` +
|
||||
`${s.indexedFiles} files indexed (${s.parsedFiles} parsed, ${s.shallowScannedFiles} art assets shallow-scanned, ${(s.elapsedMs / 1000).toFixed(1)}s)\n` +
|
||||
`${s.assetCount} assets (${s.manifestAssetCount} from ${s.manifestFiles} manifests)\n` +
|
||||
`${s.defineCount} defines, ${s.streams} streams, ${s.sourceCandidates} include candidates\n` +
|
||||
`Phase: ${s.phase} · Complete: ${s.complete}${stale}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Search paths derived from the current settings, usable even before the
|
||||
* first index snapshot exists (include links / hover / diagnostics).
|
||||
*/
|
||||
searchPaths(): SearchPaths | null {
|
||||
if (!this.projectRoot) return null;
|
||||
return buildSearchPaths(this.settings.sdkPath, this.projectRoot);
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the document scope for the current text: original parse, expanded
|
||||
* logical tree, local overlay and overlay-aware merged index. Cached by
|
||||
* URI + document version + global index epoch.
|
||||
*/
|
||||
async getScope(document: vscode.TextDocument): Promise<DocumentScope> {
|
||||
const key = document.uri.toString();
|
||||
const cached = this.localScopes.get(key);
|
||||
if (
|
||||
cached &&
|
||||
cached.version === document.version &&
|
||||
cached.indexEpoch === this.indexEpochValue
|
||||
) {
|
||||
return cached.scope;
|
||||
}
|
||||
// While a rebuild is running, avoid competing with the indexer for disk
|
||||
// I/O: serve a parse-only scope (current file + XSD context, no include
|
||||
// chain / logical expansion). The published snapshot clears this cache,
|
||||
// so the next provider call after the build gets the full local scope.
|
||||
if (this.building) {
|
||||
return this.buildCheapScope(document);
|
||||
}
|
||||
const pending = this.localScopeBuilds.get(key);
|
||||
if (pending) return pending;
|
||||
const versionAtStart = document.version;
|
||||
const promise = this.buildScope(document)
|
||||
.then((scope) => {
|
||||
this.localScopes.set(key, {
|
||||
version: versionAtStart,
|
||||
indexEpoch: this.indexEpochValue,
|
||||
scope,
|
||||
});
|
||||
return scope;
|
||||
})
|
||||
.finally(() => {
|
||||
this.localScopeBuilds.delete(key);
|
||||
});
|
||||
this.localScopeBuilds.set(key, promise);
|
||||
return promise;
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the global index with this document's local overlay attached, or
|
||||
* a minimal local-only index while the global index is still building.
|
||||
*/
|
||||
async getIndex(document: vscode.TextDocument): Promise<ModIndex | null> {
|
||||
if (!this.isRa3Workspace()) return null;
|
||||
return (await this.getScope(document)).merged;
|
||||
}
|
||||
|
||||
private async buildScope(
|
||||
document: vscode.TextDocument,
|
||||
): Promise<DocumentScope> {
|
||||
const projectRoot = this.projectRoot;
|
||||
if (!projectRoot) throw new Error("RA3 workspace root is not available");
|
||||
const searchPaths =
|
||||
this.searchPaths() ?? buildSearchPaths(this.settings.sdkPath, projectRoot);
|
||||
const readRecords = async (path: string): Promise<ParsedFile | null> =>
|
||||
this.indexer ? this.indexer.readDocument(path) : this.fallbackRead(path);
|
||||
const readDom = async (path: string): Promise<ParsedFile | null> =>
|
||||
this.indexer ? this.indexer.readDom(path) : this.fallbackRead(path);
|
||||
const scope = await buildDocumentScope(
|
||||
document.uri.fsPath,
|
||||
document.getText(),
|
||||
document.version,
|
||||
{
|
||||
projectDir: projectRoot,
|
||||
sdkDir: this.settings.sdkPath,
|
||||
searchPaths,
|
||||
readRecords,
|
||||
readDom,
|
||||
},
|
||||
);
|
||||
scope.merged = withLocalOverlay(
|
||||
this.index,
|
||||
scope.overlay,
|
||||
projectRoot,
|
||||
this.settings.sdkPath,
|
||||
);
|
||||
return scope;
|
||||
}
|
||||
|
||||
private async buildCheapScope(
|
||||
document: vscode.TextDocument,
|
||||
): Promise<DocumentScope> {
|
||||
const projectRoot = this.projectRoot;
|
||||
if (!projectRoot) throw new Error("RA3 workspace root is not available");
|
||||
const searchPaths =
|
||||
this.searchPaths() ?? buildSearchPaths(this.settings.sdkPath, projectRoot);
|
||||
const scope = await buildDocumentScope(
|
||||
document.uri.fsPath,
|
||||
document.getText(),
|
||||
document.version,
|
||||
{
|
||||
projectDir: projectRoot,
|
||||
sdkDir: this.settings.sdkPath,
|
||||
searchPaths,
|
||||
readRecords: async () => null,
|
||||
readDom: async () => null,
|
||||
},
|
||||
);
|
||||
scope.merged = withLocalOverlay(
|
||||
this.index,
|
||||
scope.overlay,
|
||||
projectRoot,
|
||||
this.settings.sdkPath,
|
||||
);
|
||||
return scope;
|
||||
}
|
||||
|
||||
/**
|
||||
* Fallback used before the first ModIndexer exists (e.g. during initial
|
||||
* activation): parses an XML file directly so the document-local scope can
|
||||
* still follow small include chains.
|
||||
*/
|
||||
private async fallbackRead(path: string): Promise<ParsedFile | null> {
|
||||
try {
|
||||
const st = await stat(path);
|
||||
if (st.size > 4 * 1024 * 1024) return null;
|
||||
const text = stripBom(await readFile(path, "utf8"));
|
||||
const lineMap = new LineMap(text);
|
||||
const parse = parseXml(text);
|
||||
return {
|
||||
file: { path: resolve(path), stat: null },
|
||||
parse,
|
||||
records: extractIndexRecords(parse, lineMap),
|
||||
lineMap,
|
||||
};
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/** Parses the (possibly unsaved) in-memory text of the active document. */
|
||||
async parseText(path: string, text: string) {
|
||||
const { parseXml, LineMap } = await import("./language/xmlParser");
|
||||
@@ -196,6 +607,7 @@ export class ModWorkspace {
|
||||
dispose(): void {
|
||||
if (this.rebuildTimer) clearTimeout(this.rebuildTimer);
|
||||
this.statusBar.dispose();
|
||||
this.output.dispose();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+32
-2
@@ -4,11 +4,15 @@ import {
|
||||
DocumentCache,
|
||||
IncludeResolveCache,
|
||||
IndexRecordsCache,
|
||||
InvalidationsEpoch,
|
||||
} from "../out/indexer/caches.js";
|
||||
import { resolve } from "node:path";
|
||||
|
||||
const stat = { mtimeMs: 1, size: 1, birthtimeMs: 1, ctimeMs: 1 };
|
||||
|
||||
function parsed(path, elements) {
|
||||
return {
|
||||
file: { path, stat: { mtimeMs: 1, size: 1 } },
|
||||
file: { path, stat },
|
||||
parse: { root: { name: "r" }, elements: new Array(elements), errors: [] },
|
||||
records: null,
|
||||
lineMap: null,
|
||||
@@ -47,7 +51,7 @@ test("DocumentCache invalidate frees budget", () => {
|
||||
test("IndexRecordsCache stores and invalidates entries", () => {
|
||||
const cache = new IndexRecordsCache();
|
||||
const entry = {
|
||||
stat: { mtimeMs: 1, size: 1 },
|
||||
stat,
|
||||
records: { assets: [], defines: [], includes: [], rootXiIncludes: [], nestedXiIncludes: [] },
|
||||
kind: "full",
|
||||
};
|
||||
@@ -57,6 +61,20 @@ test("IndexRecordsCache stores and invalidates entries", () => {
|
||||
assert.equal(cache.get("a.xml"), undefined);
|
||||
});
|
||||
|
||||
test("IndexRecordsCache exposes entries for disk persistence", () => {
|
||||
const cache = new IndexRecordsCache();
|
||||
const entry = {
|
||||
stat,
|
||||
records: { assets: [], defines: [], includes: [], rootXiIncludes: [], nestedXiIncludes: [] },
|
||||
kind: "full",
|
||||
};
|
||||
cache.set("a.xml", entry);
|
||||
const list = [...cache.entries()];
|
||||
assert.equal(list.length, 1);
|
||||
assert.equal(list[0][0], resolve("a.xml").toLowerCase());
|
||||
assert.equal(list[0][1], entry);
|
||||
});
|
||||
|
||||
test("IncludeResolveCache stores sources and manifest lookups", () => {
|
||||
const cache = new IncludeResolveCache();
|
||||
const key = "dir|DATA:static.xml";
|
||||
@@ -71,3 +89,15 @@ test("IncludeResolveCache stores sources and manifest lookups", () => {
|
||||
assert.equal(cache.get(key), undefined);
|
||||
assert.equal(cache.getManifest("static.xml"), undefined);
|
||||
});
|
||||
|
||||
test("InvalidationsEpoch tracks changes since a snapshot", () => {
|
||||
const epoch = new InvalidationsEpoch();
|
||||
const before = epoch.snapshot();
|
||||
assert.equal(epoch.changedSince(before), false);
|
||||
epoch.mark();
|
||||
assert.equal(epoch.changedSince(before), true);
|
||||
assert.equal(epoch.changedSince(epoch.snapshot()), false);
|
||||
epoch.mark();
|
||||
epoch.mark();
|
||||
assert.equal(epoch.current, 3);
|
||||
});
|
||||
|
||||
@@ -76,6 +76,8 @@ require.cache["vscode-stub"] = {
|
||||
};
|
||||
|
||||
const { Ra3CompletionProvider } = require("../out/features/completion.js");
|
||||
const { parseXml, LineMap } = require("../out/language/xmlParser.js");
|
||||
const { expandDocument } = require("../out/indexer/logicalTree.js");
|
||||
|
||||
function makeDocument(text) {
|
||||
const lineStarts = [0];
|
||||
@@ -98,9 +100,26 @@ function makeDocument(text) {
|
||||
};
|
||||
}
|
||||
|
||||
// Enum completions do not consult the index, but provideCompletionItems only
|
||||
// routes value contexts when the workspace index is present.
|
||||
const provider = new Ra3CompletionProvider({ index: {} });
|
||||
async function makeScope(text, idx) {
|
||||
const lineMap = new LineMap(text);
|
||||
const parse = parseXml(text);
|
||||
const expanded = await expandDocument("test.xml", parse, {
|
||||
resolve: () => null,
|
||||
readDom: async () => null,
|
||||
});
|
||||
return { expanded, merged: idx, overlay: {} };
|
||||
}
|
||||
|
||||
// Enum completions do not consult the index, and value contexts now work
|
||||
// even before the workspace index exists.
|
||||
const makeProvider = (idx) =>
|
||||
new Ra3CompletionProvider({
|
||||
index: idx,
|
||||
isRa3Workspace: () => true,
|
||||
getScope: async (document) => makeScope(document.getText(), idx),
|
||||
});
|
||||
const provider = makeProvider({});
|
||||
const providerNoIndex = makeProvider(null);
|
||||
const token = { isCancellationRequested: false };
|
||||
|
||||
test("Surfaces enum completion works with an unclosed quote", async () => {
|
||||
@@ -155,3 +174,73 @@ test("empty unterminated value offers all enum values", async () => {
|
||||
assert.ok(labels.includes("WATER"));
|
||||
assert.ok(labels.includes("CRUSHABLE_WALL"));
|
||||
});
|
||||
|
||||
test("enum completions work without an index", async () => {
|
||||
const text =
|
||||
`<AssetDeclaration>\n <LocomotorTemplate id="x" Surfaces="G>\n <Other/>\n</LocomotorTemplate>\n</AssetDeclaration>`;
|
||||
const line1 = text.split("\n")[1];
|
||||
const pos = new Position(1, line1.indexOf("G") + 1);
|
||||
|
||||
const items = await providerNoIndex.provideCompletionItems(
|
||||
makeDocument(text),
|
||||
pos,
|
||||
token,
|
||||
);
|
||||
const labels = items.map((i) => i.label);
|
||||
assert.ok(labels.includes("GROUND"), "enum value offered without an index");
|
||||
assert.ok(!labels.includes("WATER"));
|
||||
});
|
||||
|
||||
test("element and attribute name completions work without an index", async () => {
|
||||
const text = `<AssetDeclaration>\n <LocomotorTemplate `;
|
||||
const line1 = text.split("\n")[1];
|
||||
const pos = new Position(1, line1.length);
|
||||
|
||||
const items = await providerNoIndex.provideCompletionItems(
|
||||
makeDocument(text),
|
||||
pos,
|
||||
token,
|
||||
);
|
||||
const labels = items.map((i) => i.label);
|
||||
assert.ok(labels.includes("id"));
|
||||
assert.ok(labels.includes("Surfaces"));
|
||||
});
|
||||
|
||||
test("content (child element) completions work without an index", async () => {
|
||||
const text =
|
||||
`<AssetDeclaration>\n <LocomotorTemplate id="x">\n \n </LocomotorTemplate>\n</AssetDeclaration>`;
|
||||
const pos = new Position(2, 4);
|
||||
|
||||
const items = await providerNoIndex.provideCompletionItems(
|
||||
makeDocument(text),
|
||||
pos,
|
||||
token,
|
||||
);
|
||||
const labels = items.map((i) => i.label);
|
||||
assert.ok(labels.length > 0, "child elements offered without an index");
|
||||
});
|
||||
|
||||
test("Poid attributes offer ids from the enclosing GameObject's local scope", async () => {
|
||||
const text =
|
||||
`<AssetDeclaration>\n` +
|
||||
` <GameObject id="Tank">\n` +
|
||||
` <Draws>\n` +
|
||||
` <TruckDraw id="ModuleTag_Draw" />\n` +
|
||||
` </Draws>\n` +
|
||||
` <BehaviorModules>\n` +
|
||||
` <ReconstituteStateSpecialAbility UpdateModuleId="ModuleTag_D">\n` +
|
||||
` </BehaviorModules>\n` +
|
||||
` </GameObject>\n` +
|
||||
`</AssetDeclaration>`;
|
||||
const line6 = text.split("\n")[6];
|
||||
const pos = new Position(6, line6.indexOf("ModuleTag_D") + "ModuleTag_D".length);
|
||||
|
||||
const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
|
||||
const labels = items.map((i) => i.label);
|
||||
assert.ok(labels.includes("ModuleTag_Draw"));
|
||||
assert.ok(items.every((i) => i.kind === CompletionItemKind.Value));
|
||||
assert.ok(
|
||||
items.every((i) => i.detail === "local module"),
|
||||
"Poid completions are labelled as local-scope ids",
|
||||
);
|
||||
});
|
||||
|
||||
@@ -0,0 +1,132 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { join } from "node:path";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import { DiskRecordsCache, diskCacheKey } from "../out/indexer/diskCache.js";
|
||||
|
||||
const identity = {
|
||||
projectDir: "C:/proj",
|
||||
sdkDir: "C:/sdk",
|
||||
indexSageXml: true,
|
||||
additionalDataSearchPaths: [],
|
||||
builtmodsDirs: ["C:/sdk/builtmods"],
|
||||
};
|
||||
|
||||
const sampleRecords = {
|
||||
assets: [{ type: "GameObject", id: "TankA", line: 3 }],
|
||||
defines: [{ name: "HP", value: "100", line: 2 }],
|
||||
includes: [{ type: "all", source: "Units.xml", line: 4 }],
|
||||
rootXiIncludes: [],
|
||||
nestedXiIncludes: [],
|
||||
};
|
||||
|
||||
function stampOf(file) {
|
||||
const s = fs.statSync(file);
|
||||
return {
|
||||
mtimeMs: s.mtimeMs,
|
||||
size: s.size,
|
||||
birthtimeMs: s.birthtimeMs,
|
||||
ctimeMs: s.ctimeMs,
|
||||
};
|
||||
}
|
||||
|
||||
function makeTmp(t) {
|
||||
const tmp = fs.mkdtempSync(join(os.tmpdir(), "ra3-diskcache-"));
|
||||
t.after(() => fs.rmSync(tmp, { recursive: true, force: true }));
|
||||
return tmp;
|
||||
}
|
||||
|
||||
test("disk cache roundtrip keeps records and leaves no temp file", async (t) => {
|
||||
const tmp = makeTmp(t);
|
||||
const file = join(tmp, "a.xml");
|
||||
fs.writeFileSync(file, "0123456789");
|
||||
const filePath = join(tmp, "index-records.json.gz");
|
||||
const cache = new DiskRecordsCache(filePath, identity);
|
||||
|
||||
await cache.save([
|
||||
[
|
||||
file.toLowerCase(),
|
||||
{ stat: stampOf(file), records: sampleRecords, kind: "full" },
|
||||
],
|
||||
]);
|
||||
assert.equal(fs.existsSync(`${filePath}.tmp`), false, "atomic write leaves no temp");
|
||||
|
||||
const { records, stats } = await cache.loadValidated();
|
||||
assert.equal(stats.fileExists, true);
|
||||
assert.equal(stats.keyMatched, true);
|
||||
assert.equal(stats.loaded, 1);
|
||||
assert.equal(stats.validated, 1);
|
||||
assert.equal(stats.dropped, 0);
|
||||
assert.equal(records.length, 1);
|
||||
assert.deepEqual(records[0].records, sampleRecords);
|
||||
});
|
||||
|
||||
test("stat mismatch drops the cached entry", async (t) => {
|
||||
const tmp = makeTmp(t);
|
||||
const file = join(tmp, "a.xml");
|
||||
fs.writeFileSync(file, "0123456789");
|
||||
const filePath = join(tmp, "index-records.json.gz");
|
||||
const cache = new DiskRecordsCache(filePath, identity);
|
||||
await cache.save([
|
||||
[file.toLowerCase(), { stat: stampOf(file), records: sampleRecords, kind: "full" }],
|
||||
]);
|
||||
|
||||
const past = new Date(Date.now() - 60000);
|
||||
fs.utimesSync(file, past, past);
|
||||
const { records, stats } = await cache.loadValidated();
|
||||
assert.equal(stats.validated, 0);
|
||||
assert.equal(stats.dropped, 1);
|
||||
assert.equal(records.length, 0);
|
||||
});
|
||||
|
||||
test("identity mismatch ignores the cache", async (t) => {
|
||||
const tmp = makeTmp(t);
|
||||
const file = join(tmp, "a.xml");
|
||||
fs.writeFileSync(file, "0123456789");
|
||||
const filePath = join(tmp, "index-records.json.gz");
|
||||
const cache = new DiskRecordsCache(filePath, identity);
|
||||
await cache.save([
|
||||
[file.toLowerCase(), { stat: stampOf(file), records: sampleRecords, kind: "full" }],
|
||||
]);
|
||||
|
||||
const other = new DiskRecordsCache(filePath, {
|
||||
...identity,
|
||||
sdkDir: "D:/other-sdk",
|
||||
});
|
||||
const { records, stats } = await other.loadValidated();
|
||||
assert.equal(stats.fileExists, true);
|
||||
assert.equal(stats.keyMatched, false);
|
||||
assert.equal(records.length, 0);
|
||||
});
|
||||
|
||||
test("corrupt cache file yields an empty result", async (t) => {
|
||||
const tmp = makeTmp(t);
|
||||
const filePath = join(tmp, "index-records.json.gz");
|
||||
fs.writeFileSync(filePath, "this is not gzip json");
|
||||
const cache = new DiskRecordsCache(filePath, identity);
|
||||
const { records, stats } = await cache.loadValidated();
|
||||
assert.equal(stats.fileExists, true);
|
||||
assert.equal(records.length, 0);
|
||||
});
|
||||
|
||||
test("clear removes the cache file", async (t) => {
|
||||
const tmp = makeTmp(t);
|
||||
const file = join(tmp, "a.xml");
|
||||
fs.writeFileSync(file, "0123456789");
|
||||
const filePath = join(tmp, "index-records.json.gz");
|
||||
const cache = new DiskRecordsCache(filePath, identity);
|
||||
await cache.save([
|
||||
[file.toLowerCase(), { stat: stampOf(file), records: sampleRecords, kind: "full" }],
|
||||
]);
|
||||
assert.ok(fs.existsSync(filePath));
|
||||
await cache.clear();
|
||||
assert.equal(fs.existsSync(filePath), false);
|
||||
});
|
||||
|
||||
test("diskCacheKey differs when the identity changes", () => {
|
||||
const a = diskCacheKey(identity);
|
||||
const b = diskCacheKey({ ...identity, indexSageXml: false });
|
||||
assert.notEqual(a, b);
|
||||
assert.equal(a, diskCacheKey(identity));
|
||||
});
|
||||
@@ -0,0 +1,102 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { join, parse, resolve } from "node:path";
|
||||
import fs from "node:fs";
|
||||
import os from "node:os";
|
||||
import { resolveSource } from "../out/indexer/includeResolver.js";
|
||||
import {
|
||||
ExistenceSnapshot,
|
||||
buildExistenceSnapshot,
|
||||
isDriveRoot,
|
||||
} from "../out/indexer/existence.js";
|
||||
|
||||
function makeTmp(t) {
|
||||
const tmp = fs.mkdtempSync(join(os.tmpdir(), "ra3-existence-"));
|
||||
t.after(() => fs.rmSync(tmp, { recursive: true, force: true }));
|
||||
return tmp;
|
||||
}
|
||||
|
||||
test("isDriveRoot detects filesystem roots", () => {
|
||||
assert.equal(isDriveRoot(parse(process.cwd()).root), true);
|
||||
assert.equal(isDriveRoot(process.cwd()), false);
|
||||
});
|
||||
|
||||
test("ExistenceSnapshot answers covered paths and falls back outside roots", (t) => {
|
||||
const tmp = makeTmp(t);
|
||||
const dataDir = join(tmp, "data");
|
||||
fs.mkdirSync(dataDir);
|
||||
const existing = join(dataDir, "Units.xml");
|
||||
fs.writeFileSync(existing, "<x/>");
|
||||
|
||||
const snap = new ExistenceSnapshot([dataDir]);
|
||||
assert.equal(snap.has(existing), true);
|
||||
assert.equal(snap.has(join(dataDir, "Missing.xml")), false);
|
||||
assert.equal(snap.has(join(tmp, "outside.xml")), null, "outside roots is unknown");
|
||||
assert.ok(snap.hits >= 2, "covered lookups counted as hits");
|
||||
assert.equal(snap.fallbacks, 1);
|
||||
|
||||
if (process.platform === "win32") {
|
||||
assert.equal(
|
||||
snap.has(existing.toUpperCase()),
|
||||
true,
|
||||
"lookup is case-insensitive on Windows",
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test("buildExistenceSnapshot covers bounded search bases lazily", async (t) => {
|
||||
const tmp = makeTmp(t);
|
||||
const dataDir = join(tmp, "data");
|
||||
const artDir = join(tmp, "art");
|
||||
fs.mkdirSync(join(dataDir, "sub"), { recursive: true });
|
||||
fs.mkdirSync(artDir);
|
||||
fs.writeFileSync(join(dataDir, "Units.xml"), "<x/>");
|
||||
fs.writeFileSync(join(dataDir, "sub", "Nested.xml"), "<x/>");
|
||||
fs.writeFileSync(join(artDir, "Tank.w3x"), "<x/>");
|
||||
|
||||
const snap = buildExistenceSnapshot({
|
||||
DATA: [dataDir],
|
||||
ART: [artDir],
|
||||
AUDIO: [],
|
||||
});
|
||||
assert.equal(snap.has(join(dataDir, "Units.xml")), true);
|
||||
assert.equal(snap.has(join(dataDir, "sub", "Nested.xml")), true);
|
||||
assert.equal(snap.has(join(artDir, "Tank.w3x")), true);
|
||||
assert.equal(snap.has(join(dataDir, "Missing.xml")), false);
|
||||
});
|
||||
|
||||
test("resolveSource uses the snapshot and falls back to statSync outside it", async (t) => {
|
||||
const tmp = makeTmp(t);
|
||||
const dataDir = join(tmp, "data");
|
||||
const outsideDir = join(tmp, "outside");
|
||||
fs.mkdirSync(dataDir);
|
||||
fs.mkdirSync(outsideDir);
|
||||
const units = join(dataDir, "Units.xml");
|
||||
const outside = join(outsideDir, "Extra.xml");
|
||||
fs.writeFileSync(units, "<x/>");
|
||||
fs.writeFileSync(outside, "<x/>");
|
||||
|
||||
const searchPaths = { DATA: [dataDir], ART: [], AUDIO: [] };
|
||||
const snap = buildExistenceSnapshot(searchPaths);
|
||||
|
||||
const found = resolveSource("DATA:Units.xml", null, searchPaths, snap);
|
||||
assert.equal(found.path, units);
|
||||
assert.ok(snap.hits > 0, "covered lookup served by the snapshot");
|
||||
|
||||
const missing = resolveSource("DATA:Missing.xml", null, searchPaths, snap);
|
||||
assert.equal(missing.path, null);
|
||||
|
||||
const emptySearchPaths = { DATA: [], ART: [], AUDIO: [] };
|
||||
const outsideResolved = resolveSource(
|
||||
outside,
|
||||
outsideDir,
|
||||
emptySearchPaths,
|
||||
snap,
|
||||
);
|
||||
assert.equal(
|
||||
resolve(outsideResolved.path ?? ""),
|
||||
resolve(outside),
|
||||
"uncovered path falls back to statSync",
|
||||
);
|
||||
assert.ok(snap.fallbacks > 0, "fallback counted");
|
||||
});
|
||||
@@ -0,0 +1,72 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { join } from "node:path";
|
||||
import {
|
||||
isContentRelevantPath,
|
||||
isWatcherNoisePath,
|
||||
} from "../out/indexer/fileScanner.js";
|
||||
|
||||
test("isWatcherNoisePath filters .git internals", () => {
|
||||
assert.equal(
|
||||
isWatcherNoisePath(join("C:/proj", ".git", "index")),
|
||||
true,
|
||||
".git files are noise",
|
||||
);
|
||||
assert.equal(
|
||||
isWatcherNoisePath(join("C:/proj", "data", ".git", "FETCH_HEAD")),
|
||||
true,
|
||||
"nested .git directories are noise",
|
||||
);
|
||||
assert.equal(
|
||||
isWatcherNoisePath(join("C:/proj", "Data", "Mod.xml")),
|
||||
false,
|
||||
"project XML is not noise",
|
||||
);
|
||||
assert.equal(
|
||||
isWatcherNoisePath(join("C:/proj", ".gitignore")),
|
||||
false,
|
||||
".gitignore is a real project file",
|
||||
);
|
||||
assert.equal(
|
||||
isWatcherNoisePath(join("C:/proj", "Data", "UnitCrate.xml.git")),
|
||||
true,
|
||||
"editor temp files ending in .git are noise",
|
||||
);
|
||||
assert.equal(
|
||||
isWatcherNoisePath(join("C:/proj", "Data", "UnitCrate.xml.tmp")),
|
||||
true,
|
||||
".tmp files are noise",
|
||||
);
|
||||
assert.equal(
|
||||
isWatcherNoisePath(join("C:/proj", "Data", "Mod.xml~")),
|
||||
true,
|
||||
"backup files ending in ~ are noise",
|
||||
);
|
||||
assert.equal(
|
||||
isWatcherNoisePath(join("C:/proj", "Data", ".#Mod.xml")),
|
||||
true,
|
||||
"lock files starting with .# are noise",
|
||||
);
|
||||
});
|
||||
|
||||
test("isContentRelevantPath only reacts to XML-ish content", () => {
|
||||
assert.equal(isContentRelevantPath("a.xml"), true);
|
||||
assert.equal(isContentRelevantPath("a.w3x"), true);
|
||||
assert.equal(
|
||||
isContentRelevantPath("a.manifestxml"),
|
||||
false,
|
||||
"there is no .manifestxml source format",
|
||||
);
|
||||
assert.equal(
|
||||
isContentRelevantPath("a.w3d"),
|
||||
false,
|
||||
".w3d is binary art, not text XML",
|
||||
);
|
||||
assert.equal(isContentRelevantPath("a.dds"), false);
|
||||
assert.equal(isContentRelevantPath("a.xml.git"), false);
|
||||
assert.equal(
|
||||
isContentRelevantPath("a.lua"),
|
||||
false,
|
||||
"lua is not indexed yet",
|
||||
);
|
||||
});
|
||||
@@ -0,0 +1,6 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<AssetDeclaration xmlns="uri:ea.com:eala:asset">
|
||||
<HeadlightDraw2>
|
||||
<TruckDraw id="ModuleTag_Headlight" />
|
||||
</HeadlightDraw2>
|
||||
</AssetDeclaration>
|
||||
@@ -0,0 +1,4 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<AssetDeclaration xmlns="uri:ea.com:eala:asset">
|
||||
<GameObject id="StandaloneBase" Side="Allies" />
|
||||
</AssetDeclaration>
|
||||
+1
-1
@@ -2,7 +2,7 @@
|
||||
<AssetDeclaration xmlns="uri:ea.com:eala:asset">
|
||||
<Includes>
|
||||
<Include type="all" source="Models/Tank_SKN.w3x" />
|
||||
<Include type="all" source="Models/Tank_FP.w3d" />
|
||||
<Include type="all" source="Models/Tank_FP.dat" />
|
||||
<Include type="all" source="Models/Tank_Damaged.dds" />
|
||||
</Includes>
|
||||
</AssetDeclaration>
|
||||
|
||||
+20
@@ -0,0 +1,20 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<AssetDeclaration xmlns="uri:ea.com:eala:asset">
|
||||
<Includes>
|
||||
<Include type="instance" source="Includes/StandaloneBase.xml" />
|
||||
</Includes>
|
||||
<Defines>
|
||||
<Define name="STANDALONE_HEALTH" value="200.0" />
|
||||
</Defines>
|
||||
<GameObject id="StandaloneTank" inheritFrom="StandaloneBase">
|
||||
<Draws>
|
||||
<TruckDraw id="ModuleTag_Draw" />
|
||||
<xi:include
|
||||
href="DATA:Includes/HeadlightModules.xml"
|
||||
xpointer="xmlns(n=uri:ea.com:eala:asset) xpointer(/n:HeadlightDraw2/child::*)" />
|
||||
</Draws>
|
||||
<BehaviorModules>
|
||||
<ReconstituteStateSpecialAbility UpdateModuleId="ModuleTag_Headlight" />
|
||||
</BehaviorModules>
|
||||
</GameObject>
|
||||
</AssetDeclaration>
|
||||
+105
-2
@@ -79,7 +79,7 @@ test("provides include source candidates", async () => {
|
||||
assert.ok(xml.some((c) => c.source === "DATA:static.xml"));
|
||||
});
|
||||
|
||||
test("indexes art-asset XML (.w3x / sniffed .w3d) via shallow scan and skips binary", async () => {
|
||||
test("indexes art-asset XML (.w3x / sniffed unknown extension) via shallow scan and skips binary", async () => {
|
||||
const idx = await buildIndex();
|
||||
|
||||
// The .w3x hub chain: Mod.xml -> VehicleArt.xml -> Models/Tank_SKN.w3x.
|
||||
@@ -96,7 +96,7 @@ test("indexes art-asset XML (.w3x / sniffed .w3d) via shallow scan and skips bin
|
||||
// Unknown extension with XML content is sniffed and indexed.
|
||||
assert.ok(
|
||||
idx.assetsById.get("tank_fp")?.some((d) => d.type === "W3DMesh"),
|
||||
"unknown-extension XML (.w3d) sniffed and indexed",
|
||||
"unknown-extension XML (.dat) sniffed and indexed",
|
||||
);
|
||||
|
||||
// Binary content is registered as a file but never parsed.
|
||||
@@ -133,6 +133,107 @@ test("w3x files appear in Include source completion candidates", async () => {
|
||||
);
|
||||
});
|
||||
|
||||
test("build publishes an immutable XML phase before art scanning", async () => {
|
||||
let phaseA;
|
||||
const indexer = new ModIndexer({
|
||||
projectDir: project,
|
||||
sdkDir: sdk,
|
||||
builtmodsDirs: [join(sdk, "builtmods")],
|
||||
indexSageXml: true,
|
||||
additionalDataSearchPaths: [],
|
||||
walker: new CachedDirectoryWalker(),
|
||||
});
|
||||
const idx = await indexer.build((p) => {
|
||||
phaseA = p;
|
||||
});
|
||||
|
||||
assert.ok(phaseA, "phase-A snapshot published");
|
||||
assert.equal(phaseA.complete, false);
|
||||
assert.equal(phaseA.phase, "xml");
|
||||
assert.ok(phaseA.assetsById.has("testtank"), "XML assets available in phase A");
|
||||
assert.ok(
|
||||
phaseA.assetsById.has("vanillatank"),
|
||||
"manifest assets available in phase A",
|
||||
);
|
||||
assert.equal(
|
||||
phaseA.assetsById.has("tank_skn"),
|
||||
false,
|
||||
"art assets deferred in phase A",
|
||||
);
|
||||
assert.ok(phaseA.stats.deferredArtFiles >= 2, "deferred art queue recorded");
|
||||
assert.equal(phaseA.stats.shallowScannedFiles, 0, "no art scanned during phase A");
|
||||
|
||||
assert.equal(idx.complete, true);
|
||||
assert.equal(idx.phase, "art");
|
||||
assert.ok(idx.assetsById.has("tank_skn"), "art assets present in the final index");
|
||||
assert.equal(
|
||||
phaseA.assetsById.has("tank_skn"),
|
||||
false,
|
||||
"phase-A snapshot is immutable (phase B did not mutate it)",
|
||||
);
|
||||
assert.equal(typeof idx.stats.artScanMs, "number");
|
||||
assert.equal(
|
||||
indexer.isIndexedFile(join(project, "Data", "Mod.xml")),
|
||||
true,
|
||||
"indexed files are recognized",
|
||||
);
|
||||
assert.equal(
|
||||
indexer.isIndexedFile(join(project, "Data", "NotIndexed.xml")),
|
||||
false,
|
||||
"unrelated files are not recognized",
|
||||
);
|
||||
});
|
||||
|
||||
test("stat validation re-reads a file whose mtime changed", async (t) => {
|
||||
const tmp = fs.mkdtempSync(join(os.tmpdir(), "ra3modxml-mtime-"));
|
||||
t.after(() => fs.rmSync(tmp, { recursive: true, force: true }));
|
||||
const projectDir = join(tmp, "project");
|
||||
fs.mkdirSync(join(projectDir, "Data"), { recursive: true });
|
||||
const modPath = join(projectDir, "Data", "Mod.xml");
|
||||
fs.writeFileSync(
|
||||
modPath,
|
||||
`<?xml version="1.0"?>\n<AssetDeclaration>\n <GameObject id="TankA"/>\n</AssetDeclaration>\n`,
|
||||
"utf8",
|
||||
);
|
||||
|
||||
const documentCache = new DocumentCache();
|
||||
const recordsCache = new IndexRecordsCache();
|
||||
const resolveCache = new IncludeResolveCache();
|
||||
const make = () =>
|
||||
new ModIndexer({
|
||||
projectDir,
|
||||
sdkDir: sdk,
|
||||
builtmodsDirs: [join(sdk, "builtmods")],
|
||||
indexSageXml: false,
|
||||
additionalDataSearchPaths: [],
|
||||
walker: new CachedDirectoryWalker(),
|
||||
documentCache,
|
||||
recordsCache,
|
||||
resolveCache,
|
||||
trustUnchanged: false,
|
||||
});
|
||||
|
||||
const first = await make().build();
|
||||
assert.ok(first.assetsById.has("tanka"));
|
||||
assert.equal(first.stats.recordsCacheHits, 0);
|
||||
|
||||
const past = new Date(Date.now() - 60000);
|
||||
fs.utimesSync(modPath, past, past);
|
||||
const second = await make().build();
|
||||
assert.equal(
|
||||
second.stats.recordsCacheHits,
|
||||
0,
|
||||
"mtime change invalidates the cached records",
|
||||
);
|
||||
assert.ok(second.assetsById.has("tanka"));
|
||||
|
||||
const third = await make().build();
|
||||
assert.ok(
|
||||
third.stats.recordsCacheHits > 0,
|
||||
"unchanged files are served from the records cache",
|
||||
);
|
||||
});
|
||||
|
||||
test("shallow scans and full parses are cached across rebuilds", async () => {
|
||||
const documentCache = new DocumentCache();
|
||||
const recordsCache = new IndexRecordsCache();
|
||||
@@ -218,6 +319,8 @@ test("index stats include candidate/walk phase timings", async () => {
|
||||
const idx = await buildIndex();
|
||||
assert.equal(typeof idx.stats.candidatesMs, "number");
|
||||
assert.equal(typeof idx.stats.walkMs, "number");
|
||||
assert.ok(idx.stats.snapshotHits > 0, "existence snapshot answered lookups");
|
||||
assert.equal(typeof idx.stats.snapshotFallbacks, "number");
|
||||
});
|
||||
|
||||
test("w3x with a UTF-8 BOM is indexed with correct offsets", async (t) => {
|
||||
|
||||
@@ -0,0 +1,210 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { dirname, join } from "node:path";
|
||||
import { parseXml, LineMap, stripBom } from "../out/language/xmlParser.js";
|
||||
import { extractIndexRecords } from "../out/indexer/records.js";
|
||||
import { buildSearchPaths } from "../out/indexer/includeResolver.js";
|
||||
import {
|
||||
buildDocumentScope,
|
||||
withLocalOverlay,
|
||||
} from "../out/indexer/localScope.js";
|
||||
import {
|
||||
findContainingGameObject,
|
||||
findLocalId,
|
||||
collectLocalIds,
|
||||
} from "../out/indexer/logicalTree.js";
|
||||
import { resolveReferenceTargetsForType } from "../out/indexer/refs.js";
|
||||
import { resolveElementType } from "../out/language/typeContext.js";
|
||||
|
||||
const root = dirname(dirname(fileURLToPath(import.meta.url)));
|
||||
const project = join(root, "test", "fixtures", "minimod");
|
||||
const sdk = join(root, "test", "fixtures", "fakesdk");
|
||||
const standalonePath = join(project, "Data", "Standalone.xml");
|
||||
|
||||
async function readParsed(path) {
|
||||
const text = stripBom(await readFile(path, "utf8"));
|
||||
const lineMap = new LineMap(text);
|
||||
const parse = parseXml(text);
|
||||
return {
|
||||
file: { path, stat: null },
|
||||
parse,
|
||||
records: extractIndexRecords(parse, lineMap),
|
||||
lineMap,
|
||||
};
|
||||
}
|
||||
|
||||
async function makeScope() {
|
||||
const searchPaths = buildSearchPaths(sdk, project);
|
||||
const text = await readFile(standalonePath, "utf8");
|
||||
return buildDocumentScope(standalonePath, text, 1, {
|
||||
projectDir: project,
|
||||
sdkDir: sdk,
|
||||
searchPaths,
|
||||
readRecords: readParsed,
|
||||
readDom: readParsed,
|
||||
});
|
||||
}
|
||||
|
||||
test("local overlay resolves refs for a file outside every global stream", async () => {
|
||||
const scope = await makeScope();
|
||||
const merged = withLocalOverlay(null, scope.overlay, project, sdk);
|
||||
|
||||
assert.ok(merged.local.assetsById.has("standalonetank"));
|
||||
assert.ok(merged.local.assetsById.has("standalonebase"));
|
||||
assert.ok(merged.local.defines.has("standalone_health"));
|
||||
|
||||
const targets = resolveReferenceTargetsForType(
|
||||
merged,
|
||||
"GameObject",
|
||||
"inheritFrom",
|
||||
"StandaloneBase",
|
||||
);
|
||||
assert.equal(targets.length, 1);
|
||||
assert.match(targets[0].def.file, /StandaloneBase\.xml$/);
|
||||
assert.equal(targets[0].def.origin, "project");
|
||||
});
|
||||
|
||||
test("logical xi:include expansion gives included modules their Draws context", async () => {
|
||||
const scope = await makeScope();
|
||||
const included = scope.expanded.elements.find(
|
||||
(e) =>
|
||||
e.name === "TruckDraw" &&
|
||||
e.attrs.some((a) => a.name === "id" && a.value === "ModuleTag_Headlight"),
|
||||
);
|
||||
assert.ok(included, "xi:include target spliced into the logical tree");
|
||||
assert.match(included.sourceFile, /HeadlightModules\.xml$/i);
|
||||
assert.equal(
|
||||
resolveElementType(included),
|
||||
"W3DTruckDrawModuleData",
|
||||
"included module resolves through the logical Draws parent",
|
||||
);
|
||||
|
||||
const update = scope.expanded.elements.find(
|
||||
(e) => e.name === "ReconstituteStateSpecialAbility",
|
||||
);
|
||||
assert.ok(update);
|
||||
const gameObject = findContainingGameObject(update);
|
||||
assert.equal(gameObject?.name, "GameObject");
|
||||
assert.equal(
|
||||
findLocalId(gameObject, "ModuleTag_Headlight"),
|
||||
included,
|
||||
"Poid reference can reach a module spliced in through xi:include",
|
||||
);
|
||||
|
||||
const localIds = collectLocalIds(gameObject).map((i) => i.id);
|
||||
assert.ok(localIds.includes("ModuleTag_Draw"));
|
||||
assert.ok(localIds.includes("ModuleTag_Headlight"));
|
||||
});
|
||||
|
||||
test("local overlay wins over a global definition with the same id", async () => {
|
||||
const scope = await makeScope();
|
||||
const global = {
|
||||
assets: new Map(),
|
||||
assetsById: new Map([
|
||||
[
|
||||
"standalonebase",
|
||||
[
|
||||
{
|
||||
type: "GameObject",
|
||||
id: "StandaloneBase",
|
||||
file: join(sdk, "SageXml", "VanillaBase.xml"),
|
||||
line: 1,
|
||||
origin: "sdk",
|
||||
},
|
||||
],
|
||||
],
|
||||
]),
|
||||
defines: new Map(),
|
||||
};
|
||||
const merged = withLocalOverlay(global, scope.overlay, project, sdk);
|
||||
const targets = resolveReferenceTargetsForType(
|
||||
merged,
|
||||
"GameObject",
|
||||
"inheritFrom",
|
||||
"StandaloneBase",
|
||||
);
|
||||
assert.equal(targets.length, 2);
|
||||
assert.match(targets[0].def.file, /StandaloneBase\.xml$/);
|
||||
assert.equal(targets[0].def.origin, "project");
|
||||
assert.match(targets[1].def.file, /VanillaBase\.xml$/);
|
||||
});
|
||||
|
||||
test("logical expansion terminates on xi:include cycles", async (t) => {
|
||||
const tmp = await mkdtemp(join(tmpdir(), "ra3-local-cycle-"));
|
||||
t.after(() => rm(tmp, { recursive: true, force: true }));
|
||||
const dataDir = join(tmp, "Data");
|
||||
const includesDir = join(dataDir, "Includes");
|
||||
await mkdir(includesDir, { recursive: true });
|
||||
const aPath = join(dataDir, "A.xml");
|
||||
const bPath = join(includesDir, "B.xml");
|
||||
await writeFile(
|
||||
aPath,
|
||||
`<AssetDeclaration><GameObject id="A"><xi:include href="Includes/B.xml"/></GameObject></AssetDeclaration>`,
|
||||
"utf8",
|
||||
);
|
||||
await writeFile(
|
||||
bPath,
|
||||
`<AssetDeclaration><GameObject id="B"><xi:include href="../A.xml"/></GameObject></AssetDeclaration>`,
|
||||
"utf8",
|
||||
);
|
||||
const searchPaths = buildSearchPaths(sdk, tmp);
|
||||
const text = await readFile(aPath, "utf8");
|
||||
const scope = await buildDocumentScope(aPath, text, 1, {
|
||||
projectDir: tmp,
|
||||
sdkDir: sdk,
|
||||
searchPaths,
|
||||
readRecords: readParsed,
|
||||
readDom: readParsed,
|
||||
});
|
||||
assert.ok(
|
||||
scope.expanded.elements.some(
|
||||
(e) =>
|
||||
e.name === "GameObject" &&
|
||||
e.attrs.some((a) => a.name === "id" && a.value === "A"),
|
||||
),
|
||||
"entry document survives a cycle",
|
||||
);
|
||||
});
|
||||
|
||||
test("the same xi:include target can expand under multiple parents", async (t) => {
|
||||
const tmp = await mkdtemp(join(tmpdir(), "ra3-local-shared-"));
|
||||
t.after(() => rm(tmp, { recursive: true, force: true }));
|
||||
const dataDir = join(tmp, "Data");
|
||||
const includesDir = join(dataDir, "Includes");
|
||||
await mkdir(includesDir, { recursive: true });
|
||||
const mainPath = join(dataDir, "Main.xml");
|
||||
const fragmentPath = join(includesDir, "Fragment.xml");
|
||||
await writeFile(
|
||||
fragmentPath,
|
||||
`<AssetDeclaration><HeadlightDraw2><TruckDraw id="ModuleTag_Shared"/></HeadlightDraw2></AssetDeclaration>`,
|
||||
"utf8",
|
||||
);
|
||||
await writeFile(
|
||||
mainPath,
|
||||
`<AssetDeclaration>` +
|
||||
`<GameObject id="A"><Draws><xi:include href="DATA:Includes/Fragment.xml" ` +
|
||||
`xpointer="xmlns(n=uri:ea.com:eala:asset) xpointer(/n:HeadlightDraw2/child::*)"/></Draws></GameObject>` +
|
||||
`<GameObject id="B"><Draws><xi:include href="DATA:Includes/Fragment.xml" ` +
|
||||
`xpointer="xmlns(n=uri:ea.com:eala:asset) xpointer(/n:HeadlightDraw2/child::*)"/></Draws></GameObject>` +
|
||||
`</AssetDeclaration>`,
|
||||
"utf8",
|
||||
);
|
||||
const searchPaths = buildSearchPaths(sdk, tmp);
|
||||
const text = await readFile(mainPath, "utf8");
|
||||
const scope = await buildDocumentScope(mainPath, text, 1, {
|
||||
projectDir: tmp,
|
||||
sdkDir: sdk,
|
||||
searchPaths,
|
||||
readRecords: readParsed,
|
||||
readDom: readParsed,
|
||||
});
|
||||
const shared = scope.expanded.elements.filter(
|
||||
(e) =>
|
||||
e.name === "TruckDraw" &&
|
||||
e.attrs.some((a) => a.name === "id" && a.value === "ModuleTag_Shared"),
|
||||
);
|
||||
assert.equal(shared.length, 2, "same fragment expands once per parent");
|
||||
});
|
||||
@@ -130,13 +130,13 @@ test("fallback tokens cover names, attributes and values", () => {
|
||||
|
||||
test("well-formed XML gets no semantic fallback tokens", async () => {
|
||||
const text = `<AssetDeclaration>\n <LocomotorTemplate id="x" Surfaces="GROUND"/>\n</AssetDeclaration>`;
|
||||
const provider = new Ra3SemanticTokensProvider();
|
||||
const provider = new Ra3SemanticTokensProvider({ isRa3Workspace: () => true });
|
||||
const result = await provider.provideDocumentSemanticTokens(makeDocument(text), {});
|
||||
assert.equal(result.data.length, 0);
|
||||
});
|
||||
|
||||
test("malformed XML gets semantic fallback tokens", async () => {
|
||||
const provider = new Ra3SemanticTokensProvider();
|
||||
const provider = new Ra3SemanticTokensProvider({ isRa3Workspace: () => true });
|
||||
const result = await provider.provideDocumentSemanticTokens(
|
||||
makeDocument(MALFORMED),
|
||||
{},
|
||||
|
||||
Reference in New Issue
Block a user