improve codelens

This commit is contained in:
2026-08-07 13:16:50 +02:00
parent 47807f9fed
commit 36eaaafa01
29 changed files with 2288 additions and 181 deletions
+28 -11
View File
@@ -19,7 +19,15 @@
400 条时标记为不完整,继续输入会重新请求,因此 `CrateDebris_01` 这类排在
列表后部的 id 不会因首屏截断而消失。
- **悬停提示**:元素/属性显示 XSD 文档、类型、必填/默认值;引用值显示定义位置;`$DEFINE` 显示值与定义位置;`Include source` / `xi:include href` 显示解析后的目标文件;`xi:include` 元素与属性给出 XInclude 说明。
- **引用导航**:从引用值(`CommandSet="..."``Weapon="..."``inheritFrom``<CreateObject>ID</CreateObject>` 等元素文本)跳转到定义(严格按引用类型过滤,候选由 `ra3modxml.definitionMode` 控制:`all` 列出 mod + 原版、`project-only` 优先项目内定义);`Ctrl+点击` Include / `xi:include href` 打开目标文件;Find All References 同时搜索属性值与元素文本内容;文档大纲列出顶层资产与 `$DEFINE`
- **引用导航**:从引用值(`CommandSet="..."``Weapon="..."``inheritFrom``<CreateObject>ID</CreateObject>` 等元素文本)跳转到定义(严格按引用类型过滤,候选由 `ra3modxml.definitionMode` 控制:`all` 列出 mod + 原版、`project-only` 优先项目内定义);`Ctrl+点击` Include / `xi:include href` 打开目标文件;Find All References 基于**语义引用索引**(属性引用 + simple-content 文本 + `inheritFrom`,排除 id 定义点 / Poid / `$DEFINE`,不再全文搜索);文档大纲列出顶层资产与 `$DEFINE`
- **引用计数(CodeLens)**:在“设计上应被引用”的顶部资产类型上显示
`0 references` / `1 reference` / `N references`0 也显示),点击直接打开
references peek;设置类、地图元数据、w3x 子结构等自动注册类型不显示,
避免满屏 0manifest 资产有对应 SageXml 源码时,引用按源码定义归并计数
(打开 SageXml 源码同样能看到引用数)。
- **未引用资产**`RA3 Mod XML: Find unreferenced assets…` 命令按类型列出
所有零引用的项目资产并跳转;编辑器右键菜单
`Find unreferenced assets of this type` 可直接使用光标所在资产类型。
- **当前文档局部作用域(T1)**:即使一个文件不在任何全局流里(没有从
`Data/Mod.xml` / `additionalmaps` 可达),插件也会按当前文件自身的资产、
`$DEFINE` 及其 include 链建立局部索引。`xi:include` 会在逻辑树中展开,
@@ -38,13 +46,16 @@
Include 跳转/悬停照常工作;引用类诊断会“显示但标注”
`unresolved-reference-indexing` + `(index incomplete)` 说明),不会把
未完成的索引误当成最终结论。
- **大项目性能**:索引记录(资产 / Define / Include / 行号)与 include 解析结果
跨重建缓存,保存触发的重建零 stat、零重读(Corona 实测约 2 秒);DOM 树只按需
保留并设元素预算,避免内存膨胀。编辑器外的文件改动(git pull、导出工具)会
触发防抖重建;构建期间文件再次被修改时,已发布索引会标记 `(stale)` 并自动重跑。
include 路径解析使用目录枚举建立的文件集快照(无 statSync 风暴);records
缓存会持久化到磁盘(gzip + 多信号 stat 校验 + 原子写),重启 VS Code 后冷启动
只需秒级校验,Corona 实测约 11 秒(首次全量约 2 分钟)。
- **大项目性能**:索引记录(资产 / Define / Include / 引用 / 行号)与 include
解析结果跨重建缓存,保存触发的重建零 stat、零重读(Corona 实测约 2 秒);
DOM 树只按需保留并设元素预算,避免内存膨胀。编辑器外的文件改动(git pull、
导出工具)会触发防抖重建;构建期间文件再次被修改时,已发布索引会标记
`(stale)` 并自动重跑。include 路径解析使用目录枚举建立的文件集快照(无
statSync 风暴);records 缓存会持久化到磁盘(gzip + 多信号 stat 校验 +
内容哈希 + 原子写),重启 VS Code 后冷启动只需秒级校验,Corona 实测约 11 秒
(首次全量约 2 分钟)。引用索引只从构建期实际消费的 records 构建;打开文档
时若发现当前文本的 records 与快照不一致(如外置盘重连后缓存过时),会自动
定向重建自愈;`Re-index workspace` 会对 stat 匹配的 XML 也做内容校验。
## 使用
@@ -70,6 +81,9 @@
- `RA3 Mod XML: Show index report`:查看索引统计。
- `RA3 Mod XML: Clear caches and rebuild`:清空内存/磁盘缓存并强制全量重建。
- `RA3 Mod XML: Show cache report`:查看磁盘缓存路径、大小、校验统计与命中数。
- `RA3 Mod XML: Find unreferenced assets…`:按类型查找零引用的项目资产。
- `RA3 Mod XML: Find unreferenced assets of this type`:右键菜单入口,
直接查找光标所在顶部资产类型的未引用资产。
## 开发
@@ -102,17 +116,19 @@ src/
existence.ts 文件集存在性快照(目录枚举 Set,替代逐路径 statSync
manifestParser.ts .manifest 二进制解析(移植 OpenSAGE ManifestFile.cs
fileScanner.ts 目录扫描与 Include source 候选
refs.ts 引用目标解析(按引用类型过滤)
refs.ts 引用目标解析(按引用类型过滤)+ “设计上可被引用类型”判定
referenceIndex.ts 引用记录 → 反向引用索引(定义 → 引用位置)+ 未引用报告
xpointer.ts xi:include xpointer 子集解析(纯 TS
logicalTree.ts 当前文档逻辑树(xi:include 拼接、局部作用域)
localScope.ts 文档局部索引 overlay(自身链 + include 链)
shallowScan.ts .w3x 等大体积美术资产顶层浅扫描(纯 TS,不建 DOM)
records.ts 每文件紧凑索引记录(资产/Define/Include/xi + 行号)
records.ts 每文件紧凑索引记录(资产/Define/Include/xi/引用 + 行号偏移
caches.ts 跨重建持久缓存(DocumentCache / IndexRecordsCache /
IncludeResolveCache+ 失效纪元 InvalidationsEpoch
diskCache.ts 跨会话磁盘缓存(gzip JSON、原子写、多信号 stat 校验)
indexer.ts 工作区索引器(后台、缓存、记录驱动重建、分阶段发布)
features/ completion / hover / navigation / diagnostics / semanticTokens
features/ completion / hover / navigation / references / codeLens /
unreferenced / diagnostics / semanticTokens
syntaxes/ TextMate 注入语法
tools/ XSD → 模型、AssetType 枚举提取
```
@@ -124,4 +140,5 @@ tools/ XSD → 模型、AssetType 枚举提取
- 领域说明与需求:`docs/requirements.md`
- 调研与设计决策:`docs/plan.md`
- 问题分析与修复记录:`docs/analysis-issues.md`
- 引用计数 / 语义 FAR / 未引用资产功能设计:`docs/features-reference-counts.md`
- Manifest 格式参考:OpenSAGE `src/OpenSage.Game/Data/StreamFS/ManifestFile.cs`(本仓库 `OpenSAGE/` 子目录,commit `d45d361`
+116 -1
View File
@@ -974,7 +974,8 @@ stale=true 并触发 follow-up。处理(按用户建议的扩展名白名单
- w3x 文件名启发式定向扫描(按约定后续再做)。
- `AssetIdList` 等“任意资产 ID 列表”的引用语义建模。
- Find All References 目前仍是全文搜索,未走索引(对应需求 P1 高效搜索)。
- Find All References 已改为语义引用索引(第十八轮,2026-08-05,见
`docs/features-reference-counts.md`);通用内容搜索与索引复用仍属 P1 远期。
---
@@ -1509,3 +1510,117 @@ XSD 模型中共 **371 处“子元素是 simple type”的声明(149 个不
若 Corona 上“输入 C 后菜单出现慢”仍然明显,下一个瓶颈大概率是
`getScope()` 每次文档版本变化都重建完整局部 include 链 / 逻辑树;本轮先把
候选截断与类型过滤造成的“列表错误”修掉,局部 scope 缓存优化留作独立一轮。
---
## 二十三、问题分析(2026-08-05):FAR 把定义行算作“自引用”;CodeLens 漏掉 manifest 源引用
### 问题 1Find All References 把 id 定义行也算进结果
**现象**:对任意资产执行 FAR,结果里包含它自己的 `id="..."` 定义行,看起来
像“自己引用自己”;CodeLens 计数没有这个问题。
**根因**VS Code 的 FAR 默认带 `context.includeDeclaration = true`,旧实现
把定义位置附加进返回结果。语义反向索引本身不含 id 定义点(records 提取时
已排除),CodeLens 只读反向索引,所以两者不一致。
**修复**`findReferenceLocations` 不再附加定义位置,无论 `includeDeclaration`
取值;FAR 结果与 CodeLens 计数严格一致。
### 问题 2CodeLens 不显示 manifest 定义的引用,FAR 却显示
**现象**:打开 SageXml 原版源码时,CodeLens 显示 0 引用,但从该资产执行
FAR 能看到引用。
**根因**:反向索引的站点挂在“实际解析到的定义”上。当 SageXml 源码不在
include 遍历里时,引用只解析到 manifest 定义(如 `static.manifest` 条目),
站点挂在 manifest keyCodeLens 用“当前文档定义 key”精确查表所以是 0,
FAR 把同 id/同类型的 manifest 定义也合并进来所以能看到。
**修复**:新增 `referenceSitesForDefinition`:除了精确 key,还把
`manifestSource` 可解析到当前打开文件的 manifest 定义的站点并入计数——
语义上把“manifest 引用”视作“SageXml 源码对该 asset 的引用”
Go to Definition 本来就会把 manifest 定义映射到 SageXml 源码)。
**验证**AttachTest 中 326 个带引用的 manifest 定义此前全部没有对应
`origin: "sdk"` 定义(源码未遍历),现在打开对应 SageXml 源码即可看到计数
(如 `PlayerTemplate Allies` → 8 引用)。
### 举一反三的测试(146 → 147 全绿)
- `referenceProvider.test.mjs`FAR 即使 `includeDeclaration = true` 也不返回
定义行;从引用位置发起 FAR 结果一致;
- `referenceIndex.test.mjs``referenceSitesForDefinition` 把 manifest 源
站点并入 SageXml 源码定义,其他文件不串;
- `codeLens.test.mjs`:打开 manifestSource 对应的源码文件时,CodeLens 显示
manifest 定义上挂着的引用数。
版本 **0.1.16 → 0.1.17**。
---
## 二十四、问题分析(2026-08-06):引用索引与 records 缓存不同步 / 竞争
### 现象
Corona `Data/GlobalData/Weapon/Weapon_Allied.xml` 中的
`AlliedCommandoDesertEaglesWarhead`(一个 WeaponTemplate)偶尔没有 CodeLens
FAR 显示无引用;它实际被同文件 WeaponTemplate 的 `ProjectileNugget
WarheadTemplate="..."` 引用。该问题在移动硬盘重连 + 重新打开工作区 + 校验
磁盘缓存 + 重新 indexing 之后出现;在 ProjectileNugget 里 Ctrl+点击一次后恢复;
清缓存重载不复发。
### 调查
1. **模型核对**`AlliedCommandoDesertEaglesWarhead` 的元素是
`<WeaponTemplate>``ProjectileNuggetType@WarheadTemplate` 的 refType 是
`WeaponTemplate`,语义匹配成立。
2. **真实文件验证**:直接对 Weapon_Allied.xml 做 records 提取 + 反向索引,
引用记录(line 1332)能正确挂到定义(line 1341)——**新鲜构建没有问题**。
3. 因此问题不在提取/过滤逻辑,而在“引用索引与索引状态不同步”的缓存/竞争
路径。
### 找到的不同步 / 竞争点
- **`buildReferences()` 读共享 recordsCache**workspace 持有、跨重建复用),
而不是 walk 实际消费的 records
- 外部盘重连 / 旧磁盘缓存条目“stat 全匹配但内容过时”(FAT32/exFAT 时间戳
粒度 2s、同步工具保留 mtime、size 不变)→ 资产用旧 records 入库,而新
引用缺失;watcher 事件在重连期间丢失时快照不会标 stale → 持久化问题;
- 构建中途 watcher 失效、或 feature `readDom` 重写 recordsCache,都可能让
快照的 references 与 assets 来自不同版本的 records。
- **feature 在构建中调用 `readDom`**(定义跳转的精确定位)会改写同一个
indexer 的 `files` / `recordsCache`,污染进行中的构建。
- **force 重建(Re-index)之前只比 stat 信号**:stat 相同但内容不同的陈旧
条目会被直接复用,只有清缓存才能修复。
### 修复
1. **构建期本地 records**`ModIndexer.buildRecords` 记录本次 walk 实际消费的
`{ file, records, recordsHash }``buildReferences()` 只从这里构建反向索引。
中途失效 / feature 重读不再造成“资产在但引用缺失”。
2. **构建期 readDom 闸门**`assetDefLocation` 在 `ws.isBuilding` 时退化为
行级位置,避免定义跳转改写进行中的构建。
3. **force 重建内容校验**full XML 的 records 缓存条目带 `contentHash`
`Re-index workspace` 对 stat 匹配的条目也读文件比对哈希,不一致才重解析
(w3x 浅扫描不读,保持 2.6 GB 免读)。
4. **打开文档自愈**:快照发布每文件 `recordsHashes`CodeLens / FAR 对**已保存**
文档比较当前文本的 records 哈希,不一致则定向 `invalidate` +
`scheduleRebuild("records-desync")`。records 哈希忽略行尾/空白差异,
未保存编辑不触发,避免误报和循环。
5. **磁盘缓存 v2 → v3**:旧缓存没有哈希、无法校验/自愈,一次性重建后每文件
都带哈希。
### 验证(147 → 151 全绿)
- 构建中途 invalidate 某文件的 recordsCache,最终快照仍保留该文件的引用;
- force 重建:stat 全匹配但 contentHash 不同的条目被重新解析(trusted 路径
仍复用缓存不读盘);
- 自愈:records 哈希不一致时只对干净文件触发 invalidate + records-desync
- CodeLens 集成:打开文档与快照不同步时调度定向重建。
“Ctrl+点击后恢复”的精确时序无法在代码里复现;最可能是跳转前后恰好发生了一
次重建(watcher 事件 / dirty-followup)。自愈检查让这类问题不再依赖巧合:
只要文件被打开并触发 CodeLens / FAR,不一致就会被检测并定向修复。
版本 **0.1.17 → 0.1.18**
+122
View File
@@ -0,0 +1,122 @@
# 功能设计:引用计数、语义 Find All References 与未引用资产
> 状态:已实现(v0.1.16 起)。本文档记录需求分析结论、语义定义、数据流与
> 已知边界;问题排查记录仍放在 `docs/analysis-issues.md`,功能设计单独成文。
## 一、需求与结论
原始需求:
1. 在每个顶部 asset 上显示它被引用了多少次,点击后直接打开“查找所有引用”,
而不是必须通过右键菜单;
2. 能查找“某一种类型中所有没被引用的顶部 asset”。
调研结论(2026-08-05):
- XSD 模型层面,292 个有模型类型的顶层资产中:73 个有类型化引用指向它们,
28 个只能被 `inheritFrom` 指向,**191 个在模型里根本没有任何引用指向**
(设置类、地图元数据、w3x 子结构等)。对这些类型,“0 引用”是唯一正常状态。
- AttachTest 实测:625 个项目定义中 29% 是 0 引用、48% 是 1 引用。
因此“所有资产一律显示计数”会制造大量噪音,也会让用户误以为资产是孤儿。
- 结论:**计数按类型过滤**——只在“设计上应该被引用”的类型上显示
0 也显示,因为对 GameObject 这类类型 0 是有效信号);自动注册/结构类
类型不显示。
- 旧版 Find All References 是全文搜索(正则匹配 `"id"` / `>id<`),会把
`id="X"` 定义本身、`EditorName="X"` 等非引用属性也算进去。计数如果和它
同源,必然误导;如果不同源,点击后的结果又对不上。因此本轮把 FAR 一起
升级为语义引用索引,计数与点击结果共用同一数据源。
## 二、语义定义:什么算一次“引用”
与补全 / hover / 跳转 / 诊断完全一致(`refs.ts` 是唯一判定来源):
-`xas:refType` 的属性值(如 `CommandSet``LogicCommandSet`);
-`xas:refType` 的 simple-content 文本(如 `<CreateObject>ID</CreateObject>`);
- `inheritFrom`(按元素自身类型过滤);
-`refType``isRef` 属性(按同名 ID 匹配任意声明类型)。
不算引用:
- 元素自己的 `id` 定义点(除非是 `RoadObject@id→Road` 这类跨类型 id 引用);
- Poid 管线局部引用(`ModuleId``AttachModuleId``SoundRef` 等);
- `$DEFINE` / `=` 常量值;
- 枚举、文件路径、`Include@source` / `xi:include href`
- w3x 内部父子结构关系(`W3DMesh` 等靠结构归属,不走全局引用)。
引用索引按“类型 + id + 定义位置”精确归属,同名 ID 的不同类型定义互不串扰
`WeaponTemplate:X` 的引用不会计到 `GameObject:X` 头上)。
## 三、数据流
```
parse DOM
│ records.ts extractIndexRecords(parse, lineMap, text)
IndexRecords.references[] 每文件紧凑记录(refType / selfType / value /
│ 行号 / 起止偏移),随 records 缓存跨重建复用,
│ 并持久化到磁盘缓存(缓存版本 v3,full XML
│ 附带内容哈希;快照另发布每文件 records 哈希)
indexer.ts buildReferences() 只解算本次 build 触及的文件,防止陈旧缓存泄漏
│ referenceIndex.ts buildReferenceIndex()
ModIndex.references Map<定义 key, ReferenceSite[]>
├── CodeLens 计数(O(1) 查表)
├── 语义 Find All References(返回精确位置)
└── 未引用资产报告(0 引用 = 不在反向索引中)
```
引用记录在解析期就把 `refType` / `selfType` 固化下来,反向索引构建时不需要
再解析 DOM、也不需要上下文类型解析;只需对 `assetsById` 做一次查找并按类型
过滤。快照发布时(XML 阶段 + art 阶段)各构建一次反向索引,Corona 规模下
开销远小于 include 遍历。
反向索引只从**本次构建的 walk 实际消费的 records**`ModIndexer.buildRecords`
构建,不读共享 recordsCache:中途 watcher 失效、或 feature 通过 `readDom`
重读文件,都不会让“资产在但引用缺失”的快照出现。打开文档时 CodeLens / FAR
还会比较当前文本的 records 哈希与快照,不一致就定向 invalidate 并触发
`records-desync` 重建自愈(仅对已保存文档,未保存编辑不触发)。
## 四、CodeLens 规则
- 只对根级(`AssetDeclaration` 直接子元素)带 `id` 的资产显示;
- 只对 `isReferenceTargetType()` 为真的类型显示(类型化引用目标 +
`inheritFrom` 可继承类型;见 `refs.ts``referenceTargetTypes()`);
- 0 也显示:`0 references` / `1 reference` / `N references`
- 点击执行 `ra3modxml.showReferences``editor.action.showReferences`
打开 references peek,结果与计数完全一致(不含定义本身);
- 计数除了当前定义自己的反向索引桶,还并入“manifestSource 可解析到当前
文件”的 manifest 定义桶:manifest 资产有对应 SageXml 源码时,引用直接
视作 SageXml 源码对该 asset 的引用(Go to Definition 同样把 manifest
定义映射到 SageXml 源码);
- 索引重建完成后自动 `editor.action.codeLens.refresh`,计数不会停留在旧值。
## 五、未引用资产
- 主入口:命令面板 `RA3 Mod XML: Find unreferenced assets…`
第一步 QuickPick 选类型(只列“设计上应被引用”的类型,显示每种未引用数量),
第二步列出资产(`id — 相对路径:行号`),点击跳转到定义;
- 加速入口:编辑器右键菜单
`RA3 Mod XML: Find unreferenced assets of this type`
(仅 `editorLangId == xml && ra3modxml.active` 时显示),光标在顶部资产上
时直接预选该类型,否则回退到类型选择;
- 语义:只统计 `origin === "project"` 且非 `viaInstance` 的定义;
manifest / SDK 定义永远不参与;如果某个 id 覆盖了原版资产,来自原版/其他
流的引用同样计入(否则会把覆盖件误报成未使用)。
## 六、已知边界与后续
- 当前文档局部作用域(不在任何全局流里的文件)的引用不在全局反向索引中:
这类文件打开后 CodeLens / FAR 只反映全局流;局部链内的引用暂不计数。
- Find All References 不返回定义行本身(即使 VS Code 传入
`includeDeclaration`),因此结果数量与 CodeLens 计数严格一致。
- `AssetIdList` 等“任意资产 ID 列表”的引用语义仍未建模:整个属性值按一条
引用记录处理,与现有补全/诊断保持一致(已知缺口,后续可在 records 层扩展)。
- w3x 文件本身不提取引用记录(浅扫描无 DOM);对 w3x 资产的引用从引用它的
XML 文件捕获。
- manifest 二进制里包含 BAB 编译后的完整引用图(`AssetReferenceOffset/Count` +
8 字节 `TypeId+InstanceId` 条目),目前仍跳过。将来可解析它给原版/manifest
资产提供“编译器权威”计数;mod 源码资产在编译前没有 manifest,仍需源码级
引用索引。
- 未引用资产目前是命令 + QuickPick 的“查询”形态;如果之后想要常驻浏览,
可以再加 Tree View(更重,暂不计划)。
+54 -6
View File
@@ -1,6 +1,6 @@
# 调研结论与实施计划(已按最新代码同步更新)
> 说明:本文档随实现演进持续同步。最近一次同步(2026-08-04)对齐了实现过程中新增的模块与设计变更:BAB 精确搜索路径、manifest 类型/ID 推导、上下文感知元素类型、属性级 refType / Poid 局部引用(`id` 定义点)、精确跳转范围、嵌套 `xi:include`、注入式语法高亮、bit-flag 列表补全(空格触发 / 排除已用 / 追加模式)、simple-content 元素文本引用(补全 / hover / 跳转 / 诊断 / Find All References)等。
> 说明:本文档随实现演进持续同步。最近一次同步(2026-08-05)对齐了实现过程中新增的模块与设计变更:BAB 精确搜索路径、manifest 类型/ID 推导、上下文感知元素类型、属性级 refType / Poid 局部引用(`id` 定义点)、精确跳转范围、嵌套 `xi:include`、注入式语法高亮、bit-flag 列表补全(空格触发 / 排除已用 / 追加模式)、simple-content 元素文本引用(补全 / hover / 跳转 / 诊断 / Find All References、语义引用索引 / CodeLens 引用计数 / 未引用资产命令等。
## 一、调研结论(带证据)
@@ -86,9 +86,10 @@ src/
manifestParser.ts .manifest 二进制解析 + 类型/ID 推导(纯 TS)
fileScanner.ts 目录遍历缓存 + Include source 候选收集
refs.ts 引用目标解析(属性 + 元素文本内容,按 refType / isRef /
inheritFrom 过滤,纯 TS
inheritFrom 过滤,纯 TS+ “设计上可被引用类型”判定
referenceIndex.ts 引用记录 → 反向引用索引(定义 → 引用位置)+ 未引用报告
shallowScan.ts 大体积美术资产(.w3x 等)顶层浅扫描(纯 TS,不建 DOM)
records.ts 每文件紧凑索引记录(资产/Define/Include/xi + 行号)
records.ts 每文件紧凑索引记录(资产/Define/Include/xi/引用 + 行号偏移
caches.ts 跨重建持久缓存(DocumentCache / IndexRecordsCache /
IncludeResolveCache+ 失效纪元 InvalidationsEpoch
diskCache.ts 跨会话磁盘缓存(gzip JSON、原子写、多信号 stat 校验)
@@ -99,6 +100,9 @@ src/
completion.ts 补全 provider(元素/属性/值,上下文感知;xs:list 多值按当前段过滤)
hover.ts hover provider
navigation.ts 定义/引用/文档链接/大纲
references.ts 语义 FAR / 引用上下文 / CodeLens 命令共享逻辑
codeLens.ts CodeLens 引用计数(类型过滤,0 也显示,点击开 references peek
unreferenced.ts 未引用资产 QuickPick 命令 + 右键菜单入口
diagnostics.ts 实时诊断
semanticTokens.ts 语义 token provider(文档有解析错误时接管着色)
syntaxes/
@@ -108,9 +112,10 @@ tools/
extract-asset-types.mjs OpenSAGE AssetType.cs → asset-types.json
test/
fixtures/minimod 样例 Modinclude 各种情形、同名 ID、嵌套 xi:include、manifest 回退)
*.test.mjs 11 个测试文件(xmlParser / context / completion / semanticTokens /
*.test.mjs 14 个测试文件(xmlParser / context / completion / semanticTokens /
includeResolver / manifestParser / indexer / schemaModel / refs /
typeContext / manifestTypes
typeContext / manifestTypes / referenceIndex / codeLens /
referenceProvider
```
### 关键设计决策
@@ -251,6 +256,27 @@ test/
`<` 做过滤前缀导致菜单为空——改为保留已输入的 `<`、range 从 `<` 之后
开始、插入文本不带开括号;`textContentTokenAt` 对未闭合元素用 `el.end`
作内容边界。
28. **语义引用索引 + CodeLens 计数 + 未引用资产(第十八轮,2026-08-05)**
- 动机与结论:292 个有模型类型的顶层资产中 191 个在 XSD 中没有任何
类型化引用指向(设置/地图元数据/w3x 子结构等自动注册类型),
AttachTest 实测 29% 项目定义 0 引用、48% 恰好 1 引用。因此 CodeLens
计数只显示在 `isReferenceTargetType()` 为真的类型上(类型化引用目标 +
`inheritFrom` 可继承类型),0 也显示;未引用报告默认也只列这些类型。
- 数据:`IndexRecords` 增加 `references[]`
`kind / refType / selfType / value / line / start / end`),解析期固化
引用上下文,反向索引构建零 DOM、零上下文重解析;`ModIndex.references`
为“定义 key → 引用位置”的反向表,快照发布时构建;磁盘缓存版本
v1 → v2`index-records-v2.json.gz`),v3 起附加内容/records 哈希。
- FAR 从全文搜索改为语义索引:不再把 `id="X"` 定义行、`EditorName="X"`
等非引用属性计为引用;CodeLens 显示的计数与点击打开的 references peek
严格一致;`includeDeclaration` 时才附加定义位置。
- 未引用资产:命令面板 `Find unreferenced assets…` + 编辑器右键菜单
`Find unreferenced assets of this type`;只统计 `origin === "project"`
且非 `viaInstance` 的定义;覆盖原版 id 时来自原版/其他流的引用计入。
- 边界:局部 scope(不在全局流里的文件)的引用不入全局反向索引;
`AssetIdList` 等列表引用按整值记录;w3x 不提取引用记录(对 w3x 资产的
引用从引用它的 XML 捕获);manifest 编译期引用图(AssetReference 缓冲)
留作后续。设计文档:`docs/features-reference-counts.md`
## 三、实施步骤
@@ -308,6 +334,24 @@ test/
id/define/local/include 候选返回 `isIncomplete` 让 VS Code 随输入重请求;
当前文档 local 资产优先、候选 top-N 用堆避免全量排序、Include source
先排序再截断;测试 128 → 136。
21. [x] 语义引用索引 + CodeLens + 未引用资产(第十八轮,2026-08-05):
records 引用记录提取、`referenceIndex.ts` 反向索引、语义 FAR(替换全文
搜索)、CodeLens 引用计数(类型过滤、0 也显示、点击开 references peek)、
`findUnreferencedAssets` 命令 + 右键菜单、磁盘缓存 v2;
测试 136 → 146;设计文档 `docs/features-reference-counts.md`
22. [x] FAR 定义行排除 + manifest 源引用归并(2026-08-05 修复轮):
FAR 不再因 `includeDeclaration` 附加定义行(消除“自引用”观感);
`referenceSitesForDefinition` 把 manifestSource 可解析到当前文件的
manifest 定义站点并入 CodeLens 计数(AttachTest 326 个 manifest 定义
受益);测试 146 → 147;版本 0.1.17;分析见
`docs/analysis-issues.md` 二十三。
23. [x] 引用索引与缓存同步加固(2026-08-06):反向索引改为从**构建期本地
records**`buildRecords`)构建,不再读共享 recordsCache——中途失效、
feature `readDom` 重读都不再造成“资产在但引用缺失”;构建中 `readDom`
不再被定义跳转调用(避免污染进行中的构建);force 重建(Re-index
对 stat 匹配的 full XML 做内容哈希校验;打开文档时比较 records 哈希,
不一致则定向 invalidate + `records-desync` 重建自愈;磁盘缓存 v2 → v3;
测试 147 → 151;分析见 `docs/analysis-issues.md` 二十四。
## 四、验证结果(实测)
@@ -332,7 +376,11 @@ test/
模块 `id` 定义点、Poid 局部引用、`xi:include` 不校验、`Side="Allies"` 命中
manifest 的 `PlayerTemplate`)、simple-content 文本引用(`<` 后补全不产生
`<<``<CreateObject>$1</CreateObject>` 片段、内容值按 refType 过滤、
内容 hover / Ctrl 跳转 / 诊断)
内容 hover / Ctrl 跳转 / 诊断)、引用索引(records 引用提取:attr/content/
inheritFrom、排除 id/Poid/`$DEFINE`/枚举;`buildReferenceIndex` 严格类型过滤
不串同名 IDminimod 集成:`CommandSet` / `inheritFrom` 反向命中;未引用过滤
只含 project 非 viaInstanceCodeLens 类型过滤与 0 显示;语义 FAR 排除定义
行与非引用属性)。
> 注:D: 盘移动硬盘已恢复连接;Corona 已在第八 / 九轮按上述新数据回归。
+12
View File
@@ -86,9 +86,21 @@ XML 之间的组织靠 `<Include>` 标签,共有三种语义:
美术资产目录(Corona 约 2.6 GB);索引记录与 include 解析结果同样跨重建缓存,
保存触发的重建零 stat、零重读(Corona 实测约 2 秒)。
**补充(引用计数、语义 Find All References 与未引用资产,2026-08-05**
- 每个顶部 asset 显示被引用次数(CodeLens,0 也显示),点击直接打开
references peek;只在“设计上应被引用”的类型上显示,避免设置类/地图元数据/
w3x 子结构等自动注册类型制造满屏 0;
- Find All References 改为基于语义引用索引(属性引用 + simple-content 文本 +
`inheritFrom`,排除 id 定义点 / Poid / `$DEFINE`),不再全文搜索;
- 命令 `Find unreferenced assets…`:按类型列出所有零引用的项目定义并跳转;
编辑器右键菜单 `Find unreferenced assets of this type` 预选光标所在类型;
- 设计文档见 `docs/features-reference-counts.md`
### P1:非近期目标(本期不做,但预留扩展点)
6. **高效搜索**Mod 项目巨大(Corona 约 7500 个 XML、38MB)时直接全文搜索很慢,需要一个高效的 XML 内容索引机制。
(2026-08-05:语义引用索引已落地,FAR / 引用计数 / 未引用报告不再全文搜索;
通用内容搜索与索引复用仍属远期。)
7. **索引机制的复用性**:希望索引不仅能服务 VS Code 插件,也能被其他工具(如搜索、静态分析)复用,因此索引/解析核心应设计成与编辑器无关的纯模块。
## 三、环境与参考资源
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "ra3-mod-xml",
"version": "0.1.9",
"version": "0.1.18",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "ra3-mod-xml",
"version": "0.1.9",
"version": "0.1.18",
"license": "MIT",
"dependencies": {
"fast-xml-parser": "^4.5.0"
+18 -1
View File
@@ -2,7 +2,7 @@
"name": "ra3-mod-xml",
"displayName": "RA3 Mod XML",
"description": "Red Alert 3 Mod XML tooling: syntax highlighting, completions, reference navigation and diagnostics for SAGE/BinaryAssetBuilder XML.",
"version": "0.1.15",
"version": "0.1.18",
"publisher": "ra3-mod-xml",
"license": "MIT",
"engines": {
@@ -100,6 +100,23 @@
{
"command": "ra3modxml.showCacheReport",
"title": "RA3 Mod XML: Show cache report"
},
{
"command": "ra3modxml.findUnreferencedAssets",
"title": "RA3 Mod XML: Find unreferenced assets…"
},
{
"command": "ra3modxml.findUnreferencedAssetsOfType",
"title": "RA3 Mod XML: Find unreferenced assets of this type"
}
]
},
"menus": {
"editor/context": [
{
"command": "ra3modxml.findUnreferencedAssetsOfType",
"when": "editorLangId == xml && ra3modxml.active",
"group": "navigation@50"
}
]
},
+33
View File
@@ -8,6 +8,12 @@ import {
Ra3DocumentSymbolProvider,
Ra3ReferenceProvider,
} from "./features/navigation";
import { Ra3CodeLensProvider } from "./features/codeLens";
import { showReferencesForDef } from "./features/references";
import {
findUnreferencedAssets,
findUnreferencedAssetsOfType,
} from "./features/unreferenced";
import { Ra3Diagnostics } from "./features/diagnostics";
import {
Ra3SemanticTokensProvider,
@@ -60,6 +66,12 @@ export function activate(context: vscode.ExtensionContext): void {
new Ra3DocumentSymbolProvider(ws),
),
);
context.subscriptions.push(
vscode.languages.registerCodeLensProvider(
XML_SELECTOR,
new Ra3CodeLensProvider(ws),
),
);
context.subscriptions.push(
vscode.languages.registerDocumentSemanticTokensProvider(
XML_SELECTOR,
@@ -73,6 +85,7 @@ export function activate(context: vscode.ExtensionContext): void {
// Refresh diagnostics for every open XML document whenever a new index
// snapshot is published (XML phase, art phase, stale/final rebuild).
ws.onIndexUpdate = () => {
void vscode.commands.executeCommand("editor.action.codeLens.refresh");
for (const doc of vscode.workspace.textDocuments) {
if (doc.languageId === "xml") void diagnostics.update(doc);
}
@@ -177,6 +190,7 @@ export function activate(context: vscode.ExtensionContext): void {
`Project: ${s.projectDir}\n` +
`Files: ${s.indexedFiles} (${s.parsedFiles} parsed, ${s.shallowScannedFiles} shallow-scanned, ${s.shallowCacheHits + s.recordsCacheHits} cache hits)\n` +
`Assets: ${s.assetCount} (${s.manifestAssetCount} from ${s.manifestFiles} manifests)\n` +
`References: ${s.referenceCount}\n` +
`Defines: ${s.defineCount} · Streams: ${s.streams} · Candidates: ${s.sourceCandidates}\n` +
`Phase: ${s.phase} · Complete: ${s.complete}${stale}\n` +
`Build #${ws.buildCount} (trigger: ${ws.lastTrigger})\n` +
@@ -186,6 +200,25 @@ export function activate(context: vscode.ExtensionContext): void {
);
}),
);
context.subscriptions.push(
vscode.commands.registerCommand(
"ra3modxml.showReferences",
(args: Parameters<typeof showReferencesForDef>[1]) =>
void showReferencesForDef(ws, args),
),
);
context.subscriptions.push(
vscode.commands.registerCommand(
"ra3modxml.findUnreferencedAssets",
() => void findUnreferencedAssets(ws),
),
);
context.subscriptions.push(
vscode.commands.registerCommand(
"ra3modxml.findUnreferencedAssetsOfType",
() => void findUnreferencedAssetsOfType(ws),
),
);
void ws.initialize();
}
+90
View File
@@ -0,0 +1,90 @@
import * as vscode from "vscode";
import { LineMap, parseXml } from "../language/xmlParser";
import { resolveElementType } from "../language/typeContext";
import { isReferenceTargetType } from "../indexer/refs";
import {
referenceSitesForDefinition,
scheduleRebuildIfRecordsDesync,
} from "../indexer/referenceIndex";
import type { ShowReferencesArgs } from "./references";
import type { ModWorkspace } from "../workspace";
/** Never build a DOM for huge files just to show counts (w3x safety). */
const MAX_CODELENS_TEXT = 4 * 1024 * 1024;
/**
* CodeLens reference counts on top-level assets.
*
* Only types that are reference targets by design get a lens (settings, map
* metadata, w3x sub-assets etc. would otherwise show a permanent, misleading
* "0 references"). Zero is still shown for the meaningful types: that is the
* signal users can click to inspect an unused asset.
*/
export class Ra3CodeLensProvider implements vscode.CodeLensProvider {
constructor(private ws: ModWorkspace) {}
provideCodeLenses(
document: vscode.TextDocument,
_token: vscode.CancellationToken,
): vscode.CodeLens[] {
if (!this.ws.isRa3Workspace()) return [];
const idx = this.ws.index;
if (!idx) return [];
const text = document.getText();
if (text.length > MAX_CODELENS_TEXT) return [];
scheduleRebuildIfRecordsDesync(this.ws, document);
const doc = parseXml(text);
const root = doc.root;
if (!root) return [];
const lineMap = new LineMap(text);
const lenses: vscode.CodeLens[] = [];
for (const child of root.children) {
const local = localName(child.name);
if (local === "Tags" || local === "Includes" || local === "Defines") continue;
const idAttr = child.attrs.find((a) => a.name === "id");
if (!idAttr?.hasValue) continue;
const elType = resolveElementType(child);
if (!isReferenceTargetType(elType)) continue;
const id = idAttr.value;
const line = lineMap.positionAt(idAttr.valueStart).line + 1;
const count = referenceSitesForDefinition(idx, {
type: local,
id,
file: document.uri.fsPath,
line,
}).length;
const range = new vscode.Range(
document.positionAt(child.start),
document.positionAt(child.startTagEnd),
);
const args: ShowReferencesArgs = {
uri: document.uri,
position: document.positionAt(idAttr.valueStart),
id,
type: local,
file: document.uri.fsPath,
line,
};
lenses.push(
new vscode.CodeLens(range, {
title:
count === 0
? "0 references"
: count === 1
? "1 reference"
: `${count} references`,
command: "ra3modxml.showReferences",
arguments: [args],
}),
);
}
return lenses;
}
}
function localName(tag: string): string {
const idx = tag.lastIndexOf(":");
return idx >= 0 ? tag.slice(idx + 1) : tag;
}
+14 -87
View File
@@ -14,6 +14,7 @@ import {
resolveReferenceTargetsForType,
type ReferenceTarget,
} from "../indexer/refs";
import { findReferenceLocations } from "./references";
import {
findContainingGameObject,
findLocalId,
@@ -163,6 +164,18 @@ async function assetDefLocation(
scope: DocumentScope,
currentDocument: vscode.TextDocument,
): Promise<vscode.Location | null> {
// While a rebuild is running, avoid readDom() mutating the live indexer's
// caches mid-build; a line-based location is a fine temporary fallback.
if (ws.isBuilding) {
const line = Math.max(0, def.line - 1);
return new vscode.Location(
vscode.Uri.file(def.file),
new vscode.Range(
new vscode.Position(line, 0),
new vscode.Position(line, 1),
),
);
}
if (def.origin === "manifest") {
const src = def.manifestSource;
if (src?.toUpperCase().startsWith("DATA:")) {
@@ -278,56 +291,7 @@ export class Ra3ReferenceProvider implements vscode.ReferenceProvider {
_context: vscode.ReferenceContext,
_token: vscode.CancellationToken,
): Promise<vscode.Location[] | null> {
if (!this.ws.isRa3Workspace()) return null;
const text = document.getText();
const offset = document.offsetAt(position);
const doc = parseXml(text);
const el = findElementAt(doc, offset);
if (!el) return null;
const attr = el.attrs.find(
(a) =>
(a.hasValue && offset >= a.valueStart && offset <= a.valueEnd) ||
(offset >= a.nameStart && offset <= a.nameEnd),
);
let id: string | null = null;
if (attr?.hasValue) {
id = attr.value;
} else {
// Element text content (e.g. <CreateObject>CrateDebris_01</CreateObject>).
const elType = resolveElementType(el);
const token = textContentTokenAt(text, el, offset);
if (token && isReferenceContentType(elType) && !token.value.startsWith("$")) {
id = token.value;
}
}
if (!id || id.startsWith("$")) return null;
const locations: vscode.Location[] = [];
// Matches both attribute values ("id" / 'id') and simple-content
// references (>id<); the outer delimiters are stripped from the result
// range below so the returned locations cover just the id.
const pattern = `(?:["']|>)[ \\t]*${escapeRegExp(id)}[ \\t]*(?:["']|<)`;
await findTextInWorkspace(
{ pattern, isRegExp: true },
{ include: "**/*.xml", maxResults: 2000 },
(result: { uri: vscode.Uri; matches: { range: vscode.Range }[] }) => {
if (!result.uri) return;
for (const m of result.matches) {
const start = m.range.start;
const end = m.range.end;
locations.push(
new vscode.Location(
result.uri,
new vscode.Range(
new vscode.Position(start.line, start.character + 1),
new vscode.Position(end.line, end.character - 1),
),
),
);
}
},
);
return locations.length ? locations : null;
return findReferenceLocations(this.ws, document, position);
}
}
@@ -435,40 +399,3 @@ function localName(tag: string): string {
const idx = tag.lastIndexOf(":");
return idx >= 0 ? tag.slice(idx + 1) : tag;
}
function escapeRegExp(s: string): string {
return s.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
}
/**
* `workspace.findTextInFiles` is a stable VS Code API (since 1.66) but is
* missing from the published typings, so we declare the subset we need and
* call it via a safe cast.
*/
interface TextSearchQuery {
pattern: string;
isRegExp?: boolean;
isCaseSensitive?: boolean;
isWordMatch?: boolean;
}
interface TextSearchOptions {
include?: string;
exclude?: string;
maxResults?: number;
}
function findTextInWorkspace(
query: TextSearchQuery,
options: TextSearchOptions,
callback: (result: { uri: vscode.Uri; matches: { range: vscode.Range }[] }) => void,
): Promise<void> {
const api = vscode.workspace as unknown as {
findTextInFiles(
query: TextSearchQuery,
options: TextSearchOptions,
callback: (result: { uri: vscode.Uri; matches: { range: vscode.Range }[] }) => void,
): Promise<unknown>;
};
return api.findTextInFiles(query, options, callback).then(() => undefined);
}
+227
View File
@@ -0,0 +1,227 @@
/**
* Shared semantic reference logic used by Find All References, the CodeLens
* "N references" command and (indirectly) the unreferenced-assets report.
*
* Unlike the old text-search implementation, every result here comes from
* the reverse reference index built during indexing, so the count shown on a
* top-level asset always matches the references peek opened by clicking it.
*/
import * as vscode from "vscode";
import {
findElementAt,
parseXml,
textContentTokenAt,
} from "../language/xmlParser";
import { resolveElementType } from "../language/typeContext";
import { attributesOfType, typeInfo } from "../model/schemaModel";
import {
filterAndScoreDefs,
isReferenceAttributeOfType,
isReferenceContentType,
mergeLocalAndGlobalDefs,
} from "../indexer/refs";
import {
referenceSitesForDef,
scheduleRebuildIfRecordsDesync,
} from "../indexer/referenceIndex";
import type { AssetDef, ModIndex, ReferenceSite } from "../indexer/types";
import type { ModWorkspace } from "../workspace";
export interface ReferenceContext {
id: string;
/** XSD refType when the cursor is on a typed reference; null otherwise. */
refType: string | null;
/** Element type for inheritFrom filtering; null otherwise. */
selfType: string | null;
}
/**
* Extracts the referenced id and its XSD context from the cursor position.
* Works on reference attribute values/names, simple-content text and on an
* asset's own `id` definition (where every same-id definition is a target).
*/
export function referenceContextAt(
document: vscode.TextDocument,
offset: number,
): ReferenceContext | null {
const text = document.getText();
const doc = parseXml(text);
const el = findElementAt(doc, offset);
if (!el) return null;
const elType = resolveElementType(el);
const attr = el.attrs.find(
(a) =>
(a.hasValue && offset >= a.valueStart && offset <= a.valueEnd) ||
(offset >= a.nameStart && offset <= a.nameEnd),
);
if (attr?.hasValue) {
const value = attr.value;
if (!value || value.startsWith("$") || value.startsWith("=")) return null;
const nameLower = attr.name.toLowerCase();
if (nameLower === "id") {
return { id: value, refType: null, selfType: null };
}
if (!isReferenceAttributeOfType(elType, attr.name)) return null;
if (nameLower === "inheritfrom") {
return { id: value, refType: null, selfType: elType };
}
const attrInfo = elType
? attributesOfType(elType).find((a) => a.name === attr.name)
: undefined;
return { id: value, refType: attrInfo?.refType ?? null, selfType: null };
}
if (elType && isReferenceContentType(elType)) {
const token = textContentTokenAt(text, el, offset);
if (token && !token.value.startsWith("$") && !token.value.startsWith("=")) {
const info = typeInfo(elType);
return {
id: token.value,
refType: info?.kind === "simple" ? info.refType : null,
selfType: null,
};
}
}
return null;
}
/** Definitions matching a reference context (strict type filtering). */
export function definitionsForReference(
idx: ModIndex,
ctx: ReferenceContext,
): AssetDef[] {
const defs = mergeLocalAndGlobalDefs(
idx.local?.assetsById.get(ctx.id.toLowerCase()),
idx.assetsById.get(ctx.id.toLowerCase()),
);
return filterAndScoreDefs(defs, ctx.refType, ctx.selfType).map((t) => t.def);
}
/** Union of reference sites for a set of definitions, de-duplicated. */
export function collectReferenceSites(
idx: ModIndex,
defs: readonly AssetDef[],
): ReferenceSite[] {
const sites: ReferenceSite[] = [];
const seen = new Set<string>();
for (const def of defs) {
for (const site of referenceSitesForDef(idx, def)) {
const key = `${site.file}\u0000${site.start}\u0000${site.end}\u0000${site.kind}`;
if (seen.has(key)) continue;
seen.add(key);
sites.push(site);
}
}
return sites;
}
/** Converts stored offsets to precise editor locations (fallback: line). */
export async function sitesToLocations(
ws: ModWorkspace,
sites: readonly ReferenceSite[],
): Promise<vscode.Location[]> {
const byFile = new Map<string, ReferenceSite[]>();
for (const site of sites) {
let list = byFile.get(site.file);
if (!list) {
list = [];
byFile.set(site.file, list);
}
list.push(site);
}
const locations: vscode.Location[] = [];
for (const [file, fileSites] of byFile) {
const parsed = await ws.indexer?.readDom(file);
const lineMap = parsed?.lineMap ?? null;
for (const site of fileSites) {
if (lineMap) {
const start = lineMap.positionAt(site.start);
const end = lineMap.positionAt(site.end);
locations.push(
new vscode.Location(
vscode.Uri.file(file),
new vscode.Range(
new vscode.Position(start.line, start.character),
new vscode.Position(end.line, end.character),
),
),
);
} else {
const line = Math.max(0, site.line - 1);
locations.push(
new vscode.Location(
vscode.Uri.file(file),
new vscode.Range(
new vscode.Position(line, 0),
new vscode.Position(line, 1),
),
),
);
}
}
}
return locations;
}
/**
* Semantic Find All References (used by the reference provider).
*
* Only real reference sites are returned — the asset's own `id` definition is
* never included, regardless of VS Code's `includeDeclaration` flag, so the
* result set matches the CodeLens reference count exactly.
*/
export async function findReferenceLocations(
ws: ModWorkspace,
document: vscode.TextDocument,
position: vscode.Position,
): Promise<vscode.Location[] | null> {
if (!ws.isRa3Workspace()) return null;
scheduleRebuildIfRecordsDesync(ws, document);
const scope = await ws.getScope(document);
const idx = scope.merged;
if (!idx) return null;
const ctx = referenceContextAt(document, document.offsetAt(position));
if (!ctx) return null;
const defs = definitionsForReference(idx, ctx);
if (!defs.length) return null;
const locations = await sitesToLocations(ws, collectReferenceSites(idx, defs));
return locations.length ? locations : null;
}
/** Arguments passed from the CodeLens to the showReferences command. */
export interface ShowReferencesArgs {
uri: vscode.Uri;
position: vscode.Position;
id: string;
type: string;
file: string;
line: number;
}
/** Opens the references peek for one specific asset definition. */
export async function showReferencesForDef(
ws: ModWorkspace,
args: ShowReferencesArgs,
): Promise<void> {
const idx = ws.index;
if (!idx) return;
const def: AssetDef = {
type: args.type,
id: args.id,
file: args.file,
line: args.line,
origin: "project",
};
const sites = referenceSitesForDef(idx, def);
const locations = await sitesToLocations(ws, sites);
await vscode.commands.executeCommand(
"editor.action.showReferences",
args.uri,
args.position,
locations,
);
}
+124
View File
@@ -0,0 +1,124 @@
import * as vscode from "vscode";
import { relative } from "node:path";
import { findElementAt, parseXml } from "../language/xmlParser";
import { unreferencedByType } from "../indexer/referenceIndex";
import type { ModWorkspace } from "../workspace";
interface TypePickItem extends vscode.QuickPickItem {
type: string;
}
interface AssetPickItem extends vscode.QuickPickItem {
file: string;
line: number;
}
/**
* Palette command: pick an asset type, then jump to any project asset of
* that type that has zero incoming references. Only types that are reference
* targets by design are offered, so auto-registered data (settings, map
* metadata, w3x sub-assets...) is not reported as "unused".
*/
export async function findUnreferencedAssets(
ws: ModWorkspace,
args?: { type?: string },
): Promise<void> {
if (!ws.isRa3Workspace() || !ws.index) {
void vscode.window.showInformationMessage(
"RA3 Mod XML: no index available yet.",
);
return;
}
const idx = ws.index;
const byType = unreferencedByType(idx);
let type = args?.type;
if (!type) {
if (!byType.size) {
void vscode.window.showInformationMessage(
"RA3 Mod XML: no unreferenced assets found.",
);
return;
}
const pickedType = await vscode.window.showQuickPick<TypePickItem>(
[...byType.entries()].map(([t, defs]) => ({
label: t,
description: `${defs.length} unreferenced`,
type: t,
})),
{
placeHolder: "Select an asset type",
matchOnDescription: true,
},
);
if (!pickedType) return;
type = pickedType.type;
}
const defs = byType.get(type) ?? [];
if (!defs.length) {
void vscode.window.showInformationMessage(
`RA3 Mod XML: no unreferenced ${type} assets found.`,
);
return;
}
const pickedAsset = await vscode.window.showQuickPick<AssetPickItem>(
defs.map((d) => ({
label: d.id,
description: `${displayPath(idx.projectDir, d.file)}:${d.line}`,
file: d.file,
line: d.line,
})),
{
placeHolder: `${type}: ${defs.length} unreferenced`,
matchOnDescription: true,
},
);
if (!pickedAsset) return;
const uri = vscode.Uri.file(pickedAsset.file);
const document = await vscode.workspace.openTextDocument(uri);
const line = Math.max(0, pickedAsset.line - 1);
await vscode.window.showTextDocument(document, {
selection: new vscode.Range(
new vscode.Position(line, 0),
new vscode.Position(line, 1),
),
preview: true,
});
}
/**
* Editor context-menu entry: pre-selects the asset type under the cursor.
* Falls back to the type picker when the cursor is not on a top-level asset.
*/
export async function findUnreferencedAssetsOfType(
ws: ModWorkspace,
): Promise<void> {
const editor = vscode.window.activeTextEditor;
if (editor && ws.isRa3Workspace()) {
const document = editor.document;
const offset = document.offsetAt(editor.selection.active);
const doc = parseXml(document.getText());
const el = findElementAt(doc, offset);
if (el && el.parent === doc.root) {
const local = localName(el.name);
const isStructural = local === "Tags" || local === "Includes" || local === "Defines";
const hasId = el.attrs.some((a) => a.name === "id" && a.hasValue);
if (!isStructural && hasId) {
return findUnreferencedAssets(ws, { type: local });
}
}
}
return findUnreferencedAssets(ws);
}
function localName(tag: string): string {
const idx = tag.lastIndexOf(":");
return idx >= 0 ? tag.slice(idx + 1) : tag;
}
function displayPath(projectDir: string, file: string): string {
const rel = relative(projectDir, file);
return rel && !rel.startsWith("..") ? rel : file;
}
+29
View File
@@ -10,6 +10,7 @@
*/
import { resolve } from "node:path";
import { createHash } from "node:crypto";
import type { IndexedFile, ParsedFile } from "./types";
import type { IndexRecords } from "./records";
import type { ResolveResult } from "./includeResolver";
@@ -19,6 +20,28 @@ export function normKey(path: string): string {
return resolve(path).toLowerCase();
}
/**
* SHA-1 of the BOM-stripped text a file's records were extracted from.
* Lets force rebuilds verify that a stat-matching cache entry is not stale
* (FAT32/exFAT timestamps can match after a rewrite), and lets features
* detect "open document != indexed snapshot" without reading the whole index.
*/
export function contentHash(text: string): string {
return createHash("sha1").update(text, "utf8").digest("hex");
}
/**
* Hash of a file's compact index records. Semantic records (assets, defines,
* includes, references) are what the index actually consumes, so comparing
* records hashes ignores cosmetic text changes (line endings, whitespace)
* and only fires the self-heal when the index would really be out of date.
*/
export function recordsHash(records: IndexRecords): string {
return createHash("sha1")
.update(JSON.stringify(records), "utf8")
.digest("hex");
}
/**
* LRU cache for fully parsed XML documents.
*
@@ -119,6 +142,12 @@ export interface IndexRecordsCacheEntry {
records: IndexRecords;
/** "shallow" for art-asset scans (.w3x), "full" for parsed XML. */
kind: "shallow" | "full";
/**
* Hash of the BOM-stripped file text (full parses only). Absent for
* shallow scans (avoid hashing multi-MB model files) and for cache entries
* produced before this field existed.
*/
contentHash?: string;
}
/**
+11 -1
View File
@@ -34,7 +34,14 @@ import type { IndexedFile } from "./types";
const gzipAsync = promisify(gzip);
const gunzipAsync = promisify(gunzip);
export const DISK_CACHE_VERSION = 1;
/**
* v2: per-file records now carry typed reference records (`references`),
* so caches produced by v1 (assets/defines/includes only) are stale.
* v3: full XML records carry `contentHash`, and snapshots publish per-file
* `recordsHashes` for the desync self-heal; caches without hashes cannot be
* verified, so v2 files are regenerated once.
*/
export const DISK_CACHE_VERSION = 3;
/** How many stat validations run concurrently on load. */
const VALIDATE_CONCURRENCY = 32;
@@ -53,6 +60,8 @@ export interface DiskCacheRecord {
stat: NonNullable<IndexedFile["stat"]>;
records: IndexRecords;
kind: "full" | "shallow";
/** Content hash for full XML parses (see `IndexRecordsCacheEntry`). */
contentHash?: string;
}
interface DiskCacheFile {
@@ -173,6 +182,7 @@ export class DiskRecordsCache {
stat: entry.stat,
records: entry.records,
kind: entry.kind,
contentHash: entry.contentHash,
});
}
const payload: DiskCacheFile = {
+120 -43
View File
@@ -40,14 +40,26 @@ import {
} from "./manifestParser";
import { canonicalTypeName } from "../model/schemaModel";
import { collectSourceCandidates } from "./fileScanner";
import { DocumentCache, IncludeResolveCache, IndexRecordsCache, normKey } from "./caches";
import {
contentHash,
DocumentCache,
IncludeResolveCache,
IndexRecordsCache,
normKey,
recordsHash,
} from "./caches";
import type { IndexRecordsCacheEntry } from "./caches";
import { scanXmlShallow } from "./shallowScan";
import {
extractIndexRecords,
recordsFromShallow,
type IndexRecords,
type IndexRecordXi,
} from "./records";
import {
buildReferenceIndex,
type ReferenceRecordSource,
} from "./referenceIndex";
import type {
AssetDef,
DefineDef,
@@ -55,6 +67,7 @@ import type {
IndexedFile,
ModIndex,
ParsedFile,
ReferenceSite,
SourceCandidate,
StreamInfo,
} from "./types";
@@ -110,6 +123,16 @@ export class ModIndexer {
private assetsById = new Map<string, AssetDef[]>();
private defines = new Map<string, DefineDef[]>();
private files = new Map<string, IndexedFile>();
/**
* Records exactly as the current build's walk saw them (path -> records +
* content hash). The reverse reference index is built from this map, never
* from the shared records cache, so a watcher invalidation or a feature
* re-read (readDom) mid-build cannot desync references from assets.
*/
private buildRecords = new Map<
string,
{ file: string; records: IndexRecords; recordsHash: string }
>();
private streams: StreamInfo[] = [];
private manifests = new Map<string, ManifestInfo>();
private sourceCandidates: SourceCandidate[] = [];
@@ -171,6 +194,20 @@ export class ModIndexer {
rec.stat.birthtimeMs === st.birthtimeMs &&
rec.stat.ctimeMs === st.ctimeMs
) {
// Force rebuilds (Re-index workspace) verify full-XML content even
// when every stat signal matches: external drives (FAT32/exFAT) can
// rewrite a file with the same size and coarse timestamps.
if (
this.opts.trustUnchanged === false &&
rec.kind === "full" &&
rec.contentHash
) {
const text = stripBom(await readFile(path, "utf8"));
if (contentHash(text) === rec.contentHash) {
return this.recordsParsed(path, rec);
}
return this.parseFullXml(path, st, text);
}
return this.recordsParsed(path, rec);
}
const hit = this.docs.get(key);
@@ -235,27 +272,7 @@ export class ModIndexer {
return parsed;
}
const text = stripBom(await readFile(path, "utf8"));
const lineMap = new LineMap(text);
const parse = parseXml(text);
const records = extractIndexRecords(parse, lineMap);
const parsed: ParsedFile = {
file: {
path: resolve(path),
stat: {
mtimeMs: st.mtimeMs,
size: st.size,
birthtimeMs: st.birthtimeMs,
ctimeMs: st.ctimeMs,
},
},
parse,
records,
lineMap,
};
this.docs.set(parsed);
this.recordsCache.set(key, { stat: parsed.file.stat, records, kind: "full" });
this.files.set(key, parsed.file);
return parsed;
return this.parseFullXml(path, st, text);
} catch {
const parsed: ParsedFile = {
file: { path: resolve(path), stat: null },
@@ -313,6 +330,40 @@ export class ModIndexer {
return { file, parse: null, records: entry.records, lineMap: null };
}
/**
* Parses a full XML document from its text, caches records (with a content
* hash) and the DOM, and registers the file in this build.
*/
private parseFullXml(path: string, st: Stats, text: string): ParsedFile {
const key = normKey(path);
const lineMap = new LineMap(text);
const parse = parseXml(text);
const records = extractIndexRecords(parse, lineMap, text);
const parsed: ParsedFile = {
file: {
path: resolve(path),
stat: {
mtimeMs: st.mtimeMs,
size: st.size,
birthtimeMs: st.birthtimeMs,
ctimeMs: st.ctimeMs,
},
},
parse,
records,
lineMap,
};
this.docs.set(parsed);
this.recordsCache.set(key, {
stat: parsed.file.stat,
records,
kind: "full",
contentHash: contentHash(text),
});
this.files.set(key, parsed.file);
return parsed;
}
/**
* Reads a document and guarantees a DOM parse tree. Used for root-level
* <xi:include> xpointer selection (rare) and by the document-local scope
@@ -341,27 +392,7 @@ export class ModIndexer {
}
if (st.size > MAX_PARSE_BYTES) return null;
const text = stripBom(await readFile(path, "utf8"));
const lineMap = new LineMap(text);
const parse = parseXml(text);
const records = extractIndexRecords(parse, lineMap);
const parsed: ParsedFile = {
file: {
path: resolve(path),
stat: {
mtimeMs: st.mtimeMs,
size: st.size,
birthtimeMs: st.birthtimeMs,
ctimeMs: st.ctimeMs,
},
},
parse,
records,
lineMap,
};
this.docs.set(parsed);
this.recordsCache.set(key, { stat: parsed.file.stat, records, kind: "full" });
this.files.set(key, parsed.file);
return parsed;
return this.parseFullXml(path, st, text);
} catch {
return null;
}
@@ -424,6 +455,7 @@ export class ModIndexer {
async build(onPhase?: (index: ModIndex) => void | Promise<void>): Promise<ModIndex> {
const start = Date.now();
this.buildRecords.clear();
// Root list only; directories are listed lazily on first query, so the
// XML phase does not pay an upfront recursive enumeration of the SDK.
this.existence = buildExistenceSnapshot(this.searchPaths);
@@ -561,6 +593,15 @@ export class ModIndexer {
for (const [id, defs] of this.assetsById) assetsById.set(id, defs.slice());
const defines = new Map<string, DefineDef[]>();
for (const [name, defs] of this.defines) defines.set(name, defs.slice());
const references = this.buildReferences();
const referenceCount = [...references.values()].reduce(
(sum, sites) => sum + sites.length,
0,
);
const recordsHashes = new Map<string, string>();
for (const [key, entry] of this.buildRecords) {
recordsHashes.set(key, entry.recordsHash);
}
return {
projectDir: resolve(this.opts.projectDir),
@@ -575,6 +616,8 @@ export class ModIndexer {
manifests: new Map(this.manifests),
sourceCandidates: this.sourceCandidates.slice(),
diagnostics: this.diagnostics.slice(),
references,
recordsHashes,
stats: {
projectDir: resolve(this.opts.projectDir),
sdkDir: resolve(this.opts.sdkDir),
@@ -596,6 +639,7 @@ export class ModIndexer {
walkMs: this.timings.walkMs,
artScanMs: this.timings.artScanMs,
assetCount: [...this.assets.values()].reduce((sum, byId) => sum + byId.size, 0),
referenceCount,
defineCount: this.defines.size,
manifestFiles: this.manifests.size,
manifestAssetCount,
@@ -606,6 +650,38 @@ export class ModIndexer {
};
}
/**
* Resolves the per-file reference records collected during this build
* against the current asset maps. Only files touched by this build are
* included, so stale cache entries for files that left the include graph
* never leak into the reverse index.
*/
private buildReferences(): Map<string, ReferenceSite[]> {
const sources: ReferenceRecordSource[] = [];
for (const { file, records } of this.buildRecords.values()) {
sources.push({ file, records });
}
return buildReferenceIndex(sources, {
assets: this.assets,
assetsById: this.assetsById,
});
}
/**
* Remembers the records exactly as this build saw them (plus the content
* hash when the file was fully parsed / cached with one), so snapshots can
* build references from the same source of truth as the asset maps.
*/
private noteBuildRecords(parsed: ParsedFile): void {
if (!parsed.records) return;
const key = normKey(parsed.file.path);
this.buildRecords.set(key, {
file: parsed.file.path,
records: parsed.records,
recordsHash: recordsHash(parsed.records),
});
}
// ── Include walk ──────────────────────────────────────────────────
private async walk(
@@ -670,6 +746,7 @@ export class ModIndexer {
): Promise<void> {
const records = parsed.records;
if (!records) return;
this.noteBuildRecords(parsed);
const file = parsed.file.path;
const origin = this.originOf(file);
+6 -2
View File
@@ -59,7 +59,7 @@ export async function buildDocumentScope(
const lineMap = new LineMap(text);
const parse = parseXml(text);
const builder = new OverlayBuilder(ctx);
await builder.addEntry(uri, parse, lineMap);
await builder.addEntry(uri, parse, lineMap, text);
const expanded = await expandDocument(uri, parse, {
resolve: (source, currentDir) =>
@@ -115,6 +115,8 @@ export function withLocalOverlay(
manifests: new Map(),
sourceCandidates: [],
diagnostics: [],
references: new Map(),
recordsHashes: new Map(),
stats: {
projectDir,
sdkDir,
@@ -134,6 +136,7 @@ export function withLocalOverlay(
walkMs: 0,
artScanMs: 0,
assetCount: 0,
referenceCount: 0,
defineCount: 0,
manifestFiles: 0,
manifestAssetCount: 0,
@@ -164,12 +167,13 @@ class OverlayBuilder {
path: string,
parse: XmlDocument,
lineMap: LineMap,
text: string,
): Promise<void> {
this.lineMaps.set(scopePathKey(path), lineMap);
await this.addParsed({
file: { path: resolve(path), stat: null },
parse,
records: extractIndexRecords(parse, lineMap),
records: extractIndexRecords(parse, lineMap, text),
lineMap,
}, 0);
}
+119 -3
View File
@@ -12,6 +12,12 @@
import type { LineMap, XmlDocument } from "../language/xmlParser";
import type { ShallowDocument } from "./shallowScan";
import { attributesOfType, typeInfo } from "../model/schemaModel";
import { resolveElementType } from "../language/typeContext";
import {
isReferenceAttributeOfType,
isReferenceContentType,
} from "./refs";
export interface IndexRecordAsset {
/** Top-level element name, e.g. "W3DContainer". */
@@ -42,6 +48,27 @@ export interface IndexRecordXi {
line: number;
}
export interface IndexRecordReference {
/** "attr" for attribute values, "content" for simple-content text. */
kind: "attr" | "content";
/**
* XSD reference target type (from `xas:refType`), or null for untyped
* `isRef` references and `inheritFrom` (which filters by the element's own
* type via `selfType`).
*/
refType: string | null;
/** Element type used by `inheritFrom` filtering; null otherwise. */
selfType: string | null;
/** The referenced id text (whole attribute value / trimmed content). */
value: string;
/** 1-based line of the value. */
line: number;
/** Character offset of the value start (relative to the file text). */
start: number;
/** Character offset one past the value end. */
end: number;
}
export interface IndexRecords {
assets: IndexRecordAsset[];
defines: IndexRecordDefine[];
@@ -50,6 +77,8 @@ export interface IndexRecords {
rootXiIncludes: IndexRecordXi[];
/** <xi:include> elements nested anywhere else in the document. */
nestedXiIncludes: IndexRecordXi[];
/** Typed global-asset references (attribute values + simple content). */
references: IndexRecordReference[];
}
const INCLUDE_TYPES = new Set(["all", "instance", "reference"]);
@@ -69,14 +98,21 @@ function localName(tag: string): string {
* Tags/Includes/Defines), $DEFINE constants, the top-level <Includes> block
* and root/nested <xi:include> elements.
*/
export function extractIndexRecords(parse: XmlDocument, lineMap: LineMap): IndexRecords {
export function extractIndexRecords(
parse: XmlDocument,
lineMap: LineMap,
text: string,
): IndexRecords {
const assets: IndexRecordAsset[] = [];
const defines: IndexRecordDefine[] = [];
const includes: IndexRecordInclude[] = [];
const rootXiIncludes: IndexRecordXi[] = [];
const nestedXiIncludes: IndexRecordXi[] = [];
const references: IndexRecordReference[] = [];
const root = parse.root;
if (!root) return { assets, defines, includes, rootXiIncludes, nestedXiIncludes };
if (!root) {
return { assets, defines, includes, rootXiIncludes, nestedXiIncludes, references };
}
for (const child of root.children) {
const local = localName(child.name);
@@ -142,7 +178,86 @@ export function extractIndexRecords(parse: XmlDocument, lineMap: LineMap): Index
});
}
return { assets, defines, includes, rootXiIncludes, nestedXiIncludes };
collectReferenceRecords(parse, lineMap, text, references);
return { assets, defines, includes, rootXiIncludes, nestedXiIncludes, references };
}
/**
* Walks every element of a fully parsed document and records typed
* global-asset references: reference attributes, `inheritFrom` and
* simple-content reference text. Local `id` definitions, Poid pipeline-local
* references and `$DEFINE`/`=` values are intentionally skipped (the same
* semantics as diagnostics / hover / navigation).
*
* The stored `refType` / `selfType` pair is exactly what
* `resolveReferenceTargetsForType` derives from the element context, so the
* reverse reference index can resolve these records after the whole index is
* built without re-walking the document or re-resolving element types.
*/
function collectReferenceRecords(
parse: XmlDocument,
lineMap: LineMap,
text: string,
out: IndexRecordReference[],
): void {
for (const el of parse.elements) {
const elType = resolveElementType(el);
for (const attr of el.attrs) {
if (!attr.hasValue) continue;
if (!isReferenceAttributeOfType(elType, attr.name)) continue;
const value = attr.value;
if (!value || value.startsWith("$") || value.startsWith("=")) continue;
let refType: string | null = null;
let selfType: string | null = null;
if (attr.name.toLowerCase() === "inheritfrom") {
selfType = elType;
} else if (elType) {
refType =
attributesOfType(elType).find((a) => a.name === attr.name)?.refType ??
null;
}
out.push({
kind: "attr",
refType,
selfType,
value,
line: lineOf(lineMap, attr.valueStart),
start: attr.valueStart,
end: attr.valueEnd,
});
}
if (
elType &&
isReferenceContentType(elType) &&
!el.selfClosing &&
el.closeTagStart >= 0
) {
const raw = text.slice(el.startTagEnd, el.closeTagStart);
const value = raw.trim();
if (
!value ||
value.startsWith("$") ||
value.startsWith("=") ||
value.includes("<")
) {
continue;
}
const start = el.startTagEnd + raw.indexOf(value);
const info = typeInfo(elType);
out.push({
kind: "content",
refType: info?.kind === "simple" ? info.refType : null,
selfType: null,
value,
line: lineOf(lineMap, start),
start,
end: start + value.length,
});
}
}
}
/** Converts a shallow scan (offsets) into index records (1-based lines). */
@@ -173,5 +288,6 @@ export function recordsFromShallow(scan: ShallowDocument, lineMap: LineMap): Ind
xpointer: x.xpointer,
line: lineOf(lineMap, x.start),
})),
references: [],
};
}
+219
View File
@@ -0,0 +1,219 @@
/**
* Reverse reference index built from per-file reference records.
*
* The indexer stores compact reference records per file (attribute values,
* simple-content text and inheritFrom, with XSD `refType`/`selfType` context
* captured at parse time). After the include walk, this module resolves every
* record against the final asset maps and produces:
*
* definition key -> reference sites
*
* which powers CodeLens reference counts, semantic Find All References and
* the "unreferenced assets" report. Pure TypeScript: no vscode dependency.
*/
import { extractIndexRecords, type IndexRecords } from "./records";
import {
filterAndScoreDefs,
isReferenceTargetType,
type ReferenceLookup,
} from "./refs";
import { buildSearchPaths, resolveSource } from "./includeResolver";
import { normKey, recordsHash } from "./caches";
import { LineMap, parseXml } from "../language/xmlParser";
import type { AssetDef, ModIndex, ReferenceSite } from "./types";
/** A file whose reference records should be resolved. */
export interface ReferenceRecordSource {
/** Absolute path of the referencing file. */
file: string;
records: IndexRecords;
}
/** Stable key identifying one specific asset definition. */
export function assetDefKey(
def: Pick<AssetDef, "type" | "id" | "file" | "line">,
): string {
return `${def.type}\u0000${def.id.toLowerCase()}\u0000${def.file.toLowerCase()}\u0000${def.line}`;
}
/**
* Resolves per-file reference records against the asset lookup and returns
* the reverse map. A record resolves to every definition that satisfies its
* `refType` / `selfType` context (same strict filtering as go-to-definition),
* so same-name ids of different types never share reference counts.
*/
export function buildReferenceIndex(
sources: Iterable<ReferenceRecordSource>,
lookup: ReferenceLookup,
): Map<string, ReferenceSite[]> {
const map = new Map<string, ReferenceSite[]>();
for (const { file, records } of sources) {
for (const ref of records.references) {
const defs = lookup.assetsById.get(ref.value.toLowerCase());
if (!defs?.length) continue;
const targets = filterAndScoreDefs(defs, ref.refType, ref.selfType);
for (const target of targets) {
const key = assetDefKey(target.def);
let sites = map.get(key);
if (!sites) {
sites = [];
map.set(key, sites);
}
sites.push({
file,
line: ref.line,
start: ref.start,
end: ref.end,
kind: ref.kind,
});
}
}
}
return map;
}
/** Reference sites for one definition (empty when the index has none). */
export function referenceSitesForDef(
idx: Pick<ModIndex, "references"> | null | undefined,
def: Pick<AssetDef, "type" | "id" | "file" | "line">,
): ReferenceSite[] {
return idx?.references?.get(assetDefKey(def)) ?? [];
}
function normFileKey(path: string): string {
return path.replace(/\\/g, "/").toLowerCase();
}
/**
* Reference sites that belong to a definition opened in the editor.
*
* Besides the definition's own reverse-index bucket, this unions the sites
* of manifest definitions that map back to the same XML source file via
* `manifestSource`. A manifest asset with a resolvable SageXml source is
* semantically the same asset as that XML definition, so references to it
* should show up on the source file's CodeLens too (Find All References
* already sees them because it unions every same-id/type definition).
*/
export function referenceSitesForDefinition(
idx: ModIndex,
def: Pick<AssetDef, "type" | "id" | "file" | "line">,
): ReferenceSite[] {
const sites = referenceSitesForDef(idx, def);
const byId = idx.assets.get(def.type)?.get(def.id.toLowerCase());
if (!byId?.length) return sites;
const defFile = normFileKey(def.file);
const searchPaths = buildSearchPaths(idx.sdkDir, idx.projectDir);
const seen = new Set(
sites.map((s) => `${s.file}\u0000${s.start}\u0000${s.end}\u0000${s.kind}`),
);
for (const other of byId) {
if (other.origin !== "manifest" || !other.manifestSource) continue;
const resolved = resolveSource(other.manifestSource, null, searchPaths).path;
if (!resolved || normFileKey(resolved) !== defFile) continue;
for (const site of referenceSitesForDef(idx, other)) {
const key = `${site.file}\u0000${site.start}\u0000${site.end}\u0000${site.kind}`;
if (seen.has(key)) continue;
seen.add(key);
sites.push(site);
}
}
return sites;
}
export interface UnreferencedOptions {
/**
* When true (default), only report types that are reference targets by
* design. Auto-registered / structural types (settings, map metadata,
* w3x sub-assets...) are excluded because zero references is their normal
* state.
*/
onlyReferenceTargetTypes?: boolean;
}
/**
* Project asset definitions with zero incoming references, grouped by type.
* Manifest / SDK definitions and `instance`-only assets are never reported
* (they are not part of the compiled stream in the same way).
*/
export function unreferencedByType(
idx: ModIndex,
options: UnreferencedOptions = {},
): Map<string, AssetDef[]> {
const onlyReferenceTargets = options.onlyReferenceTargetTypes ?? true;
const out = new Map<string, AssetDef[]>();
for (const [type, byId] of idx.assets) {
if (onlyReferenceTargets && !isReferenceTargetType(type)) continue;
const defs: AssetDef[] = [];
for (const arr of byId.values()) {
for (const def of arr) {
if (def.origin !== "project" || def.viaInstance) continue;
if (referenceSitesForDef(idx, def).length > 0) continue;
defs.push(def);
}
}
if (defs.length) {
defs.sort((a, b) => a.id.localeCompare(b.id));
out.set(type, defs);
}
}
return out;
}
/** Minimal workspace surface needed by the records-desync self-heal. */
export interface RecordsSyncWorkspace {
index: ModIndex | null;
invalidate(path: string): void;
scheduleRebuild(reason: string): void;
}
/** Minimal document surface (vscode.TextDocument subset, no vscode dep). */
export interface RecordsSyncDocument {
uri: { fsPath: string; scheme?: string };
isDirty?: boolean;
getText(): string;
}
/**
* True when a clean (saved) document's text no longer matches the records the
* published snapshot was built from. This catches cache entries that slipped
* through stat validation (e.g. a rewrite with preserved timestamps on an
* external drive) or watcher events lost during a drive reconnect.
*/
export function documentRecordsDesynced(
idx: ModIndex,
fsPath: string,
text: string,
): boolean {
const expected = idx.recordsHashes?.get(normKey(fsPath));
if (expected == null) return false;
const lineMap = new LineMap(text);
const records = extractIndexRecords(parseXml(text), lineMap, text);
return recordsHash(records) !== expected;
}
/**
* Self-heal: when the open, saved document's content differs from the
* snapshot's records hash, invalidate exactly that file and schedule a
* rebuild. Returns true when a rebuild was scheduled. Unsaved (dirty)
* documents are skipped — the editor text is intentionally ahead of disk.
*/
export function scheduleRebuildIfRecordsDesync(
ws: RecordsSyncWorkspace,
document: RecordsSyncDocument,
): boolean {
if (
document.isDirty ||
(document.uri.scheme != null && document.uri.scheme !== "file")
) {
return false;
}
const idx = ws.index;
if (!idx) return false;
const fsPath = document.uri.fsPath;
if (!documentRecordsDesynced(idx, fsPath, document.getText())) return false;
ws.invalidate(fsPath);
ws.scheduleRebuild("records-desync");
return true;
}
+67 -5
View File
@@ -1,17 +1,33 @@
import {
allTypeNames,
attributesOfType,
canonicalTypeName,
elementTypeName,
isAssignableTo,
typeChain,
typeInfo,
} from "../model/schemaModel";
import type { AssetDef, ModIndex } from "./types";
import type { AssetDef, LocalOverlay } from "./types";
export interface ReferenceTarget {
def: AssetDef;
score: number;
}
/**
* The subset of `ModIndex` that reference resolution needs. Kept narrow so
* the reverse reference index can resolve records against the indexer's live
* maps without constructing a full index snapshot.
*/
export interface ReferenceLookup {
/** type -> id -> definitions. */
assets: Map<string, Map<string, AssetDef[]>>;
/** id -> definitions across all types. */
assetsById: Map<string, AssetDef[]>;
/** Optional document-local overlay (consulted first). */
local?: LocalOverlay;
}
/**
* True when an attribute is a "pipeline-local" reference that the global
* asset index cannot judge:
@@ -78,7 +94,7 @@ export function isReferenceAttributeOfType(
* Returns [] when the attribute is not a typed reference or nothing matches.
*/
export function resolveReferenceTargets(
idx: ModIndex,
idx: ReferenceLookup,
elementType: string,
attrName: string,
id: string,
@@ -93,7 +109,7 @@ export function resolveReferenceTargets(
/** Same resolution, driven by a resolved XSD type name. */
export function resolveReferenceTargetsForType(
idx: ModIndex,
idx: ReferenceLookup,
typeName: string | null,
attrName: string,
id: string,
@@ -148,7 +164,7 @@ export function isReferenceContentType(typeName: string | null): boolean {
* (e.g. `GameObjectWeakRef` -> `GameObject`).
*/
export function resolveContentReferenceTargets(
idx: ModIndex,
idx: ReferenceLookup,
typeName: string | null,
id: string,
): ReferenceTarget[] {
@@ -164,7 +180,7 @@ export function resolveContentReferenceTargets(
return filterAndScoreDefs(defs, refType, null);
}
function filterAndScoreDefs(
export function filterAndScoreDefs(
defs: readonly AssetDef[],
refType: string | null,
selfType: string | null,
@@ -204,3 +220,49 @@ export function mergeLocalAndGlobalDefs(
}
return out;
}
let referenceTargetTypeSet: Set<string> | null = null;
/**
* The set of XSD types that are "reference targets by design": at least one
* typed reference attribute / simple-content reference points at them, or
* they are inheritable (`inheritFrom`). Types outside this set are
* auto-registered / structural (settings, map metadata, w3x sub-assets...),
* so a zero reference count is their normal state and counts would only be
* noise.
*/
export function referenceTargetTypes(): ReadonlySet<string> {
if (referenceTargetTypeSet) return referenceTargetTypeSet;
const set = new Set<string>();
const add = (t: string | null) => {
if (!t) return;
set.add(canonicalTypeName(t) ?? t);
};
for (const typeName of allTypeNames()) {
const info = typeInfo(typeName);
if (!info) continue;
if (info.kind === "complex") {
for (const attr of info.attributes) {
if (isLocalReferenceAttribute(typeName, attr.name)) continue;
if (attr.refType) add(attr.refType);
}
if (info.attributes.some((a) => a.name.toLowerCase() === "inheritfrom")) {
add(typeName);
}
} else if (
info.kind === "simple" &&
info.refType &&
!typeChain(typeName).includes("Poid")
) {
add(info.refType);
}
}
referenceTargetTypeSet = set;
return set;
}
/** True when the type is a designed reference target (see above). */
export function isReferenceTargetType(typeName: string | null): boolean {
if (!typeName) return false;
return referenceTargetTypes().has(canonicalTypeName(typeName) ?? typeName);
}
+32
View File
@@ -24,6 +24,22 @@ export interface AssetDef {
manifestSource?: string;
}
/**
* One resolved reference occurrence pointing at an asset definition.
* Produced by `buildReferenceIndex` from per-file reference records.
*/
export interface ReferenceSite {
/** Absolute path of the referencing file. */
file: string;
/** 1-based line of the reference value. */
line: number;
/** Character offset of the value start (relative to the file text). */
start: number;
/** Character offset one past the value end. */
end: number;
kind: "attr" | "content";
}
export interface DefineDef {
name: string;
value: string;
@@ -121,6 +137,8 @@ export interface IndexStats {
/** Time spent shallow-scanning deferred art assets (ms). */
artScanMs: number;
assetCount: number;
/** Total resolved reference sites in the reverse index. */
referenceCount: number;
defineCount: number;
manifestFiles: number;
manifestAssetCount: number;
@@ -159,6 +177,20 @@ export interface ModIndex {
sourceCandidates: SourceCandidate[];
/** Problems found while indexing (unresolved includes, cycles, ...). */
diagnostics: IndexerDiagnostic[];
/**
* Reverse reference index: asset definition key (see
* `referenceIndex.assetDefKey`) -> reference sites. Built from the compact
* per-file reference records when a snapshot is published, so counts and
* Find All References share one semantic source of truth.
*/
references: Map<string, ReferenceSite[]>;
/**
* normKey(file) -> SHA-1 of the file's compact index records as this
* snapshot consumed them. Lets open documents detect "my file changed on
* disk but the index still uses older records" and trigger a targeted
* rebuild, while ignoring cosmetic text changes (line endings, whitespace).
*/
recordsHashes: Map<string, string>;
stats: IndexStats;
/**
* Document-local overlay (when the index was obtained through the
+5
View File
@@ -92,6 +92,11 @@ export const modelMeta = {
typeCount: Object.keys(model.types).length,
};
/** All type names in the XSD model (complex + simple), in model order. */
export function allTypeNames(): string[] {
return Object.keys(model.types);
}
export function topLevelElements(): string[] {
return model.topLevelElements;
}
+9 -2
View File
@@ -92,7 +92,7 @@ export class ModWorkspace {
this.settings = readSettings();
const storageUri = context.storageUri ?? context.globalStorageUri;
if (storageUri) {
this.diskCachePath = join(storageUri.fsPath, "index-records-v1.json.gz");
this.diskCachePath = join(storageUri.fsPath, "index-records-v3.json.gz");
}
this.output = vscode.window.createOutputChannel("RA3 Mod XML");
this.statusBar = vscode.window.createStatusBarItem(
@@ -145,6 +145,11 @@ export class ModWorkspace {
async initialize(): Promise<void> {
this.projectRoot = this.detectProjectRoot();
void vscode.commands.executeCommand(
"setContext",
"ra3modxml.active",
this.projectRoot != null,
);
if (!this.projectRoot) {
this.statusBar.hide();
return;
@@ -346,6 +351,7 @@ export class ModWorkspace {
stat: rec.stat,
records: rec.records,
kind: rec.kind,
contentHash: rec.contentHash,
});
}
}
@@ -451,6 +457,7 @@ export class ModWorkspace {
`${s.projectDir}\n` +
`${s.indexedFiles} files indexed (${s.parsedFiles} parsed, ${s.shallowScannedFiles} art assets shallow-scanned, ${(s.elapsedMs / 1000).toFixed(1)}s)\n` +
`${s.assetCount} assets (${s.manifestAssetCount} from ${s.manifestFiles} manifests)\n` +
`${s.referenceCount} reference sites\n` +
`${s.defineCount} defines, ${s.streams} streams, ${s.sourceCandidates} include candidates\n` +
`Phase: ${s.phase} · Complete: ${s.complete}${stale}`;
}
@@ -589,7 +596,7 @@ export class ModWorkspace {
return {
file: { path: resolve(path), stat: null },
parse,
records: extractIndexRecords(parse, lineMap),
records: extractIndexRecords(parse, lineMap, text),
lineMap,
};
} catch {
+2 -2
View File
@@ -52,7 +52,7 @@ test("IndexRecordsCache stores and invalidates entries", () => {
const cache = new IndexRecordsCache();
const entry = {
stat,
records: { assets: [], defines: [], includes: [], rootXiIncludes: [], nestedXiIncludes: [] },
records: { assets: [], defines: [], includes: [], rootXiIncludes: [], nestedXiIncludes: [], references: [] },
kind: "full",
};
cache.set("a.xml", entry);
@@ -65,7 +65,7 @@ test("IndexRecordsCache exposes entries for disk persistence", () => {
const cache = new IndexRecordsCache();
const entry = {
stat,
records: { assets: [], defines: [], includes: [], rootXiIncludes: [], nestedXiIncludes: [] },
records: { assets: [], defines: [], includes: [], rootXiIncludes: [], nestedXiIncludes: [], references: [] },
kind: "full",
};
cache.set("a.xml", entry);
+210
View File
@@ -0,0 +1,210 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { createRequire } from "node:module";
import { fileURLToPath } from "node:url";
import { dirname, join, resolve } from "node:path";
// Minimal vscode shim: the CodeLens provider only constructs CodeLens/Range.
class Range {
constructor(start, end) {
this.start = start;
this.end = end;
}
}
class CodeLens {
constructor(range, command) {
this.range = range;
this.command = command;
}
}
const require = createRequire(import.meta.url);
const Module = require("module");
const origResolve = Module._resolveFilename;
Module._resolveFilename = function (request, ...args) {
if (request === "vscode") return "vscode-stub";
return origResolve.call(this, request, ...args);
};
require.cache["vscode-stub"] = {
id: "vscode-stub",
filename: "vscode-stub",
loaded: true,
exports: {
Range,
CodeLens,
},
};
const { Ra3CodeLensProvider } = require("../out/features/codeLens.js");
const { assetDefKey } = require("../out/indexer/referenceIndex.js");
const { normKey } = require("../out/indexer/caches.js");
const root = dirname(dirname(fileURLToPath(import.meta.url)));
const PROJECT = join(root, "test", "fixtures", "minimod");
const SDK = join(root, "test", "fixtures", "fakesdk");
// A real file whose path matches what DATA:Includes/Units.xml resolves to.
const FILE = resolve(join(PROJECT, "Data", "Includes", "Units.xml"));
const TEXT = `<AssetDeclaration>
<GameObject id="TestTank"/>
<GameObject id="BaseVehicle"/>
<CameraSettings id="S"/>
</AssetDeclaration>`;
function makeDocument(text = TEXT) {
const lineStarts = [0];
for (let i = 0; i < text.length; i++) {
if (text.charCodeAt(i) === 10) lineStarts.push(i + 1);
}
return {
uri: { fsPath: FILE },
getText: () => text,
positionAt: (offset) => {
let lo = 0;
let hi = lineStarts.length - 1;
while (lo < hi) {
const mid = (lo + hi + 1) >> 1;
if (lineStarts[mid] <= offset) lo = mid;
else hi = mid - 1;
}
return { line: lo, character: offset - lineStarts[lo] };
},
};
}
function makeIndex() {
const tankSite = {
file: "C:/mod/Data/Other.xml",
line: 7,
start: 40,
end: 48,
kind: "content",
};
const secondSite = {
file: "C:/mod/Data/Third.xml",
line: 2,
start: 12,
end: 20,
kind: "attr",
};
const references = new Map();
references.set(
assetDefKey({
type: "GameObject",
id: "TestTank",
file: FILE,
line: 2,
}),
[tankSite, secondSite],
);
references.set(
assetDefKey({
type: "GameObject",
id: "BaseVehicle",
file: FILE,
line: 3,
}),
[],
);
return {
references,
assets: new Map(),
sdkDir: SDK,
projectDir: PROJECT,
};
}
test("CodeLens shows counts on reference-target types only, including zero", () => {
const provider = new Ra3CodeLensProvider({
isRa3Workspace: () => true,
index: makeIndex(),
});
const lenses = provider.provideCodeLenses(makeDocument(), {});
assert.equal(lenses.length, 2, "no lens for auto-registered CameraSettings");
const tank = lenses.find((l) => l.command.arguments[0].id === "TestTank");
const base = lenses.find((l) => l.command.arguments[0].id === "BaseVehicle");
assert.ok(tank, "GameObject TestTank gets a lens");
assert.equal(tank.command.title, "2 references");
assert.equal(tank.command.command, "ra3modxml.showReferences");
assert.equal(tank.command.arguments[0].type, "GameObject");
assert.equal(tank.command.arguments[0].line, 2);
assert.ok(base, "zero is still displayed for reference-target types");
assert.equal(base.command.title, "0 references");
// Lenses anchor on the element start tag.
assert.equal(tank.range.start.line, 1);
assert.ok(tank.range.start.character < tank.range.end.character);
});
test("CodeLens returns nothing without a workspace or index", () => {
const noWorkspace = new Ra3CodeLensProvider({
isRa3Workspace: () => false,
index: makeIndex(),
});
assert.deepEqual(noWorkspace.provideCodeLenses(makeDocument(), {}), []);
const noIndex = new Ra3CodeLensProvider({
isRa3Workspace: () => true,
index: null,
});
assert.deepEqual(noIndex.provideCodeLenses(makeDocument(), {}), []);
});
test("CodeLens counts references attached to a manifest definition with the same SageXml source", () => {
const manifestDef = {
type: "GameObject",
id: "TestTank",
file: resolve(join(SDK, "builtmods", "static.manifest")),
line: 0,
origin: "manifest",
manifestSource: "DATA:Includes/Units.xml",
};
const site = {
file: "C:/mod/Data/Other.xml",
line: 7,
start: 40,
end: 48,
kind: "content",
};
const references = new Map();
references.set(assetDefKey(manifestDef), [site]);
const idx = {
references,
assets: new Map([
["GameObject", new Map([["testtank", [manifestDef]]])],
]),
sdkDir: SDK,
projectDir: PROJECT,
};
const provider = new Ra3CodeLensProvider({
isRa3Workspace: () => true,
index: idx,
});
const lenses = provider.provideCodeLenses(makeDocument(), {});
const tank = lenses.find((l) => l.command.arguments[0].id === "TestTank");
assert.ok(tank, "lens is shown for the SageXml-backed definition");
assert.equal(tank.command.title, "1 reference");
});
test("CodeLens schedules a targeted rebuild when the open document desyncs from the snapshot", () => {
const idx = makeIndex();
idx.recordsHashes = new Map([[normKey(FILE), "stale-hash"]]);
const calls = [];
const provider = new Ra3CodeLensProvider({
isRa3Workspace: () => true,
index: idx,
invalidate: (p) => calls.push(["invalidate", p]),
scheduleRebuild: (r) => calls.push(["schedule", r]),
});
provider.provideCodeLenses(makeDocument(), {});
assert.ok(
calls.some(([kind]) => kind === "invalidate"),
"the stale file is invalidated",
);
assert.ok(
calls.some(([kind, reason]) => kind === "schedule" && reason === "records-desync"),
"a targeted rebuild is scheduled",
);
});
+8 -1
View File
@@ -19,6 +19,7 @@ const sampleRecords = {
includes: [{ type: "all", source: "Units.xml", line: 4 }],
rootXiIncludes: [],
nestedXiIncludes: [],
references: [],
};
function stampOf(file) {
@@ -47,7 +48,12 @@ test("disk cache roundtrip keeps records and leaves no temp file", async (t) =>
await cache.save([
[
file.toLowerCase(),
{ stat: stampOf(file), records: sampleRecords, kind: "full" },
{
stat: stampOf(file),
records: sampleRecords,
kind: "full",
contentHash: "abc123",
},
],
]);
assert.equal(fs.existsSync(`${filePath}.tmp`), false, "atomic write leaves no temp");
@@ -60,6 +66,7 @@ test("disk cache roundtrip keeps records and leaves no temp file", async (t) =>
assert.equal(stats.dropped, 0);
assert.equal(records.length, 1);
assert.deepEqual(records[0].records, sampleRecords);
assert.equal(records[0].contentHash, "abc123");
});
test("stat mismatch drops the cached entry", async (t) => {
+1 -1
View File
@@ -31,7 +31,7 @@ async function readParsed(path) {
return {
file: { path, stat: null },
parse,
records: extractIndexRecords(parse, lineMap),
records: extractIndexRecords(parse, lineMap, text),
lineMap,
};
}
+51 -1
View File
@@ -24,7 +24,7 @@ test("extractIndexRecords mirrors the walk semantics", () => {
</GameObject>
</AssetDeclaration>`;
const lineMap = new LineMap(text);
const records = extractIndexRecords(parseXml(text), lineMap);
const records = extractIndexRecords(parseXml(text), lineMap, text);
assert.deepEqual(
records.assets.map((a) => [a.type, a.id, a.line]),
[
@@ -62,4 +62,54 @@ test("recordsFromShallow converts offsets to 1-based lines", () => {
assert.equal(records.assets[0].type, "W3DContainer");
assert.equal(records.assets[0].id, "A");
assert.equal(records.assets[0].line, 2);
assert.deepEqual(records.references, []);
});
test("extractIndexRecords records typed references and skips non-references", () => {
const text = `<AssetDeclaration>
<GameObject id="Tank" CommandSet="CS" inheritFrom="Base" KindOf="SELECTABLE"/>
<ObjectCreationList id="OCL">
<CreateObject>
<CreateObject>Tank</CreateObject>
</CreateObject>
</ObjectCreationList>
<CameraSettings id="S"/>
</AssetDeclaration>`;
const lineMap = new LineMap(text);
const records = extractIndexRecords(parseXml(text), lineMap, text);
const attrs = records.references.filter((r) => r.kind === "attr");
const content = records.references.filter((r) => r.kind === "content");
// Typed attribute reference keeps the XSD refType.
const cs = attrs.find((r) => r.value === "CS");
assert.ok(cs, "CommandSet reference is recorded");
assert.equal(cs.refType, "LogicCommandSet");
assert.equal(cs.selfType, null);
assert.equal(cs.line, 2);
assert.equal(records.references.some((r) => r.start === cs.start && r.end === cs.end), true);
// inheritFrom records the element type as selfType instead of refType.
const base = attrs.find((r) => r.value === "Base");
assert.ok(base, "inheritFrom reference is recorded");
assert.equal(base.refType, null);
assert.equal(base.selfType, "GameObject");
// Enums and non-reference values are not references.
assert.equal(attrs.some((r) => r.value === "SELECTABLE"), false);
// Simple-content text is recorded with its content refType and offsets.
const tank = content.find((r) => r.value === "Tank");
assert.ok(tank, "content reference is recorded");
assert.equal(tank.refType, "GameObject");
const line = text.split("\n")[4];
assert.equal(line.slice(tank.start - text.indexOf(line), tank.end - text.indexOf(line)), "Tank");
// The id definition itself is never recorded as a reference.
assert.equal(
records.references.some(
(r) => r.value === "Tank" && r.kind === "attr",
),
false,
);
});
+378
View File
@@ -0,0 +1,378 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { fileURLToPath } from "node:url";
import { dirname, join } from "node:path";
import { mkdtempSync, rmSync, statSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { ModIndexer } from "../out/indexer/indexer.js";
import { CachedDirectoryWalker } from "../out/indexer/fileScanner.js";
import {
IndexRecordsCache,
contentHash,
normKey,
recordsHash,
} from "../out/indexer/caches.js";
import {
assetDefKey,
buildReferenceIndex,
documentRecordsDesynced,
referenceSitesForDef,
referenceSitesForDefinition,
scheduleRebuildIfRecordsDesync,
unreferencedByType,
} from "../out/indexer/referenceIndex.js";
import { LineMap, parseXml } from "../out/language/xmlParser.js";
import { extractIndexRecords } from "../out/indexer/records.js";
const root = dirname(dirname(fileURLToPath(import.meta.url)));
const project = join(root, "test", "fixtures", "minimod");
const sdk = join(root, "test", "fixtures", "fakesdk");
async function buildIndex() {
const indexer = new ModIndexer({
projectDir: project,
sdkDir: sdk,
builtmodsDirs: [join(sdk, "builtmods")],
indexSageXml: true,
additionalDataSearchPaths: [],
walker: new CachedDirectoryWalker(),
});
return indexer.build();
}
function makeDef(type, id, file, line, extra = {}) {
return {
type,
id,
file,
line,
origin: "project",
...extra,
};
}
test("buildReferenceIndex resolves records with strict type filtering", () => {
const tank = makeDef("GameObject", "Tank", "C:/mod/A.xml", 2);
const gun = makeDef("WeaponTemplate", "Tank", "C:/mod/B.xml", 1);
const cs = makeDef("LogicCommandSet", "CS", "C:/mod/C.xml", 4);
const lookup = {
assets: new Map(),
assetsById: new Map([
["tank", [tank, gun]],
["cs", [cs]],
]),
};
const recordsA = {
assets: [],
defines: [],
includes: [],
rootXiIncludes: [],
nestedXiIncludes: [],
references: [
{
kind: "content",
refType: "GameObject",
selfType: null,
value: "Tank",
line: 5,
start: 40,
end: 44,
},
],
};
const recordsB = {
assets: [],
defines: [],
includes: [],
rootXiIncludes: [],
nestedXiIncludes: [],
references: [
{
kind: "attr",
refType: "LogicCommandSet",
selfType: null,
value: "CS",
line: 3,
start: 10,
end: 12,
},
],
};
const map = buildReferenceIndex(
[
{ file: "C:/mod/A.xml", records: recordsA },
{ file: "C:/mod/B.xml", records: recordsB },
],
lookup,
);
// The content reference resolves only to the GameObject definition, never
// to the same-name WeaponTemplate.
const tankSites = map.get(assetDefKey(tank));
assert.equal(tankSites.length, 1);
assert.equal(tankSites[0].file, "C:/mod/A.xml");
assert.equal(tankSites[0].kind, "content");
assert.equal(map.get(assetDefKey(gun)), undefined);
assert.equal(map.get(assetDefKey(cs)).length, 1);
});
test("records extracted from XML resolve through the reference index", () => {
const text = `<AssetDeclaration>
<GameObject id="Tank" CommandSet="CS"/>
<LogicCommandSet id="CS"/>
<ObjectCreationList id="OCL">
<CreateObject>
<CreateObject>Tank</CreateObject>
</CreateObject>
</ObjectCreationList>
</AssetDeclaration>`;
const lineMap = new LineMap(text);
const records = extractIndexRecords(parseXml(text), lineMap, text);
const file = "C:/mod/D.xml";
const tank = makeDef("GameObject", "Tank", file, 2);
const cs = makeDef("LogicCommandSet", "CS", file, 3);
const lookup = {
assets: new Map(),
assetsById: new Map([
["tank", [tank]],
["cs", [cs]],
]),
};
const map = buildReferenceIndex([{ file, records }], lookup);
const tankSites = map.get(assetDefKey(tank));
assert.equal(tankSites.length, 1);
assert.equal(tankSites[0].kind, "content");
assert.equal(records.references.find((r) => r.value === "Tank").start, tankSites[0].start);
const csSites = map.get(assetDefKey(cs));
assert.equal(csSites.length, 1);
assert.equal(csSites[0].kind, "attr");
});
test("referenceSitesForDefinition unions manifest-source sites onto the SageXml source file", () => {
const sourceFile = join(project, "Data", "Includes", "Units.xml");
const manifestDef = {
type: "GameObject",
id: "Tank",
file: join(sdk, "builtmods", "static.manifest"),
line: 0,
origin: "manifest",
manifestSource: "DATA:Includes/Units.xml",
};
const site = {
file: "C:/mod/ref.xml",
line: 3,
start: 10,
end: 14,
kind: "attr",
};
const idx = {
assets: new Map([["GameObject", new Map([["tank", [manifestDef]]])]]),
assetsById: new Map([["tank", [manifestDef]]]),
references: new Map([[assetDefKey(manifestDef), [site]]]),
projectDir: project,
sdkDir: sdk,
};
const sites = referenceSitesForDefinition(idx, {
type: "GameObject",
id: "Tank",
file: sourceFile,
line: 4,
});
assert.equal(sites.length, 1);
assert.equal(sites[0].file, "C:/mod/ref.xml");
// A different file does not inherit the manifest definition's sites.
const other = referenceSitesForDefinition(idx, {
type: "GameObject",
id: "Tank",
file: "C:/mod/elsewhere.xml",
line: 4,
});
assert.equal(other.length, 0);
});
test("the minimod indexer publishes a semantic reverse reference index", async () => {
const idx = await buildIndex();
assert.ok(idx.stats.referenceCount > 0, "reverse index is populated");
// Units.xml has CommandSet="TestTankCommandSet" and inheritFrom="BaseVehicle".
const lcs = idx.assets.get("LogicCommandSet").get("testtankcommandset")[0];
const lcsSites = idx.references.get(assetDefKey(lcs));
assert.ok(lcsSites && lcsSites.length >= 1);
assert.ok(lcsSites.some((s) => s.kind === "attr"));
const baseVehicle = idx.assets.get("GameObject").get("basevehicle")[0];
const bvSites = idx.references.get(assetDefKey(baseVehicle));
assert.ok(bvSites && bvSites.length >= 1);
});
test("references survive a records-cache invalidation during the build", async () => {
const recordsCache = new IndexRecordsCache();
const indexer = new ModIndexer({
projectDir: project,
sdkDir: sdk,
builtmodsDirs: [join(sdk, "builtmods")],
indexSageXml: true,
additionalDataSearchPaths: [],
walker: new CachedDirectoryWalker(),
recordsCache,
});
const unitsPath = join(project, "Data", "Includes", "Units.xml");
const idx = await indexer.build((phaseIndex) => {
if (!phaseIndex.complete) recordsCache.invalidate(unitsPath);
});
const lcs = idx.assets.get("LogicCommandSet").get("testtankcommandset")[0];
const sites = idx.references.get(assetDefKey(lcs));
assert.ok(
sites && sites.length >= 1,
"final snapshot keeps the walk-time reference records",
);
});
test("force rebuild verifies content even when every stat signal matches", async (t) => {
const tmp = mkdtempSync(join(tmpdir(), "ra3-refidx-"));
t.after(() => rmSync(tmp, { recursive: true, force: true }));
const file = join(tmp, "units.xml");
const diskText = `<AssetDeclaration><GameObject id="Tank"/></AssetDeclaration>`;
const cachedText = `<AssetDeclaration><GameObject id="Cached"/></AssetDeclaration>`;
writeFileSync(file, diskText);
const st = statSync(file);
const stamp = {
mtimeMs: st.mtimeMs,
size: st.size,
birthtimeMs: st.birthtimeMs,
ctimeMs: st.ctimeMs,
};
const cache = new IndexRecordsCache();
cache.set(file, {
stat: stamp,
records: extractIndexRecords(
parseXml(cachedText),
new LineMap(cachedText),
cachedText,
),
kind: "full",
contentHash: contentHash(cachedText),
});
const opts = {
projectDir: tmp,
sdkDir: tmp,
builtmodsDirs: [],
indexSageXml: false,
additionalDataSearchPaths: [],
walker: new CachedDirectoryWalker(),
recordsCache: cache,
};
const trusted = new ModIndexer({ ...opts, trustUnchanged: true });
const trustedParsed = await trusted.readDocument(file);
assert.equal(
trustedParsed.records.assets[0].id,
"Cached",
"trusted rebuilds reuse the cached records without reading",
);
const forced = new ModIndexer({ ...opts, trustUnchanged: false });
const forcedParsed = await forced.readDocument(file);
assert.equal(
forcedParsed.records.assets[0].id,
"Tank",
"force rebuild re-reads a stat-matching but content-stale entry",
);
assert.equal(cache.get(file).contentHash, contentHash(diskText));
});
test("records-desync self-heal schedules a targeted rebuild only for clean files", () => {
const file = join(project, "Data", "Includes", "Units.xml");
const emptyRecords = extractIndexRecords(
parseXml("<AssetDeclaration/>"),
new LineMap("<AssetDeclaration/>"),
"<AssetDeclaration/>",
);
const idx = {
recordsHashes: new Map([[normKey(file), recordsHash(emptyRecords)]]),
references: new Map(),
assets: new Map(),
sdkDir: sdk,
projectDir: project,
};
assert.equal(
documentRecordsDesynced(idx, file, "<AssetDeclaration/>"),
false,
);
assert.equal(
documentRecordsDesynced(
idx,
file,
"<AssetDeclaration><GameObject id=\"Tank\"/></AssetDeclaration>",
),
true,
);
let invalidated = null;
let scheduled = null;
const ws = {
index: idx,
invalidate: (p) => {
invalidated = p;
},
scheduleRebuild: (r) => {
scheduled = r;
},
};
assert.equal(
scheduleRebuildIfRecordsDesync(ws, {
uri: { fsPath: file, scheme: "file" },
isDirty: false,
getText: () => "<AssetDeclaration><GameObject id=\"Tank\"/></AssetDeclaration>",
}),
true,
);
assert.equal(invalidated, file);
assert.equal(scheduled, "records-desync");
invalidated = null;
scheduled = null;
assert.equal(
scheduleRebuildIfRecordsDesync(ws, {
uri: { fsPath: file, scheme: "file" },
isDirty: true,
getText: () => "<AssetDeclaration><GameObject id=\"Tank\"/></AssetDeclaration>",
}),
false,
);
assert.equal(scheduled, null);
});
test("unreferencedByType reports only meaningful project assets", async () => {
const idx = await buildIndex();
const map = unreferencedByType(idx);
const all = [...map.values()].flat();
assert.ok(all.length > 0, "some project assets are unreferenced");
assert.ok(
all.every((d) => d.origin === "project" && !d.viaInstance),
"only compiled-stream project definitions are reported",
);
assert.ok(
all.every((d) => !d.file.toLowerCase().includes("fakesdk")),
"SDK/manifest definitions are never reported",
);
// WeaponTemplate TestTankCannon is defined but never referenced.
const weapons = map.get("WeaponTemplate");
assert.ok(weapons.some((d) => d.id === "TestTankCannon"));
// LogicCommandSet TestTankCommandSet is referenced, so it must not appear.
const commandSets = map.get("LogicCommandSet");
assert.ok(!commandSets?.some((d) => d.id === "TestTankCommandSet"));
// referenceSitesForDef is stable across lookups and safe without a map.
const tankCannon = idx.assets.get("WeaponTemplate").get("testtankcannon")[0];
assert.deepEqual(referenceSitesForDef(idx, tankCannon), []);
assert.deepEqual(referenceSitesForDef(null, tankCannon), []);
});
+169
View File
@@ -0,0 +1,169 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { createRequire } from "node:module";
// Minimal vscode shim for the semantic reference provider.
class Position {
constructor(line, character) {
this.line = line;
this.character = character;
}
}
class Range {
constructor(start, end) {
this.start = start;
this.end = end;
}
}
class Location {
constructor(uri, range) {
this.uri = uri;
this.range = range;
}
}
const Uri = {
file: (p) => ({ fsPath: p }),
};
const require = createRequire(import.meta.url);
const Module = require("module");
const origResolve = Module._resolveFilename;
Module._resolveFilename = function (request, ...args) {
if (request === "vscode") return "vscode-stub";
return origResolve.call(this, request, ...args);
};
require.cache["vscode-stub"] = {
id: "vscode-stub",
filename: "vscode-stub",
loaded: true,
exports: {
Position,
Range,
Location,
Uri,
SymbolKind: {},
DocumentSymbol: class {},
},
};
const { Ra3ReferenceProvider } = require("../out/features/navigation.js");
const { LineMap, parseXml } = require("../out/language/xmlParser.js");
const { extractIndexRecords } = require("../out/indexer/records.js");
const { buildReferenceIndex } = require("../out/indexer/referenceIndex.js");
const FILE = "C:/mod/Data/Units.xml";
const TEXT = `<AssetDeclaration>
<GameObject id="Tank"/>
<ObjectCreationList id="OCL">
<CreateObject>
<CreateObject>Tank</CreateObject>
</CreateObject>
</ObjectCreationList>
</AssetDeclaration>`;
function makeDocument(text = TEXT) {
const lineStarts = [0];
for (let i = 0; i < text.length; i++) {
if (text.charCodeAt(i) === 10) lineStarts.push(i + 1);
}
return {
uri: { fsPath: FILE },
getText: () => text,
offsetAt: (pos) => lineStarts[pos.line] + pos.character,
positionAt: (offset) => {
let lo = 0;
let hi = lineStarts.length - 1;
while (lo < hi) {
const mid = (lo + hi + 1) >> 1;
if (lineStarts[mid] <= offset) lo = mid;
else hi = mid - 1;
}
return new Position(lo, offset - lineStarts[lo]);
},
};
}
function makeScope() {
const parse = parseXml(TEXT);
const lineMap = new LineMap(TEXT);
const records = extractIndexRecords(parse, lineMap, TEXT);
const def = {
type: "GameObject",
id: "Tank",
file: FILE,
line: 2,
origin: "project",
};
const lookup = {
assets: new Map([["GameObject", new Map([["tank", [def]]])]]),
assetsById: new Map([["tank", [def]]]),
};
const references = buildReferenceIndex([{ file: FILE, records }], lookup);
const idx = {
...lookup,
references,
complete: true,
phase: "art",
projectDir: "C:/mod",
sdkDir: "C:/sdk",
defines: new Map(),
files: new Map(),
streams: [],
manifests: new Map(),
sourceCandidates: [],
diagnostics: [],
stats: {},
};
return {
merged: idx,
};
}
function makeWs(scope) {
const parse = parseXml(TEXT);
const lineMap = new LineMap(TEXT);
return {
isRa3Workspace: () => true,
getScope: async () => scope,
indexer: {
readDom: async (path) =>
path === FILE
? { file: { path: FILE }, parse, lineMap, records: null }
: null,
},
};
}
test("semantic Find All References excludes the definition even when includeDeclaration is set", async () => {
const scope = makeScope();
const provider = new Ra3ReferenceProvider(makeWs(scope));
const document = makeDocument();
const defLine = TEXT.split("\n")[1];
const defPos = new Position(1, defLine.indexOf('id="') + 4);
const refs = await provider.provideReferences(document, defPos, {
includeDeclaration: true,
}, {});
assert.ok(refs, "references are returned");
assert.equal(refs.length, 1, "only the typed content reference is returned");
assert.equal(refs[0].uri.fsPath, FILE);
assert.equal(refs[0].range.start.line, 4);
assert.equal(
TEXT.split("\n")[4].slice(refs[0].range.start.character, refs[0].range.end.character),
"Tank",
);
});
test("FAR from the reference site itself returns the same result", async () => {
const scope = makeScope();
const provider = new Ra3ReferenceProvider(makeWs(scope));
const document = makeDocument();
const contentLine = TEXT.split("\n")[4];
const contentPos = new Position(4, contentLine.indexOf("Tank") + 1);
const refs = await provider.provideReferences(document, contentPos, {
includeDeclaration: false,
}, {});
assert.equal(refs.length, 1);
assert.equal(refs[0].range.start.line, 4);
});