6 Commits
Author SHA1 Message Date
lanyi 36eaaafa01 improve codelens 2026-08-07 13:16:50 +02:00
lanyi 47807f9fed 0.1.15 继续优化补全 2026-08-04 19:14:12 +02:00
lanyi f8187da611 0.1.13 improve completion 2026-08-04 17:53:11 +02:00
lanyi 6794896ac7 0.1.9 2026-08-04 14:28:26 +02:00
lanyi 44b2eaf273 0.1.1 2026-08-02 19:11:54 +02:00
lanyi 4cc07b9abc 支持 W3X 2026-08-02 14:32:18 +02:00
61 changed files with 9933 additions and 545 deletions
+1
View File
@@ -3,3 +3,4 @@ out/
dist/ dist/
*.vsix *.vsix
OpenSAGE/ OpenSAGE/
debug.log
+65 -9
View File
@@ -6,23 +6,62 @@
- **语法高亮**:在普通 XML 高亮之上叠加领域标记(`$DEFINE` 常量、`inheritFrom`、`xai:joinAction`、结构标签);XML 语法异常(如未闭合引号)期间由语义 token 兜底,标签/属性/值着色不中断。 - **语法高亮**:在普通 XML 高亮之上叠加领域标记(`$DEFINE` 常量、`inheritFrom`、`xai:joinAction`、结构标签);XML 语法异常(如未闭合引号)期间由语义 token 兜底,标签/属性/值着色不中断。
- **自动补全**: - **自动补全**:
- 元素名:按当前父元素的 XSD 模型补全子元素;顶层资产(`AssetDeclaration` 内)补全 `GameObject`、`WeaponTemplate` 等 295 种类型。 - 元素名:按当前父元素的 XSD 模型补全子元素;顶层资产(`AssetDeclaration` 内)补全 `GameObject`、`WeaponTemplate` 等 295 种类型。已输入 `<` 时补全保留该 `<`、只替换名称区(不会出现 `<<`);需要填文本的 simple-content 元素(如 `<CreateObject>`)补全为 `<CreateObject>$1</CreateObject>` 并自动弹出值补全,而不是无法填值的自闭合标签。
- 属性名:必填属性优先,附带类型/文档/默认值;自动提示 `xai:joinAction` 与 `xmlns:xai`。 - 属性名:必填属性优先,附带类型/文档/默认值;自动提示 `xai:joinAction` 与 `xmlns:xai`。接受补全时自动避免与上一个属性贴在一起,并按文件已有的排版补空格或换行(换行的基础缩进由编辑器提供,插件不再内嵌缩进以免叠加);数字/角度/时间等标量属性直接填入 XSD 默认值或类型示例(如 `0d`、`0s`),引用/枚举/布尔等保留真正的 `$1` 占位符并弹出值补全。
- 属性值: - 属性值:
- 引用型属性(如 `CommandSet`、`Weapon`)按 `xas:refType` 补全对应类型的资产 ID(**同名 ID 只补全匹配类型**); - 引用型属性(如 `CommandSet`、`Weapon`)按 `xas:refType` 补全对应类型的资产 ID(**同名 ID 只补全匹配类型**);
- `inheritFrom` 补全可继承的资产 ID; - `inheritFrom` 补全可继承的资产 ID;
- 枚举与位标志列表(如 `Include type`、`LocomotorTemplate@Surfaces`、`KindOf`;列表值支持空格后继续补全下一项); - 枚举与位标志列表(如 `Include type`、`LocomotorTemplate@Surfaces`、`KindOf`;列表值在空格后自动继续补全下一项,已使用的 flag 不再重复推荐,闭合值末尾可直接追加新 flag);
- 布尔值、`$DEFINE` 常量; - 布尔值、`$DEFINE` 常量;
- `<Include source>` 补全可解析的 `DATA:` / `ART:` / `AUDIO:` 与项目相对路径。 - `<Include source>` 补全可解析的 `DATA:` / `ART:` / `AUDIO:` 与项目相对路径。
- 元素文本内容:simple-content 引用元素(如 `<CreateObject>`、`<RequiredUpgrade>`、`<SpawnTemplate>`)直接在标签间补全对应类型的资产 ID(`GameObjectWeakRef` → GameObject)、枚举或 `$DEFINE`。
接受片段后的 `$1` 光标位置会立即弹出值补全,而不是属性名;引用列表超过
400 条时标记为不完整,继续输入会重新请求,因此 `CrateDebris_01` 这类排在
列表后部的 id 不会因首屏截断而消失。
- **悬停提示**:元素/属性显示 XSD 文档、类型、必填/默认值;引用值显示定义位置;`$DEFINE` 显示值与定义位置;`Include source` / `xi:include href` 显示解析后的目标文件;`xi:include` 元素与属性给出 XInclude 说明。 - **悬停提示**:元素/属性显示 XSD 文档、类型、必填/默认值;引用值显示定义位置;`$DEFINE` 显示值与定义位置;`Include source` / `xi:include href` 显示解析后的目标文件;`xi:include` 元素与属性给出 XInclude 说明。
- **引用导航**:从引用值(`CommandSet="..."`、`Weapon="..."`、`inheritFrom`)跳转到定义(严格按引用类型过滤,候选由 `ra3modxml.definitionMode` 控制:`all` 列出 mod + 原版、`project-only` 优先项目内定义);`Ctrl+点击` Include / `xi:include href` 打开目标文件;Find All References 搜索整个工作区;文档大纲列出顶层资产与 `$DEFINE`。 - **引用导航**:从引用值(`CommandSet="..."`、`Weapon="..."`、`inheritFrom`、`<CreateObject>ID</CreateObject>` 等元素文本)跳转到定义(严格按引用类型过滤,候选由 `ra3modxml.definitionMode` 控制:`all` 列出 mod + 原版、`project-only` 优先项目内定义);`Ctrl+点击` Include / `xi:include href` 打开目标文件;Find All References 基于**语义引用索引**(属性引用 + simple-content 文本 + `inheritFrom`,排除 id 定义点 / Poid / `$DEFINE`,不再全文搜索);文档大纲列出顶层资产与 `$DEFINE`。
- **错误检查**:XML 格式错误、未知元素/属性(`xi:` 等外来命名空间不误报)、顶层资产缺 `id`、重复 ID、未解析引用(含类型不匹配)、Include / 嵌套 `xi:include` 目标找不到、`$DEFINE` 未定义。 - **引用计数(CodeLens)**:在“设计上应被引用”的顶部资产类型上显示
`0 references` / `1 reference` / `N references`(0 也显示),点击直接打开
references peek;设置类、地图元数据、w3x 子结构等自动注册类型不显示,
避免满屏 0;manifest 资产有对应 SageXml 源码时,引用按源码定义归并计数
(打开 SageXml 源码同样能看到引用数)。
- **未引用资产**:`RA3 Mod XML: Find unreferenced assets…` 命令按类型列出
所有零引用的项目资产并跳转;编辑器右键菜单
`Find unreferenced assets of this type` 可直接使用光标所在资产类型。
- **当前文档局部作用域(T1)**:即使一个文件不在任何全局流里(没有从
`Data/Mod.xml` / `additionalmaps` 可达),插件也会按当前文件自身的资产、
`$DEFINE` 及其 include 链建立局部索引。`xi:include` 会在逻辑树中展开,
使 include 进来的内容获得正确的父上下文;`AttachModuleId` / `ModuleId` /
`AutoResolveBody` 等管线局部(Poid)引用可以补全、悬停与跳转到同一
GameObject 内的模块(含通过 `xi:include` 拼入的兄弟模块)。
- **错误检查**:XML 格式错误、未知元素/属性(`xi:` 等外来命名空间不误报)、顶层资产缺 `id`、重复 ID、未解析引用(含属性值与元素文本内容、类型不匹配)、Include / 嵌套 `xi:include` 目标找不到、`$DEFINE` 未定义。
- **manifest 支持**:`<Include type="reference">` 指向的 `static/global/audio.manifest`(SDK `builtmods`)会被解析,manifest 中的原版资产 ID 可用于补全/悬停/导航/诊断。 - **manifest 支持**:`<Include type="reference">` 指向的 `static/global/audio.manifest`(SDK `builtmods`)会被解析,manifest 中的原版资产 ID 可用于补全/悬停/导航/诊断。
- **美术资产(`.w3x`)**:`W3X.xml` / `ART:` include 链中的 `.w3x` 模型文件会被
索引(`W3DContainer` / `W3DMesh` / `W3DHierarchy` 等顶层资产),因此
`Model@Name`、`Hierarchy`、`Mesh` 等引用可以解析、悬停与跳转。超大模型
(几十 MB 的顶点/三角形数据)采用浅扫描——只提取顶层资产记录、不建 DOM 树,
结果在 workspace 级缓存并跨重建复用,保存文件触发的重建不会重读未变化的模型文件。
- **索引分阶段与部分可用性**:先建立 XML + manifest 索引(首建早期即可用),
w3x 美术资产随后台扫描补齐。索引完成前,语法/模型诊断、枚举与子元素补全、
Include 跳转/悬停照常工作;引用类诊断会“显示但标注”
(`unresolved-reference-indexing` + `(index incomplete)` 说明),不会把
未完成的索引误当成最终结论。
- **大项目性能**:索引记录(资产 / Define / Include / 引用 / 行号)与 include
解析结果跨重建缓存,保存触发的重建零 stat、零重读(Corona 实测约 2 秒);
DOM 树只按需保留并设元素预算,避免内存膨胀。编辑器外的文件改动(git pull、
导出工具)会触发防抖重建;构建期间文件再次被修改时,已发布索引会标记
`(stale)` 并自动重跑。include 路径解析使用目录枚举建立的文件集快照(无
statSync 风暴);records 缓存会持久化到磁盘(gzip + 多信号 stat 校验 +
内容哈希 + 原子写),重启 VS Code 后冷启动只需秒级校验,Corona 实测约 11 秒
(首次全量约 2 分钟)。引用索引只从构建期实际消费的 records 构建;打开文档
时若发现当前文本的 records 与快照不一致(如外置盘重连后缓存过时),会自动
定向重建自愈;`Re-index workspace` 会对 stat 匹配的 XML 也做内容校验。
## 使用 ## 使用
1. 用 VS Code 打开 RA3 Mod 项目文件夹(含 `Data/Mod.xml` 或 `mod.babproj`)。 1. 用 VS Code 打开 RA3 Mod 项目文件夹(含 `Data/Mod.xml` 或 `mod.babproj`)。
2. 插件自动激活并开始后台索引(状态栏显示资产数量)。 2. 插件自动激活并开始后台索引(状态栏显示阶段与资产数量;扩展也会在打开
任意 XML 文件时激活,非 RA3 工作区不会显示 RA3 专属功能)。
3. 编辑任意 `*.xml` 即可获得补全、跳转与诊断。 3. 编辑任意 `*.xml` 即可获得补全、跳转与诊断。
### 设置(`settings.json`) ### 设置(`settings.json`)
@@ -40,6 +79,11 @@
- `RA3 Mod XML: Re-index workspace`:手动重建索引。 - `RA3 Mod XML: Re-index workspace`:手动重建索引。
- `RA3 Mod XML: Show index report`:查看索引统计。 - `RA3 Mod XML: Show index report`:查看索引统计。
- `RA3 Mod XML: Clear caches and rebuild`:清空内存/磁盘缓存并强制全量重建。
- `RA3 Mod XML: Show cache report`:查看磁盘缓存路径、大小、校验统计与命中数。
- `RA3 Mod XML: Find unreferenced assets…`:按类型查找零引用的项目资产。
- `RA3 Mod XML: Find unreferenced assets of this type`:右键菜单入口,
直接查找光标所在顶部资产类型的未引用资产。
## 开发 ## 开发
@@ -69,11 +113,22 @@ src/
schemaModel.ts XSD 模型运行时(schema-model.json / asset-types.json 由 tools 生成) schemaModel.ts XSD 模型运行时(schema-model.json / asset-types.json 由 tools 生成)
indexer/ indexer/
includeResolver.ts Include 路径解析(纯 TS,移植 check_duplicate_ids.py) includeResolver.ts Include 路径解析(纯 TS,移植 check_duplicate_ids.py)
existence.ts 文件集存在性快照(目录枚举 Set,替代逐路径 statSync)
manifestParser.ts .manifest 二进制解析(移植 OpenSAGE ManifestFile.cs) manifestParser.ts .manifest 二进制解析(移植 OpenSAGE ManifestFile.cs)
fileScanner.ts 目录扫描与 Include source 候选 fileScanner.ts 目录扫描与 Include source 候选
refs.ts 引用目标解析(按引用类型过滤) refs.ts 引用目标解析(按引用类型过滤)+ “设计上可被引用类型”判定
indexer.ts 工作区索引器(后台、缓存、增量重建) referenceIndex.ts 引用记录 → 反向引用索引(定义 → 引用位置)+ 未引用报告
features/ completion / hover / navigation / diagnostics / semanticTokens xpointer.ts xi:include xpointer 子集解析(纯 TS)
logicalTree.ts 当前文档逻辑树(xi:include 拼接、局部作用域)
localScope.ts 文档局部索引 overlay(自身链 + include 链)
shallowScan.ts .w3x 等大体积美术资产顶层浅扫描(纯 TS,不建 DOM)
records.ts 每文件紧凑索引记录(资产/Define/Include/xi/引用 + 行号偏移)
caches.ts 跨重建持久缓存(DocumentCache / IndexRecordsCache /
IncludeResolveCache)+ 失效纪元 InvalidationsEpoch
diskCache.ts 跨会话磁盘缓存(gzip JSON、原子写、多信号 stat 校验)
indexer.ts 工作区索引器(后台、缓存、记录驱动重建、分阶段发布)
features/ completion / hover / navigation / references / codeLens /
unreferenced / diagnostics / semanticTokens
syntaxes/ TextMate 注入语法 syntaxes/ TextMate 注入语法
tools/ XSD → 模型、AssetType 枚举提取 tools/ XSD → 模型、AssetType 枚举提取
``` ```
@@ -85,4 +140,5 @@ tools/ XSD → 模型、AssetType 枚举提取
- 领域说明与需求:`docs/requirements.md` - 领域说明与需求:`docs/requirements.md`
- 调研与设计决策:`docs/plan.md` - 调研与设计决策:`docs/plan.md`
- 问题分析与修复记录:`docs/analysis-issues.md` - 问题分析与修复记录:`docs/analysis-issues.md`
- 引用计数 / 语义 FAR / 未引用资产功能设计:`docs/features-reference-counts.md`
- Manifest 格式参考:OpenSAGE `src/OpenSage.Game/Data/StreamFS/ManifestFile.cs`(本仓库 `OpenSAGE/` 子目录,commit `d45d361`) - Manifest 格式参考:OpenSAGE `src/OpenSage.Game/Data/StreamFS/ManifestFile.cs`(本仓库 `OpenSAGE/` 子目录,commit `d45d361`)
-1
View File
@@ -1 +0,0 @@
[0801/032923.150:ERROR:third_party\crashpad\crashpad\util\win\registration_protocol_win.cc:108] CreateFile: 拒绝访问。 (0x5)
File diff suppressed because it is too large Load Diff
+122
View File
@@ -0,0 +1,122 @@
# 功能设计:引用计数、语义 Find All References 与未引用资产
> 状态:已实现(v0.1.16 起)。本文档记录需求分析结论、语义定义、数据流与
> 已知边界;问题排查记录仍放在 `docs/analysis-issues.md`,功能设计单独成文。
## 一、需求与结论
原始需求:
1. 在每个顶部 asset 上显示它被引用了多少次,点击后直接打开“查找所有引用”,
而不是必须通过右键菜单;
2. 能查找“某一种类型中所有没被引用的顶部 asset”。
调研结论(2026-08-05):
- XSD 模型层面,292 个有模型类型的顶层资产中:73 个有类型化引用指向它们,
28 个只能被 `inheritFrom` 指向,**191 个在模型里根本没有任何引用指向**
(设置类、地图元数据、w3x 子结构等)。对这些类型,“0 引用”是唯一正常状态。
- AttachTest 实测:625 个项目定义中 29% 是 0 引用、48% 是 1 引用。
因此“所有资产一律显示计数”会制造大量噪音,也会让用户误以为资产是孤儿。
- 结论:**计数按类型过滤**——只在“设计上应该被引用”的类型上显示
(0 也显示,因为对 GameObject 这类类型 0 是有效信号);自动注册/结构类
类型不显示。
- 旧版 Find All References 是全文搜索(正则匹配 `"id"` / `>id<`),会把
`id="X"` 定义本身、`EditorName="X"` 等非引用属性也算进去。计数如果和它
同源,必然误导;如果不同源,点击后的结果又对不上。因此本轮把 FAR 一起
升级为语义引用索引,计数与点击结果共用同一数据源。
## 二、语义定义:什么算一次“引用”
与补全 / hover / 跳转 / 诊断完全一致(`refs.ts` 是唯一判定来源):
- 带 `xas:refType` 的属性值(如 `CommandSet` → `LogicCommandSet`);
- 带 `xas:refType` 的 simple-content 文本(如 `<CreateObject>ID</CreateObject>`);
- `inheritFrom`(按元素自身类型过滤);
- 无 `refType` 的 `isRef` 属性(按同名 ID 匹配任意声明类型)。
不算引用:
- 元素自己的 `id` 定义点(除非是 `RoadObject@id→Road` 这类跨类型 id 引用);
- Poid 管线局部引用(`ModuleId`、`AttachModuleId`、`SoundRef` 等);
- `$DEFINE` / `=` 常量值;
- 枚举、文件路径、`Include@source` / `xi:include href`;
- w3x 内部父子结构关系(`W3DMesh` 等靠结构归属,不走全局引用)。
引用索引按“类型 + id + 定义位置”精确归属,同名 ID 的不同类型定义互不串扰
(`WeaponTemplate:X` 的引用不会计到 `GameObject:X` 头上)。
## 三、数据流
```
parse DOM
│ records.ts extractIndexRecords(parse, lineMap, text)
▼
IndexRecords.references[] 每文件紧凑记录(refType / selfType / value /
│ 行号 / 起止偏移),随 records 缓存跨重建复用,
│ 并持久化到磁盘缓存(缓存版本 v3,full XML
│ 附带内容哈希;快照另发布每文件 records 哈希)
▼
indexer.ts buildReferences() 只解算本次 build 触及的文件,防止陈旧缓存泄漏
│ referenceIndex.ts buildReferenceIndex()
▼
ModIndex.references Map<定义 key, ReferenceSite[]>
├── CodeLens 计数(O(1) 查表)
├── 语义 Find All References(返回精确位置)
└── 未引用资产报告(0 引用 = 不在反向索引中)
```
引用记录在解析期就把 `refType` / `selfType` 固化下来,反向索引构建时不需要
再解析 DOM、也不需要上下文类型解析;只需对 `assetsById` 做一次查找并按类型
过滤。快照发布时(XML 阶段 + art 阶段)各构建一次反向索引,Corona 规模下
开销远小于 include 遍历。
反向索引只从**本次构建的 walk 实际消费的 records**(`ModIndexer.buildRecords`)
构建,不读共享 recordsCache:中途 watcher 失效、或 feature 通过 `readDom`
重读文件,都不会让“资产在但引用缺失”的快照出现。打开文档时 CodeLens / FAR
还会比较当前文本的 records 哈希与快照,不一致就定向 invalidate 并触发
`records-desync` 重建自愈(仅对已保存文档,未保存编辑不触发)。
## 四、CodeLens 规则
- 只对根级(`AssetDeclaration` 直接子元素)带 `id` 的资产显示;
- 只对 `isReferenceTargetType()` 为真的类型显示(类型化引用目标 +
`inheritFrom` 可继承类型;见 `refs.ts` 的 `referenceTargetTypes()`);
- 0 也显示:`0 references` / `1 reference` / `N references`;
- 点击执行 `ra3modxml.showReferences` → `editor.action.showReferences`
打开 references peek,结果与计数完全一致(不含定义本身);
- 计数除了当前定义自己的反向索引桶,还并入“manifestSource 可解析到当前
文件”的 manifest 定义桶:manifest 资产有对应 SageXml 源码时,引用直接
视作 SageXml 源码对该 asset 的引用(Go to Definition 同样把 manifest
定义映射到 SageXml 源码);
- 索引重建完成后自动 `editor.action.codeLens.refresh`,计数不会停留在旧值。
## 五、未引用资产
- 主入口:命令面板 `RA3 Mod XML: Find unreferenced assets…`
第一步 QuickPick 选类型(只列“设计上应被引用”的类型,显示每种未引用数量),
第二步列出资产(`id — 相对路径:行号`),点击跳转到定义;
- 加速入口:编辑器右键菜单
`RA3 Mod XML: Find unreferenced assets of this type`
(仅 `editorLangId == xml && ra3modxml.active` 时显示),光标在顶部资产上
时直接预选该类型,否则回退到类型选择;
- 语义:只统计 `origin === "project"` 且非 `viaInstance` 的定义;
manifest / SDK 定义永远不参与;如果某个 id 覆盖了原版资产,来自原版/其他
流的引用同样计入(否则会把覆盖件误报成未使用)。
## 六、已知边界与后续
- 当前文档局部作用域(不在任何全局流里的文件)的引用不在全局反向索引中:
这类文件打开后 CodeLens / FAR 只反映全局流;局部链内的引用暂不计数。
- Find All References 不返回定义行本身(即使 VS Code 传入
`includeDeclaration`),因此结果数量与 CodeLens 计数严格一致。
- `AssetIdList` 等“任意资产 ID 列表”的引用语义仍未建模:整个属性值按一条
引用记录处理,与现有补全/诊断保持一致(已知缺口,后续可在 records 层扩展)。
- w3x 文件本身不提取引用记录(浅扫描无 DOM);对 w3x 资产的引用从引用它的
XML 文件捕获。
- manifest 二进制里包含 BAB 编译后的完整引用图(`AssetReferenceOffset/Count` +
8 字节 `TypeId+InstanceId` 条目),目前仍跳过。将来可解析它给原版/manifest
资产提供“编译器权威”计数;mod 源码资产在编译前没有 manifest,仍需源码级
引用索引。
- 未引用资产目前是命令 + QuickPick 的“查询”形态;如果之后想要常驻浏览,
可以再加 Tree View(更重,暂不计划)。
+251 -25
View File
@@ -1,6 +1,6 @@
# 调研结论与实施计划(已按最新代码同步更新) # 调研结论与实施计划(已按最新代码同步更新)
> 说明:本文档随实现演进持续同步。最近一次同步(2026-08-01)对齐了实现过程中新增的模块与设计变更:BAB 精确搜索路径、manifest 类型/ID 推导、上下文感知元素类型、属性级 refType / Poid 局部引用(`id` 定义点)、精确跳转范围、嵌套 `xi:include`、注入式语法高亮等。 > 说明:本文档随实现演进持续同步。最近一次同步(2026-08-05)对齐了实现过程中新增的模块与设计变更:BAB 精确搜索路径、manifest 类型/ID 推导、上下文感知元素类型、属性级 refType / Poid 局部引用(`id` 定义点)、精确跳转范围、嵌套 `xi:include`、注入式语法高亮、bit-flag 列表补全(空格触发 / 排除已用 / 追加模式)、simple-content 元素文本引用(补全 / hover / 跳转 / 诊断 / Find All References)、语义引用索引 / CodeLens 引用计数 / 未引用资产命令等。
## 一、调研结论(带证据) ## 一、调研结论(带证据)
@@ -82,15 +82,27 @@ src/
asset-types.json 由 tools/extract-asset-types.mjs 生成(TypeId 哈希→类型名) asset-types.json 由 tools/extract-asset-types.mjs 生成(TypeId 哈希→类型名)
indexer/ indexer/
includeResolver.ts Include 路径解析(纯 TS,BAB /data /art /audio 顺序) includeResolver.ts Include 路径解析(纯 TS,BAB /data /art /audio 顺序)
existence.ts 文件集存在性快照(目录枚举 Set,替代逐路径 statSync)
manifestParser.ts .manifest 二进制解析 + 类型/ID 推导(纯 TS) manifestParser.ts .manifest 二进制解析 + 类型/ID 推导(纯 TS)
fileScanner.ts 目录遍历缓存 + Include source 候选收集 fileScanner.ts 目录遍历缓存 + Include source 候选收集
refs.ts 引用目标解析(按 refType / isRef / inheritFrom 过滤,纯 TS) refs.ts 引用目标解析(属性 + 元素文本内容,按 refType / isRef /
indexer.ts 工作区索引器(资产/Define/流/manifest 合并,LRU 解析缓存) inheritFrom 过滤,纯 TS)+ “设计上可被引用类型”判定
referenceIndex.ts 引用记录 → 反向引用索引(定义 → 引用位置)+ 未引用报告
shallowScan.ts 大体积美术资产(.w3x 等)顶层浅扫描(纯 TS,不建 DOM)
records.ts 每文件紧凑索引记录(资产/Define/Include/xi/引用 + 行号偏移)
caches.ts 跨重建持久缓存(DocumentCache / IndexRecordsCache /
IncludeResolveCache)+ 失效纪元 InvalidationsEpoch
diskCache.ts 跨会话磁盘缓存(gzip JSON、原子写、多信号 stat 校验)
indexer.ts 工作区索引器(资产/Define/流/manifest/w3x 合并,
分阶段:XML → art,中间快照可发布)
types.ts 共享类型 types.ts 共享类型
features/ features/
completion.ts 补全 provider(元素/属性/值,上下文感知;xs:list 多值按当前段过滤) completion.ts 补全 provider(元素/属性/值,上下文感知;xs:list 多值按当前段过滤)
hover.ts hover provider hover.ts hover provider
navigation.ts 定义/引用/文档链接/大纲 navigation.ts 定义/引用/文档链接/大纲
references.ts 语义 FAR / 引用上下文 / CodeLens 命令共享逻辑
codeLens.ts CodeLens 引用计数(类型过滤,0 也显示,点击开 references peek)
unreferenced.ts 未引用资产 QuickPick 命令 + 右键菜单入口
diagnostics.ts 实时诊断 diagnostics.ts 实时诊断
semanticTokens.ts 语义 token provider(文档有解析错误时接管着色) semanticTokens.ts 语义 token provider(文档有解析错误时接管着色)
syntaxes/ syntaxes/
@@ -100,15 +112,21 @@ tools/
extract-asset-types.mjs OpenSAGE AssetType.cs → asset-types.json extract-asset-types.mjs OpenSAGE AssetType.cs → asset-types.json
test/ test/
fixtures/minimod 样例 Mod(include 各种情形、同名 ID、嵌套 xi:include、manifest 回退) fixtures/minimod 样例 Mod(include 各种情形、同名 ID、嵌套 xi:include、manifest 回退)
*.test.mjs 11 个测试文件(xmlParser / context / completion / semanticTokens / *.test.mjs 14 个测试文件(xmlParser / context / completion / semanticTokens /
includeResolver / manifestParser / indexer / schemaModel / refs / includeResolver / manifestParser / indexer / schemaModel / refs /
typeContext / manifestTypes) typeContext / manifestTypes / referenceIndex / codeLens /
referenceProvider)
``` ```
### 关键设计决策 ### 关键设计决策
1. **语言激活范围**:不劫持 `*.xml`。通过 `workspaceContains:**/Data/Mod.xml`、`**/*.babproj` 激活;语法高亮为**纯注入** grammar(不声明 `language`,避免覆盖内置 XML 语法)。 1. **语言激活范围**:不劫持 `*.xml`。通过 `workspaceContains:**/Data/Mod.xml`、`**/*.babproj` 激活;语法高亮为**纯注入** grammar(不声明 `language`,避免覆盖内置 XML 语法)。
2. **索引范围与默认值**:索引“项目 Data + additionalmaps + 沿 include 可达的 SageXml 原版源码”;SDK 路径默认 `C:\Apps\RA3-MODSDK-X`(可配置)。`reference` include 解析为 `builtmods` 下对应 manifest(惰性解析、按文件缓存),manifest 缺失/无效时回退到占位 XML。 2. **索引范围与默认值**:索引“项目 Data + additionalmaps + 沿 include 可达的 SageXml 原版源码”;SDK 路径默认 `C:\Apps\RA3-MODSDK-X`(可配置)。`reference` include 解析为 `builtmods` 下对应 manifest(惰性解析、按文件缓存),manifest 缺失/无效时回退到占位 XML。
**美术资产(.w3x)**:`<Include type="all">` / `ART:` 指向的 `.w3x`(及内容嗅探为
XML 的未知扩展名文件)按其顶层资产入库(`W3DContainer` / `W3DMesh` /
`W3DHierarchy` / `W3DCollisionBox` 等),使 `Model@Name`、`Hierarchy`、`Mesh`
等引用可解析;大模型文件**浅扫描**(不建 DOM),结果缓存在 workspace 级、
跨重建复用(详见设计决策 14)。
3. **manifest 资产建模**:类型优先用哈希表,未知时从名称前缀推导;可引用 ID 取最后冒号段;类型名统一走大小写规范化(`W3dContainer` ↔ `W3DContainer`),类型匹配严格遵循 XSD 继承链。 3. **manifest 资产建模**:类型优先用哈希表,未知时从名称前缀推导;可引用 ID 取最后冒号段;类型名统一走大小写规范化(`W3dContainer` ↔ `W3DContainer`),类型匹配严格遵循 XSD 继承链。
4. **上下文感知元素类型**:同名元素按父元素类型解析(`resolveElementType` 沿解析树逐层 `childTypeOf`,失败回退全局映射),保证 `<Weapon>` 等元素的属性/引用判定正确。 4. **上下文感知元素类型**:同名元素按父元素类型解析(`resolveElementType` 沿解析树逐层 `childTypeOf`,失败回退全局映射),保证 `<Weapon>` 等元素的属性/引用判定正确。
5. **引用判定与解析**:`refType` 或 `isRef` 均视为引用;带 `refType` 时严格按类型过滤(同名 ID 不串类型);`inheritFrom` 按可继承类型过滤。**局部作用域例外**(`isLocalReferenceAttribute`):`id` 是元素自身的定义点——无 refType 或 refType 与自身类型兼容时不检查、不解析(`RoadObject@id→Road` 这类跨类型 id 引用保留检查);Poid 类型属性是管线局部引用,全局索引无法判定,不检查、不解析。 5. **引用判定与解析**:`refType` 或 `isRef` 均视为引用;带 `refType` 时严格按类型过滤(同名 ID 不串类型);`inheritFrom` 按可继承类型过滤。**局部作用域例外**(`isLocalReferenceAttribute`):`id` 是元素自身的定义点——无 refType 或 refType 与自身类型兼容时不检查、不解析(`RoadObject@id→Road` 这类跨类型 id 引用保留检查);Poid 类型属性是管线局部引用,全局索引无法判定,不检查、不解析。
@@ -120,14 +138,145 @@ test/
11. **`xs:list` 建模与多值补全**:list 简单类型继承 itemType 的枚举 / refType / isRef / 11. **`xs:list` 建模与多值补全**:list 简单类型继承 itemType 的枚举 / refType / isRef /
allowsDefine 并标记 `isList`(`LocomotorSurfaceBitFlags`、`KindOfBitFlags` 等 79 个 allowsDefine 并标记 `isList`(`LocomotorSurfaceBitFlags`、`KindOfBitFlags` 等 79 个
类型、317 处属性声明受益);补全只对“最后一个空格段”过滤,替换范围只覆盖当前段, 类型、317 处属性声明受益);补全只对“最后一个空格段”过滤,替换范围只覆盖当前段,
支持 `Surfaces="GROUND ` 之后继续输入 `W` 提示 `WATER`。 支持 `Surfaces="GROUND ` 之后继续输入 `W` 提示 `WATER`。第十三轮(2026-08-04)
补全触发与编辑体验:空格注册为触发字符;列表过滤排除已出现的 flag;当前段已是
完整枚举值且没有更长变体时进入“追加模式”(零宽 range + `insertText=" FLAG"`,
可直接在闭合值末尾/中间追加);替换范围止于光标,中间插入不会删除尾部 flag。
12. **未闭合引号的行尾恢复**:起始标签扫描到 EOF 且引号未闭合时,在第一个换行处截断 12. **未闭合引号的行尾恢复**:起始标签扫描到 EOF 且引号未闭合时,在第一个换行处截断
标签并继续解析,未闭合只影响当前行(仍上报 `Unterminated start tag`),后续元素 标签并继续解析,未闭合只影响当前行(仍上报 `Unterminated start tag`),后续元素
的补全 / hover / 诊断不中断。 的补全 / hover / 诊断不中断。第十三轮补充:恢复出的元素带 `recoveredStartTag`
标记并补挂父链;补全上下文对“光标在恢复元素内但越过 `startTagEnd`”的情况按
`text.slice(tagStart, cursor)` 重新解析部分标签,使多行书写的未闭合属性
(如 `Disposition="`)仍可获得 attribute-value 补全,且不影响全局解析。
13. **语义 token 兜底高亮**:TextMate 对未闭合引号会把后续内容当字符串吞掉(任何 13. **语义 token 兜底高亮**:TextMate 对未闭合引号会把后续内容当字符串吞掉(任何
XML 编辑器皆然);扩展注册 `DocumentSemanticTokensProvider`,仅当解析报错时用 XML 编辑器皆然);扩展注册 `DocumentSemanticTokensProvider`,仅当解析报错时用
语义 token 覆盖标签名 / 属性名 / 属性值(标准 token 类型 `type` / `property` / 语义 token 覆盖标签名 / 属性名 / 属性值(标准 token 类型 `type` / `property` /
`string`,主题自带配色)。合法文件返回空,观感与纯 TextMate 完全一致。 `string`,主题自带配色)。合法文件返回空,观感与纯 TextMate 完全一致。
14. **大体积美术资产浅扫描 + 跨重建持久缓存**(第八轮,2026-08-02):
- 实测 Corona:3788 个 w3x / 2.64 GB,163 个超过 4 MB,最大 22.8 MB;大文件是
`W3DHierarchy` + 若干 `W3DMesh`(`Vertices/V`、`Triangles/T` 等 unbounded 数值
载荷),顶层记录通常只有几个到二十几个。
- 全量解析内存放大约 17 倍(6.3 MB 文本 → +109 MB DOM),不可接受;`scanXmlShallow`
单次线性扫描只提取顶层 `name+id`、`<Includes>`、`<xi:include>`、`<Defines>`,
22 MB 文件 ~600 ms、保留内存≈0。
- 索引按扩展名三分:`.xml` 全量解析(4 MB 上限不变);`.w3x` 浅扫描;
未知扩展名嗅探文件头(`<` 开头、无 NUL)决定按 XML 浅扫描或二进制登记
(manifest 是 `*.manifest` 二进制,不存在 `.manifestxml` 源码格式)。
- `DocumentCache` / `ShallowScanCache` 由 `ModWorkspace` 持有,每次重建传入新的
`ModIndexer`;按 `mtimeMs + size` 校验,未变化不重读。Corona 第二次构建
w3x 重扫数为 0(4,829 次缓存命中)。
- 读取整个文件不可避免(顶层边界需要全量扫描),但建 DOM 不是;优化的是
"不分配子节点对象"与"跨重建不重读",两者叠加后方案可行。
15. **重建零 stat + 记录驱动索引**(第九轮,2026-08-02,v0.1.1):
- 插桩发现每次重建(含信任重建)都有约 11 万次同步 `statSync`
(`resolveSource` 的 include 存在性检查),机械盘上占 10-20s;
`IncludeResolveCache` 按(目录 + source)缓存解析结果,内容编辑不清、
创建/删除文件与强制 reindex 才清。信任重建 statSync 降到 0。
- `IndexRecordsCache` 缓存每文件紧凑索引记录(顶层资产 / Define / Include /
xi:include + 1-based 行号),信任重建完全不接触 DOM;`DocumentCache`
双重淘汰(条数 LRU + 元素预算,超预算先淘汰最大树)把 DOM 常驻内存封顶。
- w3x 缓存不再保留 LineMap(Corona 全量约 700MB),浅扫描直接产出带行号的记录。
- 实测:Corona 信任二次构建 21s → **2.0s**(statSync 0);首建后 2.5GB
堆保留确认为构建期可回收垃圾,常驻 ~100MB;强制 reindex ~5-25s。
- 候选目录扫描并行化;`stats` 新增 `candidatesMs` / `walkMs` / `resolveCalls` /
`resolveCacheHits` 供索引报告定位耗时。
16. **分阶段索引与部分可用性**(第十轮,2026-08-03):索引分两阶段发布——
阶段 A(xml)只走 XML + manifest include 链,w3x 只登记进待扫队列;
阶段 B(art)浅扫描队列并继续走 w3x 内的 include。阶段 A 结束即发布
不可变快照(`snapshotIndex` 深拷贝嵌套 Map/数组),XML/枚举/语法类功能
在首建早期即可用;引用类诊断在 `!complete || stale` 时“显示但标注”
(code 为 `*-indexing`,消息注明 index incomplete)。快照携带
`complete` / `phase` / `stale`,状态栏显示阶段。
17. **构建中失效与 stale 标记**(第十轮):watcher 的 change / create /
delete 均触发防抖重建;`InvalidationsEpoch` 记录失效次数,快照发布时若
期间有失效则标记 stale,由 dirty 机制随后重建收敛;构建失败保留上一个
快照而非清空索引。
18. **多信号文件 stamp**(第十轮):`IndexedFile.stat` 扩展为
`{ size, mtimeMs, birthtimeMs, ctimeMs }`,任一不匹配即重读,为磁盘
持久化缓存铺路(FAT32 mtime 2s 粒度、工具保留 mtime 等场景)。
19. **文件集快照替代 statSync**(第十一轮,2026-08-03):`ExistenceSnapshot`
用**惰性按目录 readdir**(只读查询到的父目录,按目录缓存)回答 include
存在性,覆盖根之外才 statSync 回退;盘符根不枚举。首版全量递归枚举使
XML 阶段从 27s 涨到 45s,已改为惰性模式(复测 24.0s)。
`stats.snapshotHits / snapshotFallbacks` 入报告;Corona 首建 75,926 次
查询全部由快照回答、0 回退。
20. **磁盘持久化缓存**(第十一轮):`DiskRecordsCache` 持久化 records 缓存
(gzip JSON、原子写、identity key);启动时并发 stat 多信号校验,不匹配
丢弃重读;构建后异步回写。Corona 缓存仅 651 KB,冷启动约 11s(原
~2.5 分钟)。
21. **缓存命令**(第十一轮):`ra3modxml.clearCache`(清内存 + 磁盘 +
强制重建)、`ra3modxml.showCacheReport`(路径/大小/校验统计/命中数)。
22. **重建插桩**(第十一轮补充):`buildCount` / `lastBuildTrigger`
(initial、save、watcher-*、config、reindex、clear-cache、
dirty-followup)进入索引/缓存报告;“RA3 Mod XML” 输出通道记录每次构建
的触发原因、phase A 发布时间与完成耗时,用于定位“首建后又重建一次”
之类的现象。
23. **watcher 噪声过滤与 URI 日志**(第十一轮补充):watcher 事件把触发
URI 写入输出通道;路径含 `.git` 段的事件直接忽略(后台 fetch /
maintenance 会周期性触碰 `.git`,不应触发重建或 stale 标记)。
24. **watcher 内容白名单**(第十一轮补充):临时文件命名模式
(`.git`/`.tmp`/`.lock`/`~`/`.swp`/`.bak`/`.orig` 后缀、`.#`/`.~`
前缀)全部忽略;`onDidChange` 只响应扩展名白名单(`.xml` / `.w3x`,
RA3 合理文本格式为 xml/w3x/lua,lua 暂未索引、manifest 为二进制)或已在
索引中的文件;创建/删除仍响应所有真实文件(影响 include 存在性)。
25. **当前文档局部链 + 逻辑树展开(第十二轮,T1)**:新增 `xpointer.ts` /
`logicalTree.ts` / `localScope.ts`。打开文件时按当前文本建立局部 overlay
(自身资产 / `$DEFINE` / include 链),并生成展开 `xi:include` 的逻辑树;
features 经 `ws.getScope(document)` 拿到 overlay-aware 索引。Poid 引用
(`AttachModuleId` / `ModuleId` / `AutoResolveBody` 等)在最近 GameObject
子树内解析;未命中不新增诊断(保守策略,避免跨文件误报)。
顶层 `<Include type="all">` 暂不并入逻辑树(保留为后续扩展)。
26. **属性补全的插入布局与类型化默认值(第十四轮,2026-08-04)**:
`attributeInsertLayout` 按临近属性的排版决定插入方式——贴引号时补空格、
一行一个属性时补换行 + 缩进、已在新行时用临近缩进替换当前行空白、
inline 风格的手动换行保留用户缩进;`attributeValuePlaceholder` 对引用/
枚举/list/布尔等建议类属性保留 `$1` + 自动触发,对标量属性填 XSD 默认值
或类型示例(`0d` / `0s` / `100%` / `0.0` / `0`),具体默认值不再弹空
suggest。
第十五轮(2026-08-04)最终结论:VS Code 插入含换行的补全文本时会把当前
行基础缩进与文本内嵌缩进相加(3+3=6、6+3=9…),因此换行前缀只插入 `\n`、
缩进交给编辑器;同时半截属性名(`hasValue=false`)不再作为缩进锚点,改用
第一个独占一行的完整属性作为规范缩进,插入换行时顺带吞掉触发补全留下的
尾随空格;属性名补全改用 `SnippetString`(`$1` 成为真正占位符),并新增
输出通道调试日志。
27. **simple-content 元素文本引用(第十六轮,2026-08-04)**:simple type
子元素(如 `ObjectCreationList` 内嵌套 `<CreateObject>`,类型
`GameObjectWeakRef`)的**标签间文本**就是资产引用。内容区补全现在区分
“复杂元素 → 子元素名”与“简单元素 → 值补全”;用户已输入 `<` 时替换范围从
`<` 开始,杜绝 `<<`;simple type 元素片段固定为 `<Name>$1</Name>`(可填
值)并自动触发值补全。hover / Ctrl 跳转 / 诊断 / Find All References
均增加内容 token 分支。只有**带 `xas:refType`** 的 simple 内容按全局引用
处理(291 处子元素声明);无类型 `AssetReference`
(`FXShaderConstantTexture@Value`、`RenderSubObjectReference@Mesh` 等
真实数据是贴图/子对象名)与 `Poid` 不参与全局解析,避免误报。
补充:真实文件中 `<` 后还有 `</…>` 时,`findTagEnd` 曾把闭合标签的 `>` 误
当成残缺开始标签的结束,生成空名/半截名伪元素,补全走 element-name 分支
导致 `<<`。修复为引号外遇到 `<` 即视为未闭合(行尾恢复),且
`elementNameItems` 的替换范围包含 `<`。实机再回归:范围含 `<` 会让 VS Code
用 `<` 做过滤前缀导致菜单为空——改为保留已输入的 `<`、range 从 `<` 之后
开始、插入文本不带开括号;`textContentTokenAt` 对未闭合元素用 `el.end`
作内容边界。
28. **语义引用索引 + CodeLens 计数 + 未引用资产(第十八轮,2026-08-05)**:
- 动机与结论:292 个有模型类型的顶层资产中 191 个在 XSD 中没有任何
类型化引用指向(设置/地图元数据/w3x 子结构等自动注册类型),
AttachTest 实测 29% 项目定义 0 引用、48% 恰好 1 引用。因此 CodeLens
计数只显示在 `isReferenceTargetType()` 为真的类型上(类型化引用目标 +
`inheritFrom` 可继承类型),0 也显示;未引用报告默认也只列这些类型。
- 数据:`IndexRecords` 增加 `references[]`
(`kind / refType / selfType / value / line / start / end`),解析期固化
引用上下文,反向索引构建零 DOM、零上下文重解析;`ModIndex.references`
为“定义 key → 引用位置”的反向表,快照发布时构建;磁盘缓存版本
v1 → v2(`index-records-v2.json.gz`),v3 起附加内容/records 哈希。
- FAR 从全文搜索改为语义索引:不再把 `id="X"` 定义行、`EditorName="X"`
等非引用属性计为引用;CodeLens 显示的计数与点击打开的 references peek
严格一致;`includeDeclaration` 时才附加定义位置。
- 未引用资产:命令面板 `Find unreferenced assets…` + 编辑器右键菜单
`Find unreferenced assets of this type`;只统计 `origin === "project"`
且非 `viaInstance` 的定义;覆盖原版 id 时来自原版/其他流的引用计入。
- 边界:局部 scope(不在全局流里的文件)的引用不入全局反向索引;
`AssetIdList` 等列表引用按整值记录;w3x 不提取引用记录(对 w3x 资产的
引用从引用它的 XML 捕获);manifest 编译期引用图(AssetReference 缓冲)
留作后续。设计文档:`docs/features-reference-counts.md`。
## 三、实施步骤 ## 三、实施步骤
@@ -136,38 +285,115 @@ test/
3. [x] 生成模型:`schema-model.json`(XSD,295 顶层元素 / 1851 类型)与 `asset-types.json`(79 个 AssetType 哈希)。 3. [x] 生成模型:`schema-model.json`(XSD,295 顶层元素 / 1851 类型)与 `asset-types.json`(79 个 AssetType 哈希)。
4. [x] 纯 TS 核心:include 解析、manifest 解析、XML 解析封装、索引器、引用解析。 4. [x] 纯 TS 核心:include 解析、manifest 解析、XML 解析封装、索引器、引用解析。
5. [x] 功能层:补全、hover、导航、诊断、大纲、高亮 grammar。 5. [x] 功能层:补全、hover、导航、诊断、大纲、高亮 grammar。
6. [x] 单测(fixture Mod,53 个用例全绿:含 xs:list 枚举、未闭合引号恢复、list 多值 6. [x] 单测(fixture Mod,73 个用例全绿:含 xs:list 枚举、未闭合引号恢复、list 多值
分段、带 vscode stub 的补全集成、语义 token 兜底)+ 编译 + `vsce package` 打包 分段、带 vscode stub 的补全集成、语义 token 兜底)+ 编译 + `vsce package` 打包
(ra3-mod-xml-0.1.0.vsix,约 495KB)。 (ra3-mod-xml-0.1.1.vsix,约 499KB)。
7. [x] 在 AttachTest / GenEvoTest / Corona 上做冒烟验证,并按真实项目反馈修复问题(详见 `docs/analysis-issues.md` 六轮分析)。 7. [x] 在 AttachTest / GenEvoTest / Corona 上做冒烟验证,并按真实项目反馈修复问题(详见 `docs/analysis-issues.md` 八轮分析)。
8. [x] w3x 美术资产索引(第八轮):新增 `shallowScan.ts` / `caches.ts`,w3x 与内容嗅探
XML 走浅扫描,Include source 补全候选加入 w3x,BOM 剥离,缓存跨重建持久化;
AttachTest 报错场景(`AUGunship_SKN`)修复,Corona 首次/二次构建实测验证。
9. [x] Corona 性能与内存优化(第九轮,v0.1.1):`records.ts` 记录驱动索引、
`IncludeResolveCache` 零 stat 重建、DOM 元素预算淘汰、w3x LineMap 移除、
候选并行扫描、阶段计时;Corona 信任重建 2.0s;确认首建 2.5GB 为可回收垃圾。
10. [x] 部分可用性 + 分阶段索引(第十轮,2026-08-03):T0 解耦(语法/模型
诊断、枚举/子元素补全、Include 链接无索引可用)、A/B 分阶段发布、watcher
触发重建 + 失效纪元 stale 标记、stat 多信号扩展;测试 73 → 79。
11. [x] 冷启动提速(第十一轮,2026-08-03):文件集快照替代 statSync、
磁盘持久化缓存(多信号校验、原子写)、clearCache/showCacheReport 命令;
测试 79 → 90;Corona 冷启动 ~11s。
12. [x] 惰性存在性快照 + 重建插桩(第十一轮补充,2026-08-03):全量目录枚举
改为惰性按目录 readdir(phase A 45s → 24.0s);新增 buildCount /
lastBuildTrigger 与输出通道日志;索引/缓存报告显示构建序号与触发原因。
13. [x] watcher 噪声过滤 + URI 日志(第十一轮补充,2026-08-03):输出通道
记录 `[watcher-*] <path>`;`.git` 段路径忽略;测试 90 → 91。
14. [x] watcher 内容白名单(第十一轮补充,2026-08-03):临时文件命名模式
过滤 + 内容变更扩展名白名单 + `isIndexedFile` 兜底;测试 91 → 92。
15. [x] T1 当前文档局部链 + include 展开(第十二轮,2026-08-03):局部
overlay(不在任何流里的文件也能解析自身引用)+ `xi:include` 逻辑树展开
+ Poid 局部作用域补全/悬停/跳转;测试 92 → 98。2026-08-04 补充构建期
闸门:`getScope` 在重建进行中只返回 parse-only scope,避免与 indexer
抢盘(版本 0.1.9)。
16. [x] bit-flag 列表补全修复(第十三轮,2026-08-04):空格触发字符、多行
未闭合标签的部分标签重解析(`recoveredStartTag` 标记 + 恢复元素父链)、
闭合值内追加模式与中间插入范围修复;测试 98 → 107(版本 0.1.10)。
17. [x] 属性补全插入体验(第十四轮,2026-08-04):闭合引号后自动补空格、
临近属性缩进对齐、标量属性类型化默认值;测试 107 → 111(版本 0.1.11)。
18. [x] 属性补全缩进叠加修复(第十五轮,2026-08-04):换行前缀只插入 `\n`
(编辑器自动补基础缩进,避免 3+3=6 式叠加)、完整属性锚点 + 首个独占一行
属性为规范缩进、`$1` 改为 SnippetString 占位符、输出通道调试日志、尾随
空格吞除;测试 111 → 113(版本 0.1.12–0.1.13)。
19. [x] simple-content 文本引用修复(第十六轮,2026-08-04):`<` 后补全范围
覆盖 `<`、simple type 元素片段 `<Name>$1</Name>`、简单元素内容值补全、
`refs.ts` 新增 `isReferenceContentType` / `resolveContentReferenceTargets`、
hover / 定义跳转 / 诊断 / Find All References 内容分支;
测试 113 → 121(版本 0.1.14)。实机回归补充:`findTagEnd` 引号外遇 `<`
视为未闭合 + 插入文本不带开括号(range 从 `<` 之后开始);测试 121 → 125,
再回归 125 → 128(未闭合元素内容 token)。
20. [x] simple-content 零宽边界与大列表补全(第十七轮,2026-08-04):
`>` 后光标归入内容区、已闭合元素 `end` 改为开区间;超过 400 条的
id/define/local/include 候选返回 `isIncomplete` 让 VS Code 随输入重请求;
当前文档 local 资产优先、候选 top-N 用堆避免全量排序、Include source
先排序再截断;测试 128 → 136。
21. [x] 语义引用索引 + CodeLens + 未引用资产(第十八轮,2026-08-05):
records 引用记录提取、`referenceIndex.ts` 反向索引、语义 FAR(替换全文
搜索)、CodeLens 引用计数(类型过滤、0 也显示、点击开 references peek)、
`findUnreferencedAssets` 命令 + 右键菜单、磁盘缓存 v2;
测试 136 → 146;设计文档 `docs/features-reference-counts.md`。
22. [x] FAR 定义行排除 + manifest 源引用归并(2026-08-05 修复轮):
FAR 不再因 `includeDeclaration` 附加定义行(消除“自引用”观感);
`referenceSitesForDefinition` 把 manifestSource 可解析到当前文件的
manifest 定义站点并入 CodeLens 计数(AttachTest 326 个 manifest 定义
受益);测试 146 → 147;版本 0.1.17;分析见
`docs/analysis-issues.md` 二十三。
23. [x] 引用索引与缓存同步加固(2026-08-06):反向索引改为从**构建期本地
records**(`buildRecords`)构建,不再读共享 recordsCache——中途失效、
feature `readDom` 重读都不再造成“资产在但引用缺失”;构建中 `readDom`
不再被定义跳转调用(避免污染进行中的构建);force 重建(Re-index)
对 stat 匹配的 full XML 做内容哈希校验;打开文档时比较 records 哈希,
不一致则定向 invalidate + `records-desync` 重建自愈;磁盘缓存 v2 → v3;
测试 147 → 151;分析见 `docs/analysis-issues.md` 二十四。
## 四、验证结果(实测) ## 四、验证结果(实测)
| 项目 | 规模 | 索引耗时 | 资产数 | 说明 | | 项目 | 规模 | 索引耗时 | 资产数 | 说明 |
|---|---|---|---|---| |---|---|---|---|---|
| AttachTest | 71 文件 | ~0.6s | 35,546(manifest 35,322) | 2 个流(static + mapmetadata) | | AttachTest | 88 文件 | ~1.2s | 35,607(manifest 35,322,w3x 浅扫 62) | 2 个流(static + mapmetadata);二次构建 354ms / 62 缓存命中 |
| GenEvoTest | 24 文件 | ~1.8s | 35,392(manifest 35,322) | 项目 ID(alliedmcv 等)正确收录 | | GenEvoTest | 66 文件 | 首次 ~2.6s / 二次 ~0.4s | 35,502(manifest 35,322,w3x 浅扫 38) | 2 个流、73 个 Define;二次构建 0 重扫 / 38 缓存命中 |
| Corona | 3,448 文件(+ 非 XML 资产路径) | ~54.5s | 55,305(manifest 35,322) | 3 个流、183 个 Define、0 诊断 | | Corona | 8,976 文件 | 首次 ~250s / 信任二次 ~2s / 强制 ~5-25s | 64,868(manifest 35,322,w3x 浅扫 4,829) | 3 个流、183 个 Define、0 诊断;二次构建 statSync 0、resolveHits 15,333 |
单元测试覆盖:XML 解析(自闭合/容错/偏移/未闭合引号行尾恢复)、补全上下文(未闭合引号仍为 attribute-value、list 多值分段)、补全集成(vscode stub 下 `LocomotorTemplate@Surfaces` 未闭合引号枚举补全、空格后第二段过滤与替换范围)、语义 token(标签/属性/值范围、合法文档返回空、malformed 返回兜底 token)、include 解析(BAB 顺序、SDK 根优先于 SageXml)、manifest 二进制解析(合成 v5 样本、类型/ID 推导)、索引器(资产/Define/流/缺失 include/嵌套 xi:include)、XSD 模型(上下文类型、`childTypeOf`、大小写规范化、属性级 refType、外来命名空间判定、`xs:list` 枚举继承与 `isList` 标记)、引用过滤(`Weapon="X"` 只跳 `WeaponTemplate`、模块 `id` 定义点、Poid 局部引用、`xi:include` 不校验、`Side="Allies"` 命中 manifest 的 `PlayerTemplate`)。 单元测试覆盖:XML 解析(自闭合/容错/偏移/未闭合引号行尾恢复 + recovered
标记)、补全上下文(未闭合引号仍为 attribute-value、多行未闭合引号、list 多值
分段、闭合引号后为 attribute-name)、补全集成(vscode stub 下
`LocomotorTemplate@Surfaces` 未闭合引号枚举补全、空格后第二段过滤与替换范围、
空格后排除已用 flag、中间插入范围止于光标、闭合值末尾追加模式、`CAN_ATTACK`
前缀保护、多行未闭合 `Disposition` 完整链路、闭合引号后补空格、一行一个属性
换行缩进、新行缩进对齐、标量类型化默认值)、语义 token(标签/属性/值范围、
合法文档返回空、malformed 返回兜底 token)、include 解析(BAB 顺序、SDK 根
优先于 SageXml)、manifest 二进制解析(合成 v5 样本、类型/ID 推导)、索引器
(资产/Define/流/缺失 include/嵌套 xi:include)、XSD 模型(上下文类型、
`childTypeOf`、大小写规范化、属性级 refType、外来命名空间判定、`xs:list`
枚举继承与 `isList` 标记)、引用过滤(`Weapon="X"` 只跳 `WeaponTemplate`、
模块 `id` 定义点、Poid 局部引用、`xi:include` 不校验、`Side="Allies"` 命中
manifest 的 `PlayerTemplate`)、simple-content 文本引用(`<` 后补全不产生
`<<`、`<CreateObject>$1</CreateObject>` 片段、内容值按 refType 过滤、
内容 hover / Ctrl 跳转 / 诊断)、引用索引(records 引用提取:attr/content/
inheritFrom、排除 id/Poid/`$DEFINE`/枚举;`buildReferenceIndex` 严格类型过滤
不串同名 ID;minimod 集成:`CommandSet` / `inheritFrom` 反向命中;未引用过滤
只含 project 非 viaInstance;CodeLens 类型过滤与 0 显示;语义 FAR 排除定义
行与非引用属性)。
> 注:`D:\Mods\CoronaMod` 位于移动硬盘,当前未连接;GenEvoTest / Corona 的回归需在 D: 盘可用时补跑(用户会另行通知)。 > 注:D: 盘移动硬盘已恢复连接;Corona 已在第八 / 九轮按上述新数据回归。
## 五、假设与开放问题 ## 五、假设与开放问题
- 假设 SDK 路径默认 `C:\Apps\RA3-MODSDK-X`(与 prompts 一致),可在设置中修改。 - 假设 SDK 路径默认 `C:\Apps\RA3-MODSDK-X`(与 prompts 一致),可在设置中修改。
- 假设补全/导航以“文本语义分析”为主,不做完整 XSD 校验(BAB 才是权威校验器)。 - 假设补全/导航以“文本语义分析”为主,不做完整 XSD 校验(BAB 才是权威校验器)。
- 开放:是否发布到 VS Code Marketplace(需要 publisher)——本期先保证本地 `vsce package` 可安装。 - 开放:是否发布到 VS Code Marketplace(需要 publisher)——本期先保证本地 `vsce package` 可安装。
- 开放:**“宏展开”式虚拟合并**(用户提议,方向已确认):解析前先把 `xi:include` - 开放:**“宏展开”式虚拟合并**(用户提议,方向已确认):`xi:include` 已按逻辑树
(以及顶层 `<Include type="all">`、`inheritFrom` 继承合并)展开成不含 include 的 展开(第十二轮,见第六节);顶层 `<Include type="all">` 与 `inheritFrom` +
文档树,再对展开后的树做 XSD 校验、补全与诊断——与 BAB 编译时把整个 Mod 合并成 `xai:joinAction` 的深合并仍未实现,后续如需要“当前文档视角的全量合并诊断”再继续。
一份大 XML 的行为一致。展开树需携带**来源追溯**(错误/跳转仍定位到原始文件),
并处理 xpointer 子集解析与 include 循环。当前仅做到:目标文件可索引、可导航、
缺失可诊断;`xi:include` 本身不参与 XSD 校验(第五轮)。
该功能也是“GameObject 内模块 id 局部作用域解析”(第四轮遗留)的地基。
详细设计备忘见下一节。
## 六、include 展开设计备忘(2026-08-01,待实施) ## 六、include 展开设计备忘(2026-08-01;xi:include 部分已实施于第十二轮)
> 目的:集中记录 include 处理相关的现状、结论与设计,下次遇到 include 问题时从这里继续, > 目的:集中记录 include 处理相关的现状、结论与设计,下次遇到 include 问题时从这里继续,
> 并在实施后把结果回写本节。 > 并在实施后把结果回写本节。
@@ -180,7 +406,7 @@ test/
| `reference` → builtmods manifest 解析 / 缺失回退占位 XML | 已实现 | | `reference` → builtmods manifest 解析 / 缺失回退占位 XML | 已实现 |
| 嵌套 `xi:include`(任意层级):目标可索引、缺失报 `include-not-found`、Ctrl+点击跳转、`href` hover 解析目标 | 已实现(第二轮 + 第五轮) | | 嵌套 `xi:include`(任意层级):目标可索引、缺失报 `include-not-found`、Ctrl+点击跳转、`href` hover 解析目标 | 已实现(第二轮 + 第五轮) |
| `xi:include` 及其属性不参与 XSD 校验(外来命名空间守卫 `isXsdElementName` / `isXsdAttributeName`) | 已实现(第五轮) | | `xi:include` 及其属性不参与 XSD 校验(外来命名空间守卫 `isXsdElementName` / `isXsdAttributeName`) | 已实现(第五轮) |
| include 目标内容“虚拟合并”进父文档的逻辑树 | **未实现**(本文档主题) | | include 目标内容“虚拟合并”进父文档的逻辑树 | 已实现 `xi:include`(第十二轮);顶层 `<Include type="all">` 仍不展开 |
### 2. 已确认的方向 ### 2. 已确认的方向
+31 -1
View File
@@ -41,9 +41,15 @@ XML 之间的组织靠 `<Include>` 标签,共有三种语义:
- 枚举值(XSD `xs:enumeration`); - 枚举值(XSD `xs:enumeration`);
- `$DEFINE` 常量(如 `$CIV_HEALTH_SMALL`); - `$DEFINE` 常量(如 `$CIV_HEALTH_SMALL`);
- `<Include source>` 补全可解析的文件路径(`DATA:` / `ART:` / `AUDIO:`)。 - `<Include source>` 补全可解析的文件路径(`DATA:` / `ART:` / `AUDIO:`)。
- **元素文本内容(simple content)**:带 `xas:refType` 的简单内容元素
(如 `<CreateObject>ID</CreateObject>`)在标签间补全对应类型的资产 ID、
枚举或 `$DEFINE`;补全出的 simple-content 元素必须是可填值的成对标签
(`<Name></Name>`),且内容区已输入 `<` 时不得产生 `<<`。
3. **引用提示(Hover)**:元素/属性悬停显示 XSD 文档、类型、默认值;资产 ID 悬停显示定义位置;`$DEFINE` 悬停显示值与定义位置。 3. **引用提示(Hover)**:元素/属性悬停显示 XSD 文档、类型、默认值;资产 ID 悬停显示定义位置;`$DEFINE` 悬停显示值与定义位置。
- 元素文本内容(simple content 引用)悬停同样显示定义位置。
4. **引用导航**: 4. **引用导航**:
- 从引用型属性值跳转到对应资产定义(Go to Definition); - 从引用型属性值跳转到对应资产定义(Go to Definition);
- 从 simple-content 引用元素(`<CreateObject>ID</CreateObject>`)的文本跳转到对应资产定义;
- 查找某资产 ID 的所有引用(Find All References); - 查找某资产 ID 的所有引用(Find All References);
- `<Include source>` / `xi:include href` 直接打开目标文件; - `<Include source>` / `xi:include href` 直接打开目标文件;
- `inheritFrom` 跳转到被继承元素; - `inheritFrom` 跳转到被继承元素;
@@ -54,6 +60,7 @@ XML 之间的组织靠 `<Include>` 标签,共有三种语义:
- 缺失必填 `id`(顶层资产); - 缺失必填 `id`(顶层资产);
- 重复 ID(同类型 + 同 id,mod 文件之间;覆盖原版 SageXml 不算冲突); - 重复 ID(同类型 + 同 id,mod 文件之间;覆盖原版 SageXml 不算冲突);
- 引用未解析(引用了不存在的资产 ID,可配置是否忽略原版 manifest 中的 ID); - 引用未解析(引用了不存在的资产 ID,可配置是否忽略原版 manifest 中的 ID);
- simple-content 引用元素的文本未解析(同属性引用规则,仅带 refType 的类型)。
- `<Include>` 目标文件找不到、Include 循环; - `<Include>` 目标文件找不到、Include 循环;
- `$DEFINE` 未定义。 - `$DEFINE` 未定义。
@@ -70,9 +77,30 @@ XML 之间的组织靠 `<Include>` 标签,共有三种语义:
- `TypeId` 哈希 → 类型名的映射来自 OpenSAGE 的 `AssetType` 枚举(本工作区可提取); - `TypeId` 哈希 → 类型名的映射来自 OpenSAGE 的 `AssetType` 枚举(本工作区可提取);
- 资产名与源文件名各自存放在独立的空字符结尾字符串缓冲区中。 - 资产名与源文件名各自存放在独立的空字符结尾字符串缓冲区中。
**补充(美术资产 `.w3x` 索引)**:
- `W3X.xml` / `ART:` include 链中的 `.w3x` 是文本 XML(建模工具导出,BAB 同样按
XML 编译),其顶层资产(`W3DContainer` / `W3DMesh` / `W3DHierarchy` 等)应参与
补全、悬停、导航与诊断——`Model@Name`、`Hierarchy`、`Mesh` 等引用依赖这些定义;
- 大模型文件(实测 Corona 最大 22.8 MB,顶点/三角形数据占大头)采用**顶层浅扫描**
(不建 DOM 树),结果在 workspace 级缓存并跨重建复用,避免每次保存都重读整个
美术资产目录(Corona 约 2.6 GB);索引记录与 include 解析结果同样跨重建缓存,
保存触发的重建零 stat、零重读(Corona 实测约 2 秒)。
**补充(引用计数、语义 Find All References 与未引用资产,2026-08-05)**:
- 每个顶部 asset 显示被引用次数(CodeLens,0 也显示),点击直接打开
references peek;只在“设计上应被引用”的类型上显示,避免设置类/地图元数据/
w3x 子结构等自动注册类型制造满屏 0;
- Find All References 改为基于语义引用索引(属性引用 + simple-content 文本 +
`inheritFrom`,排除 id 定义点 / Poid / `$DEFINE`),不再全文搜索;
- 命令 `Find unreferenced assets…`:按类型列出所有零引用的项目定义并跳转;
编辑器右键菜单 `Find unreferenced assets of this type` 预选光标所在类型;
- 设计文档见 `docs/features-reference-counts.md`。
### P1:非近期目标(本期不做,但预留扩展点) ### P1:非近期目标(本期不做,但预留扩展点)
6. **高效搜索**:Mod 项目巨大(Corona 约 7500 个 XML、38MB)时直接全文搜索很慢,需要一个高效的 XML 内容索引机制。 6. **高效搜索**:Mod 项目巨大(Corona 约 7500 个 XML、38MB)时直接全文搜索很慢,需要一个高效的 XML 内容索引机制。
(2026-08-05:语义引用索引已落地,FAR / 引用计数 / 未引用报告不再全文搜索;
通用内容搜索与索引复用仍属远期。)
7. **索引机制的复用性**:希望索引不仅能服务 VS Code 插件,也能被其他工具(如搜索、静态分析)复用,因此索引/解析核心应设计成与编辑器无关的纯模块。 7. **索引机制的复用性**:希望索引不仅能服务 VS Code 插件,也能被其他工具(如搜索、静态分析)复用,因此索引/解析核心应设计成与编辑器无关的纯模块。
## 三、环境与参考资源 ## 三、环境与参考资源
@@ -81,7 +109,9 @@ XML 之间的组织靠 `<Include>` 标签,共有三种语义:
- 中小项目:`C:\Apps\RA3-MODSDK-X\Mods\AttachTest`、`D:\Mods\CoronaMod\mods\mods\GenEvoTest`。 - 中小项目:`C:\Apps\RA3-MODSDK-X\Mods\AttachTest`、`D:\Mods\CoronaMod\mods\mods\GenEvoTest`。
- 大型项目:`D:\Mods\CoronaMod\mods\mods\corona`(自带 `xsd/`)。 - 大型项目:`D:\Mods\CoronaMod\mods\mods\corona`(自带 `xsd/`)。
- 现有工具:工作区 `check_duplicate_ids.py`(include 解析与重复 ID 检测的参考实现)。 - 现有工具:工作区 `check_duplicate_ids.py`(include 解析与重复 ID 检测的参考实现)。
- manifest 格式参考:OpenSAGE `src/OpenSage.Game/Data/StreamFS/ManifestFile.cs`(commit `d45d361`,最新分支已移除该文件)。本机网络受限未能拉取,且 manifest 为压缩/哈希的二进制,本期不实现其解析。 - manifest 格式参考:OpenSAGE `src/OpenSage.Game/Data/StreamFS/ManifestFile.cs`
(commit `d45d361`,本仓库 `OpenSAGE/` 子目录)。插件已实现 v5/v6/v7 二进制
解析(`src/indexer/manifestParser.ts`),未知 TypeId 哈希时按资产名前缀推导类型。
## 四、验收标准 ## 四、验收标准
+2 -2
View File
@@ -1,12 +1,12 @@
{ {
"name": "ra3-mod-xml", "name": "ra3-mod-xml",
"version": "0.1.0", "version": "0.1.18",
"lockfileVersion": 3, "lockfileVersion": 3,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "ra3-mod-xml", "name": "ra3-mod-xml",
"version": "0.1.0", "version": "0.1.18",
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"fast-xml-parser": "^4.5.0" "fast-xml-parser": "^4.5.0"
+27 -1
View File
@@ -2,7 +2,7 @@
"name": "ra3-mod-xml", "name": "ra3-mod-xml",
"displayName": "RA3 Mod XML", "displayName": "RA3 Mod XML",
"description": "Red Alert 3 Mod XML tooling: syntax highlighting, completions, reference navigation and diagnostics for SAGE/BinaryAssetBuilder XML.", "description": "Red Alert 3 Mod XML tooling: syntax highlighting, completions, reference navigation and diagnostics for SAGE/BinaryAssetBuilder XML.",
"version": "0.1.0", "version": "0.1.18",
"publisher": "ra3-mod-xml", "publisher": "ra3-mod-xml",
"license": "MIT", "license": "MIT",
"engines": { "engines": {
@@ -22,6 +22,7 @@
], ],
"main": "./dist/extension.js", "main": "./dist/extension.js",
"activationEvents": [ "activationEvents": [
"onLanguage:xml",
"workspaceContains:**/Data/Mod.xml", "workspaceContains:**/Data/Mod.xml",
"workspaceContains:**/mod.babproj", "workspaceContains:**/mod.babproj",
"workspaceContains:**/*.babproj" "workspaceContains:**/*.babproj"
@@ -91,6 +92,31 @@
{ {
"command": "ra3modxml.openIndexReport", "command": "ra3modxml.openIndexReport",
"title": "RA3 Mod XML: Show index report" "title": "RA3 Mod XML: Show index report"
},
{
"command": "ra3modxml.clearCache",
"title": "RA3 Mod XML: Clear caches and rebuild"
},
{
"command": "ra3modxml.showCacheReport",
"title": "RA3 Mod XML: Show cache report"
},
{
"command": "ra3modxml.findUnreferencedAssets",
"title": "RA3 Mod XML: Find unreferenced assets…"
},
{
"command": "ra3modxml.findUnreferencedAssetsOfType",
"title": "RA3 Mod XML: Find unreferenced assets of this type"
}
]
},
"menus": {
"editor/context": [
{
"command": "ra3modxml.findUnreferencedAssetsOfType",
"when": "editorLangId == xml && ra3modxml.active",
"group": "navigation@50"
} }
] ]
}, },
+84 -8
View File
@@ -8,6 +8,12 @@ import {
Ra3DocumentSymbolProvider, Ra3DocumentSymbolProvider,
Ra3ReferenceProvider, Ra3ReferenceProvider,
} from "./features/navigation"; } from "./features/navigation";
import { Ra3CodeLensProvider } from "./features/codeLens";
import { showReferencesForDef } from "./features/references";
import {
findUnreferencedAssets,
findUnreferencedAssetsOfType,
} from "./features/unreferenced";
import { Ra3Diagnostics } from "./features/diagnostics"; import { Ra3Diagnostics } from "./features/diagnostics";
import { import {
Ra3SemanticTokensProvider, Ra3SemanticTokensProvider,
@@ -30,6 +36,7 @@ export function activate(context: vscode.ExtensionContext): void {
":", ":",
".", ".",
"/", "/",
" ",
), ),
); );
context.subscriptions.push( context.subscriptions.push(
@@ -44,7 +51,7 @@ export function activate(context: vscode.ExtensionContext): void {
context.subscriptions.push( context.subscriptions.push(
vscode.languages.registerReferenceProvider( vscode.languages.registerReferenceProvider(
XML_SELECTOR, XML_SELECTOR,
new Ra3ReferenceProvider(), new Ra3ReferenceProvider(ws),
), ),
); );
context.subscriptions.push( context.subscriptions.push(
@@ -56,19 +63,33 @@ export function activate(context: vscode.ExtensionContext): void {
context.subscriptions.push( context.subscriptions.push(
vscode.languages.registerDocumentSymbolProvider( vscode.languages.registerDocumentSymbolProvider(
XML_SELECTOR, XML_SELECTOR,
new Ra3DocumentSymbolProvider(), new Ra3DocumentSymbolProvider(ws),
),
);
context.subscriptions.push(
vscode.languages.registerCodeLensProvider(
XML_SELECTOR,
new Ra3CodeLensProvider(ws),
), ),
); );
context.subscriptions.push( context.subscriptions.push(
vscode.languages.registerDocumentSemanticTokensProvider( vscode.languages.registerDocumentSemanticTokensProvider(
XML_SELECTOR, XML_SELECTOR,
new Ra3SemanticTokensProvider(), new Ra3SemanticTokensProvider(ws),
RA3_SEMANTIC_TOKENS_LEGEND, RA3_SEMANTIC_TOKENS_LEGEND,
), ),
); );
const diagnostics = new Ra3Diagnostics(ws); const diagnostics = new Ra3Diagnostics(ws);
context.subscriptions.push(diagnostics); context.subscriptions.push(diagnostics);
// Refresh diagnostics for every open XML document whenever a new index
// snapshot is published (XML phase, art phase, stale/final rebuild).
ws.onIndexUpdate = () => {
void vscode.commands.executeCommand("editor.action.codeLens.refresh");
for (const doc of vscode.workspace.textDocuments) {
if (doc.languageId === "xml") void diagnostics.update(doc);
}
};
const diagnosticTimers = new Map<string, ReturnType<typeof setTimeout>>(); const diagnosticTimers = new Map<string, ReturnType<typeof setTimeout>>();
const scheduleDiagnostics = (doc: vscode.TextDocument) => { const scheduleDiagnostics = (doc: vscode.TextDocument) => {
@@ -109,40 +130,95 @@ export function activate(context: vscode.ExtensionContext): void {
context.subscriptions.push( context.subscriptions.push(
vscode.workspace.onDidSaveTextDocument((doc) => { vscode.workspace.onDidSaveTextDocument((doc) => {
if (doc.languageId !== "xml") return; if (doc.languageId !== "xml") return;
ws.scheduleRebuild(); ws.invalidate(doc.uri.fsPath);
ws.scheduleRebuild("save");
void diagnostics.update(doc); void diagnostics.update(doc);
}), }),
); );
context.subscriptions.push( context.subscriptions.push(
vscode.workspace.onDidChangeConfiguration((e) => { vscode.workspace.onDidChangeConfiguration((e) => {
if (e.affectsConfiguration("ra3modxml")) ws.scheduleRebuild(); if (e.affectsConfiguration("ra3modxml")) {
// Search paths / builtmods locations may have changed: cached include
// resolutions and manifest lookups are no longer valid.
ws.invalidateExistence();
ws.scheduleRebuild("config");
}
}), }),
); );
context.subscriptions.push( context.subscriptions.push(
vscode.commands.registerCommand("ra3modxml.reindex", () => ws.rebuild()), vscode.commands.registerCommand(
"ra3modxml.reindex",
() => void ws.rebuild(true, "reindex-command"),
),
);
context.subscriptions.push(
vscode.commands.registerCommand("ra3modxml.clearCache", () => {
ws.clearCaches();
void vscode.window.showInformationMessage(
"RA3 Mod XML: caches cleared; rebuilding from scratch…",
);
}),
);
context.subscriptions.push(
vscode.commands.registerCommand("ra3modxml.showCacheReport", async () => {
void vscode.window.showInformationMessage(await ws.cacheReport(), {
modal: false,
});
}),
); );
context.subscriptions.push( context.subscriptions.push(
vscode.commands.registerCommand("ra3modxml.openIndexReport", () => { vscode.commands.registerCommand("ra3modxml.openIndexReport", () => {
const idx = ws.index; const idx = ws.index;
if (!idx) { if (!idx) {
if (ws.isBuilding) {
void vscode.window.showInformationMessage(
"RA3 Mod XML: index is still building — check the status bar. " +
"Most features become available after the XML phase.",
);
return;
}
void vscode.window.showInformationMessage( void vscode.window.showInformationMessage(
"RA3 Mod XML: no index available. Open a workspace that contains Data/Mod.xml.", "RA3 Mod XML: no index available. Open a workspace that contains Data/Mod.xml.",
); );
return; return;
} }
const s = idx.stats; const s = idx.stats;
const stale = idx.stale ? " (stale)" : "";
void vscode.window.showInformationMessage( void vscode.window.showInformationMessage(
`RA3 Mod XML index\n` + `RA3 Mod XML index\n` +
`Project: ${s.projectDir}\n` + `Project: ${s.projectDir}\n` +
`Files: ${s.indexedFiles} (${s.parsedFiles} parsed)\n` + `Files: ${s.indexedFiles} (${s.parsedFiles} parsed, ${s.shallowScannedFiles} shallow-scanned, ${s.shallowCacheHits + s.recordsCacheHits} cache hits)\n` +
`Assets: ${s.assetCount} (${s.manifestAssetCount} from ${s.manifestFiles} manifests)\n` + `Assets: ${s.assetCount} (${s.manifestAssetCount} from ${s.manifestFiles} manifests)\n` +
`References: ${s.referenceCount}\n` +
`Defines: ${s.defineCount} · Streams: ${s.streams} · Candidates: ${s.sourceCandidates}\n` + `Defines: ${s.defineCount} · Streams: ${s.streams} · Candidates: ${s.sourceCandidates}\n` +
`Indexed in ${(s.elapsedMs / 1000).toFixed(1)}s`, `Phase: ${s.phase} · Complete: ${s.complete}${stale}\n` +
`Build #${ws.buildCount} (trigger: ${ws.lastTrigger})\n` +
`Indexed in ${(s.elapsedMs / 1000).toFixed(1)}s\n` +
`XML walk: ${(s.walkMs / 1000).toFixed(1)}s · Candidates: ${(s.candidatesMs / 1000).toFixed(1)}s · Art scan: ${(s.artScanMs / 1000).toFixed(1)}s`,
{ modal: false }, { modal: false },
); );
}), }),
); );
context.subscriptions.push(
vscode.commands.registerCommand(
"ra3modxml.showReferences",
(args: Parameters<typeof showReferencesForDef>[1]) =>
void showReferencesForDef(ws, args),
),
);
context.subscriptions.push(
vscode.commands.registerCommand(
"ra3modxml.findUnreferencedAssets",
() => void findUnreferencedAssets(ws),
),
);
context.subscriptions.push(
vscode.commands.registerCommand(
"ra3modxml.findUnreferencedAssetsOfType",
() => void findUnreferencedAssetsOfType(ws),
),
);
void ws.initialize(); void ws.initialize();
} }
+90
View File
@@ -0,0 +1,90 @@
import * as vscode from "vscode";
import { LineMap, parseXml } from "../language/xmlParser";
import { resolveElementType } from "../language/typeContext";
import { isReferenceTargetType } from "../indexer/refs";
import {
referenceSitesForDefinition,
scheduleRebuildIfRecordsDesync,
} from "../indexer/referenceIndex";
import type { ShowReferencesArgs } from "./references";
import type { ModWorkspace } from "../workspace";
/** Never build a DOM for huge files just to show counts (w3x safety). */
const MAX_CODELENS_TEXT = 4 * 1024 * 1024;
/**
* CodeLens reference counts on top-level assets.
*
* Only types that are reference targets by design get a lens (settings, map
* metadata, w3x sub-assets etc. would otherwise show a permanent, misleading
* "0 references"). Zero is still shown for the meaningful types: that is the
* signal users can click to inspect an unused asset.
*/
export class Ra3CodeLensProvider implements vscode.CodeLensProvider {
constructor(private ws: ModWorkspace) {}
provideCodeLenses(
document: vscode.TextDocument,
_token: vscode.CancellationToken,
): vscode.CodeLens[] {
if (!this.ws.isRa3Workspace()) return [];
const idx = this.ws.index;
if (!idx) return [];
const text = document.getText();
if (text.length > MAX_CODELENS_TEXT) return [];
scheduleRebuildIfRecordsDesync(this.ws, document);
const doc = parseXml(text);
const root = doc.root;
if (!root) return [];
const lineMap = new LineMap(text);
const lenses: vscode.CodeLens[] = [];
for (const child of root.children) {
const local = localName(child.name);
if (local === "Tags" || local === "Includes" || local === "Defines") continue;
const idAttr = child.attrs.find((a) => a.name === "id");
if (!idAttr?.hasValue) continue;
const elType = resolveElementType(child);
if (!isReferenceTargetType(elType)) continue;
const id = idAttr.value;
const line = lineMap.positionAt(idAttr.valueStart).line + 1;
const count = referenceSitesForDefinition(idx, {
type: local,
id,
file: document.uri.fsPath,
line,
}).length;
const range = new vscode.Range(
document.positionAt(child.start),
document.positionAt(child.startTagEnd),
);
const args: ShowReferencesArgs = {
uri: document.uri,
position: document.positionAt(idAttr.valueStart),
id,
type: local,
file: document.uri.fsPath,
line,
};
lenses.push(
new vscode.CodeLens(range, {
title:
count === 0
? "0 references"
: count === 1
? "1 reference"
: `${count} references`,
command: "ra3modxml.showReferences",
arguments: [args],
}),
);
}
return lenses;
}
}
function localName(tag: string): string {
const idx = tag.lastIndexOf(":");
return idx >= 0 ? tag.slice(idx + 1) : tag;
}
+566 -66
View File
@@ -1,5 +1,6 @@
import * as vscode from "vscode"; import * as vscode from "vscode";
import { parseXml, type XmlElement } from "../language/xmlParser"; import type { XmlAttribute, XmlElement } from "../language/xmlParser";
import { textContentTokenAt } from "../language/xmlParser";
import { import {
analyzeContext, analyzeContext,
splitListValuePrefix, splitListValuePrefix,
@@ -7,7 +8,13 @@ import {
} from "../language/context"; } from "../language/context";
import { resolveElementType } from "../language/typeContext"; import { resolveElementType } from "../language/typeContext";
import * as model from "../model/schemaModel"; import * as model from "../model/schemaModel";
import type { AttributeInfo, SimpleTypeInfo } from "../model/schemaModel";
import { isLocalReferenceAttribute } from "../indexer/refs"; import { isLocalReferenceAttribute } from "../indexer/refs";
import {
findContainingGameObject,
collectLocalIds,
type LogicalElement,
} from "../indexer/logicalTree";
import type { ModWorkspace } from "../workspace"; import type { ModWorkspace } from "../workspace";
import type { ModIndex, AssetDef } from "../indexer/types"; import type { ModIndex, AssetDef } from "../indexer/types";
@@ -20,12 +27,14 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
document: vscode.TextDocument, document: vscode.TextDocument,
position: vscode.Position, position: vscode.Position,
_token: vscode.CancellationToken, _token: vscode.CancellationToken,
): Promise<vscode.CompletionItem[]> { ): Promise<vscode.CompletionItem[] | vscode.CompletionList<vscode.CompletionItem>> {
if (!this.ws.isRa3Workspace()) return [];
const text = document.getText(); const text = document.getText();
const offset = document.offsetAt(position); const offset = document.offsetAt(position);
const doc = parseXml(text); const scope = await this.ws.getScope(document);
const doc = scope.expanded;
const ctx = analyzeContext(doc, text, offset); const ctx = analyzeContext(doc, text, offset);
const idx = this.ws.index; const idx = scope.merged;
switch (ctx.kind) { switch (ctx.kind) {
case "element-name": case "element-name":
@@ -33,9 +42,9 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
case "attribute-name": case "attribute-name":
return this.attributeNameItems(ctx, document, position); return this.attributeNameItems(ctx, document, position);
case "attribute-value": case "attribute-value":
return idx ? this.valueItems(ctx, document, position, idx) : []; return this.valueItems(ctx, document, position, idx);
case "content": case "content":
return idx ? this.contentItems(ctx, document, position, idx) : []; return this.contentItems(ctx, document, position, idx);
default: default:
return []; return [];
} }
@@ -53,10 +62,12 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
const names = this.childrenOf(parent); const names = this.childrenOf(parent);
if (!names.length) return []; if (!names.length) return [];
// The "<" already exists at the element's start: replace only the name
// area and insert the tag body WITHOUT a leading "<", otherwise the
// range text ("<") would also be used as the filter prefix and hide
// every suggestion.
const start = ctx.element const start = ctx.element
? document.offsetAt( ? ctx.element.start + (ctx.closing ? 2 : 1)
document.positionAt(ctx.element.start + (ctx.closing ? 2 : 1)),
)
: document.offsetAt(position); : document.offsetAt(position);
const range = new vscode.Range(document.positionAt(start), position); const range = new vscode.Range(document.positionAt(start), position);
const items: vscode.CompletionItem[] = []; const items: vscode.CompletionItem[] = [];
@@ -71,7 +82,7 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
(type ? `Type: ${type}` : ""); (type ? `Type: ${type}` : "");
item.documentation = docText ? new vscode.MarkdownString(docText) : undefined; item.documentation = docText ? new vscode.MarkdownString(docText) : undefined;
item.detail = type ? `RA3 XML · ${type}` : "RA3 XML"; item.detail = type ? `RA3 XML · ${type}` : "RA3 XML";
item.insertText = this.elementSnippet(child.name, type); item.insertText = this.elementSnippet(child.name, type, ctx.element == null);
items.push(item); items.push(item);
} }
return items; return items;
@@ -95,16 +106,27 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
return []; return [];
} }
private elementSnippet(name: string, type: string | null): vscode.SnippetString { private elementSnippet(
name: string,
type: string | null,
includeOpenBracket = true,
): vscode.SnippetString {
const open = includeOpenBracket ? "<" : "";
if (model.isTopLevelElement(name)) { if (model.isTopLevelElement(name)) {
return new vscode.SnippetString(`<${name} id="$1">\n\t$0\n</${name}>`); return new vscode.SnippetString(`${open}${name} id="$1">\n\t$0\n</${name}>`);
} }
const info = type ? model.typeInfo(type) : undefined; const info = type ? model.typeInfo(type) : undefined;
// Simple types hold text content (asset id / enum / define / string), so
// they need an explicit closing tag and a value placeholder instead of a
// self-closing tag that can never contain a value.
if (info?.kind === "simple") {
return new vscode.SnippetString(`${open}${name}>$1</${name}>`);
}
const hasChildren = info?.kind === "complex" && info.children.length > 0; const hasChildren = info?.kind === "complex" && info.children.length > 0;
if (hasChildren) { if (hasChildren) {
return new vscode.SnippetString(`<${name}>\n\t$0\n</${name}>`); return new vscode.SnippetString(`${open}${name}>\n\t$0\n</${name}>`);
} }
return new vscode.SnippetString(`<${name} />`); return new vscode.SnippetString(`${open}${name} />`);
} }
// ── Attribute name ──────────────────────────────────────────────── // ── Attribute name ────────────────────────────────────────────────
@@ -121,8 +143,15 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
const used = new Set(ctx.existingAttrs.map((a) => a.toLowerCase())); const used = new Set(ctx.existingAttrs.map((a) => a.toLowerCase()));
const items: vscode.CompletionItem[] = []; const items: vscode.CompletionItem[] = [];
const wordStart = findAttributeWordStart(document, position, el); const text = document.getText();
const range = new vscode.Range(document.positionAt(wordStart), position); const offset = document.offsetAt(position);
const layout = attributeInsertLayout(text, el, offset);
const range = new vscode.Range(document.positionAt(layout.rangeStart), position);
this.ws.log(
`[completion] attr-name ${el.name} existing=[${ctx.existingAttrs.join(", ")}] ` +
`range=${layout.rangeStart}..${offset} prefix=${JSON.stringify(layout.prefix)}`,
);
for (const attr of attrs) { for (const attr of attrs) {
if (used.has(attr.name.toLowerCase())) continue; if (used.has(attr.name.toLowerCase())) continue;
@@ -138,15 +167,16 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
if (attr.default != null) md.appendMarkdown(`Default: \`${attr.default}\` \n`); if (attr.default != null) md.appendMarkdown(`Default: \`${attr.default}\` \n`);
md.appendMarkdown(`Type: \`${attr.type ?? "string"}\``); md.appendMarkdown(`Type: \`${attr.type ?? "string"}\``);
item.documentation = md; item.documentation = md;
if (attr.required) { const value = this.attributeValuePlaceholder(attr, el);
item.insertText = attr.name === "id" ? 'id="$1"' : `${attr.name}="$1"`; item.insertText = new vscode.SnippetString(
} else { layout.prefix + `${attr.name}="${value.snippet}"`,
item.insertText = `${attr.name}="$1"`; );
if (value.trigger) {
item.command = {
command: "editor.action.triggerSuggest",
title: "Suggest attribute values",
};
} }
item.command = {
command: "editor.action.triggerSuggest",
title: "Suggest attribute values",
};
items.push(item); items.push(item);
} }
@@ -154,31 +184,75 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
if (!used.has("xai:joinaction")) { if (!used.has("xai:joinaction")) {
const j = new vscode.CompletionItem("xai:joinAction", vscode.CompletionItemKind.Property); const j = new vscode.CompletionItem("xai:joinAction", vscode.CompletionItemKind.Property);
j.range = range; j.range = range;
j.insertText = 'xai:joinAction="$1"'; j.insertText = new vscode.SnippetString(
layout.prefix + 'xai:joinAction="$1"',
);
j.detail = "Instance join action"; j.detail = "Instance join action";
j.documentation = new vscode.MarkdownString( j.documentation = new vscode.MarkdownString(
"Controls how this element merges with the inherited definition: `Replace` or `Remove`.", "Controls how this element merges with the inherited definition: `Replace` or `Remove`.",
); );
j.command = {
command: "editor.action.triggerSuggest",
title: "Suggest attribute values",
};
items.push(j); items.push(j);
} }
if (!used.has("xmlns:xai")) { if (!used.has("xmlns:xai")) {
const ns = new vscode.CompletionItem("xmlns:xai", vscode.CompletionItemKind.Property); const ns = new vscode.CompletionItem("xmlns:xai", vscode.CompletionItemKind.Property);
ns.range = range; ns.range = range;
ns.insertText = 'xmlns:xai="uri:ea.com:eala:asset:instance"'; ns.insertText = new vscode.SnippetString(
layout.prefix + 'xmlns:xai="uri:ea.com:eala:asset:instance"',
);
ns.detail = "xai namespace"; ns.detail = "xai namespace";
items.push(ns); items.push(ns);
} }
return items; return items;
} }
/**
* Chooses the placeholder/value inserted for a completed attribute:
* attributes whose value is picked from suggestions (references, enums,
* lists, booleans, defines, include sources, local ids) keep a `$1`
* placeholder and re-trigger the value popup; scalar attributes get the XSD
* default (or a type-appropriate example such as `0d` for angles or `0s`
* for times) so the completed value shows the expected format immediately.
*/
private attributeValuePlaceholder(
attr: AttributeInfo,
el: XmlElement,
): { snippet: string; trigger: boolean } {
if (
attr.isBoolean ||
attr.isList ||
attr.enumValues.length > 0 ||
attr.refType != null ||
attr.isRef ||
attr.name === "inheritFrom" ||
(el.name === "Include" && attr.name === "source")
) {
return { snippet: "$1", trigger: true };
}
if (attr.name === "id") {
// `id` is the element's own definition point; nothing to suggest, but
// keep the placeholder for the user to type the id.
return { snippet: "$1", trigger: false };
}
if (attr.default != null && attr.default !== "") {
return { snippet: attr.default, trigger: false };
}
const example = DEFAULT_VALUE_BY_TYPE[(attr.type ?? "").toLowerCase()];
if (example != null) return { snippet: example, trigger: false };
return { snippet: "$1", trigger: false };
}
// ── Attribute value ─────────────────────────────────────────────── // ── Attribute value ───────────────────────────────────────────────
private valueItems( private valueItems(
ctx: CompletionContext, ctx: CompletionContext,
document: vscode.TextDocument, document: vscode.TextDocument,
position: vscode.Position, position: vscode.Position,
idx: ModIndex, idx: ModIndex | null,
): vscode.CompletionItem[] { ): vscode.CompletionItem[] | vscode.CompletionList<vscode.CompletionItem> {
const el = ctx.element; const el = ctx.element;
const attr = ctx.attr; const attr = ctx.attr;
if (!el || !attr) return []; if (!el || !attr) return [];
@@ -194,15 +268,23 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
// whitespace-separated: only the token currently being edited is used for // whitespace-separated: only the token currently being edited is used for
// filtering, and the replacement range covers that token instead of the // filtering, and the replacement range covers that token instead of the
// whole value. // whole value.
const seg = attrInfo?.isList const isList = attrInfo?.isList === true;
const seg = isList
? splitListValuePrefix(rawPrefix) ? splitListValuePrefix(rawPrefix)
: { token: rawPrefix, start: 0 }; : { token: rawPrefix, start: 0 };
const prefix = seg.token; const prefix = seg.token;
const valueStartOffset = const valueStartOffset =
attr.valueStart >= 0 ? attr.valueStart : document.offsetAt(position); attr.valueStart >= 0 ? attr.valueStart : document.offsetAt(position);
const endOffset = const cursorOffset = document.offsetAt(position);
attr.quoteEnd > attr.valueEnd ? attr.valueEnd : document.offsetAt(position); // For list values the replacement must cover only the segment being
// edited; extending to the end of the whole value would delete the flags
// after the cursor when inserting in the middle of an existing list.
const endOffset = isList
? Math.min(cursorOffset, attr.valueEnd >= 0 ? attr.valueEnd : cursorOffset)
: attr.quoteEnd > attr.valueEnd
? attr.valueEnd
: cursorOffset;
const rangeStart = valueStartOffset + seg.start; const rangeStart = valueStartOffset + seg.start;
const valueRange = new vscode.Range( const valueRange = new vscode.Range(
document.positionAt(rangeStart), document.positionAt(rangeStart),
@@ -214,10 +296,12 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
kind: vscode.CompletionItemKind, kind: vscode.CompletionItemKind,
detail: string, detail: string,
doc?: string, doc?: string,
range?: vscode.Range,
insertText?: string,
) => { ) => {
const item = new vscode.CompletionItem(label, kind); const item = new vscode.CompletionItem(label, kind);
item.range = valueRange; item.range = range ?? valueRange;
item.insertText = label; item.insertText = insertText ?? label;
item.detail = detail; item.detail = detail;
if (doc) item.documentation = new vscode.MarkdownString(doc); if (doc) item.documentation = new vscode.MarkdownString(doc);
return item; return item;
@@ -232,6 +316,7 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
); );
} }
if (isInclude && attrName === "source") { if (isInclude && attrName === "source") {
if (!idx) return [];
return this.includeSourceItems(idx, prefix, make); return this.includeSourceItems(idx, prefix, make);
} }
if (attrName === "xai:joinaction" || attrName === "joinaction") { if (attrName === "xai:joinaction" || attrName === "joinaction") {
@@ -242,19 +327,25 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
// inheritFrom: same element type first, then everything. // inheritFrom: same element type first, then everything.
if (attrName === "inheritfrom") { if (attrName === "inheritfrom") {
if (!idx) return [];
return this.assetIdItems(idx, el.name, null, prefix, make); return this.assetIdItems(idx, el.name, null, prefix, make);
} }
// `id` attributes are definitions and Poid attributes are pipeline-local // `id` attributes are definitions and Poid attributes are pipeline-local
// references; offering global asset ids for them would be wrong. // references; offering global asset ids for them would be wrong.
if (attrInfo && isLocalReferenceAttribute(elType, attr.name)) { if (attrInfo && isLocalReferenceAttribute(elType, attr.name)) {
return []; if (attrName === "id") return [];
return this.localIdItems(el as LogicalElement, prefix, make);
} }
if (attrInfo?.refType) { if (attrInfo?.refType) {
if (!idx) return [];
return this.assetIdItems(idx, null, attrInfo.refType, prefix, make); return this.assetIdItems(idx, null, attrInfo.refType, prefix, make);
} }
if (attrInfo?.enumValues?.length) { if (attrInfo?.enumValues?.length) {
if (attrInfo.isList) {
return this.listEnumItems(attrInfo, rawPrefix, seg, valueRange, make);
}
return attrInfo.enumValues return attrInfo.enumValues
.filter((v) => v.toLowerCase().startsWith(prefix.toLowerCase())) .filter((v) => v.toLowerCase().startsWith(prefix.toLowerCase()))
.map((v) => make(v, vscode.CompletionItemKind.EnumMember, attrInfo.type ?? "enum")); .map((v) => make(v, vscode.CompletionItemKind.EnumMember, attrInfo.type ?? "enum"));
@@ -264,28 +355,87 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
.filter((v) => v.startsWith(prefix.toLowerCase())) .filter((v) => v.startsWith(prefix.toLowerCase()))
.map((v) => make(v, vscode.CompletionItemKind.Value, "boolean")); .map((v) => make(v, vscode.CompletionItemKind.Value, "boolean"));
} }
if (attrInfo?.allowsDefine) { if (idx && attrInfo?.allowsDefine) {
return this.defineItems(idx, prefix, make); return this.defineItems(idx, prefix, make);
} }
return []; return [];
} }
/**
* Completions for xs:list enum values (whitespace-separated bit flags).
*
* Only the segment being edited is used for filtering, and values already
* present earlier in the list are excluded so adding a flag never re-offers
* an existing one. When the current segment is already a complete flag and
* no other flag extends it (e.g. "GROUND" -> "GROUND_EDGE" would disable
* this), the remaining flags are offered as insertions after the cursor
* (" FLAG") so flags can be appended to an already-closed value.
*/
private listEnumItems(
attrInfo: { enumValues: string[]; type?: string | null },
rawPrefix: string,
seg: { token: string; start: number },
valueRange: vscode.Range,
make: (
label: string,
kind: vscode.CompletionItemKind,
detail: string,
doc?: string,
range?: vscode.Range,
insertText?: string,
) => vscode.CompletionItem,
): vscode.CompletionItem[] {
const used = new Set(
rawPrefix
.slice(0, seg.start)
.split(/\s+/)
.map((t) => t.toLowerCase())
.filter(Boolean),
);
const token = seg.token.toLowerCase();
const exact = token !== "" && attrInfo.enumValues.some((v) => v.toLowerCase() === token);
const extendable = attrInfo.enumValues.some(
(v) => v.toLowerCase().startsWith(token) && v.toLowerCase() !== token,
);
const append = exact && !extendable;
const range = append
? new vscode.Range(valueRange.end, valueRange.end)
: valueRange;
const filtered = attrInfo.enumValues.filter((v) => {
const lower = v.toLowerCase();
if (used.has(lower)) return false;
if (append) return lower !== token;
if (exact && lower === token) return false;
return lower.startsWith(token);
});
return filtered.map((v) =>
make(
v,
vscode.CompletionItemKind.EnumMember,
attrInfo.type ?? "enum",
undefined,
range,
append ? ` ${v}` : v,
),
);
}
private includeSourceItems( private includeSourceItems(
idx: ModIndex, idx: ModIndex,
prefix: string, prefix: string,
make: (label: string, kind: vscode.CompletionItemKind, detail: string, doc?: string) => vscode.CompletionItem, make: (label: string, kind: vscode.CompletionItemKind, detail: string, doc?: string) => vscode.CompletionItem,
): vscode.CompletionItem[] { ): vscode.CompletionItem[] | vscode.CompletionList<vscode.CompletionItem> {
const lower = prefix.toLowerCase(); const lower = prefix.toLowerCase();
const candidates = idx.sourceCandidates const candidates = idx.sourceCandidates.filter((c) =>
.filter((c) => c.source.toLowerCase().includes(lower)) c.source.toLowerCase().includes(lower),
.slice(0, MAX_VALUE_ITEMS); );
const priority: Record<string, number> = { "": 0, DATA: 1, ART: 2, AUDIO: 3 }; const priority: Record<string, number> = { "": 0, DATA: 1, ART: 2, AUDIO: 3 };
candidates.sort( candidates.sort(
(a, b) => (a, b) =>
(priority[a.prefix ?? ""] ?? 4) - (priority[b.prefix ?? ""] ?? 4) || (priority[a.prefix ?? ""] ?? 4) - (priority[b.prefix ?? ""] ?? 4) ||
a.source.localeCompare(b.source), a.source.localeCompare(b.source),
); );
return candidates.map((c) => { const items = candidates.map((c) => {
const item = make(c.source, vscode.CompletionItemKind.File, "Include source"); const item = make(c.source, vscode.CompletionItemKind.File, "Include source");
item.detail = c.path; item.detail = c.path;
item.documentation = new vscode.MarkdownString( item.documentation = new vscode.MarkdownString(
@@ -293,6 +443,7 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
); );
return item; return item;
}); });
return this.limitItems(items, items.length);
} }
private assetIdItems( private assetIdItems(
@@ -301,20 +452,37 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
refType: string | null, refType: string | null,
prefix: string, prefix: string,
make: (label: string, kind: vscode.CompletionItemKind, detail: string, doc?: string) => vscode.CompletionItem, make: (label: string, kind: vscode.CompletionItemKind, detail: string, doc?: string) => vscode.CompletionItem,
): vscode.CompletionItem[] { ): vscode.CompletionItem[] | vscode.CompletionList<vscode.CompletionItem> {
const lower = prefix.toLowerCase(); const lower = prefix.toLowerCase();
const scored: { def: AssetDef; score: number }[] = []; const scored: { def: AssetDef; score: number }[] = [];
const seen = new Set<string>();
const consider = (def: AssetDef) => { const consider = (def: AssetDef) => {
const key = `${def.type}:${def.id.toLowerCase()}:${def.file}:${def.line}`;
if (seen.has(key)) return;
seen.add(key);
if (!def.id.toLowerCase().startsWith(lower)) return; if (!def.id.toLowerCase().startsWith(lower)) return;
let score = 3; let score = 3;
if (refType && model.isAssignableTo(def.type, refType)) score = 1; if (refType && model.isAssignableTo(def.type, refType)) score = 1;
if (selfType && model.isAssignableTo(def.type, selfType)) score = 0; if (selfType && model.isAssignableTo(def.type, selfType)) score = 0;
if (def.origin === "project") score -= 0.2; if (def.origin === "project") score -= 0.2;
if (def.stream === "local") score -= 0.4;
scored.push({ def, score }); scored.push({ def, score });
}; };
const targetType = selfType ?? refType; const targetType = selfType ?? refType;
const localAssets = idx.local?.assets;
const localById = idx.local?.assetsById;
if (localAssets || localById) {
if (!targetType) {
for (const list of localById!.values()) for (const d of list) consider(d);
} else {
for (const [typeName, byId] of localAssets!) {
if (!model.isAssignableTo(typeName, targetType)) continue;
for (const list of byId.values()) for (const d of list) consider(d);
}
}
}
if (!targetType) { if (!targetType) {
for (const list of idx.assetsById.values()) for (const d of list) consider(d); for (const list of idx.assetsById.values()) for (const d of list) consider(d);
} else { } else {
@@ -324,8 +492,8 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
} }
} }
scored.sort((a, b) => a.score - b.score || a.def.id.localeCompare(b.def.id)); const top = topScoredDefs(scored, MAX_VALUE_ITEMS);
return scored.slice(0, MAX_VALUE_ITEMS).map(({ def }) => { const items = top.map(({ def }) => {
const origin = def.origin === "manifest" ? `manifest (${def.manifestSource ?? ""})` : def.origin; const origin = def.origin === "manifest" ? `manifest (${def.manifestSource ?? ""})` : def.origin;
const doc = new vscode.MarkdownString(); const doc = new vscode.MarkdownString();
doc.appendCodeblock(def.id); doc.appendCodeblock(def.id);
@@ -334,63 +502,375 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
doc.appendMarkdown(`**Origin**: ${origin}`); doc.appendMarkdown(`**Origin**: ${origin}`);
return make(def.id, vscode.CompletionItemKind.Value, `${def.type} · ${origin}`, doc.value); return make(def.id, vscode.CompletionItemKind.Value, `${def.type} · ${origin}`, doc.value);
}); });
return this.limitItems(items, scored.length);
} }
private defineItems( private defineItems(
idx: ModIndex, idx: ModIndex,
prefix: string, prefix: string,
make: (label: string, kind: vscode.CompletionItemKind, detail: string, doc?: string) => vscode.CompletionItem, make: (label: string, kind: vscode.CompletionItemKind, detail: string, doc?: string) => vscode.CompletionItem,
): vscode.CompletionItem[] { ): vscode.CompletionItem[] | vscode.CompletionList<vscode.CompletionItem> {
const lower = prefix.replace(/^[=$]*/, "").toLowerCase(); const lower = prefix.replace(/^[=$]*/, "").toLowerCase();
const items: vscode.CompletionItem[] = []; const items: vscode.CompletionItem[] = [];
for (const [key, defs] of idx.defines) { const seen = new Set<string>();
if (!key.includes(lower)) continue; for (const defines of [idx.local?.defines, idx.defines]) {
const def = defs[0]; if (!defines) continue;
const label = `$${def.name}`; for (const [key, defs] of defines) {
const item = make(label, vscode.CompletionItemKind.Constant, "Define", def.value); if (!key.includes(lower)) continue;
item.insertText = label; const def = defs[0];
items.push(item); const dedupe = `${def.name.toLowerCase()}:${def.file}:${def.line}`;
if (seen.has(dedupe)) continue;
seen.add(dedupe);
const label = `$${def.name}`;
const item = make(label, vscode.CompletionItemKind.Constant, "Define", def.value);
item.insertText = label;
items.push(item);
}
} }
return items.slice(0, MAX_VALUE_ITEMS); return this.limitItems(items, items.length);
}
private localIdItems(
el: LogicalElement,
prefix: string,
make: (label: string, kind: vscode.CompletionItemKind, detail: string, doc?: string) => vscode.CompletionItem,
): vscode.CompletionItem[] | vscode.CompletionList<vscode.CompletionItem> {
const root = findContainingGameObject(el);
if (!root) return [];
const lower = prefix.toLowerCase();
const items: vscode.CompletionItem[] = [];
for (const { id } of collectLocalIds(root)) {
if (!id.toLowerCase().startsWith(lower)) continue;
items.push(
make(
id,
vscode.CompletionItemKind.Value,
"local module",
"Pipeline-local id in the enclosing GameObject (includes xi:include targets).",
),
);
}
return this.limitItems(items, items.length);
}
/**
* VS Code filters the returned items client-side while the user keeps
* typing. Once the result is capped, the list must be marked incomplete so
* the provider is asked again with the narrower prefix; otherwise a wanted
* id (e.g. CrateDebris_01) can be silently cut off behind the first 400
* alphabetically-earlier candidates and never reappear.
*/
private limitItems<T extends vscode.CompletionItem>(
items: T[],
total: number,
): T[] | vscode.CompletionList<T> {
if (total <= MAX_VALUE_ITEMS) return items;
return new vscode.CompletionList(items.slice(0, MAX_VALUE_ITEMS), true);
} }
// ── Element content ─────────────────────────────────────────────── // ── Element content ───────────────────────────────────────────────
private contentItems( private contentItems(
ctx: CompletionContext, ctx: CompletionContext,
_document: vscode.TextDocument, document: vscode.TextDocument,
_position: vscode.Position, position: vscode.Position,
_idx: ModIndex, idx: ModIndex | null,
): vscode.CompletionItem[] { ): vscode.CompletionItem[] | vscode.CompletionList<vscode.CompletionItem> {
const el = ctx.element; const el = ctx.element;
if (!el) return []; if (!el) return [];
// Reuse element-name suggestions with a plain replacement range. const elType = resolveElementType(el);
const info = elType ? model.typeInfo(elType) : undefined;
// Simple-content element: the text between the tags is the value itself
// (e.g. <CreateObject>CrateDebris_01</CreateObject>), so offer value
// completions (asset ids / enums / defines) instead of child elements.
if (info?.kind === "simple") {
return this.simpleContentItems(el, elType, info, document, position, idx);
}
return this.contentChildItems(el, document, position);
}
private simpleContentItems(
el: XmlElement,
elType: string | null,
info: SimpleTypeInfo,
document: vscode.TextDocument,
position: vscode.Position,
idx: ModIndex | null,
): vscode.CompletionItem[] | vscode.CompletionList<vscode.CompletionItem> {
const text = document.getText();
const offset = document.offsetAt(position);
const token = textContentTokenAt(text, el, offset);
const prefix = token ? text.slice(token.start, Math.min(offset, token.end)) : "";
const rawPrefix = text.slice(el.startTagEnd, offset);
const isList = info.isList === true;
const seg = isList
? splitListValuePrefix(rawPrefix)
: { token: prefix, start: token ? token.start - el.startTagEnd : 0 };
const rangeStart = token ? token.start : offset;
const valueRange = new vscode.Range(
document.positionAt(rangeStart),
document.positionAt(Math.max(rangeStart, offset)),
);
const make = (
label: string,
kind: vscode.CompletionItemKind,
detail: string,
doc?: string,
range?: vscode.Range,
insertText?: string,
) => {
const item = new vscode.CompletionItem(label, kind);
item.range = range ?? valueRange;
item.insertText = insertText ?? label;
item.detail = detail;
if (doc) item.documentation = new vscode.MarkdownString(doc);
return item;
};
// Typed asset references only (isRef without refType is used by real
// data for shader constants / mesh sub-object names, not global ids).
if (info.refType) {
if (!idx) return [];
return this.assetIdItems(idx, null, info.refType, seg.token, make);
}
if (info.enumValues.length) {
if (isList) return this.listEnumItems(info, rawPrefix, seg, valueRange, make);
return info.enumValues
.filter((v) => v.toLowerCase().startsWith(seg.token.toLowerCase()))
.map((v) => make(v, vscode.CompletionItemKind.EnumMember, elType ?? "enum"));
}
if (idx && info.allowsDefine) {
return this.defineItems(idx, seg.token, make);
}
return [];
}
private contentChildItems(
el: XmlElement,
document: vscode.TextDocument,
position: vscode.Position,
): vscode.CompletionItem[] {
const elType = resolveElementType(el); const elType = resolveElementType(el);
const names = elType ? model.childrenOfType(elType) : model.childrenOfElement(el.name); const names = elType ? model.childrenOfType(elType) : model.childrenOfElement(el.name);
const items: vscode.CompletionItem[] = []; const items: vscode.CompletionItem[] = [];
if (!names.length) return items;
const text = document.getText();
const offset = document.offsetAt(position);
// When the user already typed "<" (optionally followed by a partial
// name), keep that "<" and replace only the name area; the inserted
// snippet then has no leading "<" so the range text stays a valid
// filter prefix ("", "Cr", ...) instead of "<" (which would hide every
// item). Without a typed "<" the full "<Name>…</Name>" is inserted.
const { rangeStart, typedOpen } = contentElementRange(text, offset);
const range = new vscode.Range(document.positionAt(rangeStart), position);
for (const child of names) { for (const child of names) {
const item = new vscode.CompletionItem(child.name, vscode.CompletionItemKind.Field); const item = new vscode.CompletionItem(child.name, vscode.CompletionItemKind.Field);
item.insertText = this.elementSnippet(child.name, child.type); item.range = range;
item.insertText = this.elementSnippet(child.name, child.type, !typedOpen);
const type = child.type; const type = child.type;
const info = type ? model.typeInfo(type) : undefined; const info = type ? model.typeInfo(type) : undefined;
item.detail = type ? `RA3 XML · ${type}` : "RA3 XML"; item.detail = type ? `RA3 XML · ${type}` : "RA3 XML";
const doc = child.doc || (info?.kind === "complex" ? info.doc : ""); const doc = child.doc || (info?.kind === "complex" ? info.doc : "");
if (doc) item.documentation = new vscode.MarkdownString(doc); if (doc) item.documentation = new vscode.MarkdownString(doc);
if (info?.kind === "simple" && this.simpleContentValueKind(info)) {
item.command = {
command: "editor.action.triggerSuggest",
title: "Suggest content value",
};
}
items.push(item); items.push(item);
} }
return items; return items;
} }
private simpleContentValueKind(info: SimpleTypeInfo): boolean {
return (
info.refType != null ||
info.enumValues.length > 0 ||
info.allowsDefine
);
}
} }
function findAttributeWordStart( interface ScoredDef {
document: vscode.TextDocument, def: AssetDef;
position: vscode.Position, score: number;
el: { start: number }, }
): number {
const offset = document.offsetAt(position); function compareScoredDefs(a: ScoredDef, b: ScoredDef): number {
const tagStart = el.start; return a.score - b.score || a.def.id.localeCompare(b.def.id);
}
/**
* Returns the best `limit` scored definitions without sorting the whole
* candidate list. A max-heap keeps the worst item of the current top set at
* the root, so every additional candidate only needs an O(log limit) check.
*/
function topScoredDefs(scored: ScoredDef[], limit: number): ScoredDef[] {
if (scored.length <= limit) {
scored.sort(compareScoredDefs);
return scored;
}
const better = (a: ScoredDef, b: ScoredDef) => compareScoredDefs(a, b) < 0;
const heap: ScoredDef[] = [];
const swap = (i: number, j: number) => {
const t = heap[i];
heap[i] = heap[j];
heap[j] = t;
};
const siftUp = (i: number) => {
while (i > 0) {
const parent = (i - 1) >> 1;
if (better(heap[i], heap[parent])) {
swap(i, parent);
i = parent;
} else {
break;
}
}
};
const siftDown = (i: number) => {
for (;;) {
const left = i * 2 + 1;
const right = left + 1;
let worst = i;
if (left < heap.length && better(heap[left], heap[worst])) worst = left;
if (right < heap.length && better(heap[right], heap[worst])) worst = right;
if (worst === i) break;
swap(i, worst);
i = worst;
}
};
for (const entry of scored) {
if (heap.length < limit) {
heap.push(entry);
siftUp(heap.length - 1);
} else if (better(entry, heap[0])) {
heap[0] = entry;
siftDown(0);
}
}
heap.sort(compareScoredDefs);
return heap;
}
/**
* Replacement range for a child-element completion in element content:
* - when the user typed "<" (optionally followed by a partial name), the
* "<" is kept and the range covers the name area after it;
* - a partial name typed without "<" is replaced as a word;
* - whitespace-only content leaves the range empty at the cursor.
* Typing a closing tag ("</" / "</Name") never suggests child elements.
*/
function contentElementRange(
text: string,
offset: number,
): { rangeStart: number; typedOpen: boolean } {
let j = offset;
while (j > 0 && /[ \t]/.test(text[j - 1])) j--;
let i = j;
while (i > 0 && /[A-Za-z0-9_:.-]/.test(text[i - 1])) i--;
if (i >= 2 && text.slice(i - 2, i) === "</") {
return { rangeStart: offset, typedOpen: false };
}
if (i > 0 && text[i - 1] === "<") {
return { rangeStart: i, typedOpen: true };
}
if (i < j) return { rangeStart: i, typedOpen: false };
return { rangeStart: offset, typedOpen: false };
}
interface AttributeInsertLayout {
/** Offset where the completed attribute name starts replacing the text. */
rangeStart: number;
/** Text to insert before the attribute name (space / newline + indent). */
prefix: string;
}
/**
* Computes the insertion layout for an attribute-name completion:
* - a leading space when the cursor sits directly against a closing quote;
* - a plain newline when the element's attributes are laid out one per line
* (the editor supplies the new line's base indentation itself; embedding
* our own indent here would be ADDED on top of it, e.g. 3+3=6, 6+3=9);
* - replacing the current line's whitespace with that indentation when the
* user already started a new line.
*
* The indentation anchor is deliberately NOT the last parsed attribute:
* a half-typed attribute name on the cursor's new line has no value yet, and
* using its line indentation would copy the editor's own auto-indent (which
* can grow line by line) into every inserted attribute. Instead we use the
* first complete attribute that starts on its own line, which is stable and
* pre-existing, and only fall back to the last complete attribute when the
* whole element is inline.
*/
function attributeInsertLayout(
text: string,
el: XmlElement,
offset: number,
): AttributeInsertLayout {
const wordStart = findAttributeWordStart(text, offset, el.start);
const attrs = el.attrs;
const complete = attrs.filter((a) => a.hasValue);
const last = complete.length ? complete[complete.length - 1] : null;
const lastEnd = last ? attributeEndOffset(last) : -1;
const alreadyOnNewLine = lastEnd >= 0 && text.slice(lastEnd, offset).includes("\n");
// Canonical indent anchor: the first complete attribute that starts on its
// own line. Fall back to the last complete attribute for inline elements.
let anchor: XmlAttribute | null = null;
let onePerLine = false;
let prevEnd = el.start + 1 + el.name.length;
for (const a of complete) {
if (text.slice(prevEnd, a.nameStart).includes("\n")) {
anchor = a;
onePerLine = true;
break;
}
prevEnd = attributeEndOffset(a);
}
if (!anchor && complete.length) anchor = complete[complete.length - 1];
const indent = anchor
? text.slice(0, anchor.nameStart).match(/[ \t]*$/)?.[0] ?? ""
: "";
if (!onePerLine) {
if (alreadyOnNewLine) {
// Inline-style file, but the user started a new line: keep whatever
// indentation they already typed.
return { rangeStart: wordStart, prefix: "" };
}
const needsSpace = wordStart > el.start + 1 && !/\s/.test(text[wordStart - 1]);
return { rangeStart: wordStart, prefix: needsSpace ? " " : "" };
}
if (alreadyOnNewLine) {
const lineStart = text.lastIndexOf("\n", offset - 1) + 1;
return { rangeStart: lineStart, prefix: indent };
}
// Insert on a new line. The editor adds the current line's indentation to
// the new line, so we must NOT embed our own indent here (it would
// compound). If whitespace was typed between the previous attribute and
// the cursor (e.g. a space used to trigger the suggestion popup), consume
// it so it does not linger as a trailing space.
const wsStart =
lastEnd >= 0 &&
wordStart > lastEnd &&
/^[ \t]*$/.test(text.slice(lastEnd, wordStart))
? lastEnd
: wordStart;
return { rangeStart: wsStart, prefix: "\n" };
}
function attributeEndOffset(attr: XmlAttribute): number {
return attr.quoteEnd >= 0 ? attr.quoteEnd : attr.nameEnd;
}
function findAttributeWordStart(text: string, offset: number, tagStart: number): number {
let i = offset; let i = offset;
const text = document.getText();
while (i > tagStart) { while (i > tagStart) {
const c = text[i - 1]; const c = text[i - 1];
if (/[\s=<>"/]/.test(c)) break; if (/[\s=<>"/]/.test(c)) break;
@@ -398,3 +878,23 @@ function findAttributeWordStart(
} }
return i; return i;
} }
/** Type-appropriate example values for common RA3 XSD scalar types. */
const DEFAULT_VALUE_BY_TYPE: Record<string, string> = {
angle: "0d",
time: "0s",
velocity: "0.0",
percentage: "100%",
sagereal: "0.0",
sageint: "0",
sageunsignedint: "0",
float: "0.0",
double: "0.0",
int: "0",
unsignedint: "0",
unsignedbyte: "0",
byte: "0",
short: "0",
long: "0",
decimal: "0.0",
};
+157 -25
View File
@@ -1,15 +1,20 @@
import * as vscode from "vscode"; import * as vscode from "vscode";
import { dirname } from "node:path"; import { dirname } from "node:path";
import { LineMap, parseXml, type XmlElement } from "../language/xmlParser"; import { LineMap, type XmlElement } from "../language/xmlParser";
import { resolveElementType } from "../language/typeContext"; import { resolveElementType } from "../language/typeContext";
import { resolveSource, buildSearchPaths } from "../indexer/includeResolver"; import { resolveSource, buildSearchPaths } from "../indexer/includeResolver";
import * as model from "../model/schemaModel"; import * as model from "../model/schemaModel";
import type { ModWorkspace } from "../workspace"; import type { ModWorkspace } from "../workspace";
import type { ModIndex } from "../indexer/types"; import type { ModIndex } from "../indexer/types";
import { import {
isReferenceContentType,
isReferenceAttributeOfType, isReferenceAttributeOfType,
mergeLocalAndGlobalDefs,
resolveContentReferenceTargets,
resolveReferenceTargetsForType, resolveReferenceTargetsForType,
} from "../indexer/refs"; } from "../indexer/refs";
import type { LogicalElement } from "../indexer/logicalTree";
import { scopePathKey } from "../indexer/localScope";
export class Ra3Diagnostics { export class Ra3Diagnostics {
private collection: vscode.DiagnosticCollection; private collection: vscode.DiagnosticCollection;
@@ -19,15 +24,19 @@ export class Ra3Diagnostics {
} }
async update(document: vscode.TextDocument): Promise<void> { async update(document: vscode.TextDocument): Promise<void> {
const idx = this.ws.index; if (!this.ws.isRa3Workspace()) {
if (!idx) {
this.collection.set(document.uri, []); this.collection.set(document.uri, []);
return; return;
} }
const scope = await this.ws.getScope(document);
const idx = scope.merged;
const text = document.getText(); const text = document.getText();
const lineMap = new LineMap(text); const lineMap = new LineMap(text);
const doc = parseXml(text); const doc = scope.parse;
const diags: vscode.Diagnostic[] = []; const diags: vscode.Diagnostic[] = [];
// Reference/duplicate checks are provisional while the index is
// incomplete or stale: "not found" may be a false positive.
const provisional = idx ? !idx.complete || idx.stale === true : false;
for (const err of doc.errors) { for (const err of doc.errors) {
diags.push( diags.push(
@@ -44,7 +53,15 @@ export class Ra3Diagnostics {
} }
if (doc.root) { if (doc.root) {
this.checkElements(doc.root, doc, lineMap, idx, document, diags); this.checkElements(
scope.expanded.root,
scope.expanded,
lineMap,
idx,
document,
diags,
provisional,
);
} }
this.collection.set(document.uri, diags); this.collection.set(document.uri, diags);
@@ -59,19 +76,29 @@ export class Ra3Diagnostics {
} }
private checkElements( private checkElements(
root: XmlElement, root: LogicalElement | null,
doc: { elements: XmlElement[] }, doc: { elements: LogicalElement[] },
lineMap: LineMap, lineMap: LineMap,
idx: ModIndex, idx: ModIndex | null,
document: vscode.TextDocument, document: vscode.TextDocument,
diags: vscode.Diagnostic[], diags: vscode.Diagnostic[],
provisional: boolean,
): void { ): void {
const settings = this.ws.settings; const settings = this.ws.settings;
const fileDuplicates = new Map<string, { line: number }>(); const fileDuplicates = new Map<string, { line: number }>();
for (const el of doc.elements) { for (const el of doc.elements) {
// Only report diagnostics for nodes that belong to the document being
// edited. Nodes spliced in through xi:include keep their own source
// file and are diagnosed when that file is opened.
if (scopePathKey(el.sourceFile) !== scopePathKey(document.uri.fsPath)) {
continue;
}
const local = localName(el.name); const local = localName(el.name);
const isTopLevel = el.parent === root && !["Tags", "Includes", "Defines"].includes(local); const isTopLevel =
root !== null &&
el.parent === root &&
!["Tags", "Includes", "Defines"].includes(local);
const range = tagRange(document, el); const range = tagRange(document, el);
// Top-level assets must have an id. // Top-level assets must have an id.
@@ -111,6 +138,7 @@ export class Ra3Diagnostics {
document, document,
idx, idx,
diags, diags,
provisional,
); );
} }
} }
@@ -169,8 +197,10 @@ export class Ra3Diagnostics {
document, document,
idx, idx,
diags, diags,
provisional,
); );
} }
this.checkContentReferences(el, elType, document, idx, diags, provisional);
} }
// Include-specific checks. // Include-specific checks.
@@ -184,12 +214,17 @@ export class Ra3Diagnostics {
type: string, type: string,
id: string, id: string,
document: vscode.TextDocument, document: vscode.TextDocument,
idx: ModIndex, idx: ModIndex | null,
diags: vscode.Diagnostic[], diags: vscode.Diagnostic[],
provisional: boolean,
): void { ): void {
if (!idx) return;
const byType = idx.assets.get(type); const byType = idx.assets.get(type);
const defs = byType?.get(id.toLowerCase()); const defs = mergeLocalAndGlobalDefs(
if (!defs || defs.length < 2) return; idx.local?.assets.get(type)?.get(id.toLowerCase()),
byType?.get(id.toLowerCase()),
);
if (defs.length < 2) return;
const self = defs.filter( const self = defs.filter(
(d) => (d) =>
d.origin === "project" && d.origin === "project" &&
@@ -211,7 +246,8 @@ export class Ra3Diagnostics {
diags.push( diags.push(
this.diag( this.diag(
range, range,
`Duplicate id "${id}" for <${type}> (also defined in ${other.file})`, `Duplicate id "${id}" for <${type}> (also defined in ${other.file})` +
(provisional ? " (based on a partial index)" : ""),
vscode.DiagnosticSeverity.Error, vscode.DiagnosticSeverity.Error,
"duplicate-id", "duplicate-id",
), ),
@@ -225,8 +261,9 @@ export class Ra3Diagnostics {
value: string, value: string,
attr: { valueStart: number; valueEnd: number }, attr: { valueStart: number; valueEnd: number },
document: vscode.TextDocument, document: vscode.TextDocument,
idx: ModIndex, idx: ModIndex | null,
diags: vscode.Diagnostic[], diags: vscode.Diagnostic[],
provisional: boolean,
): void { ): void {
if (!value) return; if (!value) return;
const range = new vscode.Range( const range = new vscode.Range(
@@ -238,13 +275,19 @@ export class Ra3Diagnostics {
const defineRe = /\$([A-Za-z_][A-Za-z0-9_]*)/g; const defineRe = /\$([A-Za-z_][A-Za-z0-9_]*)/g;
let m: RegExpExecArray | null; let m: RegExpExecArray | null;
while ((m = defineRe.exec(value)) !== null) { while ((m = defineRe.exec(value)) !== null) {
if (!idx.defines.has(m[1].toLowerCase())) { if (
idx &&
!(idx.local?.defines.has(m[1].toLowerCase()) ??
idx.defines.has(m[1].toLowerCase()))
) {
const code = provisional ? "undefined-define-indexing" : "undefined-define";
diags.push( diags.push(
this.diag( this.diag(
range, range,
`Undefined define "$${m[1]}"`, `Undefined define "$${m[1]}"` +
(provisional ? " (index incomplete — may be a false positive)" : ""),
vscode.DiagnosticSeverity.Warning, vscode.DiagnosticSeverity.Warning,
"undefined-define", code,
), ),
); );
} }
@@ -253,10 +296,13 @@ export class Ra3Diagnostics {
if (value.startsWith("$") || value.startsWith("=")) return; if (value.startsWith("$") || value.startsWith("=")) return;
const severity = this.ws.settings.reportUnresolvedReferences; const severity = this.ws.settings.reportUnresolvedReferences;
if (severity === "none") return; if (severity === "none") return;
if (!idx) return;
if (!isReferenceAttributeOfType(elType, attrName)) return; if (!isReferenceAttributeOfType(elType, attrName)) return;
const targets = resolveReferenceTargetsForType(idx, elType, attrName, value); const targets = resolveReferenceTargetsForType(idx, elType, attrName, value);
if (targets.length) return; if (targets.length) return;
const anyDef = idx.assetsById.has(value.toLowerCase()); const anyDef =
(idx.local?.assetsById.has(value.toLowerCase()) ?? false) ||
idx.assetsById.has(value.toLowerCase());
const attrRef = model const attrRef = model
.attributesOfType(elType) .attributesOfType(elType)
.find((a) => a.name === attrName); .find((a) => a.name === attrName);
@@ -265,16 +311,96 @@ export class Ra3Diagnostics {
: attrRef?.isRef : attrRef?.isRef
? "of the expected declared type" ? "of the expected declared type"
: "matching"; : "matching";
const code = provisional ? "unresolved-reference-indexing" : "unresolved-reference";
const baseMessage = anyDef
? `Reference "${value}" has no definition ${expected} (ids with the same name exist for other types)`
: `Unresolved reference "${value}" (not found in the current index)`;
diags.push( diags.push(
this.diag( this.diag(
range, range,
anyDef provisional
? `Reference "${value}" has no definition ${expected} (ids with the same name exist for other types)` ? `${baseMessage} (index incomplete — may be a false positive)`
: `Unresolved reference "${value}" (not found in the current index)`, : baseMessage,
severity === "warning" severity === "warning"
? vscode.DiagnosticSeverity.Warning ? vscode.DiagnosticSeverity.Warning
: vscode.DiagnosticSeverity.Information, : vscode.DiagnosticSeverity.Information,
"unresolved-reference", code,
),
);
}
private checkContentReferences(
el: XmlElement,
elType: string | null,
document: vscode.TextDocument,
idx: ModIndex | null,
diags: vscode.Diagnostic[],
provisional: boolean,
): void {
// Only simple-content elements carry a text value; complex elements'
// "content" is child markup and must not be scanned for value refs.
const info = elType ? model.typeInfo(elType) : undefined;
if (info?.kind !== "simple") return;
if (el.selfClosing || el.closeTagStart < 0) return;
const text = document.getText();
const raw = text.slice(el.startTagEnd, el.closeTagStart);
const value = raw.trim();
if (!value) return;
const valueStart = el.startTagEnd + raw.indexOf(value);
const range = new vscode.Range(
document.positionAt(valueStart),
document.positionAt(valueStart + value.length),
);
// Undefined $DEFINE references.
const defineRe = /\$([A-Za-z_][A-Za-z0-9_]*)/g;
let m: RegExpExecArray | null;
while ((m = defineRe.exec(value)) !== null) {
if (
idx &&
!(
idx.local?.defines.has(m[1].toLowerCase()) ??
idx.defines.has(m[1].toLowerCase())
)
) {
const code = provisional ? "undefined-define-indexing" : "undefined-define";
diags.push(
this.diag(
range,
`Undefined define "$${m[1]}"` +
(provisional ? " (index incomplete — may be a false positive)" : ""),
vscode.DiagnosticSeverity.Warning,
code,
),
);
}
}
if (value.startsWith("$") || value.startsWith("=")) return;
const severity = this.ws.settings.reportUnresolvedReferences;
if (severity === "none" || !idx) return;
if (!isReferenceContentType(elType)) return;
const targets = resolveContentReferenceTargets(idx, elType, value);
if (targets.length) return;
const anyDef =
(idx.local?.assetsById.has(value.toLowerCase()) ?? false) ||
idx.assetsById.has(value.toLowerCase());
const refType = info.refType;
const expected = refType ? `of type \`${refType}\`` : "of the expected declared type";
const code = provisional ? "unresolved-reference-indexing" : "unresolved-reference";
const baseMessage = anyDef
? `Reference "${value}" has no definition ${expected} (ids with the same name exist for other types)`
: `Unresolved reference "${value}" (not found in the current index)`;
diags.push(
this.diag(
range,
provisional
? `${baseMessage} (index incomplete — may be a false positive)`
: baseMessage,
severity === "warning"
? vscode.DiagnosticSeverity.Warning
: vscode.DiagnosticSeverity.Information,
code,
), ),
); );
} }
@@ -282,7 +408,7 @@ export class Ra3Diagnostics {
private checkInclude( private checkInclude(
el: XmlElement, el: XmlElement,
document: vscode.TextDocument, document: vscode.TextDocument,
idx: ModIndex, idx: ModIndex | null,
diags: vscode.Diagnostic[], diags: vscode.Diagnostic[],
): void { ): void {
const typeAttr = el.attrs.find((a) => a.name === "type"); const typeAttr = el.attrs.find((a) => a.name === "type");
@@ -301,12 +427,18 @@ export class Ra3Diagnostics {
); );
} }
if (!sourceAttr?.hasValue) return; if (!sourceAttr?.hasValue) return;
const searchPaths = idx
? buildSearchPaths(idx.sdkDir, idx.projectDir)
: this.ws.searchPaths();
if (!searchPaths) return;
const resolved = resolveSource( const resolved = resolveSource(
sourceAttr.value, sourceAttr.value,
dirname(document.uri.fsPath), dirname(document.uri.fsPath),
buildSearchPaths(idx.sdkDir, idx.projectDir), searchPaths,
); );
if (!resolved.path && !idx.sourceCandidates.some((c) => c.source === sourceAttr.value)) { const candidateHit =
idx?.sourceCandidates.some((c) => c.source === sourceAttr.value) ?? false;
if (!resolved.path && !candidateHit) {
diags.push( diags.push(
this.diag( this.diag(
new vscode.Range( new vscode.Range(
+148 -35
View File
@@ -1,13 +1,22 @@
import * as vscode from "vscode"; import * as vscode from "vscode";
import { findElementAt, parseXml } from "../language/xmlParser"; import { findElementAt, textContentTokenAt } from "../language/xmlParser";
import { resolveElementType } from "../language/typeContext"; import { resolveElementType } from "../language/typeContext";
import * as model from "../model/schemaModel"; import * as model from "../model/schemaModel";
import type { ModWorkspace } from "../workspace"; import type { ModWorkspace } from "../workspace";
import type { ModIndex } from "../indexer/types";
import { import {
isLocalReferenceAttribute,
isReferenceAttributeOfType, isReferenceAttributeOfType,
isReferenceContentType,
resolveContentReferenceTargets,
resolveReferenceTargetsForType, resolveReferenceTargetsForType,
type ReferenceTarget,
} from "../indexer/refs"; } from "../indexer/refs";
import {
findContainingGameObject,
findLocalId,
type LogicalElement,
} from "../indexer/logicalTree";
import { scopePathKey, type DocumentScope } from "../indexer/localScope";
import { dirname } from "node:path"; import { dirname } from "node:path";
import { buildSearchPaths, resolveSource } from "../indexer/includeResolver"; import { buildSearchPaths, resolveSource } from "../indexer/includeResolver";
@@ -19,9 +28,11 @@ export class Ra3HoverProvider implements vscode.HoverProvider {
position: vscode.Position, position: vscode.Position,
_token: vscode.CancellationToken, _token: vscode.CancellationToken,
): Promise<vscode.Hover | null> { ): Promise<vscode.Hover | null> {
const text = document.getText(); if (!this.ws.isRa3Workspace()) return null;
const offset = document.offsetAt(position); const offset = document.offsetAt(position);
const doc = parseXml(text); const text = document.getText();
const scope = await this.ws.getScope(document);
const doc = scope.expanded;
const el = findElementAt(doc, offset); const el = findElementAt(doc, offset);
if (!el) return null; if (!el) return null;
const elType = resolveElementType(el); const elType = resolveElementType(el);
@@ -35,9 +46,15 @@ export class Ra3HoverProvider implements vscode.HoverProvider {
// Attribute value. // Attribute value.
for (const attr of el.attrs) { for (const attr of el.attrs) {
if (attr.hasValue && offset >= attr.valueStart && offset <= attr.valueEnd) { if (attr.hasValue && offset >= attr.valueStart && offset <= attr.valueEnd) {
return this.valueHover(el, elType, attr.name, attr.value, document, this.ws.index); return this.valueHover(el, elType, attr.name, attr.value, document, scope);
} }
} }
// Element text content (e.g. <CreateObject>CrateDebris_01</CreateObject>).
const contentToken = textContentTokenAt(text, el, offset);
if (contentToken) {
const h = this.contentHover(elType, contentToken.value, document, scope);
if (h) return h;
}
// Element name. // Element name.
const nameStart = el.start + 1; const nameStart = el.start + 1;
if (offset >= nameStart && offset <= nameStart + el.name.length) { if (offset >= nameStart && offset <= nameStart + el.name.length) {
@@ -116,22 +133,19 @@ export class Ra3HoverProvider implements vscode.HoverProvider {
attrName: string, attrName: string,
value: string, value: string,
document: vscode.TextDocument, document: vscode.TextDocument,
idx: ModIndex | null, scope: DocumentScope,
): vscode.Hover | null { ): vscode.Hover | null {
const idx = scope.merged;
const md = new vscode.MarkdownString(); const md = new vscode.MarkdownString();
// $DEFINE reference. // $DEFINE reference.
const defineMatch = /\$([A-Za-z_][A-Za-z0-9_]*)/.exec(value); const defineMatch = /\$([A-Za-z_][A-Za-z0-9_]*)/.exec(value);
if (defineMatch && idx) { if (defineMatch && idx) {
const defs = idx.defines.get(defineMatch[1].toLowerCase()); const defs =
if (defs?.length) { idx.local?.defines.get(defineMatch[1].toLowerCase()) ??
const d = defs[0]; idx.defines.get(defineMatch[1].toLowerCase());
md.appendMarkdown(`**Define** \`$${d.name}\` \n`); const h = this.defineHover(defs, document);
md.appendCodeblock(d.value); if (h) return h;
const rel = relativePath(document, d.file);
md.appendMarkdown(`Defined in \`${rel}:${d.line}\``);
return new vscode.Hover(md);
}
} }
// Include source. // Include source.
@@ -139,11 +153,14 @@ export class Ra3HoverProvider implements vscode.HoverProvider {
(el.name === "Include" && attrName === "source") || (el.name === "Include" && attrName === "source") ||
(el.name === "xi:include" && attrName === "href") (el.name === "xi:include" && attrName === "href")
) { ) {
const resolved = idx const searchPaths = idx
? buildSearchPaths(idx.sdkDir, idx.projectDir)
: this.ws.searchPaths();
const resolved = searchPaths
? resolveSource( ? resolveSource(
value, value,
dirname(document.uri.fsPath), dirname(document.uri.fsPath),
buildSearchPaths(idx.sdkDir, idx.projectDir), searchPaths,
).path ).path
: null; : null;
if (resolved) { if (resolved) {
@@ -161,22 +178,29 @@ export class Ra3HoverProvider implements vscode.HoverProvider {
return new vscode.Hover(md); return new vscode.Hover(md);
} }
// Pipeline-local (Poid) references: resolve inside the enclosing
// GameObject's logical subtree (including xi:include targets).
if (
isLocalReferenceAttribute(elType, attrName) &&
attrName.toLowerCase() !== "id"
) {
return this.localIdHover(scope, el as LogicalElement, value, document);
}
// Asset reference / inheritFrom. // Asset reference / inheritFrom.
if (idx) { if (!idx) {
if (isReferenceAttributeOfType(elType, attrName)) {
md.appendMarkdown(
"Index is still building — references cannot be resolved yet.",
);
return new vscode.Hover(md);
}
return null;
}
{
if (!isReferenceAttributeOfType(elType, attrName)) return null; if (!isReferenceAttributeOfType(elType, attrName)) return null;
const targets = resolveReferenceTargetsForType(idx, elType, attrName, value); const targets = resolveReferenceTargetsForType(idx, elType, attrName, value);
if (targets.length) { if (targets.length) return this.definitionsHover(targets, document);
const md2 = new vscode.MarkdownString();
md2.appendMarkdown(`**${targets.length} definition${targets.length > 1 ? "s" : ""}** \n`);
for (const { def: d } of targets.slice(0, 8)) {
const loc =
d.origin === "manifest"
? `manifest \`${d.manifestSource ?? d.file}\``
: `\`${relativePath(document, d.file)}:${d.line}\``;
md2.appendMarkdown(`- \`${d.type}\` · ${loc} \n`);
}
return new vscode.Hover(md2);
}
const attrRef = model const attrRef = model
.attributesOfType(elType) .attributesOfType(elType)
.find((a) => a.name === attrName); .find((a) => a.name === attrName);
@@ -185,14 +209,103 @@ export class Ra3HoverProvider implements vscode.HoverProvider {
: attrRef?.isRef : attrRef?.isRef
? " of the expected declared type" ? " of the expected declared type"
: ""; : "";
md.appendMarkdown( return this.noDefinitionHover(expected);
`No matching definition${expected} in the current index` + }
" (may exist in a compiled manifest or vanilla data).", }
);
/**
* Hover for text inside a simple-content element whose type is a typed
* asset reference (e.g. <CreateObject> with GameObjectWeakRef).
*/
private contentHover(
elType: string | null,
value: string,
document: vscode.TextDocument,
scope: DocumentScope,
): vscode.Hover | null {
const idx = scope.merged;
const defineMatch = /\$([A-Za-z_][A-Za-z0-9_]*)/.exec(value);
if (defineMatch && idx) {
const defs =
idx.local?.defines.get(defineMatch[1].toLowerCase()) ??
idx.defines.get(defineMatch[1].toLowerCase());
const h = this.defineHover(defs, document);
if (h) return h;
}
if (!isReferenceContentType(elType)) return null;
if (!idx) {
const md = new vscode.MarkdownString();
md.appendMarkdown("Index is still building — references cannot be resolved yet.");
return new vscode.Hover(md); return new vscode.Hover(md);
} }
const targets = resolveContentReferenceTargets(idx, elType, value);
if (targets.length) return this.definitionsHover(targets, document);
const info = elType ? model.typeInfo(elType) : undefined;
const refType = info?.kind === "simple" ? info.refType : null;
return this.noDefinitionHover(
refType ? ` of type \`${refType}\`` : " of the expected declared type",
);
}
return null; private defineHover(
defs: { name: string; value: string; file: string; line: number }[] | undefined,
document: vscode.TextDocument,
): vscode.Hover | null {
if (!defs?.length) return null;
const d = defs[0];
const md = new vscode.MarkdownString();
md.appendMarkdown(`**Define** \`$${d.name}\` \n`);
md.appendCodeblock(d.value);
const rel = relativePath(document, d.file);
md.appendMarkdown(`Defined in \`${rel}:${d.line}\``);
return new vscode.Hover(md);
}
private definitionsHover(
targets: ReferenceTarget[],
document: vscode.TextDocument,
): vscode.Hover {
const md2 = new vscode.MarkdownString();
md2.appendMarkdown(`**${targets.length} definition${targets.length > 1 ? "s" : ""}** \n`);
for (const { def: d } of targets.slice(0, 8)) {
const loc =
d.origin === "manifest"
? `manifest \`${d.manifestSource ?? d.file}\``
: `\`${relativePath(document, d.file)}:${d.line}\``;
md2.appendMarkdown(`- \`${d.type}\` · ${loc} \n`);
}
return new vscode.Hover(md2);
}
private noDefinitionHover(expected: string): vscode.Hover {
const md = new vscode.MarkdownString();
md.appendMarkdown(
`No matching definition${expected} in the current index` +
" (may exist in a compiled manifest or vanilla data).",
);
return new vscode.Hover(md);
}
private localIdHover(
scope: DocumentScope,
el: LogicalElement,
value: string,
document: vscode.TextDocument,
): vscode.Hover | null {
const root = findContainingGameObject(el);
if (!root) return null;
const target = findLocalId(root, value);
if (!target) return null;
const idAttr = target.attrs.find((a) => a.name === "id");
if (!idAttr?.hasValue) return null;
const lineMap = scope.lineMaps.get(scopePathKey(target.sourceFile));
const line = lineMap ? lineMap.positionAt(idAttr.valueStart).line + 1 : 0;
const md = new vscode.MarkdownString();
md.appendMarkdown(`**Local pipeline id** \`${value}\` \n`);
md.appendCodeblock(`<${target.name}>`);
const rel = relativePath(document, target.sourceFile);
md.appendMarkdown(`Defined in \`${rel}:${line}\``);
return new vscode.Hover(md);
} }
} }
+163 -92
View File
@@ -1,13 +1,26 @@
import * as vscode from "vscode"; import * as vscode from "vscode";
import { dirname } from "node:path"; import { dirname } from "node:path";
import { findElementAt, parseXml } from "../language/xmlParser"; import { findElementAt, parseXml, textContentTokenAt } from "../language/xmlParser";
import { resolveElementType } from "../language/typeContext"; import { resolveElementType } from "../language/typeContext";
import { import {
buildSearchPaths, buildSearchPaths,
resolveSource, resolveSource,
type SearchPaths, type SearchPaths,
} from "../indexer/includeResolver"; } from "../indexer/includeResolver";
import { resolveReferenceTargetsForType } from "../indexer/refs"; import {
isLocalReferenceAttribute,
isReferenceContentType,
resolveContentReferenceTargets,
resolveReferenceTargetsForType,
type ReferenceTarget,
} from "../indexer/refs";
import { findReferenceLocations } from "./references";
import {
findContainingGameObject,
findLocalId,
type LogicalElement,
} from "../indexer/logicalTree";
import { scopePathKey, type DocumentScope } from "../indexer/localScope";
import type { ModWorkspace } from "../workspace"; import type { ModWorkspace } from "../workspace";
import type { AssetDef, ModIndex } from "../indexer/types"; import type { AssetDef, ModIndex } from "../indexer/types";
@@ -25,11 +38,11 @@ export class Ra3DefinitionProvider implements vscode.DefinitionProvider {
position: vscode.Position, position: vscode.Position,
_token: vscode.CancellationToken, _token: vscode.CancellationToken,
): Promise<vscode.Location | vscode.Location[] | null> { ): Promise<vscode.Location | vscode.Location[] | null> {
const idx = this.ws.index; if (!this.ws.isRa3Workspace()) return null;
if (!idx) return null; const scope = await this.ws.getScope(document);
const text = document.getText(); const idx = scope.merged;
const offset = document.offsetAt(position); const offset = document.offsetAt(position);
const doc = parseXml(text); const doc = scope.expanded;
const el = findElementAt(doc, offset); const el = findElementAt(doc, offset);
if (!el) return null; if (!el) return null;
const elType = resolveElementType(el); const elType = resolveElementType(el);
@@ -37,7 +50,19 @@ export class Ra3DefinitionProvider implements vscode.DefinitionProvider {
const attr = el.attrs.find( const attr = el.attrs.find(
(a) => a.hasValue && offset >= a.valueStart && offset <= a.valueEnd, (a) => a.hasValue && offset >= a.valueStart && offset <= a.valueEnd,
); );
if (!attr) return null; if (!attr) {
// Element text content (e.g. <CreateObject>CrateDebris_01</CreateObject>).
if (idx && isReferenceContentType(elType)) {
const token = textContentTokenAt(document.getText(), el, offset);
if (token && !token.value.startsWith("$")) {
const targets = resolveContentReferenceTargets(idx, elType, token.value);
if (targets.length) {
return this.referenceLocations(scope, idx, targets, document);
}
}
}
return null;
}
const value = attr.value; const value = attr.value;
const nameLower = attr.name.toLowerCase(); const nameLower = attr.name.toLowerCase();
@@ -46,34 +71,85 @@ export class Ra3DefinitionProvider implements vscode.DefinitionProvider {
(el.name === "Include" && nameLower === "source") || (el.name === "Include" && nameLower === "source") ||
(el.name === "include" && nameLower === "href") (el.name === "include" && nameLower === "href")
) { ) {
const resolved = const searchPaths = idx ? searchPathsFor(idx) : this.ws.searchPaths();
resolveSource(value, dirname(document.uri.fsPath), searchPathsFor(idx)).path ?? const resolved = searchPaths
idx.sourceCandidates.find((c) => c.source === value)?.path ?? ? resolveSource(value, dirname(document.uri.fsPath), searchPaths).path
null; : null;
return resolved const fallback = idx?.sourceCandidates.find((c) => c.source === value)?.path;
? new vscode.Location(vscode.Uri.file(resolved), new vscode.Position(0, 0)) const target = resolved ?? fallback ?? null;
return target
? new vscode.Location(vscode.Uri.file(target), new vscode.Position(0, 0))
: null; : null;
} }
// Asset reference / inheritFrom (filtered by the attribute's ref type). // Asset reference / inheritFrom (filtered by the attribute's ref type).
if (!idx) return null;
if (value && !value.startsWith("$")) { if (value && !value.startsWith("$")) {
let targets = resolveReferenceTargetsForType(idx, elType, attr.name, value);
if (!targets.length) return null;
if ( if (
this.ws.settings.definitionMode === "project-only" && isLocalReferenceAttribute(elType, attr.name) &&
targets.some((t) => t.def.origin === "project") nameLower !== "id"
) { ) {
targets = targets.filter((t) => t.def.origin === "project"); const local = this.localIdLocation(
scope,
el as LogicalElement,
value,
);
if (local) return local;
} }
const locations: vscode.Location[] = []; const targets = resolveReferenceTargetsForType(idx, elType, attr.name, value);
for (const { def } of targets.slice(0, 8)) { if (!targets.length) return null;
const loc = await assetDefLocation(this.ws, def, idx); return this.referenceLocations(scope, idx, targets, document);
if (loc) locations.push(loc);
}
return locations.length ? locations : null;
} }
return null; return null;
} }
private async referenceLocations(
scope: DocumentScope,
idx: ModIndex,
targets: ReferenceTarget[],
document: vscode.TextDocument,
): Promise<vscode.Location[] | null> {
let filtered = targets;
if (
this.ws.settings.definitionMode === "project-only" &&
targets.some((t) => t.def.origin === "project")
) {
filtered = targets.filter((t) => t.def.origin === "project");
}
const locations: vscode.Location[] = [];
for (const { def } of filtered.slice(0, 8)) {
const loc = await assetDefLocation(this.ws, def, idx, scope, document);
if (loc) locations.push(loc);
}
return locations.length ? locations : null;
}
private localIdLocation(
scope: DocumentScope,
el: LogicalElement,
value: string,
): vscode.Location | null {
const root = findContainingGameObject(el);
if (!root) return null;
const target = findLocalId(root, value);
if (!target) return null;
const idAttr = target.attrs.find((a) => a.name === "id");
if (!idAttr?.hasValue) return null;
const lineMap = scope.lineMaps.get(scopePathKey(target.sourceFile));
if (!lineMap) {
return new vscode.Location(
vscode.Uri.file(target.sourceFile),
new vscode.Position(0, 0),
);
}
return new vscode.Location(
vscode.Uri.file(target.sourceFile),
new vscode.Range(
toVscodePosition(lineMap.positionAt(idAttr.valueStart)),
toVscodePosition(lineMap.positionAt(idAttr.valueEnd)),
),
);
}
} }
/** /**
@@ -85,7 +161,21 @@ async function assetDefLocation(
ws: ModWorkspace, ws: ModWorkspace,
def: AssetDef, def: AssetDef,
idx: ModIndex, idx: ModIndex,
scope: DocumentScope,
currentDocument: vscode.TextDocument,
): Promise<vscode.Location | null> { ): Promise<vscode.Location | null> {
// While a rebuild is running, avoid readDom() mutating the live indexer's
// caches mid-build; a line-based location is a fine temporary fallback.
if (ws.isBuilding) {
const line = Math.max(0, def.line - 1);
return new vscode.Location(
vscode.Uri.file(def.file),
new vscode.Range(
new vscode.Position(line, 0),
new vscode.Position(line, 1),
),
);
}
if (def.origin === "manifest") { if (def.origin === "manifest") {
const src = def.manifestSource; const src = def.manifestSource;
if (src?.toUpperCase().startsWith("DATA:")) { if (src?.toUpperCase().startsWith("DATA:")) {
@@ -100,6 +190,11 @@ async function assetDefLocation(
return null; return null;
} }
if (scopePathKey(def.file) === scopePathKey(currentDocument.uri.fsPath)) {
const precise = locationInCurrentDocument(scope, def.id, currentDocument);
if (precise) return precise;
}
return ( return (
(await locationInDocument(ws, def.file, def.id)) ?? (await locationInDocument(ws, def.file, def.id)) ??
new vscode.Location( new vscode.Location(
@@ -109,6 +204,36 @@ async function assetDefLocation(
); );
} }
function locationInCurrentDocument(
scope: DocumentScope,
id: string,
document: vscode.TextDocument,
): vscode.Location | null {
const el = scope.parse.elements.find((e) =>
e.attrs.some(
(a) => a.name === "id" && a.value.toLowerCase() === id.toLowerCase(),
),
);
if (!el) return null;
const idAttr = el.attrs.find((a) => a.name === "id");
if (idAttr?.hasValue) {
return new vscode.Location(
document.uri,
new vscode.Range(
document.positionAt(idAttr.valueStart),
document.positionAt(idAttr.valueEnd),
),
);
}
return new vscode.Location(
document.uri,
new vscode.Range(
document.positionAt(el.start),
document.positionAt(el.startTagEnd),
),
);
}
/** /**
* Finds the precise range of an asset definition inside an XML file: the id * Finds the precise range of an asset definition inside an XML file: the id
* attribute value when present, otherwise the element start tag. * attribute value when present, otherwise the element start tag.
@@ -118,7 +243,9 @@ async function locationInDocument(
file: string, file: string,
id: string, id: string,
): Promise<vscode.Location | null> { ): Promise<vscode.Location | null> {
const parsed = await ws.indexer?.readDocument(file); // readDom (not readDocument) guarantees a DOM even when the compact
// records cache already has an entry for the file.
const parsed = await ws.indexer?.readDom(file);
if (parsed?.parse && parsed.lineMap) { if (parsed?.parse && parsed.lineMap) {
const el = parsed.parse.elements.find( const el = parsed.parse.elements.find(
(e) => (e) =>
@@ -156,39 +283,15 @@ function toVscodePosition(p: { line: number; character: number }): vscode.Positi
// ── Find all references ───────────────────────────────────────────── // ── Find all references ─────────────────────────────────────────────
export class Ra3ReferenceProvider implements vscode.ReferenceProvider { export class Ra3ReferenceProvider implements vscode.ReferenceProvider {
constructor(private ws: ModWorkspace) {}
async provideReferences( async provideReferences(
document: vscode.TextDocument, document: vscode.TextDocument,
position: vscode.Position, position: vscode.Position,
_context: vscode.ReferenceContext, _context: vscode.ReferenceContext,
_token: vscode.CancellationToken, _token: vscode.CancellationToken,
): Promise<vscode.Location[] | null> { ): Promise<vscode.Location[] | null> {
const text = document.getText(); return findReferenceLocations(this.ws, document, position);
const offset = document.offsetAt(position);
const doc = parseXml(text);
const el = findElementAt(doc, offset);
if (!el) return null;
const attr = el.attrs.find(
(a) =>
(a.hasValue && offset >= a.valueStart && offset <= a.valueEnd) ||
(offset >= a.nameStart && offset <= a.nameEnd),
);
if (!attr?.hasValue) return null;
const id = attr.value;
if (!id || id.startsWith("$")) return null;
const locations: vscode.Location[] = [];
const pattern = `["']${escapeRegExp(id)}["']`;
await findTextInWorkspace(
{ pattern, isRegExp: true },
{ include: "**/*.xml", maxResults: 2000 },
(result: { uri: vscode.Uri; matches: { range: vscode.Range }[] }) => {
if (!result.uri) return;
for (const m of result.matches) {
locations.push(new vscode.Location(result.uri, m.range));
}
},
);
return locations.length ? locations : null;
} }
} }
@@ -201,8 +304,10 @@ export class Ra3DocumentLinkProvider implements vscode.DocumentLinkProvider {
document: vscode.TextDocument, document: vscode.TextDocument,
_token: vscode.CancellationToken, _token: vscode.CancellationToken,
): Promise<vscode.DocumentLink[]> { ): Promise<vscode.DocumentLink[]> {
if (!this.ws.isRa3Workspace()) return [];
const idx = this.ws.index; const idx = this.ws.index;
if (!idx) return []; const searchPaths = idx ? searchPathsFor(idx) : this.ws.searchPaths();
if (!searchPaths) return [];
const text = document.getText(); const text = document.getText();
const doc = parseXml(text); const doc = parseXml(text);
const links: vscode.DocumentLink[] = []; const links: vscode.DocumentLink[] = [];
@@ -214,9 +319,9 @@ export class Ra3DocumentLinkProvider implements vscode.DocumentLinkProvider {
resolveSource( resolveSource(
srcAttr.value, srcAttr.value,
dirname(document.uri.fsPath), dirname(document.uri.fsPath),
searchPathsFor(idx), searchPaths,
).path ?? ).path ??
idx.sourceCandidates.find((c) => c.source === srcAttr.value)?.path ?? idx?.sourceCandidates.find((c) => c.source === srcAttr.value)?.path ??
null; null;
if (!target) continue; if (!target) continue;
links.push( links.push(
@@ -236,10 +341,13 @@ export class Ra3DocumentLinkProvider implements vscode.DocumentLinkProvider {
// ── Document symbols (outline) ────────────────────────────────────── // ── Document symbols (outline) ──────────────────────────────────────
export class Ra3DocumentSymbolProvider implements vscode.DocumentSymbolProvider { export class Ra3DocumentSymbolProvider implements vscode.DocumentSymbolProvider {
constructor(private ws: ModWorkspace) {}
async provideDocumentSymbols( async provideDocumentSymbols(
document: vscode.TextDocument, document: vscode.TextDocument,
_token: vscode.CancellationToken, _token: vscode.CancellationToken,
): Promise<vscode.DocumentSymbol[]> { ): Promise<vscode.DocumentSymbol[]> {
if (!this.ws.isRa3Workspace()) return [];
const text = document.getText(); const text = document.getText();
const doc = parseXml(text); const doc = parseXml(text);
const root = doc.root; const root = doc.root;
@@ -291,40 +399,3 @@ function localName(tag: string): string {
const idx = tag.lastIndexOf(":"); const idx = tag.lastIndexOf(":");
return idx >= 0 ? tag.slice(idx + 1) : tag; return idx >= 0 ? tag.slice(idx + 1) : tag;
} }
function escapeRegExp(s: string): string {
return s.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
}
/**
* `workspace.findTextInFiles` is a stable VS Code API (since 1.66) but is
* missing from the published typings, so we declare the subset we need and
* call it via a safe cast.
*/
interface TextSearchQuery {
pattern: string;
isRegExp?: boolean;
isCaseSensitive?: boolean;
isWordMatch?: boolean;
}
interface TextSearchOptions {
include?: string;
exclude?: string;
maxResults?: number;
}
function findTextInWorkspace(
query: TextSearchQuery,
options: TextSearchOptions,
callback: (result: { uri: vscode.Uri; matches: { range: vscode.Range }[] }) => void,
): Promise<void> {
const api = vscode.workspace as unknown as {
findTextInFiles(
query: TextSearchQuery,
options: TextSearchOptions,
callback: (result: { uri: vscode.Uri; matches: { range: vscode.Range }[] }) => void,
): Promise<unknown>;
};
return api.findTextInFiles(query, options, callback).then(() => undefined);
}
+227
View File
@@ -0,0 +1,227 @@
/**
* Shared semantic reference logic used by Find All References, the CodeLens
* "N references" command and (indirectly) the unreferenced-assets report.
*
* Unlike the old text-search implementation, every result here comes from
* the reverse reference index built during indexing, so the count shown on a
* top-level asset always matches the references peek opened by clicking it.
*/
import * as vscode from "vscode";
import {
findElementAt,
parseXml,
textContentTokenAt,
} from "../language/xmlParser";
import { resolveElementType } from "../language/typeContext";
import { attributesOfType, typeInfo } from "../model/schemaModel";
import {
filterAndScoreDefs,
isReferenceAttributeOfType,
isReferenceContentType,
mergeLocalAndGlobalDefs,
} from "../indexer/refs";
import {
referenceSitesForDef,
scheduleRebuildIfRecordsDesync,
} from "../indexer/referenceIndex";
import type { AssetDef, ModIndex, ReferenceSite } from "../indexer/types";
import type { ModWorkspace } from "../workspace";
export interface ReferenceContext {
id: string;
/** XSD refType when the cursor is on a typed reference; null otherwise. */
refType: string | null;
/** Element type for inheritFrom filtering; null otherwise. */
selfType: string | null;
}
/**
* Extracts the referenced id and its XSD context from the cursor position.
* Works on reference attribute values/names, simple-content text and on an
* asset's own `id` definition (where every same-id definition is a target).
*/
export function referenceContextAt(
document: vscode.TextDocument,
offset: number,
): ReferenceContext | null {
const text = document.getText();
const doc = parseXml(text);
const el = findElementAt(doc, offset);
if (!el) return null;
const elType = resolveElementType(el);
const attr = el.attrs.find(
(a) =>
(a.hasValue && offset >= a.valueStart && offset <= a.valueEnd) ||
(offset >= a.nameStart && offset <= a.nameEnd),
);
if (attr?.hasValue) {
const value = attr.value;
if (!value || value.startsWith("$") || value.startsWith("=")) return null;
const nameLower = attr.name.toLowerCase();
if (nameLower === "id") {
return { id: value, refType: null, selfType: null };
}
if (!isReferenceAttributeOfType(elType, attr.name)) return null;
if (nameLower === "inheritfrom") {
return { id: value, refType: null, selfType: elType };
}
const attrInfo = elType
? attributesOfType(elType).find((a) => a.name === attr.name)
: undefined;
return { id: value, refType: attrInfo?.refType ?? null, selfType: null };
}
if (elType && isReferenceContentType(elType)) {
const token = textContentTokenAt(text, el, offset);
if (token && !token.value.startsWith("$") && !token.value.startsWith("=")) {
const info = typeInfo(elType);
return {
id: token.value,
refType: info?.kind === "simple" ? info.refType : null,
selfType: null,
};
}
}
return null;
}
/** Definitions matching a reference context (strict type filtering). */
export function definitionsForReference(
idx: ModIndex,
ctx: ReferenceContext,
): AssetDef[] {
const defs = mergeLocalAndGlobalDefs(
idx.local?.assetsById.get(ctx.id.toLowerCase()),
idx.assetsById.get(ctx.id.toLowerCase()),
);
return filterAndScoreDefs(defs, ctx.refType, ctx.selfType).map((t) => t.def);
}
/** Union of reference sites for a set of definitions, de-duplicated. */
export function collectReferenceSites(
idx: ModIndex,
defs: readonly AssetDef[],
): ReferenceSite[] {
const sites: ReferenceSite[] = [];
const seen = new Set<string>();
for (const def of defs) {
for (const site of referenceSitesForDef(idx, def)) {
const key = `${site.file}\u0000${site.start}\u0000${site.end}\u0000${site.kind}`;
if (seen.has(key)) continue;
seen.add(key);
sites.push(site);
}
}
return sites;
}
/** Converts stored offsets to precise editor locations (fallback: line). */
export async function sitesToLocations(
ws: ModWorkspace,
sites: readonly ReferenceSite[],
): Promise<vscode.Location[]> {
const byFile = new Map<string, ReferenceSite[]>();
for (const site of sites) {
let list = byFile.get(site.file);
if (!list) {
list = [];
byFile.set(site.file, list);
}
list.push(site);
}
const locations: vscode.Location[] = [];
for (const [file, fileSites] of byFile) {
const parsed = await ws.indexer?.readDom(file);
const lineMap = parsed?.lineMap ?? null;
for (const site of fileSites) {
if (lineMap) {
const start = lineMap.positionAt(site.start);
const end = lineMap.positionAt(site.end);
locations.push(
new vscode.Location(
vscode.Uri.file(file),
new vscode.Range(
new vscode.Position(start.line, start.character),
new vscode.Position(end.line, end.character),
),
),
);
} else {
const line = Math.max(0, site.line - 1);
locations.push(
new vscode.Location(
vscode.Uri.file(file),
new vscode.Range(
new vscode.Position(line, 0),
new vscode.Position(line, 1),
),
),
);
}
}
}
return locations;
}
/**
* Semantic Find All References (used by the reference provider).
*
* Only real reference sites are returned — the asset's own `id` definition is
* never included, regardless of VS Code's `includeDeclaration` flag, so the
* result set matches the CodeLens reference count exactly.
*/
export async function findReferenceLocations(
ws: ModWorkspace,
document: vscode.TextDocument,
position: vscode.Position,
): Promise<vscode.Location[] | null> {
if (!ws.isRa3Workspace()) return null;
scheduleRebuildIfRecordsDesync(ws, document);
const scope = await ws.getScope(document);
const idx = scope.merged;
if (!idx) return null;
const ctx = referenceContextAt(document, document.offsetAt(position));
if (!ctx) return null;
const defs = definitionsForReference(idx, ctx);
if (!defs.length) return null;
const locations = await sitesToLocations(ws, collectReferenceSites(idx, defs));
return locations.length ? locations : null;
}
/** Arguments passed from the CodeLens to the showReferences command. */
export interface ShowReferencesArgs {
uri: vscode.Uri;
position: vscode.Position;
id: string;
type: string;
file: string;
line: number;
}
/** Opens the references peek for one specific asset definition. */
export async function showReferencesForDef(
ws: ModWorkspace,
args: ShowReferencesArgs,
): Promise<void> {
const idx = ws.index;
if (!idx) return;
const def: AssetDef = {
type: args.type,
id: args.id,
file: args.file,
line: args.line,
origin: "project",
};
const sites = referenceSitesForDef(idx, def);
const locations = await sitesToLocations(ws, sites);
await vscode.commands.executeCommand(
"editor.action.showReferences",
args.uri,
args.position,
locations,
);
}
+6
View File
@@ -1,6 +1,7 @@
import * as vscode from "vscode"; import * as vscode from "vscode";
import { parseXml } from "../language/xmlParser"; import { parseXml } from "../language/xmlParser";
import { buildSemanticTokenRanges } from "../language/semanticTokens"; import { buildSemanticTokenRanges } from "../language/semanticTokens";
import type { ModWorkspace } from "../workspace";
const TOKEN_TYPES = ["type", "property", "string"] as const; const TOKEN_TYPES = ["type", "property", "string"] as const;
@@ -20,10 +21,15 @@ export const RA3_SEMANTIC_TOKENS_LEGEND = new vscode.SemanticTokensLegend([
export class Ra3SemanticTokensProvider export class Ra3SemanticTokensProvider
implements vscode.DocumentSemanticTokensProvider implements vscode.DocumentSemanticTokensProvider
{ {
constructor(private ws: ModWorkspace) {}
async provideDocumentSemanticTokens( async provideDocumentSemanticTokens(
document: vscode.TextDocument, document: vscode.TextDocument,
_token: vscode.CancellationToken, _token: vscode.CancellationToken,
): Promise<vscode.SemanticTokens> { ): Promise<vscode.SemanticTokens> {
if (!this.ws.isRa3Workspace()) {
return new vscode.SemanticTokens(new Uint32Array(0));
}
const text = document.getText(); const text = document.getText();
const doc = parseXml(text); const doc = parseXml(text);
if (doc.errors.length === 0) { if (doc.errors.length === 0) {
+124
View File
@@ -0,0 +1,124 @@
import * as vscode from "vscode";
import { relative } from "node:path";
import { findElementAt, parseXml } from "../language/xmlParser";
import { unreferencedByType } from "../indexer/referenceIndex";
import type { ModWorkspace } from "../workspace";
interface TypePickItem extends vscode.QuickPickItem {
type: string;
}
interface AssetPickItem extends vscode.QuickPickItem {
file: string;
line: number;
}
/**
* Palette command: pick an asset type, then jump to any project asset of
* that type that has zero incoming references. Only types that are reference
* targets by design are offered, so auto-registered data (settings, map
* metadata, w3x sub-assets...) is not reported as "unused".
*/
export async function findUnreferencedAssets(
ws: ModWorkspace,
args?: { type?: string },
): Promise<void> {
if (!ws.isRa3Workspace() || !ws.index) {
void vscode.window.showInformationMessage(
"RA3 Mod XML: no index available yet.",
);
return;
}
const idx = ws.index;
const byType = unreferencedByType(idx);
let type = args?.type;
if (!type) {
if (!byType.size) {
void vscode.window.showInformationMessage(
"RA3 Mod XML: no unreferenced assets found.",
);
return;
}
const pickedType = await vscode.window.showQuickPick<TypePickItem>(
[...byType.entries()].map(([t, defs]) => ({
label: t,
description: `${defs.length} unreferenced`,
type: t,
})),
{
placeHolder: "Select an asset type",
matchOnDescription: true,
},
);
if (!pickedType) return;
type = pickedType.type;
}
const defs = byType.get(type) ?? [];
if (!defs.length) {
void vscode.window.showInformationMessage(
`RA3 Mod XML: no unreferenced ${type} assets found.`,
);
return;
}
const pickedAsset = await vscode.window.showQuickPick<AssetPickItem>(
defs.map((d) => ({
label: d.id,
description: `${displayPath(idx.projectDir, d.file)}:${d.line}`,
file: d.file,
line: d.line,
})),
{
placeHolder: `${type}: ${defs.length} unreferenced`,
matchOnDescription: true,
},
);
if (!pickedAsset) return;
const uri = vscode.Uri.file(pickedAsset.file);
const document = await vscode.workspace.openTextDocument(uri);
const line = Math.max(0, pickedAsset.line - 1);
await vscode.window.showTextDocument(document, {
selection: new vscode.Range(
new vscode.Position(line, 0),
new vscode.Position(line, 1),
),
preview: true,
});
}
/**
* Editor context-menu entry: pre-selects the asset type under the cursor.
* Falls back to the type picker when the cursor is not on a top-level asset.
*/
export async function findUnreferencedAssetsOfType(
ws: ModWorkspace,
): Promise<void> {
const editor = vscode.window.activeTextEditor;
if (editor && ws.isRa3Workspace()) {
const document = editor.document;
const offset = document.offsetAt(editor.selection.active);
const doc = parseXml(document.getText());
const el = findElementAt(doc, offset);
if (el && el.parent === doc.root) {
const local = localName(el.name);
const isStructural = local === "Tags" || local === "Includes" || local === "Defines";
const hasId = el.attrs.some((a) => a.name === "id" && a.hasValue);
if (!isStructural && hasId) {
return findUnreferencedAssets(ws, { type: local });
}
}
}
return findUnreferencedAssets(ws);
}
function localName(tag: string): string {
const idx = tag.lastIndexOf(":");
return idx >= 0 ? tag.slice(idx + 1) : tag;
}
function displayPath(projectDir: string, file: string): string {
const rel = relative(projectDir, file);
return rel && !rel.startsWith("..") ? rel : file;
}
+292
View File
@@ -0,0 +1,292 @@
/**
* Caches shared by the indexer.
*
* The workspace owns one instance of each cache and passes them into every
* ModIndexer, so a rebuild (which creates a fresh indexer) does not re-read
* files whose stat (mtime/size) is unchanged. This is what makes indexing
* large art-asset corpora (Corona: ~3800 .w3x files, 2.6 GB) practical:
* after the first build, save-triggered rebuilds only re-scan files that
* actually changed.
*/
import { resolve } from "node:path";
import { createHash } from "node:crypto";
import type { IndexedFile, ParsedFile } from "./types";
import type { IndexRecords } from "./records";
import type { ResolveResult } from "./includeResolver";
/** Case-insensitive absolute path key. */
export function normKey(path: string): string {
return resolve(path).toLowerCase();
}
/**
* SHA-1 of the BOM-stripped text a file's records were extracted from.
* Lets force rebuilds verify that a stat-matching cache entry is not stale
* (FAT32/exFAT timestamps can match after a rewrite), and lets features
* detect "open document != indexed snapshot" without reading the whole index.
*/
export function contentHash(text: string): string {
return createHash("sha1").update(text, "utf8").digest("hex");
}
/**
* Hash of a file's compact index records. Semantic records (assets, defines,
* includes, references) are what the index actually consumes, so comparing
* records hashes ignores cosmetic text changes (line endings, whitespace)
* and only fires the self-heal when the index would really be out of date.
*/
export function recordsHash(records: IndexRecords): string {
return createHash("sha1")
.update(JSON.stringify(records), "utf8")
.digest("hex");
}
/**
* LRU cache for fully parsed XML documents.
*
* Parse trees of huge mods can be memory-heavy (~17x the source text), so
* retention is bounded twice: by entry count (least-recently-used eviction)
* and by a total element budget (largest trees are evicted first). Evicted
* entries are re-read from disk on demand.
*/
export class DocumentCache {
private map = new Map<string, ParsedFile>();
private totalElements = 0;
constructor(
private capacity = 64,
private elementBudget = 2_000_000,
) {}
get(path: string): ParsedFile | undefined {
const key = normKey(path);
const hit = this.map.get(key);
if (!hit) return undefined;
this.map.delete(key);
this.map.set(key, hit);
return hit;
}
/** Number of cached documents (for diagnostics). */
get size(): number {
return this.map.size;
}
/** Total elements held by cached parse trees (for diagnostics). */
get elements(): number {
return this.totalElements;
}
set(parsed: ParsedFile): void {
const key = normKey(parsed.file.path);
const prev = this.map.get(key);
if (prev) this.totalElements -= elementCount(prev);
this.map.delete(key);
this.map.set(key, parsed);
this.totalElements += elementCount(parsed);
this.evict();
}
invalidate(path: string): void {
const key = normKey(path);
const prev = this.map.get(key);
if (prev) this.totalElements -= elementCount(prev);
this.map.delete(key);
}
clear(): void {
this.map.clear();
this.totalElements = 0;
}
private evict(): void {
while (this.map.size > this.capacity || this.totalElements > this.elementBudget) {
if (this.map.size === 0) break;
if (this.map.size > this.capacity) {
// Over capacity: drop the least recently used entry.
const oldest = this.map.keys().next().value;
if (oldest === undefined) break;
this.remove(oldest);
} else {
// Over the element budget: drop the largest tree, which frees the
// most memory per eviction.
let largestKey: string | undefined;
let largest = -1;
for (const [key, value] of this.map) {
const n = elementCount(value);
if (n > largest) {
largest = n;
largestKey = key;
}
}
if (largestKey === undefined || largest <= 0) break;
this.remove(largestKey);
}
}
}
private remove(key: string): void {
const prev = this.map.get(key);
if (prev) this.totalElements -= elementCount(prev);
this.map.delete(key);
}
}
function elementCount(parsed: ParsedFile): number {
return parsed.parse?.elements.length ?? 0;
}
export interface IndexRecordsCacheEntry {
stat: IndexedFile["stat"];
records: IndexRecords;
/** "shallow" for art-asset scans (.w3x), "full" for parsed XML. */
kind: "shallow" | "full";
/**
* Hash of the BOM-stripped file text (full parses only). Absent for
* shallow scans (avoid hashing multi-MB model files) and for cache entries
* produced before this field existed.
*/
contentHash?: string;
}
/**
* Cache for per-file index records (top-level assets, defines, includes,
* xi:include targets). Records are tiny compared to DOM trees or line maps of
* multi-megabyte model files, so the capacity comfortably covers a whole mod
* (Corona: ~9k files) and rebuilds never re-read unchanged files.
*/
export class IndexRecordsCache {
private map = new Map<string, IndexRecordsCacheEntry>();
constructor(private capacity = 16384) {}
get(path: string): IndexRecordsCacheEntry | undefined {
const key = normKey(path);
const hit = this.map.get(key);
if (!hit) return undefined;
this.map.delete(key);
this.map.set(key, hit);
return hit;
}
/** Number of cached record sets (for diagnostics). */
get size(): number {
return this.map.size;
}
/** Iterates [normalized key, entry] pairs (used by disk persistence). */
entries(): IterableIterator<[string, IndexRecordsCacheEntry]> {
return this.map.entries();
}
set(path: string, entry: IndexRecordsCacheEntry): void {
const key = normKey(path);
this.map.delete(key);
this.map.set(key, entry);
if (this.map.size > this.capacity) {
const oldest = this.map.keys().next().value;
if (oldest !== undefined) this.map.delete(oldest);
}
}
invalidate(path: string): void {
this.map.delete(normKey(path));
}
clear(): void {
this.map.clear();
}
}
/**
* Cache for Include/@source (and xi:include/@href) resolution results.
*
* Resolving a source performs synchronous statSync existence checks against
* every search base; a Corona build issues ~110k of them (tens of seconds on
* a mechanical drive). Content edits never change *existence*, so this cache
* survives content rebuilds and is only cleared when files are created or
* deleted (or on a forced reindex).
*/
export class IncludeResolveCache {
private map = new Map<string, ResolveResult>();
private manifestMap = new Map<string, string | null>();
constructor(private capacity = 262144) {}
get(key: string): ResolveResult | undefined {
const hit = this.map.get(key);
if (!hit) return undefined;
this.map.delete(key);
this.map.set(key, hit);
return hit;
}
set(key: string, result: ResolveResult): void {
this.map.delete(key);
this.map.set(key, result);
if (this.map.size > this.capacity) {
const oldest = this.map.keys().next().value;
if (oldest !== undefined) this.map.delete(oldest);
}
}
getManifest(key: string): string | null | undefined {
const hit = this.manifestMap.get(key);
if (hit === undefined) return undefined;
this.manifestMap.delete(key);
this.manifestMap.set(key, hit);
return hit;
}
setManifest(key: string, path: string | null): void {
this.manifestMap.delete(key);
this.manifestMap.set(key, path);
if (this.manifestMap.size > this.capacity) {
const oldest = this.manifestMap.keys().next().value;
if (oldest !== undefined) this.manifestMap.delete(oldest);
}
}
clear(): void {
this.map.clear();
this.manifestMap.clear();
}
/** Number of cached resolutions (for diagnostics). */
get size(): number {
return this.map.size;
}
}
/**
* Monotonic counter for workspace-level invalidations.
*
* A build captures `snapshot()` when it starts; if `changedSince()` is true
* when a phase snapshot is about to be published, files may have changed
* mid-build, so the published index is marked stale (and the workspace's
* dirty/rebuild mechanism converges to fresh data shortly after).
*/
export class InvalidationsEpoch {
private value = 0;
/** Records a new invalidation (content, creation or deletion). */
mark(): void {
this.value++;
}
/** Returns the current epoch value. */
snapshot(): number {
return this.value;
}
/** True when at least one invalidation happened since `epoch`. */
changedSince(epoch: number): boolean {
return this.value !== epoch;
}
/** Current epoch value (read-only accessor). */
get current(): number {
return this.value;
}
}
+218
View File
@@ -0,0 +1,218 @@
/**
* On-disk persistence for per-file index records.
*
* The records cache (top-level assets / defines / includes / xi:include with
* line numbers) is tiny compared to the source corpus (Corona: ~9k files,
* ~10 MB in memory), but rebuilding it from scratch means reading ~2.6 GB of
* art assets again. Persisting it makes a cold start cost a stat validation
* pass (~seconds on SSD, a few to tens of seconds on a mechanical drive)
* instead of a full rebuild.
*
* Correctness model (layered):
* - every cached record stores a multi-signal stamp
* `{ size, mtimeMs, birthtimeMs, ctimeMs }`;
* - on load, each file is stat-validated (no content reads); mismatches and
* missing files are dropped and re-read during the build;
* - during a session the file watcher invalidates entries precisely;
* - `ra3modxml.reindex` / `ra3modxml.clearCache` remain the final authority.
*
* The file is gzip-compressed JSON written atomically (temp + rename), keyed
* by project identity + settings so stale caches are ignored automatically.
*
* Pure TypeScript: no vscode dependency.
*/
import { createHash } from "node:crypto";
import { mkdir, readFile, rename, rm, stat, writeFile } from "node:fs/promises";
import { dirname } from "node:path";
import { gzip, gunzip } from "node:zlib";
import { promisify } from "node:util";
import type { IndexRecordsCacheEntry } from "./caches";
import type { IndexRecords } from "./records";
import type { IndexedFile } from "./types";
const gzipAsync = promisify(gzip);
const gunzipAsync = promisify(gunzip);
/**
* v2: per-file records now carry typed reference records (`references`),
* so caches produced by v1 (assets/defines/includes only) are stale.
* v3: full XML records carry `contentHash`, and snapshots publish per-file
* `recordsHashes` for the desync self-heal; caches without hashes cannot be
* verified, so v2 files are regenerated once.
*/
export const DISK_CACHE_VERSION = 3;
/** How many stat validations run concurrently on load. */
const VALIDATE_CONCURRENCY = 32;
/** Settings that change what the index contains; a mismatch ignores the cache. */
export interface DiskCacheIdentity {
projectDir: string;
sdkDir: string;
indexSageXml: boolean;
additionalDataSearchPaths: string[];
builtmodsDirs: string[];
}
export interface DiskCacheRecord {
/** Normalized cache key (see `normKey`). */
key: string;
stat: NonNullable<IndexedFile["stat"]>;
records: IndexRecords;
kind: "full" | "shallow";
/** Content hash for full XML parses (see `IndexRecordsCacheEntry`). */
contentHash?: string;
}
interface DiskCacheFile {
version: number;
key: string;
savedAt: string;
records: DiskCacheRecord[];
}
export interface DiskCacheLoadStats {
fileExists: boolean;
keyMatched: boolean;
/** Records stored in the file. */
loaded: number;
/** Records whose stat still matches (kept). */
validated: number;
/** Records dropped because the file changed, moved or was deleted. */
dropped: number;
}
export function diskCacheKey(identity: DiskCacheIdentity): string {
return createHash("sha256")
.update(JSON.stringify(identity))
.digest("hex")
.slice(0, 16);
}
export class DiskRecordsCache {
constructor(
private readonly filePath: string,
private readonly identity: DiskCacheIdentity,
) {}
get path(): string {
return this.filePath;
}
/**
* Loads and stat-validates the cache. Returns the kept records plus load
* statistics; missing/corrupt/key-mismatched caches yield an empty result
* instead of an error.
*/
async loadValidated(): Promise<{
records: DiskCacheRecord[];
stats: DiskCacheLoadStats;
}> {
const stats: DiskCacheLoadStats = {
fileExists: false,
keyMatched: false,
loaded: 0,
validated: 0,
dropped: 0,
};
let raw: DiskCacheFile | null = null;
try {
const buf = await readFile(this.filePath);
stats.fileExists = true;
const text = (await gunzipAsync(buf)).toString("utf8");
const parsed = JSON.parse(text);
if (
parsed &&
parsed.version === DISK_CACHE_VERSION &&
parsed.key === diskCacheKey(this.identity) &&
Array.isArray(parsed.records)
) {
raw = parsed as DiskCacheFile;
}
} catch {
// Missing or corrupt cache: fall through with an empty result.
}
if (!raw) return { records: [], stats };
stats.keyMatched = true;
stats.loaded = raw.records.length;
const kept: DiskCacheRecord[] = [];
for (let i = 0; i < raw.records.length; i += VALIDATE_CONCURRENCY) {
const chunk = raw.records.slice(i, i + VALIDATE_CONCURRENCY);
const results = await Promise.all(
chunk.map(async (rec): Promise<DiskCacheRecord | null> => {
try {
const s = await stat(rec.key);
if (
s.isFile() &&
s.size === rec.stat.size &&
s.mtimeMs === rec.stat.mtimeMs &&
s.birthtimeMs === rec.stat.birthtimeMs &&
s.ctimeMs === rec.stat.ctimeMs
) {
return rec;
}
} catch {
// File missing or inaccessible.
}
return null;
}),
);
for (const r of results) {
if (r) {
kept.push(r);
stats.validated++;
} else {
stats.dropped++;
}
}
}
return { records: kept, stats };
}
/** Writes the current records cache atomically (temp file + rename). */
async save(
entries: Iterable<[string, IndexRecordsCacheEntry]>,
): Promise<void> {
const records: DiskCacheRecord[] = [];
for (const [key, entry] of entries) {
if (!entry.stat) continue;
records.push({
key,
stat: entry.stat,
records: entry.records,
kind: entry.kind,
contentHash: entry.contentHash,
});
}
const payload: DiskCacheFile = {
version: DISK_CACHE_VERSION,
key: diskCacheKey(this.identity),
savedAt: new Date().toISOString(),
records,
};
const buf = await gzipAsync(Buffer.from(JSON.stringify(payload), "utf8"));
await mkdir(dirname(this.filePath), { recursive: true });
const tmp = `${this.filePath}.tmp`;
await writeFile(tmp, buf);
await rename(tmp, this.filePath);
}
/** Deletes the cache file (used by the clear-cache command). */
async clear(): Promise<void> {
try {
await rm(this.filePath, { force: true });
} catch {
// Best effort.
}
}
async status(): Promise<{ exists: boolean; sizeBytes: number }> {
try {
const s = await stat(this.filePath);
return { exists: s.isFile(), sizeBytes: s.size };
} catch {
return { exists: false, sizeBytes: 0 };
}
}
}
+117
View File
@@ -0,0 +1,117 @@
/**
* Lazy file-existence snapshot for include resolution.
*
* `resolveSource` performs synchronous `statSync` existence checks against
* every search base; a cold Corona build does ~110k of them (tens of seconds
* on a mechanical drive). Instead, existence is answered by reading the
* candidate's **parent directory** once (`readdir`, no per-file stat) and
* caching the entry set for the rest of the build. Only directories that are
* actually queried are ever listed, so a cold build pays a handful of
* readdir calls instead of an upfront recursive enumeration of every search
* root (which measurably slowed the XML phase).
*
* Correctness: the workspace clears `IncludeResolveCache` on file
* create/delete; each rebuild creates a fresh snapshot, and the debounced
* rebuild triggered by the watcher converges if anything changed mid-build.
*
* Pure TypeScript: no vscode dependency.
*/
import { existsSync, readdirSync } from "node:fs";
import { basename, dirname, parse, resolve, sep } from "node:path";
import { normKey } from "./caches";
import type { SearchPaths } from "./includeResolver";
/**
* Answers file-existence questions from lazily read directory listings.
* `has()` is authoritative for paths inside the roots and returns null for
* paths the snapshot does not cover (the caller falls back to `statSync`).
*/
export class ExistenceSnapshot {
private roots: string[] = [];
/** parent dir (normalized) -> lowercased file names, or null (no dir). */
private dirCache = new Map<string, Set<string> | null>();
/** Existence answers served from cached directory listings. */
hits = 0;
/** Paths outside the snapshot that required a statSync fallback. */
fallbacks = 0;
constructor(roots: string[]) {
// Roots and lookups must use the same normalization (case-insensitive on
// Windows), otherwise `startsWith` misses due to case differences.
this.roots = roots.map((r) => {
const n = normKey(r);
return n.endsWith(sep) ? n : n + sep;
});
}
/** true/false when covered by the snapshot, null when not covered. */
has(absPath: string): boolean | null {
const parentKey = normKey(dirname(absPath));
if (!this.isCovered(parentKey)) {
this.fallbacks++;
return null;
}
let entries = this.dirCache.get(parentKey);
if (entries === undefined) {
entries = listDirEntries(parentKey);
this.dirCache.set(parentKey, entries);
}
this.hits++;
return entries ? entries.has(basename(absPath).toLowerCase()) : false;
}
private isCovered(dirKey: string): boolean {
const key = dirKey.endsWith(sep) ? dirKey : dirKey + sep;
for (const root of this.roots) {
if (key.startsWith(root)) return true;
}
return false;
}
}
function listDirEntries(dir: string): Set<string> | null {
try {
const out = new Set<string>();
for (const entry of readdirSync(dir, { withFileTypes: true })) {
if (entry.isFile()) out.add(entry.name.toLowerCase());
}
return out;
} catch {
return null;
}
}
/**
* True when `dir` is a filesystem root (e.g. "C:\" or "/"). Such roots are
* never treated as search bases (they may contain the whole disk).
*/
export function isDriveRoot(dir: string): boolean {
const resolved = resolve(dir);
return parse(resolved).root === resolved;
}
/**
* Builds the snapshot root list from the search bases: drive roots and
* missing directories are skipped, and a root covered by a broader root is
* dropped (e.g. `sdkDir` covers `sdkDir/SageXml`). No directory is listed
* here; listings happen lazily per queried parent directory.
*/
export function buildExistenceSnapshot(searchPaths: SearchPaths): ExistenceSnapshot {
const candidates = [
...searchPaths.DATA,
...searchPaths.ART,
...searchPaths.AUDIO,
].map((r) => resolve(r));
candidates.sort((a, b) => a.length - b.length);
const roots: string[] = [];
for (const root of candidates) {
if (isDriveRoot(root)) continue;
if (!existsSync(root)) continue;
const normalized = normKey(root);
if (roots.some((r) => normalized.startsWith(r + sep))) continue;
roots.push(normalized);
}
return new ExistenceSnapshot(roots);
}
+59 -13
View File
@@ -1,7 +1,18 @@
import { readdir, stat } from "node:fs/promises"; import { readdir, stat } from "node:fs/promises";
import { join, relative, resolve } from "node:path"; import { extname, join, relative, resolve } from "node:path";
import type { FileWalker, SourceCandidate } from "./types"; import type { FileWalker, SourceCandidate } from "./types";
/**
* Extensions offered as Include/@source candidates in DATA directories:
* regular XML plus art-asset XML (.w3x) that mods include via W3X.xml hubs.
* ART/AUDIO directories already list every file.
*/
const DATA_CANDIDATE_EXTENSIONS = new Set([".xml", ".w3x"]);
function isDataCandidate(path: string): boolean {
return DATA_CANDIDATE_EXTENSIONS.has(extname(path).toLowerCase());
}
/** /**
* Recursive file list walker with a simple directory-mtime cache. Used to * Recursive file list walker with a simple directory-mtime cache. Used to
* enumerate candidate files for Include/@source completion. Directory scans * enumerate candidate files for Include/@source completion. Directory scans
@@ -50,6 +61,38 @@ export class CachedDirectoryWalker implements FileWalker {
} }
} }
/**
* True for paths whose changes can never affect the index: `.git` internals
* touched by background fetch/maintenance, and transient temp/backup files
* created by editors or other extensions (`UnitCrate.xml.git`, `*.tmp`,
* `*.lock`, `file~`, `.#file`, ...). Such events are ignored by the file
* watcher instead of triggering rebuilds.
*/
export function isWatcherNoisePath(fsPath: string): boolean {
const segments = fsPath.split(/[\\/]/);
if (segments.some((seg) => seg.toLowerCase() === ".git")) return true;
const base = segments[segments.length - 1] ?? "";
const lower = base.toLowerCase();
const noiseSuffixes = [".git", ".tmp", ".lock", "~", ".swp", ".bak", ".orig"];
if (noiseSuffixes.some((suffix) => lower.endsWith(suffix))) return true;
return lower.startsWith(".#") || lower.startsWith(".~");
}
/**
* True for files whose *content* participates in the index: XML documents
* and art-asset XML (.w3x). Reasonable text formats in RA3 mods are `.xml`,
* `.w3x` and `.lua`; lua is not indexed yet, and compiled manifests are
* binary `*.manifest` (there is no `.manifestxml` source format). Files
* already in the index with other extensions (e.g. sniffed XML) are handled
* separately via `ModIndexer.isIndexedFile`. Content changes to binary art
* (e.g. textures, `.w3d`) cannot change index records, so they do not need
* to trigger a rebuild.
*/
export function isContentRelevantPath(fsPath: string): boolean {
const ext = extname(fsPath).toLowerCase();
return ext === ".xml" || ext === ".w3x";
}
/** /**
* Builds the candidate list for Include/@source completion from a set of * Builds the candidate list for Include/@source completion from a set of
* search directories. For DATA directories only *.xml files are listed; for * search directories. For DATA directories only *.xml files are listed; for
@@ -76,29 +119,32 @@ export async function collectSourceCandidates(
out.push({ source, path: resolve(path), prefix, baseDir: resolve(baseDir) }); out.push({ source, path: resolve(path), prefix, baseDir: resolve(baseDir) });
}; };
for (const dir of dataDirs) { // List directories in parallel; the walker caches each directory by its
const files = await walker.listFiles(dir); // mtime, so rebuilds after the first are still cheap.
const [dataLists, artLists, audioLists] = await Promise.all([
Promise.all(dataDirs.map(async (dir) => ({ dir, files: await walker.listFiles(dir) }))),
Promise.all(artDirs.map(async (dir) => ({ dir, files: await walker.listFiles(dir) }))),
Promise.all(audioDirs.map(async (dir) => ({ dir, files: await walker.listFiles(dir) }))),
]);
for (const { dir, files } of dataLists) {
for (const f of files) { for (const f of files) {
if (!f.toLowerCase().endsWith(".xml")) continue; if (!isDataCandidate(f)) continue;
add(f, dir, "DATA"); add(f, dir, "DATA");
} }
if (samePath(dir, projectDataDir)) { if (samePath(dir, projectDataDir)) {
// Also offer project-relative paths (what Mod.xml itself uses). // Also offer project-relative paths (what Mod.xml itself uses).
for (const f of files) { for (const f of files) {
if (f.toLowerCase().endsWith(".xml")) add(f, projectDataDir, null); if (isDataCandidate(f)) add(f, projectDataDir, null);
} }
} }
} }
for (const dir of artDirs) { for (const { dir, files } of artLists) {
for (const f of await walker.listFiles(dir)) { for (const f of files) add(f, dir, "ART");
add(f, dir, "ART");
}
} }
for (const dir of audioDirs) { for (const { dir, files } of audioLists) {
for (const f of await walker.listFiles(dir)) { for (const f of files) add(f, dir, "AUDIO");
add(f, dir, "AUDIO");
}
} }
return out; return out;
+25 -7
View File
@@ -8,6 +8,7 @@
import { join, resolve, normalize, isAbsolute } from "node:path"; import { join, resolve, normalize, isAbsolute } from "node:path";
import { statSync } from "node:fs"; import { statSync } from "node:fs";
import type { ExistenceSnapshot } from "./existence";
export type IncludeKind = "all" | "instance" | "reference"; export type IncludeKind = "all" | "instance" | "reference";
export type SourcePrefix = "DATA" | "ART" | "AUDIO" | null; export type SourcePrefix = "DATA" | "ART" | "AUDIO" | null;
@@ -99,41 +100,58 @@ export function resolveSource(
source: string, source: string,
currentDir: string | null, currentDir: string | null,
searchPaths: SearchPaths, searchPaths: SearchPaths,
existence?: ExistenceSnapshot,
): ResolveResult { ): ResolveResult {
const raw = source.trim().replace(/\\/g, "/"); const raw = source.trim().replace(/\\/g, "/");
const { prefix, rest } = splitPrefix(raw); const { prefix, rest } = splitPrefix(raw);
if (prefix) { if (prefix) {
const bases = searchPaths[prefix] ?? []; const bases = searchPaths[prefix] ?? [];
const direct = findInBases(rest, bases); const direct = findInBases(rest, bases, existence);
if (direct) return { path: direct, prefix, raw }; if (direct) return { path: direct, prefix, raw };
if (prefix === "ART" && !rest.includes("/")) { if (prefix === "ART" && !rest.includes("/")) {
const two = rest.slice(0, 2).toLowerCase(); const two = rest.slice(0, 2).toLowerCase();
const prefixed = findInBases(`${two}/${rest}`, bases); const prefixed = findInBases(`${two}/${rest}`, bases, existence);
if (prefixed) return { path: prefixed, prefix, raw }; if (prefixed) return { path: prefixed, prefix, raw };
} }
return { path: null, prefix, raw }; return { path: null, prefix, raw };
} }
if (currentDir && isAbsolute(rest)) { if (currentDir && isAbsolute(rest)) {
return { path: fileExists(rest) ? rest : null, prefix: null, raw }; return {
path: fileExists(rest, existence) ? rest : null,
prefix: null,
raw,
};
} }
if (currentDir) { if (currentDir) {
const candidate = resolve(currentDir, rest); const candidate = resolve(currentDir, rest);
return { path: fileExists(candidate) ? candidate : null, prefix: null, raw }; return {
path: fileExists(candidate, existence) ? candidate : null,
prefix: null,
raw,
};
} }
return { path: null, prefix: null, raw }; return { path: null, prefix: null, raw };
} }
function findInBases(relPath: string, bases: string[]): string | null { function findInBases(
relPath: string,
bases: string[],
existence?: ExistenceSnapshot,
): string | null {
for (const base of bases) { for (const base of bases) {
const candidate = normalize(resolve(base, relPath)); const candidate = normalize(resolve(base, relPath));
if (fileExists(candidate)) return candidate; if (fileExists(candidate, existence)) return candidate;
} }
return null; return null;
} }
function fileExists(path: string): boolean { function fileExists(path: string, existence?: ExistenceSnapshot): boolean {
if (existence) {
const known = existence.has(path);
if (known !== null) return known;
}
try { try {
return statSync(path).isFile(); return statSync(path).isFile();
} catch { } catch {
+652 -220
View File
File diff suppressed because it is too large Load Diff
+269
View File
@@ -0,0 +1,269 @@
import { dirname, resolve } from "node:path";
import { LineMap, parseXml, type XmlDocument } from "../language/xmlParser";
import { extractIndexRecords } from "./records";
import { resolveSource, type SearchPaths } from "./includeResolver";
import { expandDocument, type LogicalDocument } from "./logicalTree";
import type {
AssetDef,
DefineDef,
LocalOverlay,
ModIndex,
ParsedFile,
} from "./types";
export interface LocalScopeContext {
projectDir: string;
sdkDir: string;
searchPaths: SearchPaths;
/** Reads a file's compact index records (full parse or shallow scan). */
readRecords(path: string): Promise<ParsedFile | null>;
/** Reads a file and guarantees a DOM parse tree. */
readDom(path: string): Promise<ParsedFile | null>;
}
/**
* Everything the features need for the currently open document: the original
* parse, the expanded logical tree, per-source line maps, the local overlay
* and the overlay-aware index for lookups.
*/
export interface DocumentScope {
uri: string;
version: number;
parse: XmlDocument;
lineMap: LineMap;
expanded: LogicalDocument;
/** scopePathKey(file) -> line map (current file + expanded include targets). */
lineMaps: Map<string, LineMap>;
/** Local assets/defines from the document itself and its include chain. */
overlay: LocalOverlay;
/** Global index with `local` attached (or a minimal standalone index). */
merged: ModIndex | null;
}
/** Normalized key used for source-file identity / line-map lookup. */
export function scopePathKey(path: string): string {
return path.replace(/\\/g, "/").toLowerCase();
}
/**
* Builds the document scope for the current (possibly unsaved) text:
* - a local overlay of assets/defines reachable from this file;
* - a logical tree with supported xi:include targets spliced in place.
*/
export async function buildDocumentScope(
uri: string,
text: string,
version: number,
ctx: LocalScopeContext,
): Promise<DocumentScope> {
const lineMap = new LineMap(text);
const parse = parseXml(text);
const builder = new OverlayBuilder(ctx);
await builder.addEntry(uri, parse, lineMap, text);
const expanded = await expandDocument(uri, parse, {
resolve: (source, currentDir) =>
resolveSource(source, currentDir, ctx.searchPaths).path,
readDom: async (path) => {
const parsed = await ctx.readDom(path);
return parsed?.parse && parsed.lineMap
? { parse: parsed.parse, lineMap: parsed.lineMap }
: null;
},
});
const lineMaps = new Map<string, LineMap>();
lineMaps.set(scopePathKey(uri), lineMap);
for (const [path, lm] of builder.lineMaps) {
if (!lineMaps.has(path)) lineMaps.set(path, lm);
}
return {
uri,
version,
parse,
lineMap,
expanded,
lineMaps,
overlay: builder.overlay,
merged: null,
};
}
/**
* Attaches a document-local overlay to a global index without copying the
* global maps. When there is no global index yet, returns a minimal standalone
* index so the local chain alone can serve completions / references.
*/
export function withLocalOverlay(
global: ModIndex | null,
overlay: LocalOverlay,
projectDir: string,
sdkDir: string,
): ModIndex {
if (!global) {
return {
projectDir,
sdkDir,
complete: false,
phase: "xml",
assets: new Map(),
assetsById: new Map(),
defines: new Map(),
files: new Map(),
streams: [],
manifests: new Map(),
sourceCandidates: [],
diagnostics: [],
references: new Map(),
recordsHashes: new Map(),
stats: {
projectDir,
sdkDir,
phase: "xml",
complete: false,
indexedFiles: 0,
parsedFiles: 0,
shallowScannedFiles: 0,
deferredArtFiles: 0,
shallowCacheHits: 0,
recordsCacheHits: 0,
resolveCacheHits: 0,
resolveCalls: 0,
snapshotHits: 0,
snapshotFallbacks: 0,
candidatesMs: 0,
walkMs: 0,
artScanMs: 0,
assetCount: 0,
referenceCount: 0,
defineCount: 0,
manifestFiles: 0,
manifestAssetCount: 0,
streams: 0,
sourceCandidates: 0,
elapsedMs: 0,
},
local: overlay,
};
}
return { ...global, local: overlay };
}
const MAX_LOCAL_DEPTH = 64;
class OverlayBuilder {
readonly overlay: LocalOverlay = {
assets: new Map(),
assetsById: new Map(),
defines: new Map(),
};
readonly lineMaps = new Map<string, LineMap>();
private visited = new Set<string>();
constructor(private ctx: LocalScopeContext) {}
async addEntry(
path: string,
parse: XmlDocument,
lineMap: LineMap,
text: string,
): Promise<void> {
this.lineMaps.set(scopePathKey(path), lineMap);
await this.addParsed({
file: { path: resolve(path), stat: null },
parse,
records: extractIndexRecords(parse, lineMap, text),
lineMap,
}, 0);
}
async addFile(path: string, depth: number): Promise<void> {
if (depth > MAX_LOCAL_DEPTH) return;
const parsed = await this.ctx.readRecords(path);
if (parsed) await this.addParsed(parsed, depth);
}
private async addParsed(parsed: ParsedFile, depth: number): Promise<void> {
const path = parsed.file.path;
const key = scopePathKey(path);
if (!parsed.records || this.visited.has(key)) return;
this.visited.add(key);
if (parsed.lineMap) this.lineMaps.set(key, parsed.lineMap);
const origin = this.originOf(path);
for (const asset of parsed.records.assets) {
this.addAsset({
type: asset.type,
id: asset.id,
file: path,
line: asset.line,
origin,
stream: "local",
});
}
for (const define of parsed.records.defines) {
const entry: DefineDef = {
name: define.name,
value: define.value,
file: path,
line: define.line,
origin,
};
const arr = this.overlay.defines.get(define.name.toLowerCase());
if (arr) arr.push(entry);
else this.overlay.defines.set(define.name.toLowerCase(), [entry]);
}
for (const inc of parsed.records.includes) {
const resolved = resolveSource(inc.source, dirname(path), this.ctx.searchPaths);
if (!resolved.path) continue;
if (inc.type === "all" || inc.type === "instance") {
await this.addFile(resolved.path, depth + 1);
}
// type="reference" points at compiled manifests; their assets are
// provided by the global index, so the local text overlay skips them.
}
for (const xi of [
...parsed.records.nestedXiIncludes,
...parsed.records.rootXiIncludes,
]) {
const resolved = resolveSource(xi.href, dirname(path), this.ctx.searchPaths);
if (resolved.path) await this.addFile(resolved.path, depth + 1);
}
}
private addAsset(def: AssetDef): void {
const typeKey = def.type;
const idKey = def.id.toLowerCase();
let byId = this.overlay.assets.get(typeKey);
if (!byId) {
byId = new Map();
this.overlay.assets.set(typeKey, byId);
}
const arr = byId.get(idKey);
if (arr) {
if (arr.some((a) => a.file === def.file && a.line === def.line)) return;
arr.push(def);
} else {
byId.set(idKey, [def]);
}
const all = this.overlay.assetsById.get(idKey);
if (all) {
if (all.some((a) => a.file === def.file && a.line === def.line)) return;
all.push(def);
} else {
this.overlay.assetsById.set(idKey, [def]);
}
}
private originOf(path: string): "project" | "sdk" | "manifest" {
const p = resolve(path).toLowerCase();
const project = resolve(this.ctx.projectDir).toLowerCase();
const sdk = resolve(this.ctx.sdkDir).toLowerCase();
if (p.startsWith(project + "\\")) return "project";
if (sdk && p.startsWith(sdk + "\\")) return "sdk";
return "project";
}
}
+249
View File
@@ -0,0 +1,249 @@
import { dirname } from "node:path";
import type {
LineMap,
XmlDocument,
XmlElement,
XmlParseError,
} from "../language/xmlParser";
import { resolveElementType } from "../language/typeContext";
import { isAssignableTo } from "../model/schemaModel";
import { findXPointerContainer, localName } from "./xpointer";
/**
* A logical document is the parsed tree of the currently open file with
* supported `xi:include` targets spliced in place. Every node keeps its
* original source file and offsets so diagnostics / hover / navigation can
* map back to the real file.
*/
export interface LogicalElement extends Omit<XmlElement, "parent" | "children"> {
parent: LogicalElement | null;
children: LogicalElement[];
sourceFile: string;
}
export interface LogicalDocument {
root: LogicalElement | null;
elements: LogicalElement[];
/** Mirrors XmlDocument so existing helpers (findElementAt etc.) work. */
errors: XmlParseError[];
declarationEnd: number;
}
export interface ExpandContext {
/** Resolves an xi:include href (BAB search order). */
resolve(source: string, currentDir: string): string | null;
/** Reads a target and guarantees a DOM parse tree. */
readDom(path: string): Promise<{ parse: XmlDocument; lineMap: LineMap } | null>;
/** Include-depth guard. Defaults to 64. */
maxDepth?: number;
}
const DEFAULT_MAX_DEPTH = 64;
/**
* Builds the logical document for `entryPath` by replacing supported
* `xi:include` elements with their selected target children.
*
* The original xi:include node stays in `elements` (so hover keeps working)
* but is removed from the logical child list; its selected content is spliced
* in as siblings. Nodes are shallow-cloned shells with a rebuilt parent/child
* chain, so cached parse trees in the indexer are never mutated.
*/
export async function expandDocument(
entryPath: string,
parse: XmlDocument,
ctx: ExpandContext,
): Promise<LogicalDocument> {
const elements: LogicalElement[] = [];
const map = new Map<XmlElement, LogicalElement>();
// Pre-create a logical shell for every original element (including orphan
// nodes produced by the parser's unterminated-quote recovery). Parent
// pointers follow the original tree so type context is preserved.
for (const orig of parse.elements) {
const parent = orig.parent ? map.get(orig.parent) ?? null : null;
const clone = cloneNode(orig, parent, entryPath);
map.set(orig, clone);
elements.push(clone);
}
// Cycle guard is a recursion stack, not a global visited set: the same
// fragment may legitimately be included under several parents.
const stack = new Set<string>();
const root = parse.root ? map.get(parse.root) ?? null : null;
// Traverse from the real root plus any parser-recovery orphans (elements
// whose parent is null but are not the root).
const roots = new Set<XmlElement>();
if (parse.root) roots.add(parse.root);
for (const orig of parse.elements) {
if (orig !== parse.root && orig.parent === null) roots.add(orig);
}
for (const origRoot of roots) {
const logicalRoot = map.get(origRoot)!;
await expandChildren(
origRoot,
logicalRoot,
entryPath,
0,
ctx,
elements,
stack,
map,
);
}
return { root, elements, errors: parse.errors, declarationEnd: parse.declarationEnd };
}
function cloneNode(
el: XmlElement,
parent: LogicalElement | null,
sourceFile: string,
): LogicalElement {
return { ...el, parent, children: [], sourceFile };
}
async function expandChildren(
origParent: XmlElement,
logicalParent: LogicalElement,
file: string,
depth: number,
ctx: ExpandContext,
elements: LogicalElement[],
stack: Set<string>,
map?: Map<XmlElement, LogicalElement>,
): Promise<void> {
for (const child of origParent.children) {
await handleChild(child, logicalParent, file, depth, ctx, elements, stack, map);
}
}
async function handleChild(
orig: XmlElement,
logicalParent: LogicalElement,
file: string,
depth: number,
ctx: ExpandContext,
elements: LogicalElement[],
stack: Set<string>,
map?: Map<XmlElement, LogicalElement>,
): Promise<void> {
const isXi = orig.name.toLowerCase().startsWith("xi:") &&
localName(orig.name).toLowerCase() === "include";
if (isXi) {
// Keep the xi:include itself discoverable (hover), but let its selected
// content replace it in the logical child list.
if (!map?.has(orig)) elements.push(cloneNode(orig, logicalParent, file));
await expandXi(orig, logicalParent, file, depth, ctx, elements, stack);
return;
}
let clone = map?.get(orig);
if (!clone) {
clone = cloneNode(orig, logicalParent, file);
elements.push(clone);
}
logicalParent.children.push(clone);
await expandChildren(orig, clone, file, depth + 1, ctx, elements, stack, map);
}
async function expandXi(
xi: XmlElement,
logicalParent: LogicalElement,
parentFile: string,
depth: number,
ctx: ExpandContext,
elements: LogicalElement[],
stack: Set<string>,
): Promise<void> {
const href = xi.attrs.find((a) => a.name === "href")?.value;
if (!href) return;
const resolved = ctx.resolve(href, dirname(parentFile));
if (!resolved) return;
const key = normPath(resolved);
if (stack.has(key)) return; // include cycle
if (depth > (ctx.maxDepth ?? DEFAULT_MAX_DEPTH)) return;
stack.add(key);
try {
const target = await ctx.readDom(resolved);
if (!target?.parse?.root) return;
const xpointer = xi.attrs.find((a) => a.name === "xpointer")?.value ?? "";
const selected = xpointer
? findXPointerContainer(target.parse, xpointer)?.children ?? []
: target.parse.root.children;
for (const sel of selected) {
await handleChild(sel, logicalParent, resolved, depth + 1, ctx, elements, stack);
}
} finally {
stack.delete(key);
}
}
/** True when a logical element's resolved XSD type is a GameObject. */
export function isGameObjectElement(el: LogicalElement): boolean {
const type = resolveElementType(el);
return type != null && isAssignableTo(type, "GameObject");
}
/** Nearest ancestor whose resolved type is a GameObject (or subclass). */
export function findContainingGameObject(
el: LogicalElement,
): LogicalElement | null {
let cur = el.parent;
while (cur) {
if (isGameObjectElement(cur)) return cur;
cur = cur.parent;
}
return null;
}
export interface LocalIdInfo {
id: string;
el: LogicalElement;
}
/**
* Collects every `id` defined inside a GameObject subtree (including modules
* spliced in through xi:include). These are the candidates for Poid-typed
* pipeline-local references such as AttachModuleId / ModuleId.
*/
export function collectLocalIds(root: LogicalElement): LocalIdInfo[] {
const out: LocalIdInfo[] = [];
const seen = new Set<string>();
const stack: LogicalElement[] = [root];
while (stack.length) {
const el = stack.pop()!;
const idAttr = el.attrs.find((a) => a.name === "id");
if (idAttr?.hasValue) {
const key = idAttr.value.toLowerCase();
if (!seen.has(key)) {
seen.add(key);
out.push({ id: idAttr.value, el });
}
}
for (const child of el.children) stack.push(child);
}
return out;
}
/** Finds an id inside a GameObject subtree (case-insensitive). */
export function findLocalId(
root: LogicalElement,
id: string,
): LogicalElement | null {
const wanted = id.toLowerCase();
const stack: LogicalElement[] = [root];
while (stack.length) {
const el = stack.pop()!;
const idAttr = el.attrs.find((a) => a.name === "id");
if (idAttr?.hasValue && idAttr.value.toLowerCase() === wanted) return el;
for (const child of el.children) stack.push(child);
}
return null;
}
function normPath(path: string): string {
return path.replace(/\\/g, "/").toLowerCase();
}
+293
View File
@@ -0,0 +1,293 @@
/**
* Compact per-file index records.
*
* A rebuild only needs each file's top-level assets, defines, includes and
* xi:include targets — not the DOM. Extracting these records at parse time
* and caching them across rebuilds lets Corona-scale rebuilds skip both the
* DOM and the file I/O for unchanged files, while keeping the DOM cache small
* for on-demand features (hover / navigation / outline).
*
* Pure TypeScript: no vscode dependency.
*/
import type { LineMap, XmlDocument } from "../language/xmlParser";
import type { ShallowDocument } from "./shallowScan";
import { attributesOfType, typeInfo } from "../model/schemaModel";
import { resolveElementType } from "../language/typeContext";
import {
isReferenceAttributeOfType,
isReferenceContentType,
} from "./refs";
export interface IndexRecordAsset {
/** Top-level element name, e.g. "W3DContainer". */
type: string;
id: string;
/** 1-based line of the id attribute value. */
line: number;
}
export interface IndexRecordDefine {
name: string;
value: string;
/** 1-based line of the <Define> element. */
line: number;
}
export interface IndexRecordInclude {
type: "all" | "instance" | "reference" | null;
source: string;
/** 1-based line of the <Include> element. */
line: number;
}
export interface IndexRecordXi {
href: string;
xpointer: string | null;
/** 1-based line of the <xi:include> element. */
line: number;
}
export interface IndexRecordReference {
/** "attr" for attribute values, "content" for simple-content text. */
kind: "attr" | "content";
/**
* XSD reference target type (from `xas:refType`), or null for untyped
* `isRef` references and `inheritFrom` (which filters by the element's own
* type via `selfType`).
*/
refType: string | null;
/** Element type used by `inheritFrom` filtering; null otherwise. */
selfType: string | null;
/** The referenced id text (whole attribute value / trimmed content). */
value: string;
/** 1-based line of the value. */
line: number;
/** Character offset of the value start (relative to the file text). */
start: number;
/** Character offset one past the value end. */
end: number;
}
export interface IndexRecords {
assets: IndexRecordAsset[];
defines: IndexRecordDefine[];
includes: IndexRecordInclude[];
/** <xi:include> elements that are direct children of the root. */
rootXiIncludes: IndexRecordXi[];
/** <xi:include> elements nested anywhere else in the document. */
nestedXiIncludes: IndexRecordXi[];
/** Typed global-asset references (attribute values + simple content). */
references: IndexRecordReference[];
}
const INCLUDE_TYPES = new Set(["all", "instance", "reference"]);
function lineOf(lineMap: LineMap, offset: number): number {
return lineMap.positionAt(offset).line + 1;
}
function localName(tag: string): string {
const idx = tag.lastIndexOf(":");
return idx >= 0 ? tag.slice(idx + 1) : tag;
}
/**
* Extracts the index records of a fully parsed document. Mirrors the walk
* semantics of the indexer exactly: top-level assets (excluding
* Tags/Includes/Defines), $DEFINE constants, the top-level <Includes> block
* and root/nested <xi:include> elements.
*/
export function extractIndexRecords(
parse: XmlDocument,
lineMap: LineMap,
text: string,
): IndexRecords {
const assets: IndexRecordAsset[] = [];
const defines: IndexRecordDefine[] = [];
const includes: IndexRecordInclude[] = [];
const rootXiIncludes: IndexRecordXi[] = [];
const nestedXiIncludes: IndexRecordXi[] = [];
const references: IndexRecordReference[] = [];
const root = parse.root;
if (!root) {
return { assets, defines, includes, rootXiIncludes, nestedXiIncludes, references };
}
for (const child of root.children) {
const local = localName(child.name);
if (local === "Tags" || local === "Includes" || local === "Defines") continue;
if (local === "include") {
const href = child.attrs.find((a) => a.name === "href")?.value;
if (href) {
rootXiIncludes.push({
href,
xpointer: child.attrs.find((a) => a.name === "xpointer")?.value ?? null,
line: lineOf(lineMap, child.start),
});
}
continue;
}
const idAttr = child.attrs.find((a) => a.name === "id");
if (idAttr) {
assets.push({ type: local, id: idAttr.value, line: lineOf(lineMap, idAttr.valueStart) });
}
}
for (const child of root.children) {
if (localName(child.name) !== "Defines") continue;
for (const define of child.children) {
if (localName(define.name) !== "Define") continue;
const name = define.attrs.find((a) => a.name === "name")?.value;
if (!name) continue;
defines.push({
name,
value: define.attrs.find((a) => a.name === "value")?.value ?? "",
line: lineOf(lineMap, define.start),
});
}
}
const includesElem = root.children.find((c) => localName(c.name) === "Includes");
if (includesElem) {
for (const inc of includesElem.children) {
if (localName(inc.name) !== "Include") continue;
const source = inc.attrs.find((a) => a.name === "source")?.value;
if (!source) continue;
const type = inc.attrs.find((a) => a.name === "type")?.value;
includes.push({
type:
type && INCLUDE_TYPES.has(type)
? (type as "all" | "instance" | "reference")
: null,
source,
line: lineOf(lineMap, inc.start),
});
}
}
for (const el of parse.elements) {
if (localName(el.name) !== "include") continue;
if (el.parent === root) continue; // already handled above
const href = el.attrs.find((a) => a.name === "href")?.value;
if (!href) continue;
nestedXiIncludes.push({
href,
xpointer: el.attrs.find((a) => a.name === "xpointer")?.value ?? null,
line: lineOf(lineMap, el.start),
});
}
collectReferenceRecords(parse, lineMap, text, references);
return { assets, defines, includes, rootXiIncludes, nestedXiIncludes, references };
}
/**
* Walks every element of a fully parsed document and records typed
* global-asset references: reference attributes, `inheritFrom` and
* simple-content reference text. Local `id` definitions, Poid pipeline-local
* references and `$DEFINE`/`=` values are intentionally skipped (the same
* semantics as diagnostics / hover / navigation).
*
* The stored `refType` / `selfType` pair is exactly what
* `resolveReferenceTargetsForType` derives from the element context, so the
* reverse reference index can resolve these records after the whole index is
* built without re-walking the document or re-resolving element types.
*/
function collectReferenceRecords(
parse: XmlDocument,
lineMap: LineMap,
text: string,
out: IndexRecordReference[],
): void {
for (const el of parse.elements) {
const elType = resolveElementType(el);
for (const attr of el.attrs) {
if (!attr.hasValue) continue;
if (!isReferenceAttributeOfType(elType, attr.name)) continue;
const value = attr.value;
if (!value || value.startsWith("$") || value.startsWith("=")) continue;
let refType: string | null = null;
let selfType: string | null = null;
if (attr.name.toLowerCase() === "inheritfrom") {
selfType = elType;
} else if (elType) {
refType =
attributesOfType(elType).find((a) => a.name === attr.name)?.refType ??
null;
}
out.push({
kind: "attr",
refType,
selfType,
value,
line: lineOf(lineMap, attr.valueStart),
start: attr.valueStart,
end: attr.valueEnd,
});
}
if (
elType &&
isReferenceContentType(elType) &&
!el.selfClosing &&
el.closeTagStart >= 0
) {
const raw = text.slice(el.startTagEnd, el.closeTagStart);
const value = raw.trim();
if (
!value ||
value.startsWith("$") ||
value.startsWith("=") ||
value.includes("<")
) {
continue;
}
const start = el.startTagEnd + raw.indexOf(value);
const info = typeInfo(elType);
out.push({
kind: "content",
refType: info?.kind === "simple" ? info.refType : null,
selfType: null,
value,
line: lineOf(lineMap, start),
start,
end: start + value.length,
});
}
}
}
/** Converts a shallow scan (offsets) into index records (1-based lines). */
export function recordsFromShallow(scan: ShallowDocument, lineMap: LineMap): IndexRecords {
return {
assets: scan.assets.map((a) => ({
type: a.name,
id: a.id,
line: lineOf(lineMap, a.idValueStart),
})),
defines: scan.defines.map((d) => ({
name: d.name,
value: d.value,
line: lineOf(lineMap, d.start),
})),
includes: scan.includes.map((i) => ({
type: i.type,
source: i.source,
line: lineOf(lineMap, i.start),
})),
rootXiIncludes: scan.rootXiIncludes.map((x) => ({
href: x.href,
xpointer: x.xpointer,
line: lineOf(lineMap, x.start),
})),
nestedXiIncludes: scan.nestedXiIncludes.map((x) => ({
href: x.href,
xpointer: x.xpointer,
line: lineOf(lineMap, x.start),
})),
references: [],
};
}
+219
View File
@@ -0,0 +1,219 @@
/**
* Reverse reference index built from per-file reference records.
*
* The indexer stores compact reference records per file (attribute values,
* simple-content text and inheritFrom, with XSD `refType`/`selfType` context
* captured at parse time). After the include walk, this module resolves every
* record against the final asset maps and produces:
*
* definition key -> reference sites
*
* which powers CodeLens reference counts, semantic Find All References and
* the "unreferenced assets" report. Pure TypeScript: no vscode dependency.
*/
import { extractIndexRecords, type IndexRecords } from "./records";
import {
filterAndScoreDefs,
isReferenceTargetType,
type ReferenceLookup,
} from "./refs";
import { buildSearchPaths, resolveSource } from "./includeResolver";
import { normKey, recordsHash } from "./caches";
import { LineMap, parseXml } from "../language/xmlParser";
import type { AssetDef, ModIndex, ReferenceSite } from "./types";
/** A file whose reference records should be resolved. */
export interface ReferenceRecordSource {
/** Absolute path of the referencing file. */
file: string;
records: IndexRecords;
}
/** Stable key identifying one specific asset definition. */
export function assetDefKey(
def: Pick<AssetDef, "type" | "id" | "file" | "line">,
): string {
return `${def.type}\u0000${def.id.toLowerCase()}\u0000${def.file.toLowerCase()}\u0000${def.line}`;
}
/**
* Resolves per-file reference records against the asset lookup and returns
* the reverse map. A record resolves to every definition that satisfies its
* `refType` / `selfType` context (same strict filtering as go-to-definition),
* so same-name ids of different types never share reference counts.
*/
export function buildReferenceIndex(
sources: Iterable<ReferenceRecordSource>,
lookup: ReferenceLookup,
): Map<string, ReferenceSite[]> {
const map = new Map<string, ReferenceSite[]>();
for (const { file, records } of sources) {
for (const ref of records.references) {
const defs = lookup.assetsById.get(ref.value.toLowerCase());
if (!defs?.length) continue;
const targets = filterAndScoreDefs(defs, ref.refType, ref.selfType);
for (const target of targets) {
const key = assetDefKey(target.def);
let sites = map.get(key);
if (!sites) {
sites = [];
map.set(key, sites);
}
sites.push({
file,
line: ref.line,
start: ref.start,
end: ref.end,
kind: ref.kind,
});
}
}
}
return map;
}
/** Reference sites for one definition (empty when the index has none). */
export function referenceSitesForDef(
idx: Pick<ModIndex, "references"> | null | undefined,
def: Pick<AssetDef, "type" | "id" | "file" | "line">,
): ReferenceSite[] {
return idx?.references?.get(assetDefKey(def)) ?? [];
}
function normFileKey(path: string): string {
return path.replace(/\\/g, "/").toLowerCase();
}
/**
* Reference sites that belong to a definition opened in the editor.
*
* Besides the definition's own reverse-index bucket, this unions the sites
* of manifest definitions that map back to the same XML source file via
* `manifestSource`. A manifest asset with a resolvable SageXml source is
* semantically the same asset as that XML definition, so references to it
* should show up on the source file's CodeLens too (Find All References
* already sees them because it unions every same-id/type definition).
*/
export function referenceSitesForDefinition(
idx: ModIndex,
def: Pick<AssetDef, "type" | "id" | "file" | "line">,
): ReferenceSite[] {
const sites = referenceSitesForDef(idx, def);
const byId = idx.assets.get(def.type)?.get(def.id.toLowerCase());
if (!byId?.length) return sites;
const defFile = normFileKey(def.file);
const searchPaths = buildSearchPaths(idx.sdkDir, idx.projectDir);
const seen = new Set(
sites.map((s) => `${s.file}\u0000${s.start}\u0000${s.end}\u0000${s.kind}`),
);
for (const other of byId) {
if (other.origin !== "manifest" || !other.manifestSource) continue;
const resolved = resolveSource(other.manifestSource, null, searchPaths).path;
if (!resolved || normFileKey(resolved) !== defFile) continue;
for (const site of referenceSitesForDef(idx, other)) {
const key = `${site.file}\u0000${site.start}\u0000${site.end}\u0000${site.kind}`;
if (seen.has(key)) continue;
seen.add(key);
sites.push(site);
}
}
return sites;
}
export interface UnreferencedOptions {
/**
* When true (default), only report types that are reference targets by
* design. Auto-registered / structural types (settings, map metadata,
* w3x sub-assets...) are excluded because zero references is their normal
* state.
*/
onlyReferenceTargetTypes?: boolean;
}
/**
* Project asset definitions with zero incoming references, grouped by type.
* Manifest / SDK definitions and `instance`-only assets are never reported
* (they are not part of the compiled stream in the same way).
*/
export function unreferencedByType(
idx: ModIndex,
options: UnreferencedOptions = {},
): Map<string, AssetDef[]> {
const onlyReferenceTargets = options.onlyReferenceTargetTypes ?? true;
const out = new Map<string, AssetDef[]>();
for (const [type, byId] of idx.assets) {
if (onlyReferenceTargets && !isReferenceTargetType(type)) continue;
const defs: AssetDef[] = [];
for (const arr of byId.values()) {
for (const def of arr) {
if (def.origin !== "project" || def.viaInstance) continue;
if (referenceSitesForDef(idx, def).length > 0) continue;
defs.push(def);
}
}
if (defs.length) {
defs.sort((a, b) => a.id.localeCompare(b.id));
out.set(type, defs);
}
}
return out;
}
/** Minimal workspace surface needed by the records-desync self-heal. */
export interface RecordsSyncWorkspace {
index: ModIndex | null;
invalidate(path: string): void;
scheduleRebuild(reason: string): void;
}
/** Minimal document surface (vscode.TextDocument subset, no vscode dep). */
export interface RecordsSyncDocument {
uri: { fsPath: string; scheme?: string };
isDirty?: boolean;
getText(): string;
}
/**
* True when a clean (saved) document's text no longer matches the records the
* published snapshot was built from. This catches cache entries that slipped
* through stat validation (e.g. a rewrite with preserved timestamps on an
* external drive) or watcher events lost during a drive reconnect.
*/
export function documentRecordsDesynced(
idx: ModIndex,
fsPath: string,
text: string,
): boolean {
const expected = idx.recordsHashes?.get(normKey(fsPath));
if (expected == null) return false;
const lineMap = new LineMap(text);
const records = extractIndexRecords(parseXml(text), lineMap, text);
return recordsHash(records) !== expected;
}
/**
* Self-heal: when the open, saved document's content differs from the
* snapshot's records hash, invalidate exactly that file and schedule a
* rebuild. Returns true when a rebuild was scheduled. Unsaved (dirty)
* documents are skipped — the editor text is intentionally ahead of disk.
*/
export function scheduleRebuildIfRecordsDesync(
ws: RecordsSyncWorkspace,
document: RecordsSyncDocument,
): boolean {
if (
document.isDirty ||
(document.uri.scheme != null && document.uri.scheme !== "file")
) {
return false;
}
const idx = ws.index;
if (!idx) return false;
const fsPath = document.uri.fsPath;
if (!documentRecordsDesynced(idx, fsPath, document.getText())) return false;
ws.invalidate(fsPath);
ws.scheduleRebuild("records-desync");
return true;
}
+144 -5
View File
@@ -1,15 +1,33 @@
import { import {
allTypeNames,
attributesOfType, attributesOfType,
canonicalTypeName,
elementTypeName, elementTypeName,
isAssignableTo, isAssignableTo,
typeChain,
typeInfo,
} from "../model/schemaModel"; } from "../model/schemaModel";
import type { AssetDef, ModIndex } from "./types"; import type { AssetDef, LocalOverlay } from "./types";
export interface ReferenceTarget { export interface ReferenceTarget {
def: AssetDef; def: AssetDef;
score: number; score: number;
} }
/**
* The subset of `ModIndex` that reference resolution needs. Kept narrow so
* the reverse reference index can resolve records against the indexer's live
* maps without constructing a full index snapshot.
*/
export interface ReferenceLookup {
/** type -> id -> definitions. */
assets: Map<string, Map<string, AssetDef[]>>;
/** id -> definitions across all types. */
assetsById: Map<string, AssetDef[]>;
/** Optional document-local overlay (consulted first). */
local?: LocalOverlay;
}
/** /**
* True when an attribute is a "pipeline-local" reference that the global * True when an attribute is a "pipeline-local" reference that the global
* asset index cannot judge: * asset index cannot judge:
@@ -76,7 +94,7 @@ export function isReferenceAttributeOfType(
* Returns [] when the attribute is not a typed reference or nothing matches. * Returns [] when the attribute is not a typed reference or nothing matches.
*/ */
export function resolveReferenceTargets( export function resolveReferenceTargets(
idx: ModIndex, idx: ReferenceLookup,
elementType: string, elementType: string,
attrName: string, attrName: string,
id: string, id: string,
@@ -91,13 +109,16 @@ export function resolveReferenceTargets(
/** Same resolution, driven by a resolved XSD type name. */ /** Same resolution, driven by a resolved XSD type name. */
export function resolveReferenceTargetsForType( export function resolveReferenceTargetsForType(
idx: ModIndex, idx: ReferenceLookup,
typeName: string | null, typeName: string | null,
attrName: string, attrName: string,
id: string, id: string,
): ReferenceTarget[] { ): ReferenceTarget[] {
const defs = idx.assetsById.get(id.toLowerCase()); const defs = mergeLocalAndGlobalDefs(
if (!defs?.length) return []; idx.local?.assetsById.get(id.toLowerCase()),
idx.assetsById.get(id.toLowerCase()),
);
if (!defs.length) return [];
const nameLower = attrName.toLowerCase(); const nameLower = attrName.toLowerCase();
let refType: string | null = null; let refType: string | null = null;
@@ -114,6 +135,56 @@ export function resolveReferenceTargetsForType(
refType = attr.refType; refType = attr.refType;
} }
return filterAndScoreDefs(defs, refType, selfType);
}
/**
* True when an element's text content is a typed reference to a global
* asset: the element's resolved XSD type is a simple type carrying an
* `xas:refType` (e.g. `<CreateObject>` with `GameObjectWeakRef`).
*
* Only *typed* refs are treated as content references. Generic untyped
* `AssetReference` content is used by real data for shader constants,
* mesh sub-object names and other values that are not global asset ids
* (`FXShaderConstantTexture@Value`, `RenderSubObjectReference@Mesh`), so
* resolving those globally would produce false hover/navigation/diagnostics.
* Poid pipeline-local ids are excluded for the same reason.
*/
export function isReferenceContentType(typeName: string | null): boolean {
if (!typeName) return false;
const info = typeInfo(typeName);
if (info?.kind !== "simple") return false;
if (typeChain(typeName).includes("Poid")) return false;
return info.refType != null;
}
/**
* Resolves the definitions an element's text content should point to,
* filtered by the element type's `xas:refType`
* (e.g. `GameObjectWeakRef` -> `GameObject`).
*/
export function resolveContentReferenceTargets(
idx: ReferenceLookup,
typeName: string | null,
id: string,
): ReferenceTarget[] {
if (!isReferenceContentType(typeName)) return [];
if (!typeName) return [];
const defs = mergeLocalAndGlobalDefs(
idx.local?.assetsById.get(id.toLowerCase()),
idx.assetsById.get(id.toLowerCase()),
);
if (!defs.length) return [];
const info = typeInfo(typeName);
const refType = info?.kind === "simple" ? info.refType : null;
return filterAndScoreDefs(defs, refType, null);
}
export function filterAndScoreDefs(
defs: readonly AssetDef[],
refType: string | null,
selfType: string | null,
): ReferenceTarget[] {
const targets: ReferenceTarget[] = []; const targets: ReferenceTarget[] = [];
for (const def of defs) { for (const def of defs) {
if (refType && !isAssignableTo(def.type, refType)) continue; if (refType && !isAssignableTo(def.type, refType)) continue;
@@ -127,3 +198,71 @@ export function resolveReferenceTargetsForType(
targets.sort((a, b) => a.score - b.score || a.def.id.localeCompare(b.def.id)); targets.sort((a, b) => a.score - b.score || a.def.id.localeCompare(b.def.id));
return targets; return targets;
} }
/**
* Merges document-local definitions with the global index, keeping local
* entries first and de-duplicating definitions that exist in both.
*/
export function mergeLocalAndGlobalDefs(
local: readonly AssetDef[] | undefined,
global: readonly AssetDef[] | undefined,
): AssetDef[] {
const seen = new Set<string>();
const out: AssetDef[] = [];
for (const list of [local, global]) {
if (!list) continue;
for (const def of list) {
const key = `${def.type}\u0000${def.id.toLowerCase()}\u0000${def.file}\u0000${def.line}`;
if (seen.has(key)) continue;
seen.add(key);
out.push(def);
}
}
return out;
}
let referenceTargetTypeSet: Set<string> | null = null;
/**
* The set of XSD types that are "reference targets by design": at least one
* typed reference attribute / simple-content reference points at them, or
* they are inheritable (`inheritFrom`). Types outside this set are
* auto-registered / structural (settings, map metadata, w3x sub-assets...),
* so a zero reference count is their normal state and counts would only be
* noise.
*/
export function referenceTargetTypes(): ReadonlySet<string> {
if (referenceTargetTypeSet) return referenceTargetTypeSet;
const set = new Set<string>();
const add = (t: string | null) => {
if (!t) return;
set.add(canonicalTypeName(t) ?? t);
};
for (const typeName of allTypeNames()) {
const info = typeInfo(typeName);
if (!info) continue;
if (info.kind === "complex") {
for (const attr of info.attributes) {
if (isLocalReferenceAttribute(typeName, attr.name)) continue;
if (attr.refType) add(attr.refType);
}
if (info.attributes.some((a) => a.name.toLowerCase() === "inheritfrom")) {
add(typeName);
}
} else if (
info.kind === "simple" &&
info.refType &&
!typeChain(typeName).includes("Poid")
) {
add(info.refType);
}
}
referenceTargetTypeSet = set;
return set;
}
/** True when the type is a designed reference target (see above). */
export function isReferenceTargetType(typeName: string | null): boolean {
if (!typeName) return false;
return referenceTargetTypes().has(canonicalTypeName(typeName) ?? typeName);
}
+274
View File
@@ -0,0 +1,274 @@
/**
* Shallow XML scanner for large art-asset documents (e.g. .w3x).
*
* Full XML parsing builds an element tree whose memory footprint is roughly
* 17x the source text (measured on Corona model files). Model exports like
* <W3DMesh> contain hundreds of thousands of tiny numeric elements
* (Vertices/V, Triangles/T, ...) that the extension never needs: the index
* only consumes top-level asset definitions (name + id), top-level
* <Includes>, nested <xi:include> targets and <Defines> constants.
*
* This scanner performs a single linear pass without building child nodes,
* so multi-megabyte model files can be indexed in linear time with ~zero
* retained memory.
*
* Pure TypeScript: no vscode dependency, reusable outside the extension.
*/
export interface ShallowAssetRecord {
/** Top-level element name, e.g. "W3DContainer". */
name: string;
/** Value of the id attribute. */
id: string;
/** Offset of the first id value character. */
idValueStart: number;
/** Offset one past the last id value character. */
idValueEnd: number;
/** Offset of the element's "<". */
start: number;
/** Offset one past the ">" of the start tag. */
startTagEnd: number;
}
export interface ShallowIncludeRecord {
/** "all" | "instance" | "reference", or null when absent/unknown. */
type: "all" | "instance" | "reference" | null;
source: string;
/** Offset of the <Include> element. */
start: number;
}
export interface ShallowXiIncludeRecord {
href: string;
xpointer: string | null;
/** Offset of the <xi:include> element. */
start: number;
}
export interface ShallowDefineRecord {
name: string;
value: string;
/** Offset of the <Define> element. */
start: number;
}
export interface ShallowScanError {
message: string;
offset: number;
}
export interface ShallowDocument {
assets: ShallowAssetRecord[];
includes: ShallowIncludeRecord[];
/** <xi:include> elements that are direct children of the root. */
rootXiIncludes: ShallowXiIncludeRecord[];
/** <xi:include> elements nested anywhere else in the document. */
nestedXiIncludes: ShallowXiIncludeRecord[];
defines: ShallowDefineRecord[];
errors: ShallowScanError[];
}
interface AttrHit {
value: string;
valueStart: number;
valueEnd: number;
}
/**
* Matches a complete XML tag while respecting quoted attribute values, so a
* ">" or "/" inside a value never terminates the tag early.
*/
const TAG_RE = /<(?:"[^"]*"|'[^']*'|[^'"<>])*>/g;
export function scanXmlShallow(text: string): ShallowDocument {
const errors: ShallowScanError[] = [];
const assets: ShallowAssetRecord[] = [];
const includes: ShallowIncludeRecord[] = [];
const rootXiIncludes: ShallowXiIncludeRecord[] = [];
const nestedXiIncludes: ShallowXiIncludeRecord[] = [];
const defines: ShallowDefineRecord[] = [];
// Depth of the currently open element stack. The document root opens at
// depth 0 -> 1, so its direct children open when depth === 1.
let depth = 0;
let inIncludes = false;
let inDefines = false;
let i = 0;
while (i < text.length) {
const lt = text.indexOf("<", i);
if (lt < 0) break;
// Non-element constructs: comments, CDATA, DOCTYPE and processing
// instructions are skipped whole so their content never looks like tags.
if (text.startsWith("<!--", lt)) {
const close = text.indexOf("-->", lt + 4);
if (close < 0) {
errors.push({ message: "Unterminated comment", offset: lt });
break;
}
i = close + 3;
continue;
}
if (text.startsWith("<![CDATA[", lt)) {
const close = text.indexOf("]]>", lt + 9);
if (close < 0) {
errors.push({ message: "Unterminated CDATA section", offset: lt });
break;
}
i = close + 3;
continue;
}
if (text.startsWith("<!", lt)) {
const close = text.indexOf(">", lt + 2);
if (close < 0) {
errors.push({ message: "Unterminated DOCTYPE", offset: lt });
break;
}
i = close + 1;
continue;
}
if (text.startsWith("<?", lt)) {
const close = text.indexOf("?>", lt + 2);
if (close < 0) {
errors.push({ message: "Unterminated processing instruction", offset: lt });
break;
}
i = close + 2;
continue;
}
TAG_RE.lastIndex = lt;
const m = TAG_RE.exec(text);
if (!m) {
errors.push({ message: "Unterminated tag", offset: lt });
break;
}
const tag = m[0];
const gt = m.index + tag.length - 1;
const inner = tag.slice(1, -1);
const closing = inner.startsWith("/");
const selfClosing = !closing && /\/\s*$/.test(inner);
const body = closing ? inner.slice(1) : inner;
let nameEnd = 0;
while (nameEnd < body.length && !/[\s/>]/.test(body[nameEnd])) nameEnd++;
const name = body.slice(0, nameEnd);
const base = lt + 1;
if (!closing) {
// Top-level elements (direct children of the root).
if (depth === 1) {
inIncludes = name === "Includes";
inDefines = name === "Defines";
const idAttr = findAttr(inner, base, "id");
if (idAttr && idAttr.value) {
assets.push({
name,
id: idAttr.value,
idValueStart: idAttr.valueStart,
idValueEnd: idAttr.valueEnd,
start: lt,
startTagEnd: gt + 1,
});
}
}
// <Include> entries inside the top-level <Includes> block.
if (inIncludes && depth === 2 && name === "Include") {
const type = findAttr(inner, base, "type");
const source = findAttr(inner, base, "source");
if (source?.value) {
includes.push({
type:
type?.value === "all" || type?.value === "instance" || type?.value === "reference"
? type.value
: null,
source: source.value,
start: lt,
});
}
}
// <Define> entries inside the top-level <Defines> block.
if (inDefines && depth === 2 && name === "Define") {
const nameAttr = findAttr(inner, base, "name");
const valueAttr = findAttr(inner, base, "value");
if (nameAttr?.value) {
defines.push({ name: nameAttr.value, value: valueAttr?.value ?? "", start: lt });
}
}
// Nested <xi:include> (or any *:include) anywhere in the document.
if (localName(name) === "include") {
const href = findAttr(inner, base, "href");
const xpointer = findAttr(inner, base, "xpointer");
if (href?.value) {
const rec: ShallowXiIncludeRecord = {
href: href.value,
xpointer: xpointer?.value ?? null,
start: lt,
};
if (depth === 1) rootXiIncludes.push(rec);
else nestedXiIncludes.push(rec);
}
}
if (!selfClosing) depth++;
} else {
depth--;
}
i = gt + 1;
}
return { assets, includes, rootXiIncludes, nestedXiIncludes, defines, errors };
}
/**
* Scans the attributes of a tag (its inner text, without "<" and ">") for
* the first occurrence of `want` and returns its value with absolute offsets
* (base = offset one past the "<").
*/
function findAttr(inner: string, base: number, want: string): AttrHit | null {
let i = 0;
while (i < inner.length) {
while (i < inner.length && /\s/.test(inner[i])) i++;
// Self-closing marker or end of tag: no more attributes.
if (i >= inner.length || inner[i] === "/" || inner[i] === ">") break;
const nameStart = i;
while (i < inner.length && !/[\s=/>]/.test(inner[i])) i++;
const name = inner.slice(nameStart, i);
while (i < inner.length && /\s/.test(inner[i])) i++;
if (inner[i] === "=") {
i++;
while (i < inner.length && /\s/.test(inner[i])) i++;
const q = inner[i];
if (q === '"' || q === "'") {
const vStart = i + 1;
const vEnd = inner.indexOf(q, vStart);
if (vEnd < 0) {
return name === want ? { value: "", valueStart: -1, valueEnd: -1 } : null;
}
if (name === want) {
return {
value: inner.slice(vStart, vEnd),
valueStart: base + vStart,
valueEnd: base + vEnd,
};
}
i = vEnd + 1;
continue;
}
// Unquoted value (tolerated).
const vs = i;
while (i < inner.length && !/[\s>]/.test(inner[i])) i++;
if (name === want) {
return { value: inner.slice(vs, i), valueStart: base + vs, valueEnd: base + i };
}
continue;
}
// Attribute without a value (rare but tolerated).
if (name === want) return { value: "", valueStart: -1, valueEnd: -1 };
}
return null;
}
function localName(name: string): string {
const idx = name.lastIndexOf(":");
return idx >= 0 ? name.slice(idx + 1) : name;
}
+134 -1
View File
@@ -1,6 +1,8 @@
import type { XmlDocument } from "../language/xmlParser"; import type { XmlDocument } from "../language/xmlParser";
import type { ManifestInfo } from "./manifestParser"; import type { ManifestInfo } from "./manifestParser";
import type { LineMap } from "../language/xmlParser"; import type { LineMap } from "../language/xmlParser";
import type { IndexRecords } from "./records";
import type { DocumentCache, IncludeResolveCache, IndexRecordsCache } from "./caches";
export type AssetOrigin = "project" | "sdk" | "manifest"; export type AssetOrigin = "project" | "sdk" | "manifest";
@@ -22,6 +24,22 @@ export interface AssetDef {
manifestSource?: string; manifestSource?: string;
} }
/**
* One resolved reference occurrence pointing at an asset definition.
* Produced by `buildReferenceIndex` from per-file reference records.
*/
export interface ReferenceSite {
/** Absolute path of the referencing file. */
file: string;
/** 1-based line of the reference value. */
line: number;
/** Character offset of the value start (relative to the file text). */
start: number;
/** Character offset one past the value end. */
end: number;
kind: "attr" | "content";
}
export interface DefineDef { export interface DefineDef {
name: string; name: string;
value: string; value: string;
@@ -30,9 +48,36 @@ export interface DefineDef {
origin: AssetOrigin; origin: AssetOrigin;
} }
/**
* Document-local overlay produced by `localScope.ts`. It contains assets /
* defines reachable from the currently open document (its own text plus its
* include chain), even when that file is not part of any global stream.
*
* The overlay is attached to a `ModIndex` as `local` rather than merged into
* the global maps, so large indexes (Corona: ~65k assets) are never copied
* on every keystroke. Lookup helpers consult `local` first.
*/
export interface LocalOverlay {
/** type -> id -> definitions. */
assets: Map<string, Map<string, AssetDef[]>>;
/** id -> definitions across all types. */
assetsById: Map<string, AssetDef[]>;
/** `$NAME` -> definitions. */
defines: Map<string, DefineDef[]>;
}
export interface IndexedFile { export interface IndexedFile {
path: string; path: string;
stat: { mtimeMs: number; size: number } | null; /**
* Multi-signal file stamp used to validate cached entries without
* re-reading content: size, last-write time, creation time and change
* time. Creation/change time catch tools that rewrite a file while
* preserving its mtime (e.g. temp-file + rename exporters) — important on
* removable drives where mtime resolution can be coarse (FAT32: 2 s).
*/
stat:
| { mtimeMs: number; size: number; birthtimeMs: number; ctimeMs: number }
| null;
} }
export interface StreamInfo { export interface StreamInfo {
@@ -63,9 +108,37 @@ export interface IndexerDiagnostic {
export interface IndexStats { export interface IndexStats {
projectDir: string; projectDir: string;
sdkDir: string; sdkDir: string;
/** Last finished phase ("xml" or "art"). */
phase: "xml" | "art";
/** Whether the index is fully complete (art assets included). */
complete: boolean;
indexedFiles: number; indexedFiles: number;
parsedFiles: number; parsedFiles: number;
/** Art-asset documents indexed via shallow scan (no DOM tree). */
shallowScannedFiles: number;
/** Art files registered during the XML phase, scanned later in phase B. */
deferredArtFiles: number;
/** Shallow scans served from the persistent cache (unchanged files). */
shallowCacheHits: number;
/** Parsed XML files served from the persistent records cache. */
recordsCacheHits: number;
/** Include/xi:include resolutions served from the resolve cache. */
resolveCacheHits: number;
/** Include/xi:include resolutions performed during this build. */
resolveCalls: number;
/** Existence checks answered by the directory snapshot (no statSync). */
snapshotHits: number;
/** Existence checks outside the snapshot that fell back to statSync. */
snapshotFallbacks: number;
/** Time spent enumerating Include source candidates (ms). */
candidatesMs: number;
/** Time spent walking the include graph (ms). */
walkMs: number;
/** Time spent shallow-scanning deferred art assets (ms). */
artScanMs: number;
assetCount: number; assetCount: number;
/** Total resolved reference sites in the reverse index. */
referenceCount: number;
defineCount: number; defineCount: number;
manifestFiles: number; manifestFiles: number;
manifestAssetCount: number; manifestAssetCount: number;
@@ -77,6 +150,19 @@ export interface IndexStats {
export interface ModIndex { export interface ModIndex {
projectDir: string; projectDir: string;
sdkDir: string; sdkDir: string;
/**
* True when every indexing phase (including the art-asset shallow scan)
* has finished. Features use this to decide whether unresolved references
* are final errors or provisional "may be a false positive" diagnostics.
*/
complete: boolean;
/** Last finished phase: "xml" (XML + manifests) or "art" (final). */
phase: "xml" | "art";
/**
* True when files changed while this snapshot was being built, so some
* entries may be stale. A follow-up rebuild is scheduled by the workspace.
*/
stale?: boolean;
/** type -> id -> definitions (project + sdk + manifest, deduplicated). */ /** type -> id -> definitions (project + sdk + manifest, deduplicated). */
assets: Map<string, Map<string, AssetDef[]>>; assets: Map<string, Map<string, AssetDef[]>>;
/** id -> definitions across all types. */ /** id -> definitions across all types. */
@@ -91,7 +177,27 @@ export interface ModIndex {
sourceCandidates: SourceCandidate[]; sourceCandidates: SourceCandidate[];
/** Problems found while indexing (unresolved includes, cycles, ...). */ /** Problems found while indexing (unresolved includes, cycles, ...). */
diagnostics: IndexerDiagnostic[]; diagnostics: IndexerDiagnostic[];
/**
* Reverse reference index: asset definition key (see
* `referenceIndex.assetDefKey`) -> reference sites. Built from the compact
* per-file reference records when a snapshot is published, so counts and
* Find All References share one semantic source of truth.
*/
references: Map<string, ReferenceSite[]>;
/**
* normKey(file) -> SHA-1 of the file's compact index records as this
* snapshot consumed them. Lets open documents detect "my file changed on
* disk but the index still uses older records" and trigger a targeted
* rebuild, while ignoring cosmetic text changes (line endings, whitespace).
*/
recordsHashes: Map<string, string>;
stats: IndexStats; stats: IndexStats;
/**
* Document-local overlay (when the index was obtained through the
* workspace's `getScope` / `getIndex` path). Optional so plain indexer
* snapshots remain overlay-free.
*/
local?: LocalOverlay;
} }
export interface IndexOptions { export interface IndexOptions {
@@ -102,6 +208,23 @@ export interface IndexOptions {
additionalDataSearchPaths: string[]; additionalDataSearchPaths: string[];
/** Directory walker used to enumerate files for source completion. */ /** Directory walker used to enumerate files for source completion. */
walker: FileWalker; walker: FileWalker;
/** Optional parse-tree cache shared across rebuilds (owned by the workspace). */
documentCache?: DocumentCache;
/** Optional index-records cache shared across rebuilds (owned by the workspace). */
recordsCache?: IndexRecordsCache;
/** Optional include-resolution cache shared across rebuilds (owned by the workspace). */
resolveCache?: IncludeResolveCache;
/**
* When true, cached documents whose path is not in `changedFiles` are used
* without a per-file stat. The workspace enables this while a file watcher
* invalidates caches for changed paths; a forced reindex passes false.
*/
trustUnchanged?: boolean;
/**
* Normalized paths (see `normKey`) known to have changed since the caches
* were populated. Only consulted when `trustUnchanged` is true.
*/
changedFiles?: ReadonlySet<string>;
} }
export interface FileWalker { export interface FileWalker {
@@ -121,5 +244,15 @@ export interface ParseCache {
export interface ParsedFile { export interface ParsedFile {
file: IndexedFile; file: IndexedFile;
parse: XmlDocument | null; parse: XmlDocument | null;
/**
* Compact index records (assets/defines/includes/xi:include with lines).
* Present for every indexable XML document; null for binary files.
*/
records: IndexRecords | null;
lineMap: LineMap | null; lineMap: LineMap | null;
/**
* True when the file is an art-asset XML that was only registered during
* the XML phase (its shallow scan is deferred to the art phase).
*/
deferredArt?: boolean;
} }
+24
View File
@@ -0,0 +1,24 @@
import type { XmlDocument, XmlElement } from "../language/xmlParser";
/** Lowercases nothing; returns the part after the last ":" in a tag name. */
export function localName(tag: string): string {
const idx = tag.lastIndexOf(":");
return idx >= 0 ? tag.slice(idx + 1) : tag;
}
/**
* Resolves the xpointer subset used by real RA3 mods:
* xmlns(n=uri:ea.com:eala:asset) xpointer(/n:ElementName/child::*)
*
* Returns the container element whose children are selected by the xpointer,
* or null when the form is unsupported / the container is missing.
*/
export function findXPointerContainer(
doc: XmlDocument,
xpointer: string,
): XmlElement | null {
const m = /xpointer\(\/\w+:(\w+)\/child::\*\)/.exec(xpointer);
if (!m) return null;
const name = m[1];
return doc.elements.find((el) => localName(el.name) === name) ?? null;
}
+49 -4
View File
@@ -1,4 +1,5 @@
import type { XmlAttribute, XmlDocument, XmlElement } from "./xmlParser"; import type { XmlAttribute, XmlDocument, XmlElement } from "./xmlParser";
import { elementContainsOffset, parseTag } from "./xmlParser";
export type ContextKind = export type ContextKind =
| "element-name" | "element-name"
@@ -30,18 +31,35 @@ export function analyzeContext(
let container: XmlElement | null = null; let container: XmlElement | null = null;
for (const el of doc.elements) { for (const el of doc.elements) {
if (el.end < 0) continue; if (el.end < 0) continue;
if (offset >= el.start && offset <= el.end) { if (elementContainsOffset(el, offset)) {
if (!container || el.depth > container.depth) container = el; if (!container || el.depth > container.depth) container = el;
} }
} }
if (!container) return empty("none"); if (!container) return empty("none");
// Inside the start tag of the element. // Inside the start tag of the element. The boundary right after `>` is the
if (offset >= container.start && offset <= container.startTagEnd) { // start of the content (e.g. the `$1` cursor in
// `<CreateObject>$1</CreateObject>`), not another attribute slot; only an
// unterminated start tag whose `>` has not been typed yet still belongs to
// the start tag at its recovered end.
const atTagEnd = offset === container.startTagEnd;
const tagClosed =
atTagEnd && container.startTagEnd > container.start &&
text[container.startTagEnd - 1] === ">";
if (offset >= container.start && (offset < container.startTagEnd || (atTagEnd && !tagClosed))) {
return analyzeStartTag(container, text, offset); return analyzeStartTag(container, text, offset);
} }
// A start tag that had to be recovered (e.g. an attribute quote is still
// open) is truncated at the first line break, so attributes typed on later
// lines of the same tag are not part of the parsed element. Re-parse the
// partial tag up to the cursor so value/attribute completion keeps working
// while typing in the malformed tag.
if (container.recoveredStartTag) {
return analyzeRecoveredStartTag(container, text, offset);
}
// Otherwise the cursor is in element content. // Otherwise the cursor is in element content.
return { return {
kind: "content", kind: "content",
@@ -53,6 +71,33 @@ export function analyzeContext(
}; };
} }
/**
* Classifies the cursor inside a recovered (truncated) start tag by re-parsing
* the raw tag content up to the cursor. The original element only kept the
* attributes that fit on its first line; re-parsing the partial text recovers
* attributes typed on later lines without affecting the rest of the document.
*/
function analyzeRecoveredStartTag(
el: XmlElement,
text: string,
offset: number,
): CompletionContext {
const raw = parseTag(text.slice(el.start + 1, offset), el.start + 1);
const partial: XmlElement = {
name: raw.name,
attrs: raw.attrs,
children: [],
parent: el.parent,
start: raw.start,
startTagEnd: offset,
end: offset,
selfClosing: raw.selfClosing,
closeTagStart: -1,
depth: el.depth,
};
return analyzeStartTag(partial, text, offset);
}
function analyzeStartTag( function analyzeStartTag(
el: XmlElement, el: XmlElement,
text: string, text: string,
@@ -83,7 +128,7 @@ function analyzeStartTag(
attr.hasValue && attr.hasValue &&
attr.quoteStart >= 0 && attr.quoteStart >= 0 &&
offset >= attr.quoteStart && offset >= attr.quoteStart &&
(attr.quoteEnd < 0 || offset <= attr.quoteEnd) (attr.quoteEnd < 0 || offset < attr.quoteEnd)
) { ) {
const start = attr.valueStart; const start = attr.valueStart;
const prefix = offset > start ? text.slice(start, offset) : ""; const prefix = offset > start ? text.slice(start, offset) : "";
+99 -3
View File
@@ -43,6 +43,12 @@ export interface XmlElement {
/** Offset of "</" of the closing tag, or -1 when self-closing. */ /** Offset of "</" of the closing tag, or -1 when self-closing. */
closeTagStart: number; closeTagStart: number;
depth: number; depth: number;
/**
* True when the start tag was unterminated and recovered at a line break.
* Attributes typed on later lines of the same tag are not part of the
* parsed element; completion re-parses the partial tag up to the cursor.
*/
recoveredStartTag?: boolean;
} }
export interface XmlParseError { export interface XmlParseError {
@@ -66,6 +72,14 @@ export interface Position {
character: number; character: number;
} }
/**
* Removes a leading UTF-8 byte-order mark (U+FEFF) so source offsets match
* the text as editors expose it (VS Code strips the BOM from document text).
*/
export function stripBom(text: string): string {
return text.charCodeAt(0) === 0xfeff ? text.slice(1) : text;
}
/** Precomputes line start offsets for offset <-> position conversion. */ /** Precomputes line start offsets for offset <-> position conversion. */
export class LineMap { export class LineMap {
private lineStarts: number[] = [0]; private lineStarts: number[] = [0];
@@ -111,7 +125,7 @@ interface RawTag {
const NAME_RE = /[A-Za-z_][\w:.-]*/y; const NAME_RE = /[A-Za-z_][\w:.-]*/y;
function parseTag(content: string, contentStart: number): RawTag { export function parseTag(content: string, contentStart: number): RawTag {
const base = contentStart; const base = contentStart;
let j = 0; let j = 0;
while (j < content.length && /\s/.test(content[j])) { while (j < content.length && /\s/.test(content[j])) {
@@ -349,8 +363,15 @@ export function parseXml(text: string): XmlDocument {
const raw = parseTag(content, i + 1); const raw = parseTag(content, i + 1);
if (raw.name) { if (raw.name) {
const el = buildElement(raw, stack.length); const el = buildElement(raw, stack.length);
el.recoveredStartTag = true;
elements.push(el); elements.push(el);
root = root ?? el; if (stack.length === 0) {
root = root ?? el;
} else {
const parent = stack[stack.length - 1];
parent.children.push(el);
el.parent = parent;
}
stack.push(el); stack.push(el);
} }
i = recoverTo + 1; i = recoverTo + 1;
@@ -399,6 +420,13 @@ function findTagEnd(text: string, from: number): number {
if (c === quote) quote = null; if (c === quote) quote = null;
} else if (c === '"' || c === "'") { } else if (c === '"' || c === "'") {
quote = c; quote = c;
} else if (c === "<") {
// A new tag start before the current tag's ">" means the ">" we would
// find later belongs to that other tag (typically a closing tag after
// a just-typed "<" in element content). Treat the current tag as
// unterminated so the parser recovers at the line break: the context
// stays "content" and the completion range can cover the typed "<".
return -1;
} else if (c === ">") { } else if (c === ">") {
return i; return i;
} }
@@ -427,7 +455,7 @@ export function findElementAt(doc: XmlDocument, offset: number): XmlElement | nu
let best: XmlElement | null = null; let best: XmlElement | null = null;
for (const el of doc.elements) { for (const el of doc.elements) {
if (el.end < 0) continue; if (el.end < 0) continue;
if (offset >= el.start && offset <= el.end) { if (elementContainsOffset(el, offset)) {
if (!best || el.depth > best.depth) { if (!best || el.depth > best.depth) {
best = el; best = el;
} }
@@ -436,6 +464,74 @@ export function findElementAt(doc: XmlDocument, offset: number): XmlElement | nu
return best; return best;
} }
/**
* Whether `offset` belongs to an element's span.
*
* The end offset is exclusive for a completed element (closing tag or
* self-closing tag): a cursor right after `</Name>` belongs to the parent's
* content, not the child. The one exception is an unclosed element whose
* parser-recovered `end` is the document end: a cursor at EOF is still
* inside the element being typed.
*/
export function elementContainsOffset(el: XmlElement, offset: number): boolean {
if (offset < el.start) return false;
if (offset < el.end) return true;
if (offset > el.end) return false;
return !el.selfClosing && el.closeTagStart < 0;
}
export interface TextToken {
value: string;
/** Absolute offset of the first character of the token. */
start: number;
/** Absolute offset one past the last character of the token. */
end: number;
}
/**
* Returns the whitespace-delimited text token inside an element's content
* that contains `offset`, with absolute source offsets. Used for
* simple-content elements (e.g. `<CreateObject>CrateDebris_01</CreateObject>`)
* by completion, hover, navigation and diagnostics. Returns null when the
* offset is not inside text content (start tag, closing tag, self-closing).
*/
export function textContentTokenAt(
text: string,
el: XmlElement,
offset: number,
): TextToken | null {
if (el.selfClosing) return null;
const contentEnd = el.closeTagStart >= 0 ? el.closeTagStart : el.end;
if (contentEnd <= el.startTagEnd) return null;
if (offset <= el.startTagEnd || offset > contentEnd) return null;
const contentStart = el.startTagEnd;
// A cursor right before the closing tag is still inside the content; clamp
// the relative position to the content length in that case.
const rel = Math.min(offset - contentStart, contentEnd - contentStart);
let tokenStart = rel;
while (tokenStart > 0 && !/\s/.test(text[contentStart + tokenStart - 1])) {
tokenStart--;
}
let tokenEnd = rel;
while (
tokenEnd < contentEnd - contentStart &&
!/\s/.test(text[contentStart + tokenEnd])
) {
tokenEnd++;
}
// The cursor may sit on trailing whitespace or at the closing-tag
// boundary; trim whitespace so the token is exactly the value word.
while (tokenEnd > tokenStart && /\s/.test(text[contentStart + tokenEnd - 1])) {
tokenEnd--;
}
if (tokenEnd <= tokenStart) return null;
return {
value: text.slice(contentStart + tokenStart, contentStart + tokenEnd),
start: contentStart + tokenStart,
end: contentStart + tokenEnd,
};
}
/** Finds an element by name that contains the offset (including its start tag). */ /** Finds an element by name that contains the offset (including its start tag). */
export function findOpenTagElementAt(doc: XmlDocument, offset: number): XmlElement | null { export function findOpenTagElementAt(doc: XmlDocument, offset: number): XmlElement | null {
const el = findElementAt(doc, offset); const el = findElementAt(doc, offset);
+5
View File
@@ -92,6 +92,11 @@ export const modelMeta = {
typeCount: Object.keys(model.types).length, typeCount: Object.keys(model.types).length,
}; };
/** All type names in the XSD model (complex + simple), in model order. */
export function allTypeNames(): string[] {
return Object.keys(model.types);
}
export function topLevelElements(): string[] { export function topLevelElements(): string[] {
return model.topLevelElements; return model.topLevelElements;
} }
+521 -21
View File
@@ -1,9 +1,35 @@
import * as vscode from "vscode"; import * as vscode from "vscode";
import { existsSync } from "node:fs"; import { existsSync } from "node:fs";
import { join, dirname } from "node:path"; import { readFile, stat } from "node:fs/promises";
import { CachedDirectoryWalker } from "./indexer/fileScanner"; import { join, dirname, resolve } from "node:path";
import {
CachedDirectoryWalker,
isContentRelevantPath,
isWatcherNoisePath,
} from "./indexer/fileScanner";
import { ModIndexer } from "./indexer/indexer"; import { ModIndexer } from "./indexer/indexer";
import type { ModIndex } from "./indexer/types"; import {
DocumentCache,
IncludeResolveCache,
IndexRecordsCache,
InvalidationsEpoch,
normKey,
} from "./indexer/caches";
import {
DiskRecordsCache,
diskCacheKey,
type DiskCacheIdentity,
type DiskCacheLoadStats,
} from "./indexer/diskCache";
import { buildSearchPaths, type SearchPaths } from "./indexer/includeResolver";
import { extractIndexRecords } from "./indexer/records";
import { LineMap, parseXml, stripBom } from "./language/xmlParser";
import {
buildDocumentScope,
withLocalOverlay,
type DocumentScope,
} from "./indexer/localScope";
import type { ModIndex, ParsedFile } from "./indexer/types";
import { readSettings, type ExtensionSettings } from "./settings"; import { readSettings, type ExtensionSettings } from "./settings";
const REBUILD_DEBOUNCE_MS = 1500; const REBUILD_DEBOUNCE_MS = 1500;
@@ -15,13 +41,60 @@ export class ModWorkspace {
settings: ExtensionSettings; settings: ExtensionSettings;
private walker = new CachedDirectoryWalker(); private walker = new CachedDirectoryWalker();
// Caches owned here survive rebuilds: a fresh ModIndexer reuses them and
// only re-reads files whose stat changed (crucial for the ~2.6 GB of .w3x
// art assets in a project like Corona).
private documentCache = new DocumentCache();
private recordsCache = new IndexRecordsCache();
private resolveCache = new IncludeResolveCache();
/**
* Monotonic invalidation counter. A build captures the epoch when it
* starts; snapshots published after any invalidation are marked stale so
* features can tell users "this index may be slightly out of date" while
* the follow-up rebuild converges.
*/
private epoch = new InvalidationsEpoch();
/** On-disk records cache (cold-start acceleration). */
private diskCachePath: string | null = null;
private diskCache: DiskRecordsCache | null = null;
private diskCacheStats: DiskCacheLoadStats = {
fileExists: false,
keyMatched: false,
loaded: 0,
validated: 0,
dropped: 0,
};
private diskSaved = false;
private saving: Promise<void> | null = null;
/** Document-local scopes (parse + expanded tree + overlay), per open doc. */
private localScopes = new Map<
string,
{ version: number; indexEpoch: number; scope: DocumentScope }
>();
private localScopeBuilds = new Map<string, Promise<DocumentScope>>();
private indexEpochValue = 0;
/** Diagnostics: how many builds ran and what triggered the last one. */
private buildCountValue = 0;
private lastBuildTrigger = "initial";
private pendingTrigger: string | null = null;
private output: vscode.OutputChannel;
/** Called whenever a new index snapshot is published (phase or final). */
onIndexUpdate?: () => void;
private context: vscode.ExtensionContext;
private watchers: vscode.FileSystemWatcher[] = [];
private statusBar: vscode.StatusBarItem; private statusBar: vscode.StatusBarItem;
private rebuildTimer: ReturnType<typeof setTimeout> | null = null; private rebuildTimer: ReturnType<typeof setTimeout> | null = null;
private building = false; private building = false;
private dirty = false; private dirty = false;
constructor(context: vscode.ExtensionContext) { constructor(context: vscode.ExtensionContext) {
this.context = context;
this.settings = readSettings(); this.settings = readSettings();
const storageUri = context.storageUri ?? context.globalStorageUri;
if (storageUri) {
this.diskCachePath = join(storageUri.fsPath, "index-records-v3.json.gz");
}
this.output = vscode.window.createOutputChannel("RA3 Mod XML");
this.statusBar = vscode.window.createStatusBarItem( this.statusBar = vscode.window.createStatusBarItem(
vscode.StatusBarAlignment.Left, vscode.StatusBarAlignment.Left,
100, 100,
@@ -29,12 +102,37 @@ export class ModWorkspace {
this.statusBar.name = "RA3 Mod XML"; this.statusBar.name = "RA3 Mod XML";
this.statusBar.command = "ra3modxml.openIndexReport"; this.statusBar.command = "ra3modxml.openIndexReport";
context.subscriptions.push(this.statusBar); context.subscriptions.push(this.statusBar);
context.subscriptions.push(
vscode.workspace.onDidCloseTextDocument((document) => {
this.localScopes.delete(document.uri.toString());
}),
);
} }
isRa3Workspace(): boolean { isRa3Workspace(): boolean {
return this.projectRoot != null; return this.projectRoot != null;
} }
/** Appends a line to the "RA3 Mod XML" output channel (debug/troubleshooting). */
log(message: string): void {
this.output.appendLine(message);
}
/** True while a rebuild is running (before any snapshot is published). */
get isBuilding(): boolean {
return this.building;
}
/** Number of index builds performed in this session. */
get buildCount(): number {
return this.buildCountValue;
}
/** Why the last build started ("initial", "save", "watcher-*", ...). */
get lastTrigger(): string {
return this.lastBuildTrigger;
}
detectProjectRoot(): string | null { detectProjectRoot(): string | null {
const folders = vscode.workspace.workspaceFolders; const folders = vscode.workspace.workspaceFolders;
if (!folders?.length) return null; if (!folders?.length) return null;
@@ -47,32 +145,134 @@ export class ModWorkspace {
async initialize(): Promise<void> { async initialize(): Promise<void> {
this.projectRoot = this.detectProjectRoot(); this.projectRoot = this.detectProjectRoot();
void vscode.commands.executeCommand(
"setContext",
"ra3modxml.active",
this.projectRoot != null,
);
if (!this.projectRoot) { if (!this.projectRoot) {
this.statusBar.hide(); this.statusBar.hide();
return; return;
} }
this.startWatching();
this.statusBar.text = "$(sync~spin) RA3 XML: indexing…"; this.statusBar.text = "$(sync~spin) RA3 XML: indexing…";
this.statusBar.show(); this.statusBar.show();
await this.rebuild(); await this.rebuild(false, "initial");
} }
scheduleRebuild(): void { /**
* Invalidates cached documents for a path (called by the file watcher and
* on document save), so the next rebuild re-reads it instead of trusting
* the cached copy.
*/
invalidate(path: string): void {
if (!path) return;
this.epoch.mark();
this.documentCache.invalidate(path);
this.recordsCache.invalidate(path);
}
/**
* Called when files are created or deleted: include-resolution results
* (which encode file existence) are no longer trustworthy.
*/
invalidateExistence(): void {
this.epoch.mark();
this.resolveCache.clear();
}
/**
* Watches the project, SDK and extra DATA roots for file changes and
* invalidates the corresponding cache entries. With caches invalidated
* precisely, rebuilds can trust every other cached file and skip per-file
* stats (huge win on mechanical drives; Corona rebuild dropped from ~38s
* to a few seconds).
*/
private startWatching(): void {
if (!this.projectRoot) return; if (!this.projectRoot) return;
const roots = new Set([
this.projectRoot,
this.settings.sdkPath,
...this.settings.additionalDataSearchPaths,
]);
for (const root of roots) {
if (!existsSync(root)) continue;
try {
const watcher = vscode.workspace.createFileSystemWatcher(
new vscode.RelativePattern(root, "**/*"),
);
watcher.onDidCreate((uri) => {
if (isWatcherNoisePath(uri.fsPath)) return;
this.output.appendLine(`[watcher-create] ${uri.fsPath}`);
this.invalidate(uri.fsPath);
this.invalidateExistence();
this.scheduleRebuild("watcher-create");
});
watcher.onDidChange((uri) => {
if (isWatcherNoisePath(uri.fsPath)) return;
// Content changes only matter for files that can change index
// records (XML-ish documents); textures/binary art changes do not.
if (
!isContentRelevantPath(uri.fsPath) &&
!this.isIndexedPath(uri.fsPath)
) {
return;
}
this.output.appendLine(`[watcher-change] ${uri.fsPath}`);
this.invalidate(uri.fsPath);
this.scheduleRebuild("watcher-change");
});
watcher.onDidDelete((uri) => {
if (isWatcherNoisePath(uri.fsPath)) return;
this.output.appendLine(`[watcher-delete] ${uri.fsPath}`);
this.invalidate(uri.fsPath);
this.invalidateExistence();
this.scheduleRebuild("watcher-delete");
});
this.watchers.push(watcher);
this.context.subscriptions.push(watcher);
} catch {
// The root may be temporarily unavailable (e.g. removable drive);
// indexing still works, just without watcher-based invalidation.
}
}
}
/** True when the path is part of the current index (any build state). */
private isIndexedPath(fsPath: string): boolean {
if (this.indexer?.isIndexedFile(fsPath)) return true;
return this.index?.files.has(normKey(fsPath)) ?? false;
}
scheduleRebuild(reason = "unknown"): void {
if (!this.projectRoot) return;
this.pendingTrigger = reason;
if (this.rebuildTimer) clearTimeout(this.rebuildTimer); if (this.rebuildTimer) clearTimeout(this.rebuildTimer);
this.rebuildTimer = setTimeout(() => { this.rebuildTimer = setTimeout(() => {
void this.rebuild(); void this.rebuild(false, this.pendingTrigger ?? reason);
}, REBUILD_DEBOUNCE_MS); }, REBUILD_DEBOUNCE_MS);
} }
async rebuild(): Promise<void> { async rebuild(force = false, trigger = "unknown"): Promise<void> {
if (!this.projectRoot) return; if (!this.projectRoot) return;
if (this.building) { if (this.building) {
this.dirty = true; this.dirty = true;
return; return;
} }
if (force) this.resolveCache.clear();
this.building = true; this.building = true;
this.buildCountValue++;
this.lastBuildTrigger = trigger;
this.output.appendLine(
`[build #${this.buildCountValue}] trigger=${trigger} force=${force} start=${new Date().toISOString()}`,
);
this.settings = readSettings(); this.settings = readSettings();
const epochAtStart = this.epoch.snapshot();
try { try {
// Cold start: seed the records cache from disk (stat-validated) so a
// fresh session does not re-read unchanged files (Corona: 2.6 GB of
// art assets) just because the in-memory caches are empty.
await this.seedRecordsFromDisk();
this.statusBar.text = "$(sync~spin) RA3 XML: indexing…"; this.statusBar.text = "$(sync~spin) RA3 XML: indexing…";
const indexer = new ModIndexer({ const indexer = new ModIndexer({
projectDir: this.projectRoot, projectDir: this.projectRoot,
@@ -81,31 +281,329 @@ export class ModWorkspace {
indexSageXml: this.settings.indexSageXml, indexSageXml: this.settings.indexSageXml,
additionalDataSearchPaths: this.settings.additionalDataSearchPaths, additionalDataSearchPaths: this.settings.additionalDataSearchPaths,
walker: this.walker, walker: this.walker,
documentCache: this.documentCache,
recordsCache: this.recordsCache,
resolveCache: this.resolveCache,
// Trust cache entries unless the user explicitly asked for a full
// verification (ra3modxml.reindex).
trustUnchanged: !force,
}); });
const started = Date.now();
this.index = await indexer.build();
this.indexer = indexer; this.indexer = indexer;
const secs = ((Date.now() - started) / 1000).toFixed(1); // The XML phase is published as soon as it is ready, so completion /
const s = this.index.stats; // navigation / diagnostics work while the art scan continues.
this.statusBar.text = `$(symbol-misc) RA3 XML: ${formatCount(s.assetCount)} assets`; const finalIndex = await indexer.build((phaseIndex) => {
this.statusBar.tooltip = this.publishIndex(phaseIndex, epochAtStart);
`${s.projectDir}\n` + });
`${s.indexedFiles} files indexed (${secs}s)\n` + this.publishIndex(finalIndex, epochAtStart);
`${s.assetCount} assets (${s.manifestAssetCount} from ${s.manifestFiles} manifests)\n` + this.output.appendLine(
`${s.defineCount} defines, ${s.streams} streams, ${s.sourceCandidates} include candidates`; `[build #${this.buildCountValue}] done in ${(finalIndex.stats.elapsedMs / 1000).toFixed(1)}s (phase=${finalIndex.phase}, assets=${finalIndex.stats.assetCount}, stale=${finalIndex.stale === true}, walk=${(finalIndex.stats.walkMs / 1000).toFixed(1)}s, candidates=${(finalIndex.stats.candidatesMs / 1000).toFixed(1)}s, art=${(finalIndex.stats.artScanMs / 1000).toFixed(1)}s)`,
);
this.saveRecordsToDisk();
} catch (err) { } catch (err) {
this.index = null; if (this.index) {
this.statusBar.text = "$(error) RA3 XML: indexing failed"; // Keep the last good snapshot (marked stale) instead of disabling the
this.statusBar.tooltip = err instanceof Error ? err.message : String(err); // extension entirely; a later rebuild can recover.
this.index.stale = true;
this.statusBar.text = "$(error) RA3 XML: indexing failed (stale index kept)";
this.statusBar.tooltip = err instanceof Error ? err.message : String(err);
this.onIndexUpdate?.();
} else {
this.statusBar.text = "$(error) RA3 XML: indexing failed";
this.statusBar.tooltip = err instanceof Error ? err.message : String(err);
}
} finally { } finally {
this.building = false; this.building = false;
if (this.dirty) { if (this.dirty) {
this.dirty = false; this.dirty = false;
void this.rebuild(); void this.rebuild(false, `dirty-followup (${this.lastBuildTrigger})`);
} else if (this.index) {
// Build is fully over: refresh diagnostics with full local scopes
// (the snapshot published while `building` was still true only got
// cheap parse-only scopes).
this.onIndexUpdate?.();
} }
} }
} }
private diskCacheIdentity(): DiskCacheIdentity | null {
if (!this.projectRoot) return null;
return {
projectDir: this.projectRoot,
sdkDir: this.settings.sdkPath,
indexSageXml: this.settings.indexSageXml,
additionalDataSearchPaths: this.settings.additionalDataSearchPaths,
builtmodsDirs: this.settings.builtmodsDirs,
};
}
/** Loads + stat-validates the disk cache into the records cache once. */
private async seedRecordsFromDisk(): Promise<void> {
if (this.recordsCache.size > 0) return;
const identity = this.diskCacheIdentity();
if (!this.diskCachePath || !identity) return;
this.diskCache = new DiskRecordsCache(this.diskCachePath, identity);
this.statusBar.text = "$(sync~spin) RA3 XML: validating cache…";
const { records, stats } = await this.diskCache.loadValidated();
this.diskCacheStats = stats;
this.diskSaved = false;
for (const rec of records) {
this.recordsCache.set(rec.key, {
stat: rec.stat,
records: rec.records,
kind: rec.kind,
contentHash: rec.contentHash,
});
}
}
/** Persists the records cache after a successful build (best-effort). */
private saveRecordsToDisk(): void {
if (!this.diskCache) return;
// Snapshot the entries now: the save runs in the background while the
// next rebuild may already be mutating the live cache.
const entries = [...this.recordsCache.entries()];
const prev = this.saving ?? Promise.resolve();
this.saving = prev
.then(async () => {
await this.diskCache!.save(entries);
this.diskSaved = true;
})
.catch(() => {
// Disk persistence is best-effort; the in-memory cache still works.
});
}
/**
* Clears every cache (in-memory + disk + directory walker) and starts a
* full forced rebuild. Used by the `ra3modxml.clearCache` command.
*/
clearCaches(): void {
this.localScopes.clear();
this.documentCache.clear();
this.recordsCache.clear();
this.resolveCache.clear();
this.walker.clear();
this.diskCacheStats = {
fileExists: false,
keyMatched: false,
loaded: 0,
validated: 0,
dropped: 0,
};
this.diskSaved = false;
void this.diskCache?.clear();
void this.rebuild(true, "clear-cache");
}
/** Human-readable cache status for the `ra3modxml.showCacheReport` command. */
async cacheReport(): Promise<string> {
const lines: string[] = ["RA3 Mod XML cache report"];
lines.push(`Disk cache: ${this.diskCachePath ?? "not available"}`);
if (this.diskCachePath) {
const status = await this.diskCache?.status();
lines.push(
` file: ${status?.exists ? `${(status.sizeBytes / 1024).toFixed(1)} KB` : "missing"}`,
);
const identity = this.diskCacheIdentity();
lines.push(` identity key: ${identity ? diskCacheKey(identity) : "-"}`);
lines.push(
` last load: file=${this.diskCacheStats.fileExists} keyMatched=${this.diskCacheStats.keyMatched} loaded=${this.diskCacheStats.loaded} validated=${this.diskCacheStats.validated} dropped=${this.diskCacheStats.dropped}`,
);
lines.push(` saved after last build: ${this.diskSaved}`);
}
lines.push(
`In-memory: ${this.recordsCache.size} record entries · ${this.documentCache.size} documents (${this.documentCache.elements} elements) · ${this.resolveCache.size} include resolutions`,
);
lines.push(`Builds: #${this.buildCount} (last trigger: ${this.lastBuildTrigger})`);
if (this.index) {
const s = this.index.stats;
lines.push(
`Last build: snapshotHits=${s.snapshotHits} snapshotFallbacks=${s.snapshotFallbacks} recordsCacheHits=${s.recordsCacheHits} shallowCacheHits=${s.shallowCacheHits}`,
);
}
return lines.join("\n");
}
/**
* Publishes an index snapshot (intermediate phase or final). If any file
* was invalidated while the snapshot was being built, it is marked stale;
* the dirty/rebuild mechanism converges shortly after.
*/
private publishIndex(index: ModIndex, epochAtStart: number): void {
if (this.epoch.changedSince(epochAtStart)) index.stale = true;
this.index = index;
this.indexEpochValue++;
// The merged index attached to a document scope changes with every
// published snapshot, so cached scopes are rebuilt lazily on next use.
this.localScopes.clear();
this.updateStatusBar(index);
this.onIndexUpdate?.();
if (!index.complete) {
this.output.appendLine(
`[build #${this.buildCountValue}] phase A published in ${(index.stats.elapsedMs / 1000).toFixed(1)}s (${index.stats.assetCount} assets, ${index.stats.deferredArtFiles} art files pending)`,
);
}
}
private updateStatusBar(idx: ModIndex): void {
const s = idx.stats;
const stale = idx.stale ? " (stale)" : "";
if (!idx.complete) {
this.statusBar.text = `$(sync~spin) RA3 XML: XML indexed, scanning art…${stale}`;
} else {
this.statusBar.text = `$(symbol-misc) RA3 XML: ${formatCount(s.assetCount)} assets${stale}`;
}
this.statusBar.tooltip =
`${s.projectDir}\n` +
`${s.indexedFiles} files indexed (${s.parsedFiles} parsed, ${s.shallowScannedFiles} art assets shallow-scanned, ${(s.elapsedMs / 1000).toFixed(1)}s)\n` +
`${s.assetCount} assets (${s.manifestAssetCount} from ${s.manifestFiles} manifests)\n` +
`${s.referenceCount} reference sites\n` +
`${s.defineCount} defines, ${s.streams} streams, ${s.sourceCandidates} include candidates\n` +
`Phase: ${s.phase} · Complete: ${s.complete}${stale}`;
}
/**
* Search paths derived from the current settings, usable even before the
* first index snapshot exists (include links / hover / diagnostics).
*/
searchPaths(): SearchPaths | null {
if (!this.projectRoot) return null;
return buildSearchPaths(this.settings.sdkPath, this.projectRoot);
}
/**
* Returns the document scope for the current text: original parse, expanded
* logical tree, local overlay and overlay-aware merged index. Cached by
* URI + document version + global index epoch.
*/
async getScope(document: vscode.TextDocument): Promise<DocumentScope> {
const key = document.uri.toString();
const cached = this.localScopes.get(key);
if (
cached &&
cached.version === document.version &&
cached.indexEpoch === this.indexEpochValue
) {
return cached.scope;
}
// While a rebuild is running, avoid competing with the indexer for disk
// I/O: serve a parse-only scope (current file + XSD context, no include
// chain / logical expansion). The published snapshot clears this cache,
// so the next provider call after the build gets the full local scope.
if (this.building) {
return this.buildCheapScope(document);
}
const pending = this.localScopeBuilds.get(key);
if (pending) return pending;
const versionAtStart = document.version;
const promise = this.buildScope(document)
.then((scope) => {
this.localScopes.set(key, {
version: versionAtStart,
indexEpoch: this.indexEpochValue,
scope,
});
return scope;
})
.finally(() => {
this.localScopeBuilds.delete(key);
});
this.localScopeBuilds.set(key, promise);
return promise;
}
/**
* Returns the global index with this document's local overlay attached, or
* a minimal local-only index while the global index is still building.
*/
async getIndex(document: vscode.TextDocument): Promise<ModIndex | null> {
if (!this.isRa3Workspace()) return null;
return (await this.getScope(document)).merged;
}
private async buildScope(
document: vscode.TextDocument,
): Promise<DocumentScope> {
const projectRoot = this.projectRoot;
if (!projectRoot) throw new Error("RA3 workspace root is not available");
const searchPaths =
this.searchPaths() ?? buildSearchPaths(this.settings.sdkPath, projectRoot);
const readRecords = async (path: string): Promise<ParsedFile | null> =>
this.indexer ? this.indexer.readDocument(path) : this.fallbackRead(path);
const readDom = async (path: string): Promise<ParsedFile | null> =>
this.indexer ? this.indexer.readDom(path) : this.fallbackRead(path);
const scope = await buildDocumentScope(
document.uri.fsPath,
document.getText(),
document.version,
{
projectDir: projectRoot,
sdkDir: this.settings.sdkPath,
searchPaths,
readRecords,
readDom,
},
);
scope.merged = withLocalOverlay(
this.index,
scope.overlay,
projectRoot,
this.settings.sdkPath,
);
return scope;
}
private async buildCheapScope(
document: vscode.TextDocument,
): Promise<DocumentScope> {
const projectRoot = this.projectRoot;
if (!projectRoot) throw new Error("RA3 workspace root is not available");
const searchPaths =
this.searchPaths() ?? buildSearchPaths(this.settings.sdkPath, projectRoot);
const scope = await buildDocumentScope(
document.uri.fsPath,
document.getText(),
document.version,
{
projectDir: projectRoot,
sdkDir: this.settings.sdkPath,
searchPaths,
readRecords: async () => null,
readDom: async () => null,
},
);
scope.merged = withLocalOverlay(
this.index,
scope.overlay,
projectRoot,
this.settings.sdkPath,
);
return scope;
}
/**
* Fallback used before the first ModIndexer exists (e.g. during initial
* activation): parses an XML file directly so the document-local scope can
* still follow small include chains.
*/
private async fallbackRead(path: string): Promise<ParsedFile | null> {
try {
const st = await stat(path);
if (st.size > 4 * 1024 * 1024) return null;
const text = stripBom(await readFile(path, "utf8"));
const lineMap = new LineMap(text);
const parse = parseXml(text);
return {
file: { path: resolve(path), stat: null },
parse,
records: extractIndexRecords(parse, lineMap, text),
lineMap,
};
} catch {
return null;
}
}
/** Parses the (possibly unsaved) in-memory text of the active document. */ /** Parses the (possibly unsaved) in-memory text of the active document. */
async parseText(path: string, text: string) { async parseText(path: string, text: string) {
const { parseXml, LineMap } = await import("./language/xmlParser"); const { parseXml, LineMap } = await import("./language/xmlParser");
@@ -113,6 +611,7 @@ export class ModWorkspace {
return { return {
file: { path, stat: null }, file: { path, stat: null },
parse, parse,
records: null,
lineMap: new LineMap(text), lineMap: new LineMap(text),
}; };
} }
@@ -120,6 +619,7 @@ export class ModWorkspace {
dispose(): void { dispose(): void {
if (this.rebuildTimer) clearTimeout(this.rebuildTimer); if (this.rebuildTimer) clearTimeout(this.rebuildTimer);
this.statusBar.dispose(); this.statusBar.dispose();
this.output.dispose();
} }
} }
+103
View File
@@ -0,0 +1,103 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import {
DocumentCache,
IncludeResolveCache,
IndexRecordsCache,
InvalidationsEpoch,
} from "../out/indexer/caches.js";
import { resolve } from "node:path";
const stat = { mtimeMs: 1, size: 1, birthtimeMs: 1, ctimeMs: 1 };
function parsed(path, elements) {
return {
file: { path, stat },
parse: { root: { name: "r" }, elements: new Array(elements), errors: [] },
records: null,
lineMap: null,
};
}
test("DocumentCache evicts the largest tree when over the element budget", () => {
const cache = new DocumentCache(64, 100);
cache.set(parsed("a.xml", 60));
cache.set(parsed("b.xml", 60));
assert.equal(cache.get("a.xml"), undefined, "largest tree evicted first");
assert.ok(cache.get("b.xml"));
});
test("DocumentCache evicts least recently used when over capacity", () => {
const cache = new DocumentCache(2, 1_000_000);
cache.set(parsed("a.xml", 10));
cache.set(parsed("b.xml", 10));
cache.set(parsed("c.xml", 10));
assert.equal(cache.get("a.xml"), undefined);
assert.ok(cache.get("b.xml"));
assert.ok(cache.get("c.xml"));
});
test("DocumentCache invalidate frees budget", () => {
const cache = new DocumentCache(64, 100);
cache.set(parsed("a.xml", 60));
cache.invalidate("a.xml");
cache.set(parsed("b.xml", 60));
assert.ok(cache.get("b.xml"), "invalidating a freed its budget share");
cache.set(parsed("c.xml", 60));
assert.equal(cache.get("a.xml"), undefined);
assert.ok(cache.get("c.xml"));
});
test("IndexRecordsCache stores and invalidates entries", () => {
const cache = new IndexRecordsCache();
const entry = {
stat,
records: { assets: [], defines: [], includes: [], rootXiIncludes: [], nestedXiIncludes: [], references: [] },
kind: "full",
};
cache.set("a.xml", entry);
assert.equal(cache.get("a.xml"), entry);
cache.invalidate("a.xml");
assert.equal(cache.get("a.xml"), undefined);
});
test("IndexRecordsCache exposes entries for disk persistence", () => {
const cache = new IndexRecordsCache();
const entry = {
stat,
records: { assets: [], defines: [], includes: [], rootXiIncludes: [], nestedXiIncludes: [], references: [] },
kind: "full",
};
cache.set("a.xml", entry);
const list = [...cache.entries()];
assert.equal(list.length, 1);
assert.equal(list[0][0], resolve("a.xml").toLowerCase());
assert.equal(list[0][1], entry);
});
test("IncludeResolveCache stores sources and manifest lookups", () => {
const cache = new IncludeResolveCache();
const key = "dir|DATA:static.xml";
const result = { path: "C:/sdk/static.xml", prefix: "DATA", raw: "DATA:static.xml" };
assert.equal(cache.get(key), undefined);
cache.set(key, result);
assert.equal(cache.get(key), result);
assert.equal(cache.getManifest("static.xml"), undefined);
cache.setManifest("static.xml", "C:/sdk/builtmods/static.manifest");
assert.equal(cache.getManifest("static.xml"), "C:/sdk/builtmods/static.manifest");
cache.clear();
assert.equal(cache.get(key), undefined);
assert.equal(cache.getManifest("static.xml"), undefined);
});
test("InvalidationsEpoch tracks changes since a snapshot", () => {
const epoch = new InvalidationsEpoch();
const before = epoch.snapshot();
assert.equal(epoch.changedSince(before), false);
epoch.mark();
assert.equal(epoch.changedSince(before), true);
assert.equal(epoch.changedSince(epoch.snapshot()), false);
epoch.mark();
epoch.mark();
assert.equal(epoch.current, 3);
});
+210
View File
@@ -0,0 +1,210 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { createRequire } from "node:module";
import { fileURLToPath } from "node:url";
import { dirname, join, resolve } from "node:path";
// Minimal vscode shim: the CodeLens provider only constructs CodeLens/Range.
class Range {
constructor(start, end) {
this.start = start;
this.end = end;
}
}
class CodeLens {
constructor(range, command) {
this.range = range;
this.command = command;
}
}
const require = createRequire(import.meta.url);
const Module = require("module");
const origResolve = Module._resolveFilename;
Module._resolveFilename = function (request, ...args) {
if (request === "vscode") return "vscode-stub";
return origResolve.call(this, request, ...args);
};
require.cache["vscode-stub"] = {
id: "vscode-stub",
filename: "vscode-stub",
loaded: true,
exports: {
Range,
CodeLens,
},
};
const { Ra3CodeLensProvider } = require("../out/features/codeLens.js");
const { assetDefKey } = require("../out/indexer/referenceIndex.js");
const { normKey } = require("../out/indexer/caches.js");
const root = dirname(dirname(fileURLToPath(import.meta.url)));
const PROJECT = join(root, "test", "fixtures", "minimod");
const SDK = join(root, "test", "fixtures", "fakesdk");
// A real file whose path matches what DATA:Includes/Units.xml resolves to.
const FILE = resolve(join(PROJECT, "Data", "Includes", "Units.xml"));
const TEXT = `<AssetDeclaration>
<GameObject id="TestTank"/>
<GameObject id="BaseVehicle"/>
<CameraSettings id="S"/>
</AssetDeclaration>`;
function makeDocument(text = TEXT) {
const lineStarts = [0];
for (let i = 0; i < text.length; i++) {
if (text.charCodeAt(i) === 10) lineStarts.push(i + 1);
}
return {
uri: { fsPath: FILE },
getText: () => text,
positionAt: (offset) => {
let lo = 0;
let hi = lineStarts.length - 1;
while (lo < hi) {
const mid = (lo + hi + 1) >> 1;
if (lineStarts[mid] <= offset) lo = mid;
else hi = mid - 1;
}
return { line: lo, character: offset - lineStarts[lo] };
},
};
}
function makeIndex() {
const tankSite = {
file: "C:/mod/Data/Other.xml",
line: 7,
start: 40,
end: 48,
kind: "content",
};
const secondSite = {
file: "C:/mod/Data/Third.xml",
line: 2,
start: 12,
end: 20,
kind: "attr",
};
const references = new Map();
references.set(
assetDefKey({
type: "GameObject",
id: "TestTank",
file: FILE,
line: 2,
}),
[tankSite, secondSite],
);
references.set(
assetDefKey({
type: "GameObject",
id: "BaseVehicle",
file: FILE,
line: 3,
}),
[],
);
return {
references,
assets: new Map(),
sdkDir: SDK,
projectDir: PROJECT,
};
}
test("CodeLens shows counts on reference-target types only, including zero", () => {
const provider = new Ra3CodeLensProvider({
isRa3Workspace: () => true,
index: makeIndex(),
});
const lenses = provider.provideCodeLenses(makeDocument(), {});
assert.equal(lenses.length, 2, "no lens for auto-registered CameraSettings");
const tank = lenses.find((l) => l.command.arguments[0].id === "TestTank");
const base = lenses.find((l) => l.command.arguments[0].id === "BaseVehicle");
assert.ok(tank, "GameObject TestTank gets a lens");
assert.equal(tank.command.title, "2 references");
assert.equal(tank.command.command, "ra3modxml.showReferences");
assert.equal(tank.command.arguments[0].type, "GameObject");
assert.equal(tank.command.arguments[0].line, 2);
assert.ok(base, "zero is still displayed for reference-target types");
assert.equal(base.command.title, "0 references");
// Lenses anchor on the element start tag.
assert.equal(tank.range.start.line, 1);
assert.ok(tank.range.start.character < tank.range.end.character);
});
test("CodeLens returns nothing without a workspace or index", () => {
const noWorkspace = new Ra3CodeLensProvider({
isRa3Workspace: () => false,
index: makeIndex(),
});
assert.deepEqual(noWorkspace.provideCodeLenses(makeDocument(), {}), []);
const noIndex = new Ra3CodeLensProvider({
isRa3Workspace: () => true,
index: null,
});
assert.deepEqual(noIndex.provideCodeLenses(makeDocument(), {}), []);
});
test("CodeLens counts references attached to a manifest definition with the same SageXml source", () => {
const manifestDef = {
type: "GameObject",
id: "TestTank",
file: resolve(join(SDK, "builtmods", "static.manifest")),
line: 0,
origin: "manifest",
manifestSource: "DATA:Includes/Units.xml",
};
const site = {
file: "C:/mod/Data/Other.xml",
line: 7,
start: 40,
end: 48,
kind: "content",
};
const references = new Map();
references.set(assetDefKey(manifestDef), [site]);
const idx = {
references,
assets: new Map([
["GameObject", new Map([["testtank", [manifestDef]]])],
]),
sdkDir: SDK,
projectDir: PROJECT,
};
const provider = new Ra3CodeLensProvider({
isRa3Workspace: () => true,
index: idx,
});
const lenses = provider.provideCodeLenses(makeDocument(), {});
const tank = lenses.find((l) => l.command.arguments[0].id === "TestTank");
assert.ok(tank, "lens is shown for the SageXml-backed definition");
assert.equal(tank.command.title, "1 reference");
});
test("CodeLens schedules a targeted rebuild when the open document desyncs from the snapshot", () => {
const idx = makeIndex();
idx.recordsHashes = new Map([[normKey(FILE), "stale-hash"]]);
const calls = [];
const provider = new Ra3CodeLensProvider({
isRa3Workspace: () => true,
index: idx,
invalidate: (p) => calls.push(["invalidate", p]),
scheduleRebuild: (r) => calls.push(["schedule", r]),
});
provider.provideCodeLenses(makeDocument(), {});
assert.ok(
calls.some(([kind]) => kind === "invalidate"),
"the stale file is invalidated",
);
assert.ok(
calls.some(([kind, reason]) => kind === "schedule" && reason === "records-desync"),
"a targeted rebuild is scheduled",
);
});
+707 -3
View File
@@ -20,6 +20,13 @@ class CompletionItem {
} }
} }
class CompletionList {
constructor(items, isIncomplete) {
this.items = items;
this.isIncomplete = isIncomplete;
}
}
class Position { class Position {
constructor(line, character) { constructor(line, character) {
this.line = line; this.line = line;
@@ -67,6 +74,7 @@ require.cache["vscode-stub"] = {
loaded: true, loaded: true,
exports: { exports: {
CompletionItem, CompletionItem,
CompletionList,
CompletionItemKind, CompletionItemKind,
Position, Position,
Range, Range,
@@ -76,6 +84,8 @@ require.cache["vscode-stub"] = {
}; };
const { Ra3CompletionProvider } = require("../out/features/completion.js"); const { Ra3CompletionProvider } = require("../out/features/completion.js");
const { parseXml, LineMap } = require("../out/language/xmlParser.js");
const { expandDocument } = require("../out/indexer/logicalTree.js");
function makeDocument(text) { function makeDocument(text) {
const lineStarts = [0]; const lineStarts = [0];
@@ -98,11 +108,33 @@ function makeDocument(text) {
}; };
} }
// Enum completions do not consult the index, but provideCompletionItems only async function makeScope(text, idx) {
// routes value contexts when the workspace index is present. const lineMap = new LineMap(text);
const provider = new Ra3CompletionProvider({ index: {} }); const parse = parseXml(text);
const expanded = await expandDocument("test.xml", parse, {
resolve: () => null,
readDom: async () => null,
});
return { expanded, merged: idx, overlay: {} };
}
// Enum completions do not consult the index, and value contexts now work
// even before the workspace index exists.
const makeProvider = (idx) =>
new Ra3CompletionProvider({
index: idx,
isRa3Workspace: () => true,
getScope: async (document) => makeScope(document.getText(), idx),
log: () => {},
});
const provider = makeProvider({});
const providerNoIndex = makeProvider(null);
const token = { isCancellationRequested: false }; const token = { isCancellationRequested: false };
function listItems(result) {
return Array.isArray(result) ? result : result.items;
}
test("Surfaces enum completion works with an unclosed quote", async () => { test("Surfaces enum completion works with an unclosed quote", async () => {
const text = const text =
`<AssetDeclaration>\n <LocomotorTemplate id="x" Surfaces="G>\n <Other/>\n</LocomotorTemplate>\n</AssetDeclaration>`; `<AssetDeclaration>\n <LocomotorTemplate id="x" Surfaces="G>\n <Other/>\n</LocomotorTemplate>\n</AssetDeclaration>`;
@@ -142,6 +174,95 @@ test("list values filter on the token after whitespace", async () => {
assert.equal(water.range.start.character, valueStart + "GROUND ".length - line1Start); assert.equal(water.range.start.character, valueStart + "GROUND ".length - line1Start);
}); });
test("list completion after a space offers only unused flags", async () => {
const text =
`<AssetDeclaration>\n <LocomotorTemplate id="x" Surfaces="GROUND ">\n <Other/>\n</LocomotorTemplate>\n</AssetDeclaration>`;
const line1 = text.split("\n")[1];
const pos = new Position(1, line1.indexOf("GROUND ") + "GROUND ".length);
const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
const labels = items.map((i) => i.label);
assert.equal(items.length, 10);
assert.ok(labels.includes("WATER"));
assert.ok(!labels.includes("GROUND"));
// The replacement range is empty at the cursor: existing flags are kept.
const water = items.find((i) => i.label === "WATER");
assert.equal(water.range.start.character, pos.character);
assert.equal(water.range.end.character, pos.character);
});
test("inserting a flag in the middle of a list does not delete trailing flags", async () => {
const text =
`<AssetDeclaration>\n <LocomotorTemplate id="x" Surfaces="GROUND WATER">\n <Other/>\n</LocomotorTemplate>\n</AssetDeclaration>`;
const line1 = text.split("\n")[1];
// Cursor right after the space between GROUND and WATER.
const pos = new Position(1, line1.indexOf("GROUND ") + "GROUND ".length);
const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
const labels = items.map((i) => i.label);
assert.ok(labels.includes("WATER"));
assert.ok(!labels.includes("GROUND"));
const water = items.find((i) => i.label === "WATER");
assert.equal(water.insertText, "WATER");
// The range must end at the cursor, not at the end of the whole value
// (which would replace the trailing "WATER" when accepting a suggestion).
assert.equal(water.range.start.character, pos.character);
assert.equal(water.range.end.character, pos.character);
});
test("complete flag at the end of a closed value appends the remaining flags", async () => {
const text =
`<AssetDeclaration>\n <LocomotorTemplate id="x" Surfaces="GROUND">\n <Other/>\n</LocomotorTemplate>\n</AssetDeclaration>`;
const line1 = text.split("\n")[1];
const pos = new Position(1, line1.indexOf("GROUND") + "GROUND".length);
const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
const labels = items.map((i) => i.label);
assert.equal(items.length, 10);
assert.ok(!labels.includes("GROUND"));
const water = items.find((i) => i.label === "WATER");
assert.ok(water, "remaining flags are offered at the end of a complete flag");
assert.equal(water.insertText, " WATER");
assert.equal(water.range.start.character, pos.character);
assert.equal(water.range.end.character, pos.character);
});
test("prefix-extended flags keep prefix filtering instead of append mode", async () => {
const text =
`<AssetDeclaration>\n <ObjectFilter Include="CAN_ATTACK">\n <Other/>\n</ObjectFilter>\n</AssetDeclaration>`;
const line1 = text.split("\n")[1];
const pos = new Position(1, line1.indexOf("CAN_ATTACK") + "CAN_ATTACK".length);
const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
const labels = items.map((i) => i.label);
assert.ok(labels.includes("CAN_ATTACK_WALLS"));
assert.ok(labels.includes("CAN_ATTACK_STEALTHED"));
assert.ok(!labels.includes("WATER"));
const walls = items.find((i) => i.label === "CAN_ATTACK_WALLS");
assert.equal(walls.insertText, "CAN_ATTACK_WALLS");
// The range replaces the typed token instead of inserting after it.
assert.equal(walls.range.start.character, pos.character - "CAN_ATTACK".length);
assert.equal(walls.range.end.character, pos.character);
});
test("multi-line unterminated Disposition value offers remaining flags", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject\n` +
` Options="IGNORE_ALL_OBJECTS"\n` +
` Disposition="RANDOM_FORCE `;
const line4 = text.split("\n")[4];
const pos = new Position(4, line4.length);
const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
const labels = items.map((i) => i.label);
assert.ok(labels.includes("DISPOSITION_NONE"));
assert.ok(labels.includes("FLOATING"));
assert.ok(!labels.includes("RANDOM_FORCE"));
});
test("empty unterminated value offers all enum values", async () => { test("empty unterminated value offers all enum values", async () => {
const text = const text =
`<AssetDeclaration>\n <LocomotorTemplate id="x" Surfaces=">\n <Other/>\n</LocomotorTemplate>\n</AssetDeclaration>`; `<AssetDeclaration>\n <LocomotorTemplate id="x" Surfaces=">\n <Other/>\n</LocomotorTemplate>\n</AssetDeclaration>`;
@@ -155,3 +276,586 @@ test("empty unterminated value offers all enum values", async () => {
assert.ok(labels.includes("WATER")); assert.ok(labels.includes("WATER"));
assert.ok(labels.includes("CRUSHABLE_WALL")); assert.ok(labels.includes("CRUSHABLE_WALL"));
}); });
test("enum completions work without an index", async () => {
const text =
`<AssetDeclaration>\n <LocomotorTemplate id="x" Surfaces="G>\n <Other/>\n</LocomotorTemplate>\n</AssetDeclaration>`;
const line1 = text.split("\n")[1];
const pos = new Position(1, line1.indexOf("G") + 1);
const items = await providerNoIndex.provideCompletionItems(
makeDocument(text),
pos,
token,
);
const labels = items.map((i) => i.label);
assert.ok(labels.includes("GROUND"), "enum value offered without an index");
assert.ok(!labels.includes("WATER"));
});
test("element and attribute name completions work without an index", async () => {
const text = `<AssetDeclaration>\n <LocomotorTemplate `;
const line1 = text.split("\n")[1];
const pos = new Position(1, line1.length);
const items = await providerNoIndex.provideCompletionItems(
makeDocument(text),
pos,
token,
);
const labels = items.map((i) => i.label);
assert.ok(labels.includes("id"));
assert.ok(labels.includes("Surfaces"));
});
test("attribute completion after a closed quote inserts a space", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject Options="IGNORE_ALL_OBJECTS" Disposition="RANDOM_FORCE RELATIVE_ANGLE">`;
const line2 = text.split("\n")[2];
const pos = new Position(
2,
line2.indexOf('RELATIVE_ANGLE"') + 'RELATIVE_ANGLE"'.length,
);
const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
const count = items.find((i) => i.label === "Count");
assert.ok(count, "Count is a valid next attribute");
assert.equal(count.insertText.value, ' Count="1"');
// The replacement range is empty at the cursor; the space comes from the
// insert text so the attribute never glues to the closing quote.
assert.equal(count.range.start.character, pos.character);
assert.equal(count.range.end.character, pos.character);
});
test("attribute completion on one-per-line elements adds a newline", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject\n` +
` Options="IGNORE_ALL_OBJECTS"\n` +
` Disposition="RANDOM_FORCE RELATIVE_ANGLE">`;
const line4 = text.split("\n")[4];
const pos = new Position(
4,
line4.indexOf('RELATIVE_ANGLE"') + 'RELATIVE_ANGLE"'.length,
);
const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
const count = items.find((i) => i.label === "Count");
assert.ok(count);
// The editor adds the new line's base indentation itself; embedding our
// own indent here would be added on top of it and compound line by line.
assert.equal(count.insertText.value, '\nCount="1"');
assert.equal(count.range.start.character, pos.character);
assert.equal(count.range.end.character, pos.character);
});
test("attribute completion on a new line aligns with the neighbor indent", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject\n` +
` Options="IGNORE_ALL_OBJECTS"\n` +
` Disposition="RANDOM_FORCE RELATIVE_ANGLE"\n` +
` `;
const pos = new Position(5, 6);
const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
const count = items.find((i) => i.label === "Count");
assert.ok(count);
assert.equal(count.insertText.value, ' Count="1"');
// The range replaces the whitespace already typed on the new line.
assert.equal(count.range.start.character, 0);
assert.equal(count.range.end.character, 6);
});
test("a half-typed attribute on a new line does not drive the indent", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject\n` +
` Options="IGNORE_ALL_OBJECTS"\n` +
` Disposition="RANDOM_FORCE RELATIVE_ANGLE"\n` +
` Count="1"\n` +
` C`;
// The "C" line carries a large editor auto-indent (20 spaces) that must
// NOT become the anchor for the completed attribute.
const pos = new Position(6, 21);
const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
const createFX = items.find((i) => i.label === "CreateFX");
assert.ok(createFX);
assert.equal(createFX.insertText.value, ' CreateFX="$1"');
// The range covers the auto-indented whitespace and the typed "C".
assert.equal(createFX.range.start.character, 0);
assert.equal(createFX.range.end.character, 21);
});
test("whitespace used to trigger the popup is consumed on newline insert", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject\n` +
` Options="IGNORE_ALL_OBJECTS"\n` +
` Disposition="RANDOM_FORCE RELATIVE_ANGLE" >`;
const line4 = text.split("\n")[4];
const pos = new Position(
4,
line4.indexOf('RELATIVE_ANGLE" ') + 'RELATIVE_ANGLE" '.length,
);
const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
const count = items.find((i) => i.label === "Count");
assert.ok(count);
assert.equal(count.insertText.value, '\nCount="1"');
// The range starts at the previous attribute's closing quote, so the
// typed space is replaced by the newline instead of lingering.
assert.equal(count.range.start.character, pos.character - 1);
assert.equal(count.range.end.character, pos.character);
});
test("scalar attributes get typed default values, suggestion attributes keep $1", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject `;
const pos = new Position(2, text.split("\n")[2].length);
const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
const item = (label) => items.find((i) => i.label === label);
// XSD default wins.
assert.equal(item("Count").insertText.value, 'Count="1"');
// Type-based examples for scalars without an XSD default.
assert.equal(item("FadeTime").insertText.value, 'FadeTime="0s"');
assert.equal(item("DispositionAngle").insertText.value, 'DispositionAngle="0d"');
// Suggestion-driven values keep the $1 placeholder and re-trigger suggest.
assert.equal(item("CreateFX").insertText.value, 'CreateFX="$1"');
assert.ok(item("CreateFX").command, "reference values re-trigger suggest");
assert.equal(item("Options").insertText.value, 'Options="$1"');
assert.ok(item("Options").command, "list values re-trigger suggest");
assert.equal(item("DisabledWhileBusy").insertText.value, 'DisabledWhileBusy="$1"');
assert.ok(item("DisabledWhileBusy").command, "boolean values re-trigger suggest");
// Concrete defaults do not pop an empty suggest widget.
assert.equal(item("Count").command, undefined);
});
test("content (child element) completions work without an index", async () => {
const text =
`<AssetDeclaration>\n <LocomotorTemplate id="x">\n \n </LocomotorTemplate>\n</AssetDeclaration>`;
const pos = new Position(2, 4);
const items = await providerNoIndex.provideCompletionItems(
makeDocument(text),
pos,
token,
);
const labels = items.map((i) => i.label);
assert.ok(labels.length > 0, "child elements offered without an index");
});
test("Poid attributes offer ids from the enclosing GameObject's local scope", async () => {
const text =
`<AssetDeclaration>\n` +
` <GameObject id="Tank">\n` +
` <Draws>\n` +
` <TruckDraw id="ModuleTag_Draw" />\n` +
` </Draws>\n` +
` <BehaviorModules>\n` +
` <ReconstituteStateSpecialAbility UpdateModuleId="ModuleTag_D">\n` +
` </BehaviorModules>\n` +
` </GameObject>\n` +
`</AssetDeclaration>`;
const line6 = text.split("\n")[6];
const pos = new Position(6, line6.indexOf("ModuleTag_D") + "ModuleTag_D".length);
const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
const labels = items.map((i) => i.label);
assert.ok(labels.includes("ModuleTag_Draw"));
assert.ok(items.every((i) => i.kind === CompletionItemKind.Value));
assert.ok(
items.every((i) => i.detail === "local module"),
"Poid completions are labelled as local-scope ids",
);
});
test("accepting a child element after a typed < replaces the < instead of doubling it", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject\n` +
` Options="IGNORE_ALL_OBJECTS"\n` +
` Disposition="RANDOM_FORCE RELATIVE_ANGLE ABSOLUTE_ANGLE"\n` +
` MinForceMagnitude="2.0"\n` +
` MaxForceMagnitude="7.0"\n` +
` DispositionIntensity="5.0"\n` +
` MinLifetime="1.0s"\n` +
` MaxLifetime="3.s"\n` +
` MinForcePitch="90d"\n` +
` MaxForcePitch="75d">\n` +
` <Offset x="26.13" y="4.87" z="15.99"></Offset>\n` +
` <`;
const lines = text.split("\n");
const last = lines.length - 1;
const pos = new Position(last, lines[last].length);
const document = makeDocument(text);
const items = await providerNoIndex.provideCompletionItems(document, pos, token);
const createObject = items.find((i) => i.label === "CreateObject");
const offset = items.find((i) => i.label === "Offset");
assert.ok(createObject);
assert.ok(offset);
// The typed "<" stays in the document; the range covers only the name
// area after it (empty here), and the snippet has no leading "<", so the
// final document never contains "<<" and the filter prefix is not "<".
const ltOffset = document.offsetAt(createObject.range.start);
assert.equal(ltOffset, document.offsetAt(pos));
assert.equal(createObject.range.end.line, last);
assert.equal(createObject.range.end.character, lines[last].length);
assert.equal(offset.range.start.character, createObject.range.start.character);
// Simple-content children must be open/close pairs with a value
// placeholder, never a self-closing tag, and re-trigger value suggest.
assert.equal(createObject.insertText.value, "CreateObject>$1</CreateObject>");
assert.ok(createObject.command, "simple-content child re-triggers suggest");
const applied =
text.slice(0, document.offsetAt(createObject.range.start)) +
createObject.insertText.value +
text.slice(document.offsetAt(createObject.range.end));
assert.ok(!applied.includes("<<"), "no doubled angle bracket after accepting");
assert.match(
applied.split("\n")[last],
/<CreateObject>\$1<\/CreateObject>/,
);
});
test("no << when a closing tag follows the typed < (real file shape)", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject\n` +
` Options="IGNORE_ALL_OBJECTS"\n` +
` Disposition="RANDOM_FORCE RELATIVE_ANGLE ABSOLUTE_ANGLE">\n` +
` <Offset x="26.13" y="4.87" z="15.99"></Offset>\n` +
` <\n` +
` </CreateObject>\n` +
` </ObjectCreationList>\n` +
`</AssetDeclaration>`;
const lines = text.split("\n");
const last = 6;
const pos = new Position(last, lines[last].length);
const document = makeDocument(text);
const items = await providerNoIndex.provideCompletionItems(document, pos, token);
const createObject = items.find((i) => i.label === "CreateObject");
assert.ok(createObject, "child element still offered after a lone <");
assert.equal(document.offsetAt(createObject.range.start), document.offsetAt(pos));
assert.equal(createObject.insertText.value, "CreateObject>$1</CreateObject>");
const applied =
text.slice(0, document.offsetAt(createObject.range.start)) +
createObject.insertText.value +
text.slice(document.offsetAt(createObject.range.end));
assert.ok(!applied.includes("<<"), "no doubled angle bracket");
const appliedLines = applied.split("\n");
assert.match(appliedLines[6], /<CreateObject>\$1<\/CreateObject>/);
});
test("no << when a partial child name was typed before the closing tag", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject>\n` +
` <Cr\n` +
` </CreateObject>\n` +
` </ObjectCreationList>\n` +
`</AssetDeclaration>`;
const lines = text.split("\n");
const last = 3;
const pos = new Position(last, lines[last].length);
const document = makeDocument(text);
const items = await providerNoIndex.provideCompletionItems(document, pos, token);
const createObject = items.find((i) => i.label === "CreateObject");
assert.ok(createObject);
// The "<" stays; the range covers only the typed partial name "Cr", and
// the snippet has no leading "<".
const startOffset = document.offsetAt(createObject.range.start);
assert.equal(text[startOffset], "C");
assert.equal(createObject.insertText.value, "CreateObject>$1</CreateObject>");
const applied =
text.slice(0, startOffset) +
createObject.insertText.value +
text.slice(document.offsetAt(createObject.range.end));
assert.ok(!applied.includes("<<"), "no doubled angle bracket for partial names");
assert.match(applied.split("\n")[last], /<CreateObject>\$1<\/CreateObject>/);
});
test("content completion without a typed < inserts the full tag", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject>\n` +
` \n` +
` </CreateObject>\n` +
` </ObjectCreationList>\n` +
`</AssetDeclaration>`;
const pos = new Position(3, 6);
const items = await providerNoIndex.provideCompletionItems(
makeDocument(text),
pos,
token,
);
const createObject = items.find((i) => i.label === "CreateObject");
assert.ok(createObject);
// No "<" was typed: the snippet includes the opening bracket and the
// replacement range is empty at the cursor.
assert.equal(createObject.insertText.value, "<CreateObject>$1</CreateObject>");
assert.equal(createObject.range.start.character, 6);
assert.equal(createObject.range.end.character, 6);
});
test("simple-content element offers typed asset ids as the text value", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject>\n` +
` <CreateObject>C</CreateObject>\n` +
` </CreateObject>\n` +
` </ObjectCreationList>\n` +
`</AssetDeclaration>`;
const line = text.split("\n")[3];
const pos = new Position(3, line.indexOf(">C") + 2);
const go = {
type: "GameObject",
id: "CrateDebris_01",
file: "Crates.xml",
line: 1,
origin: "project",
};
const weapon = {
type: "WeaponTemplate",
id: "CrateWeapon_01",
file: "Weapons.xml",
line: 1,
origin: "project",
};
const idx = {
assets: new Map([
["GameObject", new Map([["cratedebris_01", [go]]])],
["WeaponTemplate", new Map([["crateweapon_01", [weapon]]])],
]),
assetsById: new Map([
["cratedebris_01", [go]],
["crateweapon_01", [weapon]],
]),
};
const items = await makeProvider(idx).provideCompletionItems(
makeDocument(text),
pos,
token,
);
const labels = items.map((i) => i.label);
assert.ok(labels.includes("CrateDebris_01"));
assert.ok(
!labels.includes("CrateWeapon_01"),
"content refs are filtered by the element's refType (GameObject)",
);
const item = items.find((i) => i.label === "CrateDebris_01");
assert.equal(item.range.start.character, line.indexOf(">C") + 1);
assert.equal(item.range.end.character, pos.character);
assert.equal(item.insertText, "CrateDebris_01");
});
test("simple-content value completion works before the closing tag is typed", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject>\n` +
` <CreateObject>C`;
const line = text.split("\n")[3];
const pos = new Position(3, line.length);
const go = {
type: "GameObject",
id: "CrateDebris_01",
file: "Crates.xml",
line: 1,
origin: "project",
};
const idx = {
assets: new Map([["GameObject", new Map([["cratedebris_01", [go]]])]]),
assetsById: new Map([["cratedebris_01", [go]]]),
};
const items = await makeProvider(idx).provideCompletionItems(
makeDocument(text),
pos,
token,
);
const labels = items.map((i) => i.label);
assert.ok(labels.includes("CrateDebris_01"));
const item = items.find((i) => i.label === "CrateDebris_01");
// The unclosed element's end is the document end, so the typed "C" is
// still a real token and the range covers it.
assert.equal(item.range.start.character, line.indexOf(">C") + 1);
assert.equal(item.range.end.character, pos.character);
});
test("content start after accepting a simple-content snippet offers values, not attributes", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject>\n` +
` <CreateObject></CreateObject>\n` +
` </CreateObject>\n` +
` </ObjectCreationList>\n` +
`</AssetDeclaration>`;
const line = text.split("\n")[3];
const pos = new Position(3, line.indexOf(">") + 1);
const go = {
type: "GameObject",
id: "CrateDebris_01",
file: "Crates.xml",
line: 1,
origin: "project",
};
const idx = {
assets: new Map([["GameObject", new Map([["cratedebris_01", [go]]])]]),
assetsById: new Map([["cratedebris_01", [go]]]),
};
const result = await makeProvider(idx).provideCompletionItems(
makeDocument(text),
pos,
token,
);
const labels = listItems(result).map((i) => i.label);
assert.ok(labels.includes("CrateDebris_01"));
assert.ok(!labels.includes("xai:joinAction"));
assert.ok(!labels.includes("xmlns:xai"));
});
test("large asset-id lists are incomplete so narrower prefixes can re-request", async () => {
const defs = [];
for (let i = 0; i < 450; i++) {
defs.push({
type: "GameObject",
id: `C${String(i).padStart(3, "0")}`,
file: `C${i}.xml`,
line: 1,
origin: "project",
});
}
defs.push({
type: "GameObject",
id: "CrateDebris_01",
file: "Crates.xml",
line: 1,
origin: "project",
});
const byId = new Map();
const gameObjects = new Map();
const assets = new Map([["GameObject", gameObjects]]);
for (const def of defs) {
const key = def.id.toLowerCase();
byId.set(key, [def]);
gameObjects.set(key, [def]);
}
const idx = { assets, assetsById: byId };
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject>\n` +
` <CreateObject>C</CreateObject>\n` +
` </CreateObject>\n` +
` </ObjectCreationList>\n` +
`</AssetDeclaration>`;
const line = text.split("\n")[3];
const pos = new Position(3, line.indexOf(">C") + 2);
const first = await makeProvider(idx).provideCompletionItems(
makeDocument(text),
pos,
token,
);
assert.equal(Array.isArray(first), false);
assert.equal(first.isIncomplete, true, "capped list asks VS Code to recompute");
assert.equal(first.items.length, 400);
assert.ok(
!first.items.some((i) => i.label === "CrateDebris_01"),
"the target is beyond the initial 400 and must be found by a re-request",
);
// isIncomplete makes VS Code call the provider again as the prefix narrows.
const text2 = text.replace(">C<", ">Cr<");
const line2 = text2.split("\n")[3];
const second = await makeProvider(idx).provideCompletionItems(
makeDocument(text2),
new Position(3, line2.indexOf(">Cr") + 3),
token,
);
const secondItems = listItems(second);
assert.ok(
secondItems.some((i) => i.label === "CrateDebris_01"),
"narrower prefix re-request reaches the previously cut-off id",
);
});
test("current-file local overlay assets survive the global 400 cap", async () => {
const defs = [];
for (let i = 0; i < 450; i++) {
defs.push({
type: "GameObject",
id: `C${String(i).padStart(3, "0")}`,
file: `C${i}.xml`,
line: 1,
origin: "project",
});
}
const byId = new Map();
const gameObjects = new Map();
const assets = new Map([["GameObject", gameObjects]]);
for (const def of defs) {
const key = def.id.toLowerCase();
byId.set(key, [def]);
gameObjects.set(key, [def]);
}
const localGo = {
type: "GameObject",
id: "CrateDebris_01",
file: "Crates.xml",
line: 1,
origin: "project",
stream: "local",
};
const idx = {
assets,
assetsById: byId,
local: {
assets: new Map([
["GameObject", new Map([["cratedebris_01", [localGo]]])],
]),
assetsById: new Map([["cratedebris_01", [localGo]]]),
defines: new Map(),
},
};
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject>\n` +
` <CreateObject>C</CreateObject>\n` +
` </CreateObject>\n` +
` </ObjectCreationList>\n` +
`</AssetDeclaration>`;
const line = text.split("\n")[3];
const result = await makeProvider(idx).provideCompletionItems(
makeDocument(text),
new Position(3, line.indexOf(">C") + 2),
token,
);
assert.equal(Array.isArray(result), false);
assert.equal(result.isIncomplete, true);
assert.ok(result.items.some((i) => i.label === "CrateDebris_01"));
});
+281
View File
@@ -0,0 +1,281 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { createRequire } from "node:module";
// Minimal vscode shim for hover / definition / diagnostics providers.
const CompletionItemKind = {};
class Position {
constructor(line, character) {
this.line = line;
this.character = character;
}
}
class Range {
constructor(start, end) {
this.start = start;
this.end = end;
}
}
class MarkdownString {
constructor(value) {
this.value = value ?? "";
}
appendMarkdown(text) {
this.value += text;
return this;
}
appendCodeblock(text) {
this.value += "\n```\n" + text + "\n```\n";
return this;
}
}
class Hover {
constructor(contents) {
this.contents = contents;
}
}
class Location {
constructor(uri, range) {
this.uri = uri;
this.range = range;
}
}
class Diagnostic {
constructor(range, message, severity) {
this.range = range;
this.message = message;
this.severity = severity;
}
}
class FakeDiagnosticCollection {
constructor() {
this.last = null;
}
set(uri, diags) {
this.last = { uri, diags };
}
delete() {}
dispose() {}
}
const Uri = {
file: (p) => ({ fsPath: p }),
};
const workspace = {
getWorkspaceFolder: (uri) => ({ uri: { fsPath: "C:/mod" } }),
};
const languages = {
createDiagnosticCollection: () => new FakeDiagnosticCollection(),
};
const require = createRequire(import.meta.url);
const Module = require("module");
const origResolve = Module._resolveFilename;
Module._resolveFilename = function (request, ...args) {
if (request === "vscode") return "vscode-stub";
return origResolve.call(this, request, ...args);
};
require.cache["vscode-stub"] = {
id: "vscode-stub",
filename: "vscode-stub",
loaded: true,
exports: {
CompletionItemKind,
Position,
Range,
MarkdownString,
Hover,
Location,
Diagnostic,
DiagnosticSeverity: { Error: 0, Warning: 1, Information: 2, Hint: 3 },
Uri,
workspace,
languages,
},
};
const { Ra3HoverProvider } = require("../out/features/hover.js");
const { Ra3DefinitionProvider } = require("../out/features/navigation.js");
const { Ra3Diagnostics } = require("../out/features/diagnostics.js");
const { parseXml, LineMap } = require("../out/language/xmlParser.js");
const { expandDocument } = require("../out/indexer/logicalTree.js");
const URI = "C:/mod/Data/Crates.xml";
function makeDocument(text, uri = URI) {
const lineStarts = [0];
for (let i = 0; i < text.length; i++) {
if (text.charCodeAt(i) === 10) lineStarts.push(i + 1);
}
return {
uri: { fsPath: uri },
getText: () => text,
offsetAt: (pos) => lineStarts[pos.line] + pos.character,
positionAt: (offset) => {
let lo = 0;
let hi = lineStarts.length - 1;
while (lo < hi) {
const mid = (lo + hi + 1) >> 1;
if (lineStarts[mid] <= offset) lo = mid;
else hi = mid - 1;
}
return new Position(lo, offset - lineStarts[lo]);
},
};
}
async function makeScope(text, idx) {
const lineMap = new LineMap(text);
const parse = parseXml(text);
const expanded = await expandDocument(URI, parse, {
resolve: () => null,
readDom: async () => null,
});
return {
uri: URI,
version: 1,
parse,
lineMap,
expanded,
lineMaps: new Map(),
overlay: {},
merged: idx,
};
}
function makeIdx(defs) {
const assets = new Map();
const assetsById = new Map();
for (const def of defs) {
const idKey = def.id.toLowerCase();
let byId = assets.get(def.type);
if (!byId) {
byId = new Map();
assets.set(def.type, byId);
}
byId.set(idKey, [def]);
assetsById.set(idKey, [def]);
}
return {
assets,
assetsById,
defines: new Map(),
projectDir: "C:/mod",
sdkDir: "C:/sdk",
files: new Map(),
streams: [],
manifests: new Map(),
sourceCandidates: [],
diagnostics: [],
stats: {},
};
}
const TEXT =
`<AssetDeclaration>\n` +
` <GameObject id="CrateDebris_01"/>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject>\n` +
` <CreateObject>CrateDebris_01</CreateObject>\n` +
` </CreateObject>\n` +
` </ObjectCreationList>\n` +
`</AssetDeclaration>`;
test("hover on simple-content text shows the referenced definition", async () => {
const def = {
type: "GameObject",
id: "CrateDebris_01",
file: URI,
line: 2,
origin: "project",
};
const idx = makeIdx([def]);
const scope = await makeScope(TEXT, idx);
const provider = new Ra3HoverProvider({
isRa3Workspace: () => true,
getScope: async () => scope,
searchPaths: () => null,
});
const line = TEXT.split("\n")[4];
const pos = new Position(4, line.indexOf("CrateDebris_01") + 3);
const hover = await provider.provideHover(makeDocument(TEXT), pos, {});
assert.ok(hover, "hover is returned for typed content text");
assert.match(hover.contents.value, /1 definition/);
assert.match(hover.contents.value, /GameObject/);
});
test("Ctrl+click on simple-content text jumps to the definition", async () => {
const def = {
type: "GameObject",
id: "CrateDebris_01",
file: URI,
line: 2,
origin: "project",
};
const scope = await makeScope(TEXT, makeIdx([def]));
const provider = new Ra3DefinitionProvider({
isRa3Workspace: () => true,
getScope: async () => scope,
settings: { definitionMode: "all" },
indexer: null,
});
const line = TEXT.split("\n")[4];
const pos = new Position(4, line.indexOf("CrateDebris_01") + 3);
const locations = await provider.provideDefinition(makeDocument(TEXT), pos, {});
assert.ok(locations && locations.length === 1, "content definition resolves");
const defStart = TEXT.indexOf('id="CrateDebris_01"') + 'id="'.length;
const defEnd = defStart + "CrateDebris_01".length;
assert.deepEqual(
{
start: locations[0].range.start,
end: locations[0].range.end,
},
{
start: makeDocument(TEXT).positionAt(defStart),
end: makeDocument(TEXT).positionAt(defEnd),
},
);
});
test("diagnostics report unresolved typed content references only", async () => {
const text =
`<AssetDeclaration>\n` +
` <GameObject id="CrateDebris_01"/>\n` +
` <ObjectFilter>\n` +
` <IncludeThing>SomeValue</IncludeThing>\n` +
` </ObjectFilter>\n` +
` <ObjectCreationList id="OCL">\n` +
` <CreateObject>\n` +
` <CreateObject>CrateDebris_01</CreateObject>\n` +
` <CreateObject>MissingThing</CreateObject>\n` +
` </CreateObject>\n` +
` </ObjectCreationList>\n` +
`</AssetDeclaration>`;
const def = {
type: "GameObject",
id: "CrateDebris_01",
file: URI,
line: 2,
origin: "project",
};
const scope = await makeScope(text, makeIdx([def]));
const collection = new FakeDiagnosticCollection();
const provider = new Ra3Diagnostics({
isRa3Workspace: () => true,
getScope: async () => scope,
settings: {
diagnoseUnknownElements: false,
reportUnresolvedReferences: "warning",
},
});
provider["collection"] = collection;
await provider.update(makeDocument(text));
const messages = collection.last.diags.map((d) => d.message);
assert.ok(
messages.some((m) => m.includes('Unresolved reference "MissingThing"')),
"typed content refs are diagnosed",
);
assert.ok(
!messages.some((m) => m.includes("SomeValue")),
"untyped WeakReference content is not diagnosed as a global ref",
);
});
+72
View File
@@ -34,6 +34,78 @@ test("closed quote still resolves value context", () => {
assert.equal(ctx.valuePrefix, "GROUND"); assert.equal(ctx.valuePrefix, "GROUND");
}); });
test("multi-line unterminated quote keeps attribute-value context", () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject\n` +
` Options="IGNORE_ALL_OBJECTS"\n` +
` Disposition="`;
const cursor = text.length;
const doc = parseXml(text);
const ctx = analyzeContext(doc, text, cursor);
assert.equal(ctx.kind, "attribute-value");
assert.equal(ctx.attr?.name, "Disposition");
assert.equal(ctx.valuePrefix, "");
assert.equal(ctx.element?.name, "CreateObject");
});
test("multi-line unterminated value prefix includes typed flags", () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject\n` +
` Options="IGNORE_ALL_OBJECTS"\n` +
` Disposition="RANDOM_FORCE `;
const cursor = text.length;
const doc = parseXml(text);
const ctx = analyzeContext(doc, text, cursor);
assert.equal(ctx.kind, "attribute-value");
assert.equal(ctx.attr?.name, "Disposition");
assert.equal(ctx.valuePrefix, "RANDOM_FORCE ");
});
test("cursor after a closed quote is an attribute-name context", () => {
const text = `<Locomotor id="x" Surfaces="GROUND">\n</Locomotor>`;
const cursor = text.indexOf('GROUND"') + 'GROUND"'.length;
const doc = parseXml(text);
const ctx = analyzeContext(doc, text, cursor);
assert.equal(ctx.kind, "attribute-name");
assert.equal(ctx.attr, null);
});
test("cursor exactly after an opening tag with a closing tag is content", () => {
const text = `<CreateObject></CreateObject>`;
const cursor = text.indexOf(">") + 1;
const doc = parseXml(text);
const ctx = analyzeContext(doc, text, cursor);
assert.equal(ctx.kind, "content");
assert.equal(ctx.element?.name, "CreateObject");
});
test("cursor after a typed > but before the closing tag is content too", () => {
const text = `<CreateObject>`;
const cursor = text.length;
const doc = parseXml(text);
const ctx = analyzeContext(doc, text, cursor);
assert.equal(ctx.kind, "content");
assert.equal(ctx.element?.name, "CreateObject");
});
test("cursor after a closed child element belongs to the parent content", () => {
const text =
`<AssetDeclaration>` +
`<ObjectCreationList>` +
`<CreateObject>X</CreateObject>` +
`</ObjectCreationList>` +
`</AssetDeclaration>`;
const cursor = text.indexOf("</CreateObject>") + "</CreateObject>".length;
const doc = parseXml(text);
const ctx = analyzeContext(doc, text, cursor);
assert.equal(ctx.kind, "content");
assert.equal(ctx.element?.name, "ObjectCreationList");
});
test("splitListValuePrefix isolates the token being edited", () => { test("splitListValuePrefix isolates the token being edited", () => {
assert.deepEqual(splitListValuePrefix("GROUND WA"), { token: "WA", start: 7 }); assert.deepEqual(splitListValuePrefix("GROUND WA"), { token: "WA", start: 7 });
assert.deepEqual(splitListValuePrefix("GROUND "), { token: "", start: 7 }); assert.deepEqual(splitListValuePrefix("GROUND "), { token: "", start: 7 });
+139
View File
@@ -0,0 +1,139 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { join } from "node:path";
import fs from "node:fs";
import os from "node:os";
import { DiskRecordsCache, diskCacheKey } from "../out/indexer/diskCache.js";
const identity = {
projectDir: "C:/proj",
sdkDir: "C:/sdk",
indexSageXml: true,
additionalDataSearchPaths: [],
builtmodsDirs: ["C:/sdk/builtmods"],
};
const sampleRecords = {
assets: [{ type: "GameObject", id: "TankA", line: 3 }],
defines: [{ name: "HP", value: "100", line: 2 }],
includes: [{ type: "all", source: "Units.xml", line: 4 }],
rootXiIncludes: [],
nestedXiIncludes: [],
references: [],
};
function stampOf(file) {
const s = fs.statSync(file);
return {
mtimeMs: s.mtimeMs,
size: s.size,
birthtimeMs: s.birthtimeMs,
ctimeMs: s.ctimeMs,
};
}
function makeTmp(t) {
const tmp = fs.mkdtempSync(join(os.tmpdir(), "ra3-diskcache-"));
t.after(() => fs.rmSync(tmp, { recursive: true, force: true }));
return tmp;
}
test("disk cache roundtrip keeps records and leaves no temp file", async (t) => {
const tmp = makeTmp(t);
const file = join(tmp, "a.xml");
fs.writeFileSync(file, "0123456789");
const filePath = join(tmp, "index-records.json.gz");
const cache = new DiskRecordsCache(filePath, identity);
await cache.save([
[
file.toLowerCase(),
{
stat: stampOf(file),
records: sampleRecords,
kind: "full",
contentHash: "abc123",
},
],
]);
assert.equal(fs.existsSync(`${filePath}.tmp`), false, "atomic write leaves no temp");
const { records, stats } = await cache.loadValidated();
assert.equal(stats.fileExists, true);
assert.equal(stats.keyMatched, true);
assert.equal(stats.loaded, 1);
assert.equal(stats.validated, 1);
assert.equal(stats.dropped, 0);
assert.equal(records.length, 1);
assert.deepEqual(records[0].records, sampleRecords);
assert.equal(records[0].contentHash, "abc123");
});
test("stat mismatch drops the cached entry", async (t) => {
const tmp = makeTmp(t);
const file = join(tmp, "a.xml");
fs.writeFileSync(file, "0123456789");
const filePath = join(tmp, "index-records.json.gz");
const cache = new DiskRecordsCache(filePath, identity);
await cache.save([
[file.toLowerCase(), { stat: stampOf(file), records: sampleRecords, kind: "full" }],
]);
const past = new Date(Date.now() - 60000);
fs.utimesSync(file, past, past);
const { records, stats } = await cache.loadValidated();
assert.equal(stats.validated, 0);
assert.equal(stats.dropped, 1);
assert.equal(records.length, 0);
});
test("identity mismatch ignores the cache", async (t) => {
const tmp = makeTmp(t);
const file = join(tmp, "a.xml");
fs.writeFileSync(file, "0123456789");
const filePath = join(tmp, "index-records.json.gz");
const cache = new DiskRecordsCache(filePath, identity);
await cache.save([
[file.toLowerCase(), { stat: stampOf(file), records: sampleRecords, kind: "full" }],
]);
const other = new DiskRecordsCache(filePath, {
...identity,
sdkDir: "D:/other-sdk",
});
const { records, stats } = await other.loadValidated();
assert.equal(stats.fileExists, true);
assert.equal(stats.keyMatched, false);
assert.equal(records.length, 0);
});
test("corrupt cache file yields an empty result", async (t) => {
const tmp = makeTmp(t);
const filePath = join(tmp, "index-records.json.gz");
fs.writeFileSync(filePath, "this is not gzip json");
const cache = new DiskRecordsCache(filePath, identity);
const { records, stats } = await cache.loadValidated();
assert.equal(stats.fileExists, true);
assert.equal(records.length, 0);
});
test("clear removes the cache file", async (t) => {
const tmp = makeTmp(t);
const file = join(tmp, "a.xml");
fs.writeFileSync(file, "0123456789");
const filePath = join(tmp, "index-records.json.gz");
const cache = new DiskRecordsCache(filePath, identity);
await cache.save([
[file.toLowerCase(), { stat: stampOf(file), records: sampleRecords, kind: "full" }],
]);
assert.ok(fs.existsSync(filePath));
await cache.clear();
assert.equal(fs.existsSync(filePath), false);
});
test("diskCacheKey differs when the identity changes", () => {
const a = diskCacheKey(identity);
const b = diskCacheKey({ ...identity, indexSageXml: false });
assert.notEqual(a, b);
assert.equal(a, diskCacheKey(identity));
});
+102
View File
@@ -0,0 +1,102 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { join, parse, resolve } from "node:path";
import fs from "node:fs";
import os from "node:os";
import { resolveSource } from "../out/indexer/includeResolver.js";
import {
ExistenceSnapshot,
buildExistenceSnapshot,
isDriveRoot,
} from "../out/indexer/existence.js";
function makeTmp(t) {
const tmp = fs.mkdtempSync(join(os.tmpdir(), "ra3-existence-"));
t.after(() => fs.rmSync(tmp, { recursive: true, force: true }));
return tmp;
}
test("isDriveRoot detects filesystem roots", () => {
assert.equal(isDriveRoot(parse(process.cwd()).root), true);
assert.equal(isDriveRoot(process.cwd()), false);
});
test("ExistenceSnapshot answers covered paths and falls back outside roots", (t) => {
const tmp = makeTmp(t);
const dataDir = join(tmp, "data");
fs.mkdirSync(dataDir);
const existing = join(dataDir, "Units.xml");
fs.writeFileSync(existing, "<x/>");
const snap = new ExistenceSnapshot([dataDir]);
assert.equal(snap.has(existing), true);
assert.equal(snap.has(join(dataDir, "Missing.xml")), false);
assert.equal(snap.has(join(tmp, "outside.xml")), null, "outside roots is unknown");
assert.ok(snap.hits >= 2, "covered lookups counted as hits");
assert.equal(snap.fallbacks, 1);
if (process.platform === "win32") {
assert.equal(
snap.has(existing.toUpperCase()),
true,
"lookup is case-insensitive on Windows",
);
}
});
test("buildExistenceSnapshot covers bounded search bases lazily", async (t) => {
const tmp = makeTmp(t);
const dataDir = join(tmp, "data");
const artDir = join(tmp, "art");
fs.mkdirSync(join(dataDir, "sub"), { recursive: true });
fs.mkdirSync(artDir);
fs.writeFileSync(join(dataDir, "Units.xml"), "<x/>");
fs.writeFileSync(join(dataDir, "sub", "Nested.xml"), "<x/>");
fs.writeFileSync(join(artDir, "Tank.w3x"), "<x/>");
const snap = buildExistenceSnapshot({
DATA: [dataDir],
ART: [artDir],
AUDIO: [],
});
assert.equal(snap.has(join(dataDir, "Units.xml")), true);
assert.equal(snap.has(join(dataDir, "sub", "Nested.xml")), true);
assert.equal(snap.has(join(artDir, "Tank.w3x")), true);
assert.equal(snap.has(join(dataDir, "Missing.xml")), false);
});
test("resolveSource uses the snapshot and falls back to statSync outside it", async (t) => {
const tmp = makeTmp(t);
const dataDir = join(tmp, "data");
const outsideDir = join(tmp, "outside");
fs.mkdirSync(dataDir);
fs.mkdirSync(outsideDir);
const units = join(dataDir, "Units.xml");
const outside = join(outsideDir, "Extra.xml");
fs.writeFileSync(units, "<x/>");
fs.writeFileSync(outside, "<x/>");
const searchPaths = { DATA: [dataDir], ART: [], AUDIO: [] };
const snap = buildExistenceSnapshot(searchPaths);
const found = resolveSource("DATA:Units.xml", null, searchPaths, snap);
assert.equal(found.path, units);
assert.ok(snap.hits > 0, "covered lookup served by the snapshot");
const missing = resolveSource("DATA:Missing.xml", null, searchPaths, snap);
assert.equal(missing.path, null);
const emptySearchPaths = { DATA: [], ART: [], AUDIO: [] };
const outsideResolved = resolveSource(
outside,
outsideDir,
emptySearchPaths,
snap,
);
assert.equal(
resolve(outsideResolved.path ?? ""),
resolve(outside),
"uncovered path falls back to statSync",
);
assert.ok(snap.fallbacks > 0, "fallback counted");
});
+72
View File
@@ -0,0 +1,72 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { join } from "node:path";
import {
isContentRelevantPath,
isWatcherNoisePath,
} from "../out/indexer/fileScanner.js";
test("isWatcherNoisePath filters .git internals", () => {
assert.equal(
isWatcherNoisePath(join("C:/proj", ".git", "index")),
true,
".git files are noise",
);
assert.equal(
isWatcherNoisePath(join("C:/proj", "data", ".git", "FETCH_HEAD")),
true,
"nested .git directories are noise",
);
assert.equal(
isWatcherNoisePath(join("C:/proj", "Data", "Mod.xml")),
false,
"project XML is not noise",
);
assert.equal(
isWatcherNoisePath(join("C:/proj", ".gitignore")),
false,
".gitignore is a real project file",
);
assert.equal(
isWatcherNoisePath(join("C:/proj", "Data", "UnitCrate.xml.git")),
true,
"editor temp files ending in .git are noise",
);
assert.equal(
isWatcherNoisePath(join("C:/proj", "Data", "UnitCrate.xml.tmp")),
true,
".tmp files are noise",
);
assert.equal(
isWatcherNoisePath(join("C:/proj", "Data", "Mod.xml~")),
true,
"backup files ending in ~ are noise",
);
assert.equal(
isWatcherNoisePath(join("C:/proj", "Data", ".#Mod.xml")),
true,
"lock files starting with .# are noise",
);
});
test("isContentRelevantPath only reacts to XML-ish content", () => {
assert.equal(isContentRelevantPath("a.xml"), true);
assert.equal(isContentRelevantPath("a.w3x"), true);
assert.equal(
isContentRelevantPath("a.manifestxml"),
false,
"there is no .manifestxml source format",
);
assert.equal(
isContentRelevantPath("a.w3d"),
false,
".w3d is binary art, not text XML",
);
assert.equal(isContentRelevantPath("a.dds"), false);
assert.equal(isContentRelevantPath("a.xml.git"), false);
assert.equal(
isContentRelevantPath("a.lua"),
false,
"lua is not indexed yet",
);
});
@@ -0,0 +1,6 @@
<?xml version="1.0" encoding="utf-8"?>
<AssetDeclaration xmlns="uri:ea.com:eala:asset">
<HeadlightDraw2>
<TruckDraw id="ModuleTag_Headlight" />
</HeadlightDraw2>
</AssetDeclaration>
@@ -0,0 +1,2 @@
This is a fake DDS payload used to verify that binary assets included from
an XML hub are registered as files but never parsed as XML.
@@ -0,0 +1,4 @@
<?xml version="1.0" encoding="UTF-8"?>
<AssetDeclaration xmlns="uri:ea.com:eala:asset">
<W3DMesh id="Tank_FP" />
</AssetDeclaration>
@@ -0,0 +1,5 @@
<?xml version="1.0" encoding="UTF-8"?>
<AssetDeclaration xmlns="uri:ea.com:eala:asset">
<W3DContainer id="Tank_SKN" Hierarchy="Tank_SKL" />
<W3DHierarchy id="Tank_SKL" />
</AssetDeclaration>
@@ -0,0 +1,4 @@
<?xml version="1.0" encoding="utf-8"?>
<AssetDeclaration xmlns="uri:ea.com:eala:asset">
<GameObject id="StandaloneBase" Side="Allies" />
</AssetDeclaration>
+8
View File
@@ -0,0 +1,8 @@
<?xml version="1.0" encoding="utf-8"?>
<AssetDeclaration xmlns="uri:ea.com:eala:asset">
<Includes>
<Include type="all" source="Models/Tank_SKN.w3x" />
<Include type="all" source="Models/Tank_FP.dat" />
<Include type="all" source="Models/Tank_Damaged.dds" />
</Includes>
</AssetDeclaration>
+1
View File
@@ -9,5 +9,6 @@
<Include type="all" source="Includes/Weapons.xml" /> <Include type="all" source="Includes/Weapons.xml" />
<Include type="all" source="Includes/Shared.xml" /> <Include type="all" source="Includes/Shared.xml" />
<Include type="all" source="Includes/Refs.xml" /> <Include type="all" source="Includes/Refs.xml" />
<Include type="all" source="Includes/VehicleArt.xml" />
</Includes> </Includes>
</AssetDeclaration> </AssetDeclaration>
+20
View File
@@ -0,0 +1,20 @@
<?xml version="1.0" encoding="utf-8"?>
<AssetDeclaration xmlns="uri:ea.com:eala:asset">
<Includes>
<Include type="instance" source="Includes/StandaloneBase.xml" />
</Includes>
<Defines>
<Define name="STANDALONE_HEALTH" value="200.0" />
</Defines>
<GameObject id="StandaloneTank" inheritFrom="StandaloneBase">
<Draws>
<TruckDraw id="ModuleTag_Draw" />
<xi:include
href="DATA:Includes/HeadlightModules.xml"
xpointer="xmlns(n=uri:ea.com:eala:asset) xpointer(/n:HeadlightDraw2/child::*)" />
</Draws>
<BehaviorModules>
<ReconstituteStateSpecialAbility UpdateModuleId="ModuleTag_Headlight" />
</BehaviorModules>
</GameObject>
</AssetDeclaration>
+290
View File
@@ -2,8 +2,16 @@ import { test } from "node:test";
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { fileURLToPath } from "node:url"; import { fileURLToPath } from "node:url";
import { dirname, join } from "node:path"; import { dirname, join } from "node:path";
import fs from "node:fs";
import os from "node:os";
import { ModIndexer } from "../out/indexer/indexer.js"; import { ModIndexer } from "../out/indexer/indexer.js";
import { CachedDirectoryWalker } from "../out/indexer/fileScanner.js"; import { CachedDirectoryWalker } from "../out/indexer/fileScanner.js";
import {
DocumentCache,
IncludeResolveCache,
IndexRecordsCache,
} from "../out/indexer/caches.js";
import { resolveReferenceTargetsForType } from "../out/indexer/refs.js";
const root = dirname(dirname(fileURLToPath(import.meta.url))); const root = dirname(dirname(fileURLToPath(import.meta.url)));
const project = join(root, "test", "fixtures", "minimod"); const project = join(root, "test", "fixtures", "minimod");
@@ -70,3 +78,285 @@ test("provides include source candidates", async () => {
assert.ok(xml.some((c) => c.source === "Includes/Units.xml")); assert.ok(xml.some((c) => c.source === "Includes/Units.xml"));
assert.ok(xml.some((c) => c.source === "DATA:static.xml")); assert.ok(xml.some((c) => c.source === "DATA:static.xml"));
}); });
test("indexes art-asset XML (.w3x / sniffed unknown extension) via shallow scan and skips binary", async () => {
const idx = await buildIndex();
// The .w3x hub chain: Mod.xml -> VehicleArt.xml -> Models/Tank_SKN.w3x.
const skn = idx.assetsById.get("tank_skn");
assert.ok(skn?.some((d) => d.type === "W3DContainer"), "W3DContainer from .w3x indexed");
assert.ok(
idx.assetsById.get("tank_skl")?.some((d) => d.type === "W3DHierarchy"),
"W3DHierarchy from .w3x indexed",
);
const container = skn.find((d) => d.type === "W3DContainer");
assert.match(container.file, /Tank_SKN\.w3x$/);
assert.ok(container.line > 0, "definition line recorded from shallow scan");
// Unknown extension with XML content is sniffed and indexed.
assert.ok(
idx.assetsById.get("tank_fp")?.some((d) => d.type === "W3DMesh"),
"unknown-extension XML (.dat) sniffed and indexed",
);
// Binary content is registered as a file but never parsed.
assert.equal(idx.assetsById.get("tank_damaged"), undefined, "binary asset never indexed");
assert.ok(
idx.files.has(
join(project, "Data", "Includes", "Models", "Tank_Damaged.dds").toLowerCase(),
),
"binary include target registered as a file",
);
// The reported Harbinger scenario: <Model Name="Tank_SKN"/> (refType
// BaseRenderAssetType) must resolve to the W3DContainer defined in the w3x.
const targets = resolveReferenceTargetsForType(
idx,
"ScriptedModelDrawModel",
"Name",
"Tank_SKN",
);
assert.equal(targets.length, 1);
assert.equal(targets[0].def.type, "W3DContainer");
assert.match(targets[0].def.file, /Tank_SKN\.w3x$/);
});
test("w3x files appear in Include source completion candidates", async () => {
const idx = await buildIndex();
assert.ok(
idx.sourceCandidates.some((c) => c.source === "Includes/Models/Tank_SKN.w3x"),
"project-relative w3x candidate",
);
assert.ok(
idx.sourceCandidates.some((c) => c.source === "DATA:Includes/Models/Tank_SKN.w3x"),
"DATA: w3x candidate",
);
});
test("build publishes an immutable XML phase before art scanning", async () => {
let phaseA;
const indexer = new ModIndexer({
projectDir: project,
sdkDir: sdk,
builtmodsDirs: [join(sdk, "builtmods")],
indexSageXml: true,
additionalDataSearchPaths: [],
walker: new CachedDirectoryWalker(),
});
const idx = await indexer.build((p) => {
phaseA = p;
});
assert.ok(phaseA, "phase-A snapshot published");
assert.equal(phaseA.complete, false);
assert.equal(phaseA.phase, "xml");
assert.ok(phaseA.assetsById.has("testtank"), "XML assets available in phase A");
assert.ok(
phaseA.assetsById.has("vanillatank"),
"manifest assets available in phase A",
);
assert.equal(
phaseA.assetsById.has("tank_skn"),
false,
"art assets deferred in phase A",
);
assert.ok(phaseA.stats.deferredArtFiles >= 2, "deferred art queue recorded");
assert.equal(phaseA.stats.shallowScannedFiles, 0, "no art scanned during phase A");
assert.equal(idx.complete, true);
assert.equal(idx.phase, "art");
assert.ok(idx.assetsById.has("tank_skn"), "art assets present in the final index");
assert.equal(
phaseA.assetsById.has("tank_skn"),
false,
"phase-A snapshot is immutable (phase B did not mutate it)",
);
assert.equal(typeof idx.stats.artScanMs, "number");
assert.equal(
indexer.isIndexedFile(join(project, "Data", "Mod.xml")),
true,
"indexed files are recognized",
);
assert.equal(
indexer.isIndexedFile(join(project, "Data", "NotIndexed.xml")),
false,
"unrelated files are not recognized",
);
});
test("stat validation re-reads a file whose mtime changed", async (t) => {
const tmp = fs.mkdtempSync(join(os.tmpdir(), "ra3modxml-mtime-"));
t.after(() => fs.rmSync(tmp, { recursive: true, force: true }));
const projectDir = join(tmp, "project");
fs.mkdirSync(join(projectDir, "Data"), { recursive: true });
const modPath = join(projectDir, "Data", "Mod.xml");
fs.writeFileSync(
modPath,
`<?xml version="1.0"?>\n<AssetDeclaration>\n <GameObject id="TankA"/>\n</AssetDeclaration>\n`,
"utf8",
);
const documentCache = new DocumentCache();
const recordsCache = new IndexRecordsCache();
const resolveCache = new IncludeResolveCache();
const make = () =>
new ModIndexer({
projectDir,
sdkDir: sdk,
builtmodsDirs: [join(sdk, "builtmods")],
indexSageXml: false,
additionalDataSearchPaths: [],
walker: new CachedDirectoryWalker(),
documentCache,
recordsCache,
resolveCache,
trustUnchanged: false,
});
const first = await make().build();
assert.ok(first.assetsById.has("tanka"));
assert.equal(first.stats.recordsCacheHits, 0);
const past = new Date(Date.now() - 60000);
fs.utimesSync(modPath, past, past);
const second = await make().build();
assert.equal(
second.stats.recordsCacheHits,
0,
"mtime change invalidates the cached records",
);
assert.ok(second.assetsById.has("tanka"));
const third = await make().build();
assert.ok(
third.stats.recordsCacheHits > 0,
"unchanged files are served from the records cache",
);
});
test("shallow scans and full parses are cached across rebuilds", async () => {
const documentCache = new DocumentCache();
const recordsCache = new IndexRecordsCache();
const resolveCache = new IncludeResolveCache();
const makeIndexer = () =>
new ModIndexer({
projectDir: project,
sdkDir: sdk,
builtmodsDirs: [join(sdk, "builtmods")],
indexSageXml: true,
additionalDataSearchPaths: [],
walker: new CachedDirectoryWalker(),
documentCache,
recordsCache,
resolveCache,
});
const first = await makeIndexer().build();
const second = await makeIndexer().build();
assert.equal(first.stats.shallowScannedFiles, 2, "w3x + sniffed w3d scanned on first build");
assert.equal(second.stats.shallowScannedFiles, 0, "unchanged art assets are not re-scanned");
assert.equal(second.stats.shallowCacheHits, 2);
assert.ok(second.stats.recordsCacheHits > 0, "XML records served from cache");
assert.ok(second.stats.resolveCacheHits > 0, "include resolutions served from cache");
assert.equal(second.stats.resolveCalls, 0, "no include re-resolved on a trusted rebuild");
assert.equal(second.stats.assetCount, first.stats.assetCount);
assert.equal(second.stats.indexedFiles, first.stats.indexedFiles);
});
test("trusted rebuilds skip unchanged files; invalidation forces re-reads", async () => {
const documentCache = new DocumentCache();
const recordsCache = new IndexRecordsCache();
const resolveCache = new IncludeResolveCache();
const opts = () => ({
projectDir: project,
sdkDir: sdk,
builtmodsDirs: [join(sdk, "builtmods")],
indexSageXml: true,
additionalDataSearchPaths: [],
walker: new CachedDirectoryWalker(),
documentCache,
recordsCache,
resolveCache,
});
// Trusted rebuild: cached files are reused without any per-file stat/read.
const first = await new ModIndexer({ ...opts(), trustUnchanged: true }).build();
assert.equal(first.stats.shallowScannedFiles, 2);
const second = await new ModIndexer({ ...opts(), trustUnchanged: true }).build();
assert.equal(second.stats.shallowScannedFiles, 0);
assert.equal(second.stats.shallowCacheHits, 2);
assert.ok(second.stats.recordsCacheHits > 0);
// Simulate the file watcher / save handler: invalidate one art asset.
// The next trusted rebuild must re-scan exactly that file.
recordsCache.invalidate(join(project, "Data", "Includes", "Models", "Tank_SKN.w3x"));
const third = await new ModIndexer({ ...opts(), trustUnchanged: true }).build();
assert.equal(third.stats.shallowScannedFiles, 1);
assert.equal(third.stats.shallowCacheHits, 1);
assert.ok(
third.assetsById.get("tank_skn")?.some((d) => d.type === "W3DContainer"),
"re-scanned w3x asset present",
);
// Invalidate one XML file: exactly its records are re-extracted.
recordsCache.invalidate(join(project, "Data", "Includes", "Units.xml"));
const fourth = await new ModIndexer({ ...opts(), trustUnchanged: true }).build();
assert.equal(fourth.stats.recordsCacheHits, third.stats.recordsCacheHits - 1);
assert.ok(
fourth.assetsById.get("testtank")?.some((d) => d.type === "GameObject"),
"re-parsed XML asset present",
);
// A forced rebuild (ra3modxml.reindex) verifies stats but still reuses
// content caches for unchanged files.
const forced = await new ModIndexer({ ...opts(), trustUnchanged: false }).build();
assert.equal(forced.stats.shallowScannedFiles, 0);
assert.equal(forced.stats.shallowCacheHits, 2);
});
test("index stats include candidate/walk phase timings", async () => {
const idx = await buildIndex();
assert.equal(typeof idx.stats.candidatesMs, "number");
assert.equal(typeof idx.stats.walkMs, "number");
assert.ok(idx.stats.snapshotHits > 0, "existence snapshot answered lookups");
assert.equal(typeof idx.stats.snapshotFallbacks, "number");
});
test("w3x with a UTF-8 BOM is indexed with correct offsets", async (t) => {
const tmp = fs.mkdtempSync(join(os.tmpdir(), "ra3modxml-bom-"));
t.after(() => fs.rmSync(tmp, { recursive: true, force: true }));
const projectDir = join(tmp, "project");
fs.mkdirSync(join(projectDir, "Data", "Models"), { recursive: true });
fs.writeFileSync(
join(projectDir, "Data", "Mod.xml"),
`<?xml version="1.0" encoding="utf-8"?>
<AssetDeclaration>
<Includes>
<Include type="all" source="Models/Tank_BOM.w3x"/>
</Includes>
</AssetDeclaration>`,
"utf8",
);
fs.writeFileSync(
join(projectDir, "Data", "Models", "Tank_BOM.w3x"),
"\uFEFF<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n" +
"<AssetDeclaration>\n" +
" <W3DContainer id=\"Tank_BOM\" />\n" +
"</AssetDeclaration>\n",
"utf8",
);
const idx = await new ModIndexer({
projectDir,
sdkDir: sdk,
builtmodsDirs: [join(sdk, "builtmods")],
indexSageXml: false,
additionalDataSearchPaths: [],
walker: new CachedDirectoryWalker(),
}).build();
const def = idx.assetsById.get("tank_bom")?.find((d) => d.type === "W3DContainer");
assert.ok(def, "BOM-prefixed w3x asset indexed");
assert.equal(def.line, 3, "id line is correct despite the BOM");
});
+210
View File
@@ -0,0 +1,210 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { fileURLToPath } from "node:url";
import { dirname, join } from "node:path";
import { parseXml, LineMap, stripBom } from "../out/language/xmlParser.js";
import { extractIndexRecords } from "../out/indexer/records.js";
import { buildSearchPaths } from "../out/indexer/includeResolver.js";
import {
buildDocumentScope,
withLocalOverlay,
} from "../out/indexer/localScope.js";
import {
findContainingGameObject,
findLocalId,
collectLocalIds,
} from "../out/indexer/logicalTree.js";
import { resolveReferenceTargetsForType } from "../out/indexer/refs.js";
import { resolveElementType } from "../out/language/typeContext.js";
const root = dirname(dirname(fileURLToPath(import.meta.url)));
const project = join(root, "test", "fixtures", "minimod");
const sdk = join(root, "test", "fixtures", "fakesdk");
const standalonePath = join(project, "Data", "Standalone.xml");
async function readParsed(path) {
const text = stripBom(await readFile(path, "utf8"));
const lineMap = new LineMap(text);
const parse = parseXml(text);
return {
file: { path, stat: null },
parse,
records: extractIndexRecords(parse, lineMap, text),
lineMap,
};
}
async function makeScope() {
const searchPaths = buildSearchPaths(sdk, project);
const text = await readFile(standalonePath, "utf8");
return buildDocumentScope(standalonePath, text, 1, {
projectDir: project,
sdkDir: sdk,
searchPaths,
readRecords: readParsed,
readDom: readParsed,
});
}
test("local overlay resolves refs for a file outside every global stream", async () => {
const scope = await makeScope();
const merged = withLocalOverlay(null, scope.overlay, project, sdk);
assert.ok(merged.local.assetsById.has("standalonetank"));
assert.ok(merged.local.assetsById.has("standalonebase"));
assert.ok(merged.local.defines.has("standalone_health"));
const targets = resolveReferenceTargetsForType(
merged,
"GameObject",
"inheritFrom",
"StandaloneBase",
);
assert.equal(targets.length, 1);
assert.match(targets[0].def.file, /StandaloneBase\.xml$/);
assert.equal(targets[0].def.origin, "project");
});
test("logical xi:include expansion gives included modules their Draws context", async () => {
const scope = await makeScope();
const included = scope.expanded.elements.find(
(e) =>
e.name === "TruckDraw" &&
e.attrs.some((a) => a.name === "id" && a.value === "ModuleTag_Headlight"),
);
assert.ok(included, "xi:include target spliced into the logical tree");
assert.match(included.sourceFile, /HeadlightModules\.xml$/i);
assert.equal(
resolveElementType(included),
"W3DTruckDrawModuleData",
"included module resolves through the logical Draws parent",
);
const update = scope.expanded.elements.find(
(e) => e.name === "ReconstituteStateSpecialAbility",
);
assert.ok(update);
const gameObject = findContainingGameObject(update);
assert.equal(gameObject?.name, "GameObject");
assert.equal(
findLocalId(gameObject, "ModuleTag_Headlight"),
included,
"Poid reference can reach a module spliced in through xi:include",
);
const localIds = collectLocalIds(gameObject).map((i) => i.id);
assert.ok(localIds.includes("ModuleTag_Draw"));
assert.ok(localIds.includes("ModuleTag_Headlight"));
});
test("local overlay wins over a global definition with the same id", async () => {
const scope = await makeScope();
const global = {
assets: new Map(),
assetsById: new Map([
[
"standalonebase",
[
{
type: "GameObject",
id: "StandaloneBase",
file: join(sdk, "SageXml", "VanillaBase.xml"),
line: 1,
origin: "sdk",
},
],
],
]),
defines: new Map(),
};
const merged = withLocalOverlay(global, scope.overlay, project, sdk);
const targets = resolveReferenceTargetsForType(
merged,
"GameObject",
"inheritFrom",
"StandaloneBase",
);
assert.equal(targets.length, 2);
assert.match(targets[0].def.file, /StandaloneBase\.xml$/);
assert.equal(targets[0].def.origin, "project");
assert.match(targets[1].def.file, /VanillaBase\.xml$/);
});
test("logical expansion terminates on xi:include cycles", async (t) => {
const tmp = await mkdtemp(join(tmpdir(), "ra3-local-cycle-"));
t.after(() => rm(tmp, { recursive: true, force: true }));
const dataDir = join(tmp, "Data");
const includesDir = join(dataDir, "Includes");
await mkdir(includesDir, { recursive: true });
const aPath = join(dataDir, "A.xml");
const bPath = join(includesDir, "B.xml");
await writeFile(
aPath,
`<AssetDeclaration><GameObject id="A"><xi:include href="Includes/B.xml"/></GameObject></AssetDeclaration>`,
"utf8",
);
await writeFile(
bPath,
`<AssetDeclaration><GameObject id="B"><xi:include href="../A.xml"/></GameObject></AssetDeclaration>`,
"utf8",
);
const searchPaths = buildSearchPaths(sdk, tmp);
const text = await readFile(aPath, "utf8");
const scope = await buildDocumentScope(aPath, text, 1, {
projectDir: tmp,
sdkDir: sdk,
searchPaths,
readRecords: readParsed,
readDom: readParsed,
});
assert.ok(
scope.expanded.elements.some(
(e) =>
e.name === "GameObject" &&
e.attrs.some((a) => a.name === "id" && a.value === "A"),
),
"entry document survives a cycle",
);
});
test("the same xi:include target can expand under multiple parents", async (t) => {
const tmp = await mkdtemp(join(tmpdir(), "ra3-local-shared-"));
t.after(() => rm(tmp, { recursive: true, force: true }));
const dataDir = join(tmp, "Data");
const includesDir = join(dataDir, "Includes");
await mkdir(includesDir, { recursive: true });
const mainPath = join(dataDir, "Main.xml");
const fragmentPath = join(includesDir, "Fragment.xml");
await writeFile(
fragmentPath,
`<AssetDeclaration><HeadlightDraw2><TruckDraw id="ModuleTag_Shared"/></HeadlightDraw2></AssetDeclaration>`,
"utf8",
);
await writeFile(
mainPath,
`<AssetDeclaration>` +
`<GameObject id="A"><Draws><xi:include href="DATA:Includes/Fragment.xml" ` +
`xpointer="xmlns(n=uri:ea.com:eala:asset) xpointer(/n:HeadlightDraw2/child::*)"/></Draws></GameObject>` +
`<GameObject id="B"><Draws><xi:include href="DATA:Includes/Fragment.xml" ` +
`xpointer="xmlns(n=uri:ea.com:eala:asset) xpointer(/n:HeadlightDraw2/child::*)"/></Draws></GameObject>` +
`</AssetDeclaration>`,
"utf8",
);
const searchPaths = buildSearchPaths(sdk, tmp);
const text = await readFile(mainPath, "utf8");
const scope = await buildDocumentScope(mainPath, text, 1, {
projectDir: tmp,
sdkDir: sdk,
searchPaths,
readRecords: readParsed,
readDom: readParsed,
});
const shared = scope.expanded.elements.filter(
(e) =>
e.name === "TruckDraw" &&
e.attrs.some((a) => a.name === "id" && a.value === "ModuleTag_Shared"),
);
assert.equal(shared.length, 2, "same fragment expands once per parent");
});
+115
View File
@@ -0,0 +1,115 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { parseXml, LineMap } from "../out/language/xmlParser.js";
import { extractIndexRecords, recordsFromShallow } from "../out/indexer/records.js";
import { scanXmlShallow } from "../out/indexer/shallowScan.js";
test("extractIndexRecords mirrors the walk semantics", () => {
const text = `<?xml version="1.0"?>
<AssetDeclaration>
<Defines>
<Define name="HP" value="100"/>
</Defines>
<Includes>
<Include type="all" source="Units.xml"/>
<Include type="instance" source="Base.xml"/>
</Includes>
<GameObject id="Tank" CommandSet="TankCommandSet"/>
<WeaponTemplate id="TankGun"/>
<xi:include href="DATA:Extra.xml" xpointer="xmlns(n=uri:ea.com:eala:asset) xpointer(/n:Extra/child::*)"/>
<GameObject id="Tank2">
<Draws>
<xi:include href="DATA:Nested.xml"/>
</Draws>
</GameObject>
</AssetDeclaration>`;
const lineMap = new LineMap(text);
const records = extractIndexRecords(parseXml(text), lineMap, text);
assert.deepEqual(
records.assets.map((a) => [a.type, a.id, a.line]),
[
["GameObject", "Tank", 10],
["WeaponTemplate", "TankGun", 11],
["GameObject", "Tank2", 13],
],
);
assert.deepEqual(
records.defines.map((d) => [d.name, d.value]),
[["HP", "100"]],
);
assert.deepEqual(
records.includes.map((i) => [i.type, i.source]),
[
["all", "Units.xml"],
["instance", "Base.xml"],
],
);
assert.deepEqual(
records.rootXiIncludes.map((x) => [x.href, x.line]),
[["DATA:Extra.xml", 12]],
);
assert.deepEqual(
records.nestedXiIncludes.map((x) => [x.href, x.line]),
[["DATA:Nested.xml", 15]],
);
});
test("recordsFromShallow converts offsets to 1-based lines", () => {
const text = `<AssetDeclaration>\n <W3DContainer id="A"/>\n</AssetDeclaration>`;
const lineMap = new LineMap(text);
const records = recordsFromShallow(scanXmlShallow(text), lineMap);
assert.equal(records.assets.length, 1);
assert.equal(records.assets[0].type, "W3DContainer");
assert.equal(records.assets[0].id, "A");
assert.equal(records.assets[0].line, 2);
assert.deepEqual(records.references, []);
});
test("extractIndexRecords records typed references and skips non-references", () => {
const text = `<AssetDeclaration>
<GameObject id="Tank" CommandSet="CS" inheritFrom="Base" KindOf="SELECTABLE"/>
<ObjectCreationList id="OCL">
<CreateObject>
<CreateObject>Tank</CreateObject>
</CreateObject>
</ObjectCreationList>
<CameraSettings id="S"/>
</AssetDeclaration>`;
const lineMap = new LineMap(text);
const records = extractIndexRecords(parseXml(text), lineMap, text);
const attrs = records.references.filter((r) => r.kind === "attr");
const content = records.references.filter((r) => r.kind === "content");
// Typed attribute reference keeps the XSD refType.
const cs = attrs.find((r) => r.value === "CS");
assert.ok(cs, "CommandSet reference is recorded");
assert.equal(cs.refType, "LogicCommandSet");
assert.equal(cs.selfType, null);
assert.equal(cs.line, 2);
assert.equal(records.references.some((r) => r.start === cs.start && r.end === cs.end), true);
// inheritFrom records the element type as selfType instead of refType.
const base = attrs.find((r) => r.value === "Base");
assert.ok(base, "inheritFrom reference is recorded");
assert.equal(base.refType, null);
assert.equal(base.selfType, "GameObject");
// Enums and non-reference values are not references.
assert.equal(attrs.some((r) => r.value === "SELECTABLE"), false);
// Simple-content text is recorded with its content refType and offsets.
const tank = content.find((r) => r.value === "Tank");
assert.ok(tank, "content reference is recorded");
assert.equal(tank.refType, "GameObject");
const line = text.split("\n")[4];
assert.equal(line.slice(tank.start - text.indexOf(line), tank.end - text.indexOf(line)), "Tank");
// The id definition itself is never recorded as a reference.
assert.equal(
records.references.some(
(r) => r.value === "Tank" && r.kind === "attr",
),
false,
);
});
+378
View File
@@ -0,0 +1,378 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { fileURLToPath } from "node:url";
import { dirname, join } from "node:path";
import { mkdtempSync, rmSync, statSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { ModIndexer } from "../out/indexer/indexer.js";
import { CachedDirectoryWalker } from "../out/indexer/fileScanner.js";
import {
IndexRecordsCache,
contentHash,
normKey,
recordsHash,
} from "../out/indexer/caches.js";
import {
assetDefKey,
buildReferenceIndex,
documentRecordsDesynced,
referenceSitesForDef,
referenceSitesForDefinition,
scheduleRebuildIfRecordsDesync,
unreferencedByType,
} from "../out/indexer/referenceIndex.js";
import { LineMap, parseXml } from "../out/language/xmlParser.js";
import { extractIndexRecords } from "../out/indexer/records.js";
const root = dirname(dirname(fileURLToPath(import.meta.url)));
const project = join(root, "test", "fixtures", "minimod");
const sdk = join(root, "test", "fixtures", "fakesdk");
async function buildIndex() {
const indexer = new ModIndexer({
projectDir: project,
sdkDir: sdk,
builtmodsDirs: [join(sdk, "builtmods")],
indexSageXml: true,
additionalDataSearchPaths: [],
walker: new CachedDirectoryWalker(),
});
return indexer.build();
}
function makeDef(type, id, file, line, extra = {}) {
return {
type,
id,
file,
line,
origin: "project",
...extra,
};
}
test("buildReferenceIndex resolves records with strict type filtering", () => {
const tank = makeDef("GameObject", "Tank", "C:/mod/A.xml", 2);
const gun = makeDef("WeaponTemplate", "Tank", "C:/mod/B.xml", 1);
const cs = makeDef("LogicCommandSet", "CS", "C:/mod/C.xml", 4);
const lookup = {
assets: new Map(),
assetsById: new Map([
["tank", [tank, gun]],
["cs", [cs]],
]),
};
const recordsA = {
assets: [],
defines: [],
includes: [],
rootXiIncludes: [],
nestedXiIncludes: [],
references: [
{
kind: "content",
refType: "GameObject",
selfType: null,
value: "Tank",
line: 5,
start: 40,
end: 44,
},
],
};
const recordsB = {
assets: [],
defines: [],
includes: [],
rootXiIncludes: [],
nestedXiIncludes: [],
references: [
{
kind: "attr",
refType: "LogicCommandSet",
selfType: null,
value: "CS",
line: 3,
start: 10,
end: 12,
},
],
};
const map = buildReferenceIndex(
[
{ file: "C:/mod/A.xml", records: recordsA },
{ file: "C:/mod/B.xml", records: recordsB },
],
lookup,
);
// The content reference resolves only to the GameObject definition, never
// to the same-name WeaponTemplate.
const tankSites = map.get(assetDefKey(tank));
assert.equal(tankSites.length, 1);
assert.equal(tankSites[0].file, "C:/mod/A.xml");
assert.equal(tankSites[0].kind, "content");
assert.equal(map.get(assetDefKey(gun)), undefined);
assert.equal(map.get(assetDefKey(cs)).length, 1);
});
test("records extracted from XML resolve through the reference index", () => {
const text = `<AssetDeclaration>
<GameObject id="Tank" CommandSet="CS"/>
<LogicCommandSet id="CS"/>
<ObjectCreationList id="OCL">
<CreateObject>
<CreateObject>Tank</CreateObject>
</CreateObject>
</ObjectCreationList>
</AssetDeclaration>`;
const lineMap = new LineMap(text);
const records = extractIndexRecords(parseXml(text), lineMap, text);
const file = "C:/mod/D.xml";
const tank = makeDef("GameObject", "Tank", file, 2);
const cs = makeDef("LogicCommandSet", "CS", file, 3);
const lookup = {
assets: new Map(),
assetsById: new Map([
["tank", [tank]],
["cs", [cs]],
]),
};
const map = buildReferenceIndex([{ file, records }], lookup);
const tankSites = map.get(assetDefKey(tank));
assert.equal(tankSites.length, 1);
assert.equal(tankSites[0].kind, "content");
assert.equal(records.references.find((r) => r.value === "Tank").start, tankSites[0].start);
const csSites = map.get(assetDefKey(cs));
assert.equal(csSites.length, 1);
assert.equal(csSites[0].kind, "attr");
});
test("referenceSitesForDefinition unions manifest-source sites onto the SageXml source file", () => {
const sourceFile = join(project, "Data", "Includes", "Units.xml");
const manifestDef = {
type: "GameObject",
id: "Tank",
file: join(sdk, "builtmods", "static.manifest"),
line: 0,
origin: "manifest",
manifestSource: "DATA:Includes/Units.xml",
};
const site = {
file: "C:/mod/ref.xml",
line: 3,
start: 10,
end: 14,
kind: "attr",
};
const idx = {
assets: new Map([["GameObject", new Map([["tank", [manifestDef]]])]]),
assetsById: new Map([["tank", [manifestDef]]]),
references: new Map([[assetDefKey(manifestDef), [site]]]),
projectDir: project,
sdkDir: sdk,
};
const sites = referenceSitesForDefinition(idx, {
type: "GameObject",
id: "Tank",
file: sourceFile,
line: 4,
});
assert.equal(sites.length, 1);
assert.equal(sites[0].file, "C:/mod/ref.xml");
// A different file does not inherit the manifest definition's sites.
const other = referenceSitesForDefinition(idx, {
type: "GameObject",
id: "Tank",
file: "C:/mod/elsewhere.xml",
line: 4,
});
assert.equal(other.length, 0);
});
test("the minimod indexer publishes a semantic reverse reference index", async () => {
const idx = await buildIndex();
assert.ok(idx.stats.referenceCount > 0, "reverse index is populated");
// Units.xml has CommandSet="TestTankCommandSet" and inheritFrom="BaseVehicle".
const lcs = idx.assets.get("LogicCommandSet").get("testtankcommandset")[0];
const lcsSites = idx.references.get(assetDefKey(lcs));
assert.ok(lcsSites && lcsSites.length >= 1);
assert.ok(lcsSites.some((s) => s.kind === "attr"));
const baseVehicle = idx.assets.get("GameObject").get("basevehicle")[0];
const bvSites = idx.references.get(assetDefKey(baseVehicle));
assert.ok(bvSites && bvSites.length >= 1);
});
test("references survive a records-cache invalidation during the build", async () => {
const recordsCache = new IndexRecordsCache();
const indexer = new ModIndexer({
projectDir: project,
sdkDir: sdk,
builtmodsDirs: [join(sdk, "builtmods")],
indexSageXml: true,
additionalDataSearchPaths: [],
walker: new CachedDirectoryWalker(),
recordsCache,
});
const unitsPath = join(project, "Data", "Includes", "Units.xml");
const idx = await indexer.build((phaseIndex) => {
if (!phaseIndex.complete) recordsCache.invalidate(unitsPath);
});
const lcs = idx.assets.get("LogicCommandSet").get("testtankcommandset")[0];
const sites = idx.references.get(assetDefKey(lcs));
assert.ok(
sites && sites.length >= 1,
"final snapshot keeps the walk-time reference records",
);
});
test("force rebuild verifies content even when every stat signal matches", async (t) => {
const tmp = mkdtempSync(join(tmpdir(), "ra3-refidx-"));
t.after(() => rmSync(tmp, { recursive: true, force: true }));
const file = join(tmp, "units.xml");
const diskText = `<AssetDeclaration><GameObject id="Tank"/></AssetDeclaration>`;
const cachedText = `<AssetDeclaration><GameObject id="Cached"/></AssetDeclaration>`;
writeFileSync(file, diskText);
const st = statSync(file);
const stamp = {
mtimeMs: st.mtimeMs,
size: st.size,
birthtimeMs: st.birthtimeMs,
ctimeMs: st.ctimeMs,
};
const cache = new IndexRecordsCache();
cache.set(file, {
stat: stamp,
records: extractIndexRecords(
parseXml(cachedText),
new LineMap(cachedText),
cachedText,
),
kind: "full",
contentHash: contentHash(cachedText),
});
const opts = {
projectDir: tmp,
sdkDir: tmp,
builtmodsDirs: [],
indexSageXml: false,
additionalDataSearchPaths: [],
walker: new CachedDirectoryWalker(),
recordsCache: cache,
};
const trusted = new ModIndexer({ ...opts, trustUnchanged: true });
const trustedParsed = await trusted.readDocument(file);
assert.equal(
trustedParsed.records.assets[0].id,
"Cached",
"trusted rebuilds reuse the cached records without reading",
);
const forced = new ModIndexer({ ...opts, trustUnchanged: false });
const forcedParsed = await forced.readDocument(file);
assert.equal(
forcedParsed.records.assets[0].id,
"Tank",
"force rebuild re-reads a stat-matching but content-stale entry",
);
assert.equal(cache.get(file).contentHash, contentHash(diskText));
});
test("records-desync self-heal schedules a targeted rebuild only for clean files", () => {
const file = join(project, "Data", "Includes", "Units.xml");
const emptyRecords = extractIndexRecords(
parseXml("<AssetDeclaration/>"),
new LineMap("<AssetDeclaration/>"),
"<AssetDeclaration/>",
);
const idx = {
recordsHashes: new Map([[normKey(file), recordsHash(emptyRecords)]]),
references: new Map(),
assets: new Map(),
sdkDir: sdk,
projectDir: project,
};
assert.equal(
documentRecordsDesynced(idx, file, "<AssetDeclaration/>"),
false,
);
assert.equal(
documentRecordsDesynced(
idx,
file,
"<AssetDeclaration><GameObject id=\"Tank\"/></AssetDeclaration>",
),
true,
);
let invalidated = null;
let scheduled = null;
const ws = {
index: idx,
invalidate: (p) => {
invalidated = p;
},
scheduleRebuild: (r) => {
scheduled = r;
},
};
assert.equal(
scheduleRebuildIfRecordsDesync(ws, {
uri: { fsPath: file, scheme: "file" },
isDirty: false,
getText: () => "<AssetDeclaration><GameObject id=\"Tank\"/></AssetDeclaration>",
}),
true,
);
assert.equal(invalidated, file);
assert.equal(scheduled, "records-desync");
invalidated = null;
scheduled = null;
assert.equal(
scheduleRebuildIfRecordsDesync(ws, {
uri: { fsPath: file, scheme: "file" },
isDirty: true,
getText: () => "<AssetDeclaration><GameObject id=\"Tank\"/></AssetDeclaration>",
}),
false,
);
assert.equal(scheduled, null);
});
test("unreferencedByType reports only meaningful project assets", async () => {
const idx = await buildIndex();
const map = unreferencedByType(idx);
const all = [...map.values()].flat();
assert.ok(all.length > 0, "some project assets are unreferenced");
assert.ok(
all.every((d) => d.origin === "project" && !d.viaInstance),
"only compiled-stream project definitions are reported",
);
assert.ok(
all.every((d) => !d.file.toLowerCase().includes("fakesdk")),
"SDK/manifest definitions are never reported",
);
// WeaponTemplate TestTankCannon is defined but never referenced.
const weapons = map.get("WeaponTemplate");
assert.ok(weapons.some((d) => d.id === "TestTankCannon"));
// LogicCommandSet TestTankCommandSet is referenced, so it must not appear.
const commandSets = map.get("LogicCommandSet");
assert.ok(!commandSets?.some((d) => d.id === "TestTankCommandSet"));
// referenceSitesForDef is stable across lookups and safe without a map.
const tankCannon = idx.assets.get("WeaponTemplate").get("testtankcannon")[0];
assert.deepEqual(referenceSitesForDef(idx, tankCannon), []);
assert.deepEqual(referenceSitesForDef(null, tankCannon), []);
});
+169
View File
@@ -0,0 +1,169 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { createRequire } from "node:module";
// Minimal vscode shim for the semantic reference provider.
class Position {
constructor(line, character) {
this.line = line;
this.character = character;
}
}
class Range {
constructor(start, end) {
this.start = start;
this.end = end;
}
}
class Location {
constructor(uri, range) {
this.uri = uri;
this.range = range;
}
}
const Uri = {
file: (p) => ({ fsPath: p }),
};
const require = createRequire(import.meta.url);
const Module = require("module");
const origResolve = Module._resolveFilename;
Module._resolveFilename = function (request, ...args) {
if (request === "vscode") return "vscode-stub";
return origResolve.call(this, request, ...args);
};
require.cache["vscode-stub"] = {
id: "vscode-stub",
filename: "vscode-stub",
loaded: true,
exports: {
Position,
Range,
Location,
Uri,
SymbolKind: {},
DocumentSymbol: class {},
},
};
const { Ra3ReferenceProvider } = require("../out/features/navigation.js");
const { LineMap, parseXml } = require("../out/language/xmlParser.js");
const { extractIndexRecords } = require("../out/indexer/records.js");
const { buildReferenceIndex } = require("../out/indexer/referenceIndex.js");
const FILE = "C:/mod/Data/Units.xml";
const TEXT = `<AssetDeclaration>
<GameObject id="Tank"/>
<ObjectCreationList id="OCL">
<CreateObject>
<CreateObject>Tank</CreateObject>
</CreateObject>
</ObjectCreationList>
</AssetDeclaration>`;
function makeDocument(text = TEXT) {
const lineStarts = [0];
for (let i = 0; i < text.length; i++) {
if (text.charCodeAt(i) === 10) lineStarts.push(i + 1);
}
return {
uri: { fsPath: FILE },
getText: () => text,
offsetAt: (pos) => lineStarts[pos.line] + pos.character,
positionAt: (offset) => {
let lo = 0;
let hi = lineStarts.length - 1;
while (lo < hi) {
const mid = (lo + hi + 1) >> 1;
if (lineStarts[mid] <= offset) lo = mid;
else hi = mid - 1;
}
return new Position(lo, offset - lineStarts[lo]);
},
};
}
function makeScope() {
const parse = parseXml(TEXT);
const lineMap = new LineMap(TEXT);
const records = extractIndexRecords(parse, lineMap, TEXT);
const def = {
type: "GameObject",
id: "Tank",
file: FILE,
line: 2,
origin: "project",
};
const lookup = {
assets: new Map([["GameObject", new Map([["tank", [def]]])]]),
assetsById: new Map([["tank", [def]]]),
};
const references = buildReferenceIndex([{ file: FILE, records }], lookup);
const idx = {
...lookup,
references,
complete: true,
phase: "art",
projectDir: "C:/mod",
sdkDir: "C:/sdk",
defines: new Map(),
files: new Map(),
streams: [],
manifests: new Map(),
sourceCandidates: [],
diagnostics: [],
stats: {},
};
return {
merged: idx,
};
}
function makeWs(scope) {
const parse = parseXml(TEXT);
const lineMap = new LineMap(TEXT);
return {
isRa3Workspace: () => true,
getScope: async () => scope,
indexer: {
readDom: async (path) =>
path === FILE
? { file: { path: FILE }, parse, lineMap, records: null }
: null,
},
};
}
test("semantic Find All References excludes the definition even when includeDeclaration is set", async () => {
const scope = makeScope();
const provider = new Ra3ReferenceProvider(makeWs(scope));
const document = makeDocument();
const defLine = TEXT.split("\n")[1];
const defPos = new Position(1, defLine.indexOf('id="') + 4);
const refs = await provider.provideReferences(document, defPos, {
includeDeclaration: true,
}, {});
assert.ok(refs, "references are returned");
assert.equal(refs.length, 1, "only the typed content reference is returned");
assert.equal(refs[0].uri.fsPath, FILE);
assert.equal(refs[0].range.start.line, 4);
assert.equal(
TEXT.split("\n")[4].slice(refs[0].range.start.character, refs[0].range.end.character),
"Tank",
);
});
test("FAR from the reference site itself returns the same result", async () => {
const scope = makeScope();
const provider = new Ra3ReferenceProvider(makeWs(scope));
const document = makeDocument();
const contentLine = TEXT.split("\n")[4];
const contentPos = new Position(4, contentLine.indexOf("Tank") + 1);
const refs = await provider.provideReferences(document, contentPos, {
includeDeclaration: false,
}, {});
assert.equal(refs.length, 1);
assert.equal(refs[0].range.start.line, 4);
});
+58
View File
@@ -8,6 +8,8 @@ import {
isLocalReferenceAttribute, isLocalReferenceAttribute,
isReferenceAttribute, isReferenceAttribute,
isReferenceAttributeOfType, isReferenceAttributeOfType,
isReferenceContentType,
resolveContentReferenceTargets,
resolveReferenceTargets, resolveReferenceTargets,
resolveReferenceTargetsForType, resolveReferenceTargetsForType,
} from "../out/indexer/refs.js"; } from "../out/indexer/refs.js";
@@ -242,3 +244,59 @@ test("xi:include elements are outside the XSD model and unvalidated", () => {
assert.equal(model.isXsdAttributeName(a.name), true, a.name); assert.equal(model.isXsdAttributeName(a.name), true, a.name);
} }
}); });
test("typed simple content resolves like a typed attribute reference", () => {
// <CreateObject>CrateDebris_01</CreateObject> uses GameObjectWeakRef:
// the content is a GameObject reference, not a child element.
assert.equal(isReferenceContentType("GameObjectWeakRef"), true);
const idx = {
assetsById: new Map([
[
"cratedebris_01",
[
{
type: "GameObject",
id: "CrateDebris_01",
file: "Crates.xml",
line: 2,
origin: "project",
},
{
type: "WeaponTemplate",
id: "CrateDebris_01",
file: "Weapons.xml",
line: 4,
origin: "project",
},
],
],
]),
assets: new Map(),
projectDir: ".",
sdkDir: ".",
defines: new Map(),
files: new Map(),
streams: [],
manifests: new Map(),
sourceCandidates: [],
diagnostics: [],
stats: {},
};
const targets = resolveContentReferenceTargets(idx, "GameObjectWeakRef", "CrateDebris_01");
assert.equal(targets.length, 1);
assert.equal(targets[0].def.type, "GameObject");
});
test("untyped and pipeline-local content is not a global reference", () => {
// Generic AssetReference content is used for shader constants and model
// sub-object names, not global asset ids; Poid is pipeline-local.
assert.equal(isReferenceContentType("AssetReference"), false);
assert.equal(isReferenceContentType("Poid"), false);
assert.equal(isReferenceContentType("string"), false);
const idx = { assetsById: new Map(), assets: new Map(), defines: new Map() };
assert.equal(
resolveContentReferenceTargets(idx, "AssetReference", "Anything").length,
0,
);
assert.equal(resolveContentReferenceTargets(idx, "Poid", "Anything").length, 0);
});
+2 -2
View File
@@ -130,13 +130,13 @@ test("fallback tokens cover names, attributes and values", () => {
test("well-formed XML gets no semantic fallback tokens", async () => { test("well-formed XML gets no semantic fallback tokens", async () => {
const text = `<AssetDeclaration>\n <LocomotorTemplate id="x" Surfaces="GROUND"/>\n</AssetDeclaration>`; const text = `<AssetDeclaration>\n <LocomotorTemplate id="x" Surfaces="GROUND"/>\n</AssetDeclaration>`;
const provider = new Ra3SemanticTokensProvider(); const provider = new Ra3SemanticTokensProvider({ isRa3Workspace: () => true });
const result = await provider.provideDocumentSemanticTokens(makeDocument(text), {}); const result = await provider.provideDocumentSemanticTokens(makeDocument(text), {});
assert.equal(result.data.length, 0); assert.equal(result.data.length, 0);
}); });
test("malformed XML gets semantic fallback tokens", async () => { test("malformed XML gets semantic fallback tokens", async () => {
const provider = new Ra3SemanticTokensProvider(); const provider = new Ra3SemanticTokensProvider({ isRa3Workspace: () => true });
const result = await provider.provideDocumentSemanticTokens( const result = await provider.provideDocumentSemanticTokens(
makeDocument(MALFORMED), makeDocument(MALFORMED),
{}, {},
+122
View File
@@ -0,0 +1,122 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { scanXmlShallow } from "../out/indexer/shallowScan.js";
test("extracts top-level assets, includes, defines and xi:include", () => {
const text = `<?xml version="1.0"?>
<AssetDeclaration xmlns="uri:ea.com:eala:asset">
<Defines>
<Define name="MODEL_SCALE" value="1.0"/>
</Defines>
<Includes>
<Include type="all" source="Art/Model_SKN.w3x"/>
<Include type="reference" source="DATA:static.xml"/>
</Includes>
<W3DContainer id="Model_SKN" Hierarchy="Model_SKL">
<SubObject SubObjectID="GUN">
<RenderObject><Mesh>Model_SKN.GUN</Mesh></RenderObject>
</SubObject>
</W3DContainer>
<W3DMesh id="Model_SKN.GUN"/>
<xi:include href="DATA:Includes/Extra.xml" xpointer="xmlns(n=uri:ea.com:eala:asset) xpointer(/n:Extra/child::*)"/>
</AssetDeclaration>`;
const doc = scanXmlShallow(text);
assert.equal(doc.errors.length, 0);
assert.deepEqual(
doc.assets.map((a) => `${a.name}:${a.id}`),
["W3DContainer:Model_SKN", "W3DMesh:Model_SKN.GUN"],
);
assert.equal(doc.includes.length, 2);
assert.equal(doc.includes[0].type, "all");
assert.equal(doc.includes[0].source, "Art/Model_SKN.w3x");
assert.equal(doc.includes[1].type, "reference");
assert.equal(doc.defines.length, 1);
assert.equal(doc.defines[0].name, "MODEL_SCALE");
assert.equal(doc.defines[0].value, "1.0");
assert.equal(doc.rootXiIncludes.length, 1);
assert.equal(doc.rootXiIncludes[0].href, "DATA:Includes/Extra.xml");
assert.match(doc.rootXiIncludes[0].xpointer, /xpointer\(/);
assert.equal(doc.nestedXiIncludes.length, 0);
// Recorded offsets must slice the original text back out.
const container = doc.assets[0];
assert.equal(text.slice(container.idValueStart, container.idValueEnd), "Model_SKN");
assert.equal(text.slice(container.start, container.startTagEnd).startsWith("<W3DContainer"), true);
const mesh = doc.assets[1];
assert.equal(text.slice(mesh.idValueStart, mesh.idValueEnd), "Model_SKN.GUN");
});
test("handles > and / inside quoted values, self-closing tags and comments", () => {
const text = `<?xml version="1.0"?>
<!-- a > comment with < inside -->
<AssetDeclaration xmlns="uri:ea.com:eala:asset">
<W3DContainer id="Weird" Description="a > b < c" />
<W3DMesh id="X" VertexData="a / b"/>
</AssetDeclaration>`;
const doc = scanXmlShallow(text);
assert.equal(doc.errors.length, 0);
assert.deepEqual(
doc.assets.map((a) => a.id),
["Weird", "X"],
);
});
test("skips CDATA and processing instructions", () => {
const text = `<AssetDeclaration>
<!-- <W3DMesh id="FAKE"/> -->
<![CDATA[ <W3DMesh id="FAKE2"/> ]]>
<?xml-stylesheet href="x"?>
<W3DContainer id="REAL"/>
</AssetDeclaration>`;
const doc = scanXmlShallow(text);
assert.equal(doc.errors.length, 0);
assert.deepEqual(
doc.assets.map((a) => a.id),
["REAL"],
);
});
test("reports unterminated tags and keeps partial results", () => {
const doc = scanXmlShallow('<AssetDeclaration><W3DMesh id="A"/><W3DMesh id="B"');
assert.ok(doc.errors.length >= 1);
assert.deepEqual(
doc.assets.map((a) => a.id),
["A"],
);
});
test("nested module payload does not create asset records", () => {
// Only top-level elements with an id are assets; the hundreds of thousands
// of numeric V/T elements inside a mesh are not.
const doc = scanXmlShallow(`<AssetDeclaration>
<W3DMesh id="MESH">
<Vertices><V X="1" Y="2" Z="3"/><V X="4" Y="5" Z="6"/></Vertices>
<Triangles><T A="0" B="1" C="2"/></Triangles>
</W3DMesh>
</AssetDeclaration>`);
assert.deepEqual(
doc.assets.map((a) => a.id),
["MESH"],
);
});
test("distinguishes root-level and nested xi:include", () => {
const doc = scanXmlShallow(`<AssetDeclaration>
<xi:include href="DATA:Top.xml"/>
<W3DMesh id="MESH">
<SubObject>
<RenderObject>
<xi:include href="DATA:Nested.xml" xpointer="xmlns(n=uri:ea.com:eala:asset) xpointer(/n:X/child::*)"/>
</RenderObject>
</SubObject>
</W3DMesh>
</AssetDeclaration>`);
assert.deepEqual(
doc.rootXiIncludes.map((x) => x.href),
["DATA:Top.xml"],
);
assert.deepEqual(
doc.nestedXiIncludes.map((x) => x.href),
["DATA:Nested.xml"],
);
});
+112 -1
View File
@@ -1,6 +1,16 @@
import { test } from "node:test"; import { test } from "node:test";
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { parseXml, findElementAt } from "../out/language/xmlParser.js"; import {
parseXml,
findElementAt,
stripBom,
textContentTokenAt,
} from "../out/language/xmlParser.js";
test("stripBom removes a leading UTF-8 byte-order mark", () => {
assert.equal(stripBom("\uFEFF<A/>"), "<A/>");
assert.equal(stripBom("<A/>"), "<A/>");
});
test("parses elements, attributes and positions", () => { test("parses elements, attributes and positions", () => {
const text = `<AssetDeclaration xmlns="uri:ea.com:eala:asset">\n\t<GameObject id="X" KindOf="A B"/>\n</AssetDeclaration>`; const text = `<AssetDeclaration xmlns="uri:ea.com:eala:asset">\n\t<GameObject id="X" KindOf="A B"/>\n</AssetDeclaration>`;
@@ -39,6 +49,27 @@ test("findElementAt returns innermost element", () => {
assert.equal(at.name, "C"); assert.equal(at.name, "C");
}); });
test("findElementAt treats a completed element's end as exclusive", () => {
const text = `<A><B>X</B></A>`;
const doc = parseXml(text);
const b = doc.elements.find((e) => e.name === "B");
const at = findElementAt(doc, b.end);
assert.equal(at?.name, "A", "cursor after </B> belongs to the parent");
const self = `<A><B/></A>`;
const selfDoc = parseXml(self);
const b2 = selfDoc.elements.find((e) => e.name === "B");
const atSelf = findElementAt(selfDoc, b2.end);
assert.equal(atSelf?.name, "A", "cursor after <B/> belongs to the parent");
});
test("findElementAt still includes EOF inside an unclosed element", () => {
const text = `<A><B>C`;
const doc = parseXml(text);
const b = doc.elements.find((e) => e.name === "B");
assert.equal(findElementAt(doc, text.length)?.name, "B");
});
test("tolerates partial input while typing", () => { test("tolerates partial input while typing", () => {
const text = `<AssetDeclaration>\n\t<GameObject id="TestTank" Com`; const text = `<AssetDeclaration>\n\t<GameObject id="TestTank" Com`;
const doc = parseXml(text); const doc = parseXml(text);
@@ -62,6 +93,16 @@ test("recovers from an unterminated attribute value at end of line", () => {
assert.ok(b.start > a.start); assert.ok(b.start > a.start);
}); });
test("marks recovered start tags for partial completion re-parsing", () => {
const text = `<AssetDeclaration>\n <A x="1" y="abc\n <B/>\n </A>\n</AssetDeclaration>`;
const doc = parseXml(text);
const a = doc.elements.find((e) => e.name === "A");
assert.equal(a.recoveredStartTag, true);
// Well-formed elements parsed normally do not carry the marker.
const b = doc.elements.find((e) => e.name === "B");
assert.equal(b.recoveredStartTag, undefined);
});
test("reports an unterminated attribute value at EOF", () => { test("reports an unterminated attribute value at EOF", () => {
const text = `<A x="abc`; const text = `<A x="abc`;
const doc = parseXml(text); const doc = parseXml(text);
@@ -71,3 +112,73 @@ test("reports an unterminated attribute value at EOF", () => {
assert.equal(a.attrs[0].value, "abc"); assert.equal(a.attrs[0].value, "abc");
assert.equal(a.attrs[0].quoteEnd, -1); assert.equal(a.attrs[0].quoteEnd, -1);
}); });
test("textContentTokenAt returns the token inside element content", () => {
const text =
`<AssetDeclaration>` +
`<CreateObject> CrateDebris_01 </CreateObject>` +
`</AssetDeclaration>`;
const doc = parseXml(text);
const el = doc.elements.find((e) => e.name === "CreateObject");
const tokenStart = text.indexOf("Crate");
const cursor = tokenStart + 3;
const token = textContentTokenAt(text, el, cursor);
assert.deepEqual(token, {
value: "CrateDebris_01",
start: tokenStart,
end: tokenStart + "CrateDebris_01".length,
});
// The start tag, closing tag and whitespace-only content are not tokens.
assert.equal(textContentTokenAt(text, el, el.start + 1), null);
assert.equal(textContentTokenAt(text, el, text.indexOf("</CreateObject") + 1), null);
const empty = `<A><B/></A><C> </C>`;
const c = parseXml(empty).elements.find((e) => e.name === "C");
assert.equal(textContentTokenAt(empty, c, c.startTagEnd + 1), null);
});
test("textContentTokenAt works before the closing tag is typed", () => {
const text = `<A><B>C`;
const doc = parseXml(text);
const b = doc.elements.find((e) => e.name === "B");
assert.ok(b);
assert.equal(b.closeTagStart, -1);
const token = textContentTokenAt(text, b, text.length);
assert.deepEqual(token, {
value: "C",
start: text.indexOf("C"),
end: text.length,
});
});
test("a typed < in content followed by a closing tag does not swallow it", () => {
// In a real file the "<" the user just typed is followed by
// "</CreateObject>" on the next line. The parser must NOT treat that
// closing tag's ">" as the end of the malformed start tag (which would
// create a bogus empty-name element and break content completion).
const text =
`<AssetDeclaration>` +
`<ObjectCreationList>` +
`<CreateObject>\n\t<Offset/>\n\t<\n</CreateObject>` +
`</ObjectCreationList>` +
`</AssetDeclaration>`;
const doc = parseXml(text);
assert.ok(doc.errors.some((e) => /Unterminated start tag/.test(e.message)));
assert.ok(!doc.elements.some((e) => e.name === ""), "no bogus empty-name element");
const co = doc.elements.find((e) => e.name === "CreateObject");
assert.ok(co, "outer CreateObject still parsed");
assert.equal(co.end, text.indexOf("</CreateObject>") + "</CreateObject>".length);
});
test("a partial child name in content is recovered, not glued to the closing tag", () => {
const text = `<A><B>\n\t<Cr\n</B></A>`;
const doc = parseXml(text);
const cr = doc.elements.find((e) => e.name === "Cr");
assert.ok(cr, "partial name is recovered as an element shell");
assert.equal(cr.recoveredStartTag, true);
// The mismatched closing tag later closes the recovered shell (parser
// recovery), so the element stays a valid container for completion.
assert.equal(cr.closeTagStart, text.indexOf("</B>"));
assert.ok(cr.end > cr.startTagEnd);
const b = doc.elements.find((e) => e.name === "B");
assert.equal(b.end, text.length);
});