5 Commits
Author SHA1 Message Date
lanyi 47807f9fed 0.1.15 继续优化补全 2026-08-04 19:14:12 +02:00
lanyi f8187da611 0.1.13 improve completion 2026-08-04 17:53:11 +02:00
lanyi 6794896ac7 0.1.9 2026-08-04 14:28:26 +02:00
lanyi 44b2eaf273 0.1.1 2026-08-02 19:11:54 +02:00
lanyi 4cc07b9abc 支持 W3X 2026-08-02 14:32:18 +02:00
52 changed files with 7753 additions and 472 deletions
+1
View File
@@ -3,3 +3,4 @@ out/
dist/ dist/
*.vsix *.vsix
OpenSAGE/ OpenSAGE/
debug.log
+46 -7
View File
@@ -6,23 +6,51 @@
- **语法高亮**:在普通 XML 高亮之上叠加领域标记(`$DEFINE` 常量、`inheritFrom`、`xai:joinAction`、结构标签);XML 语法异常(如未闭合引号)期间由语义 token 兜底,标签/属性/值着色不中断。 - **语法高亮**:在普通 XML 高亮之上叠加领域标记(`$DEFINE` 常量、`inheritFrom`、`xai:joinAction`、结构标签);XML 语法异常(如未闭合引号)期间由语义 token 兜底,标签/属性/值着色不中断。
- **自动补全**: - **自动补全**:
- 元素名:按当前父元素的 XSD 模型补全子元素;顶层资产(`AssetDeclaration` 内)补全 `GameObject`、`WeaponTemplate` 等 295 种类型。 - 元素名:按当前父元素的 XSD 模型补全子元素;顶层资产(`AssetDeclaration` 内)补全 `GameObject`、`WeaponTemplate` 等 295 种类型。已输入 `<` 时补全保留该 `<`、只替换名称区(不会出现 `<<`);需要填文本的 simple-content 元素(如 `<CreateObject>`)补全为 `<CreateObject>$1</CreateObject>` 并自动弹出值补全,而不是无法填值的自闭合标签。
- 属性名:必填属性优先,附带类型/文档/默认值;自动提示 `xai:joinAction` 与 `xmlns:xai`。 - 属性名:必填属性优先,附带类型/文档/默认值;自动提示 `xai:joinAction` 与 `xmlns:xai`。接受补全时自动避免与上一个属性贴在一起,并按文件已有的排版补空格或换行(换行的基础缩进由编辑器提供,插件不再内嵌缩进以免叠加);数字/角度/时间等标量属性直接填入 XSD 默认值或类型示例(如 `0d`、`0s`),引用/枚举/布尔等保留真正的 `$1` 占位符并弹出值补全。
- 属性值: - 属性值:
- 引用型属性(如 `CommandSet`、`Weapon`)按 `xas:refType` 补全对应类型的资产 ID(**同名 ID 只补全匹配类型**); - 引用型属性(如 `CommandSet`、`Weapon`)按 `xas:refType` 补全对应类型的资产 ID(**同名 ID 只补全匹配类型**);
- `inheritFrom` 补全可继承的资产 ID; - `inheritFrom` 补全可继承的资产 ID;
- 枚举与位标志列表(如 `Include type`、`LocomotorTemplate@Surfaces`、`KindOf`;列表值支持空格后继续补全下一项); - 枚举与位标志列表(如 `Include type`、`LocomotorTemplate@Surfaces`、`KindOf`;列表值在空格后自动继续补全下一项,已使用的 flag 不再重复推荐,闭合值末尾可直接追加新 flag);
- 布尔值、`$DEFINE` 常量; - 布尔值、`$DEFINE` 常量;
- `<Include source>` 补全可解析的 `DATA:` / `ART:` / `AUDIO:` 与项目相对路径。 - `<Include source>` 补全可解析的 `DATA:` / `ART:` / `AUDIO:` 与项目相对路径。
- 元素文本内容:simple-content 引用元素(如 `<CreateObject>`、`<RequiredUpgrade>`、`<SpawnTemplate>`)直接在标签间补全对应类型的资产 ID(`GameObjectWeakRef` → GameObject)、枚举或 `$DEFINE`。
接受片段后的 `$1` 光标位置会立即弹出值补全,而不是属性名;引用列表超过
400 条时标记为不完整,继续输入会重新请求,因此 `CrateDebris_01` 这类排在
列表后部的 id 不会因首屏截断而消失。
- **悬停提示**:元素/属性显示 XSD 文档、类型、必填/默认值;引用值显示定义位置;`$DEFINE` 显示值与定义位置;`Include source` / `xi:include href` 显示解析后的目标文件;`xi:include` 元素与属性给出 XInclude 说明。 - **悬停提示**:元素/属性显示 XSD 文档、类型、必填/默认值;引用值显示定义位置;`$DEFINE` 显示值与定义位置;`Include source` / `xi:include href` 显示解析后的目标文件;`xi:include` 元素与属性给出 XInclude 说明。
- **引用导航**:从引用值(`CommandSet="..."`、`Weapon="..."`、`inheritFrom`)跳转到定义(严格按引用类型过滤,候选由 `ra3modxml.definitionMode` 控制:`all` 列出 mod + 原版、`project-only` 优先项目内定义);`Ctrl+点击` Include / `xi:include href` 打开目标文件;Find All References 搜索整个工作区;文档大纲列出顶层资产与 `$DEFINE`。 - **引用导航**:从引用值(`CommandSet="..."`、`Weapon="..."`、`inheritFrom`、`<CreateObject>ID</CreateObject>` 等元素文本)跳转到定义(严格按引用类型过滤,候选由 `ra3modxml.definitionMode` 控制:`all` 列出 mod + 原版、`project-only` 优先项目内定义);`Ctrl+点击` Include / `xi:include href` 打开目标文件;Find All References 同时搜索属性值与元素文本内容;文档大纲列出顶层资产与 `$DEFINE`。
- **错误检查**:XML 格式错误、未知元素/属性(`xi:` 等外来命名空间不误报)、顶层资产缺 `id`、重复 ID、未解析引用(含类型不匹配)、Include / 嵌套 `xi:include` 目标找不到、`$DEFINE` 未定义。 - **当前文档局部作用域(T1)**:即使一个文件不在任何全局流里(没有从
`Data/Mod.xml` / `additionalmaps` 可达),插件也会按当前文件自身的资产、
`$DEFINE` 及其 include 链建立局部索引。`xi:include` 会在逻辑树中展开,
使 include 进来的内容获得正确的父上下文;`AttachModuleId` / `ModuleId` /
`AutoResolveBody` 等管线局部(Poid)引用可以补全、悬停与跳转到同一
GameObject 内的模块(含通过 `xi:include` 拼入的兄弟模块)。
- **错误检查**:XML 格式错误、未知元素/属性(`xi:` 等外来命名空间不误报)、顶层资产缺 `id`、重复 ID、未解析引用(含属性值与元素文本内容、类型不匹配)、Include / 嵌套 `xi:include` 目标找不到、`$DEFINE` 未定义。
- **manifest 支持**:`<Include type="reference">` 指向的 `static/global/audio.manifest`(SDK `builtmods`)会被解析,manifest 中的原版资产 ID 可用于补全/悬停/导航/诊断。 - **manifest 支持**:`<Include type="reference">` 指向的 `static/global/audio.manifest`(SDK `builtmods`)会被解析,manifest 中的原版资产 ID 可用于补全/悬停/导航/诊断。
- **美术资产(`.w3x`)**:`W3X.xml` / `ART:` include 链中的 `.w3x` 模型文件会被
索引(`W3DContainer` / `W3DMesh` / `W3DHierarchy` 等顶层资产),因此
`Model@Name`、`Hierarchy`、`Mesh` 等引用可以解析、悬停与跳转。超大模型
(几十 MB 的顶点/三角形数据)采用浅扫描——只提取顶层资产记录、不建 DOM 树,
结果在 workspace 级缓存并跨重建复用,保存文件触发的重建不会重读未变化的模型文件。
- **索引分阶段与部分可用性**:先建立 XML + manifest 索引(首建早期即可用),
w3x 美术资产随后台扫描补齐。索引完成前,语法/模型诊断、枚举与子元素补全、
Include 跳转/悬停照常工作;引用类诊断会“显示但标注”
(`unresolved-reference-indexing` + `(index incomplete)` 说明),不会把
未完成的索引误当成最终结论。
- **大项目性能**:索引记录(资产 / Define / Include / 行号)与 include 解析结果
跨重建缓存,保存触发的重建零 stat、零重读(Corona 实测约 2 秒);DOM 树只按需
保留并设元素预算,避免内存膨胀。编辑器外的文件改动(git pull、导出工具)会
触发防抖重建;构建期间文件再次被修改时,已发布索引会标记 `(stale)` 并自动重跑。
include 路径解析使用目录枚举建立的文件集快照(无 statSync 风暴);records
缓存会持久化到磁盘(gzip + 多信号 stat 校验 + 原子写),重启 VS Code 后冷启动
只需秒级校验,Corona 实测约 11 秒(首次全量约 2 分钟)。
## 使用 ## 使用
1. 用 VS Code 打开 RA3 Mod 项目文件夹(含 `Data/Mod.xml` 或 `mod.babproj`)。 1. 用 VS Code 打开 RA3 Mod 项目文件夹(含 `Data/Mod.xml` 或 `mod.babproj`)。
2. 插件自动激活并开始后台索引(状态栏显示资产数量)。 2. 插件自动激活并开始后台索引(状态栏显示阶段与资产数量;扩展也会在打开
任意 XML 文件时激活,非 RA3 工作区不会显示 RA3 专属功能)。
3. 编辑任意 `*.xml` 即可获得补全、跳转与诊断。 3. 编辑任意 `*.xml` 即可获得补全、跳转与诊断。
### 设置(`settings.json`) ### 设置(`settings.json`)
@@ -40,6 +68,8 @@
- `RA3 Mod XML: Re-index workspace`:手动重建索引。 - `RA3 Mod XML: Re-index workspace`:手动重建索引。
- `RA3 Mod XML: Show index report`:查看索引统计。 - `RA3 Mod XML: Show index report`:查看索引统计。
- `RA3 Mod XML: Clear caches and rebuild`:清空内存/磁盘缓存并强制全量重建。
- `RA3 Mod XML: Show cache report`:查看磁盘缓存路径、大小、校验统计与命中数。
## 开发 ## 开发
@@ -69,10 +99,19 @@ src/
schemaModel.ts XSD 模型运行时(schema-model.json / asset-types.json 由 tools 生成) schemaModel.ts XSD 模型运行时(schema-model.json / asset-types.json 由 tools 生成)
indexer/ indexer/
includeResolver.ts Include 路径解析(纯 TS,移植 check_duplicate_ids.py) includeResolver.ts Include 路径解析(纯 TS,移植 check_duplicate_ids.py)
existence.ts 文件集存在性快照(目录枚举 Set,替代逐路径 statSync)
manifestParser.ts .manifest 二进制解析(移植 OpenSAGE ManifestFile.cs) manifestParser.ts .manifest 二进制解析(移植 OpenSAGE ManifestFile.cs)
fileScanner.ts 目录扫描与 Include source 候选 fileScanner.ts 目录扫描与 Include source 候选
refs.ts 引用目标解析(按引用类型过滤) refs.ts 引用目标解析(按引用类型过滤)
indexer.ts 工作区索引器(后台、缓存、增量重建) xpointer.ts xi:include xpointer 子集解析(纯 TS)
logicalTree.ts 当前文档逻辑树(xi:include 拼接、局部作用域)
localScope.ts 文档局部索引 overlay(自身链 + include 链)
shallowScan.ts .w3x 等大体积美术资产顶层浅扫描(纯 TS,不建 DOM)
records.ts 每文件紧凑索引记录(资产/Define/Include/xi + 行号)
caches.ts 跨重建持久缓存(DocumentCache / IndexRecordsCache /
IncludeResolveCache)+ 失效纪元 InvalidationsEpoch
diskCache.ts 跨会话磁盘缓存(gzip JSON、原子写、多信号 stat 校验)
indexer.ts 工作区索引器(后台、缓存、记录驱动重建、分阶段发布)
features/ completion / hover / navigation / diagnostics / semanticTokens features/ completion / hover / navigation / diagnostics / semanticTokens
syntaxes/ TextMate 注入语法 syntaxes/ TextMate 注入语法
tools/ XSD → 模型、AssetType 枚举提取 tools/ XSD → 模型、AssetType 枚举提取
-1
View File
@@ -1 +0,0 @@
[0801/032923.150:ERROR:third_party\crashpad\crashpad\util\win\registration_protocol_win.cc:108] CreateFile: 拒绝访问。 (0x5)
+919
View File
@@ -590,3 +590,922 @@ token:文档出现解析错误时,由插件用自己的容错解析树继续
配色,可能与 TextMate XML 配色略有差异——只在打字过程中出现,可接受; 配色,可能与 TextMate XML 配色略有差异——只在打字过程中出现,可接受;
- 未实现 `provideDocumentSemanticTokensEdits`(delta 版本),每次全量计算;单文件 - 未实现 `provideDocumentSemanticTokensEdits`(delta 版本),每次全量计算;单文件
解析在 KB 级,开销可忽略。 解析在 KB 级,开销可忽略。
---
## 十三、问题分析(第八轮,2026-08-02):`.w3x` 美术资产未被索引(AUGunship_SKN)
### 问题
AttachTest `Harbinger Gunship\GameObject.xml` 中 `<Model Name="AUGunship_SKN"/>`
报两条错误:
1. Problems 面板诊断:`Unresolved reference "AUGunship_SKN" (not found in the current index)`
(`unresolved-reference`,来自 `features/diagnostics.ts`);
2. 悬停提示:`No matching definition of type BaseRenderAssetType in the current index
(may exist in a compiled manifest or vanilla data).`(来自 `features/hover.ts`)。
两条是同一缺失定义的两种呈现(诊断 + hover),不是两个独立 bug。
`W3DContainer:AUGunship_SKN` 确实由 mod 定义——但定义在
`Harbinger Gunship\W3X\AUGUNSHIP_SKN.w3x` 里,而索引器只解析 `.xml`
(manifest 是 `*.manifest` 二进制,不存在 `.manifestxml` 源码格式)。
### 关键事实(实测)
1. **`.w3x` 是文本 XML**:文件头即 `<?xml ...?>` + `<AssetDeclaration>`,内容为
`<W3DContainer id="AUGUNSHIP_SKN" Hierarchy="AUGUNSHIP_SKL">` + `<SubObject>` 子树。
附带 UTF-8 BOM(抽样 292 个 Corona w3x:0 个 UTF-16,40 个带 BOM)。
2. **w3x 通过 `<Include type="all">` 链进索引**:`Mod.xml → … → W3X.xml → W3X/*.w3x`,
也常用 `ART:xxx.w3x`(SDK 根、项目 `Art` 等搜索路径)。索引器此前只把 w3x 登记为
文件(`stream.files`),从不解析,因此其顶层资产不在 `assetsById` 中。
3. **类型匹配本来是对的**:`Model@Name` 的 refType 是 `BaseRenderAssetType`,
`W3DContainer → BaseRenderAssetType` 可赋值(`isAssignableTo` = true)。缺的只是定义。
4. **manifest / SageXml 支持早已存在**(第二轮/第三轮),但该资产是 mod 自己的 w3x,
不在 `builtmods/*.manifest` 也不在 SageXml——hover 的 "may exist in a compiled manifest
or vanilla data" 只是通用兜底文案。
### 规模调查(Corona,D: 盘已连接)
- **3788 个 w3x,共 2.64 GB**;最大 22.8 MB;163 个超过原 4 MB 解析上限,11 个超 10 MB。
- 大文件结构符合"建模软件导出"模式:顶层是少量固定 W3D 资产
(`W3DHierarchy` + 若干 `W3DMesh` / `W3DContainer` / `W3DCollisionBox`,
有的还带 `<Includes>` 引用 `ART:*.xml`);体积大头是 `W3DMesh` 内的
`Vertices/V`、`Normals/N`、`TexCoords/T`、`Triangles/T` 等 `maxOccurs="unbounded"`
数值元素。例如 21.7 MB 的 `CBRefinery_BLD.W3X` 只有 22 个顶层记录,
Vertices+Triangles 块占约 12 MB(55%)。
- 全量建 DOM 的代价(实测 6.3 MB Aegis 文件):407 ms、193,651 个元素、
**堆内存 +109 MB(约 17 倍文本体积)**;22 MB 文件外推约 1.5 s + ~380 MB/份。
浅扫描(只取顶层记录):6.3 MB ~200 ms、22 MB ~600 ms,保留内存近似为零。
- **读整个文件不可避免,但建树不是**:浅扫描仍然是线性扫描全文(要知道顶层元素边界
就必须扫完),只是不分配子节点对象——所以"事后优化"完全有意义,且是必要项。
### 修复
1. **新增浅扫描模块** `src/indexer/shallowScan.ts`(纯 TS):
单次线性扫描,只提取顶层元素 `name + id`(含精确 offset)、顶层 `<Includes>`
的 `Include@type/source`、任意层级 `<xi:include>` 的 `href/xpointer`、`<Defines>`
常量;注释 / CDATA / DOCTYPE / PI 整体跳过;属性解析兼容引号内 `>` 与 `/`。
2. **索引模式三分**:`.xml` 全量解析(4 MB 上限不变);
`.w3x` 一律浅扫描;未知扩展名先嗅探文件头(512 字节,BOM/空白后以 `<` 开头且无
NUL 字节 → 按 XML 浅扫描,否则按二进制仅登记)。w3x 自身的 `<Includes>` 与
嵌套 `xi:include` 会继续被 walk(BAB 语义)。
3. **持久缓存**:`DocumentCache` / `ShallowScanCache` 移到 `src/indexer/caches.ts`,
由 `ModWorkspace` 持有并传入每次新建的 `ModIndexer`;读取时按 `mtimeMs + size`
校验,未变化不重读。这是 w3x 方案在 Corona 上可用的前提(否则每次保存都重读 2.6 GB)。
4. **BOM 剥离**:`xmlParser` 新增 `stripBom()`,全量解析与浅扫描前统一剥离,
保证第一行偏移与编辑器一致。
5. **Include source 补全候选**:DATA 目录与项目相对路径候选加入 `.w3x`
(`fileScanner`),`W3X/xxx.w3x`、`ART/xxx.w3x` 可补全。
6. 索引报告/状态栏新增 `shallowScannedFiles` / `shallowCacheHits` 统计。
### 验证
- 单元测试 **53 → 63 全绿**:新增 `test/shallowScan.test.mjs`(顶层资产 / Includes /
xi:include / Defines / 引号内 `>` / CDATA / 未闭合标签 / 数值载荷不产生记录);
indexer 新增 w3x 链、未知扩展名 XML 嗅探(fixture 用 `.dat`)、二进制 `.dds`
跳过、跨重建缓存命中、
BOM 偏移断言;xmlParser 新增 `stripBom` 用例。
- AttachTest 实机:
- 首次构建 1.2 s,浅扫 62 个文件;`Model@Name=AUGunship_SKN` →
`W3DContainer @ …\W3X\AUGUNSHIP_SKN.w3x:3`;`Hierarchy=AUGunship_SKL`、
`AUGunship_FP` 同样解析;资产数 35,546 → 35,607。
- 第二次构建 354 ms,0 次重扫,62 次缓存命中。
- Corona 实机(D: 盘):
- 首次构建 241 s:8,976 个文件、**浅扫 4,829 个**、资产 64,868
(manifest 35,322)、3 个流、183 个 Define。
- 第二次构建 38 s:0 次重扫、4,829 次缓存命中、资产数一致。
- `CBREFINERY_BLD` 正确解析到 `W3DHierarchy @ cb/CBRefinery_BLD.w3x:16` 与
`W3DContainer @ …:776412`。
### 边界与后续
- 首次建索引较慢(Corona ~4 分钟,机械盘):读 2.6 GB 是下限,浅扫描本身 ~30-90 s;
后续可做并行扫描或把 w3x 顶层记录做成独立小缓存文件。
- 第二次构建仍有 ~38 s(主要是对 ~9k 文件逐文件 stat + 重建 Map,机械盘随机读);
后续可做"文件清单 + stat 快照"级缓存。
- 浅扫描对 w3x 不做 XSD 校验(编辑器特性不注册 w3x 语言);若用户手动把 `*.w3x`
关联为 xml,完整解析仍会发生在打开的文档上(VS Code 自身行为)。
- UTF-16 XML 的 w3x 会被嗅探判定为二进制(文件头含 NUL);实测生态中不存在,
如遇可再扩展解码。
---
## 十四、问题分析(第九轮,2026-08-02):Corona 重建性能与内存
### 目标与基线
第八轮后 Corona 的实测基线:
| 场景 | 耗时 |
|---|---|
| 首次全量建索引 | ~180-290s(机械盘波动) |
| 信任二次构建(保存触发) | 8-21s |
| 强制 reindex | 26-38s |
| 首建后保留堆 | ~2.5GB(原因不明) |
### 调查 1:二次构建为何仍有 8-21s
插桩 `fs.statSync` 后发现:**每次构建(含信任重建)都会执行约 11 万次同步 statSync**,
全部来自 `resolveSource` 的 include 目标存在性检查(BAB 搜索路径逐 base `statSync`)。
机械盘上这就是 10-20s 的来源——即使文件内容全部命中缓存,include 解析仍在逐路径 stat。
修复:新增 `IncludeResolveCache`(workspace 级、跨重建复用):
- 键 = 当前文件目录 + source 字符串;命中直接返回,不 stat;
- 内容编辑不影响"文件是否存在",因此保存触发的重建**不清除**解析缓存;
- 文件创建/删除(watcher `onDidCreate` / `onDidDelete`)与 `ra3modxml.reindex`
(强制校验)清除缓存;
- `reference` 的 manifest 查找(`builtmods/*.manifest` 存在性)同样缓存;
- 配置变更(搜索路径 / builtmods 目录)时清除。
### 调查 2:首建后保留堆 2.5GB 是什么
逐级清空测量(构建 → 清 DOM 缓存 → 清 records → 清 walker → 清索引 Map):
- `DocumentCache`(元素预算 1M、64 条):实际只保留 64 条 / 1788 个元素 ≈ 1MB;
- `IndexRecordsCache`(8976 个文件的紧凑记录):约 11MB;
- 目录 walker:约 4MB;
- 全部索引 Map(manifests + assets + assetsById + defines + files + streams +
candidates + diagnostics):约 75MB;
- 全部清空并强制 GC 后,堆回到基线(0MB 保留)。
结论:2.5GB 是**构建期可回收垃圾**(60MB XML 的 DOM 瞬态 + 2.6GB w3x 扫描文本/行映射
的分配压力),不是常驻泄漏;VSCode 正常 GC 压力下会回收。扩展常驻内存约为
基线 + ~100MB。顺带修复了一个潜在常驻风险:w3x 的 `LineMap`(Corona 全量约 700MB)
不再随 `ShallowScanCache` 保留,浅扫描结果以"带行号的紧凑记录"形式缓存。
### 其他改动
1. **`IndexRecordsCache` 取代 DOM 依赖的重建**:新增 `src/indexer/records.ts`,
每个文件解析时提取紧凑索引记录(顶层资产 / Define / Include / xi:include +
1-based 行号),跨重建缓存;信任重建完全不接触 DOM。
2. **`DocumentCache` 双重淘汰**:条数(LRU)+ 元素预算(超预算先淘汰最大树),
把 DOM 常驻内存封顶;DOM 只服务于按需特性(跳转精确范围等)。
3. **候选目录扫描并行化**(`collectSourceCandidates` 各目录 `Promise.all`)。
4. **阶段计时**:`stats.candidatesMs` / `stats.walkMs` / `stats.resolveCalls` /
`stats.resolveCacheHits` 进入索引报告,便于后续定位耗时。
5. 版本 0.1.0 → **0.1.1**。
### 实测(Corona,D: 机械盘)
| 场景 | 优化前 | 优化后 |
|---|---|---|
| 首次全量建索引 | ~180-290s | ~250s(含 2.6GB w3x 读取+浅扫描、~7.6 万次冷 statSync) |
| 信任二次构建 | 8-21s | **2.0s**(statSync 0 次,resolveHits 15,333) |
| 强制 reindex | 26-38s | ~5s(保留解析缓存时);显式命令会清解析缓存,约 15-25s |
| 首建后保留堆 | ~2.5GB(疑为泄漏) | 确认是可回收垃圾;常驻 ~100MB |
单元测试 **63 → 73 全绿**:新增 `records.test.mjs`(DOM→记录提取、浅扫描→记录)、
`caches.test.mjs`(元素预算淘汰、LRU、records/resolve 缓存),indexer 测试补充
resolve 缓存命中与"信任重建 0 次重解析"断言。
### 遗留
- 首次全量建索引仍受限于机械盘读 2.6GB + 冷 statSync(~4 分钟);后续可考虑
并行浅扫描(worker)或把 w3x 顶层记录持久化到磁盘缓存。
- `ra3modxml.reindex` 出于正确性会清空解析缓存(可能 ~15-25s);如接受 watcher
可靠性可改为保留。
---
## 十五、问题分析(第十轮,2026-08-03):索引未完成时的部分可用性与分阶段索引
### 目标
第九轮遗留:Corona 首次全量建索引约 4 分钟(2.6GB w3x 读取 + 冷 statSync),
期间插件所有功能被整体关闭(`ws.index == null`)。本轮让插件在索引完成前
分层可用,并把最耗时的 w3x 扫描放到后台阶段。
### 现状证据
- `diagnostics.update` 无索引时直接清空诊断:连 XML 语法错误、未知元素、缺 id
这类不依赖索引的检查也被关闭;
- 补全的 `attribute-value` / `content` 分支无索引直接返回空,但枚举 / 布尔 /
Include type / 子元素补全并不需要索引(`contentItems` 甚至没用 idx);
- hover / 文档链接的 Include 解析只依赖搜索路径(settings),不需要索引;
- Find All References 完全不使用索引。
### 设计
1. **索引状态模型**:`ModIndex` 增加 `complete` / `phase`(`"xml" | "art"`)/
`stale`;`stats` 增加 `phase` / `complete` / `deferredArtFiles` /
`artScanMs`。
2. **分阶段索引**:
- 阶段 A(xml):walk include 链时只**登记** w3x / 嗅探 XML 文件
(`readDocument` 的 `deferArt` 模式),不读内容;manifest、项目 XML、
mapmetadata 全部正常索引;结束后发布不可变快照;
- 阶段 B(art):按队列浅扫描 w3x、应用资产记录,并继续走 w3x 内的
Include / xi:include(此时新遇到的美术文件立即扫描,不再入队);
- 快照不可变性的关键:`addAsset` 在阶段 B 仍会向数组 push,所以阶段 A
发布时必须复制 `assets` / `assetsById` / `defines` / `streams.files` /
`diagnostics`(`snapshotIndex` 深拷贝嵌套 Map/数组)。
3. **部分可用性(T0 解耦)**:
- 诊断:语法错误、未知元素/属性、缺 id、同文件重复 ID、Include 目标存在性
不再依赖索引;引用 / `$DEFINE` / 跨文件重复在索引未完成或 stale 时
“显示但标注”:code 变为 `unresolved-reference-indexing` /
`undefined-define-indexing`,消息追加 `(index incomplete — may be a
false positive)`;跨文件重复追加 `(based on a partial index)`;
- 补全:枚举 / 布尔 / Include type / `xai:joinAction` / 子元素(content)
在无索引时可用;资产 ID / define / include source 仍需索引;
- hover / 文档链接:Include source / `xi:include href` 用 settings 搜索路径
即可解析;无索引时引用值 hover 提示 “Index is still building”。
4. **构建中文件被修改**:
- watcher 的 change / create / delete 现在都会 `scheduleRebuild()`(此前
只 `invalidate`,外部修改后没有任何东西触发重建);
- 新增 `InvalidationsEpoch`:`invalidate` / `invalidateExistence` 递增;
构建开始时记录 epoch,发布每个快照(含最终)时若 epoch 变化则标记
`stale`(状态栏显示 `(stale)`);dirty 机制保证构建结束后立即再重建收敛;
- 构建失败时保留上一个可用快照并标记 stale,不再清空索引。
5. **stat 结构扩展**:`IndexedFile.stat` 从 `{ mtimeMs, size }` 变为
`{ mtimeMs, size, birthtimeMs, ctimeMs }`,为磁盘持久化缓存铺路
(FAT32 的 mtime 只有 2 秒粒度,多信号可捕捉“保留 mtime 的整体替换写入”)。
### 验证
- 单元测试 **73 → 79 全绿**:
- indexer:phase-A 快照发布(XML/manifest 资产可用、美术资产缺席)、快照
不可变性、`deferredArtFiles` 统计、`artScanMs`、mtime 变更强制重读;
- caches:`InvalidationsEpoch` 递增 / 快照语义;
- completion:无索引时枚举 / 元素名 / 属性名 / 子元素补全仍工作。
- `tsc` + esbuild 构建通过。
### 实机复现与补充修复(2026-08-03)
在 Corona 上实测新版索引器:
- 阶段 A(xml)**27.0s** 发布:54,283 资产(含 manifest 35,322)、8,399 文件、
4,797 个 w3x 待扫;最终 **118.0s**:64,868 资产、8,976 文件、4,829 浅扫
(artScanMs ≈ 91s,walkMs ≈ 26s)。
- `OnSeaUnitCrate.xml` 中的引用在阶段 A 即可解析(`Locomotor=JapanEggLocomotor`
、`CommandSet=EmptyCommandSet` 均有候选);`LocomotorSet@Condition` 是
19 个枚举值的模型属性,无索引时即可补全。
由此确认两个真实体验问题并修复:
1. **状态栏初始不显示 indexing**:`workspaceContains` 激活事件在大目录上扫描
较慢,打开工作区后扩展尚未激活,看起来“没有任何功能”。`activationEvents`
增加 `onLanguage:xml`,打开任意 XML 文件即激活;同时各 provider 增加
`isRa3Workspace()` 守卫,避免在非 RA3 工作区误补全/误诊断。
2. 索引构建中执行 “Show index report” 提示 “no index available” 有误导 →
新增 `ws.isBuilding`,构建中改为提示 “index is still building”。
版本 **0.1.1 → 0.1.2**(重新打包 `ra3-mod-xml-0.1.2.vsix`)。
---
## 十六、问题分析(第十一轮,2026-08-03):冷启动磁盘缓存与首建 statSync 消除
### 目标
第十轮后 Corona 首次建索引仍需 ~2 分钟,且每次新会话(重启 VS Code)都要
重来一遍。本轮做两件事:
1. **文件集快照替代 statSync**:`resolveSource` 的逐 base `statSync` 存在性
检查(Corona 首建约 11 万次)改为目录枚举建立的 `Set` 查询;
2. **磁盘持久化缓存**:records 缓存(含 w3x 浅扫记录)跨会话落盘,冷启动
只做 stat 校验,不再重读 2.6GB 美术资产。
### 设计
1. **ExistenceSnapshot**(`src/indexer/existence.ts`):
- **惰性按目录 readdir**:只在实际查询某个候选路径时读取它的父目录
(`readdirSync` + `withFileTypes`,不 stat 单个文件),结果按目录缓存;
不做首建前的全量递归枚举(该方案曾让 XML 阶段从 27s 涨到 45s);
- 覆盖根判定:候选父目录落在根内 → 目录条目 Set 查询(`hits`);落在
根外 → `statSync` 回退(`fallbacks`);
- 盘符根(`C:\` 等)与不存在的根不枚举,避免遍历整个磁盘;
- 子根被更宽的根覆盖时跳过(如 `sdkDir` 覆盖 `sdkDir/SageXml`)。
2. **DiskRecordsCache**(`src/indexer/diskCache.ts`):
- gzip JSON,原子写(tmp + rename),版本号 + identity key
(项目/SDK/设置哈希,配置变化自动忽略旧缓存);
- 每条记录保存多信号 stamp `{ size, mtimeMs, birthtimeMs, ctimeMs }`;
- 加载时并发(32)stat 校验,不匹配/缺失丢弃,构建时重读;
- 缺失/损坏/身份不符 → 空结果,不报错。
3. **workspace 集成**:
- 构建前若内存 records 缓存为空,从磁盘加载并校验(状态栏显示
“validating cache…”);构建成功后异步回写(entries 先快照,避免与
下一次构建竞争);
- 新命令 `ra3modxml.clearCache`(清内存 + 删磁盘文件 + 强制重建)与
`ra3modxml.showCacheReport`(缓存路径/大小/加载校验统计/命中数)。
### 实测(Corona)
| 场景 | 结果 |
|---|---|
| 首次构建 | 118.7s(phaseA **24.0s**);snapshotHits **75,926**、fallbacks **0**、resolveCalls 11,061 |
| 保存缓存 | 0.1s;gzip 后 **651 KB** |
| 加载 + stat 校验 | 0.2s(8,976 条全部校验通过) |
| 新会话二次构建 | **10.9s**(w3x 重扫 0、shallowCacheHits 4,829、recordsCacheHits 4,166) |
冷启动(加载 + 校验 + 构建)约 **11s**,对比之前每次 ~2.5 分钟。
### 测试与验证
- 单元测试 **79 → 90 全绿**:
- `existence.test.mjs`:覆盖/未覆盖判定、hits/fallbacks、盘符根识别、
搜索 base 枚举、`resolveSource` 快照命中与 statSync 回退;
- `diskCache.test.mjs`:roundtrip、原子写无残留 tmp、stat 变更丢弃、
identity 不符忽略、损坏文件空结果、clear、key 稳定性;
- `caches.test.mjs` 增加 `entries()`;indexer 统计断言 snapshotHits。
- `tsc` + esbuild + `vsce package` 通过。
### 遗留
- 首次构建 phase A 现在包含快照目录枚举成本(Corona 实测约 +10-17s,后续
构建由 walker 缓存吸收);如 SDK 根目录特别大可再优化根覆盖策略。
- w3x 并行/流水线浅扫描仍未做(HDD 收益存疑,SSD 再做)。
### 补充(用户实测反馈,2026-08-03)
用户在 0.1.3 上删除 workspace storage 缓存后实测:
- XML 阶段约 45s(比之前 27s 多)→ 根因是磁盘缓存轮实现的**全量目录枚举
快照**在首建前递归枚举 SDK 根等搜索根。已改为惰性按目录 readdir,复测
phase A **24.0s**,statSync 消除效果不变(75,926 次快照命中、0 回退)。
- “show index report 显示 Indexed in 1.1s、0 浅扫、8995 缓存命中”与观察
不一致 → 首建完成后又发生了一次信任重建(follow-up)。为定位触发源,
新增**重建插桩**:`buildCount` / `lastBuildTrigger`(initial、save、
watcher-create/change/delete、config、reindex-command、clear-cache、
dirty-followup)+ “RA3 Mod XML” 输出通道(每次构建记录 trigger、phase A
发布时间、完成耗时);索引报告与缓存报告均显示构建序号与触发原因。
### 补充 2(0.1.4 日志复现,2026-08-03)
0.1.4 输出通道日志:
- build #1:phase A 31.4s、done 120.1s、**stale=true**;
- build #2:`dirty-followup (initial)`,0.6s(报告计时不一致的直接来源);
- build #3/#4/#5:每 ~35s 一次 `watcher-change`,每次 0.5s 重建。
build #1 的 stale=true 与周期性 watcher-change 均不符合预期:说明有后台进程
在周期性触碰被监视目录(最可疑是 `.git` 内部文件,如后台 fetch/maintenance)。
处理:
1. watcher 事件现在把**触发 URI 写入输出通道**(`[watcher-change] <path>`),
可直接定位是哪个文件/目录在变化;
2. 新增 `isWatcherNoisePath`:路径含 `.git` 段的事件直接忽略(不 invalidate、
不标记 stale、不触发重建);单元测试 90 → 91。
版本 **0.1.4 → 0.1.5**。
### 补充 3(0.1.5 日志复现,2026-08-03)
0.1.5 日志中周期性重建已消失,但首建期间仍有一次:
```
[watcher-change] d:\...\corona\Data\Neutral\Crate\UnitCrate.xml.git
```
该文件**并不存在**(疑似其他 VSCode 扩展产生的瞬时临时文件),且它是
`*.xml.git` 文件名,不是 `.git` 目录段,绕过了上一轮过滤;它也导致 build #1
stale=true 并触发 follow-up。处理(按用户建议的扩展名白名单思路):
1. `isWatcherNoisePath` 增加临时文件命名模式:`.git` / `.tmp` / `.lock` /
`~` / `.swp` / `.bak` / `.orig` 后缀,以及 `.#` / `.~` 前缀;
2. `onDidChange` 只响应**内容相关**文件:扩展名白名单(`.xml` / `.w3x`;
领域修正:RA3 合理文本格式为 xml/w3x/lua,lua 暂未索引,manifest 为
`*.manifest` 二进制,不存在 `.manifestxml`)或已在当前索引中的文件
(`ModIndexer.isIndexedFile`,覆盖被嗅探为 XML 的未知扩展名);纹理等
二进制内容变更不触发重建;
3. 创建/删除仍对所有真实文件响应(影响 include 存在性),临时文件模式除外。
测试 91 → 92;版本 **0.1.5 → 0.1.6**。
### 遗留(此处的磁盘缓存与文件集快照已在第十一轮完成,T1 已在第十二轮完成)
- w3x 文件名启发式定向扫描(按约定后续再做)。
- `AssetIdList` 等“任意资产 ID 列表”的引用语义建模。
- Find All References 目前仍是全文搜索,未走索引(对应需求 P1 高效搜索)。
---
## 十七、问题分析(第十二轮,2026-08-03):当前文档局部链与 include 逻辑树展开(T1)
### 目标
上一轮遗留的 T1:让**不在任何全局流里的文件**也能解析自身引用,并为 GameObject
内模块 `id` 的局部作用域(`AttachModuleId` / `ModuleId` / `AutoResolveBody` 等)
铺路。两件事一起做:
1. **T1a 文档局部 overlay**:当前打开的文档(含未保存文本)自身资产 / `$DEFINE`
及其 include 链进入一个轻量局部索引,与全局索引叠加使用;
2. **T1b 逻辑树展开**:`xi:include` 按现有 `xpointer` 子集拼入当前文档的逻辑树,
使 include 进来的内容获得正确的父上下文,并让 Poid 引用能在同一 GameObject
子树内解析。
### 现状证据
- 全局索引只从 `Data/Mod.xml` 与 `additionalmaps/mapmetadata_*.xml` 出发;
`Data/Standalone.xml` 这类未进流的文件,其自身 `GameObject` / `$DEFINE` 不会
出现在 `ws.index`,`inheritFrom`、`CommandSet` 全部无法解析;
- `xi:include` 此前只做到“目标文件可索引 / 缺失可诊断”,没有拼入当前文档树;
include 进来的 `TruckDraw` 等模块拿不到 `Draws` 子元素的上下文类型;
- `isLocalReferenceAttribute` 对所有 Poid 属性一律“不检查、不解析”,导致
同一 GameObject 内完全可静态判断的模块引用也没有 hover / 跳转 / 补全。
### 设计
1. **共享 `xpointer.ts`**:`localName` / `findXPointerContainer` 从 indexer 迁出,
全局索引与局部展开共用同一份 xpointer 子集实现。
2. **`logicalTree.ts`**:
- 按解析器扁平元素表预建逻辑节点壳(保留原始 `sourceFile` 与偏移),再按
真实根 + 容错孤儿根遍历,重建 parent/child 链;
- `xi:include` 解析 href → `readDom` 目标 → 按 `xpointer` 选择容器子节点 →
拼入逻辑父元素;目标缺失 / 环 / 超深时跳过(环与深度用 visited + 64);
- 保留 xi 节点本身在 `elements` 中,hover 仍可解释 XInclude。
3. **`localScope.ts`**:
- `buildDocumentScope` 返回原始 parse、逻辑树、per-source LineMap、局部
overlay 与 overlay-aware merged index;
- overlay 沿 `<Include type="all/instance">` 与 `xi:include` 递归收集资产 /
Define;`reference` 指向 manifest,资产由全局索引提供,不重复解析;
- `withLocalOverlay` 不复制全局 Map(Corona ~65k 资产),只把 overlay 挂到
`ModIndex.local`,查询函数“局部优先、全局兜底”。
4. **workspace 集成**:`getScope(document)` 按 URI + 文档 version + 全局索引
代次缓存;全局索引发布 / 文件关闭时失效;首个 indexer 创建前的兜底读取走
`fallbackRead`(小 XML 直接解析)。
5. **features 接入**:
- diagnostics / hover / navigation / completion 改从 `getScope` 取逻辑树与
merged index;诊断只上报 `sourceFile === 当前文件` 的节点;
- 引用解析 / 补全 / Define 查询支持 `idx.local` 优先;
- Poid 属性:`AttachModuleId` / `ModuleId` / `UpdateModuleId` 等可在最近
GameObject 子树内补全、hover、跳转;未命中**不新增诊断**(保守,避免
“武器引用另一 GameObject 模块”这类跨文件语义误报);
- 导航对当前未保存文件优先用编辑器文本定位,再回退磁盘 DOM。
### 验证
- 单元测试 **92 → 98 全绿**:
- `localScope.test.mjs`:未进流文件 overlay(自身资产 / Define / instance
include 链)、inheritFrom 局部解析、xi:include 展开后模块上下文类型、
Poid 引用找到 include 进来的兄弟模块、局部定义优先于全局同名定义、
xi:include 环终止;
- `completion.test.mjs`:Poid 属性只补全所在 GameObject 子树内的 id。
- `tsc` / `esbuild` 构建通过(打包验证见版本发布步骤)。
### 边界与后续
- 顶层 `<Include type="all">` 仍**不**并入逻辑树:它通常是独立整文件或大体积
w3x,展开收益与风险不成比例;如需要“当前文档视角的全量合并诊断”再单独做。
- `AttachModuleId` 若出现在独立 `WeaponTemplate`(不在 GameObject 子树内),
本轮仍不解析;等真实样本确认跨文件语义后再扩展。
- 未命中 Poid 不报诊断是刻意保守,后续可加配置项开启。
版本 **0.1.7 → 0.1.8**。
### 补充:构建期局部作用域闸门(2026-08-04)
用户清空缓存后在 Corona 上测得 phase A **101.2s** / 完成 **254.6s**,明显高于
第十一轮冷建基线(phase A 约 24s / 总约 118s)。代码审查确认 T1 没有改动 indexer
的构建路径(`localScope` / `logicalTree` 不参与 `build()`),但自动诊断在构建中
会触发 `getScope()`,可能和 indexer 抢磁盘 / CPU。
修复:`getScope()` 在 `building === true` 时返回 **parse-only 轻量 scope**
(只解析当前文件,不沿 include 链读盘、不展开逻辑树);构建完全结束后再刷新
全量局部 scope。这样首建期间诊断 / 补全仍可用,但不会拖慢索引。索引报告与
输出通道同步增加 `walk / candidates / art` 耗时分解,便于下次直接定位慢在哪一段。
版本 **0.1.8 → 0.1.9**。
---
## 十八、问题分析(第十三轮,2026-08-04):bit-flag 列表补全的三层问题
### 现象
对 `xs:list` 枚举(bit flag,如 `CreateObject@Disposition`、
`LocomotorTemplate@Surfaces`):
1. 刚打开引号时(`Disposition="`)能补全;
2. 输入第一项后再输入空格,**不触发**补全;
3. 已经闭合的 `Disposition="RANDOM_FORCE RELATIVE_ANGLE"` 想在中间插入或末尾
追加 flag,**不触发**补全。
### 根因(三个独立问题)
**1. 空格没有注册为补全触发字符**
`extension.ts` 注册 provider 时只传了 `< " = : . /`。VS Code 只在输入 word
字符或已注册触发字符时自动弹出补全;空格两者都不是,所以“打 flag → 空格”
永远不会自动弹出。刚打开引号能补全正是因为 `"` 已注册。
**2. 多行未闭合引号下,解析恢复丢失后续行属性**
解析器对未闭合开始标签的恢复策略是“截到第一个换行”(`xmlParser.ts`),因此
像用户示例这样属性逐行书写的标签,`<CreateObject` 之后各行的 `Options` /
`Disposition` 全部丢失,`startTagEnd` 停在 `<CreateObject` 行尾。光标在后续
行时 `analyzeContext` 走 `content` 分支——实测 `kind: content, attrs: []`。
也就是说,**按用户贴出的原文状态,当前代码其实并不会补全 Disposition**;
观察到“能补全”的编辑状态里引号/`>` 多半已闭合。另外恢复分支没有把恢复出的
元素挂到父元素下(`parent = null`),即使补上上下文分析,类型解析也会退回
全局映射(`CreateObject` → `GameObjectWeakRef`)而找不到 `Disposition`。
**3. 闭合引号内“插入/追加 flag”的体验与范围问题**
- 光标在完整值末尾(`...RELATIVE_ANGLE|"`)时,当前 token 恰好等于完整枚举
值,`startsWith` 过滤只剩它自己 → 看起来“没有补全”;
- 替换范围 bug:引号闭合时 `endOffset` 固定取整个值的末尾而不是光标位置。
实测光标在 `RANDOM_FORCE | RELATIVE_ANGLE` 中间时 range 为 `(28..42)`,
选中任意 flag 会删掉 `RELATIVE_ANGLE` 及之后的内容;
- 光标恰好贴在闭合引号后面(`"|`)时,`offset <= quoteEnd` 把引号算进
prefix(`RELATIVE_ANGLE"`),返回 0 项。
### 修复
1. **触发字符**:`registerCompletionItemProvider` 增加 `" "`(空格)。副作用:
属性之间按空格会弹属性名补全(加分项),文本内容按空格会弹子元素补全。
2. **多行未闭合标签的补全**:
- 解析器给恢复出的元素打 `recoveredStartTag` 标记,并**补挂父链**
(与正常分支一致,`parent.children.push` + `el.parent = parent`);
- `analyzeContext` 在光标越过 `startTagEnd` 且元素带标记时,把
`text.slice(tagStart, cursor)` 作为部分标签重新 `parseTag` 一次,
再走同一套 start-tag 分类。全局解析恢复策略不变,后续文档解析/诊断
不受影响。
- 引号判定从 `offset <= quoteEnd` 改为 `offset < quoteEnd`:光标在闭合
引号之后进入 attribute-name 上下文。
3. **list 补全范围与过滤**(`completion.ts`):
- list 值替换范围改为 `min(cursor, valueEnd)`,只覆盖当前段;非 list
保持整值替换;
- 排除列表中已出现的 flag(空格后只推荐剩余项);
- 追加模式:当前段已是完整枚举值且没有其它枚举以它为前缀时,给出零宽
range、`insertText = " FLAG"`,可直接在闭合值末尾/列表中间追加;
前缀保护是必要的——实测 820 个 list 枚举里有 10569 对严格前缀关系
(如 `CAN_ATTACK` → `CAN_ATTACK_WALLS`)。
### 举一反三的测试(98 → 107 全绿)
- `xmlParser.test.mjs`:恢复元素带 `recoveredStartTag` 标记、正常元素不带;
- `context.test.mjs`:用户示例的多行未闭合引号(`Disposition="`)仍为
attribute-value 且 prefix 正确;输入 flag + 空格后 prefix 含完整值;闭合
引号之后为 attribute-name;
- `completion.test.mjs`:
- 空格后只推荐未使用的 flag(10 项,不含 GROUND),range 零宽在光标处;
- 列表中间插入不会删掉尾部 flag(range 止于光标);
- 闭合值末尾完整 flag → 追加模式(`insertText: " WATER"`);
- `CAN_ATTACK` 有更长变体时保持前缀过滤,不进入追加模式;
- 多行未闭合 `Disposition="RANDOM_FORCE ` 经完整 provider 链路返回剩余
flag(不含 RANDOM_FORCE)。
版本 **0.1.9 → 0.1.10**。
---
## 十九、问题分析(第十四轮,2026-08-04):属性补全的插入体验
### 现象
写完一个属性值并关闭引号后,会立刻触发下一个属性的补全菜单(由 `"` 触发字符
带来,方便)。但按 Enter 接受补全时不会补空格,结果属性与上一个属性的闭合
引号贴在一起:
```xml
Disposition="RANDOM_FORCE RELATIVE_ANGLE"Count="$1"
```
连续接受会变成 `...RELATIVE_ANGLE"Count="$1"CreateFX="$1"DestinationPlayer="$1"`。
另外提出两个功能请求:
1. 新属性自动参考临近属性的缩进(很多 XML 的属性统一换行缩进);
2. 补全的 `$1` 占位符在允许时变成更有意义的值(数字 → 数字、角度 → 角度、
时间 → 时间),顺带提示值的格式。
### 修复
**1. 插入布局(`completion.ts` 新增 `attributeInsertLayout`)**
- 光标紧贴上一个属性的闭合引号时,插入文本前补一个空格(inline 布局);
- 临近属性是“一行一个”布局(相邻属性之间的原文含换行)时,插入
`\n + 上一个属性的缩进`;
- 用户已经回车换行时,用临近缩进替换当前行已有的空白(对齐);
- inline 风格的文件里用户手动换行,则保留用户自己打的缩进,不强改;
- `xai:joinAction` / `xmlns:xai` 辅助项同样享受前缀与触发。
**2. 类型化默认值(`completion.ts` 新增 `attributeValuePlaceholder`)**
- 引用 / 枚举 / list / 布尔 / `inheritFrom` / `Include@source` / `id` 保留
`$1` 占位并自动触发值补全(这些值靠候选选择,不能瞎猜);
- 标量属性优先用 XSD `default`(如 `Count="1"`),没有默认值时按类型给示例:
`Angle → 0d`、`Time → 0s`、`Percentage → 100%`、`Velocity → 0.0`、
`SageReal/float → 0.0`、`SageInt/unsigned → 0`;
- 填了具体默认值后不再弹空的 suggest 窗口;`allowsDefine` 的数值属性现在也
直接给数值示例(`$DEFINE` 仍可在值内手动触发补全)。
### 举一反三的测试(107 → 111 全绿)
- 闭合引号后接受属性 → ` Count="1"`(空格),range 零宽在光标处;
- 一行一个属性 → `\n Count="1"`(换行 + 缩进);
- 已在新行 → range 覆盖当前行空白,插入 ` Count="1"` 对齐;
- 标量默认值:`Count="1"`(XSD 默认)、`FadeTime="0s"`、`DispositionAngle="0d"`;
- 建议类属性:`CreateFX="$1"`、`Options="$1"`、`DisabledWhileBusy="$1"` 均带
trigger 命令;具体默认值不带。
版本 **0.1.10 → 0.1.11**。
---
## 二十、问题分析(第十五轮,2026-08-04):属性补全的缩进叠加与 `$1` 占位符
### 现象
连续接受属性补全时,缩进不是稳定对齐,而是逐行递增。用户分步实测(0.1.12):
- 关闭引号 → 属性候选菜单 → 直接 Enter:第一次补全 `Count="1"` 就落在
6 个 Tab(`Disposition` 是 3 个 Tab);
- 按空格再次触发 → Enter:`CreateFX="$1"` 落在 9 个 Tab;
- 再 Enter(CreateFX 带触发命令,菜单自动重开):`DestinationPlayer="$1"`
落在 12 个 Tab。
```xml
<CreateObject
Options="IGNORE_ALL_OBJECTS"
Disposition="RANDOM_FORCE RELATIVE_ANGLE ABSOLUTE_ANGLE"
Count="1"
CreateFX="$1"
DestinationPlayer="$1"
```
### 根因
VS Code 在插入**含换行的补全文本**时,会给新行套用当前行的基础缩进,并与
补全文本里嵌入的缩进**相加**(而不是替换):
```text
我们插入 \n + 3 Tab → 落盘 = 当前行 3 Tab + 我们 3 Tab = 6 Tab
下一行:当前行 6 Tab + 我们 3 Tab = 9 Tab
再下一行:9 + 3 = 12 Tab
```
与实测的 6 / 9 / 12 完全一致。关键证据是**第一次补全就已多缩进**:0.1.12
第一次只插入 `\n` + 3 个 Tab(锚点是首个独占一行的 `Options`),落盘却是
6 个——问题不在我们读取了谁的缩进,而在于补全文本自带的缩进被编辑器叠加。
排查过程中还发现一个放大因素并已修复:`attributeInsertLayout` 原先以
“最后一个被解析出的属性”为锚点,用户在自动缩进的新行上输入的半截属性名
(`C`、`D`…,`hasValue = false`)也会被当成锚点,把编辑器自动缩进抄进补全
行;即使叠加根因修掉,这个因素也会让缩进更容易跑偏。
### 修复(0.1.13)
1. **换行时只插入 `\n`,不再嵌入缩进**:编辑器自动补当前行的基础缩进
(3 Tab),叠加量为 0,后续行稳定在 3 Tab;已在新行时仍显式替换为规范
缩进(该路径不插入换行,不受叠加影响)。
2. **锚点只用完整属性**(`hasValue` 为真),并优先取**第一个独占一行的完整
属性**作为规范缩进;半截属性名不参与缩进计算,整行内联时才回退到最后一个
完整属性。
3. **`$1` 改为真正的 snippet 占位符**:属性名补全统一用 `SnippetString`,
文档中不再出现字面 `$1`;接受补全后光标落在引号内的占位处,弹出的也是值
补全菜单。`Count="1"` 这类具体默认值同样用 snippet(无占位符,光标落在
闭合引号后)。
4. **尾随空格**:插入换行时,若上一个属性与光标之间只有空白(例如为触发补全
按的空格),把这段空白一并纳入替换范围,不再残留尾随空格。
5. **调试日志**:`ModWorkspace.log()` 输出到 “RA3 Mod XML” 输出通道;
attribute-name 补全每次记录 `existing / range / prefix`(JSON 转义),用于
对比“我们插入的内容”与“落盘的内容”,定位编辑器侧改写。
### 测试环境说明
当前单测(node + vscode stub)**不能复现 VS Code 的 suggest 弹窗、snippet
缩进与 auto-indent 行为**,这类问题只能靠实机 + 输出通道日志确认。后续若要
自动化,需要引入 `@vscode/test-electron` 做扩展宿主集成测试(本期未做,记录
为候选)。
### 测试(111 → 113 全绿)
- 新行上输入半截属性名(行缩进 20 个空格)→ 补全仍用规范缩进,range 覆盖
整段自动缩进与已输入字符;
- 为触发补全按的空格被新行替换吞掉,不再残留尾随空格;
- 属性名补全断言改为 `insertText.value`(SnippetString),换行插入断言为
`\nCount="1"`(缩进由编辑器提供)。
版本 **0.1.11 → 0.1.13**(0.1.12 为中间版本,仅含锚点与尾随空格修复,
未解决叠加;0.1.13 为最终修复)。
---
## 二十一、问题分析(第十六轮,2026-08-04):元素文本内容(simple content)引用与 `<<` 补全
### 现象
用户样例(`ObjectCreationList` → `CreateObject` 内输入 `<`):
1. 补全菜单出现 `CreateObject`,接受后变成 `<<CreateObject />`——两个尖括号
非法 XML;
2. 嵌套 `<CreateObject>` 被补全成自闭合 `<CreateObject />`,无法在标签内填
单位的 id;正确写法应是 `<CreateObject>CrateDebris_01</CreateObject>`
(原版 `SageXml\GlobalData\ObjectCreationLists.xml` 与 AttachTest
`ObjectCreationList.xml` 均为此形态);
3. 手动改成 `<CreateObject>C</CreateObject>` 后,内容区不出现 C 开头的
GameObject id 补全;
4. 对 `<CreateObject>CrateDebris_01</CreateObject>` 的文本 hover 与
Ctrl+点击导航均无效。
### 根因(三个独立缺陷叠加)
**A. 内容区补全没有处理“用户已输入 `<`”**
`<` 是补全触发字符(`extension.ts`)。`completion.ts` 的 `contentItems`
返回子元素候选时既不设置 `range`,`insertText` 又是完整的 `<Name…>`,VS Code
就把完整标签**追加**在已输入的 `<` 后面 → `<<Name />`。影响所有内容区子元素
补全(`Offset`、`RequiredUpgrade` 等同样中招),不只 `CreateObject`。
**B. `elementSnippet` 不知道“simple type 元素要填文本”**
XSD 中 `CreateObjectNugget` 明确声明
`<xs:element name="CreateObject" type="GameObjectWeakRef"/>`;`GameObjectWeakRef`
是 simple type,标签内容是 GameObject id。`elementSnippet` 只区分顶层 / 有子
元素的复杂类型 / 其他,simple type 落入 `<Name />` 分支,于是生成了永远填不了
值的自闭合标签。
**C. 所有 feature 都只认属性值,不认元素文本内容**
- `contentItems` 对 simple type 元素查“子元素列表”得到空 → 内容区无补全;
- `hover` 只检查属性名/属性值/元素名,文本区间不返回任何结果;
- `provideDefinition` 找不到属性直接 `return null`;
- `checkValueReferences` 只对属性做未解析引用检查;
- Find All References 的正则只匹配 `["']id["']`。
### 类似情况盘点(模型统计)
XSD 模型中共 **371 处“子元素是 simple type”的声明(149 个不同元素名)**:
| 类别 | 数量 | 例子 |
|---|---|---|
| 带 `xas:refType` 的引用内容 | 291 | `CreateObject`→GameObject、`RequiredUpgrade`/`ForbiddenUpgrade`→UpgradeTemplate、`SpawnTemplate`→GameObject、`Filename`→AudioFile、`RequiredObject`→GameObject |
| 无类型 `AssetReference`(isRef 无 refType) | 9 | `FXShaderConstantTexture@Value`、`RenderSubObjectReference@Mesh/CollisionBox` |
| 枚举内容 | 22 | `SourceMustNotHaveBeenDisabledThisFrameByType` 等 |
| 允许 `$DEFINE` 的内容 | 22 | 各类数值内容 |
| 普通字符串 | 27 | `DisplayName` 等 |
关键取证:`FXShaderConstantTexture@Value` 在真实 XML 里填的是**贴图名、数值、
布尔**(如 `AUMCV`、`1.000000`、`false`),`RenderSubObjectReference@Mesh`
在 w3x 里填的是**同一模型文件内的子对象名**——它们虽然被 XSD 标成
`AssetReference`(isRef),却不是全局资产 ID。因此**只有带 refType 的 simple
内容**才按全局引用处理;无类型 `AssetReference` 与 `Poid` 一律不参与全局
补全 / hover / 跳转 / 诊断,避免误报。
### 修复
1. **内容区 `<` 处理**(`completion.ts`):`contentChildItems` 检测光标前是否
已有 `<`(可带半截名字),有则把替换范围起点设为该 `<`,接受完整
`<Name…>` 片段时是“替换”而非“追加”,从根上消灭 `<<`;`</` 输入不触发子元素
补全。
2. **simple type 元素片段**:`elementSnippet` 对 simple type 返回
`<Name>$1</Name>`(带占位符),引用/枚举/define 内容在插入后自动弹值补全
(`editor.action.triggerSuggest`)。
3. **内容值补全**:`contentItems` 增加 simple-content 分支——元素自身类型是
simple 时按“值”补全:refType 资产 ID(严格按类型过滤)、枚举、`$DEFINE`;
替换范围只覆盖当前文本 token(`xmlParser.textContentTokenAt`)。
4. **引用层纯函数**(`refs.ts`):新增 `isReferenceContentType(typeName)`
(simple + refType + 排除 Poid)与 `resolveContentReferenceTargets(...)`,
复用属性引用的类型过滤/评分逻辑。
5. **hover / 定义跳转 / 诊断 / Find All References**:均增加元素文本 token
分支;Find All References 的搜索模式扩展为同时匹配属性值(`"id"`)与
内容引用(`>id<`),并把结果范围裁剪到 id 本身。
### 举一反三的测试(113 → 121 全绿)
- `xmlParser.test.mjs`:`textContentTokenAt` 返回内容 token、边界与空白内容
返回 null;
- `refs.test.mjs`:`GameObjectWeakRef` 内容只解析到 GameObject(同名
WeaponTemplate 排除);`AssetReference` / `Poid` / `string` 不是全局内容
引用;
- `completion.test.mjs`:用户原始场景——`<` 后接受 CreateObject 的 range 覆盖
`<`、`insertText` 为 `<CreateObject>$1</CreateObject>`、应用后不含 `<<`;
`<CreateObject>C</CreateObject>` 内只补 C 开头的 GameObject(类型过滤 +
token range);
- `contentFeatures.test.mjs`(新增):hover 显示内容引用定义、Ctrl+点击跳到
同文件精确定位、诊断只报“带 refType 的内容引用”且不误报 WeakReference
内容。
版本 **0.1.13 → 0.1.14**。
### 补充(0.1.14 实机回归,2026-08-04):真实文件里 `<` 后仍有闭合标签
单元测试最初只覆盖了“文件在 `<` 处结束”的形态,用户实机复测时 `<` 后面还有
`</CreateObject>`,`<<` 依然出现(这次是 `<<Tag>(光标在中间)</Tag>`)。
**第二层根因**:解析器的 `findTagEnd` 扫描残缺开始标签时会一直找到**后面闭合
标签的 `>`**,把它当成这个开始标签的结束,于是生成一个空名/半截名的伪元素
(如 name 为 `""` 或 `"Cr"`)。光标落在该伪元素的 start tag 内 → 补全走
`element-name` 分支,而该分支的替换范围只从 `<` **之后**开始,插入完整的
`<Name>…</Name>` 片段就变成 `<<Name>`。
**修复**(两层):
1. `findTagEnd` 在引号外遇到 `<`(在找到 `>` 之前)直接视为“未闭合开始标签”,
走原有的行尾恢复路径:`<` 单独出现时不再产生伪元素,内容区保持 content
上下文;`<Cr` 半截名则恢复为 `recoveredStartTag` 元素壳,仍由内容分支补全
(该分支的 range 已包含 `<`)。
2. `elementNameItems` 的替换范围改为从元素 `start`(即 `<`)开始,即使后续
解析仍产生伪元素,也不会再 `<<`。
测试补 121 → 125:`<` 后跟闭合标签不吞掉闭合标签、半截名恢复为元素壳、真实
文件形态(`<` 后有多行属性与闭合标签)接受 CreateObject 后不含 `<<`、半截名
`<Cr` 同样不含 `<<`。
### 补充 2(0.1.14 实机回归):`<<` 修掉后补全菜单反而消失
把 `<` 纳入替换范围后,单测直接调用 provider 全部通过,但 VS Code 实机里
`<` 后**不再弹出任何补全**;`<CreateObject>C`(尚未输入闭合标签)也不触发,
必须补上 `</CreateObject>` 才有候选。
**根因**:VS Code 用“替换范围内的文本”作为过滤前缀。范围包含 `<` 时,前缀
就是 `<`,所有标签名都不匹配 → 菜单为空;范围为零宽且元素未闭合时,
`textContentTokenAt` 因 `closeTagStart < 0` 返回 null,range 落在光标处、
没有覆盖已输入的 `C`,同样过滤不到。
**修复**(回到标准补全模式):
1. **已输入的 `<` 保留,不放进替换范围**:`contentChildItems` /
`elementNameItems` 的 range 从 `<` **之后**开始,插入文本**不再带开括号**
(`CreateObject>$1</CreateObject>`),应用结果仍是单个 `<`;没有输入 `<`
(Ctrl+Space 内容补全)时才插入完整 `<Name>…</Name>`。这样过滤前缀是
`""` / `Cr`,菜单正常显示。
2. **未闭合元素也能提取内容 token**:`textContentTokenAt` 在
`closeTagStart < 0` 时用 `el.end` 作为内容边界,`<CreateObject>C`(EOF)
也能覆盖已输入的 `C` 并给出前缀过滤。
测试 125 → 128:三个真实形态(孤立 `<`、`<Cr` 半截名、未闭合
`<CreateObject>C`)均断言插入文本不带开括号、range 覆盖正确文本、应用结果
无 `<<`,并新增“未输入 `<` 时插入完整标签”用例。
---
## 二十二、问题分析(第十七轮,2026-08-04):simple-content 补全变成属性补全;大列表截断后 CrateDebris 消失
### 现象
1. 接受 `<CreateObject>$1</CreateObject>` 片段后,弹出的不是 GameObject id
补全,而是 `xai:joinAction` 与 `xmlns:xai` 两个属性名候选;
2. 不接受这两个候选,直接在标签间输入 `C`:菜单要等较久才出现,且包含大量
C 开头的候选,但 **CrateDebris_01 不在其中**;继续输入 `r` / `D` / `e` /
`b` / `r` 后候选反而越来越少直至消失;
3. 快速输入 `Cr`(不等第一次菜单)则 CrateDebris_01 正常出现。
### 根因(两个独立问题)
**A. `>` 之后的零宽光标被当成“还在 start tag 内”**
`analyzeContext` 原来用 `offset <= startTagEnd` 判定光标在开始标签内。补全
片段 `<CreateObject>$1</CreateObject>` 接受后,`$1` 恰好落在
`startTagEnd`(`>` 后一格)这个零宽位置,于是被分到 `attribute-name`,
返回的自然是 `xai:joinAction` / `xmlns:xai`。
同类边界还有:已闭合元素的 `end` 之前也用了 `<=`,导致光标刚越过
`</CreateObject>` 时仍被当作“在子元素内容里”,而不是父元素的内容区。
**B. 引用列表超过 400 条后被硬截断,且没有告诉 VS Code“列表不完整”**
`assetIdItems` 对匹配到的 id 排序后直接 `slice(0, 400)`。VS Code 收到列表后
会在客户端按已输入的字符继续过滤这 400 条,**不会再次调用 provider**。
于是:
- 输入 `C` 时 CrateDebris_01 若排在 400 名之后,它从一开始就不在列表里;
- 继续输入 `r` 只是在这 400 条里过滤,CrateDebris_01 永远不会出现;
- 快速输入 `Cr` 时第一次请求的前缀已经是 `Cr`,匹配数小于 400,所以能看到。
`includeSourceItems` 还有一个变体:先 `slice(0, 400)` 再排序,同样可能把
真正优先级高的候选切掉;`defineItems` / `localIdItems` 也有同类截断隐患。
### 修复
1. **开始标签边界语义修正**(`context.ts` / `xmlParser.ts`):
- `>` 之后(`offset === startTagEnd`)一律视为内容区,只有 `>` 尚未输入
的残缺开始标签仍保持 `attribute-name`;
- 新增 `elementContainsOffset`:已闭合 / 自闭合元素的 `end` 为开区间,
光标在 `</Child>` 之后回到父元素内容;未闭合元素在 EOF 仍算在元素内。
2. **大列表标记 incomplete**(`completion.ts`):`assetIdItems`、
`includeSourceItems`、`defineItems`、`localIdItems` 一旦超过 400 条就返回
`CompletionList(..., isIncomplete: true)`。VS Code 会在用户继续输入时重新
请求 provider,窄前缀下 CrateDebris_01 这类“前 400 之外”的 id 不再丢失。
3. **局部优先 + 避免全量排序**:当前文档局部 overlay 的资产(`stream:
"local"`)在评分中额外降 0.4,先于全局项目候选;`topScoredDefs` 用
大小堆只保留前 400,不再对全部匹配 id 做 `sort`,降低大项目首键延迟。
4. **Include source 先排序再截断**:修掉“先 slice 后 sort”的顺序问题。
### 类似情况盘点
- 任何带 refType 的属性值 / `inheritFrom` / simple-content 引用,只要匹配数
超过 400,都存在“越输越少但目标永不出现”的风险——本轮统一用 incomplete
解决;
- 光标紧贴 `>` 后的零宽位置(简单元素片段 `$1`)与紧贴 `</Child>` 后的位置
是同一类“边界误归属”,本轮一并修掉;
- `Include@source` 的“先截断后排序”是同一家族 bug,已修复。
### 举一反三的测试(128 → 136 全绿)
- `context.test.mjs`:`>` 后且有闭合标签 → content;`>` 后但闭合标签尚未
输入 → content;`</Child>` 之后 → 父元素 content;
- `xmlParser.test.mjs`:已闭合 / 自闭合元素的 `end` 为开区间;未闭合元素
EOF 仍属于该元素;
- `completion.test.mjs`:simple-content 片段 `$1` 位置返回值补全而不是
`xai:joinAction` / `xmlns:xai`;450+ 候选时返回 incomplete 列表,窄前缀
重请求后找到 CrateDebris_01;当前文件 local overlay 资产即使总数超过
400 也保留在前 400。
版本 **0.1.14 → 0.1.15**。
### 遗留
若 Corona 上“输入 C 后菜单出现慢”仍然明显,下一个瓶颈大概率是
`getScope()` 每次文档版本变化都重建完整局部 include 链 / 逻辑树;本轮先把
候选截断与类型过滤造成的“列表错误”修掉,局部 scope 缓存优化留作独立一轮。
+201 -23
View File
@@ -1,6 +1,6 @@
# 调研结论与实施计划(已按最新代码同步更新) # 调研结论与实施计划(已按最新代码同步更新)
> 说明:本文档随实现演进持续同步。最近一次同步(2026-08-01)对齐了实现过程中新增的模块与设计变更:BAB 精确搜索路径、manifest 类型/ID 推导、上下文感知元素类型、属性级 refType / Poid 局部引用(`id` 定义点)、精确跳转范围、嵌套 `xi:include`、注入式语法高亮等。 > 说明:本文档随实现演进持续同步。最近一次同步(2026-08-04)对齐了实现过程中新增的模块与设计变更:BAB 精确搜索路径、manifest 类型/ID 推导、上下文感知元素类型、属性级 refType / Poid 局部引用(`id` 定义点)、精确跳转范围、嵌套 `xi:include`、注入式语法高亮、bit-flag 列表补全(空格触发 / 排除已用 / 追加模式)、simple-content 元素文本引用(补全 / hover / 跳转 / 诊断 / Find All References)等。
## 一、调研结论(带证据) ## 一、调研结论(带证据)
@@ -82,10 +82,18 @@ src/
asset-types.json 由 tools/extract-asset-types.mjs 生成(TypeId 哈希→类型名) asset-types.json 由 tools/extract-asset-types.mjs 生成(TypeId 哈希→类型名)
indexer/ indexer/
includeResolver.ts Include 路径解析(纯 TS,BAB /data /art /audio 顺序) includeResolver.ts Include 路径解析(纯 TS,BAB /data /art /audio 顺序)
existence.ts 文件集存在性快照(目录枚举 Set,替代逐路径 statSync)
manifestParser.ts .manifest 二进制解析 + 类型/ID 推导(纯 TS) manifestParser.ts .manifest 二进制解析 + 类型/ID 推导(纯 TS)
fileScanner.ts 目录遍历缓存 + Include source 候选收集 fileScanner.ts 目录遍历缓存 + Include source 候选收集
refs.ts 引用目标解析(按 refType / isRef / inheritFrom 过滤,纯 TS) refs.ts 引用目标解析(属性 + 元素文本内容,按 refType / isRef /
indexer.ts 工作区索引器(资产/Define/流/manifest 合并,LRU 解析缓存) inheritFrom 过滤,纯 TS)
shallowScan.ts 大体积美术资产(.w3x 等)顶层浅扫描(纯 TS,不建 DOM)
records.ts 每文件紧凑索引记录(资产/Define/Include/xi + 行号)
caches.ts 跨重建持久缓存(DocumentCache / IndexRecordsCache /
IncludeResolveCache)+ 失效纪元 InvalidationsEpoch
diskCache.ts 跨会话磁盘缓存(gzip JSON、原子写、多信号 stat 校验)
indexer.ts 工作区索引器(资产/Define/流/manifest/w3x 合并,
分阶段:XML → art,中间快照可发布)
types.ts 共享类型 types.ts 共享类型
features/ features/
completion.ts 补全 provider(元素/属性/值,上下文感知;xs:list 多值按当前段过滤) completion.ts 补全 provider(元素/属性/值,上下文感知;xs:list 多值按当前段过滤)
@@ -109,6 +117,11 @@ test/
1. **语言激活范围**:不劫持 `*.xml`。通过 `workspaceContains:**/Data/Mod.xml`、`**/*.babproj` 激活;语法高亮为**纯注入** grammar(不声明 `language`,避免覆盖内置 XML 语法)。 1. **语言激活范围**:不劫持 `*.xml`。通过 `workspaceContains:**/Data/Mod.xml`、`**/*.babproj` 激活;语法高亮为**纯注入** grammar(不声明 `language`,避免覆盖内置 XML 语法)。
2. **索引范围与默认值**:索引“项目 Data + additionalmaps + 沿 include 可达的 SageXml 原版源码”;SDK 路径默认 `C:\Apps\RA3-MODSDK-X`(可配置)。`reference` include 解析为 `builtmods` 下对应 manifest(惰性解析、按文件缓存),manifest 缺失/无效时回退到占位 XML。 2. **索引范围与默认值**:索引“项目 Data + additionalmaps + 沿 include 可达的 SageXml 原版源码”;SDK 路径默认 `C:\Apps\RA3-MODSDK-X`(可配置)。`reference` include 解析为 `builtmods` 下对应 manifest(惰性解析、按文件缓存),manifest 缺失/无效时回退到占位 XML。
**美术资产(.w3x)**:`<Include type="all">` / `ART:` 指向的 `.w3x`(及内容嗅探为
XML 的未知扩展名文件)按其顶层资产入库(`W3DContainer` / `W3DMesh` /
`W3DHierarchy` / `W3DCollisionBox` 等),使 `Model@Name`、`Hierarchy`、`Mesh`
等引用可解析;大模型文件**浅扫描**(不建 DOM),结果缓存在 workspace 级、
跨重建复用(详见设计决策 14)。
3. **manifest 资产建模**:类型优先用哈希表,未知时从名称前缀推导;可引用 ID 取最后冒号段;类型名统一走大小写规范化(`W3dContainer` ↔ `W3DContainer`),类型匹配严格遵循 XSD 继承链。 3. **manifest 资产建模**:类型优先用哈希表,未知时从名称前缀推导;可引用 ID 取最后冒号段;类型名统一走大小写规范化(`W3dContainer` ↔ `W3DContainer`),类型匹配严格遵循 XSD 继承链。
4. **上下文感知元素类型**:同名元素按父元素类型解析(`resolveElementType` 沿解析树逐层 `childTypeOf`,失败回退全局映射),保证 `<Weapon>` 等元素的属性/引用判定正确。 4. **上下文感知元素类型**:同名元素按父元素类型解析(`resolveElementType` 沿解析树逐层 `childTypeOf`,失败回退全局映射),保证 `<Weapon>` 等元素的属性/引用判定正确。
5. **引用判定与解析**:`refType` 或 `isRef` 均视为引用;带 `refType` 时严格按类型过滤(同名 ID 不串类型);`inheritFrom` 按可继承类型过滤。**局部作用域例外**(`isLocalReferenceAttribute`):`id` 是元素自身的定义点——无 refType 或 refType 与自身类型兼容时不检查、不解析(`RoadObject@id→Road` 这类跨类型 id 引用保留检查);Poid 类型属性是管线局部引用,全局索引无法判定,不检查、不解析。 5. **引用判定与解析**:`refType` 或 `isRef` 均视为引用;带 `refType` 时严格按类型过滤(同名 ID 不串类型);`inheritFrom` 按可继承类型过滤。**局部作用域例外**(`isLocalReferenceAttribute`):`id` 是元素自身的定义点——无 refType 或 refType 与自身类型兼容时不检查、不解析(`RoadObject@id→Road` 这类跨类型 id 引用保留检查);Poid 类型属性是管线局部引用,全局索引无法判定,不检查、不解析。
@@ -120,14 +133,124 @@ test/
11. **`xs:list` 建模与多值补全**:list 简单类型继承 itemType 的枚举 / refType / isRef / 11. **`xs:list` 建模与多值补全**:list 简单类型继承 itemType 的枚举 / refType / isRef /
allowsDefine 并标记 `isList`(`LocomotorSurfaceBitFlags`、`KindOfBitFlags` 等 79 个 allowsDefine 并标记 `isList`(`LocomotorSurfaceBitFlags`、`KindOfBitFlags` 等 79 个
类型、317 处属性声明受益);补全只对“最后一个空格段”过滤,替换范围只覆盖当前段, 类型、317 处属性声明受益);补全只对“最后一个空格段”过滤,替换范围只覆盖当前段,
支持 `Surfaces="GROUND ` 之后继续输入 `W` 提示 `WATER`。 支持 `Surfaces="GROUND ` 之后继续输入 `W` 提示 `WATER`。第十三轮(2026-08-04)
补全触发与编辑体验:空格注册为触发字符;列表过滤排除已出现的 flag;当前段已是
完整枚举值且没有更长变体时进入“追加模式”(零宽 range + `insertText=" FLAG"`,
可直接在闭合值末尾/中间追加);替换范围止于光标,中间插入不会删除尾部 flag。
12. **未闭合引号的行尾恢复**:起始标签扫描到 EOF 且引号未闭合时,在第一个换行处截断 12. **未闭合引号的行尾恢复**:起始标签扫描到 EOF 且引号未闭合时,在第一个换行处截断
标签并继续解析,未闭合只影响当前行(仍上报 `Unterminated start tag`),后续元素 标签并继续解析,未闭合只影响当前行(仍上报 `Unterminated start tag`),后续元素
的补全 / hover / 诊断不中断。 的补全 / hover / 诊断不中断。第十三轮补充:恢复出的元素带 `recoveredStartTag`
标记并补挂父链;补全上下文对“光标在恢复元素内但越过 `startTagEnd`”的情况按
`text.slice(tagStart, cursor)` 重新解析部分标签,使多行书写的未闭合属性
(如 `Disposition="`)仍可获得 attribute-value 补全,且不影响全局解析。
13. **语义 token 兜底高亮**:TextMate 对未闭合引号会把后续内容当字符串吞掉(任何 13. **语义 token 兜底高亮**:TextMate 对未闭合引号会把后续内容当字符串吞掉(任何
XML 编辑器皆然);扩展注册 `DocumentSemanticTokensProvider`,仅当解析报错时用 XML 编辑器皆然);扩展注册 `DocumentSemanticTokensProvider`,仅当解析报错时用
语义 token 覆盖标签名 / 属性名 / 属性值(标准 token 类型 `type` / `property` / 语义 token 覆盖标签名 / 属性名 / 属性值(标准 token 类型 `type` / `property` /
`string`,主题自带配色)。合法文件返回空,观感与纯 TextMate 完全一致。 `string`,主题自带配色)。合法文件返回空,观感与纯 TextMate 完全一致。
14. **大体积美术资产浅扫描 + 跨重建持久缓存**(第八轮,2026-08-02):
- 实测 Corona:3788 个 w3x / 2.64 GB,163 个超过 4 MB,最大 22.8 MB;大文件是
`W3DHierarchy` + 若干 `W3DMesh`(`Vertices/V`、`Triangles/T` 等 unbounded 数值
载荷),顶层记录通常只有几个到二十几个。
- 全量解析内存放大约 17 倍(6.3 MB 文本 → +109 MB DOM),不可接受;`scanXmlShallow`
单次线性扫描只提取顶层 `name+id`、`<Includes>`、`<xi:include>`、`<Defines>`,
22 MB 文件 ~600 ms、保留内存≈0。
- 索引按扩展名三分:`.xml` 全量解析(4 MB 上限不变);`.w3x` 浅扫描;
未知扩展名嗅探文件头(`<` 开头、无 NUL)决定按 XML 浅扫描或二进制登记
(manifest 是 `*.manifest` 二进制,不存在 `.manifestxml` 源码格式)。
- `DocumentCache` / `ShallowScanCache` 由 `ModWorkspace` 持有,每次重建传入新的
`ModIndexer`;按 `mtimeMs + size` 校验,未变化不重读。Corona 第二次构建
w3x 重扫数为 0(4,829 次缓存命中)。
- 读取整个文件不可避免(顶层边界需要全量扫描),但建 DOM 不是;优化的是
"不分配子节点对象"与"跨重建不重读",两者叠加后方案可行。
15. **重建零 stat + 记录驱动索引**(第九轮,2026-08-02,v0.1.1):
- 插桩发现每次重建(含信任重建)都有约 11 万次同步 `statSync`
(`resolveSource` 的 include 存在性检查),机械盘上占 10-20s;
`IncludeResolveCache` 按(目录 + source)缓存解析结果,内容编辑不清、
创建/删除文件与强制 reindex 才清。信任重建 statSync 降到 0。
- `IndexRecordsCache` 缓存每文件紧凑索引记录(顶层资产 / Define / Include /
xi:include + 1-based 行号),信任重建完全不接触 DOM;`DocumentCache`
双重淘汰(条数 LRU + 元素预算,超预算先淘汰最大树)把 DOM 常驻内存封顶。
- w3x 缓存不再保留 LineMap(Corona 全量约 700MB),浅扫描直接产出带行号的记录。
- 实测:Corona 信任二次构建 21s → **2.0s**(statSync 0);首建后 2.5GB
堆保留确认为构建期可回收垃圾,常驻 ~100MB;强制 reindex ~5-25s。
- 候选目录扫描并行化;`stats` 新增 `candidatesMs` / `walkMs` / `resolveCalls` /
`resolveCacheHits` 供索引报告定位耗时。
16. **分阶段索引与部分可用性**(第十轮,2026-08-03):索引分两阶段发布——
阶段 A(xml)只走 XML + manifest include 链,w3x 只登记进待扫队列;
阶段 B(art)浅扫描队列并继续走 w3x 内的 include。阶段 A 结束即发布
不可变快照(`snapshotIndex` 深拷贝嵌套 Map/数组),XML/枚举/语法类功能
在首建早期即可用;引用类诊断在 `!complete || stale` 时“显示但标注”
(code 为 `*-indexing`,消息注明 index incomplete)。快照携带
`complete` / `phase` / `stale`,状态栏显示阶段。
17. **构建中失效与 stale 标记**(第十轮):watcher 的 change / create /
delete 均触发防抖重建;`InvalidationsEpoch` 记录失效次数,快照发布时若
期间有失效则标记 stale,由 dirty 机制随后重建收敛;构建失败保留上一个
快照而非清空索引。
18. **多信号文件 stamp**(第十轮):`IndexedFile.stat` 扩展为
`{ size, mtimeMs, birthtimeMs, ctimeMs }`,任一不匹配即重读,为磁盘
持久化缓存铺路(FAT32 mtime 2s 粒度、工具保留 mtime 等场景)。
19. **文件集快照替代 statSync**(第十一轮,2026-08-03):`ExistenceSnapshot`
用**惰性按目录 readdir**(只读查询到的父目录,按目录缓存)回答 include
存在性,覆盖根之外才 statSync 回退;盘符根不枚举。首版全量递归枚举使
XML 阶段从 27s 涨到 45s,已改为惰性模式(复测 24.0s)。
`stats.snapshotHits / snapshotFallbacks` 入报告;Corona 首建 75,926 次
查询全部由快照回答、0 回退。
20. **磁盘持久化缓存**(第十一轮):`DiskRecordsCache` 持久化 records 缓存
(gzip JSON、原子写、identity key);启动时并发 stat 多信号校验,不匹配
丢弃重读;构建后异步回写。Corona 缓存仅 651 KB,冷启动约 11s(原
~2.5 分钟)。
21. **缓存命令**(第十一轮):`ra3modxml.clearCache`(清内存 + 磁盘 +
强制重建)、`ra3modxml.showCacheReport`(路径/大小/校验统计/命中数)。
22. **重建插桩**(第十一轮补充):`buildCount` / `lastBuildTrigger`
(initial、save、watcher-*、config、reindex、clear-cache、
dirty-followup)进入索引/缓存报告;“RA3 Mod XML” 输出通道记录每次构建
的触发原因、phase A 发布时间与完成耗时,用于定位“首建后又重建一次”
之类的现象。
23. **watcher 噪声过滤与 URI 日志**(第十一轮补充):watcher 事件把触发
URI 写入输出通道;路径含 `.git` 段的事件直接忽略(后台 fetch /
maintenance 会周期性触碰 `.git`,不应触发重建或 stale 标记)。
24. **watcher 内容白名单**(第十一轮补充):临时文件命名模式
(`.git`/`.tmp`/`.lock`/`~`/`.swp`/`.bak`/`.orig` 后缀、`.#`/`.~`
前缀)全部忽略;`onDidChange` 只响应扩展名白名单(`.xml` / `.w3x`,
RA3 合理文本格式为 xml/w3x/lua,lua 暂未索引、manifest 为二进制)或已在
索引中的文件;创建/删除仍响应所有真实文件(影响 include 存在性)。
25. **当前文档局部链 + 逻辑树展开(第十二轮,T1)**:新增 `xpointer.ts` /
`logicalTree.ts` / `localScope.ts`。打开文件时按当前文本建立局部 overlay
(自身资产 / `$DEFINE` / include 链),并生成展开 `xi:include` 的逻辑树;
features 经 `ws.getScope(document)` 拿到 overlay-aware 索引。Poid 引用
(`AttachModuleId` / `ModuleId` / `AutoResolveBody` 等)在最近 GameObject
子树内解析;未命中不新增诊断(保守策略,避免跨文件误报)。
顶层 `<Include type="all">` 暂不并入逻辑树(保留为后续扩展)。
26. **属性补全的插入布局与类型化默认值(第十四轮,2026-08-04)**:
`attributeInsertLayout` 按临近属性的排版决定插入方式——贴引号时补空格、
一行一个属性时补换行 + 缩进、已在新行时用临近缩进替换当前行空白、
inline 风格的手动换行保留用户缩进;`attributeValuePlaceholder` 对引用/
枚举/list/布尔等建议类属性保留 `$1` + 自动触发,对标量属性填 XSD 默认值
或类型示例(`0d` / `0s` / `100%` / `0.0` / `0`),具体默认值不再弹空
suggest。
第十五轮(2026-08-04)最终结论:VS Code 插入含换行的补全文本时会把当前
行基础缩进与文本内嵌缩进相加(3+3=6、6+3=9…),因此换行前缀只插入 `\n`、
缩进交给编辑器;同时半截属性名(`hasValue=false`)不再作为缩进锚点,改用
第一个独占一行的完整属性作为规范缩进,插入换行时顺带吞掉触发补全留下的
尾随空格;属性名补全改用 `SnippetString`(`$1` 成为真正占位符),并新增
输出通道调试日志。
27. **simple-content 元素文本引用(第十六轮,2026-08-04)**:simple type
子元素(如 `ObjectCreationList` 内嵌套 `<CreateObject>`,类型
`GameObjectWeakRef`)的**标签间文本**就是资产引用。内容区补全现在区分
“复杂元素 → 子元素名”与“简单元素 → 值补全”;用户已输入 `<` 时替换范围从
`<` 开始,杜绝 `<<`;simple type 元素片段固定为 `<Name>$1</Name>`(可填
值)并自动触发值补全。hover / Ctrl 跳转 / 诊断 / Find All References
均增加内容 token 分支。只有**带 `xas:refType`** 的 simple 内容按全局引用
处理(291 处子元素声明);无类型 `AssetReference`
(`FXShaderConstantTexture@Value`、`RenderSubObjectReference@Mesh` 等
真实数据是贴图/子对象名)与 `Poid` 不参与全局解析,避免误报。
补充:真实文件中 `<` 后还有 `</…>` 时,`findTagEnd` 曾把闭合标签的 `>` 误
当成残缺开始标签的结束,生成空名/半截名伪元素,补全走 element-name 分支
导致 `<<`。修复为引号外遇到 `<` 即视为未闭合(行尾恢复),且
`elementNameItems` 的替换范围包含 `<`。实机再回归:范围含 `<` 会让 VS Code
用 `<` 做过滤前缀导致菜单为空——改为保留已输入的 `<`、range 从 `<` 之后
开始、插入文本不带开括号;`textContentTokenAt` 对未闭合元素用 `el.end`
作内容边界。
## 三、实施步骤 ## 三、实施步骤
@@ -136,38 +259,93 @@ test/
3. [x] 生成模型:`schema-model.json`(XSD,295 顶层元素 / 1851 类型)与 `asset-types.json`(79 个 AssetType 哈希)。 3. [x] 生成模型:`schema-model.json`(XSD,295 顶层元素 / 1851 类型)与 `asset-types.json`(79 个 AssetType 哈希)。
4. [x] 纯 TS 核心:include 解析、manifest 解析、XML 解析封装、索引器、引用解析。 4. [x] 纯 TS 核心:include 解析、manifest 解析、XML 解析封装、索引器、引用解析。
5. [x] 功能层:补全、hover、导航、诊断、大纲、高亮 grammar。 5. [x] 功能层:补全、hover、导航、诊断、大纲、高亮 grammar。
6. [x] 单测(fixture Mod,53 个用例全绿:含 xs:list 枚举、未闭合引号恢复、list 多值 6. [x] 单测(fixture Mod,73 个用例全绿:含 xs:list 枚举、未闭合引号恢复、list 多值
分段、带 vscode stub 的补全集成、语义 token 兜底)+ 编译 + `vsce package` 打包 分段、带 vscode stub 的补全集成、语义 token 兜底)+ 编译 + `vsce package` 打包
(ra3-mod-xml-0.1.0.vsix,约 495KB)。 (ra3-mod-xml-0.1.1.vsix,约 499KB)。
7. [x] 在 AttachTest / GenEvoTest / Corona 上做冒烟验证,并按真实项目反馈修复问题(详见 `docs/analysis-issues.md` 六轮分析)。 7. [x] 在 AttachTest / GenEvoTest / Corona 上做冒烟验证,并按真实项目反馈修复问题(详见 `docs/analysis-issues.md` 八轮分析)。
8. [x] w3x 美术资产索引(第八轮):新增 `shallowScan.ts` / `caches.ts`,w3x 与内容嗅探
XML 走浅扫描,Include source 补全候选加入 w3x,BOM 剥离,缓存跨重建持久化;
AttachTest 报错场景(`AUGunship_SKN`)修复,Corona 首次/二次构建实测验证。
9. [x] Corona 性能与内存优化(第九轮,v0.1.1):`records.ts` 记录驱动索引、
`IncludeResolveCache` 零 stat 重建、DOM 元素预算淘汰、w3x LineMap 移除、
候选并行扫描、阶段计时;Corona 信任重建 2.0s;确认首建 2.5GB 为可回收垃圾。
10. [x] 部分可用性 + 分阶段索引(第十轮,2026-08-03):T0 解耦(语法/模型
诊断、枚举/子元素补全、Include 链接无索引可用)、A/B 分阶段发布、watcher
触发重建 + 失效纪元 stale 标记、stat 多信号扩展;测试 73 → 79。
11. [x] 冷启动提速(第十一轮,2026-08-03):文件集快照替代 statSync、
磁盘持久化缓存(多信号校验、原子写)、clearCache/showCacheReport 命令;
测试 79 → 90;Corona 冷启动 ~11s。
12. [x] 惰性存在性快照 + 重建插桩(第十一轮补充,2026-08-03):全量目录枚举
改为惰性按目录 readdir(phase A 45s → 24.0s);新增 buildCount /
lastBuildTrigger 与输出通道日志;索引/缓存报告显示构建序号与触发原因。
13. [x] watcher 噪声过滤 + URI 日志(第十一轮补充,2026-08-03):输出通道
记录 `[watcher-*] <path>`;`.git` 段路径忽略;测试 90 → 91。
14. [x] watcher 内容白名单(第十一轮补充,2026-08-03):临时文件命名模式
过滤 + 内容变更扩展名白名单 + `isIndexedFile` 兜底;测试 91 → 92。
15. [x] T1 当前文档局部链 + include 展开(第十二轮,2026-08-03):局部
overlay(不在任何流里的文件也能解析自身引用)+ `xi:include` 逻辑树展开
+ Poid 局部作用域补全/悬停/跳转;测试 92 → 98。2026-08-04 补充构建期
闸门:`getScope` 在重建进行中只返回 parse-only scope,避免与 indexer
抢盘(版本 0.1.9)。
16. [x] bit-flag 列表补全修复(第十三轮,2026-08-04):空格触发字符、多行
未闭合标签的部分标签重解析(`recoveredStartTag` 标记 + 恢复元素父链)、
闭合值内追加模式与中间插入范围修复;测试 98 → 107(版本 0.1.10)。
17. [x] 属性补全插入体验(第十四轮,2026-08-04):闭合引号后自动补空格、
临近属性缩进对齐、标量属性类型化默认值;测试 107 → 111(版本 0.1.11)。
18. [x] 属性补全缩进叠加修复(第十五轮,2026-08-04):换行前缀只插入 `\n`
(编辑器自动补基础缩进,避免 3+3=6 式叠加)、完整属性锚点 + 首个独占一行
属性为规范缩进、`$1` 改为 SnippetString 占位符、输出通道调试日志、尾随
空格吞除;测试 111 → 113(版本 0.1.12–0.1.13)。
19. [x] simple-content 文本引用修复(第十六轮,2026-08-04):`<` 后补全范围
覆盖 `<`、simple type 元素片段 `<Name>$1</Name>`、简单元素内容值补全、
`refs.ts` 新增 `isReferenceContentType` / `resolveContentReferenceTargets`、
hover / 定义跳转 / 诊断 / Find All References 内容分支;
测试 113 → 121(版本 0.1.14)。实机回归补充:`findTagEnd` 引号外遇 `<`
视为未闭合 + 插入文本不带开括号(range 从 `<` 之后开始);测试 121 → 125,
再回归 125 → 128(未闭合元素内容 token)。
20. [x] simple-content 零宽边界与大列表补全(第十七轮,2026-08-04):
`>` 后光标归入内容区、已闭合元素 `end` 改为开区间;超过 400 条的
id/define/local/include 候选返回 `isIncomplete` 让 VS Code 随输入重请求;
当前文档 local 资产优先、候选 top-N 用堆避免全量排序、Include source
先排序再截断;测试 128 → 136。
## 四、验证结果(实测) ## 四、验证结果(实测)
| 项目 | 规模 | 索引耗时 | 资产数 | 说明 | | 项目 | 规模 | 索引耗时 | 资产数 | 说明 |
|---|---|---|---|---| |---|---|---|---|---|
| AttachTest | 71 文件 | ~0.6s | 35,546(manifest 35,322) | 2 个流(static + mapmetadata) | | AttachTest | 88 文件 | ~1.2s | 35,607(manifest 35,322,w3x 浅扫 62) | 2 个流(static + mapmetadata);二次构建 354ms / 62 缓存命中 |
| GenEvoTest | 24 文件 | ~1.8s | 35,392(manifest 35,322) | 项目 ID(alliedmcv 等)正确收录 | | GenEvoTest | 66 文件 | 首次 ~2.6s / 二次 ~0.4s | 35,502(manifest 35,322,w3x 浅扫 38) | 2 个流、73 个 Define;二次构建 0 重扫 / 38 缓存命中 |
| Corona | 3,448 文件(+ 非 XML 资产路径) | ~54.5s | 55,305(manifest 35,322) | 3 个流、183 个 Define、0 诊断 | | Corona | 8,976 文件 | 首次 ~250s / 信任二次 ~2s / 强制 ~5-25s | 64,868(manifest 35,322,w3x 浅扫 4,829) | 3 个流、183 个 Define、0 诊断;二次构建 statSync 0、resolveHits 15,333 |
单元测试覆盖:XML 解析(自闭合/容错/偏移/未闭合引号行尾恢复)、补全上下文(未闭合引号仍为 attribute-value、list 多值分段)、补全集成(vscode stub 下 `LocomotorTemplate@Surfaces` 未闭合引号枚举补全、空格后第二段过滤与替换范围)、语义 token(标签/属性/值范围、合法文档返回空、malformed 返回兜底 token)、include 解析(BAB 顺序、SDK 根优先于 SageXml)、manifest 二进制解析(合成 v5 样本、类型/ID 推导)、索引器(资产/Define/流/缺失 include/嵌套 xi:include)、XSD 模型(上下文类型、`childTypeOf`、大小写规范化、属性级 refType、外来命名空间判定、`xs:list` 枚举继承与 `isList` 标记)、引用过滤(`Weapon="X"` 只跳 `WeaponTemplate`、模块 `id` 定义点、Poid 局部引用、`xi:include` 不校验、`Side="Allies"` 命中 manifest 的 `PlayerTemplate`)。 单元测试覆盖:XML 解析(自闭合/容错/偏移/未闭合引号行尾恢复 + recovered
标记)、补全上下文(未闭合引号仍为 attribute-value、多行未闭合引号、list 多值
分段、闭合引号后为 attribute-name)、补全集成(vscode stub 下
`LocomotorTemplate@Surfaces` 未闭合引号枚举补全、空格后第二段过滤与替换范围、
空格后排除已用 flag、中间插入范围止于光标、闭合值末尾追加模式、`CAN_ATTACK`
前缀保护、多行未闭合 `Disposition` 完整链路、闭合引号后补空格、一行一个属性
换行缩进、新行缩进对齐、标量类型化默认值)、语义 token(标签/属性/值范围、
合法文档返回空、malformed 返回兜底 token)、include 解析(BAB 顺序、SDK 根
优先于 SageXml)、manifest 二进制解析(合成 v5 样本、类型/ID 推导)、索引器
(资产/Define/流/缺失 include/嵌套 xi:include)、XSD 模型(上下文类型、
`childTypeOf`、大小写规范化、属性级 refType、外来命名空间判定、`xs:list`
枚举继承与 `isList` 标记)、引用过滤(`Weapon="X"` 只跳 `WeaponTemplate`、
模块 `id` 定义点、Poid 局部引用、`xi:include` 不校验、`Side="Allies"` 命中
manifest 的 `PlayerTemplate`)、simple-content 文本引用(`<` 后补全不产生
`<<`、`<CreateObject>$1</CreateObject>` 片段、内容值按 refType 过滤、
内容 hover / Ctrl 跳转 / 诊断)。
> 注:`D:\Mods\CoronaMod` 位于移动硬盘,当前未连接;GenEvoTest / Corona 的回归需在 D: 盘可用时补跑(用户会另行通知)。 > 注:D: 盘移动硬盘已恢复连接;Corona 已在第八 / 九轮按上述新数据回归。
## 五、假设与开放问题 ## 五、假设与开放问题
- 假设 SDK 路径默认 `C:\Apps\RA3-MODSDK-X`(与 prompts 一致),可在设置中修改。 - 假设 SDK 路径默认 `C:\Apps\RA3-MODSDK-X`(与 prompts 一致),可在设置中修改。
- 假设补全/导航以“文本语义分析”为主,不做完整 XSD 校验(BAB 才是权威校验器)。 - 假设补全/导航以“文本语义分析”为主,不做完整 XSD 校验(BAB 才是权威校验器)。
- 开放:是否发布到 VS Code Marketplace(需要 publisher)——本期先保证本地 `vsce package` 可安装。 - 开放:是否发布到 VS Code Marketplace(需要 publisher)——本期先保证本地 `vsce package` 可安装。
- 开放:**“宏展开”式虚拟合并**(用户提议,方向已确认):解析前先把 `xi:include` - 开放:**“宏展开”式虚拟合并**(用户提议,方向已确认):`xi:include` 已按逻辑树
(以及顶层 `<Include type="all">`、`inheritFrom` 继承合并)展开成不含 include 的 展开(第十二轮,见第六节);顶层 `<Include type="all">` 与 `inheritFrom` +
文档树,再对展开后的树做 XSD 校验、补全与诊断——与 BAB 编译时把整个 Mod 合并成 `xai:joinAction` 的深合并仍未实现,后续如需要“当前文档视角的全量合并诊断”再继续。
一份大 XML 的行为一致。展开树需携带**来源追溯**(错误/跳转仍定位到原始文件),
并处理 xpointer 子集解析与 include 循环。当前仅做到:目标文件可索引、可导航、
缺失可诊断;`xi:include` 本身不参与 XSD 校验(第五轮)。
该功能也是“GameObject 内模块 id 局部作用域解析”(第四轮遗留)的地基。
详细设计备忘见下一节。
## 六、include 展开设计备忘(2026-08-01,待实施) ## 六、include 展开设计备忘(2026-08-01;xi:include 部分已实施于第十二轮)
> 目的:集中记录 include 处理相关的现状、结论与设计,下次遇到 include 问题时从这里继续, > 目的:集中记录 include 处理相关的现状、结论与设计,下次遇到 include 问题时从这里继续,
> 并在实施后把结果回写本节。 > 并在实施后把结果回写本节。
@@ -180,7 +358,7 @@ test/
| `reference` → builtmods manifest 解析 / 缺失回退占位 XML | 已实现 | | `reference` → builtmods manifest 解析 / 缺失回退占位 XML | 已实现 |
| 嵌套 `xi:include`(任意层级):目标可索引、缺失报 `include-not-found`、Ctrl+点击跳转、`href` hover 解析目标 | 已实现(第二轮 + 第五轮) | | 嵌套 `xi:include`(任意层级):目标可索引、缺失报 `include-not-found`、Ctrl+点击跳转、`href` hover 解析目标 | 已实现(第二轮 + 第五轮) |
| `xi:include` 及其属性不参与 XSD 校验(外来命名空间守卫 `isXsdElementName` / `isXsdAttributeName`) | 已实现(第五轮) | | `xi:include` 及其属性不参与 XSD 校验(外来命名空间守卫 `isXsdElementName` / `isXsdAttributeName`) | 已实现(第五轮) |
| include 目标内容“虚拟合并”进父文档的逻辑树 | **未实现**(本文档主题) | | include 目标内容“虚拟合并”进父文档的逻辑树 | 已实现 `xi:include`(第十二轮);顶层 `<Include type="all">` 仍不展开 |
### 2. 已确认的方向 ### 2. 已确认的方向
+19 -1
View File
@@ -41,9 +41,15 @@ XML 之间的组织靠 `<Include>` 标签,共有三种语义:
- 枚举值(XSD `xs:enumeration`); - 枚举值(XSD `xs:enumeration`);
- `$DEFINE` 常量(如 `$CIV_HEALTH_SMALL`); - `$DEFINE` 常量(如 `$CIV_HEALTH_SMALL`);
- `<Include source>` 补全可解析的文件路径(`DATA:` / `ART:` / `AUDIO:`)。 - `<Include source>` 补全可解析的文件路径(`DATA:` / `ART:` / `AUDIO:`)。
- **元素文本内容(simple content)**:带 `xas:refType` 的简单内容元素
(如 `<CreateObject>ID</CreateObject>`)在标签间补全对应类型的资产 ID、
枚举或 `$DEFINE`;补全出的 simple-content 元素必须是可填值的成对标签
(`<Name></Name>`),且内容区已输入 `<` 时不得产生 `<<`。
3. **引用提示(Hover)**:元素/属性悬停显示 XSD 文档、类型、默认值;资产 ID 悬停显示定义位置;`$DEFINE` 悬停显示值与定义位置。 3. **引用提示(Hover)**:元素/属性悬停显示 XSD 文档、类型、默认值;资产 ID 悬停显示定义位置;`$DEFINE` 悬停显示值与定义位置。
- 元素文本内容(simple content 引用)悬停同样显示定义位置。
4. **引用导航**: 4. **引用导航**:
- 从引用型属性值跳转到对应资产定义(Go to Definition); - 从引用型属性值跳转到对应资产定义(Go to Definition);
- 从 simple-content 引用元素(`<CreateObject>ID</CreateObject>`)的文本跳转到对应资产定义;
- 查找某资产 ID 的所有引用(Find All References); - 查找某资产 ID 的所有引用(Find All References);
- `<Include source>` / `xi:include href` 直接打开目标文件; - `<Include source>` / `xi:include href` 直接打开目标文件;
- `inheritFrom` 跳转到被继承元素; - `inheritFrom` 跳转到被继承元素;
@@ -54,6 +60,7 @@ XML 之间的组织靠 `<Include>` 标签,共有三种语义:
- 缺失必填 `id`(顶层资产); - 缺失必填 `id`(顶层资产);
- 重复 ID(同类型 + 同 id,mod 文件之间;覆盖原版 SageXml 不算冲突); - 重复 ID(同类型 + 同 id,mod 文件之间;覆盖原版 SageXml 不算冲突);
- 引用未解析(引用了不存在的资产 ID,可配置是否忽略原版 manifest 中的 ID); - 引用未解析(引用了不存在的资产 ID,可配置是否忽略原版 manifest 中的 ID);
- simple-content 引用元素的文本未解析(同属性引用规则,仅带 refType 的类型)。
- `<Include>` 目标文件找不到、Include 循环; - `<Include>` 目标文件找不到、Include 循环;
- `$DEFINE` 未定义。 - `$DEFINE` 未定义。
@@ -70,6 +77,15 @@ XML 之间的组织靠 `<Include>` 标签,共有三种语义:
- `TypeId` 哈希 → 类型名的映射来自 OpenSAGE 的 `AssetType` 枚举(本工作区可提取); - `TypeId` 哈希 → 类型名的映射来自 OpenSAGE 的 `AssetType` 枚举(本工作区可提取);
- 资产名与源文件名各自存放在独立的空字符结尾字符串缓冲区中。 - 资产名与源文件名各自存放在独立的空字符结尾字符串缓冲区中。
**补充(美术资产 `.w3x` 索引)**:
- `W3X.xml` / `ART:` include 链中的 `.w3x` 是文本 XML(建模工具导出,BAB 同样按
XML 编译),其顶层资产(`W3DContainer` / `W3DMesh` / `W3DHierarchy` 等)应参与
补全、悬停、导航与诊断——`Model@Name`、`Hierarchy`、`Mesh` 等引用依赖这些定义;
- 大模型文件(实测 Corona 最大 22.8 MB,顶点/三角形数据占大头)采用**顶层浅扫描**
(不建 DOM 树),结果在 workspace 级缓存并跨重建复用,避免每次保存都重读整个
美术资产目录(Corona 约 2.6 GB);索引记录与 include 解析结果同样跨重建缓存,
保存触发的重建零 stat、零重读(Corona 实测约 2 秒)。
### P1:非近期目标(本期不做,但预留扩展点) ### P1:非近期目标(本期不做,但预留扩展点)
6. **高效搜索**:Mod 项目巨大(Corona 约 7500 个 XML、38MB)时直接全文搜索很慢,需要一个高效的 XML 内容索引机制。 6. **高效搜索**:Mod 项目巨大(Corona 约 7500 个 XML、38MB)时直接全文搜索很慢,需要一个高效的 XML 内容索引机制。
@@ -81,7 +97,9 @@ XML 之间的组织靠 `<Include>` 标签,共有三种语义:
- 中小项目:`C:\Apps\RA3-MODSDK-X\Mods\AttachTest`、`D:\Mods\CoronaMod\mods\mods\GenEvoTest`。 - 中小项目:`C:\Apps\RA3-MODSDK-X\Mods\AttachTest`、`D:\Mods\CoronaMod\mods\mods\GenEvoTest`。
- 大型项目:`D:\Mods\CoronaMod\mods\mods\corona`(自带 `xsd/`)。 - 大型项目:`D:\Mods\CoronaMod\mods\mods\corona`(自带 `xsd/`)。
- 现有工具:工作区 `check_duplicate_ids.py`(include 解析与重复 ID 检测的参考实现)。 - 现有工具:工作区 `check_duplicate_ids.py`(include 解析与重复 ID 检测的参考实现)。
- manifest 格式参考:OpenSAGE `src/OpenSage.Game/Data/StreamFS/ManifestFile.cs`(commit `d45d361`,最新分支已移除该文件)。本机网络受限未能拉取,且 manifest 为压缩/哈希的二进制,本期不实现其解析。 - manifest 格式参考:OpenSAGE `src/OpenSage.Game/Data/StreamFS/ManifestFile.cs`
(commit `d45d361`,本仓库 `OpenSAGE/` 子目录)。插件已实现 v5/v6/v7 二进制
解析(`src/indexer/manifestParser.ts`),未知 TypeId 哈希时按资产名前缀推导类型。
## 四、验收标准 ## 四、验收标准
+2 -2
View File
@@ -1,12 +1,12 @@
{ {
"name": "ra3-mod-xml", "name": "ra3-mod-xml",
"version": "0.1.0", "version": "0.1.9",
"lockfileVersion": 3, "lockfileVersion": 3,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "ra3-mod-xml", "name": "ra3-mod-xml",
"version": "0.1.0", "version": "0.1.9",
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"fast-xml-parser": "^4.5.0" "fast-xml-parser": "^4.5.0"
+10 -1
View File
@@ -2,7 +2,7 @@
"name": "ra3-mod-xml", "name": "ra3-mod-xml",
"displayName": "RA3 Mod XML", "displayName": "RA3 Mod XML",
"description": "Red Alert 3 Mod XML tooling: syntax highlighting, completions, reference navigation and diagnostics for SAGE/BinaryAssetBuilder XML.", "description": "Red Alert 3 Mod XML tooling: syntax highlighting, completions, reference navigation and diagnostics for SAGE/BinaryAssetBuilder XML.",
"version": "0.1.0", "version": "0.1.15",
"publisher": "ra3-mod-xml", "publisher": "ra3-mod-xml",
"license": "MIT", "license": "MIT",
"engines": { "engines": {
@@ -22,6 +22,7 @@
], ],
"main": "./dist/extension.js", "main": "./dist/extension.js",
"activationEvents": [ "activationEvents": [
"onLanguage:xml",
"workspaceContains:**/Data/Mod.xml", "workspaceContains:**/Data/Mod.xml",
"workspaceContains:**/mod.babproj", "workspaceContains:**/mod.babproj",
"workspaceContains:**/*.babproj" "workspaceContains:**/*.babproj"
@@ -91,6 +92,14 @@
{ {
"command": "ra3modxml.openIndexReport", "command": "ra3modxml.openIndexReport",
"title": "RA3 Mod XML: Show index report" "title": "RA3 Mod XML: Show index report"
},
{
"command": "ra3modxml.clearCache",
"title": "RA3 Mod XML: Clear caches and rebuild"
},
{
"command": "ra3modxml.showCacheReport",
"title": "RA3 Mod XML: Show cache report"
} }
] ]
}, },
+51 -8
View File
@@ -30,6 +30,7 @@ export function activate(context: vscode.ExtensionContext): void {
":", ":",
".", ".",
"/", "/",
" ",
), ),
); );
context.subscriptions.push( context.subscriptions.push(
@@ -44,7 +45,7 @@ export function activate(context: vscode.ExtensionContext): void {
context.subscriptions.push( context.subscriptions.push(
vscode.languages.registerReferenceProvider( vscode.languages.registerReferenceProvider(
XML_SELECTOR, XML_SELECTOR,
new Ra3ReferenceProvider(), new Ra3ReferenceProvider(ws),
), ),
); );
context.subscriptions.push( context.subscriptions.push(
@@ -56,19 +57,26 @@ export function activate(context: vscode.ExtensionContext): void {
context.subscriptions.push( context.subscriptions.push(
vscode.languages.registerDocumentSymbolProvider( vscode.languages.registerDocumentSymbolProvider(
XML_SELECTOR, XML_SELECTOR,
new Ra3DocumentSymbolProvider(), new Ra3DocumentSymbolProvider(ws),
), ),
); );
context.subscriptions.push( context.subscriptions.push(
vscode.languages.registerDocumentSemanticTokensProvider( vscode.languages.registerDocumentSemanticTokensProvider(
XML_SELECTOR, XML_SELECTOR,
new Ra3SemanticTokensProvider(), new Ra3SemanticTokensProvider(ws),
RA3_SEMANTIC_TOKENS_LEGEND, RA3_SEMANTIC_TOKENS_LEGEND,
), ),
); );
const diagnostics = new Ra3Diagnostics(ws); const diagnostics = new Ra3Diagnostics(ws);
context.subscriptions.push(diagnostics); context.subscriptions.push(diagnostics);
// Refresh diagnostics for every open XML document whenever a new index
// snapshot is published (XML phase, art phase, stale/final rebuild).
ws.onIndexUpdate = () => {
for (const doc of vscode.workspace.textDocuments) {
if (doc.languageId === "xml") void diagnostics.update(doc);
}
};
const diagnosticTimers = new Map<string, ReturnType<typeof setTimeout>>(); const diagnosticTimers = new Map<string, ReturnType<typeof setTimeout>>();
const scheduleDiagnostics = (doc: vscode.TextDocument) => { const scheduleDiagnostics = (doc: vscode.TextDocument) => {
@@ -109,36 +117,71 @@ export function activate(context: vscode.ExtensionContext): void {
context.subscriptions.push( context.subscriptions.push(
vscode.workspace.onDidSaveTextDocument((doc) => { vscode.workspace.onDidSaveTextDocument((doc) => {
if (doc.languageId !== "xml") return; if (doc.languageId !== "xml") return;
ws.scheduleRebuild(); ws.invalidate(doc.uri.fsPath);
ws.scheduleRebuild("save");
void diagnostics.update(doc); void diagnostics.update(doc);
}), }),
); );
context.subscriptions.push( context.subscriptions.push(
vscode.workspace.onDidChangeConfiguration((e) => { vscode.workspace.onDidChangeConfiguration((e) => {
if (e.affectsConfiguration("ra3modxml")) ws.scheduleRebuild(); if (e.affectsConfiguration("ra3modxml")) {
// Search paths / builtmods locations may have changed: cached include
// resolutions and manifest lookups are no longer valid.
ws.invalidateExistence();
ws.scheduleRebuild("config");
}
}), }),
); );
context.subscriptions.push( context.subscriptions.push(
vscode.commands.registerCommand("ra3modxml.reindex", () => ws.rebuild()), vscode.commands.registerCommand(
"ra3modxml.reindex",
() => void ws.rebuild(true, "reindex-command"),
),
);
context.subscriptions.push(
vscode.commands.registerCommand("ra3modxml.clearCache", () => {
ws.clearCaches();
void vscode.window.showInformationMessage(
"RA3 Mod XML: caches cleared; rebuilding from scratch…",
);
}),
);
context.subscriptions.push(
vscode.commands.registerCommand("ra3modxml.showCacheReport", async () => {
void vscode.window.showInformationMessage(await ws.cacheReport(), {
modal: false,
});
}),
); );
context.subscriptions.push( context.subscriptions.push(
vscode.commands.registerCommand("ra3modxml.openIndexReport", () => { vscode.commands.registerCommand("ra3modxml.openIndexReport", () => {
const idx = ws.index; const idx = ws.index;
if (!idx) { if (!idx) {
if (ws.isBuilding) {
void vscode.window.showInformationMessage(
"RA3 Mod XML: index is still building — check the status bar. " +
"Most features become available after the XML phase.",
);
return;
}
void vscode.window.showInformationMessage( void vscode.window.showInformationMessage(
"RA3 Mod XML: no index available. Open a workspace that contains Data/Mod.xml.", "RA3 Mod XML: no index available. Open a workspace that contains Data/Mod.xml.",
); );
return; return;
} }
const s = idx.stats; const s = idx.stats;
const stale = idx.stale ? " (stale)" : "";
void vscode.window.showInformationMessage( void vscode.window.showInformationMessage(
`RA3 Mod XML index\n` + `RA3 Mod XML index\n` +
`Project: ${s.projectDir}\n` + `Project: ${s.projectDir}\n` +
`Files: ${s.indexedFiles} (${s.parsedFiles} parsed)\n` + `Files: ${s.indexedFiles} (${s.parsedFiles} parsed, ${s.shallowScannedFiles} shallow-scanned, ${s.shallowCacheHits + s.recordsCacheHits} cache hits)\n` +
`Assets: ${s.assetCount} (${s.manifestAssetCount} from ${s.manifestFiles} manifests)\n` + `Assets: ${s.assetCount} (${s.manifestAssetCount} from ${s.manifestFiles} manifests)\n` +
`Defines: ${s.defineCount} · Streams: ${s.streams} · Candidates: ${s.sourceCandidates}\n` + `Defines: ${s.defineCount} · Streams: ${s.streams} · Candidates: ${s.sourceCandidates}\n` +
`Indexed in ${(s.elapsedMs / 1000).toFixed(1)}s`, `Phase: ${s.phase} · Complete: ${s.complete}${stale}\n` +
`Build #${ws.buildCount} (trigger: ${ws.lastTrigger})\n` +
`Indexed in ${(s.elapsedMs / 1000).toFixed(1)}s\n` +
`XML walk: ${(s.walkMs / 1000).toFixed(1)}s · Candidates: ${(s.candidatesMs / 1000).toFixed(1)}s · Art scan: ${(s.artScanMs / 1000).toFixed(1)}s`,
{ modal: false }, { modal: false },
); );
}), }),
+557 -57
View File
@@ -1,5 +1,6 @@
import * as vscode from "vscode"; import * as vscode from "vscode";
import { parseXml, type XmlElement } from "../language/xmlParser"; import type { XmlAttribute, XmlElement } from "../language/xmlParser";
import { textContentTokenAt } from "../language/xmlParser";
import { import {
analyzeContext, analyzeContext,
splitListValuePrefix, splitListValuePrefix,
@@ -7,7 +8,13 @@ import {
} from "../language/context"; } from "../language/context";
import { resolveElementType } from "../language/typeContext"; import { resolveElementType } from "../language/typeContext";
import * as model from "../model/schemaModel"; import * as model from "../model/schemaModel";
import type { AttributeInfo, SimpleTypeInfo } from "../model/schemaModel";
import { isLocalReferenceAttribute } from "../indexer/refs"; import { isLocalReferenceAttribute } from "../indexer/refs";
import {
findContainingGameObject,
collectLocalIds,
type LogicalElement,
} from "../indexer/logicalTree";
import type { ModWorkspace } from "../workspace"; import type { ModWorkspace } from "../workspace";
import type { ModIndex, AssetDef } from "../indexer/types"; import type { ModIndex, AssetDef } from "../indexer/types";
@@ -20,12 +27,14 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
document: vscode.TextDocument, document: vscode.TextDocument,
position: vscode.Position, position: vscode.Position,
_token: vscode.CancellationToken, _token: vscode.CancellationToken,
): Promise<vscode.CompletionItem[]> { ): Promise<vscode.CompletionItem[] | vscode.CompletionList<vscode.CompletionItem>> {
if (!this.ws.isRa3Workspace()) return [];
const text = document.getText(); const text = document.getText();
const offset = document.offsetAt(position); const offset = document.offsetAt(position);
const doc = parseXml(text); const scope = await this.ws.getScope(document);
const doc = scope.expanded;
const ctx = analyzeContext(doc, text, offset); const ctx = analyzeContext(doc, text, offset);
const idx = this.ws.index; const idx = scope.merged;
switch (ctx.kind) { switch (ctx.kind) {
case "element-name": case "element-name":
@@ -33,9 +42,9 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
case "attribute-name": case "attribute-name":
return this.attributeNameItems(ctx, document, position); return this.attributeNameItems(ctx, document, position);
case "attribute-value": case "attribute-value":
return idx ? this.valueItems(ctx, document, position, idx) : []; return this.valueItems(ctx, document, position, idx);
case "content": case "content":
return idx ? this.contentItems(ctx, document, position, idx) : []; return this.contentItems(ctx, document, position, idx);
default: default:
return []; return [];
} }
@@ -53,10 +62,12 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
const names = this.childrenOf(parent); const names = this.childrenOf(parent);
if (!names.length) return []; if (!names.length) return [];
// The "<" already exists at the element's start: replace only the name
// area and insert the tag body WITHOUT a leading "<", otherwise the
// range text ("<") would also be used as the filter prefix and hide
// every suggestion.
const start = ctx.element const start = ctx.element
? document.offsetAt( ? ctx.element.start + (ctx.closing ? 2 : 1)
document.positionAt(ctx.element.start + (ctx.closing ? 2 : 1)),
)
: document.offsetAt(position); : document.offsetAt(position);
const range = new vscode.Range(document.positionAt(start), position); const range = new vscode.Range(document.positionAt(start), position);
const items: vscode.CompletionItem[] = []; const items: vscode.CompletionItem[] = [];
@@ -71,7 +82,7 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
(type ? `Type: ${type}` : ""); (type ? `Type: ${type}` : "");
item.documentation = docText ? new vscode.MarkdownString(docText) : undefined; item.documentation = docText ? new vscode.MarkdownString(docText) : undefined;
item.detail = type ? `RA3 XML · ${type}` : "RA3 XML"; item.detail = type ? `RA3 XML · ${type}` : "RA3 XML";
item.insertText = this.elementSnippet(child.name, type); item.insertText = this.elementSnippet(child.name, type, ctx.element == null);
items.push(item); items.push(item);
} }
return items; return items;
@@ -95,16 +106,27 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
return []; return [];
} }
private elementSnippet(name: string, type: string | null): vscode.SnippetString { private elementSnippet(
name: string,
type: string | null,
includeOpenBracket = true,
): vscode.SnippetString {
const open = includeOpenBracket ? "<" : "";
if (model.isTopLevelElement(name)) { if (model.isTopLevelElement(name)) {
return new vscode.SnippetString(`<${name} id="$1">\n\t$0\n</${name}>`); return new vscode.SnippetString(`${open}${name} id="$1">\n\t$0\n</${name}>`);
} }
const info = type ? model.typeInfo(type) : undefined; const info = type ? model.typeInfo(type) : undefined;
// Simple types hold text content (asset id / enum / define / string), so
// they need an explicit closing tag and a value placeholder instead of a
// self-closing tag that can never contain a value.
if (info?.kind === "simple") {
return new vscode.SnippetString(`${open}${name}>$1</${name}>`);
}
const hasChildren = info?.kind === "complex" && info.children.length > 0; const hasChildren = info?.kind === "complex" && info.children.length > 0;
if (hasChildren) { if (hasChildren) {
return new vscode.SnippetString(`<${name}>\n\t$0\n</${name}>`); return new vscode.SnippetString(`${open}${name}>\n\t$0\n</${name}>`);
} }
return new vscode.SnippetString(`<${name} />`); return new vscode.SnippetString(`${open}${name} />`);
} }
// ── Attribute name ──────────────────────────────────────────────── // ── Attribute name ────────────────────────────────────────────────
@@ -121,8 +143,15 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
const used = new Set(ctx.existingAttrs.map((a) => a.toLowerCase())); const used = new Set(ctx.existingAttrs.map((a) => a.toLowerCase()));
const items: vscode.CompletionItem[] = []; const items: vscode.CompletionItem[] = [];
const wordStart = findAttributeWordStart(document, position, el); const text = document.getText();
const range = new vscode.Range(document.positionAt(wordStart), position); const offset = document.offsetAt(position);
const layout = attributeInsertLayout(text, el, offset);
const range = new vscode.Range(document.positionAt(layout.rangeStart), position);
this.ws.log(
`[completion] attr-name ${el.name} existing=[${ctx.existingAttrs.join(", ")}] ` +
`range=${layout.rangeStart}..${offset} prefix=${JSON.stringify(layout.prefix)}`,
);
for (const attr of attrs) { for (const attr of attrs) {
if (used.has(attr.name.toLowerCase())) continue; if (used.has(attr.name.toLowerCase())) continue;
@@ -138,15 +167,16 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
if (attr.default != null) md.appendMarkdown(`Default: \`${attr.default}\` \n`); if (attr.default != null) md.appendMarkdown(`Default: \`${attr.default}\` \n`);
md.appendMarkdown(`Type: \`${attr.type ?? "string"}\``); md.appendMarkdown(`Type: \`${attr.type ?? "string"}\``);
item.documentation = md; item.documentation = md;
if (attr.required) { const value = this.attributeValuePlaceholder(attr, el);
item.insertText = attr.name === "id" ? 'id="$1"' : `${attr.name}="$1"`; item.insertText = new vscode.SnippetString(
} else { layout.prefix + `${attr.name}="${value.snippet}"`,
item.insertText = `${attr.name}="$1"`; );
} if (value.trigger) {
item.command = { item.command = {
command: "editor.action.triggerSuggest", command: "editor.action.triggerSuggest",
title: "Suggest attribute values", title: "Suggest attribute values",
}; };
}
items.push(item); items.push(item);
} }
@@ -154,31 +184,75 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
if (!used.has("xai:joinaction")) { if (!used.has("xai:joinaction")) {
const j = new vscode.CompletionItem("xai:joinAction", vscode.CompletionItemKind.Property); const j = new vscode.CompletionItem("xai:joinAction", vscode.CompletionItemKind.Property);
j.range = range; j.range = range;
j.insertText = 'xai:joinAction="$1"'; j.insertText = new vscode.SnippetString(
layout.prefix + 'xai:joinAction="$1"',
);
j.detail = "Instance join action"; j.detail = "Instance join action";
j.documentation = new vscode.MarkdownString( j.documentation = new vscode.MarkdownString(
"Controls how this element merges with the inherited definition: `Replace` or `Remove`.", "Controls how this element merges with the inherited definition: `Replace` or `Remove`.",
); );
j.command = {
command: "editor.action.triggerSuggest",
title: "Suggest attribute values",
};
items.push(j); items.push(j);
} }
if (!used.has("xmlns:xai")) { if (!used.has("xmlns:xai")) {
const ns = new vscode.CompletionItem("xmlns:xai", vscode.CompletionItemKind.Property); const ns = new vscode.CompletionItem("xmlns:xai", vscode.CompletionItemKind.Property);
ns.range = range; ns.range = range;
ns.insertText = 'xmlns:xai="uri:ea.com:eala:asset:instance"'; ns.insertText = new vscode.SnippetString(
layout.prefix + 'xmlns:xai="uri:ea.com:eala:asset:instance"',
);
ns.detail = "xai namespace"; ns.detail = "xai namespace";
items.push(ns); items.push(ns);
} }
return items; return items;
} }
/**
* Chooses the placeholder/value inserted for a completed attribute:
* attributes whose value is picked from suggestions (references, enums,
* lists, booleans, defines, include sources, local ids) keep a `$1`
* placeholder and re-trigger the value popup; scalar attributes get the XSD
* default (or a type-appropriate example such as `0d` for angles or `0s`
* for times) so the completed value shows the expected format immediately.
*/
private attributeValuePlaceholder(
attr: AttributeInfo,
el: XmlElement,
): { snippet: string; trigger: boolean } {
if (
attr.isBoolean ||
attr.isList ||
attr.enumValues.length > 0 ||
attr.refType != null ||
attr.isRef ||
attr.name === "inheritFrom" ||
(el.name === "Include" && attr.name === "source")
) {
return { snippet: "$1", trigger: true };
}
if (attr.name === "id") {
// `id` is the element's own definition point; nothing to suggest, but
// keep the placeholder for the user to type the id.
return { snippet: "$1", trigger: false };
}
if (attr.default != null && attr.default !== "") {
return { snippet: attr.default, trigger: false };
}
const example = DEFAULT_VALUE_BY_TYPE[(attr.type ?? "").toLowerCase()];
if (example != null) return { snippet: example, trigger: false };
return { snippet: "$1", trigger: false };
}
// ── Attribute value ─────────────────────────────────────────────── // ── Attribute value ───────────────────────────────────────────────
private valueItems( private valueItems(
ctx: CompletionContext, ctx: CompletionContext,
document: vscode.TextDocument, document: vscode.TextDocument,
position: vscode.Position, position: vscode.Position,
idx: ModIndex, idx: ModIndex | null,
): vscode.CompletionItem[] { ): vscode.CompletionItem[] | vscode.CompletionList<vscode.CompletionItem> {
const el = ctx.element; const el = ctx.element;
const attr = ctx.attr; const attr = ctx.attr;
if (!el || !attr) return []; if (!el || !attr) return [];
@@ -194,15 +268,23 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
// whitespace-separated: only the token currently being edited is used for // whitespace-separated: only the token currently being edited is used for
// filtering, and the replacement range covers that token instead of the // filtering, and the replacement range covers that token instead of the
// whole value. // whole value.
const seg = attrInfo?.isList const isList = attrInfo?.isList === true;
const seg = isList
? splitListValuePrefix(rawPrefix) ? splitListValuePrefix(rawPrefix)
: { token: rawPrefix, start: 0 }; : { token: rawPrefix, start: 0 };
const prefix = seg.token; const prefix = seg.token;
const valueStartOffset = const valueStartOffset =
attr.valueStart >= 0 ? attr.valueStart : document.offsetAt(position); attr.valueStart >= 0 ? attr.valueStart : document.offsetAt(position);
const endOffset = const cursorOffset = document.offsetAt(position);
attr.quoteEnd > attr.valueEnd ? attr.valueEnd : document.offsetAt(position); // For list values the replacement must cover only the segment being
// edited; extending to the end of the whole value would delete the flags
// after the cursor when inserting in the middle of an existing list.
const endOffset = isList
? Math.min(cursorOffset, attr.valueEnd >= 0 ? attr.valueEnd : cursorOffset)
: attr.quoteEnd > attr.valueEnd
? attr.valueEnd
: cursorOffset;
const rangeStart = valueStartOffset + seg.start; const rangeStart = valueStartOffset + seg.start;
const valueRange = new vscode.Range( const valueRange = new vscode.Range(
document.positionAt(rangeStart), document.positionAt(rangeStart),
@@ -214,10 +296,12 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
kind: vscode.CompletionItemKind, kind: vscode.CompletionItemKind,
detail: string, detail: string,
doc?: string, doc?: string,
range?: vscode.Range,
insertText?: string,
) => { ) => {
const item = new vscode.CompletionItem(label, kind); const item = new vscode.CompletionItem(label, kind);
item.range = valueRange; item.range = range ?? valueRange;
item.insertText = label; item.insertText = insertText ?? label;
item.detail = detail; item.detail = detail;
if (doc) item.documentation = new vscode.MarkdownString(doc); if (doc) item.documentation = new vscode.MarkdownString(doc);
return item; return item;
@@ -232,6 +316,7 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
); );
} }
if (isInclude && attrName === "source") { if (isInclude && attrName === "source") {
if (!idx) return [];
return this.includeSourceItems(idx, prefix, make); return this.includeSourceItems(idx, prefix, make);
} }
if (attrName === "xai:joinaction" || attrName === "joinaction") { if (attrName === "xai:joinaction" || attrName === "joinaction") {
@@ -242,19 +327,25 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
// inheritFrom: same element type first, then everything. // inheritFrom: same element type first, then everything.
if (attrName === "inheritfrom") { if (attrName === "inheritfrom") {
if (!idx) return [];
return this.assetIdItems(idx, el.name, null, prefix, make); return this.assetIdItems(idx, el.name, null, prefix, make);
} }
// `id` attributes are definitions and Poid attributes are pipeline-local // `id` attributes are definitions and Poid attributes are pipeline-local
// references; offering global asset ids for them would be wrong. // references; offering global asset ids for them would be wrong.
if (attrInfo && isLocalReferenceAttribute(elType, attr.name)) { if (attrInfo && isLocalReferenceAttribute(elType, attr.name)) {
return []; if (attrName === "id") return [];
return this.localIdItems(el as LogicalElement, prefix, make);
} }
if (attrInfo?.refType) { if (attrInfo?.refType) {
if (!idx) return [];
return this.assetIdItems(idx, null, attrInfo.refType, prefix, make); return this.assetIdItems(idx, null, attrInfo.refType, prefix, make);
} }
if (attrInfo?.enumValues?.length) { if (attrInfo?.enumValues?.length) {
if (attrInfo.isList) {
return this.listEnumItems(attrInfo, rawPrefix, seg, valueRange, make);
}
return attrInfo.enumValues return attrInfo.enumValues
.filter((v) => v.toLowerCase().startsWith(prefix.toLowerCase())) .filter((v) => v.toLowerCase().startsWith(prefix.toLowerCase()))
.map((v) => make(v, vscode.CompletionItemKind.EnumMember, attrInfo.type ?? "enum")); .map((v) => make(v, vscode.CompletionItemKind.EnumMember, attrInfo.type ?? "enum"));
@@ -264,28 +355,87 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
.filter((v) => v.startsWith(prefix.toLowerCase())) .filter((v) => v.startsWith(prefix.toLowerCase()))
.map((v) => make(v, vscode.CompletionItemKind.Value, "boolean")); .map((v) => make(v, vscode.CompletionItemKind.Value, "boolean"));
} }
if (attrInfo?.allowsDefine) { if (idx && attrInfo?.allowsDefine) {
return this.defineItems(idx, prefix, make); return this.defineItems(idx, prefix, make);
} }
return []; return [];
} }
/**
* Completions for xs:list enum values (whitespace-separated bit flags).
*
* Only the segment being edited is used for filtering, and values already
* present earlier in the list are excluded so adding a flag never re-offers
* an existing one. When the current segment is already a complete flag and
* no other flag extends it (e.g. "GROUND" -> "GROUND_EDGE" would disable
* this), the remaining flags are offered as insertions after the cursor
* (" FLAG") so flags can be appended to an already-closed value.
*/
private listEnumItems(
attrInfo: { enumValues: string[]; type?: string | null },
rawPrefix: string,
seg: { token: string; start: number },
valueRange: vscode.Range,
make: (
label: string,
kind: vscode.CompletionItemKind,
detail: string,
doc?: string,
range?: vscode.Range,
insertText?: string,
) => vscode.CompletionItem,
): vscode.CompletionItem[] {
const used = new Set(
rawPrefix
.slice(0, seg.start)
.split(/\s+/)
.map((t) => t.toLowerCase())
.filter(Boolean),
);
const token = seg.token.toLowerCase();
const exact = token !== "" && attrInfo.enumValues.some((v) => v.toLowerCase() === token);
const extendable = attrInfo.enumValues.some(
(v) => v.toLowerCase().startsWith(token) && v.toLowerCase() !== token,
);
const append = exact && !extendable;
const range = append
? new vscode.Range(valueRange.end, valueRange.end)
: valueRange;
const filtered = attrInfo.enumValues.filter((v) => {
const lower = v.toLowerCase();
if (used.has(lower)) return false;
if (append) return lower !== token;
if (exact && lower === token) return false;
return lower.startsWith(token);
});
return filtered.map((v) =>
make(
v,
vscode.CompletionItemKind.EnumMember,
attrInfo.type ?? "enum",
undefined,
range,
append ? ` ${v}` : v,
),
);
}
private includeSourceItems( private includeSourceItems(
idx: ModIndex, idx: ModIndex,
prefix: string, prefix: string,
make: (label: string, kind: vscode.CompletionItemKind, detail: string, doc?: string) => vscode.CompletionItem, make: (label: string, kind: vscode.CompletionItemKind, detail: string, doc?: string) => vscode.CompletionItem,
): vscode.CompletionItem[] { ): vscode.CompletionItem[] | vscode.CompletionList<vscode.CompletionItem> {
const lower = prefix.toLowerCase(); const lower = prefix.toLowerCase();
const candidates = idx.sourceCandidates const candidates = idx.sourceCandidates.filter((c) =>
.filter((c) => c.source.toLowerCase().includes(lower)) c.source.toLowerCase().includes(lower),
.slice(0, MAX_VALUE_ITEMS); );
const priority: Record<string, number> = { "": 0, DATA: 1, ART: 2, AUDIO: 3 }; const priority: Record<string, number> = { "": 0, DATA: 1, ART: 2, AUDIO: 3 };
candidates.sort( candidates.sort(
(a, b) => (a, b) =>
(priority[a.prefix ?? ""] ?? 4) - (priority[b.prefix ?? ""] ?? 4) || (priority[a.prefix ?? ""] ?? 4) - (priority[b.prefix ?? ""] ?? 4) ||
a.source.localeCompare(b.source), a.source.localeCompare(b.source),
); );
return candidates.map((c) => { const items = candidates.map((c) => {
const item = make(c.source, vscode.CompletionItemKind.File, "Include source"); const item = make(c.source, vscode.CompletionItemKind.File, "Include source");
item.detail = c.path; item.detail = c.path;
item.documentation = new vscode.MarkdownString( item.documentation = new vscode.MarkdownString(
@@ -293,6 +443,7 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
); );
return item; return item;
}); });
return this.limitItems(items, items.length);
} }
private assetIdItems( private assetIdItems(
@@ -301,20 +452,37 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
refType: string | null, refType: string | null,
prefix: string, prefix: string,
make: (label: string, kind: vscode.CompletionItemKind, detail: string, doc?: string) => vscode.CompletionItem, make: (label: string, kind: vscode.CompletionItemKind, detail: string, doc?: string) => vscode.CompletionItem,
): vscode.CompletionItem[] { ): vscode.CompletionItem[] | vscode.CompletionList<vscode.CompletionItem> {
const lower = prefix.toLowerCase(); const lower = prefix.toLowerCase();
const scored: { def: AssetDef; score: number }[] = []; const scored: { def: AssetDef; score: number }[] = [];
const seen = new Set<string>();
const consider = (def: AssetDef) => { const consider = (def: AssetDef) => {
const key = `${def.type}:${def.id.toLowerCase()}:${def.file}:${def.line}`;
if (seen.has(key)) return;
seen.add(key);
if (!def.id.toLowerCase().startsWith(lower)) return; if (!def.id.toLowerCase().startsWith(lower)) return;
let score = 3; let score = 3;
if (refType && model.isAssignableTo(def.type, refType)) score = 1; if (refType && model.isAssignableTo(def.type, refType)) score = 1;
if (selfType && model.isAssignableTo(def.type, selfType)) score = 0; if (selfType && model.isAssignableTo(def.type, selfType)) score = 0;
if (def.origin === "project") score -= 0.2; if (def.origin === "project") score -= 0.2;
if (def.stream === "local") score -= 0.4;
scored.push({ def, score }); scored.push({ def, score });
}; };
const targetType = selfType ?? refType; const targetType = selfType ?? refType;
const localAssets = idx.local?.assets;
const localById = idx.local?.assetsById;
if (localAssets || localById) {
if (!targetType) {
for (const list of localById!.values()) for (const d of list) consider(d);
} else {
for (const [typeName, byId] of localAssets!) {
if (!model.isAssignableTo(typeName, targetType)) continue;
for (const list of byId.values()) for (const d of list) consider(d);
}
}
}
if (!targetType) { if (!targetType) {
for (const list of idx.assetsById.values()) for (const d of list) consider(d); for (const list of idx.assetsById.values()) for (const d of list) consider(d);
} else { } else {
@@ -324,8 +492,8 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
} }
} }
scored.sort((a, b) => a.score - b.score || a.def.id.localeCompare(b.def.id)); const top = topScoredDefs(scored, MAX_VALUE_ITEMS);
return scored.slice(0, MAX_VALUE_ITEMS).map(({ def }) => { const items = top.map(({ def }) => {
const origin = def.origin === "manifest" ? `manifest (${def.manifestSource ?? ""})` : def.origin; const origin = def.origin === "manifest" ? `manifest (${def.manifestSource ?? ""})` : def.origin;
const doc = new vscode.MarkdownString(); const doc = new vscode.MarkdownString();
doc.appendCodeblock(def.id); doc.appendCodeblock(def.id);
@@ -334,63 +502,375 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
doc.appendMarkdown(`**Origin**: ${origin}`); doc.appendMarkdown(`**Origin**: ${origin}`);
return make(def.id, vscode.CompletionItemKind.Value, `${def.type} · ${origin}`, doc.value); return make(def.id, vscode.CompletionItemKind.Value, `${def.type} · ${origin}`, doc.value);
}); });
return this.limitItems(items, scored.length);
} }
private defineItems( private defineItems(
idx: ModIndex, idx: ModIndex,
prefix: string, prefix: string,
make: (label: string, kind: vscode.CompletionItemKind, detail: string, doc?: string) => vscode.CompletionItem, make: (label: string, kind: vscode.CompletionItemKind, detail: string, doc?: string) => vscode.CompletionItem,
): vscode.CompletionItem[] { ): vscode.CompletionItem[] | vscode.CompletionList<vscode.CompletionItem> {
const lower = prefix.replace(/^[=$]*/, "").toLowerCase(); const lower = prefix.replace(/^[=$]*/, "").toLowerCase();
const items: vscode.CompletionItem[] = []; const items: vscode.CompletionItem[] = [];
for (const [key, defs] of idx.defines) { const seen = new Set<string>();
for (const defines of [idx.local?.defines, idx.defines]) {
if (!defines) continue;
for (const [key, defs] of defines) {
if (!key.includes(lower)) continue; if (!key.includes(lower)) continue;
const def = defs[0]; const def = defs[0];
const dedupe = `${def.name.toLowerCase()}:${def.file}:${def.line}`;
if (seen.has(dedupe)) continue;
seen.add(dedupe);
const label = `$${def.name}`; const label = `$${def.name}`;
const item = make(label, vscode.CompletionItemKind.Constant, "Define", def.value); const item = make(label, vscode.CompletionItemKind.Constant, "Define", def.value);
item.insertText = label; item.insertText = label;
items.push(item); items.push(item);
} }
return items.slice(0, MAX_VALUE_ITEMS); }
return this.limitItems(items, items.length);
}
private localIdItems(
el: LogicalElement,
prefix: string,
make: (label: string, kind: vscode.CompletionItemKind, detail: string, doc?: string) => vscode.CompletionItem,
): vscode.CompletionItem[] | vscode.CompletionList<vscode.CompletionItem> {
const root = findContainingGameObject(el);
if (!root) return [];
const lower = prefix.toLowerCase();
const items: vscode.CompletionItem[] = [];
for (const { id } of collectLocalIds(root)) {
if (!id.toLowerCase().startsWith(lower)) continue;
items.push(
make(
id,
vscode.CompletionItemKind.Value,
"local module",
"Pipeline-local id in the enclosing GameObject (includes xi:include targets).",
),
);
}
return this.limitItems(items, items.length);
}
/**
* VS Code filters the returned items client-side while the user keeps
* typing. Once the result is capped, the list must be marked incomplete so
* the provider is asked again with the narrower prefix; otherwise a wanted
* id (e.g. CrateDebris_01) can be silently cut off behind the first 400
* alphabetically-earlier candidates and never reappear.
*/
private limitItems<T extends vscode.CompletionItem>(
items: T[],
total: number,
): T[] | vscode.CompletionList<T> {
if (total <= MAX_VALUE_ITEMS) return items;
return new vscode.CompletionList(items.slice(0, MAX_VALUE_ITEMS), true);
} }
// ── Element content ─────────────────────────────────────────────── // ── Element content ───────────────────────────────────────────────
private contentItems( private contentItems(
ctx: CompletionContext, ctx: CompletionContext,
_document: vscode.TextDocument, document: vscode.TextDocument,
_position: vscode.Position, position: vscode.Position,
_idx: ModIndex, idx: ModIndex | null,
): vscode.CompletionItem[] { ): vscode.CompletionItem[] | vscode.CompletionList<vscode.CompletionItem> {
const el = ctx.element; const el = ctx.element;
if (!el) return []; if (!el) return [];
// Reuse element-name suggestions with a plain replacement range. const elType = resolveElementType(el);
const info = elType ? model.typeInfo(elType) : undefined;
// Simple-content element: the text between the tags is the value itself
// (e.g. <CreateObject>CrateDebris_01</CreateObject>), so offer value
// completions (asset ids / enums / defines) instead of child elements.
if (info?.kind === "simple") {
return this.simpleContentItems(el, elType, info, document, position, idx);
}
return this.contentChildItems(el, document, position);
}
private simpleContentItems(
el: XmlElement,
elType: string | null,
info: SimpleTypeInfo,
document: vscode.TextDocument,
position: vscode.Position,
idx: ModIndex | null,
): vscode.CompletionItem[] | vscode.CompletionList<vscode.CompletionItem> {
const text = document.getText();
const offset = document.offsetAt(position);
const token = textContentTokenAt(text, el, offset);
const prefix = token ? text.slice(token.start, Math.min(offset, token.end)) : "";
const rawPrefix = text.slice(el.startTagEnd, offset);
const isList = info.isList === true;
const seg = isList
? splitListValuePrefix(rawPrefix)
: { token: prefix, start: token ? token.start - el.startTagEnd : 0 };
const rangeStart = token ? token.start : offset;
const valueRange = new vscode.Range(
document.positionAt(rangeStart),
document.positionAt(Math.max(rangeStart, offset)),
);
const make = (
label: string,
kind: vscode.CompletionItemKind,
detail: string,
doc?: string,
range?: vscode.Range,
insertText?: string,
) => {
const item = new vscode.CompletionItem(label, kind);
item.range = range ?? valueRange;
item.insertText = insertText ?? label;
item.detail = detail;
if (doc) item.documentation = new vscode.MarkdownString(doc);
return item;
};
// Typed asset references only (isRef without refType is used by real
// data for shader constants / mesh sub-object names, not global ids).
if (info.refType) {
if (!idx) return [];
return this.assetIdItems(idx, null, info.refType, seg.token, make);
}
if (info.enumValues.length) {
if (isList) return this.listEnumItems(info, rawPrefix, seg, valueRange, make);
return info.enumValues
.filter((v) => v.toLowerCase().startsWith(seg.token.toLowerCase()))
.map((v) => make(v, vscode.CompletionItemKind.EnumMember, elType ?? "enum"));
}
if (idx && info.allowsDefine) {
return this.defineItems(idx, seg.token, make);
}
return [];
}
private contentChildItems(
el: XmlElement,
document: vscode.TextDocument,
position: vscode.Position,
): vscode.CompletionItem[] {
const elType = resolveElementType(el); const elType = resolveElementType(el);
const names = elType ? model.childrenOfType(elType) : model.childrenOfElement(el.name); const names = elType ? model.childrenOfType(elType) : model.childrenOfElement(el.name);
const items: vscode.CompletionItem[] = []; const items: vscode.CompletionItem[] = [];
if (!names.length) return items;
const text = document.getText();
const offset = document.offsetAt(position);
// When the user already typed "<" (optionally followed by a partial
// name), keep that "<" and replace only the name area; the inserted
// snippet then has no leading "<" so the range text stays a valid
// filter prefix ("", "Cr", ...) instead of "<" (which would hide every
// item). Without a typed "<" the full "<Name>…</Name>" is inserted.
const { rangeStart, typedOpen } = contentElementRange(text, offset);
const range = new vscode.Range(document.positionAt(rangeStart), position);
for (const child of names) { for (const child of names) {
const item = new vscode.CompletionItem(child.name, vscode.CompletionItemKind.Field); const item = new vscode.CompletionItem(child.name, vscode.CompletionItemKind.Field);
item.insertText = this.elementSnippet(child.name, child.type); item.range = range;
item.insertText = this.elementSnippet(child.name, child.type, !typedOpen);
const type = child.type; const type = child.type;
const info = type ? model.typeInfo(type) : undefined; const info = type ? model.typeInfo(type) : undefined;
item.detail = type ? `RA3 XML · ${type}` : "RA3 XML"; item.detail = type ? `RA3 XML · ${type}` : "RA3 XML";
const doc = child.doc || (info?.kind === "complex" ? info.doc : ""); const doc = child.doc || (info?.kind === "complex" ? info.doc : "");
if (doc) item.documentation = new vscode.MarkdownString(doc); if (doc) item.documentation = new vscode.MarkdownString(doc);
if (info?.kind === "simple" && this.simpleContentValueKind(info)) {
item.command = {
command: "editor.action.triggerSuggest",
title: "Suggest content value",
};
}
items.push(item); items.push(item);
} }
return items; return items;
} }
private simpleContentValueKind(info: SimpleTypeInfo): boolean {
return (
info.refType != null ||
info.enumValues.length > 0 ||
info.allowsDefine
);
}
} }
function findAttributeWordStart( interface ScoredDef {
document: vscode.TextDocument, def: AssetDef;
position: vscode.Position, score: number;
el: { start: number }, }
): number {
const offset = document.offsetAt(position); function compareScoredDefs(a: ScoredDef, b: ScoredDef): number {
const tagStart = el.start; return a.score - b.score || a.def.id.localeCompare(b.def.id);
}
/**
* Returns the best `limit` scored definitions without sorting the whole
* candidate list. A max-heap keeps the worst item of the current top set at
* the root, so every additional candidate only needs an O(log limit) check.
*/
function topScoredDefs(scored: ScoredDef[], limit: number): ScoredDef[] {
if (scored.length <= limit) {
scored.sort(compareScoredDefs);
return scored;
}
const better = (a: ScoredDef, b: ScoredDef) => compareScoredDefs(a, b) < 0;
const heap: ScoredDef[] = [];
const swap = (i: number, j: number) => {
const t = heap[i];
heap[i] = heap[j];
heap[j] = t;
};
const siftUp = (i: number) => {
while (i > 0) {
const parent = (i - 1) >> 1;
if (better(heap[i], heap[parent])) {
swap(i, parent);
i = parent;
} else {
break;
}
}
};
const siftDown = (i: number) => {
for (;;) {
const left = i * 2 + 1;
const right = left + 1;
let worst = i;
if (left < heap.length && better(heap[left], heap[worst])) worst = left;
if (right < heap.length && better(heap[right], heap[worst])) worst = right;
if (worst === i) break;
swap(i, worst);
i = worst;
}
};
for (const entry of scored) {
if (heap.length < limit) {
heap.push(entry);
siftUp(heap.length - 1);
} else if (better(entry, heap[0])) {
heap[0] = entry;
siftDown(0);
}
}
heap.sort(compareScoredDefs);
return heap;
}
/**
* Replacement range for a child-element completion in element content:
* - when the user typed "<" (optionally followed by a partial name), the
* "<" is kept and the range covers the name area after it;
* - a partial name typed without "<" is replaced as a word;
* - whitespace-only content leaves the range empty at the cursor.
* Typing a closing tag ("</" / "</Name") never suggests child elements.
*/
function contentElementRange(
text: string,
offset: number,
): { rangeStart: number; typedOpen: boolean } {
let j = offset;
while (j > 0 && /[ \t]/.test(text[j - 1])) j--;
let i = j;
while (i > 0 && /[A-Za-z0-9_:.-]/.test(text[i - 1])) i--;
if (i >= 2 && text.slice(i - 2, i) === "</") {
return { rangeStart: offset, typedOpen: false };
}
if (i > 0 && text[i - 1] === "<") {
return { rangeStart: i, typedOpen: true };
}
if (i < j) return { rangeStart: i, typedOpen: false };
return { rangeStart: offset, typedOpen: false };
}
interface AttributeInsertLayout {
/** Offset where the completed attribute name starts replacing the text. */
rangeStart: number;
/** Text to insert before the attribute name (space / newline + indent). */
prefix: string;
}
/**
* Computes the insertion layout for an attribute-name completion:
* - a leading space when the cursor sits directly against a closing quote;
* - a plain newline when the element's attributes are laid out one per line
* (the editor supplies the new line's base indentation itself; embedding
* our own indent here would be ADDED on top of it, e.g. 3+3=6, 6+3=9);
* - replacing the current line's whitespace with that indentation when the
* user already started a new line.
*
* The indentation anchor is deliberately NOT the last parsed attribute:
* a half-typed attribute name on the cursor's new line has no value yet, and
* using its line indentation would copy the editor's own auto-indent (which
* can grow line by line) into every inserted attribute. Instead we use the
* first complete attribute that starts on its own line, which is stable and
* pre-existing, and only fall back to the last complete attribute when the
* whole element is inline.
*/
function attributeInsertLayout(
text: string,
el: XmlElement,
offset: number,
): AttributeInsertLayout {
const wordStart = findAttributeWordStart(text, offset, el.start);
const attrs = el.attrs;
const complete = attrs.filter((a) => a.hasValue);
const last = complete.length ? complete[complete.length - 1] : null;
const lastEnd = last ? attributeEndOffset(last) : -1;
const alreadyOnNewLine = lastEnd >= 0 && text.slice(lastEnd, offset).includes("\n");
// Canonical indent anchor: the first complete attribute that starts on its
// own line. Fall back to the last complete attribute for inline elements.
let anchor: XmlAttribute | null = null;
let onePerLine = false;
let prevEnd = el.start + 1 + el.name.length;
for (const a of complete) {
if (text.slice(prevEnd, a.nameStart).includes("\n")) {
anchor = a;
onePerLine = true;
break;
}
prevEnd = attributeEndOffset(a);
}
if (!anchor && complete.length) anchor = complete[complete.length - 1];
const indent = anchor
? text.slice(0, anchor.nameStart).match(/[ \t]*$/)?.[0] ?? ""
: "";
if (!onePerLine) {
if (alreadyOnNewLine) {
// Inline-style file, but the user started a new line: keep whatever
// indentation they already typed.
return { rangeStart: wordStart, prefix: "" };
}
const needsSpace = wordStart > el.start + 1 && !/\s/.test(text[wordStart - 1]);
return { rangeStart: wordStart, prefix: needsSpace ? " " : "" };
}
if (alreadyOnNewLine) {
const lineStart = text.lastIndexOf("\n", offset - 1) + 1;
return { rangeStart: lineStart, prefix: indent };
}
// Insert on a new line. The editor adds the current line's indentation to
// the new line, so we must NOT embed our own indent here (it would
// compound). If whitespace was typed between the previous attribute and
// the cursor (e.g. a space used to trigger the suggestion popup), consume
// it so it does not linger as a trailing space.
const wsStart =
lastEnd >= 0 &&
wordStart > lastEnd &&
/^[ \t]*$/.test(text.slice(lastEnd, wordStart))
? lastEnd
: wordStart;
return { rangeStart: wsStart, prefix: "\n" };
}
function attributeEndOffset(attr: XmlAttribute): number {
return attr.quoteEnd >= 0 ? attr.quoteEnd : attr.nameEnd;
}
function findAttributeWordStart(text: string, offset: number, tagStart: number): number {
let i = offset; let i = offset;
const text = document.getText();
while (i > tagStart) { while (i > tagStart) {
const c = text[i - 1]; const c = text[i - 1];
if (/[\s=<>"/]/.test(c)) break; if (/[\s=<>"/]/.test(c)) break;
@@ -398,3 +878,23 @@ function findAttributeWordStart(
} }
return i; return i;
} }
/** Type-appropriate example values for common RA3 XSD scalar types. */
const DEFAULT_VALUE_BY_TYPE: Record<string, string> = {
angle: "0d",
time: "0s",
velocity: "0.0",
percentage: "100%",
sagereal: "0.0",
sageint: "0",
sageunsignedint: "0",
float: "0.0",
double: "0.0",
int: "0",
unsignedint: "0",
unsignedbyte: "0",
byte: "0",
short: "0",
long: "0",
decimal: "0.0",
};
+157 -25
View File
@@ -1,15 +1,20 @@
import * as vscode from "vscode"; import * as vscode from "vscode";
import { dirname } from "node:path"; import { dirname } from "node:path";
import { LineMap, parseXml, type XmlElement } from "../language/xmlParser"; import { LineMap, type XmlElement } from "../language/xmlParser";
import { resolveElementType } from "../language/typeContext"; import { resolveElementType } from "../language/typeContext";
import { resolveSource, buildSearchPaths } from "../indexer/includeResolver"; import { resolveSource, buildSearchPaths } from "../indexer/includeResolver";
import * as model from "../model/schemaModel"; import * as model from "../model/schemaModel";
import type { ModWorkspace } from "../workspace"; import type { ModWorkspace } from "../workspace";
import type { ModIndex } from "../indexer/types"; import type { ModIndex } from "../indexer/types";
import { import {
isReferenceContentType,
isReferenceAttributeOfType, isReferenceAttributeOfType,
mergeLocalAndGlobalDefs,
resolveContentReferenceTargets,
resolveReferenceTargetsForType, resolveReferenceTargetsForType,
} from "../indexer/refs"; } from "../indexer/refs";
import type { LogicalElement } from "../indexer/logicalTree";
import { scopePathKey } from "../indexer/localScope";
export class Ra3Diagnostics { export class Ra3Diagnostics {
private collection: vscode.DiagnosticCollection; private collection: vscode.DiagnosticCollection;
@@ -19,15 +24,19 @@ export class Ra3Diagnostics {
} }
async update(document: vscode.TextDocument): Promise<void> { async update(document: vscode.TextDocument): Promise<void> {
const idx = this.ws.index; if (!this.ws.isRa3Workspace()) {
if (!idx) {
this.collection.set(document.uri, []); this.collection.set(document.uri, []);
return; return;
} }
const scope = await this.ws.getScope(document);
const idx = scope.merged;
const text = document.getText(); const text = document.getText();
const lineMap = new LineMap(text); const lineMap = new LineMap(text);
const doc = parseXml(text); const doc = scope.parse;
const diags: vscode.Diagnostic[] = []; const diags: vscode.Diagnostic[] = [];
// Reference/duplicate checks are provisional while the index is
// incomplete or stale: "not found" may be a false positive.
const provisional = idx ? !idx.complete || idx.stale === true : false;
for (const err of doc.errors) { for (const err of doc.errors) {
diags.push( diags.push(
@@ -44,7 +53,15 @@ export class Ra3Diagnostics {
} }
if (doc.root) { if (doc.root) {
this.checkElements(doc.root, doc, lineMap, idx, document, diags); this.checkElements(
scope.expanded.root,
scope.expanded,
lineMap,
idx,
document,
diags,
provisional,
);
} }
this.collection.set(document.uri, diags); this.collection.set(document.uri, diags);
@@ -59,19 +76,29 @@ export class Ra3Diagnostics {
} }
private checkElements( private checkElements(
root: XmlElement, root: LogicalElement | null,
doc: { elements: XmlElement[] }, doc: { elements: LogicalElement[] },
lineMap: LineMap, lineMap: LineMap,
idx: ModIndex, idx: ModIndex | null,
document: vscode.TextDocument, document: vscode.TextDocument,
diags: vscode.Diagnostic[], diags: vscode.Diagnostic[],
provisional: boolean,
): void { ): void {
const settings = this.ws.settings; const settings = this.ws.settings;
const fileDuplicates = new Map<string, { line: number }>(); const fileDuplicates = new Map<string, { line: number }>();
for (const el of doc.elements) { for (const el of doc.elements) {
// Only report diagnostics for nodes that belong to the document being
// edited. Nodes spliced in through xi:include keep their own source
// file and are diagnosed when that file is opened.
if (scopePathKey(el.sourceFile) !== scopePathKey(document.uri.fsPath)) {
continue;
}
const local = localName(el.name); const local = localName(el.name);
const isTopLevel = el.parent === root && !["Tags", "Includes", "Defines"].includes(local); const isTopLevel =
root !== null &&
el.parent === root &&
!["Tags", "Includes", "Defines"].includes(local);
const range = tagRange(document, el); const range = tagRange(document, el);
// Top-level assets must have an id. // Top-level assets must have an id.
@@ -111,6 +138,7 @@ export class Ra3Diagnostics {
document, document,
idx, idx,
diags, diags,
provisional,
); );
} }
} }
@@ -169,8 +197,10 @@ export class Ra3Diagnostics {
document, document,
idx, idx,
diags, diags,
provisional,
); );
} }
this.checkContentReferences(el, elType, document, idx, diags, provisional);
} }
// Include-specific checks. // Include-specific checks.
@@ -184,12 +214,17 @@ export class Ra3Diagnostics {
type: string, type: string,
id: string, id: string,
document: vscode.TextDocument, document: vscode.TextDocument,
idx: ModIndex, idx: ModIndex | null,
diags: vscode.Diagnostic[], diags: vscode.Diagnostic[],
provisional: boolean,
): void { ): void {
if (!idx) return;
const byType = idx.assets.get(type); const byType = idx.assets.get(type);
const defs = byType?.get(id.toLowerCase()); const defs = mergeLocalAndGlobalDefs(
if (!defs || defs.length < 2) return; idx.local?.assets.get(type)?.get(id.toLowerCase()),
byType?.get(id.toLowerCase()),
);
if (defs.length < 2) return;
const self = defs.filter( const self = defs.filter(
(d) => (d) =>
d.origin === "project" && d.origin === "project" &&
@@ -211,7 +246,8 @@ export class Ra3Diagnostics {
diags.push( diags.push(
this.diag( this.diag(
range, range,
`Duplicate id "${id}" for <${type}> (also defined in ${other.file})`, `Duplicate id "${id}" for <${type}> (also defined in ${other.file})` +
(provisional ? " (based on a partial index)" : ""),
vscode.DiagnosticSeverity.Error, vscode.DiagnosticSeverity.Error,
"duplicate-id", "duplicate-id",
), ),
@@ -225,8 +261,9 @@ export class Ra3Diagnostics {
value: string, value: string,
attr: { valueStart: number; valueEnd: number }, attr: { valueStart: number; valueEnd: number },
document: vscode.TextDocument, document: vscode.TextDocument,
idx: ModIndex, idx: ModIndex | null,
diags: vscode.Diagnostic[], diags: vscode.Diagnostic[],
provisional: boolean,
): void { ): void {
if (!value) return; if (!value) return;
const range = new vscode.Range( const range = new vscode.Range(
@@ -238,13 +275,19 @@ export class Ra3Diagnostics {
const defineRe = /\$([A-Za-z_][A-Za-z0-9_]*)/g; const defineRe = /\$([A-Za-z_][A-Za-z0-9_]*)/g;
let m: RegExpExecArray | null; let m: RegExpExecArray | null;
while ((m = defineRe.exec(value)) !== null) { while ((m = defineRe.exec(value)) !== null) {
if (!idx.defines.has(m[1].toLowerCase())) { if (
idx &&
!(idx.local?.defines.has(m[1].toLowerCase()) ??
idx.defines.has(m[1].toLowerCase()))
) {
const code = provisional ? "undefined-define-indexing" : "undefined-define";
diags.push( diags.push(
this.diag( this.diag(
range, range,
`Undefined define "$${m[1]}"`, `Undefined define "$${m[1]}"` +
(provisional ? " (index incomplete — may be a false positive)" : ""),
vscode.DiagnosticSeverity.Warning, vscode.DiagnosticSeverity.Warning,
"undefined-define", code,
), ),
); );
} }
@@ -253,10 +296,13 @@ export class Ra3Diagnostics {
if (value.startsWith("$") || value.startsWith("=")) return; if (value.startsWith("$") || value.startsWith("=")) return;
const severity = this.ws.settings.reportUnresolvedReferences; const severity = this.ws.settings.reportUnresolvedReferences;
if (severity === "none") return; if (severity === "none") return;
if (!idx) return;
if (!isReferenceAttributeOfType(elType, attrName)) return; if (!isReferenceAttributeOfType(elType, attrName)) return;
const targets = resolveReferenceTargetsForType(idx, elType, attrName, value); const targets = resolveReferenceTargetsForType(idx, elType, attrName, value);
if (targets.length) return; if (targets.length) return;
const anyDef = idx.assetsById.has(value.toLowerCase()); const anyDef =
(idx.local?.assetsById.has(value.toLowerCase()) ?? false) ||
idx.assetsById.has(value.toLowerCase());
const attrRef = model const attrRef = model
.attributesOfType(elType) .attributesOfType(elType)
.find((a) => a.name === attrName); .find((a) => a.name === attrName);
@@ -265,16 +311,96 @@ export class Ra3Diagnostics {
: attrRef?.isRef : attrRef?.isRef
? "of the expected declared type" ? "of the expected declared type"
: "matching"; : "matching";
const code = provisional ? "unresolved-reference-indexing" : "unresolved-reference";
const baseMessage = anyDef
? `Reference "${value}" has no definition ${expected} (ids with the same name exist for other types)`
: `Unresolved reference "${value}" (not found in the current index)`;
diags.push( diags.push(
this.diag( this.diag(
range, range,
anyDef provisional
? `Reference "${value}" has no definition ${expected} (ids with the same name exist for other types)` ? `${baseMessage} (index incomplete — may be a false positive)`
: `Unresolved reference "${value}" (not found in the current index)`, : baseMessage,
severity === "warning" severity === "warning"
? vscode.DiagnosticSeverity.Warning ? vscode.DiagnosticSeverity.Warning
: vscode.DiagnosticSeverity.Information, : vscode.DiagnosticSeverity.Information,
"unresolved-reference", code,
),
);
}
private checkContentReferences(
el: XmlElement,
elType: string | null,
document: vscode.TextDocument,
idx: ModIndex | null,
diags: vscode.Diagnostic[],
provisional: boolean,
): void {
// Only simple-content elements carry a text value; complex elements'
// "content" is child markup and must not be scanned for value refs.
const info = elType ? model.typeInfo(elType) : undefined;
if (info?.kind !== "simple") return;
if (el.selfClosing || el.closeTagStart < 0) return;
const text = document.getText();
const raw = text.slice(el.startTagEnd, el.closeTagStart);
const value = raw.trim();
if (!value) return;
const valueStart = el.startTagEnd + raw.indexOf(value);
const range = new vscode.Range(
document.positionAt(valueStart),
document.positionAt(valueStart + value.length),
);
// Undefined $DEFINE references.
const defineRe = /\$([A-Za-z_][A-Za-z0-9_]*)/g;
let m: RegExpExecArray | null;
while ((m = defineRe.exec(value)) !== null) {
if (
idx &&
!(
idx.local?.defines.has(m[1].toLowerCase()) ??
idx.defines.has(m[1].toLowerCase())
)
) {
const code = provisional ? "undefined-define-indexing" : "undefined-define";
diags.push(
this.diag(
range,
`Undefined define "$${m[1]}"` +
(provisional ? " (index incomplete — may be a false positive)" : ""),
vscode.DiagnosticSeverity.Warning,
code,
),
);
}
}
if (value.startsWith("$") || value.startsWith("=")) return;
const severity = this.ws.settings.reportUnresolvedReferences;
if (severity === "none" || !idx) return;
if (!isReferenceContentType(elType)) return;
const targets = resolveContentReferenceTargets(idx, elType, value);
if (targets.length) return;
const anyDef =
(idx.local?.assetsById.has(value.toLowerCase()) ?? false) ||
idx.assetsById.has(value.toLowerCase());
const refType = info.refType;
const expected = refType ? `of type \`${refType}\`` : "of the expected declared type";
const code = provisional ? "unresolved-reference-indexing" : "unresolved-reference";
const baseMessage = anyDef
? `Reference "${value}" has no definition ${expected} (ids with the same name exist for other types)`
: `Unresolved reference "${value}" (not found in the current index)`;
diags.push(
this.diag(
range,
provisional
? `${baseMessage} (index incomplete — may be a false positive)`
: baseMessage,
severity === "warning"
? vscode.DiagnosticSeverity.Warning
: vscode.DiagnosticSeverity.Information,
code,
), ),
); );
} }
@@ -282,7 +408,7 @@ export class Ra3Diagnostics {
private checkInclude( private checkInclude(
el: XmlElement, el: XmlElement,
document: vscode.TextDocument, document: vscode.TextDocument,
idx: ModIndex, idx: ModIndex | null,
diags: vscode.Diagnostic[], diags: vscode.Diagnostic[],
): void { ): void {
const typeAttr = el.attrs.find((a) => a.name === "type"); const typeAttr = el.attrs.find((a) => a.name === "type");
@@ -301,12 +427,18 @@ export class Ra3Diagnostics {
); );
} }
if (!sourceAttr?.hasValue) return; if (!sourceAttr?.hasValue) return;
const searchPaths = idx
? buildSearchPaths(idx.sdkDir, idx.projectDir)
: this.ws.searchPaths();
if (!searchPaths) return;
const resolved = resolveSource( const resolved = resolveSource(
sourceAttr.value, sourceAttr.value,
dirname(document.uri.fsPath), dirname(document.uri.fsPath),
buildSearchPaths(idx.sdkDir, idx.projectDir), searchPaths,
); );
if (!resolved.path && !idx.sourceCandidates.some((c) => c.source === sourceAttr.value)) { const candidateHit =
idx?.sourceCandidates.some((c) => c.source === sourceAttr.value) ?? false;
if (!resolved.path && !candidateHit) {
diags.push( diags.push(
this.diag( this.diag(
new vscode.Range( new vscode.Range(
+141 -28
View File
@@ -1,13 +1,22 @@
import * as vscode from "vscode"; import * as vscode from "vscode";
import { findElementAt, parseXml } from "../language/xmlParser"; import { findElementAt, textContentTokenAt } from "../language/xmlParser";
import { resolveElementType } from "../language/typeContext"; import { resolveElementType } from "../language/typeContext";
import * as model from "../model/schemaModel"; import * as model from "../model/schemaModel";
import type { ModWorkspace } from "../workspace"; import type { ModWorkspace } from "../workspace";
import type { ModIndex } from "../indexer/types";
import { import {
isLocalReferenceAttribute,
isReferenceAttributeOfType, isReferenceAttributeOfType,
isReferenceContentType,
resolveContentReferenceTargets,
resolveReferenceTargetsForType, resolveReferenceTargetsForType,
type ReferenceTarget,
} from "../indexer/refs"; } from "../indexer/refs";
import {
findContainingGameObject,
findLocalId,
type LogicalElement,
} from "../indexer/logicalTree";
import { scopePathKey, type DocumentScope } from "../indexer/localScope";
import { dirname } from "node:path"; import { dirname } from "node:path";
import { buildSearchPaths, resolveSource } from "../indexer/includeResolver"; import { buildSearchPaths, resolveSource } from "../indexer/includeResolver";
@@ -19,9 +28,11 @@ export class Ra3HoverProvider implements vscode.HoverProvider {
position: vscode.Position, position: vscode.Position,
_token: vscode.CancellationToken, _token: vscode.CancellationToken,
): Promise<vscode.Hover | null> { ): Promise<vscode.Hover | null> {
const text = document.getText(); if (!this.ws.isRa3Workspace()) return null;
const offset = document.offsetAt(position); const offset = document.offsetAt(position);
const doc = parseXml(text); const text = document.getText();
const scope = await this.ws.getScope(document);
const doc = scope.expanded;
const el = findElementAt(doc, offset); const el = findElementAt(doc, offset);
if (!el) return null; if (!el) return null;
const elType = resolveElementType(el); const elType = resolveElementType(el);
@@ -35,9 +46,15 @@ export class Ra3HoverProvider implements vscode.HoverProvider {
// Attribute value. // Attribute value.
for (const attr of el.attrs) { for (const attr of el.attrs) {
if (attr.hasValue && offset >= attr.valueStart && offset <= attr.valueEnd) { if (attr.hasValue && offset >= attr.valueStart && offset <= attr.valueEnd) {
return this.valueHover(el, elType, attr.name, attr.value, document, this.ws.index); return this.valueHover(el, elType, attr.name, attr.value, document, scope);
} }
} }
// Element text content (e.g. <CreateObject>CrateDebris_01</CreateObject>).
const contentToken = textContentTokenAt(text, el, offset);
if (contentToken) {
const h = this.contentHover(elType, contentToken.value, document, scope);
if (h) return h;
}
// Element name. // Element name.
const nameStart = el.start + 1; const nameStart = el.start + 1;
if (offset >= nameStart && offset <= nameStart + el.name.length) { if (offset >= nameStart && offset <= nameStart + el.name.length) {
@@ -116,22 +133,19 @@ export class Ra3HoverProvider implements vscode.HoverProvider {
attrName: string, attrName: string,
value: string, value: string,
document: vscode.TextDocument, document: vscode.TextDocument,
idx: ModIndex | null, scope: DocumentScope,
): vscode.Hover | null { ): vscode.Hover | null {
const idx = scope.merged;
const md = new vscode.MarkdownString(); const md = new vscode.MarkdownString();
// $DEFINE reference. // $DEFINE reference.
const defineMatch = /\$([A-Za-z_][A-Za-z0-9_]*)/.exec(value); const defineMatch = /\$([A-Za-z_][A-Za-z0-9_]*)/.exec(value);
if (defineMatch && idx) { if (defineMatch && idx) {
const defs = idx.defines.get(defineMatch[1].toLowerCase()); const defs =
if (defs?.length) { idx.local?.defines.get(defineMatch[1].toLowerCase()) ??
const d = defs[0]; idx.defines.get(defineMatch[1].toLowerCase());
md.appendMarkdown(`**Define** \`$${d.name}\` \n`); const h = this.defineHover(defs, document);
md.appendCodeblock(d.value); if (h) return h;
const rel = relativePath(document, d.file);
md.appendMarkdown(`Defined in \`${rel}:${d.line}\``);
return new vscode.Hover(md);
}
} }
// Include source. // Include source.
@@ -139,11 +153,14 @@ export class Ra3HoverProvider implements vscode.HoverProvider {
(el.name === "Include" && attrName === "source") || (el.name === "Include" && attrName === "source") ||
(el.name === "xi:include" && attrName === "href") (el.name === "xi:include" && attrName === "href")
) { ) {
const resolved = idx const searchPaths = idx
? buildSearchPaths(idx.sdkDir, idx.projectDir)
: this.ws.searchPaths();
const resolved = searchPaths
? resolveSource( ? resolveSource(
value, value,
dirname(document.uri.fsPath), dirname(document.uri.fsPath),
buildSearchPaths(idx.sdkDir, idx.projectDir), searchPaths,
).path ).path
: null; : null;
if (resolved) { if (resolved) {
@@ -161,11 +178,93 @@ export class Ra3HoverProvider implements vscode.HoverProvider {
return new vscode.Hover(md); return new vscode.Hover(md);
} }
// Pipeline-local (Poid) references: resolve inside the enclosing
// GameObject's logical subtree (including xi:include targets).
if (
isLocalReferenceAttribute(elType, attrName) &&
attrName.toLowerCase() !== "id"
) {
return this.localIdHover(scope, el as LogicalElement, value, document);
}
// Asset reference / inheritFrom. // Asset reference / inheritFrom.
if (idx) { if (!idx) {
if (isReferenceAttributeOfType(elType, attrName)) {
md.appendMarkdown(
"Index is still building — references cannot be resolved yet.",
);
return new vscode.Hover(md);
}
return null;
}
{
if (!isReferenceAttributeOfType(elType, attrName)) return null; if (!isReferenceAttributeOfType(elType, attrName)) return null;
const targets = resolveReferenceTargetsForType(idx, elType, attrName, value); const targets = resolveReferenceTargetsForType(idx, elType, attrName, value);
if (targets.length) { if (targets.length) return this.definitionsHover(targets, document);
const attrRef = model
.attributesOfType(elType)
.find((a) => a.name === attrName);
const expected = attrRef?.refType
? ` of type \`${attrRef.refType}\``
: attrRef?.isRef
? " of the expected declared type"
: "";
return this.noDefinitionHover(expected);
}
}
/**
* Hover for text inside a simple-content element whose type is a typed
* asset reference (e.g. <CreateObject> with GameObjectWeakRef).
*/
private contentHover(
elType: string | null,
value: string,
document: vscode.TextDocument,
scope: DocumentScope,
): vscode.Hover | null {
const idx = scope.merged;
const defineMatch = /\$([A-Za-z_][A-Za-z0-9_]*)/.exec(value);
if (defineMatch && idx) {
const defs =
idx.local?.defines.get(defineMatch[1].toLowerCase()) ??
idx.defines.get(defineMatch[1].toLowerCase());
const h = this.defineHover(defs, document);
if (h) return h;
}
if (!isReferenceContentType(elType)) return null;
if (!idx) {
const md = new vscode.MarkdownString();
md.appendMarkdown("Index is still building — references cannot be resolved yet.");
return new vscode.Hover(md);
}
const targets = resolveContentReferenceTargets(idx, elType, value);
if (targets.length) return this.definitionsHover(targets, document);
const info = elType ? model.typeInfo(elType) : undefined;
const refType = info?.kind === "simple" ? info.refType : null;
return this.noDefinitionHover(
refType ? ` of type \`${refType}\`` : " of the expected declared type",
);
}
private defineHover(
defs: { name: string; value: string; file: string; line: number }[] | undefined,
document: vscode.TextDocument,
): vscode.Hover | null {
if (!defs?.length) return null;
const d = defs[0];
const md = new vscode.MarkdownString();
md.appendMarkdown(`**Define** \`$${d.name}\` \n`);
md.appendCodeblock(d.value);
const rel = relativePath(document, d.file);
md.appendMarkdown(`Defined in \`${rel}:${d.line}\``);
return new vscode.Hover(md);
}
private definitionsHover(
targets: ReferenceTarget[],
document: vscode.TextDocument,
): vscode.Hover {
const md2 = new vscode.MarkdownString(); const md2 = new vscode.MarkdownString();
md2.appendMarkdown(`**${targets.length} definition${targets.length > 1 ? "s" : ""}** \n`); md2.appendMarkdown(`**${targets.length} definition${targets.length > 1 ? "s" : ""}** \n`);
for (const { def: d } of targets.slice(0, 8)) { for (const { def: d } of targets.slice(0, 8)) {
@@ -177,14 +276,9 @@ export class Ra3HoverProvider implements vscode.HoverProvider {
} }
return new vscode.Hover(md2); return new vscode.Hover(md2);
} }
const attrRef = model
.attributesOfType(elType) private noDefinitionHover(expected: string): vscode.Hover {
.find((a) => a.name === attrName); const md = new vscode.MarkdownString();
const expected = attrRef?.refType
? ` of type \`${attrRef.refType}\``
: attrRef?.isRef
? " of the expected declared type"
: "";
md.appendMarkdown( md.appendMarkdown(
`No matching definition${expected} in the current index` + `No matching definition${expected} in the current index` +
" (may exist in a compiled manifest or vanilla data).", " (may exist in a compiled manifest or vanilla data).",
@@ -192,7 +286,26 @@ export class Ra3HoverProvider implements vscode.HoverProvider {
return new vscode.Hover(md); return new vscode.Hover(md);
} }
return null; private localIdHover(
scope: DocumentScope,
el: LogicalElement,
value: string,
document: vscode.TextDocument,
): vscode.Hover | null {
const root = findContainingGameObject(el);
if (!root) return null;
const target = findLocalId(root, value);
if (!target) return null;
const idAttr = target.attrs.find((a) => a.name === "id");
if (!idAttr?.hasValue) return null;
const lineMap = scope.lineMaps.get(scopePathKey(target.sourceFile));
const line = lineMap ? lineMap.positionAt(idAttr.valueStart).line + 1 : 0;
const md = new vscode.MarkdownString();
md.appendMarkdown(`**Local pipeline id** \`${value}\` \n`);
md.appendCodeblock(`<${target.name}>`);
const rel = relativePath(document, target.sourceFile);
md.appendMarkdown(`Defined in \`${rel}:${line}\``);
return new vscode.Hover(md);
} }
} }
+170 -26
View File
@@ -1,13 +1,25 @@
import * as vscode from "vscode"; import * as vscode from "vscode";
import { dirname } from "node:path"; import { dirname } from "node:path";
import { findElementAt, parseXml } from "../language/xmlParser"; import { findElementAt, parseXml, textContentTokenAt } from "../language/xmlParser";
import { resolveElementType } from "../language/typeContext"; import { resolveElementType } from "../language/typeContext";
import { import {
buildSearchPaths, buildSearchPaths,
resolveSource, resolveSource,
type SearchPaths, type SearchPaths,
} from "../indexer/includeResolver"; } from "../indexer/includeResolver";
import { resolveReferenceTargetsForType } from "../indexer/refs"; import {
isLocalReferenceAttribute,
isReferenceContentType,
resolveContentReferenceTargets,
resolveReferenceTargetsForType,
type ReferenceTarget,
} from "../indexer/refs";
import {
findContainingGameObject,
findLocalId,
type LogicalElement,
} from "../indexer/logicalTree";
import { scopePathKey, type DocumentScope } from "../indexer/localScope";
import type { ModWorkspace } from "../workspace"; import type { ModWorkspace } from "../workspace";
import type { AssetDef, ModIndex } from "../indexer/types"; import type { AssetDef, ModIndex } from "../indexer/types";
@@ -25,11 +37,11 @@ export class Ra3DefinitionProvider implements vscode.DefinitionProvider {
position: vscode.Position, position: vscode.Position,
_token: vscode.CancellationToken, _token: vscode.CancellationToken,
): Promise<vscode.Location | vscode.Location[] | null> { ): Promise<vscode.Location | vscode.Location[] | null> {
const idx = this.ws.index; if (!this.ws.isRa3Workspace()) return null;
if (!idx) return null; const scope = await this.ws.getScope(document);
const text = document.getText(); const idx = scope.merged;
const offset = document.offsetAt(position); const offset = document.offsetAt(position);
const doc = parseXml(text); const doc = scope.expanded;
const el = findElementAt(doc, offset); const el = findElementAt(doc, offset);
if (!el) return null; if (!el) return null;
const elType = resolveElementType(el); const elType = resolveElementType(el);
@@ -37,7 +49,19 @@ export class Ra3DefinitionProvider implements vscode.DefinitionProvider {
const attr = el.attrs.find( const attr = el.attrs.find(
(a) => a.hasValue && offset >= a.valueStart && offset <= a.valueEnd, (a) => a.hasValue && offset >= a.valueStart && offset <= a.valueEnd,
); );
if (!attr) return null; if (!attr) {
// Element text content (e.g. <CreateObject>CrateDebris_01</CreateObject>).
if (idx && isReferenceContentType(elType)) {
const token = textContentTokenAt(document.getText(), el, offset);
if (token && !token.value.startsWith("$")) {
const targets = resolveContentReferenceTargets(idx, elType, token.value);
if (targets.length) {
return this.referenceLocations(scope, idx, targets, document);
}
}
}
return null;
}
const value = attr.value; const value = attr.value;
const nameLower = attr.name.toLowerCase(); const nameLower = attr.name.toLowerCase();
@@ -46,33 +70,84 @@ export class Ra3DefinitionProvider implements vscode.DefinitionProvider {
(el.name === "Include" && nameLower === "source") || (el.name === "Include" && nameLower === "source") ||
(el.name === "include" && nameLower === "href") (el.name === "include" && nameLower === "href")
) { ) {
const resolved = const searchPaths = idx ? searchPathsFor(idx) : this.ws.searchPaths();
resolveSource(value, dirname(document.uri.fsPath), searchPathsFor(idx)).path ?? const resolved = searchPaths
idx.sourceCandidates.find((c) => c.source === value)?.path ?? ? resolveSource(value, dirname(document.uri.fsPath), searchPaths).path
null; : null;
return resolved const fallback = idx?.sourceCandidates.find((c) => c.source === value)?.path;
? new vscode.Location(vscode.Uri.file(resolved), new vscode.Position(0, 0)) const target = resolved ?? fallback ?? null;
return target
? new vscode.Location(vscode.Uri.file(target), new vscode.Position(0, 0))
: null; : null;
} }
// Asset reference / inheritFrom (filtered by the attribute's ref type). // Asset reference / inheritFrom (filtered by the attribute's ref type).
if (!idx) return null;
if (value && !value.startsWith("$")) { if (value && !value.startsWith("$")) {
let targets = resolveReferenceTargetsForType(idx, elType, attr.name, value); if (
isLocalReferenceAttribute(elType, attr.name) &&
nameLower !== "id"
) {
const local = this.localIdLocation(
scope,
el as LogicalElement,
value,
);
if (local) return local;
}
const targets = resolveReferenceTargetsForType(idx, elType, attr.name, value);
if (!targets.length) return null; if (!targets.length) return null;
return this.referenceLocations(scope, idx, targets, document);
}
return null;
}
private async referenceLocations(
scope: DocumentScope,
idx: ModIndex,
targets: ReferenceTarget[],
document: vscode.TextDocument,
): Promise<vscode.Location[] | null> {
let filtered = targets;
if ( if (
this.ws.settings.definitionMode === "project-only" && this.ws.settings.definitionMode === "project-only" &&
targets.some((t) => t.def.origin === "project") targets.some((t) => t.def.origin === "project")
) { ) {
targets = targets.filter((t) => t.def.origin === "project"); filtered = targets.filter((t) => t.def.origin === "project");
} }
const locations: vscode.Location[] = []; const locations: vscode.Location[] = [];
for (const { def } of targets.slice(0, 8)) { for (const { def } of filtered.slice(0, 8)) {
const loc = await assetDefLocation(this.ws, def, idx); const loc = await assetDefLocation(this.ws, def, idx, scope, document);
if (loc) locations.push(loc); if (loc) locations.push(loc);
} }
return locations.length ? locations : null; return locations.length ? locations : null;
} }
return null;
private localIdLocation(
scope: DocumentScope,
el: LogicalElement,
value: string,
): vscode.Location | null {
const root = findContainingGameObject(el);
if (!root) return null;
const target = findLocalId(root, value);
if (!target) return null;
const idAttr = target.attrs.find((a) => a.name === "id");
if (!idAttr?.hasValue) return null;
const lineMap = scope.lineMaps.get(scopePathKey(target.sourceFile));
if (!lineMap) {
return new vscode.Location(
vscode.Uri.file(target.sourceFile),
new vscode.Position(0, 0),
);
}
return new vscode.Location(
vscode.Uri.file(target.sourceFile),
new vscode.Range(
toVscodePosition(lineMap.positionAt(idAttr.valueStart)),
toVscodePosition(lineMap.positionAt(idAttr.valueEnd)),
),
);
} }
} }
@@ -85,6 +160,8 @@ async function assetDefLocation(
ws: ModWorkspace, ws: ModWorkspace,
def: AssetDef, def: AssetDef,
idx: ModIndex, idx: ModIndex,
scope: DocumentScope,
currentDocument: vscode.TextDocument,
): Promise<vscode.Location | null> { ): Promise<vscode.Location | null> {
if (def.origin === "manifest") { if (def.origin === "manifest") {
const src = def.manifestSource; const src = def.manifestSource;
@@ -100,6 +177,11 @@ async function assetDefLocation(
return null; return null;
} }
if (scopePathKey(def.file) === scopePathKey(currentDocument.uri.fsPath)) {
const precise = locationInCurrentDocument(scope, def.id, currentDocument);
if (precise) return precise;
}
return ( return (
(await locationInDocument(ws, def.file, def.id)) ?? (await locationInDocument(ws, def.file, def.id)) ??
new vscode.Location( new vscode.Location(
@@ -109,6 +191,36 @@ async function assetDefLocation(
); );
} }
function locationInCurrentDocument(
scope: DocumentScope,
id: string,
document: vscode.TextDocument,
): vscode.Location | null {
const el = scope.parse.elements.find((e) =>
e.attrs.some(
(a) => a.name === "id" && a.value.toLowerCase() === id.toLowerCase(),
),
);
if (!el) return null;
const idAttr = el.attrs.find((a) => a.name === "id");
if (idAttr?.hasValue) {
return new vscode.Location(
document.uri,
new vscode.Range(
document.positionAt(idAttr.valueStart),
document.positionAt(idAttr.valueEnd),
),
);
}
return new vscode.Location(
document.uri,
new vscode.Range(
document.positionAt(el.start),
document.positionAt(el.startTagEnd),
),
);
}
/** /**
* Finds the precise range of an asset definition inside an XML file: the id * Finds the precise range of an asset definition inside an XML file: the id
* attribute value when present, otherwise the element start tag. * attribute value when present, otherwise the element start tag.
@@ -118,7 +230,9 @@ async function locationInDocument(
file: string, file: string,
id: string, id: string,
): Promise<vscode.Location | null> { ): Promise<vscode.Location | null> {
const parsed = await ws.indexer?.readDocument(file); // readDom (not readDocument) guarantees a DOM even when the compact
// records cache already has an entry for the file.
const parsed = await ws.indexer?.readDom(file);
if (parsed?.parse && parsed.lineMap) { if (parsed?.parse && parsed.lineMap) {
const el = parsed.parse.elements.find( const el = parsed.parse.elements.find(
(e) => (e) =>
@@ -156,12 +270,15 @@ function toVscodePosition(p: { line: number; character: number }): vscode.Positi
// ── Find all references ───────────────────────────────────────────── // ── Find all references ─────────────────────────────────────────────
export class Ra3ReferenceProvider implements vscode.ReferenceProvider { export class Ra3ReferenceProvider implements vscode.ReferenceProvider {
constructor(private ws: ModWorkspace) {}
async provideReferences( async provideReferences(
document: vscode.TextDocument, document: vscode.TextDocument,
position: vscode.Position, position: vscode.Position,
_context: vscode.ReferenceContext, _context: vscode.ReferenceContext,
_token: vscode.CancellationToken, _token: vscode.CancellationToken,
): Promise<vscode.Location[] | null> { ): Promise<vscode.Location[] | null> {
if (!this.ws.isRa3Workspace()) return null;
const text = document.getText(); const text = document.getText();
const offset = document.offsetAt(position); const offset = document.offsetAt(position);
const doc = parseXml(text); const doc = parseXml(text);
@@ -172,19 +289,41 @@ export class Ra3ReferenceProvider implements vscode.ReferenceProvider {
(a.hasValue && offset >= a.valueStart && offset <= a.valueEnd) || (a.hasValue && offset >= a.valueStart && offset <= a.valueEnd) ||
(offset >= a.nameStart && offset <= a.nameEnd), (offset >= a.nameStart && offset <= a.nameEnd),
); );
if (!attr?.hasValue) return null; let id: string | null = null;
const id = attr.value; if (attr?.hasValue) {
id = attr.value;
} else {
// Element text content (e.g. <CreateObject>CrateDebris_01</CreateObject>).
const elType = resolveElementType(el);
const token = textContentTokenAt(text, el, offset);
if (token && isReferenceContentType(elType) && !token.value.startsWith("$")) {
id = token.value;
}
}
if (!id || id.startsWith("$")) return null; if (!id || id.startsWith("$")) return null;
const locations: vscode.Location[] = []; const locations: vscode.Location[] = [];
const pattern = `["']${escapeRegExp(id)}["']`; // Matches both attribute values ("id" / 'id') and simple-content
// references (>id<); the outer delimiters are stripped from the result
// range below so the returned locations cover just the id.
const pattern = `(?:["']|>)[ \\t]*${escapeRegExp(id)}[ \\t]*(?:["']|<)`;
await findTextInWorkspace( await findTextInWorkspace(
{ pattern, isRegExp: true }, { pattern, isRegExp: true },
{ include: "**/*.xml", maxResults: 2000 }, { include: "**/*.xml", maxResults: 2000 },
(result: { uri: vscode.Uri; matches: { range: vscode.Range }[] }) => { (result: { uri: vscode.Uri; matches: { range: vscode.Range }[] }) => {
if (!result.uri) return; if (!result.uri) return;
for (const m of result.matches) { for (const m of result.matches) {
locations.push(new vscode.Location(result.uri, m.range)); const start = m.range.start;
const end = m.range.end;
locations.push(
new vscode.Location(
result.uri,
new vscode.Range(
new vscode.Position(start.line, start.character + 1),
new vscode.Position(end.line, end.character - 1),
),
),
);
} }
}, },
); );
@@ -201,8 +340,10 @@ export class Ra3DocumentLinkProvider implements vscode.DocumentLinkProvider {
document: vscode.TextDocument, document: vscode.TextDocument,
_token: vscode.CancellationToken, _token: vscode.CancellationToken,
): Promise<vscode.DocumentLink[]> { ): Promise<vscode.DocumentLink[]> {
if (!this.ws.isRa3Workspace()) return [];
const idx = this.ws.index; const idx = this.ws.index;
if (!idx) return []; const searchPaths = idx ? searchPathsFor(idx) : this.ws.searchPaths();
if (!searchPaths) return [];
const text = document.getText(); const text = document.getText();
const doc = parseXml(text); const doc = parseXml(text);
const links: vscode.DocumentLink[] = []; const links: vscode.DocumentLink[] = [];
@@ -214,9 +355,9 @@ export class Ra3DocumentLinkProvider implements vscode.DocumentLinkProvider {
resolveSource( resolveSource(
srcAttr.value, srcAttr.value,
dirname(document.uri.fsPath), dirname(document.uri.fsPath),
searchPathsFor(idx), searchPaths,
).path ?? ).path ??
idx.sourceCandidates.find((c) => c.source === srcAttr.value)?.path ?? idx?.sourceCandidates.find((c) => c.source === srcAttr.value)?.path ??
null; null;
if (!target) continue; if (!target) continue;
links.push( links.push(
@@ -236,10 +377,13 @@ export class Ra3DocumentLinkProvider implements vscode.DocumentLinkProvider {
// ── Document symbols (outline) ────────────────────────────────────── // ── Document symbols (outline) ──────────────────────────────────────
export class Ra3DocumentSymbolProvider implements vscode.DocumentSymbolProvider { export class Ra3DocumentSymbolProvider implements vscode.DocumentSymbolProvider {
constructor(private ws: ModWorkspace) {}
async provideDocumentSymbols( async provideDocumentSymbols(
document: vscode.TextDocument, document: vscode.TextDocument,
_token: vscode.CancellationToken, _token: vscode.CancellationToken,
): Promise<vscode.DocumentSymbol[]> { ): Promise<vscode.DocumentSymbol[]> {
if (!this.ws.isRa3Workspace()) return [];
const text = document.getText(); const text = document.getText();
const doc = parseXml(text); const doc = parseXml(text);
const root = doc.root; const root = doc.root;
+6
View File
@@ -1,6 +1,7 @@
import * as vscode from "vscode"; import * as vscode from "vscode";
import { parseXml } from "../language/xmlParser"; import { parseXml } from "../language/xmlParser";
import { buildSemanticTokenRanges } from "../language/semanticTokens"; import { buildSemanticTokenRanges } from "../language/semanticTokens";
import type { ModWorkspace } from "../workspace";
const TOKEN_TYPES = ["type", "property", "string"] as const; const TOKEN_TYPES = ["type", "property", "string"] as const;
@@ -20,10 +21,15 @@ export const RA3_SEMANTIC_TOKENS_LEGEND = new vscode.SemanticTokensLegend([
export class Ra3SemanticTokensProvider export class Ra3SemanticTokensProvider
implements vscode.DocumentSemanticTokensProvider implements vscode.DocumentSemanticTokensProvider
{ {
constructor(private ws: ModWorkspace) {}
async provideDocumentSemanticTokens( async provideDocumentSemanticTokens(
document: vscode.TextDocument, document: vscode.TextDocument,
_token: vscode.CancellationToken, _token: vscode.CancellationToken,
): Promise<vscode.SemanticTokens> { ): Promise<vscode.SemanticTokens> {
if (!this.ws.isRa3Workspace()) {
return new vscode.SemanticTokens(new Uint32Array(0));
}
const text = document.getText(); const text = document.getText();
const doc = parseXml(text); const doc = parseXml(text);
if (doc.errors.length === 0) { if (doc.errors.length === 0) {
+263
View File
@@ -0,0 +1,263 @@
/**
* Caches shared by the indexer.
*
* The workspace owns one instance of each cache and passes them into every
* ModIndexer, so a rebuild (which creates a fresh indexer) does not re-read
* files whose stat (mtime/size) is unchanged. This is what makes indexing
* large art-asset corpora (Corona: ~3800 .w3x files, 2.6 GB) practical:
* after the first build, save-triggered rebuilds only re-scan files that
* actually changed.
*/
import { resolve } from "node:path";
import type { IndexedFile, ParsedFile } from "./types";
import type { IndexRecords } from "./records";
import type { ResolveResult } from "./includeResolver";
/** Case-insensitive absolute path key. */
export function normKey(path: string): string {
return resolve(path).toLowerCase();
}
/**
* LRU cache for fully parsed XML documents.
*
* Parse trees of huge mods can be memory-heavy (~17x the source text), so
* retention is bounded twice: by entry count (least-recently-used eviction)
* and by a total element budget (largest trees are evicted first). Evicted
* entries are re-read from disk on demand.
*/
export class DocumentCache {
private map = new Map<string, ParsedFile>();
private totalElements = 0;
constructor(
private capacity = 64,
private elementBudget = 2_000_000,
) {}
get(path: string): ParsedFile | undefined {
const key = normKey(path);
const hit = this.map.get(key);
if (!hit) return undefined;
this.map.delete(key);
this.map.set(key, hit);
return hit;
}
/** Number of cached documents (for diagnostics). */
get size(): number {
return this.map.size;
}
/** Total elements held by cached parse trees (for diagnostics). */
get elements(): number {
return this.totalElements;
}
set(parsed: ParsedFile): void {
const key = normKey(parsed.file.path);
const prev = this.map.get(key);
if (prev) this.totalElements -= elementCount(prev);
this.map.delete(key);
this.map.set(key, parsed);
this.totalElements += elementCount(parsed);
this.evict();
}
invalidate(path: string): void {
const key = normKey(path);
const prev = this.map.get(key);
if (prev) this.totalElements -= elementCount(prev);
this.map.delete(key);
}
clear(): void {
this.map.clear();
this.totalElements = 0;
}
private evict(): void {
while (this.map.size > this.capacity || this.totalElements > this.elementBudget) {
if (this.map.size === 0) break;
if (this.map.size > this.capacity) {
// Over capacity: drop the least recently used entry.
const oldest = this.map.keys().next().value;
if (oldest === undefined) break;
this.remove(oldest);
} else {
// Over the element budget: drop the largest tree, which frees the
// most memory per eviction.
let largestKey: string | undefined;
let largest = -1;
for (const [key, value] of this.map) {
const n = elementCount(value);
if (n > largest) {
largest = n;
largestKey = key;
}
}
if (largestKey === undefined || largest <= 0) break;
this.remove(largestKey);
}
}
}
private remove(key: string): void {
const prev = this.map.get(key);
if (prev) this.totalElements -= elementCount(prev);
this.map.delete(key);
}
}
function elementCount(parsed: ParsedFile): number {
return parsed.parse?.elements.length ?? 0;
}
export interface IndexRecordsCacheEntry {
stat: IndexedFile["stat"];
records: IndexRecords;
/** "shallow" for art-asset scans (.w3x), "full" for parsed XML. */
kind: "shallow" | "full";
}
/**
* Cache for per-file index records (top-level assets, defines, includes,
* xi:include targets). Records are tiny compared to DOM trees or line maps of
* multi-megabyte model files, so the capacity comfortably covers a whole mod
* (Corona: ~9k files) and rebuilds never re-read unchanged files.
*/
export class IndexRecordsCache {
private map = new Map<string, IndexRecordsCacheEntry>();
constructor(private capacity = 16384) {}
get(path: string): IndexRecordsCacheEntry | undefined {
const key = normKey(path);
const hit = this.map.get(key);
if (!hit) return undefined;
this.map.delete(key);
this.map.set(key, hit);
return hit;
}
/** Number of cached record sets (for diagnostics). */
get size(): number {
return this.map.size;
}
/** Iterates [normalized key, entry] pairs (used by disk persistence). */
entries(): IterableIterator<[string, IndexRecordsCacheEntry]> {
return this.map.entries();
}
set(path: string, entry: IndexRecordsCacheEntry): void {
const key = normKey(path);
this.map.delete(key);
this.map.set(key, entry);
if (this.map.size > this.capacity) {
const oldest = this.map.keys().next().value;
if (oldest !== undefined) this.map.delete(oldest);
}
}
invalidate(path: string): void {
this.map.delete(normKey(path));
}
clear(): void {
this.map.clear();
}
}
/**
* Cache for Include/@source (and xi:include/@href) resolution results.
*
* Resolving a source performs synchronous statSync existence checks against
* every search base; a Corona build issues ~110k of them (tens of seconds on
* a mechanical drive). Content edits never change *existence*, so this cache
* survives content rebuilds and is only cleared when files are created or
* deleted (or on a forced reindex).
*/
export class IncludeResolveCache {
private map = new Map<string, ResolveResult>();
private manifestMap = new Map<string, string | null>();
constructor(private capacity = 262144) {}
get(key: string): ResolveResult | undefined {
const hit = this.map.get(key);
if (!hit) return undefined;
this.map.delete(key);
this.map.set(key, hit);
return hit;
}
set(key: string, result: ResolveResult): void {
this.map.delete(key);
this.map.set(key, result);
if (this.map.size > this.capacity) {
const oldest = this.map.keys().next().value;
if (oldest !== undefined) this.map.delete(oldest);
}
}
getManifest(key: string): string | null | undefined {
const hit = this.manifestMap.get(key);
if (hit === undefined) return undefined;
this.manifestMap.delete(key);
this.manifestMap.set(key, hit);
return hit;
}
setManifest(key: string, path: string | null): void {
this.manifestMap.delete(key);
this.manifestMap.set(key, path);
if (this.manifestMap.size > this.capacity) {
const oldest = this.manifestMap.keys().next().value;
if (oldest !== undefined) this.manifestMap.delete(oldest);
}
}
clear(): void {
this.map.clear();
this.manifestMap.clear();
}
/** Number of cached resolutions (for diagnostics). */
get size(): number {
return this.map.size;
}
}
/**
* Monotonic counter for workspace-level invalidations.
*
* A build captures `snapshot()` when it starts; if `changedSince()` is true
* when a phase snapshot is about to be published, files may have changed
* mid-build, so the published index is marked stale (and the workspace's
* dirty/rebuild mechanism converges to fresh data shortly after).
*/
export class InvalidationsEpoch {
private value = 0;
/** Records a new invalidation (content, creation or deletion). */
mark(): void {
this.value++;
}
/** Returns the current epoch value. */
snapshot(): number {
return this.value;
}
/** True when at least one invalidation happened since `epoch`. */
changedSince(epoch: number): boolean {
return this.value !== epoch;
}
/** Current epoch value (read-only accessor). */
get current(): number {
return this.value;
}
}
+208
View File
@@ -0,0 +1,208 @@
/**
* On-disk persistence for per-file index records.
*
* The records cache (top-level assets / defines / includes / xi:include with
* line numbers) is tiny compared to the source corpus (Corona: ~9k files,
* ~10 MB in memory), but rebuilding it from scratch means reading ~2.6 GB of
* art assets again. Persisting it makes a cold start cost a stat validation
* pass (~seconds on SSD, a few to tens of seconds on a mechanical drive)
* instead of a full rebuild.
*
* Correctness model (layered):
* - every cached record stores a multi-signal stamp
* `{ size, mtimeMs, birthtimeMs, ctimeMs }`;
* - on load, each file is stat-validated (no content reads); mismatches and
* missing files are dropped and re-read during the build;
* - during a session the file watcher invalidates entries precisely;
* - `ra3modxml.reindex` / `ra3modxml.clearCache` remain the final authority.
*
* The file is gzip-compressed JSON written atomically (temp + rename), keyed
* by project identity + settings so stale caches are ignored automatically.
*
* Pure TypeScript: no vscode dependency.
*/
import { createHash } from "node:crypto";
import { mkdir, readFile, rename, rm, stat, writeFile } from "node:fs/promises";
import { dirname } from "node:path";
import { gzip, gunzip } from "node:zlib";
import { promisify } from "node:util";
import type { IndexRecordsCacheEntry } from "./caches";
import type { IndexRecords } from "./records";
import type { IndexedFile } from "./types";
const gzipAsync = promisify(gzip);
const gunzipAsync = promisify(gunzip);
export const DISK_CACHE_VERSION = 1;
/** How many stat validations run concurrently on load. */
const VALIDATE_CONCURRENCY = 32;
/** Settings that change what the index contains; a mismatch ignores the cache. */
export interface DiskCacheIdentity {
projectDir: string;
sdkDir: string;
indexSageXml: boolean;
additionalDataSearchPaths: string[];
builtmodsDirs: string[];
}
export interface DiskCacheRecord {
/** Normalized cache key (see `normKey`). */
key: string;
stat: NonNullable<IndexedFile["stat"]>;
records: IndexRecords;
kind: "full" | "shallow";
}
interface DiskCacheFile {
version: number;
key: string;
savedAt: string;
records: DiskCacheRecord[];
}
export interface DiskCacheLoadStats {
fileExists: boolean;
keyMatched: boolean;
/** Records stored in the file. */
loaded: number;
/** Records whose stat still matches (kept). */
validated: number;
/** Records dropped because the file changed, moved or was deleted. */
dropped: number;
}
export function diskCacheKey(identity: DiskCacheIdentity): string {
return createHash("sha256")
.update(JSON.stringify(identity))
.digest("hex")
.slice(0, 16);
}
export class DiskRecordsCache {
constructor(
private readonly filePath: string,
private readonly identity: DiskCacheIdentity,
) {}
get path(): string {
return this.filePath;
}
/**
* Loads and stat-validates the cache. Returns the kept records plus load
* statistics; missing/corrupt/key-mismatched caches yield an empty result
* instead of an error.
*/
async loadValidated(): Promise<{
records: DiskCacheRecord[];
stats: DiskCacheLoadStats;
}> {
const stats: DiskCacheLoadStats = {
fileExists: false,
keyMatched: false,
loaded: 0,
validated: 0,
dropped: 0,
};
let raw: DiskCacheFile | null = null;
try {
const buf = await readFile(this.filePath);
stats.fileExists = true;
const text = (await gunzipAsync(buf)).toString("utf8");
const parsed = JSON.parse(text);
if (
parsed &&
parsed.version === DISK_CACHE_VERSION &&
parsed.key === diskCacheKey(this.identity) &&
Array.isArray(parsed.records)
) {
raw = parsed as DiskCacheFile;
}
} catch {
// Missing or corrupt cache: fall through with an empty result.
}
if (!raw) return { records: [], stats };
stats.keyMatched = true;
stats.loaded = raw.records.length;
const kept: DiskCacheRecord[] = [];
for (let i = 0; i < raw.records.length; i += VALIDATE_CONCURRENCY) {
const chunk = raw.records.slice(i, i + VALIDATE_CONCURRENCY);
const results = await Promise.all(
chunk.map(async (rec): Promise<DiskCacheRecord | null> => {
try {
const s = await stat(rec.key);
if (
s.isFile() &&
s.size === rec.stat.size &&
s.mtimeMs === rec.stat.mtimeMs &&
s.birthtimeMs === rec.stat.birthtimeMs &&
s.ctimeMs === rec.stat.ctimeMs
) {
return rec;
}
} catch {
// File missing or inaccessible.
}
return null;
}),
);
for (const r of results) {
if (r) {
kept.push(r);
stats.validated++;
} else {
stats.dropped++;
}
}
}
return { records: kept, stats };
}
/** Writes the current records cache atomically (temp file + rename). */
async save(
entries: Iterable<[string, IndexRecordsCacheEntry]>,
): Promise<void> {
const records: DiskCacheRecord[] = [];
for (const [key, entry] of entries) {
if (!entry.stat) continue;
records.push({
key,
stat: entry.stat,
records: entry.records,
kind: entry.kind,
});
}
const payload: DiskCacheFile = {
version: DISK_CACHE_VERSION,
key: diskCacheKey(this.identity),
savedAt: new Date().toISOString(),
records,
};
const buf = await gzipAsync(Buffer.from(JSON.stringify(payload), "utf8"));
await mkdir(dirname(this.filePath), { recursive: true });
const tmp = `${this.filePath}.tmp`;
await writeFile(tmp, buf);
await rename(tmp, this.filePath);
}
/** Deletes the cache file (used by the clear-cache command). */
async clear(): Promise<void> {
try {
await rm(this.filePath, { force: true });
} catch {
// Best effort.
}
}
async status(): Promise<{ exists: boolean; sizeBytes: number }> {
try {
const s = await stat(this.filePath);
return { exists: s.isFile(), sizeBytes: s.size };
} catch {
return { exists: false, sizeBytes: 0 };
}
}
}
+117
View File
@@ -0,0 +1,117 @@
/**
* Lazy file-existence snapshot for include resolution.
*
* `resolveSource` performs synchronous `statSync` existence checks against
* every search base; a cold Corona build does ~110k of them (tens of seconds
* on a mechanical drive). Instead, existence is answered by reading the
* candidate's **parent directory** once (`readdir`, no per-file stat) and
* caching the entry set for the rest of the build. Only directories that are
* actually queried are ever listed, so a cold build pays a handful of
* readdir calls instead of an upfront recursive enumeration of every search
* root (which measurably slowed the XML phase).
*
* Correctness: the workspace clears `IncludeResolveCache` on file
* create/delete; each rebuild creates a fresh snapshot, and the debounced
* rebuild triggered by the watcher converges if anything changed mid-build.
*
* Pure TypeScript: no vscode dependency.
*/
import { existsSync, readdirSync } from "node:fs";
import { basename, dirname, parse, resolve, sep } from "node:path";
import { normKey } from "./caches";
import type { SearchPaths } from "./includeResolver";
/**
* Answers file-existence questions from lazily read directory listings.
* `has()` is authoritative for paths inside the roots and returns null for
* paths the snapshot does not cover (the caller falls back to `statSync`).
*/
export class ExistenceSnapshot {
private roots: string[] = [];
/** parent dir (normalized) -> lowercased file names, or null (no dir). */
private dirCache = new Map<string, Set<string> | null>();
/** Existence answers served from cached directory listings. */
hits = 0;
/** Paths outside the snapshot that required a statSync fallback. */
fallbacks = 0;
constructor(roots: string[]) {
// Roots and lookups must use the same normalization (case-insensitive on
// Windows), otherwise `startsWith` misses due to case differences.
this.roots = roots.map((r) => {
const n = normKey(r);
return n.endsWith(sep) ? n : n + sep;
});
}
/** true/false when covered by the snapshot, null when not covered. */
has(absPath: string): boolean | null {
const parentKey = normKey(dirname(absPath));
if (!this.isCovered(parentKey)) {
this.fallbacks++;
return null;
}
let entries = this.dirCache.get(parentKey);
if (entries === undefined) {
entries = listDirEntries(parentKey);
this.dirCache.set(parentKey, entries);
}
this.hits++;
return entries ? entries.has(basename(absPath).toLowerCase()) : false;
}
private isCovered(dirKey: string): boolean {
const key = dirKey.endsWith(sep) ? dirKey : dirKey + sep;
for (const root of this.roots) {
if (key.startsWith(root)) return true;
}
return false;
}
}
function listDirEntries(dir: string): Set<string> | null {
try {
const out = new Set<string>();
for (const entry of readdirSync(dir, { withFileTypes: true })) {
if (entry.isFile()) out.add(entry.name.toLowerCase());
}
return out;
} catch {
return null;
}
}
/**
* True when `dir` is a filesystem root (e.g. "C:\" or "/"). Such roots are
* never treated as search bases (they may contain the whole disk).
*/
export function isDriveRoot(dir: string): boolean {
const resolved = resolve(dir);
return parse(resolved).root === resolved;
}
/**
* Builds the snapshot root list from the search bases: drive roots and
* missing directories are skipped, and a root covered by a broader root is
* dropped (e.g. `sdkDir` covers `sdkDir/SageXml`). No directory is listed
* here; listings happen lazily per queried parent directory.
*/
export function buildExistenceSnapshot(searchPaths: SearchPaths): ExistenceSnapshot {
const candidates = [
...searchPaths.DATA,
...searchPaths.ART,
...searchPaths.AUDIO,
].map((r) => resolve(r));
candidates.sort((a, b) => a.length - b.length);
const roots: string[] = [];
for (const root of candidates) {
if (isDriveRoot(root)) continue;
if (!existsSync(root)) continue;
const normalized = normKey(root);
if (roots.some((r) => normalized.startsWith(r + sep))) continue;
roots.push(normalized);
}
return new ExistenceSnapshot(roots);
}
+59 -13
View File
@@ -1,7 +1,18 @@
import { readdir, stat } from "node:fs/promises"; import { readdir, stat } from "node:fs/promises";
import { join, relative, resolve } from "node:path"; import { extname, join, relative, resolve } from "node:path";
import type { FileWalker, SourceCandidate } from "./types"; import type { FileWalker, SourceCandidate } from "./types";
/**
* Extensions offered as Include/@source candidates in DATA directories:
* regular XML plus art-asset XML (.w3x) that mods include via W3X.xml hubs.
* ART/AUDIO directories already list every file.
*/
const DATA_CANDIDATE_EXTENSIONS = new Set([".xml", ".w3x"]);
function isDataCandidate(path: string): boolean {
return DATA_CANDIDATE_EXTENSIONS.has(extname(path).toLowerCase());
}
/** /**
* Recursive file list walker with a simple directory-mtime cache. Used to * Recursive file list walker with a simple directory-mtime cache. Used to
* enumerate candidate files for Include/@source completion. Directory scans * enumerate candidate files for Include/@source completion. Directory scans
@@ -50,6 +61,38 @@ export class CachedDirectoryWalker implements FileWalker {
} }
} }
/**
* True for paths whose changes can never affect the index: `.git` internals
* touched by background fetch/maintenance, and transient temp/backup files
* created by editors or other extensions (`UnitCrate.xml.git`, `*.tmp`,
* `*.lock`, `file~`, `.#file`, ...). Such events are ignored by the file
* watcher instead of triggering rebuilds.
*/
export function isWatcherNoisePath(fsPath: string): boolean {
const segments = fsPath.split(/[\\/]/);
if (segments.some((seg) => seg.toLowerCase() === ".git")) return true;
const base = segments[segments.length - 1] ?? "";
const lower = base.toLowerCase();
const noiseSuffixes = [".git", ".tmp", ".lock", "~", ".swp", ".bak", ".orig"];
if (noiseSuffixes.some((suffix) => lower.endsWith(suffix))) return true;
return lower.startsWith(".#") || lower.startsWith(".~");
}
/**
* True for files whose *content* participates in the index: XML documents
* and art-asset XML (.w3x). Reasonable text formats in RA3 mods are `.xml`,
* `.w3x` and `.lua`; lua is not indexed yet, and compiled manifests are
* binary `*.manifest` (there is no `.manifestxml` source format). Files
* already in the index with other extensions (e.g. sniffed XML) are handled
* separately via `ModIndexer.isIndexedFile`. Content changes to binary art
* (e.g. textures, `.w3d`) cannot change index records, so they do not need
* to trigger a rebuild.
*/
export function isContentRelevantPath(fsPath: string): boolean {
const ext = extname(fsPath).toLowerCase();
return ext === ".xml" || ext === ".w3x";
}
/** /**
* Builds the candidate list for Include/@source completion from a set of * Builds the candidate list for Include/@source completion from a set of
* search directories. For DATA directories only *.xml files are listed; for * search directories. For DATA directories only *.xml files are listed; for
@@ -76,29 +119,32 @@ export async function collectSourceCandidates(
out.push({ source, path: resolve(path), prefix, baseDir: resolve(baseDir) }); out.push({ source, path: resolve(path), prefix, baseDir: resolve(baseDir) });
}; };
for (const dir of dataDirs) { // List directories in parallel; the walker caches each directory by its
const files = await walker.listFiles(dir); // mtime, so rebuilds after the first are still cheap.
const [dataLists, artLists, audioLists] = await Promise.all([
Promise.all(dataDirs.map(async (dir) => ({ dir, files: await walker.listFiles(dir) }))),
Promise.all(artDirs.map(async (dir) => ({ dir, files: await walker.listFiles(dir) }))),
Promise.all(audioDirs.map(async (dir) => ({ dir, files: await walker.listFiles(dir) }))),
]);
for (const { dir, files } of dataLists) {
for (const f of files) { for (const f of files) {
if (!f.toLowerCase().endsWith(".xml")) continue; if (!isDataCandidate(f)) continue;
add(f, dir, "DATA"); add(f, dir, "DATA");
} }
if (samePath(dir, projectDataDir)) { if (samePath(dir, projectDataDir)) {
// Also offer project-relative paths (what Mod.xml itself uses). // Also offer project-relative paths (what Mod.xml itself uses).
for (const f of files) { for (const f of files) {
if (f.toLowerCase().endsWith(".xml")) add(f, projectDataDir, null); if (isDataCandidate(f)) add(f, projectDataDir, null);
} }
} }
} }
for (const dir of artDirs) { for (const { dir, files } of artLists) {
for (const f of await walker.listFiles(dir)) { for (const f of files) add(f, dir, "ART");
add(f, dir, "ART");
}
}
for (const dir of audioDirs) {
for (const f of await walker.listFiles(dir)) {
add(f, dir, "AUDIO");
} }
for (const { dir, files } of audioLists) {
for (const f of files) add(f, dir, "AUDIO");
} }
return out; return out;
+25 -7
View File
@@ -8,6 +8,7 @@
import { join, resolve, normalize, isAbsolute } from "node:path"; import { join, resolve, normalize, isAbsolute } from "node:path";
import { statSync } from "node:fs"; import { statSync } from "node:fs";
import type { ExistenceSnapshot } from "./existence";
export type IncludeKind = "all" | "instance" | "reference"; export type IncludeKind = "all" | "instance" | "reference";
export type SourcePrefix = "DATA" | "ART" | "AUDIO" | null; export type SourcePrefix = "DATA" | "ART" | "AUDIO" | null;
@@ -99,41 +100,58 @@ export function resolveSource(
source: string, source: string,
currentDir: string | null, currentDir: string | null,
searchPaths: SearchPaths, searchPaths: SearchPaths,
existence?: ExistenceSnapshot,
): ResolveResult { ): ResolveResult {
const raw = source.trim().replace(/\\/g, "/"); const raw = source.trim().replace(/\\/g, "/");
const { prefix, rest } = splitPrefix(raw); const { prefix, rest } = splitPrefix(raw);
if (prefix) { if (prefix) {
const bases = searchPaths[prefix] ?? []; const bases = searchPaths[prefix] ?? [];
const direct = findInBases(rest, bases); const direct = findInBases(rest, bases, existence);
if (direct) return { path: direct, prefix, raw }; if (direct) return { path: direct, prefix, raw };
if (prefix === "ART" && !rest.includes("/")) { if (prefix === "ART" && !rest.includes("/")) {
const two = rest.slice(0, 2).toLowerCase(); const two = rest.slice(0, 2).toLowerCase();
const prefixed = findInBases(`${two}/${rest}`, bases); const prefixed = findInBases(`${two}/${rest}`, bases, existence);
if (prefixed) return { path: prefixed, prefix, raw }; if (prefixed) return { path: prefixed, prefix, raw };
} }
return { path: null, prefix, raw }; return { path: null, prefix, raw };
} }
if (currentDir && isAbsolute(rest)) { if (currentDir && isAbsolute(rest)) {
return { path: fileExists(rest) ? rest : null, prefix: null, raw }; return {
path: fileExists(rest, existence) ? rest : null,
prefix: null,
raw,
};
} }
if (currentDir) { if (currentDir) {
const candidate = resolve(currentDir, rest); const candidate = resolve(currentDir, rest);
return { path: fileExists(candidate) ? candidate : null, prefix: null, raw }; return {
path: fileExists(candidate, existence) ? candidate : null,
prefix: null,
raw,
};
} }
return { path: null, prefix: null, raw }; return { path: null, prefix: null, raw };
} }
function findInBases(relPath: string, bases: string[]): string | null { function findInBases(
relPath: string,
bases: string[],
existence?: ExistenceSnapshot,
): string | null {
for (const base of bases) { for (const base of bases) {
const candidate = normalize(resolve(base, relPath)); const candidate = normalize(resolve(base, relPath));
if (fileExists(candidate)) return candidate; if (fileExists(candidate, existence)) return candidate;
} }
return null; return null;
} }
function fileExists(path: string): boolean { function fileExists(path: string, existence?: ExistenceSnapshot): boolean {
if (existence) {
const known = existence.has(path);
if (known !== null) return known;
}
try { try {
return statSync(path).isFile(); return statSync(path).isFile();
} catch { } catch {
+529 -174
View File
@@ -11,15 +11,27 @@
* outside the extension. * outside the extension.
*/ */
import { readFile, readdir, stat } from "node:fs/promises"; import { open, readFile, readdir, stat } from "node:fs/promises";
import type { Stats } from "node:fs";
import { basename, dirname, extname, join, resolve } from "node:path"; import { basename, dirname, extname, join, resolve } from "node:path";
import { LineMap, parseXml, type XmlDocument, type XmlElement } from "../language/xmlParser"; import {
LineMap,
parseXml,
stripBom,
type XmlElement,
} from "../language/xmlParser";
import { import {
buildSearchPaths, buildSearchPaths,
manifestPathForReference, manifestPathForReference,
resolveSource, resolveSource,
type ResolveResult,
type SearchPaths, type SearchPaths,
} from "./includeResolver"; } from "./includeResolver";
import {
buildExistenceSnapshot,
type ExistenceSnapshot,
} from "./existence";
import { findXPointerContainer, localName } from "./xpointer";
import { import {
deriveAssetId, deriveAssetId,
deriveAssetType, deriveAssetType,
@@ -28,6 +40,14 @@ import {
} from "./manifestParser"; } from "./manifestParser";
import { canonicalTypeName } from "../model/schemaModel"; import { canonicalTypeName } from "../model/schemaModel";
import { collectSourceCandidates } from "./fileScanner"; import { collectSourceCandidates } from "./fileScanner";
import { DocumentCache, IncludeResolveCache, IndexRecordsCache, normKey } from "./caches";
import type { IndexRecordsCacheEntry } from "./caches";
import { scanXmlShallow } from "./shallowScan";
import {
extractIndexRecords,
recordsFromShallow,
type IndexRecordXi,
} from "./records";
import type { import type {
AssetDef, AssetDef,
DefineDef, DefineDef,
@@ -42,54 +62,50 @@ import type {
const MAX_DEPTH = 300; const MAX_DEPTH = 300;
/** Files above this size are never parsed (safety against binary blobs). */ /** Files above this size are never parsed (safety against binary blobs). */
const MAX_PARSE_BYTES = 4 * 1024 * 1024; const MAX_PARSE_BYTES = 4 * 1024 * 1024;
/** Only these extensions are treated as XML documents. */
const XML_EXTENSIONS = new Set([".xml", ".manifestxml"]);
function normKey(path: string): string {
return resolve(path).toLowerCase();
}
/** /**
* LRU cache for parsed documents. The parse trees of huge mods can be * Fully parsed XML documents. `.xml` files are small enough that a full DOM
* memory-heavy, so only a bounded number of recent documents is retained; * is affordable. (Compiled manifests are binary `*.manifest` files parsed by
* evicted entries are re-read from disk on demand. * `manifestParser`; there is no `.manifestxml` source format.)
*/ */
export class DocumentCache { const FULL_XML_EXTENSIONS = new Set([".xml"]);
private map = new Map<string, ParsedFile>(); /**
* XML documents whose top-level structure is all the index needs (art-asset
* files exported by modeling tools, e.g. .w3x). They are shallow-scanned so
* multi-megabyte vertex/triangle payloads never become a DOM.
*/
const SHALLOW_XML_EXTENSIONS = new Set([".w3x"]);
/** Bytes peeked when deciding whether an unknown extension is XML text. */
const SNIFF_BYTES = 512;
constructor(private capacity = 64) {} type XmlMode = "full" | "shallow" | "binary";
get(path: string): ParsedFile | undefined {
const key = normKey(path);
const hit = this.map.get(key);
if (!hit) return undefined;
this.map.delete(key);
this.map.set(key, hit);
return hit;
}
set(parsed: ParsedFile): void {
const key = normKey(parsed.file.path);
this.map.delete(key);
this.map.set(key, parsed);
if (this.map.size > this.capacity) {
const oldest = this.map.keys().next().value;
if (oldest !== undefined) this.map.delete(oldest);
}
}
invalidate(path: string): void {
this.map.delete(normKey(path));
}
clear(): void {
this.map.clear();
}
}
export class ModIndexer { export class ModIndexer {
private searchPaths: SearchPaths; private searchPaths: SearchPaths;
private docs = new DocumentCache(); private docs: DocumentCache;
private recordsCache: IndexRecordsCache;
private resolveCache: IncludeResolveCache;
/** Directory-based file existence snapshot (avoids cold statSync storms). */
private existence: ExistenceSnapshot | null = null;
private scanCounters = {
shallowScannedFiles: 0,
shallowCacheHits: 0,
recordsCacheHits: 0,
resolveCacheHits: 0,
resolveCalls: 0,
};
private timings = { candidatesMs: 0, walkMs: 0, artScanMs: 0 };
/**
* Phase A ("xml") registers art-asset XML files (`.w3x` and sniffed XML)
* without reading their content; the queue is drained by phase B ("art"),
* which shallow-scans them and walks any includes they contain.
*/
private deferArtScan = false;
private artQueue: {
path: string;
stream: StreamInfo;
depth: number;
viaInstance: boolean;
}[] = [];
private assets = new Map<string, Map<string, AssetDef[]>>(); private assets = new Map<string, Map<string, AssetDef[]>>();
private assetsById = new Map<string, AssetDef[]>(); private assetsById = new Map<string, AssetDef[]>();
private defines = new Map<string, DefineDef[]>(); private defines = new Map<string, DefineDef[]>();
@@ -106,55 +122,322 @@ export class ModIndexer {
this.searchPaths = buildSearchPaths(opts.sdkDir, opts.projectDir, { this.searchPaths = buildSearchPaths(opts.sdkDir, opts.projectDir, {
DATA: opts.additionalDataSearchPaths, DATA: opts.additionalDataSearchPaths,
}); });
// Caches may be owned by the workspace so they survive rebuilds.
this.docs = opts.documentCache ?? new DocumentCache();
this.recordsCache = opts.recordsCache ?? new IndexRecordsCache();
this.resolveCache = opts.resolveCache ?? new IncludeResolveCache();
} }
/** Re-reads and caches a document; null when unreadable. */ /**
async readDocument(path: string): Promise<ParsedFile | null> { * Returns a document for indexing/navigation:
const hit = this.docs.get(path); * - `.xml` files are fully parsed (bounded by MAX_PARSE_BYTES);
if (hit) return hit; * - `.w3x` (and unknown-extension files whose content looks like XML) are
* shallow-scanned, so huge model files never become a DOM;
* - everything else is registered as a file but never parsed.
*
* Cached entries are reused when the file stat is unchanged, which lets a
* workspace-owned cache survive rebuilds.
*/
async readDocument(
path: string,
opts?: { deferArt?: boolean },
): Promise<ParsedFile | null> {
const key = normKey(path);
const trust =
this.opts.trustUnchanged === true && !this.opts.changedFiles?.has(key);
// Trusted fast path: a file that the watcher has not reported as changed
// is reused without any stat / content sniff / read at all. This is what
// makes save-triggered rebuilds cheap on huge corpora (Corona: ~9k files,
// ~2.6 GB of art assets on a mechanical drive).
if (trust) {
const rec = this.recordsCache.get(key);
if (rec) return this.recordsParsed(path, rec);
const cached = this.docs.get(key);
if (cached) {
this.files.set(key, cached.file);
return cached;
}
}
// Untrusted / cache miss: verify the stat against caches, then read.
try { try {
const [st, text] = await Promise.all([stat(path), readFile(path, "utf8")]); const st = await stat(path);
if (st.size > MAX_PARSE_BYTES) { const rec = this.recordsCache.get(key);
const file: IndexedFile = { path: resolve(path), stat: { mtimeMs: st.mtimeMs, size: st.size } }; if (
const parsed: ParsedFile = { file, parse: null, lineMap: null }; rec?.stat &&
rec.stat.mtimeMs === st.mtimeMs &&
rec.stat.size === st.size &&
rec.stat.birthtimeMs === st.birthtimeMs &&
rec.stat.ctimeMs === st.ctimeMs
) {
return this.recordsParsed(path, rec);
}
const hit = this.docs.get(key);
if (
hit?.file.stat &&
hit.file.stat.mtimeMs === st.mtimeMs &&
hit.file.stat.size === st.size &&
hit.file.stat.birthtimeMs === st.birthtimeMs &&
hit.file.stat.ctimeMs === st.ctimeMs
) {
this.files.set(key, hit.file);
return hit;
}
const mode = await this.detectXmlMode(path);
if (mode === "shallow") {
// Phase A: register the art file, defer the shallow scan to phase B.
if (opts?.deferArt) {
const file: IndexedFile = {
path: resolve(path),
stat: {
mtimeMs: st.mtimeMs,
size: st.size,
birthtimeMs: st.birthtimeMs,
ctimeMs: st.ctimeMs,
},
};
// Deliberately NOT stored in the DocumentCache: a deferred entry
// has no records, and a later phase-B read must re-scan it.
this.files.set(key, file);
return { file, parse: null, records: null, lineMap: null, deferredArt: true };
}
return this.scanShallow(path, st);
}
if (mode === "binary") {
const file: IndexedFile = {
path: resolve(path),
stat: {
mtimeMs: st.mtimeMs,
size: st.size,
birthtimeMs: st.birthtimeMs,
ctimeMs: st.ctimeMs,
},
};
const parsed: ParsedFile = { file, parse: null, records: null, lineMap: null };
this.docs.set(parsed); this.docs.set(parsed);
this.files.set(normKey(parsed.file.path), file); this.files.set(key, file);
return parsed; return parsed;
} }
if (st.size > MAX_PARSE_BYTES) {
const file: IndexedFile = {
path: resolve(path),
stat: {
mtimeMs: st.mtimeMs,
size: st.size,
birthtimeMs: st.birthtimeMs,
ctimeMs: st.ctimeMs,
},
};
const parsed: ParsedFile = { file, parse: null, records: null, lineMap: null };
this.docs.set(parsed);
this.files.set(key, file);
return parsed;
}
const text = stripBom(await readFile(path, "utf8"));
const lineMap = new LineMap(text);
const parse = parseXml(text); const parse = parseXml(text);
const records = extractIndexRecords(parse, lineMap);
const parsed: ParsedFile = { const parsed: ParsedFile = {
file: { path: resolve(path), stat: { mtimeMs: st.mtimeMs, size: st.size } }, file: {
path: resolve(path),
stat: {
mtimeMs: st.mtimeMs,
size: st.size,
birthtimeMs: st.birthtimeMs,
ctimeMs: st.ctimeMs,
},
},
parse, parse,
lineMap: new LineMap(text), records,
lineMap,
}; };
this.docs.set(parsed); this.docs.set(parsed);
this.files.set(normKey(parsed.file.path), parsed.file); this.recordsCache.set(key, { stat: parsed.file.stat, records, kind: "full" });
this.files.set(key, parsed.file);
return parsed; return parsed;
} catch { } catch {
const parsed: ParsedFile = { const parsed: ParsedFile = {
file: { path: resolve(path), stat: null }, file: { path: resolve(path), stat: null },
parse: null, parse: null,
records: null,
lineMap: null, lineMap: null,
}; };
this.docs.set(parsed); this.docs.set(parsed);
this.files.set(normKey(parsed.file.path), parsed.file); this.files.set(key, parsed.file);
return parsed; return parsed;
} }
} }
/**
* Shallow-scans a large art-asset XML document (no DOM built) and caches
* its compact index records. The transient LineMap used to compute record
* lines is discarded, so the cache never retains megabytes of line offsets
* for model files (Corona w3x alone would otherwise keep ~700 MB).
*/
private async scanShallow(path: string, st: Stats): Promise<ParsedFile | null> {
const key = normKey(path);
try {
const text = stripBom(await readFile(path, "utf8"));
const lineMap = new LineMap(text);
const records = recordsFromShallow(scanXmlShallow(text), lineMap);
const parsed: ParsedFile = {
file: {
path: resolve(path),
stat: {
mtimeMs: st.mtimeMs,
size: st.size,
birthtimeMs: st.birthtimeMs,
ctimeMs: st.ctimeMs,
},
},
parse: null,
records,
lineMap: null,
};
this.recordsCache.set(key, { stat: parsed.file.stat, records, kind: "shallow" });
this.files.set(key, parsed.file);
this.scanCounters.shallowScannedFiles++;
return parsed;
} catch {
return null;
}
}
/** Builds a lean ParsedFile from cached index records (no DOM / line map). */
private recordsParsed(path: string, entry: IndexRecordsCacheEntry): ParsedFile {
if (entry.kind === "shallow") this.scanCounters.shallowCacheHits++;
else this.scanCounters.recordsCacheHits++;
const file: IndexedFile = { path: resolve(path), stat: entry.stat };
this.files.set(normKey(path), file);
return { file, parse: null, records: entry.records, lineMap: null };
}
/**
* Reads a document and guarantees a DOM parse tree. Used for root-level
* <xi:include> xpointer selection (rare) and by the document-local scope
* (logical include expansion + precise definition locations).
*/
async readDom(path: string): Promise<ParsedFile | null> {
const key = normKey(path);
const cached = this.docs.get(key);
if (cached?.parse?.root) {
this.files.set(key, cached.file);
return cached;
}
try {
const st = await stat(path);
const hit = this.docs.get(key);
if (
hit?.parse?.root &&
hit.file.stat &&
hit.file.stat.mtimeMs === st.mtimeMs &&
hit.file.stat.size === st.size &&
hit.file.stat.birthtimeMs === st.birthtimeMs &&
hit.file.stat.ctimeMs === st.ctimeMs
) {
this.files.set(key, hit.file);
return hit;
}
if (st.size > MAX_PARSE_BYTES) return null;
const text = stripBom(await readFile(path, "utf8"));
const lineMap = new LineMap(text);
const parse = parseXml(text);
const records = extractIndexRecords(parse, lineMap);
const parsed: ParsedFile = {
file: {
path: resolve(path),
stat: {
mtimeMs: st.mtimeMs,
size: st.size,
birthtimeMs: st.birthtimeMs,
ctimeMs: st.ctimeMs,
},
},
parse,
records,
lineMap,
};
this.docs.set(parsed);
this.recordsCache.set(key, { stat: parsed.file.stat, records, kind: "full" });
this.files.set(key, parsed.file);
return parsed;
} catch {
return null;
}
}
/** Decides how a resolved include target should be consumed. */
private async detectXmlMode(path: string): Promise<XmlMode> {
const ext = extname(path).toLowerCase();
if (FULL_XML_EXTENSIONS.has(ext)) return "full";
if (SHALLOW_XML_EXTENSIONS.has(ext)) return "shallow";
return (await looksLikeXml(path)) ? "shallow" : "binary";
}
/**
* Resolves an include source through the cross-rebuild cache. Existence
* does not change on content edits, so trusted rebuilds pay zero statSync
* for include resolution (Corona does ~110k checks per build otherwise).
*/
private resolveCached(source: string, currentDir: string | null): ResolveResult {
const key = `${normKey(currentDir ?? "")}\u0000${source}`;
const hit = this.resolveCache.get(key);
if (hit) {
this.scanCounters.resolveCacheHits++;
return hit;
}
this.scanCounters.resolveCalls++;
const result = resolveSource(
source,
currentDir,
this.searchPaths,
this.existence ?? undefined,
);
this.resolveCache.set(key, result);
return result;
}
/** Cached manifest lookup for `reference` includes. */
private manifestPathCached(source: string): string | null {
const key = source.toLowerCase();
const hit = this.resolveCache.getManifest(key);
if (hit !== undefined) {
this.scanCounters.resolveCacheHits++;
return hit;
}
this.scanCounters.resolveCalls++;
const path = manifestPathForReference(source, this.opts.builtmodsDirs);
this.resolveCache.setManifest(key, path);
return path;
}
/** Returns the cached parse if present (does not read from disk). */ /** Returns the cached parse if present (does not read from disk). */
cachedDocument(path: string): ParsedFile | undefined { cachedDocument(path: string): ParsedFile | undefined {
return this.docs.get(path); return this.docs.get(path);
} }
async build(): Promise<ModIndex> { /** True when the file is part of the current build's index. */
isIndexedFile(path: string): boolean {
return this.files.has(normKey(path));
}
async build(onPhase?: (index: ModIndex) => void | Promise<void>): Promise<ModIndex> {
const start = Date.now(); const start = Date.now();
// Root list only; directories are listed lazily on first query, so the
// XML phase does not pay an upfront recursive enumeration of the SDK.
this.existence = buildExistenceSnapshot(this.searchPaths);
const projectData = await findCaseInsensitiveDir(join(this.opts.projectDir, "Data")); const projectData = await findCaseInsensitiveDir(join(this.opts.projectDir, "Data"));
const additionalMaps = projectData const additionalMaps = projectData
? await findCaseInsensitiveDir(join(projectData, "additionalmaps")) ? await findCaseInsensitiveDir(join(projectData, "additionalmaps"))
: null; : null;
// ── Streams ── // ── Streams ──
// Phase A: walk the include graph without reading art-asset content.
// `.w3x` (and sniffed XML) files are registered and queued; their
// top-level assets and nested includes are processed in phase B.
this.deferArtScan = true;
const walkStart = Date.now();
const staticEntry = projectData ? join(projectData, "Mod.xml") : null; const staticEntry = projectData ? join(projectData, "Mod.xml") : null;
if (staticEntry) { if (staticEntry) {
const stream: StreamInfo = { name: "static", entry: staticEntry, files: new Set() }; const stream: StreamInfo = { name: "static", entry: staticEntry, files: new Set() };
@@ -183,8 +466,10 @@ export class ModIndexer {
await this.walk(entry, "all", stream, 0); await this.walk(entry, "all", stream, 0);
} }
} }
this.timings.walkMs = Date.now() - walkStart;
// ── Source completion candidates ── // ── Source completion candidates ──
const candidatesStart = Date.now();
const dataDirs = [ const dataDirs = [
projectData ?? join(this.opts.projectDir, "Data"), projectData ?? join(this.opts.projectDir, "Data"),
join(this.opts.sdkDir, "SageXml"), join(this.opts.sdkDir, "SageXml"),
@@ -229,37 +514,94 @@ export class ModIndexer {
...sdkRootCandidates, ...sdkRootCandidates,
...this.sourceCandidates, ...this.sourceCandidates,
]); ]);
this.timings.candidatesMs = Date.now() - candidatesStart;
// Publish the XML phase as an immutable snapshot: features get usable
// completions/navigation/diagnostics for XML + manifest data immediately,
// while the slow art scan continues in the background.
const xmlPhase = this.snapshotIndex("xml", false, start);
if (onPhase) await onPhase(xmlPhase);
// ── Phase B: art assets ──
this.deferArtScan = false;
const artStart = Date.now();
while (this.artQueue.length) {
const entry = this.artQueue.shift()!;
const parsed = await this.readDocument(entry.path);
if (parsed?.records) {
await this.applyRecords(parsed, entry.stream, entry.depth, entry.viaInstance);
}
}
this.timings.artScanMs = Date.now() - artStart;
return this.snapshotIndex("art", true, start);
}
/**
* Produces a copy of the current index state. Phase-A snapshots must be
* immutable: phase B keeps mutating the live maps after the snapshot has
* been handed to features, so every nested map/array is cloned here.
*/
private snapshotIndex(
phase: "xml" | "art",
complete: boolean,
startedAt: number,
): ModIndex {
const manifestAssetCount = [...this.manifests.values()].reduce( const manifestAssetCount = [...this.manifests.values()].reduce(
(sum, m) => sum + m.assets.length, (sum, m) => sum + m.assets.length,
0, 0,
); );
const assets = new Map<string, Map<string, AssetDef[]>>();
for (const [type, byId] of this.assets) {
const copied = new Map<string, AssetDef[]>();
for (const [id, defs] of byId) copied.set(id, defs.slice());
assets.set(type, copied);
}
const assetsById = new Map<string, AssetDef[]>();
for (const [id, defs] of this.assetsById) assetsById.set(id, defs.slice());
const defines = new Map<string, DefineDef[]>();
for (const [name, defs] of this.defines) defines.set(name, defs.slice());
return { return {
projectDir: resolve(this.opts.projectDir), projectDir: resolve(this.opts.projectDir),
sdkDir: resolve(this.opts.sdkDir), sdkDir: resolve(this.opts.sdkDir),
assets: this.assets, complete,
assetsById: this.assetsById, phase,
defines: this.defines, assets,
files: this.files, assetsById,
streams: this.streams, defines,
manifests: this.manifests, files: new Map(this.files),
sourceCandidates: this.sourceCandidates, streams: this.streams.map((s) => ({ ...s, files: new Set(s.files) })),
diagnostics: this.diagnostics, manifests: new Map(this.manifests),
sourceCandidates: this.sourceCandidates.slice(),
diagnostics: this.diagnostics.slice(),
stats: { stats: {
projectDir: resolve(this.opts.projectDir), projectDir: resolve(this.opts.projectDir),
sdkDir: resolve(this.opts.sdkDir), sdkDir: resolve(this.opts.sdkDir),
phase,
complete,
indexedFiles: this.files.size, indexedFiles: this.files.size,
parsedFiles: [...this.files.values()].filter( parsedFiles: [...this.files.values()].filter(
(f) => f.stat != null && f.stat.size <= MAX_PARSE_BYTES, (f) => f.stat != null && f.stat.size <= MAX_PARSE_BYTES,
).length, ).length,
shallowScannedFiles: this.scanCounters.shallowScannedFiles,
deferredArtFiles: this.artQueue.length,
shallowCacheHits: this.scanCounters.shallowCacheHits,
recordsCacheHits: this.scanCounters.recordsCacheHits,
resolveCacheHits: this.scanCounters.resolveCacheHits,
resolveCalls: this.scanCounters.resolveCalls,
snapshotHits: this.existence?.hits ?? 0,
snapshotFallbacks: this.existence?.fallbacks ?? 0,
candidatesMs: this.timings.candidatesMs,
walkMs: this.timings.walkMs,
artScanMs: this.timings.artScanMs,
assetCount: [...this.assets.values()].reduce((sum, byId) => sum + byId.size, 0), assetCount: [...this.assets.values()].reduce((sum, byId) => sum + byId.size, 0),
defineCount: this.defines.size, defineCount: this.defines.size,
manifestFiles: this.manifests.size, manifestFiles: this.manifests.size,
manifestAssetCount, manifestAssetCount,
streams: this.streams.length, streams: this.streams.length,
sourceCandidates: this.sourceCandidates.length, sourceCandidates: this.sourceCandidates.length,
elapsedMs: Date.now() - start, elapsedMs: Date.now() - startedAt,
}, },
}; };
} }
@@ -294,142 +636,145 @@ export class ModIndexer {
stream.files.add(key); stream.files.add(key);
// Binary assets (w3x/dds/...) are referenced but never parsed. // readDocument returns compact index records for every indexable XML
if (!isXmlPath(path)) return; // document (full parse or shallow scan), or a bare file registration
// for binary / unparseable targets. During phase A, art-asset XML files
const parsed = await this.readDocument(path); // are registered and queued instead of scanned (deferredArt).
if (!parsed?.parse?.root) return; const parsed = await this.readDocument(path, this.deferArtScan ? { deferArt: true } : undefined);
const root = parsed.parse.root; if (!parsed) return;
if (parsed.deferredArt) {
for (const child of root.children) { this.artQueue.push({
const local = localName(child.name); path: parsed.file.path,
if (local === "Tags" || local === "Includes" || local === "Defines") continue; stream,
if (local === "include") { depth,
await this.handleXiInclude(child, parsed, stream, depth);
continue;
}
const idAttr = child.attrs.find((a) => a.name === "id");
if (idAttr) {
this.addAsset({
type: local,
id: idAttr.value,
file: parsed.file.path,
line: lineOf(parsed, idAttr.valueStart),
origin: this.originOf(parsed.file.path),
stream: stream.name,
viaInstance: mode === "instance", viaInstance: mode === "instance",
}); });
return;
}
if (parsed.records) {
await this.applyRecords(parsed, stream, depth, mode === "instance");
return;
} }
} }
for (const child of root.children) { /**
if (localName(child.name) !== "Defines") continue; * Applies a document's compact index records: top-level assets, defines,
for (const define of child.children) { * <Includes>, nested <xi:include> and root-level <xi:include> targets.
if (localName(define.name) !== "Define") continue; * Works identically for fully parsed XML and shallow-scanned art assets.
const name = define.attrs.find((a) => a.name === "name")?.value; */
const value = define.attrs.find((a) => a.name === "value")?.value; private async applyRecords(
if (!name) continue; parsed: ParsedFile,
stream: StreamInfo,
depth: number,
viaInstance: boolean,
): Promise<void> {
const records = parsed.records;
if (!records) return;
const file = parsed.file.path;
const origin = this.originOf(file);
for (const asset of records.assets) {
this.addAsset({
type: asset.type,
id: asset.id,
file,
line: asset.line,
origin,
stream: stream.name,
viaInstance,
});
}
for (const define of records.defines) {
const entry: DefineDef = { const entry: DefineDef = {
name, name: define.name,
value: value ?? "", value: define.value,
file: parsed.file.path, file,
line: lineOf(parsed, define.start), line: define.line,
origin: this.originOf(parsed.file.path), origin,
}; };
const arr = this.defines.get(name.toLowerCase()); const arr = this.defines.get(define.name.toLowerCase());
if (arr) arr.push(entry); if (arr) arr.push(entry);
else this.defines.set(name.toLowerCase(), [entry]); else this.defines.set(define.name.toLowerCase(), [entry]);
}
} }
const includesElem = root.children.find((c) => localName(c.name) === "Includes"); for (const inc of records.includes) {
if (includesElem) { const resolved = this.resolveCached(inc.source, dirname(file));
for (const inc of includesElem.children) {
if (localName(inc.name) !== "Include") continue;
const type = inc.attrs.find((a) => a.name === "type")?.value;
const source = inc.attrs.find((a) => a.name === "source")?.value;
if (!source) continue;
const resolved = resolveSource(source, dirname(parsed.file.path), this.searchPaths);
if (!resolved.path) { if (!resolved.path) {
this.diagnostics.push({ this.diagnostics.push({
file: parsed.file.path, file,
line: lineOf(parsed, inc.start), line: inc.line,
message: `Include target not found: ${source}`, message: `Include target not found: ${inc.source}`,
severity: "warning", severity: "warning",
code: "include-not-found", code: "include-not-found",
}); });
continue; continue;
} }
if (type === "all" || type === "instance") { if (inc.type === "all" || inc.type === "instance") {
await this.walk(resolved.path, type === "all" ? "all" : "instance", stream, depth + 1); await this.walk(
} else if (type === "reference") { resolved.path,
const manifestPath = manifestPathForReference(source, this.opts.builtmodsDirs); inc.type === "all" ? "all" : "instance",
stream,
depth + 1,
);
} else if (inc.type === "reference") {
const manifestPath = this.manifestPathCached(inc.source);
if (manifestPath) { if (manifestPath) {
const loaded = await this.loadManifest(manifestPath, stream.name); const loaded = await this.loadManifest(manifestPath, stream.name);
if (!loaded && isXmlPath(resolved.path)) { if (!loaded) await this.walk(resolved.path, "instance", stream, depth + 1);
// The manifest could not be parsed (missing/invalid): fall back } else {
// to the placeholder XML so its content is still available.
await this.walk(resolved.path, "instance", stream, depth + 1); await this.walk(resolved.path, "instance", stream, depth + 1);
} }
} else if (isXmlPath(resolved.path)) {
// reference to a real XML file: treat its assets as available
await this.walk(resolved.path, "instance", stream, depth + 1);
}
}
} }
} }
// Nested <xi:include> anywhere in the tree (not just under the root): for (const xi of records.nestedXiIncludes) {
// the target content is inlined into the parent element. We make the const resolved = this.resolveCached(xi.href, dirname(file));
// target file available and surface missing targets instead of ignoring
// them silently.
for (const el of parsed.parse.elements) {
if (localName(el.name) !== "include") continue;
if (el.parent === root) continue; // already handled in the loop above
const href = el.attrs.find((a) => a.name === "href")?.value;
if (!href) continue;
const resolved = resolveSource(href, dirname(parsed.file.path), this.searchPaths);
if (!resolved.path) { if (!resolved.path) {
this.diagnostics.push({ this.diagnostics.push({
file: parsed.file.path, file,
line: lineOf(parsed, el.start), line: xi.line,
message: `xi:include target not found: ${href}`, message: `xi:include target not found: ${xi.href}`,
severity: "warning", severity: "warning",
code: "include-not-found", code: "include-not-found",
}); });
continue; continue;
} }
stream.files.add(normKey(resolved.path)); stream.files.add(normKey(resolved.path));
if (isXmlPath(resolved.path)) {
await this.walk(resolved.path, "all", stream, depth + 1); await this.walk(resolved.path, "all", stream, depth + 1);
} }
for (const xi of records.rootXiIncludes) {
await this.handleRootXiInclude(xi, file, stream, depth);
} }
} }
private async handleXiInclude( /**
xi: XmlElement, * Root-level <xi:include> with an xpointer selects a named container's
parent: ParsedFile, * children in the target document, so the target's DOM is needed. These
* targets are rare, so they are parsed on demand (the tree is also cached).
*/
private async handleRootXiInclude(
xi: IndexRecordXi,
parentFile: string,
stream: StreamInfo, stream: StreamInfo,
depth: number, depth: number,
): Promise<void> { ): Promise<void> {
const href = xi.attrs.find((a) => a.name === "href")?.value; const resolved = this.resolveCached(xi.href, dirname(parentFile));
if (!href) return;
const resolved = resolveSource(href, dirname(parent.file.path), this.searchPaths);
if (!resolved.path) { if (!resolved.path) {
this.diagnostics.push({ this.diagnostics.push({
file: parent.file.path, file: parentFile,
line: lineOf(parent, xi.start), line: xi.line,
message: `xi:include target not found: ${href}`, message: `xi:include target not found: ${xi.href}`,
severity: "warning", severity: "warning",
code: "include-not-found", code: "include-not-found",
}); });
return; return;
} }
if (!isXmlPath(resolved.path)) return;
const target = await this.readDocument(resolved.path);
if (!target?.parse?.root) return;
const xpointer = xi.attrs.find((a) => a.name === "xpointer")?.value ?? ""; const target = await this.readDom(resolved.path);
if (target?.parse?.root) {
const xpointer = xi.xpointer ?? "";
let candidates: XmlElement[]; let candidates: XmlElement[];
if (xpointer) { if (xpointer) {
const container = findXPointerContainer(target.parse, xpointer); const container = findXPointerContainer(target.parse, xpointer);
@@ -452,8 +797,9 @@ export class ModIndexer {
}); });
} }
} }
stream.files.add(normKey(target.file.path)); }
await this.walk(target.file.path, "all", stream, depth + 1); stream.files.add(normKey(resolved.path));
await this.walk(resolved.path, "all", stream, depth + 1);
} }
// ── Manifest loading ────────────────────────────────────────────── // ── Manifest loading ──────────────────────────────────────────────
@@ -541,19 +887,37 @@ export class ModIndexer {
// ── Module-level helpers ───────────────────────────────────────────── // ── Module-level helpers ─────────────────────────────────────────────
function localName(tag: string): string {
const idx = tag.lastIndexOf(":");
return idx >= 0 ? tag.slice(idx + 1) : tag;
}
function lineOf(parsed: ParsedFile, offset: number): number { function lineOf(parsed: ParsedFile, offset: number): number {
if (!parsed.lineMap) return 0; if (!parsed.lineMap) return 0;
return parsed.lineMap.positionAt(offset).line + 1; return parsed.lineMap.positionAt(offset).line + 1;
} }
function isXmlPath(path: string): boolean { /**
const ext = extname(path).toLowerCase(); * Content sniffing for include targets with unknown extensions (e.g. art
return XML_EXTENSIONS.has(ext); * formats beyond .w3x): a small header that starts with "<" after an
* optional UTF-8 BOM / whitespace and contains no NUL bytes is treated as
* XML text; anything else is a binary asset (registered, never parsed).
*/
async function looksLikeXml(path: string): Promise<boolean> {
try {
const fh = await open(path, "r");
try {
const buf = Buffer.alloc(SNIFF_BYTES);
const { bytesRead } = await fh.read(buf, 0, SNIFF_BYTES, 0);
const head = buf.subarray(0, bytesRead);
if (head.includes(0)) return false;
let i = 0;
if (head[0] === 0xef && head[1] === 0xbb && head[2] === 0xbf) i = 3;
while (i < head.length && (head[i] === 0x20 || head[i] === 0x09 || head[i] === 0x0a || head[i] === 0x0d)) {
i++;
}
return i < head.length && head[i] === 0x3c; // "<"
} finally {
await fh.close();
}
} catch {
return false;
}
} }
async function findCaseInsensitiveDir(dir: string): Promise<string | null> { async function findCaseInsensitiveDir(dir: string): Promise<string | null> {
@@ -570,15 +934,6 @@ async function findCaseInsensitiveDir(dir: string): Promise<string | null> {
} }
} }
function findXPointerContainer(doc: XmlDocument, xpointer: string): XmlElement | null {
// Supports the form used by the mods:
// xmlns(n=uri:ea.com:eala:asset) xpointer(/n:ElementName/child::*)
const m = /xpointer\(\/\w+:(\w+)\/child::\*\)/.exec(xpointer);
if (!m) return null;
const name = m[1];
return doc.elements.find((el) => localName(el.name) === name) ?? null;
}
/** Keeps the first candidate for each case-insensitive source string. */ /** Keeps the first candidate for each case-insensitive source string. */
function dedupeSourceCandidates(candidates: SourceCandidate[]): SourceCandidate[] { function dedupeSourceCandidates(candidates: SourceCandidate[]): SourceCandidate[] {
const seen = new Set<string>(); const seen = new Set<string>();
+265
View File
@@ -0,0 +1,265 @@
import { dirname, resolve } from "node:path";
import { LineMap, parseXml, type XmlDocument } from "../language/xmlParser";
import { extractIndexRecords } from "./records";
import { resolveSource, type SearchPaths } from "./includeResolver";
import { expandDocument, type LogicalDocument } from "./logicalTree";
import type {
AssetDef,
DefineDef,
LocalOverlay,
ModIndex,
ParsedFile,
} from "./types";
export interface LocalScopeContext {
projectDir: string;
sdkDir: string;
searchPaths: SearchPaths;
/** Reads a file's compact index records (full parse or shallow scan). */
readRecords(path: string): Promise<ParsedFile | null>;
/** Reads a file and guarantees a DOM parse tree. */
readDom(path: string): Promise<ParsedFile | null>;
}
/**
* Everything the features need for the currently open document: the original
* parse, the expanded logical tree, per-source line maps, the local overlay
* and the overlay-aware index for lookups.
*/
export interface DocumentScope {
uri: string;
version: number;
parse: XmlDocument;
lineMap: LineMap;
expanded: LogicalDocument;
/** scopePathKey(file) -> line map (current file + expanded include targets). */
lineMaps: Map<string, LineMap>;
/** Local assets/defines from the document itself and its include chain. */
overlay: LocalOverlay;
/** Global index with `local` attached (or a minimal standalone index). */
merged: ModIndex | null;
}
/** Normalized key used for source-file identity / line-map lookup. */
export function scopePathKey(path: string): string {
return path.replace(/\\/g, "/").toLowerCase();
}
/**
* Builds the document scope for the current (possibly unsaved) text:
* - a local overlay of assets/defines reachable from this file;
* - a logical tree with supported xi:include targets spliced in place.
*/
export async function buildDocumentScope(
uri: string,
text: string,
version: number,
ctx: LocalScopeContext,
): Promise<DocumentScope> {
const lineMap = new LineMap(text);
const parse = parseXml(text);
const builder = new OverlayBuilder(ctx);
await builder.addEntry(uri, parse, lineMap);
const expanded = await expandDocument(uri, parse, {
resolve: (source, currentDir) =>
resolveSource(source, currentDir, ctx.searchPaths).path,
readDom: async (path) => {
const parsed = await ctx.readDom(path);
return parsed?.parse && parsed.lineMap
? { parse: parsed.parse, lineMap: parsed.lineMap }
: null;
},
});
const lineMaps = new Map<string, LineMap>();
lineMaps.set(scopePathKey(uri), lineMap);
for (const [path, lm] of builder.lineMaps) {
if (!lineMaps.has(path)) lineMaps.set(path, lm);
}
return {
uri,
version,
parse,
lineMap,
expanded,
lineMaps,
overlay: builder.overlay,
merged: null,
};
}
/**
* Attaches a document-local overlay to a global index without copying the
* global maps. When there is no global index yet, returns a minimal standalone
* index so the local chain alone can serve completions / references.
*/
export function withLocalOverlay(
global: ModIndex | null,
overlay: LocalOverlay,
projectDir: string,
sdkDir: string,
): ModIndex {
if (!global) {
return {
projectDir,
sdkDir,
complete: false,
phase: "xml",
assets: new Map(),
assetsById: new Map(),
defines: new Map(),
files: new Map(),
streams: [],
manifests: new Map(),
sourceCandidates: [],
diagnostics: [],
stats: {
projectDir,
sdkDir,
phase: "xml",
complete: false,
indexedFiles: 0,
parsedFiles: 0,
shallowScannedFiles: 0,
deferredArtFiles: 0,
shallowCacheHits: 0,
recordsCacheHits: 0,
resolveCacheHits: 0,
resolveCalls: 0,
snapshotHits: 0,
snapshotFallbacks: 0,
candidatesMs: 0,
walkMs: 0,
artScanMs: 0,
assetCount: 0,
defineCount: 0,
manifestFiles: 0,
manifestAssetCount: 0,
streams: 0,
sourceCandidates: 0,
elapsedMs: 0,
},
local: overlay,
};
}
return { ...global, local: overlay };
}
const MAX_LOCAL_DEPTH = 64;
class OverlayBuilder {
readonly overlay: LocalOverlay = {
assets: new Map(),
assetsById: new Map(),
defines: new Map(),
};
readonly lineMaps = new Map<string, LineMap>();
private visited = new Set<string>();
constructor(private ctx: LocalScopeContext) {}
async addEntry(
path: string,
parse: XmlDocument,
lineMap: LineMap,
): Promise<void> {
this.lineMaps.set(scopePathKey(path), lineMap);
await this.addParsed({
file: { path: resolve(path), stat: null },
parse,
records: extractIndexRecords(parse, lineMap),
lineMap,
}, 0);
}
async addFile(path: string, depth: number): Promise<void> {
if (depth > MAX_LOCAL_DEPTH) return;
const parsed = await this.ctx.readRecords(path);
if (parsed) await this.addParsed(parsed, depth);
}
private async addParsed(parsed: ParsedFile, depth: number): Promise<void> {
const path = parsed.file.path;
const key = scopePathKey(path);
if (!parsed.records || this.visited.has(key)) return;
this.visited.add(key);
if (parsed.lineMap) this.lineMaps.set(key, parsed.lineMap);
const origin = this.originOf(path);
for (const asset of parsed.records.assets) {
this.addAsset({
type: asset.type,
id: asset.id,
file: path,
line: asset.line,
origin,
stream: "local",
});
}
for (const define of parsed.records.defines) {
const entry: DefineDef = {
name: define.name,
value: define.value,
file: path,
line: define.line,
origin,
};
const arr = this.overlay.defines.get(define.name.toLowerCase());
if (arr) arr.push(entry);
else this.overlay.defines.set(define.name.toLowerCase(), [entry]);
}
for (const inc of parsed.records.includes) {
const resolved = resolveSource(inc.source, dirname(path), this.ctx.searchPaths);
if (!resolved.path) continue;
if (inc.type === "all" || inc.type === "instance") {
await this.addFile(resolved.path, depth + 1);
}
// type="reference" points at compiled manifests; their assets are
// provided by the global index, so the local text overlay skips them.
}
for (const xi of [
...parsed.records.nestedXiIncludes,
...parsed.records.rootXiIncludes,
]) {
const resolved = resolveSource(xi.href, dirname(path), this.ctx.searchPaths);
if (resolved.path) await this.addFile(resolved.path, depth + 1);
}
}
private addAsset(def: AssetDef): void {
const typeKey = def.type;
const idKey = def.id.toLowerCase();
let byId = this.overlay.assets.get(typeKey);
if (!byId) {
byId = new Map();
this.overlay.assets.set(typeKey, byId);
}
const arr = byId.get(idKey);
if (arr) {
if (arr.some((a) => a.file === def.file && a.line === def.line)) return;
arr.push(def);
} else {
byId.set(idKey, [def]);
}
const all = this.overlay.assetsById.get(idKey);
if (all) {
if (all.some((a) => a.file === def.file && a.line === def.line)) return;
all.push(def);
} else {
this.overlay.assetsById.set(idKey, [def]);
}
}
private originOf(path: string): "project" | "sdk" | "manifest" {
const p = resolve(path).toLowerCase();
const project = resolve(this.ctx.projectDir).toLowerCase();
const sdk = resolve(this.ctx.sdkDir).toLowerCase();
if (p.startsWith(project + "\\")) return "project";
if (sdk && p.startsWith(sdk + "\\")) return "sdk";
return "project";
}
}
+249
View File
@@ -0,0 +1,249 @@
import { dirname } from "node:path";
import type {
LineMap,
XmlDocument,
XmlElement,
XmlParseError,
} from "../language/xmlParser";
import { resolveElementType } from "../language/typeContext";
import { isAssignableTo } from "../model/schemaModel";
import { findXPointerContainer, localName } from "./xpointer";
/**
* A logical document is the parsed tree of the currently open file with
* supported `xi:include` targets spliced in place. Every node keeps its
* original source file and offsets so diagnostics / hover / navigation can
* map back to the real file.
*/
export interface LogicalElement extends Omit<XmlElement, "parent" | "children"> {
parent: LogicalElement | null;
children: LogicalElement[];
sourceFile: string;
}
export interface LogicalDocument {
root: LogicalElement | null;
elements: LogicalElement[];
/** Mirrors XmlDocument so existing helpers (findElementAt etc.) work. */
errors: XmlParseError[];
declarationEnd: number;
}
export interface ExpandContext {
/** Resolves an xi:include href (BAB search order). */
resolve(source: string, currentDir: string): string | null;
/** Reads a target and guarantees a DOM parse tree. */
readDom(path: string): Promise<{ parse: XmlDocument; lineMap: LineMap } | null>;
/** Include-depth guard. Defaults to 64. */
maxDepth?: number;
}
const DEFAULT_MAX_DEPTH = 64;
/**
* Builds the logical document for `entryPath` by replacing supported
* `xi:include` elements with their selected target children.
*
* The original xi:include node stays in `elements` (so hover keeps working)
* but is removed from the logical child list; its selected content is spliced
* in as siblings. Nodes are shallow-cloned shells with a rebuilt parent/child
* chain, so cached parse trees in the indexer are never mutated.
*/
export async function expandDocument(
entryPath: string,
parse: XmlDocument,
ctx: ExpandContext,
): Promise<LogicalDocument> {
const elements: LogicalElement[] = [];
const map = new Map<XmlElement, LogicalElement>();
// Pre-create a logical shell for every original element (including orphan
// nodes produced by the parser's unterminated-quote recovery). Parent
// pointers follow the original tree so type context is preserved.
for (const orig of parse.elements) {
const parent = orig.parent ? map.get(orig.parent) ?? null : null;
const clone = cloneNode(orig, parent, entryPath);
map.set(orig, clone);
elements.push(clone);
}
// Cycle guard is a recursion stack, not a global visited set: the same
// fragment may legitimately be included under several parents.
const stack = new Set<string>();
const root = parse.root ? map.get(parse.root) ?? null : null;
// Traverse from the real root plus any parser-recovery orphans (elements
// whose parent is null but are not the root).
const roots = new Set<XmlElement>();
if (parse.root) roots.add(parse.root);
for (const orig of parse.elements) {
if (orig !== parse.root && orig.parent === null) roots.add(orig);
}
for (const origRoot of roots) {
const logicalRoot = map.get(origRoot)!;
await expandChildren(
origRoot,
logicalRoot,
entryPath,
0,
ctx,
elements,
stack,
map,
);
}
return { root, elements, errors: parse.errors, declarationEnd: parse.declarationEnd };
}
function cloneNode(
el: XmlElement,
parent: LogicalElement | null,
sourceFile: string,
): LogicalElement {
return { ...el, parent, children: [], sourceFile };
}
async function expandChildren(
origParent: XmlElement,
logicalParent: LogicalElement,
file: string,
depth: number,
ctx: ExpandContext,
elements: LogicalElement[],
stack: Set<string>,
map?: Map<XmlElement, LogicalElement>,
): Promise<void> {
for (const child of origParent.children) {
await handleChild(child, logicalParent, file, depth, ctx, elements, stack, map);
}
}
async function handleChild(
orig: XmlElement,
logicalParent: LogicalElement,
file: string,
depth: number,
ctx: ExpandContext,
elements: LogicalElement[],
stack: Set<string>,
map?: Map<XmlElement, LogicalElement>,
): Promise<void> {
const isXi = orig.name.toLowerCase().startsWith("xi:") &&
localName(orig.name).toLowerCase() === "include";
if (isXi) {
// Keep the xi:include itself discoverable (hover), but let its selected
// content replace it in the logical child list.
if (!map?.has(orig)) elements.push(cloneNode(orig, logicalParent, file));
await expandXi(orig, logicalParent, file, depth, ctx, elements, stack);
return;
}
let clone = map?.get(orig);
if (!clone) {
clone = cloneNode(orig, logicalParent, file);
elements.push(clone);
}
logicalParent.children.push(clone);
await expandChildren(orig, clone, file, depth + 1, ctx, elements, stack, map);
}
async function expandXi(
xi: XmlElement,
logicalParent: LogicalElement,
parentFile: string,
depth: number,
ctx: ExpandContext,
elements: LogicalElement[],
stack: Set<string>,
): Promise<void> {
const href = xi.attrs.find((a) => a.name === "href")?.value;
if (!href) return;
const resolved = ctx.resolve(href, dirname(parentFile));
if (!resolved) return;
const key = normPath(resolved);
if (stack.has(key)) return; // include cycle
if (depth > (ctx.maxDepth ?? DEFAULT_MAX_DEPTH)) return;
stack.add(key);
try {
const target = await ctx.readDom(resolved);
if (!target?.parse?.root) return;
const xpointer = xi.attrs.find((a) => a.name === "xpointer")?.value ?? "";
const selected = xpointer
? findXPointerContainer(target.parse, xpointer)?.children ?? []
: target.parse.root.children;
for (const sel of selected) {
await handleChild(sel, logicalParent, resolved, depth + 1, ctx, elements, stack);
}
} finally {
stack.delete(key);
}
}
/** True when a logical element's resolved XSD type is a GameObject. */
export function isGameObjectElement(el: LogicalElement): boolean {
const type = resolveElementType(el);
return type != null && isAssignableTo(type, "GameObject");
}
/** Nearest ancestor whose resolved type is a GameObject (or subclass). */
export function findContainingGameObject(
el: LogicalElement,
): LogicalElement | null {
let cur = el.parent;
while (cur) {
if (isGameObjectElement(cur)) return cur;
cur = cur.parent;
}
return null;
}
export interface LocalIdInfo {
id: string;
el: LogicalElement;
}
/**
* Collects every `id` defined inside a GameObject subtree (including modules
* spliced in through xi:include). These are the candidates for Poid-typed
* pipeline-local references such as AttachModuleId / ModuleId.
*/
export function collectLocalIds(root: LogicalElement): LocalIdInfo[] {
const out: LocalIdInfo[] = [];
const seen = new Set<string>();
const stack: LogicalElement[] = [root];
while (stack.length) {
const el = stack.pop()!;
const idAttr = el.attrs.find((a) => a.name === "id");
if (idAttr?.hasValue) {
const key = idAttr.value.toLowerCase();
if (!seen.has(key)) {
seen.add(key);
out.push({ id: idAttr.value, el });
}
}
for (const child of el.children) stack.push(child);
}
return out;
}
/** Finds an id inside a GameObject subtree (case-insensitive). */
export function findLocalId(
root: LogicalElement,
id: string,
): LogicalElement | null {
const wanted = id.toLowerCase();
const stack: LogicalElement[] = [root];
while (stack.length) {
const el = stack.pop()!;
const idAttr = el.attrs.find((a) => a.name === "id");
if (idAttr?.hasValue && idAttr.value.toLowerCase() === wanted) return el;
for (const child of el.children) stack.push(child);
}
return null;
}
function normPath(path: string): string {
return path.replace(/\\/g, "/").toLowerCase();
}
+177
View File
@@ -0,0 +1,177 @@
/**
* Compact per-file index records.
*
* A rebuild only needs each file's top-level assets, defines, includes and
* xi:include targets — not the DOM. Extracting these records at parse time
* and caching them across rebuilds lets Corona-scale rebuilds skip both the
* DOM and the file I/O for unchanged files, while keeping the DOM cache small
* for on-demand features (hover / navigation / outline).
*
* Pure TypeScript: no vscode dependency.
*/
import type { LineMap, XmlDocument } from "../language/xmlParser";
import type { ShallowDocument } from "./shallowScan";
export interface IndexRecordAsset {
/** Top-level element name, e.g. "W3DContainer". */
type: string;
id: string;
/** 1-based line of the id attribute value. */
line: number;
}
export interface IndexRecordDefine {
name: string;
value: string;
/** 1-based line of the <Define> element. */
line: number;
}
export interface IndexRecordInclude {
type: "all" | "instance" | "reference" | null;
source: string;
/** 1-based line of the <Include> element. */
line: number;
}
export interface IndexRecordXi {
href: string;
xpointer: string | null;
/** 1-based line of the <xi:include> element. */
line: number;
}
export interface IndexRecords {
assets: IndexRecordAsset[];
defines: IndexRecordDefine[];
includes: IndexRecordInclude[];
/** <xi:include> elements that are direct children of the root. */
rootXiIncludes: IndexRecordXi[];
/** <xi:include> elements nested anywhere else in the document. */
nestedXiIncludes: IndexRecordXi[];
}
const INCLUDE_TYPES = new Set(["all", "instance", "reference"]);
function lineOf(lineMap: LineMap, offset: number): number {
return lineMap.positionAt(offset).line + 1;
}
function localName(tag: string): string {
const idx = tag.lastIndexOf(":");
return idx >= 0 ? tag.slice(idx + 1) : tag;
}
/**
* Extracts the index records of a fully parsed document. Mirrors the walk
* semantics of the indexer exactly: top-level assets (excluding
* Tags/Includes/Defines), $DEFINE constants, the top-level <Includes> block
* and root/nested <xi:include> elements.
*/
export function extractIndexRecords(parse: XmlDocument, lineMap: LineMap): IndexRecords {
const assets: IndexRecordAsset[] = [];
const defines: IndexRecordDefine[] = [];
const includes: IndexRecordInclude[] = [];
const rootXiIncludes: IndexRecordXi[] = [];
const nestedXiIncludes: IndexRecordXi[] = [];
const root = parse.root;
if (!root) return { assets, defines, includes, rootXiIncludes, nestedXiIncludes };
for (const child of root.children) {
const local = localName(child.name);
if (local === "Tags" || local === "Includes" || local === "Defines") continue;
if (local === "include") {
const href = child.attrs.find((a) => a.name === "href")?.value;
if (href) {
rootXiIncludes.push({
href,
xpointer: child.attrs.find((a) => a.name === "xpointer")?.value ?? null,
line: lineOf(lineMap, child.start),
});
}
continue;
}
const idAttr = child.attrs.find((a) => a.name === "id");
if (idAttr) {
assets.push({ type: local, id: idAttr.value, line: lineOf(lineMap, idAttr.valueStart) });
}
}
for (const child of root.children) {
if (localName(child.name) !== "Defines") continue;
for (const define of child.children) {
if (localName(define.name) !== "Define") continue;
const name = define.attrs.find((a) => a.name === "name")?.value;
if (!name) continue;
defines.push({
name,
value: define.attrs.find((a) => a.name === "value")?.value ?? "",
line: lineOf(lineMap, define.start),
});
}
}
const includesElem = root.children.find((c) => localName(c.name) === "Includes");
if (includesElem) {
for (const inc of includesElem.children) {
if (localName(inc.name) !== "Include") continue;
const source = inc.attrs.find((a) => a.name === "source")?.value;
if (!source) continue;
const type = inc.attrs.find((a) => a.name === "type")?.value;
includes.push({
type:
type && INCLUDE_TYPES.has(type)
? (type as "all" | "instance" | "reference")
: null,
source,
line: lineOf(lineMap, inc.start),
});
}
}
for (const el of parse.elements) {
if (localName(el.name) !== "include") continue;
if (el.parent === root) continue; // already handled above
const href = el.attrs.find((a) => a.name === "href")?.value;
if (!href) continue;
nestedXiIncludes.push({
href,
xpointer: el.attrs.find((a) => a.name === "xpointer")?.value ?? null,
line: lineOf(lineMap, el.start),
});
}
return { assets, defines, includes, rootXiIncludes, nestedXiIncludes };
}
/** Converts a shallow scan (offsets) into index records (1-based lines). */
export function recordsFromShallow(scan: ShallowDocument, lineMap: LineMap): IndexRecords {
return {
assets: scan.assets.map((a) => ({
type: a.name,
id: a.id,
line: lineOf(lineMap, a.idValueStart),
})),
defines: scan.defines.map((d) => ({
name: d.name,
value: d.value,
line: lineOf(lineMap, d.start),
})),
includes: scan.includes.map((i) => ({
type: i.type,
source: i.source,
line: lineOf(lineMap, i.start),
})),
rootXiIncludes: scan.rootXiIncludes.map((x) => ({
href: x.href,
xpointer: x.xpointer,
line: lineOf(lineMap, x.start),
})),
nestedXiIncludes: scan.nestedXiIncludes.map((x) => ({
href: x.href,
xpointer: x.xpointer,
line: lineOf(lineMap, x.start),
})),
};
}
+79 -2
View File
@@ -2,6 +2,8 @@ import {
attributesOfType, attributesOfType,
elementTypeName, elementTypeName,
isAssignableTo, isAssignableTo,
typeChain,
typeInfo,
} from "../model/schemaModel"; } from "../model/schemaModel";
import type { AssetDef, ModIndex } from "./types"; import type { AssetDef, ModIndex } from "./types";
@@ -96,8 +98,11 @@ export function resolveReferenceTargetsForType(
attrName: string, attrName: string,
id: string, id: string,
): ReferenceTarget[] { ): ReferenceTarget[] {
const defs = idx.assetsById.get(id.toLowerCase()); const defs = mergeLocalAndGlobalDefs(
if (!defs?.length) return []; idx.local?.assetsById.get(id.toLowerCase()),
idx.assetsById.get(id.toLowerCase()),
);
if (!defs.length) return [];
const nameLower = attrName.toLowerCase(); const nameLower = attrName.toLowerCase();
let refType: string | null = null; let refType: string | null = null;
@@ -114,6 +119,56 @@ export function resolveReferenceTargetsForType(
refType = attr.refType; refType = attr.refType;
} }
return filterAndScoreDefs(defs, refType, selfType);
}
/**
* True when an element's text content is a typed reference to a global
* asset: the element's resolved XSD type is a simple type carrying an
* `xas:refType` (e.g. `<CreateObject>` with `GameObjectWeakRef`).
*
* Only *typed* refs are treated as content references. Generic untyped
* `AssetReference` content is used by real data for shader constants,
* mesh sub-object names and other values that are not global asset ids
* (`FXShaderConstantTexture@Value`, `RenderSubObjectReference@Mesh`), so
* resolving those globally would produce false hover/navigation/diagnostics.
* Poid pipeline-local ids are excluded for the same reason.
*/
export function isReferenceContentType(typeName: string | null): boolean {
if (!typeName) return false;
const info = typeInfo(typeName);
if (info?.kind !== "simple") return false;
if (typeChain(typeName).includes("Poid")) return false;
return info.refType != null;
}
/**
* Resolves the definitions an element's text content should point to,
* filtered by the element type's `xas:refType`
* (e.g. `GameObjectWeakRef` -> `GameObject`).
*/
export function resolveContentReferenceTargets(
idx: ModIndex,
typeName: string | null,
id: string,
): ReferenceTarget[] {
if (!isReferenceContentType(typeName)) return [];
if (!typeName) return [];
const defs = mergeLocalAndGlobalDefs(
idx.local?.assetsById.get(id.toLowerCase()),
idx.assetsById.get(id.toLowerCase()),
);
if (!defs.length) return [];
const info = typeInfo(typeName);
const refType = info?.kind === "simple" ? info.refType : null;
return filterAndScoreDefs(defs, refType, null);
}
function filterAndScoreDefs(
defs: readonly AssetDef[],
refType: string | null,
selfType: string | null,
): ReferenceTarget[] {
const targets: ReferenceTarget[] = []; const targets: ReferenceTarget[] = [];
for (const def of defs) { for (const def of defs) {
if (refType && !isAssignableTo(def.type, refType)) continue; if (refType && !isAssignableTo(def.type, refType)) continue;
@@ -127,3 +182,25 @@ export function resolveReferenceTargetsForType(
targets.sort((a, b) => a.score - b.score || a.def.id.localeCompare(b.def.id)); targets.sort((a, b) => a.score - b.score || a.def.id.localeCompare(b.def.id));
return targets; return targets;
} }
/**
* Merges document-local definitions with the global index, keeping local
* entries first and de-duplicating definitions that exist in both.
*/
export function mergeLocalAndGlobalDefs(
local: readonly AssetDef[] | undefined,
global: readonly AssetDef[] | undefined,
): AssetDef[] {
const seen = new Set<string>();
const out: AssetDef[] = [];
for (const list of [local, global]) {
if (!list) continue;
for (const def of list) {
const key = `${def.type}\u0000${def.id.toLowerCase()}\u0000${def.file}\u0000${def.line}`;
if (seen.has(key)) continue;
seen.add(key);
out.push(def);
}
}
return out;
}
+274
View File
@@ -0,0 +1,274 @@
/**
* Shallow XML scanner for large art-asset documents (e.g. .w3x).
*
* Full XML parsing builds an element tree whose memory footprint is roughly
* 17x the source text (measured on Corona model files). Model exports like
* <W3DMesh> contain hundreds of thousands of tiny numeric elements
* (Vertices/V, Triangles/T, ...) that the extension never needs: the index
* only consumes top-level asset definitions (name + id), top-level
* <Includes>, nested <xi:include> targets and <Defines> constants.
*
* This scanner performs a single linear pass without building child nodes,
* so multi-megabyte model files can be indexed in linear time with ~zero
* retained memory.
*
* Pure TypeScript: no vscode dependency, reusable outside the extension.
*/
export interface ShallowAssetRecord {
/** Top-level element name, e.g. "W3DContainer". */
name: string;
/** Value of the id attribute. */
id: string;
/** Offset of the first id value character. */
idValueStart: number;
/** Offset one past the last id value character. */
idValueEnd: number;
/** Offset of the element's "<". */
start: number;
/** Offset one past the ">" of the start tag. */
startTagEnd: number;
}
export interface ShallowIncludeRecord {
/** "all" | "instance" | "reference", or null when absent/unknown. */
type: "all" | "instance" | "reference" | null;
source: string;
/** Offset of the <Include> element. */
start: number;
}
export interface ShallowXiIncludeRecord {
href: string;
xpointer: string | null;
/** Offset of the <xi:include> element. */
start: number;
}
export interface ShallowDefineRecord {
name: string;
value: string;
/** Offset of the <Define> element. */
start: number;
}
export interface ShallowScanError {
message: string;
offset: number;
}
export interface ShallowDocument {
assets: ShallowAssetRecord[];
includes: ShallowIncludeRecord[];
/** <xi:include> elements that are direct children of the root. */
rootXiIncludes: ShallowXiIncludeRecord[];
/** <xi:include> elements nested anywhere else in the document. */
nestedXiIncludes: ShallowXiIncludeRecord[];
defines: ShallowDefineRecord[];
errors: ShallowScanError[];
}
interface AttrHit {
value: string;
valueStart: number;
valueEnd: number;
}
/**
* Matches a complete XML tag while respecting quoted attribute values, so a
* ">" or "/" inside a value never terminates the tag early.
*/
const TAG_RE = /<(?:"[^"]*"|'[^']*'|[^'"<>])*>/g;
export function scanXmlShallow(text: string): ShallowDocument {
const errors: ShallowScanError[] = [];
const assets: ShallowAssetRecord[] = [];
const includes: ShallowIncludeRecord[] = [];
const rootXiIncludes: ShallowXiIncludeRecord[] = [];
const nestedXiIncludes: ShallowXiIncludeRecord[] = [];
const defines: ShallowDefineRecord[] = [];
// Depth of the currently open element stack. The document root opens at
// depth 0 -> 1, so its direct children open when depth === 1.
let depth = 0;
let inIncludes = false;
let inDefines = false;
let i = 0;
while (i < text.length) {
const lt = text.indexOf("<", i);
if (lt < 0) break;
// Non-element constructs: comments, CDATA, DOCTYPE and processing
// instructions are skipped whole so their content never looks like tags.
if (text.startsWith("<!--", lt)) {
const close = text.indexOf("-->", lt + 4);
if (close < 0) {
errors.push({ message: "Unterminated comment", offset: lt });
break;
}
i = close + 3;
continue;
}
if (text.startsWith("<![CDATA[", lt)) {
const close = text.indexOf("]]>", lt + 9);
if (close < 0) {
errors.push({ message: "Unterminated CDATA section", offset: lt });
break;
}
i = close + 3;
continue;
}
if (text.startsWith("<!", lt)) {
const close = text.indexOf(">", lt + 2);
if (close < 0) {
errors.push({ message: "Unterminated DOCTYPE", offset: lt });
break;
}
i = close + 1;
continue;
}
if (text.startsWith("<?", lt)) {
const close = text.indexOf("?>", lt + 2);
if (close < 0) {
errors.push({ message: "Unterminated processing instruction", offset: lt });
break;
}
i = close + 2;
continue;
}
TAG_RE.lastIndex = lt;
const m = TAG_RE.exec(text);
if (!m) {
errors.push({ message: "Unterminated tag", offset: lt });
break;
}
const tag = m[0];
const gt = m.index + tag.length - 1;
const inner = tag.slice(1, -1);
const closing = inner.startsWith("/");
const selfClosing = !closing && /\/\s*$/.test(inner);
const body = closing ? inner.slice(1) : inner;
let nameEnd = 0;
while (nameEnd < body.length && !/[\s/>]/.test(body[nameEnd])) nameEnd++;
const name = body.slice(0, nameEnd);
const base = lt + 1;
if (!closing) {
// Top-level elements (direct children of the root).
if (depth === 1) {
inIncludes = name === "Includes";
inDefines = name === "Defines";
const idAttr = findAttr(inner, base, "id");
if (idAttr && idAttr.value) {
assets.push({
name,
id: idAttr.value,
idValueStart: idAttr.valueStart,
idValueEnd: idAttr.valueEnd,
start: lt,
startTagEnd: gt + 1,
});
}
}
// <Include> entries inside the top-level <Includes> block.
if (inIncludes && depth === 2 && name === "Include") {
const type = findAttr(inner, base, "type");
const source = findAttr(inner, base, "source");
if (source?.value) {
includes.push({
type:
type?.value === "all" || type?.value === "instance" || type?.value === "reference"
? type.value
: null,
source: source.value,
start: lt,
});
}
}
// <Define> entries inside the top-level <Defines> block.
if (inDefines && depth === 2 && name === "Define") {
const nameAttr = findAttr(inner, base, "name");
const valueAttr = findAttr(inner, base, "value");
if (nameAttr?.value) {
defines.push({ name: nameAttr.value, value: valueAttr?.value ?? "", start: lt });
}
}
// Nested <xi:include> (or any *:include) anywhere in the document.
if (localName(name) === "include") {
const href = findAttr(inner, base, "href");
const xpointer = findAttr(inner, base, "xpointer");
if (href?.value) {
const rec: ShallowXiIncludeRecord = {
href: href.value,
xpointer: xpointer?.value ?? null,
start: lt,
};
if (depth === 1) rootXiIncludes.push(rec);
else nestedXiIncludes.push(rec);
}
}
if (!selfClosing) depth++;
} else {
depth--;
}
i = gt + 1;
}
return { assets, includes, rootXiIncludes, nestedXiIncludes, defines, errors };
}
/**
* Scans the attributes of a tag (its inner text, without "<" and ">") for
* the first occurrence of `want` and returns its value with absolute offsets
* (base = offset one past the "<").
*/
function findAttr(inner: string, base: number, want: string): AttrHit | null {
let i = 0;
while (i < inner.length) {
while (i < inner.length && /\s/.test(inner[i])) i++;
// Self-closing marker or end of tag: no more attributes.
if (i >= inner.length || inner[i] === "/" || inner[i] === ">") break;
const nameStart = i;
while (i < inner.length && !/[\s=/>]/.test(inner[i])) i++;
const name = inner.slice(nameStart, i);
while (i < inner.length && /\s/.test(inner[i])) i++;
if (inner[i] === "=") {
i++;
while (i < inner.length && /\s/.test(inner[i])) i++;
const q = inner[i];
if (q === '"' || q === "'") {
const vStart = i + 1;
const vEnd = inner.indexOf(q, vStart);
if (vEnd < 0) {
return name === want ? { value: "", valueStart: -1, valueEnd: -1 } : null;
}
if (name === want) {
return {
value: inner.slice(vStart, vEnd),
valueStart: base + vStart,
valueEnd: base + vEnd,
};
}
i = vEnd + 1;
continue;
}
// Unquoted value (tolerated).
const vs = i;
while (i < inner.length && !/[\s>]/.test(inner[i])) i++;
if (name === want) {
return { value: inner.slice(vs, i), valueStart: base + vs, valueEnd: base + i };
}
continue;
}
// Attribute without a value (rare but tolerated).
if (name === want) return { value: "", valueStart: -1, valueEnd: -1 };
}
return null;
}
function localName(name: string): string {
const idx = name.lastIndexOf(":");
return idx >= 0 ? name.slice(idx + 1) : name;
}
+102 -1
View File
@@ -1,6 +1,8 @@
import type { XmlDocument } from "../language/xmlParser"; import type { XmlDocument } from "../language/xmlParser";
import type { ManifestInfo } from "./manifestParser"; import type { ManifestInfo } from "./manifestParser";
import type { LineMap } from "../language/xmlParser"; import type { LineMap } from "../language/xmlParser";
import type { IndexRecords } from "./records";
import type { DocumentCache, IncludeResolveCache, IndexRecordsCache } from "./caches";
export type AssetOrigin = "project" | "sdk" | "manifest"; export type AssetOrigin = "project" | "sdk" | "manifest";
@@ -30,9 +32,36 @@ export interface DefineDef {
origin: AssetOrigin; origin: AssetOrigin;
} }
/**
* Document-local overlay produced by `localScope.ts`. It contains assets /
* defines reachable from the currently open document (its own text plus its
* include chain), even when that file is not part of any global stream.
*
* The overlay is attached to a `ModIndex` as `local` rather than merged into
* the global maps, so large indexes (Corona: ~65k assets) are never copied
* on every keystroke. Lookup helpers consult `local` first.
*/
export interface LocalOverlay {
/** type -> id -> definitions. */
assets: Map<string, Map<string, AssetDef[]>>;
/** id -> definitions across all types. */
assetsById: Map<string, AssetDef[]>;
/** `$NAME` -> definitions. */
defines: Map<string, DefineDef[]>;
}
export interface IndexedFile { export interface IndexedFile {
path: string; path: string;
stat: { mtimeMs: number; size: number } | null; /**
* Multi-signal file stamp used to validate cached entries without
* re-reading content: size, last-write time, creation time and change
* time. Creation/change time catch tools that rewrite a file while
* preserving its mtime (e.g. temp-file + rename exporters) — important on
* removable drives where mtime resolution can be coarse (FAT32: 2 s).
*/
stat:
| { mtimeMs: number; size: number; birthtimeMs: number; ctimeMs: number }
| null;
} }
export interface StreamInfo { export interface StreamInfo {
@@ -63,8 +92,34 @@ export interface IndexerDiagnostic {
export interface IndexStats { export interface IndexStats {
projectDir: string; projectDir: string;
sdkDir: string; sdkDir: string;
/** Last finished phase ("xml" or "art"). */
phase: "xml" | "art";
/** Whether the index is fully complete (art assets included). */
complete: boolean;
indexedFiles: number; indexedFiles: number;
parsedFiles: number; parsedFiles: number;
/** Art-asset documents indexed via shallow scan (no DOM tree). */
shallowScannedFiles: number;
/** Art files registered during the XML phase, scanned later in phase B. */
deferredArtFiles: number;
/** Shallow scans served from the persistent cache (unchanged files). */
shallowCacheHits: number;
/** Parsed XML files served from the persistent records cache. */
recordsCacheHits: number;
/** Include/xi:include resolutions served from the resolve cache. */
resolveCacheHits: number;
/** Include/xi:include resolutions performed during this build. */
resolveCalls: number;
/** Existence checks answered by the directory snapshot (no statSync). */
snapshotHits: number;
/** Existence checks outside the snapshot that fell back to statSync. */
snapshotFallbacks: number;
/** Time spent enumerating Include source candidates (ms). */
candidatesMs: number;
/** Time spent walking the include graph (ms). */
walkMs: number;
/** Time spent shallow-scanning deferred art assets (ms). */
artScanMs: number;
assetCount: number; assetCount: number;
defineCount: number; defineCount: number;
manifestFiles: number; manifestFiles: number;
@@ -77,6 +132,19 @@ export interface IndexStats {
export interface ModIndex { export interface ModIndex {
projectDir: string; projectDir: string;
sdkDir: string; sdkDir: string;
/**
* True when every indexing phase (including the art-asset shallow scan)
* has finished. Features use this to decide whether unresolved references
* are final errors or provisional "may be a false positive" diagnostics.
*/
complete: boolean;
/** Last finished phase: "xml" (XML + manifests) or "art" (final). */
phase: "xml" | "art";
/**
* True when files changed while this snapshot was being built, so some
* entries may be stale. A follow-up rebuild is scheduled by the workspace.
*/
stale?: boolean;
/** type -> id -> definitions (project + sdk + manifest, deduplicated). */ /** type -> id -> definitions (project + sdk + manifest, deduplicated). */
assets: Map<string, Map<string, AssetDef[]>>; assets: Map<string, Map<string, AssetDef[]>>;
/** id -> definitions across all types. */ /** id -> definitions across all types. */
@@ -92,6 +160,12 @@ export interface ModIndex {
/** Problems found while indexing (unresolved includes, cycles, ...). */ /** Problems found while indexing (unresolved includes, cycles, ...). */
diagnostics: IndexerDiagnostic[]; diagnostics: IndexerDiagnostic[];
stats: IndexStats; stats: IndexStats;
/**
* Document-local overlay (when the index was obtained through the
* workspace's `getScope` / `getIndex` path). Optional so plain indexer
* snapshots remain overlay-free.
*/
local?: LocalOverlay;
} }
export interface IndexOptions { export interface IndexOptions {
@@ -102,6 +176,23 @@ export interface IndexOptions {
additionalDataSearchPaths: string[]; additionalDataSearchPaths: string[];
/** Directory walker used to enumerate files for source completion. */ /** Directory walker used to enumerate files for source completion. */
walker: FileWalker; walker: FileWalker;
/** Optional parse-tree cache shared across rebuilds (owned by the workspace). */
documentCache?: DocumentCache;
/** Optional index-records cache shared across rebuilds (owned by the workspace). */
recordsCache?: IndexRecordsCache;
/** Optional include-resolution cache shared across rebuilds (owned by the workspace). */
resolveCache?: IncludeResolveCache;
/**
* When true, cached documents whose path is not in `changedFiles` are used
* without a per-file stat. The workspace enables this while a file watcher
* invalidates caches for changed paths; a forced reindex passes false.
*/
trustUnchanged?: boolean;
/**
* Normalized paths (see `normKey`) known to have changed since the caches
* were populated. Only consulted when `trustUnchanged` is true.
*/
changedFiles?: ReadonlySet<string>;
} }
export interface FileWalker { export interface FileWalker {
@@ -121,5 +212,15 @@ export interface ParseCache {
export interface ParsedFile { export interface ParsedFile {
file: IndexedFile; file: IndexedFile;
parse: XmlDocument | null; parse: XmlDocument | null;
/**
* Compact index records (assets/defines/includes/xi:include with lines).
* Present for every indexable XML document; null for binary files.
*/
records: IndexRecords | null;
lineMap: LineMap | null; lineMap: LineMap | null;
/**
* True when the file is an art-asset XML that was only registered during
* the XML phase (its shallow scan is deferred to the art phase).
*/
deferredArt?: boolean;
} }
+24
View File
@@ -0,0 +1,24 @@
import type { XmlDocument, XmlElement } from "../language/xmlParser";
/** Lowercases nothing; returns the part after the last ":" in a tag name. */
export function localName(tag: string): string {
const idx = tag.lastIndexOf(":");
return idx >= 0 ? tag.slice(idx + 1) : tag;
}
/**
* Resolves the xpointer subset used by real RA3 mods:
* xmlns(n=uri:ea.com:eala:asset) xpointer(/n:ElementName/child::*)
*
* Returns the container element whose children are selected by the xpointer,
* or null when the form is unsupported / the container is missing.
*/
export function findXPointerContainer(
doc: XmlDocument,
xpointer: string,
): XmlElement | null {
const m = /xpointer\(\/\w+:(\w+)\/child::\*\)/.exec(xpointer);
if (!m) return null;
const name = m[1];
return doc.elements.find((el) => localName(el.name) === name) ?? null;
}
+49 -4
View File
@@ -1,4 +1,5 @@
import type { XmlAttribute, XmlDocument, XmlElement } from "./xmlParser"; import type { XmlAttribute, XmlDocument, XmlElement } from "./xmlParser";
import { elementContainsOffset, parseTag } from "./xmlParser";
export type ContextKind = export type ContextKind =
| "element-name" | "element-name"
@@ -30,18 +31,35 @@ export function analyzeContext(
let container: XmlElement | null = null; let container: XmlElement | null = null;
for (const el of doc.elements) { for (const el of doc.elements) {
if (el.end < 0) continue; if (el.end < 0) continue;
if (offset >= el.start && offset <= el.end) { if (elementContainsOffset(el, offset)) {
if (!container || el.depth > container.depth) container = el; if (!container || el.depth > container.depth) container = el;
} }
} }
if (!container) return empty("none"); if (!container) return empty("none");
// Inside the start tag of the element. // Inside the start tag of the element. The boundary right after `>` is the
if (offset >= container.start && offset <= container.startTagEnd) { // start of the content (e.g. the `$1` cursor in
// `<CreateObject>$1</CreateObject>`), not another attribute slot; only an
// unterminated start tag whose `>` has not been typed yet still belongs to
// the start tag at its recovered end.
const atTagEnd = offset === container.startTagEnd;
const tagClosed =
atTagEnd && container.startTagEnd > container.start &&
text[container.startTagEnd - 1] === ">";
if (offset >= container.start && (offset < container.startTagEnd || (atTagEnd && !tagClosed))) {
return analyzeStartTag(container, text, offset); return analyzeStartTag(container, text, offset);
} }
// A start tag that had to be recovered (e.g. an attribute quote is still
// open) is truncated at the first line break, so attributes typed on later
// lines of the same tag are not part of the parsed element. Re-parse the
// partial tag up to the cursor so value/attribute completion keeps working
// while typing in the malformed tag.
if (container.recoveredStartTag) {
return analyzeRecoveredStartTag(container, text, offset);
}
// Otherwise the cursor is in element content. // Otherwise the cursor is in element content.
return { return {
kind: "content", kind: "content",
@@ -53,6 +71,33 @@ export function analyzeContext(
}; };
} }
/**
* Classifies the cursor inside a recovered (truncated) start tag by re-parsing
* the raw tag content up to the cursor. The original element only kept the
* attributes that fit on its first line; re-parsing the partial text recovers
* attributes typed on later lines without affecting the rest of the document.
*/
function analyzeRecoveredStartTag(
el: XmlElement,
text: string,
offset: number,
): CompletionContext {
const raw = parseTag(text.slice(el.start + 1, offset), el.start + 1);
const partial: XmlElement = {
name: raw.name,
attrs: raw.attrs,
children: [],
parent: el.parent,
start: raw.start,
startTagEnd: offset,
end: offset,
selfClosing: raw.selfClosing,
closeTagStart: -1,
depth: el.depth,
};
return analyzeStartTag(partial, text, offset);
}
function analyzeStartTag( function analyzeStartTag(
el: XmlElement, el: XmlElement,
text: string, text: string,
@@ -83,7 +128,7 @@ function analyzeStartTag(
attr.hasValue && attr.hasValue &&
attr.quoteStart >= 0 && attr.quoteStart >= 0 &&
offset >= attr.quoteStart && offset >= attr.quoteStart &&
(attr.quoteEnd < 0 || offset <= attr.quoteEnd) (attr.quoteEnd < 0 || offset < attr.quoteEnd)
) { ) {
const start = attr.valueStart; const start = attr.valueStart;
const prefix = offset > start ? text.slice(start, offset) : ""; const prefix = offset > start ? text.slice(start, offset) : "";
+98 -2
View File
@@ -43,6 +43,12 @@ export interface XmlElement {
/** Offset of "</" of the closing tag, or -1 when self-closing. */ /** Offset of "</" of the closing tag, or -1 when self-closing. */
closeTagStart: number; closeTagStart: number;
depth: number; depth: number;
/**
* True when the start tag was unterminated and recovered at a line break.
* Attributes typed on later lines of the same tag are not part of the
* parsed element; completion re-parses the partial tag up to the cursor.
*/
recoveredStartTag?: boolean;
} }
export interface XmlParseError { export interface XmlParseError {
@@ -66,6 +72,14 @@ export interface Position {
character: number; character: number;
} }
/**
* Removes a leading UTF-8 byte-order mark (U+FEFF) so source offsets match
* the text as editors expose it (VS Code strips the BOM from document text).
*/
export function stripBom(text: string): string {
return text.charCodeAt(0) === 0xfeff ? text.slice(1) : text;
}
/** Precomputes line start offsets for offset <-> position conversion. */ /** Precomputes line start offsets for offset <-> position conversion. */
export class LineMap { export class LineMap {
private lineStarts: number[] = [0]; private lineStarts: number[] = [0];
@@ -111,7 +125,7 @@ interface RawTag {
const NAME_RE = /[A-Za-z_][\w:.-]*/y; const NAME_RE = /[A-Za-z_][\w:.-]*/y;
function parseTag(content: string, contentStart: number): RawTag { export function parseTag(content: string, contentStart: number): RawTag {
const base = contentStart; const base = contentStart;
let j = 0; let j = 0;
while (j < content.length && /\s/.test(content[j])) { while (j < content.length && /\s/.test(content[j])) {
@@ -349,8 +363,15 @@ export function parseXml(text: string): XmlDocument {
const raw = parseTag(content, i + 1); const raw = parseTag(content, i + 1);
if (raw.name) { if (raw.name) {
const el = buildElement(raw, stack.length); const el = buildElement(raw, stack.length);
el.recoveredStartTag = true;
elements.push(el); elements.push(el);
if (stack.length === 0) {
root = root ?? el; root = root ?? el;
} else {
const parent = stack[stack.length - 1];
parent.children.push(el);
el.parent = parent;
}
stack.push(el); stack.push(el);
} }
i = recoverTo + 1; i = recoverTo + 1;
@@ -399,6 +420,13 @@ function findTagEnd(text: string, from: number): number {
if (c === quote) quote = null; if (c === quote) quote = null;
} else if (c === '"' || c === "'") { } else if (c === '"' || c === "'") {
quote = c; quote = c;
} else if (c === "<") {
// A new tag start before the current tag's ">" means the ">" we would
// find later belongs to that other tag (typically a closing tag after
// a just-typed "<" in element content). Treat the current tag as
// unterminated so the parser recovers at the line break: the context
// stays "content" and the completion range can cover the typed "<".
return -1;
} else if (c === ">") { } else if (c === ">") {
return i; return i;
} }
@@ -427,7 +455,7 @@ export function findElementAt(doc: XmlDocument, offset: number): XmlElement | nu
let best: XmlElement | null = null; let best: XmlElement | null = null;
for (const el of doc.elements) { for (const el of doc.elements) {
if (el.end < 0) continue; if (el.end < 0) continue;
if (offset >= el.start && offset <= el.end) { if (elementContainsOffset(el, offset)) {
if (!best || el.depth > best.depth) { if (!best || el.depth > best.depth) {
best = el; best = el;
} }
@@ -436,6 +464,74 @@ export function findElementAt(doc: XmlDocument, offset: number): XmlElement | nu
return best; return best;
} }
/**
* Whether `offset` belongs to an element's span.
*
* The end offset is exclusive for a completed element (closing tag or
* self-closing tag): a cursor right after `</Name>` belongs to the parent's
* content, not the child. The one exception is an unclosed element whose
* parser-recovered `end` is the document end: a cursor at EOF is still
* inside the element being typed.
*/
export function elementContainsOffset(el: XmlElement, offset: number): boolean {
if (offset < el.start) return false;
if (offset < el.end) return true;
if (offset > el.end) return false;
return !el.selfClosing && el.closeTagStart < 0;
}
export interface TextToken {
value: string;
/** Absolute offset of the first character of the token. */
start: number;
/** Absolute offset one past the last character of the token. */
end: number;
}
/**
* Returns the whitespace-delimited text token inside an element's content
* that contains `offset`, with absolute source offsets. Used for
* simple-content elements (e.g. `<CreateObject>CrateDebris_01</CreateObject>`)
* by completion, hover, navigation and diagnostics. Returns null when the
* offset is not inside text content (start tag, closing tag, self-closing).
*/
export function textContentTokenAt(
text: string,
el: XmlElement,
offset: number,
): TextToken | null {
if (el.selfClosing) return null;
const contentEnd = el.closeTagStart >= 0 ? el.closeTagStart : el.end;
if (contentEnd <= el.startTagEnd) return null;
if (offset <= el.startTagEnd || offset > contentEnd) return null;
const contentStart = el.startTagEnd;
// A cursor right before the closing tag is still inside the content; clamp
// the relative position to the content length in that case.
const rel = Math.min(offset - contentStart, contentEnd - contentStart);
let tokenStart = rel;
while (tokenStart > 0 && !/\s/.test(text[contentStart + tokenStart - 1])) {
tokenStart--;
}
let tokenEnd = rel;
while (
tokenEnd < contentEnd - contentStart &&
!/\s/.test(text[contentStart + tokenEnd])
) {
tokenEnd++;
}
// The cursor may sit on trailing whitespace or at the closing-tag
// boundary; trim whitespace so the token is exactly the value word.
while (tokenEnd > tokenStart && /\s/.test(text[contentStart + tokenEnd - 1])) {
tokenEnd--;
}
if (tokenEnd <= tokenStart) return null;
return {
value: text.slice(contentStart + tokenStart, contentStart + tokenEnd),
start: contentStart + tokenStart,
end: contentStart + tokenEnd,
};
}
/** Finds an element by name that contains the offset (including its start tag). */ /** Finds an element by name that contains the offset (including its start tag). */
export function findOpenTagElementAt(doc: XmlDocument, offset: number): XmlElement | null { export function findOpenTagElementAt(doc: XmlDocument, offset: number): XmlElement | null {
const el = findElementAt(doc, offset); const el = findElementAt(doc, offset);
+512 -19
View File
@@ -1,9 +1,35 @@
import * as vscode from "vscode"; import * as vscode from "vscode";
import { existsSync } from "node:fs"; import { existsSync } from "node:fs";
import { join, dirname } from "node:path"; import { readFile, stat } from "node:fs/promises";
import { CachedDirectoryWalker } from "./indexer/fileScanner"; import { join, dirname, resolve } from "node:path";
import {
CachedDirectoryWalker,
isContentRelevantPath,
isWatcherNoisePath,
} from "./indexer/fileScanner";
import { ModIndexer } from "./indexer/indexer"; import { ModIndexer } from "./indexer/indexer";
import type { ModIndex } from "./indexer/types"; import {
DocumentCache,
IncludeResolveCache,
IndexRecordsCache,
InvalidationsEpoch,
normKey,
} from "./indexer/caches";
import {
DiskRecordsCache,
diskCacheKey,
type DiskCacheIdentity,
type DiskCacheLoadStats,
} from "./indexer/diskCache";
import { buildSearchPaths, type SearchPaths } from "./indexer/includeResolver";
import { extractIndexRecords } from "./indexer/records";
import { LineMap, parseXml, stripBom } from "./language/xmlParser";
import {
buildDocumentScope,
withLocalOverlay,
type DocumentScope,
} from "./indexer/localScope";
import type { ModIndex, ParsedFile } from "./indexer/types";
import { readSettings, type ExtensionSettings } from "./settings"; import { readSettings, type ExtensionSettings } from "./settings";
const REBUILD_DEBOUNCE_MS = 1500; const REBUILD_DEBOUNCE_MS = 1500;
@@ -15,13 +41,60 @@ export class ModWorkspace {
settings: ExtensionSettings; settings: ExtensionSettings;
private walker = new CachedDirectoryWalker(); private walker = new CachedDirectoryWalker();
// Caches owned here survive rebuilds: a fresh ModIndexer reuses them and
// only re-reads files whose stat changed (crucial for the ~2.6 GB of .w3x
// art assets in a project like Corona).
private documentCache = new DocumentCache();
private recordsCache = new IndexRecordsCache();
private resolveCache = new IncludeResolveCache();
/**
* Monotonic invalidation counter. A build captures the epoch when it
* starts; snapshots published after any invalidation are marked stale so
* features can tell users "this index may be slightly out of date" while
* the follow-up rebuild converges.
*/
private epoch = new InvalidationsEpoch();
/** On-disk records cache (cold-start acceleration). */
private diskCachePath: string | null = null;
private diskCache: DiskRecordsCache | null = null;
private diskCacheStats: DiskCacheLoadStats = {
fileExists: false,
keyMatched: false,
loaded: 0,
validated: 0,
dropped: 0,
};
private diskSaved = false;
private saving: Promise<void> | null = null;
/** Document-local scopes (parse + expanded tree + overlay), per open doc. */
private localScopes = new Map<
string,
{ version: number; indexEpoch: number; scope: DocumentScope }
>();
private localScopeBuilds = new Map<string, Promise<DocumentScope>>();
private indexEpochValue = 0;
/** Diagnostics: how many builds ran and what triggered the last one. */
private buildCountValue = 0;
private lastBuildTrigger = "initial";
private pendingTrigger: string | null = null;
private output: vscode.OutputChannel;
/** Called whenever a new index snapshot is published (phase or final). */
onIndexUpdate?: () => void;
private context: vscode.ExtensionContext;
private watchers: vscode.FileSystemWatcher[] = [];
private statusBar: vscode.StatusBarItem; private statusBar: vscode.StatusBarItem;
private rebuildTimer: ReturnType<typeof setTimeout> | null = null; private rebuildTimer: ReturnType<typeof setTimeout> | null = null;
private building = false; private building = false;
private dirty = false; private dirty = false;
constructor(context: vscode.ExtensionContext) { constructor(context: vscode.ExtensionContext) {
this.context = context;
this.settings = readSettings(); this.settings = readSettings();
const storageUri = context.storageUri ?? context.globalStorageUri;
if (storageUri) {
this.diskCachePath = join(storageUri.fsPath, "index-records-v1.json.gz");
}
this.output = vscode.window.createOutputChannel("RA3 Mod XML");
this.statusBar = vscode.window.createStatusBarItem( this.statusBar = vscode.window.createStatusBarItem(
vscode.StatusBarAlignment.Left, vscode.StatusBarAlignment.Left,
100, 100,
@@ -29,12 +102,37 @@ export class ModWorkspace {
this.statusBar.name = "RA3 Mod XML"; this.statusBar.name = "RA3 Mod XML";
this.statusBar.command = "ra3modxml.openIndexReport"; this.statusBar.command = "ra3modxml.openIndexReport";
context.subscriptions.push(this.statusBar); context.subscriptions.push(this.statusBar);
context.subscriptions.push(
vscode.workspace.onDidCloseTextDocument((document) => {
this.localScopes.delete(document.uri.toString());
}),
);
} }
isRa3Workspace(): boolean { isRa3Workspace(): boolean {
return this.projectRoot != null; return this.projectRoot != null;
} }
/** Appends a line to the "RA3 Mod XML" output channel (debug/troubleshooting). */
log(message: string): void {
this.output.appendLine(message);
}
/** True while a rebuild is running (before any snapshot is published). */
get isBuilding(): boolean {
return this.building;
}
/** Number of index builds performed in this session. */
get buildCount(): number {
return this.buildCountValue;
}
/** Why the last build started ("initial", "save", "watcher-*", ...). */
get lastTrigger(): string {
return this.lastBuildTrigger;
}
detectProjectRoot(): string | null { detectProjectRoot(): string | null {
const folders = vscode.workspace.workspaceFolders; const folders = vscode.workspace.workspaceFolders;
if (!folders?.length) return null; if (!folders?.length) return null;
@@ -51,28 +149,125 @@ export class ModWorkspace {
this.statusBar.hide(); this.statusBar.hide();
return; return;
} }
this.startWatching();
this.statusBar.text = "$(sync~spin) RA3 XML: indexing…"; this.statusBar.text = "$(sync~spin) RA3 XML: indexing…";
this.statusBar.show(); this.statusBar.show();
await this.rebuild(); await this.rebuild(false, "initial");
} }
scheduleRebuild(): void { /**
* Invalidates cached documents for a path (called by the file watcher and
* on document save), so the next rebuild re-reads it instead of trusting
* the cached copy.
*/
invalidate(path: string): void {
if (!path) return;
this.epoch.mark();
this.documentCache.invalidate(path);
this.recordsCache.invalidate(path);
}
/**
* Called when files are created or deleted: include-resolution results
* (which encode file existence) are no longer trustworthy.
*/
invalidateExistence(): void {
this.epoch.mark();
this.resolveCache.clear();
}
/**
* Watches the project, SDK and extra DATA roots for file changes and
* invalidates the corresponding cache entries. With caches invalidated
* precisely, rebuilds can trust every other cached file and skip per-file
* stats (huge win on mechanical drives; Corona rebuild dropped from ~38s
* to a few seconds).
*/
private startWatching(): void {
if (!this.projectRoot) return; if (!this.projectRoot) return;
const roots = new Set([
this.projectRoot,
this.settings.sdkPath,
...this.settings.additionalDataSearchPaths,
]);
for (const root of roots) {
if (!existsSync(root)) continue;
try {
const watcher = vscode.workspace.createFileSystemWatcher(
new vscode.RelativePattern(root, "**/*"),
);
watcher.onDidCreate((uri) => {
if (isWatcherNoisePath(uri.fsPath)) return;
this.output.appendLine(`[watcher-create] ${uri.fsPath}`);
this.invalidate(uri.fsPath);
this.invalidateExistence();
this.scheduleRebuild("watcher-create");
});
watcher.onDidChange((uri) => {
if (isWatcherNoisePath(uri.fsPath)) return;
// Content changes only matter for files that can change index
// records (XML-ish documents); textures/binary art changes do not.
if (
!isContentRelevantPath(uri.fsPath) &&
!this.isIndexedPath(uri.fsPath)
) {
return;
}
this.output.appendLine(`[watcher-change] ${uri.fsPath}`);
this.invalidate(uri.fsPath);
this.scheduleRebuild("watcher-change");
});
watcher.onDidDelete((uri) => {
if (isWatcherNoisePath(uri.fsPath)) return;
this.output.appendLine(`[watcher-delete] ${uri.fsPath}`);
this.invalidate(uri.fsPath);
this.invalidateExistence();
this.scheduleRebuild("watcher-delete");
});
this.watchers.push(watcher);
this.context.subscriptions.push(watcher);
} catch {
// The root may be temporarily unavailable (e.g. removable drive);
// indexing still works, just without watcher-based invalidation.
}
}
}
/** True when the path is part of the current index (any build state). */
private isIndexedPath(fsPath: string): boolean {
if (this.indexer?.isIndexedFile(fsPath)) return true;
return this.index?.files.has(normKey(fsPath)) ?? false;
}
scheduleRebuild(reason = "unknown"): void {
if (!this.projectRoot) return;
this.pendingTrigger = reason;
if (this.rebuildTimer) clearTimeout(this.rebuildTimer); if (this.rebuildTimer) clearTimeout(this.rebuildTimer);
this.rebuildTimer = setTimeout(() => { this.rebuildTimer = setTimeout(() => {
void this.rebuild(); void this.rebuild(false, this.pendingTrigger ?? reason);
}, REBUILD_DEBOUNCE_MS); }, REBUILD_DEBOUNCE_MS);
} }
async rebuild(): Promise<void> { async rebuild(force = false, trigger = "unknown"): Promise<void> {
if (!this.projectRoot) return; if (!this.projectRoot) return;
if (this.building) { if (this.building) {
this.dirty = true; this.dirty = true;
return; return;
} }
if (force) this.resolveCache.clear();
this.building = true; this.building = true;
this.buildCountValue++;
this.lastBuildTrigger = trigger;
this.output.appendLine(
`[build #${this.buildCountValue}] trigger=${trigger} force=${force} start=${new Date().toISOString()}`,
);
this.settings = readSettings(); this.settings = readSettings();
const epochAtStart = this.epoch.snapshot();
try { try {
// Cold start: seed the records cache from disk (stat-validated) so a
// fresh session does not re-read unchanged files (Corona: 2.6 GB of
// art assets) just because the in-memory caches are empty.
await this.seedRecordsFromDisk();
this.statusBar.text = "$(sync~spin) RA3 XML: indexing…"; this.statusBar.text = "$(sync~spin) RA3 XML: indexing…";
const indexer = new ModIndexer({ const indexer = new ModIndexer({
projectDir: this.projectRoot, projectDir: this.projectRoot,
@@ -81,31 +276,327 @@ export class ModWorkspace {
indexSageXml: this.settings.indexSageXml, indexSageXml: this.settings.indexSageXml,
additionalDataSearchPaths: this.settings.additionalDataSearchPaths, additionalDataSearchPaths: this.settings.additionalDataSearchPaths,
walker: this.walker, walker: this.walker,
documentCache: this.documentCache,
recordsCache: this.recordsCache,
resolveCache: this.resolveCache,
// Trust cache entries unless the user explicitly asked for a full
// verification (ra3modxml.reindex).
trustUnchanged: !force,
}); });
const started = Date.now();
this.index = await indexer.build();
this.indexer = indexer; this.indexer = indexer;
const secs = ((Date.now() - started) / 1000).toFixed(1); // The XML phase is published as soon as it is ready, so completion /
const s = this.index.stats; // navigation / diagnostics work while the art scan continues.
this.statusBar.text = `$(symbol-misc) RA3 XML: ${formatCount(s.assetCount)} assets`; const finalIndex = await indexer.build((phaseIndex) => {
this.statusBar.tooltip = this.publishIndex(phaseIndex, epochAtStart);
`${s.projectDir}\n` + });
`${s.indexedFiles} files indexed (${secs}s)\n` + this.publishIndex(finalIndex, epochAtStart);
`${s.assetCount} assets (${s.manifestAssetCount} from ${s.manifestFiles} manifests)\n` + this.output.appendLine(
`${s.defineCount} defines, ${s.streams} streams, ${s.sourceCandidates} include candidates`; `[build #${this.buildCountValue}] done in ${(finalIndex.stats.elapsedMs / 1000).toFixed(1)}s (phase=${finalIndex.phase}, assets=${finalIndex.stats.assetCount}, stale=${finalIndex.stale === true}, walk=${(finalIndex.stats.walkMs / 1000).toFixed(1)}s, candidates=${(finalIndex.stats.candidatesMs / 1000).toFixed(1)}s, art=${(finalIndex.stats.artScanMs / 1000).toFixed(1)}s)`,
);
this.saveRecordsToDisk();
} catch (err) { } catch (err) {
this.index = null; if (this.index) {
// Keep the last good snapshot (marked stale) instead of disabling the
// extension entirely; a later rebuild can recover.
this.index.stale = true;
this.statusBar.text = "$(error) RA3 XML: indexing failed (stale index kept)";
this.statusBar.tooltip = err instanceof Error ? err.message : String(err);
this.onIndexUpdate?.();
} else {
this.statusBar.text = "$(error) RA3 XML: indexing failed"; this.statusBar.text = "$(error) RA3 XML: indexing failed";
this.statusBar.tooltip = err instanceof Error ? err.message : String(err); this.statusBar.tooltip = err instanceof Error ? err.message : String(err);
}
} finally { } finally {
this.building = false; this.building = false;
if (this.dirty) { if (this.dirty) {
this.dirty = false; this.dirty = false;
void this.rebuild(); void this.rebuild(false, `dirty-followup (${this.lastBuildTrigger})`);
} else if (this.index) {
// Build is fully over: refresh diagnostics with full local scopes
// (the snapshot published while `building` was still true only got
// cheap parse-only scopes).
this.onIndexUpdate?.();
} }
} }
} }
private diskCacheIdentity(): DiskCacheIdentity | null {
if (!this.projectRoot) return null;
return {
projectDir: this.projectRoot,
sdkDir: this.settings.sdkPath,
indexSageXml: this.settings.indexSageXml,
additionalDataSearchPaths: this.settings.additionalDataSearchPaths,
builtmodsDirs: this.settings.builtmodsDirs,
};
}
/** Loads + stat-validates the disk cache into the records cache once. */
private async seedRecordsFromDisk(): Promise<void> {
if (this.recordsCache.size > 0) return;
const identity = this.diskCacheIdentity();
if (!this.diskCachePath || !identity) return;
this.diskCache = new DiskRecordsCache(this.diskCachePath, identity);
this.statusBar.text = "$(sync~spin) RA3 XML: validating cache…";
const { records, stats } = await this.diskCache.loadValidated();
this.diskCacheStats = stats;
this.diskSaved = false;
for (const rec of records) {
this.recordsCache.set(rec.key, {
stat: rec.stat,
records: rec.records,
kind: rec.kind,
});
}
}
/** Persists the records cache after a successful build (best-effort). */
private saveRecordsToDisk(): void {
if (!this.diskCache) return;
// Snapshot the entries now: the save runs in the background while the
// next rebuild may already be mutating the live cache.
const entries = [...this.recordsCache.entries()];
const prev = this.saving ?? Promise.resolve();
this.saving = prev
.then(async () => {
await this.diskCache!.save(entries);
this.diskSaved = true;
})
.catch(() => {
// Disk persistence is best-effort; the in-memory cache still works.
});
}
/**
* Clears every cache (in-memory + disk + directory walker) and starts a
* full forced rebuild. Used by the `ra3modxml.clearCache` command.
*/
clearCaches(): void {
this.localScopes.clear();
this.documentCache.clear();
this.recordsCache.clear();
this.resolveCache.clear();
this.walker.clear();
this.diskCacheStats = {
fileExists: false,
keyMatched: false,
loaded: 0,
validated: 0,
dropped: 0,
};
this.diskSaved = false;
void this.diskCache?.clear();
void this.rebuild(true, "clear-cache");
}
/** Human-readable cache status for the `ra3modxml.showCacheReport` command. */
async cacheReport(): Promise<string> {
const lines: string[] = ["RA3 Mod XML cache report"];
lines.push(`Disk cache: ${this.diskCachePath ?? "not available"}`);
if (this.diskCachePath) {
const status = await this.diskCache?.status();
lines.push(
` file: ${status?.exists ? `${(status.sizeBytes / 1024).toFixed(1)} KB` : "missing"}`,
);
const identity = this.diskCacheIdentity();
lines.push(` identity key: ${identity ? diskCacheKey(identity) : "-"}`);
lines.push(
` last load: file=${this.diskCacheStats.fileExists} keyMatched=${this.diskCacheStats.keyMatched} loaded=${this.diskCacheStats.loaded} validated=${this.diskCacheStats.validated} dropped=${this.diskCacheStats.dropped}`,
);
lines.push(` saved after last build: ${this.diskSaved}`);
}
lines.push(
`In-memory: ${this.recordsCache.size} record entries · ${this.documentCache.size} documents (${this.documentCache.elements} elements) · ${this.resolveCache.size} include resolutions`,
);
lines.push(`Builds: #${this.buildCount} (last trigger: ${this.lastBuildTrigger})`);
if (this.index) {
const s = this.index.stats;
lines.push(
`Last build: snapshotHits=${s.snapshotHits} snapshotFallbacks=${s.snapshotFallbacks} recordsCacheHits=${s.recordsCacheHits} shallowCacheHits=${s.shallowCacheHits}`,
);
}
return lines.join("\n");
}
/**
* Publishes an index snapshot (intermediate phase or final). If any file
* was invalidated while the snapshot was being built, it is marked stale;
* the dirty/rebuild mechanism converges shortly after.
*/
private publishIndex(index: ModIndex, epochAtStart: number): void {
if (this.epoch.changedSince(epochAtStart)) index.stale = true;
this.index = index;
this.indexEpochValue++;
// The merged index attached to a document scope changes with every
// published snapshot, so cached scopes are rebuilt lazily on next use.
this.localScopes.clear();
this.updateStatusBar(index);
this.onIndexUpdate?.();
if (!index.complete) {
this.output.appendLine(
`[build #${this.buildCountValue}] phase A published in ${(index.stats.elapsedMs / 1000).toFixed(1)}s (${index.stats.assetCount} assets, ${index.stats.deferredArtFiles} art files pending)`,
);
}
}
private updateStatusBar(idx: ModIndex): void {
const s = idx.stats;
const stale = idx.stale ? " (stale)" : "";
if (!idx.complete) {
this.statusBar.text = `$(sync~spin) RA3 XML: XML indexed, scanning art…${stale}`;
} else {
this.statusBar.text = `$(symbol-misc) RA3 XML: ${formatCount(s.assetCount)} assets${stale}`;
}
this.statusBar.tooltip =
`${s.projectDir}\n` +
`${s.indexedFiles} files indexed (${s.parsedFiles} parsed, ${s.shallowScannedFiles} art assets shallow-scanned, ${(s.elapsedMs / 1000).toFixed(1)}s)\n` +
`${s.assetCount} assets (${s.manifestAssetCount} from ${s.manifestFiles} manifests)\n` +
`${s.defineCount} defines, ${s.streams} streams, ${s.sourceCandidates} include candidates\n` +
`Phase: ${s.phase} · Complete: ${s.complete}${stale}`;
}
/**
* Search paths derived from the current settings, usable even before the
* first index snapshot exists (include links / hover / diagnostics).
*/
searchPaths(): SearchPaths | null {
if (!this.projectRoot) return null;
return buildSearchPaths(this.settings.sdkPath, this.projectRoot);
}
/**
* Returns the document scope for the current text: original parse, expanded
* logical tree, local overlay and overlay-aware merged index. Cached by
* URI + document version + global index epoch.
*/
async getScope(document: vscode.TextDocument): Promise<DocumentScope> {
const key = document.uri.toString();
const cached = this.localScopes.get(key);
if (
cached &&
cached.version === document.version &&
cached.indexEpoch === this.indexEpochValue
) {
return cached.scope;
}
// While a rebuild is running, avoid competing with the indexer for disk
// I/O: serve a parse-only scope (current file + XSD context, no include
// chain / logical expansion). The published snapshot clears this cache,
// so the next provider call after the build gets the full local scope.
if (this.building) {
return this.buildCheapScope(document);
}
const pending = this.localScopeBuilds.get(key);
if (pending) return pending;
const versionAtStart = document.version;
const promise = this.buildScope(document)
.then((scope) => {
this.localScopes.set(key, {
version: versionAtStart,
indexEpoch: this.indexEpochValue,
scope,
});
return scope;
})
.finally(() => {
this.localScopeBuilds.delete(key);
});
this.localScopeBuilds.set(key, promise);
return promise;
}
/**
* Returns the global index with this document's local overlay attached, or
* a minimal local-only index while the global index is still building.
*/
async getIndex(document: vscode.TextDocument): Promise<ModIndex | null> {
if (!this.isRa3Workspace()) return null;
return (await this.getScope(document)).merged;
}
private async buildScope(
document: vscode.TextDocument,
): Promise<DocumentScope> {
const projectRoot = this.projectRoot;
if (!projectRoot) throw new Error("RA3 workspace root is not available");
const searchPaths =
this.searchPaths() ?? buildSearchPaths(this.settings.sdkPath, projectRoot);
const readRecords = async (path: string): Promise<ParsedFile | null> =>
this.indexer ? this.indexer.readDocument(path) : this.fallbackRead(path);
const readDom = async (path: string): Promise<ParsedFile | null> =>
this.indexer ? this.indexer.readDom(path) : this.fallbackRead(path);
const scope = await buildDocumentScope(
document.uri.fsPath,
document.getText(),
document.version,
{
projectDir: projectRoot,
sdkDir: this.settings.sdkPath,
searchPaths,
readRecords,
readDom,
},
);
scope.merged = withLocalOverlay(
this.index,
scope.overlay,
projectRoot,
this.settings.sdkPath,
);
return scope;
}
private async buildCheapScope(
document: vscode.TextDocument,
): Promise<DocumentScope> {
const projectRoot = this.projectRoot;
if (!projectRoot) throw new Error("RA3 workspace root is not available");
const searchPaths =
this.searchPaths() ?? buildSearchPaths(this.settings.sdkPath, projectRoot);
const scope = await buildDocumentScope(
document.uri.fsPath,
document.getText(),
document.version,
{
projectDir: projectRoot,
sdkDir: this.settings.sdkPath,
searchPaths,
readRecords: async () => null,
readDom: async () => null,
},
);
scope.merged = withLocalOverlay(
this.index,
scope.overlay,
projectRoot,
this.settings.sdkPath,
);
return scope;
}
/**
* Fallback used before the first ModIndexer exists (e.g. during initial
* activation): parses an XML file directly so the document-local scope can
* still follow small include chains.
*/
private async fallbackRead(path: string): Promise<ParsedFile | null> {
try {
const st = await stat(path);
if (st.size > 4 * 1024 * 1024) return null;
const text = stripBom(await readFile(path, "utf8"));
const lineMap = new LineMap(text);
const parse = parseXml(text);
return {
file: { path: resolve(path), stat: null },
parse,
records: extractIndexRecords(parse, lineMap),
lineMap,
};
} catch {
return null;
}
}
/** Parses the (possibly unsaved) in-memory text of the active document. */ /** Parses the (possibly unsaved) in-memory text of the active document. */
async parseText(path: string, text: string) { async parseText(path: string, text: string) {
const { parseXml, LineMap } = await import("./language/xmlParser"); const { parseXml, LineMap } = await import("./language/xmlParser");
@@ -113,6 +604,7 @@ export class ModWorkspace {
return { return {
file: { path, stat: null }, file: { path, stat: null },
parse, parse,
records: null,
lineMap: new LineMap(text), lineMap: new LineMap(text),
}; };
} }
@@ -120,6 +612,7 @@ export class ModWorkspace {
dispose(): void { dispose(): void {
if (this.rebuildTimer) clearTimeout(this.rebuildTimer); if (this.rebuildTimer) clearTimeout(this.rebuildTimer);
this.statusBar.dispose(); this.statusBar.dispose();
this.output.dispose();
} }
} }
+103
View File
@@ -0,0 +1,103 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import {
DocumentCache,
IncludeResolveCache,
IndexRecordsCache,
InvalidationsEpoch,
} from "../out/indexer/caches.js";
import { resolve } from "node:path";
const stat = { mtimeMs: 1, size: 1, birthtimeMs: 1, ctimeMs: 1 };
function parsed(path, elements) {
return {
file: { path, stat },
parse: { root: { name: "r" }, elements: new Array(elements), errors: [] },
records: null,
lineMap: null,
};
}
test("DocumentCache evicts the largest tree when over the element budget", () => {
const cache = new DocumentCache(64, 100);
cache.set(parsed("a.xml", 60));
cache.set(parsed("b.xml", 60));
assert.equal(cache.get("a.xml"), undefined, "largest tree evicted first");
assert.ok(cache.get("b.xml"));
});
test("DocumentCache evicts least recently used when over capacity", () => {
const cache = new DocumentCache(2, 1_000_000);
cache.set(parsed("a.xml", 10));
cache.set(parsed("b.xml", 10));
cache.set(parsed("c.xml", 10));
assert.equal(cache.get("a.xml"), undefined);
assert.ok(cache.get("b.xml"));
assert.ok(cache.get("c.xml"));
});
test("DocumentCache invalidate frees budget", () => {
const cache = new DocumentCache(64, 100);
cache.set(parsed("a.xml", 60));
cache.invalidate("a.xml");
cache.set(parsed("b.xml", 60));
assert.ok(cache.get("b.xml"), "invalidating a freed its budget share");
cache.set(parsed("c.xml", 60));
assert.equal(cache.get("a.xml"), undefined);
assert.ok(cache.get("c.xml"));
});
test("IndexRecordsCache stores and invalidates entries", () => {
const cache = new IndexRecordsCache();
const entry = {
stat,
records: { assets: [], defines: [], includes: [], rootXiIncludes: [], nestedXiIncludes: [] },
kind: "full",
};
cache.set("a.xml", entry);
assert.equal(cache.get("a.xml"), entry);
cache.invalidate("a.xml");
assert.equal(cache.get("a.xml"), undefined);
});
test("IndexRecordsCache exposes entries for disk persistence", () => {
const cache = new IndexRecordsCache();
const entry = {
stat,
records: { assets: [], defines: [], includes: [], rootXiIncludes: [], nestedXiIncludes: [] },
kind: "full",
};
cache.set("a.xml", entry);
const list = [...cache.entries()];
assert.equal(list.length, 1);
assert.equal(list[0][0], resolve("a.xml").toLowerCase());
assert.equal(list[0][1], entry);
});
test("IncludeResolveCache stores sources and manifest lookups", () => {
const cache = new IncludeResolveCache();
const key = "dir|DATA:static.xml";
const result = { path: "C:/sdk/static.xml", prefix: "DATA", raw: "DATA:static.xml" };
assert.equal(cache.get(key), undefined);
cache.set(key, result);
assert.equal(cache.get(key), result);
assert.equal(cache.getManifest("static.xml"), undefined);
cache.setManifest("static.xml", "C:/sdk/builtmods/static.manifest");
assert.equal(cache.getManifest("static.xml"), "C:/sdk/builtmods/static.manifest");
cache.clear();
assert.equal(cache.get(key), undefined);
assert.equal(cache.getManifest("static.xml"), undefined);
});
test("InvalidationsEpoch tracks changes since a snapshot", () => {
const epoch = new InvalidationsEpoch();
const before = epoch.snapshot();
assert.equal(epoch.changedSince(before), false);
epoch.mark();
assert.equal(epoch.changedSince(before), true);
assert.equal(epoch.changedSince(epoch.snapshot()), false);
epoch.mark();
epoch.mark();
assert.equal(epoch.current, 3);
});
+707 -3
View File
@@ -20,6 +20,13 @@ class CompletionItem {
} }
} }
class CompletionList {
constructor(items, isIncomplete) {
this.items = items;
this.isIncomplete = isIncomplete;
}
}
class Position { class Position {
constructor(line, character) { constructor(line, character) {
this.line = line; this.line = line;
@@ -67,6 +74,7 @@ require.cache["vscode-stub"] = {
loaded: true, loaded: true,
exports: { exports: {
CompletionItem, CompletionItem,
CompletionList,
CompletionItemKind, CompletionItemKind,
Position, Position,
Range, Range,
@@ -76,6 +84,8 @@ require.cache["vscode-stub"] = {
}; };
const { Ra3CompletionProvider } = require("../out/features/completion.js"); const { Ra3CompletionProvider } = require("../out/features/completion.js");
const { parseXml, LineMap } = require("../out/language/xmlParser.js");
const { expandDocument } = require("../out/indexer/logicalTree.js");
function makeDocument(text) { function makeDocument(text) {
const lineStarts = [0]; const lineStarts = [0];
@@ -98,11 +108,33 @@ function makeDocument(text) {
}; };
} }
// Enum completions do not consult the index, but provideCompletionItems only async function makeScope(text, idx) {
// routes value contexts when the workspace index is present. const lineMap = new LineMap(text);
const provider = new Ra3CompletionProvider({ index: {} }); const parse = parseXml(text);
const expanded = await expandDocument("test.xml", parse, {
resolve: () => null,
readDom: async () => null,
});
return { expanded, merged: idx, overlay: {} };
}
// Enum completions do not consult the index, and value contexts now work
// even before the workspace index exists.
const makeProvider = (idx) =>
new Ra3CompletionProvider({
index: idx,
isRa3Workspace: () => true,
getScope: async (document) => makeScope(document.getText(), idx),
log: () => {},
});
const provider = makeProvider({});
const providerNoIndex = makeProvider(null);
const token = { isCancellationRequested: false }; const token = { isCancellationRequested: false };
function listItems(result) {
return Array.isArray(result) ? result : result.items;
}
test("Surfaces enum completion works with an unclosed quote", async () => { test("Surfaces enum completion works with an unclosed quote", async () => {
const text = const text =
`<AssetDeclaration>\n <LocomotorTemplate id="x" Surfaces="G>\n <Other/>\n</LocomotorTemplate>\n</AssetDeclaration>`; `<AssetDeclaration>\n <LocomotorTemplate id="x" Surfaces="G>\n <Other/>\n</LocomotorTemplate>\n</AssetDeclaration>`;
@@ -142,6 +174,95 @@ test("list values filter on the token after whitespace", async () => {
assert.equal(water.range.start.character, valueStart + "GROUND ".length - line1Start); assert.equal(water.range.start.character, valueStart + "GROUND ".length - line1Start);
}); });
test("list completion after a space offers only unused flags", async () => {
const text =
`<AssetDeclaration>\n <LocomotorTemplate id="x" Surfaces="GROUND ">\n <Other/>\n</LocomotorTemplate>\n</AssetDeclaration>`;
const line1 = text.split("\n")[1];
const pos = new Position(1, line1.indexOf("GROUND ") + "GROUND ".length);
const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
const labels = items.map((i) => i.label);
assert.equal(items.length, 10);
assert.ok(labels.includes("WATER"));
assert.ok(!labels.includes("GROUND"));
// The replacement range is empty at the cursor: existing flags are kept.
const water = items.find((i) => i.label === "WATER");
assert.equal(water.range.start.character, pos.character);
assert.equal(water.range.end.character, pos.character);
});
test("inserting a flag in the middle of a list does not delete trailing flags", async () => {
const text =
`<AssetDeclaration>\n <LocomotorTemplate id="x" Surfaces="GROUND WATER">\n <Other/>\n</LocomotorTemplate>\n</AssetDeclaration>`;
const line1 = text.split("\n")[1];
// Cursor right after the space between GROUND and WATER.
const pos = new Position(1, line1.indexOf("GROUND ") + "GROUND ".length);
const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
const labels = items.map((i) => i.label);
assert.ok(labels.includes("WATER"));
assert.ok(!labels.includes("GROUND"));
const water = items.find((i) => i.label === "WATER");
assert.equal(water.insertText, "WATER");
// The range must end at the cursor, not at the end of the whole value
// (which would replace the trailing "WATER" when accepting a suggestion).
assert.equal(water.range.start.character, pos.character);
assert.equal(water.range.end.character, pos.character);
});
test("complete flag at the end of a closed value appends the remaining flags", async () => {
const text =
`<AssetDeclaration>\n <LocomotorTemplate id="x" Surfaces="GROUND">\n <Other/>\n</LocomotorTemplate>\n</AssetDeclaration>`;
const line1 = text.split("\n")[1];
const pos = new Position(1, line1.indexOf("GROUND") + "GROUND".length);
const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
const labels = items.map((i) => i.label);
assert.equal(items.length, 10);
assert.ok(!labels.includes("GROUND"));
const water = items.find((i) => i.label === "WATER");
assert.ok(water, "remaining flags are offered at the end of a complete flag");
assert.equal(water.insertText, " WATER");
assert.equal(water.range.start.character, pos.character);
assert.equal(water.range.end.character, pos.character);
});
test("prefix-extended flags keep prefix filtering instead of append mode", async () => {
const text =
`<AssetDeclaration>\n <ObjectFilter Include="CAN_ATTACK">\n <Other/>\n</ObjectFilter>\n</AssetDeclaration>`;
const line1 = text.split("\n")[1];
const pos = new Position(1, line1.indexOf("CAN_ATTACK") + "CAN_ATTACK".length);
const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
const labels = items.map((i) => i.label);
assert.ok(labels.includes("CAN_ATTACK_WALLS"));
assert.ok(labels.includes("CAN_ATTACK_STEALTHED"));
assert.ok(!labels.includes("WATER"));
const walls = items.find((i) => i.label === "CAN_ATTACK_WALLS");
assert.equal(walls.insertText, "CAN_ATTACK_WALLS");
// The range replaces the typed token instead of inserting after it.
assert.equal(walls.range.start.character, pos.character - "CAN_ATTACK".length);
assert.equal(walls.range.end.character, pos.character);
});
test("multi-line unterminated Disposition value offers remaining flags", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject\n` +
` Options="IGNORE_ALL_OBJECTS"\n` +
` Disposition="RANDOM_FORCE `;
const line4 = text.split("\n")[4];
const pos = new Position(4, line4.length);
const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
const labels = items.map((i) => i.label);
assert.ok(labels.includes("DISPOSITION_NONE"));
assert.ok(labels.includes("FLOATING"));
assert.ok(!labels.includes("RANDOM_FORCE"));
});
test("empty unterminated value offers all enum values", async () => { test("empty unterminated value offers all enum values", async () => {
const text = const text =
`<AssetDeclaration>\n <LocomotorTemplate id="x" Surfaces=">\n <Other/>\n</LocomotorTemplate>\n</AssetDeclaration>`; `<AssetDeclaration>\n <LocomotorTemplate id="x" Surfaces=">\n <Other/>\n</LocomotorTemplate>\n</AssetDeclaration>`;
@@ -155,3 +276,586 @@ test("empty unterminated value offers all enum values", async () => {
assert.ok(labels.includes("WATER")); assert.ok(labels.includes("WATER"));
assert.ok(labels.includes("CRUSHABLE_WALL")); assert.ok(labels.includes("CRUSHABLE_WALL"));
}); });
test("enum completions work without an index", async () => {
const text =
`<AssetDeclaration>\n <LocomotorTemplate id="x" Surfaces="G>\n <Other/>\n</LocomotorTemplate>\n</AssetDeclaration>`;
const line1 = text.split("\n")[1];
const pos = new Position(1, line1.indexOf("G") + 1);
const items = await providerNoIndex.provideCompletionItems(
makeDocument(text),
pos,
token,
);
const labels = items.map((i) => i.label);
assert.ok(labels.includes("GROUND"), "enum value offered without an index");
assert.ok(!labels.includes("WATER"));
});
test("element and attribute name completions work without an index", async () => {
const text = `<AssetDeclaration>\n <LocomotorTemplate `;
const line1 = text.split("\n")[1];
const pos = new Position(1, line1.length);
const items = await providerNoIndex.provideCompletionItems(
makeDocument(text),
pos,
token,
);
const labels = items.map((i) => i.label);
assert.ok(labels.includes("id"));
assert.ok(labels.includes("Surfaces"));
});
test("attribute completion after a closed quote inserts a space", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject Options="IGNORE_ALL_OBJECTS" Disposition="RANDOM_FORCE RELATIVE_ANGLE">`;
const line2 = text.split("\n")[2];
const pos = new Position(
2,
line2.indexOf('RELATIVE_ANGLE"') + 'RELATIVE_ANGLE"'.length,
);
const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
const count = items.find((i) => i.label === "Count");
assert.ok(count, "Count is a valid next attribute");
assert.equal(count.insertText.value, ' Count="1"');
// The replacement range is empty at the cursor; the space comes from the
// insert text so the attribute never glues to the closing quote.
assert.equal(count.range.start.character, pos.character);
assert.equal(count.range.end.character, pos.character);
});
test("attribute completion on one-per-line elements adds a newline", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject\n` +
` Options="IGNORE_ALL_OBJECTS"\n` +
` Disposition="RANDOM_FORCE RELATIVE_ANGLE">`;
const line4 = text.split("\n")[4];
const pos = new Position(
4,
line4.indexOf('RELATIVE_ANGLE"') + 'RELATIVE_ANGLE"'.length,
);
const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
const count = items.find((i) => i.label === "Count");
assert.ok(count);
// The editor adds the new line's base indentation itself; embedding our
// own indent here would be added on top of it and compound line by line.
assert.equal(count.insertText.value, '\nCount="1"');
assert.equal(count.range.start.character, pos.character);
assert.equal(count.range.end.character, pos.character);
});
test("attribute completion on a new line aligns with the neighbor indent", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject\n` +
` Options="IGNORE_ALL_OBJECTS"\n` +
` Disposition="RANDOM_FORCE RELATIVE_ANGLE"\n` +
` `;
const pos = new Position(5, 6);
const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
const count = items.find((i) => i.label === "Count");
assert.ok(count);
assert.equal(count.insertText.value, ' Count="1"');
// The range replaces the whitespace already typed on the new line.
assert.equal(count.range.start.character, 0);
assert.equal(count.range.end.character, 6);
});
test("a half-typed attribute on a new line does not drive the indent", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject\n` +
` Options="IGNORE_ALL_OBJECTS"\n` +
` Disposition="RANDOM_FORCE RELATIVE_ANGLE"\n` +
` Count="1"\n` +
` C`;
// The "C" line carries a large editor auto-indent (20 spaces) that must
// NOT become the anchor for the completed attribute.
const pos = new Position(6, 21);
const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
const createFX = items.find((i) => i.label === "CreateFX");
assert.ok(createFX);
assert.equal(createFX.insertText.value, ' CreateFX="$1"');
// The range covers the auto-indented whitespace and the typed "C".
assert.equal(createFX.range.start.character, 0);
assert.equal(createFX.range.end.character, 21);
});
test("whitespace used to trigger the popup is consumed on newline insert", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject\n` +
` Options="IGNORE_ALL_OBJECTS"\n` +
` Disposition="RANDOM_FORCE RELATIVE_ANGLE" >`;
const line4 = text.split("\n")[4];
const pos = new Position(
4,
line4.indexOf('RELATIVE_ANGLE" ') + 'RELATIVE_ANGLE" '.length,
);
const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
const count = items.find((i) => i.label === "Count");
assert.ok(count);
assert.equal(count.insertText.value, '\nCount="1"');
// The range starts at the previous attribute's closing quote, so the
// typed space is replaced by the newline instead of lingering.
assert.equal(count.range.start.character, pos.character - 1);
assert.equal(count.range.end.character, pos.character);
});
test("scalar attributes get typed default values, suggestion attributes keep $1", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject `;
const pos = new Position(2, text.split("\n")[2].length);
const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
const item = (label) => items.find((i) => i.label === label);
// XSD default wins.
assert.equal(item("Count").insertText.value, 'Count="1"');
// Type-based examples for scalars without an XSD default.
assert.equal(item("FadeTime").insertText.value, 'FadeTime="0s"');
assert.equal(item("DispositionAngle").insertText.value, 'DispositionAngle="0d"');
// Suggestion-driven values keep the $1 placeholder and re-trigger suggest.
assert.equal(item("CreateFX").insertText.value, 'CreateFX="$1"');
assert.ok(item("CreateFX").command, "reference values re-trigger suggest");
assert.equal(item("Options").insertText.value, 'Options="$1"');
assert.ok(item("Options").command, "list values re-trigger suggest");
assert.equal(item("DisabledWhileBusy").insertText.value, 'DisabledWhileBusy="$1"');
assert.ok(item("DisabledWhileBusy").command, "boolean values re-trigger suggest");
// Concrete defaults do not pop an empty suggest widget.
assert.equal(item("Count").command, undefined);
});
test("content (child element) completions work without an index", async () => {
const text =
`<AssetDeclaration>\n <LocomotorTemplate id="x">\n \n </LocomotorTemplate>\n</AssetDeclaration>`;
const pos = new Position(2, 4);
const items = await providerNoIndex.provideCompletionItems(
makeDocument(text),
pos,
token,
);
const labels = items.map((i) => i.label);
assert.ok(labels.length > 0, "child elements offered without an index");
});
test("Poid attributes offer ids from the enclosing GameObject's local scope", async () => {
const text =
`<AssetDeclaration>\n` +
` <GameObject id="Tank">\n` +
` <Draws>\n` +
` <TruckDraw id="ModuleTag_Draw" />\n` +
` </Draws>\n` +
` <BehaviorModules>\n` +
` <ReconstituteStateSpecialAbility UpdateModuleId="ModuleTag_D">\n` +
` </BehaviorModules>\n` +
` </GameObject>\n` +
`</AssetDeclaration>`;
const line6 = text.split("\n")[6];
const pos = new Position(6, line6.indexOf("ModuleTag_D") + "ModuleTag_D".length);
const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
const labels = items.map((i) => i.label);
assert.ok(labels.includes("ModuleTag_Draw"));
assert.ok(items.every((i) => i.kind === CompletionItemKind.Value));
assert.ok(
items.every((i) => i.detail === "local module"),
"Poid completions are labelled as local-scope ids",
);
});
test("accepting a child element after a typed < replaces the < instead of doubling it", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject\n` +
` Options="IGNORE_ALL_OBJECTS"\n` +
` Disposition="RANDOM_FORCE RELATIVE_ANGLE ABSOLUTE_ANGLE"\n` +
` MinForceMagnitude="2.0"\n` +
` MaxForceMagnitude="7.0"\n` +
` DispositionIntensity="5.0"\n` +
` MinLifetime="1.0s"\n` +
` MaxLifetime="3.s"\n` +
` MinForcePitch="90d"\n` +
` MaxForcePitch="75d">\n` +
` <Offset x="26.13" y="4.87" z="15.99"></Offset>\n` +
` <`;
const lines = text.split("\n");
const last = lines.length - 1;
const pos = new Position(last, lines[last].length);
const document = makeDocument(text);
const items = await providerNoIndex.provideCompletionItems(document, pos, token);
const createObject = items.find((i) => i.label === "CreateObject");
const offset = items.find((i) => i.label === "Offset");
assert.ok(createObject);
assert.ok(offset);
// The typed "<" stays in the document; the range covers only the name
// area after it (empty here), and the snippet has no leading "<", so the
// final document never contains "<<" and the filter prefix is not "<".
const ltOffset = document.offsetAt(createObject.range.start);
assert.equal(ltOffset, document.offsetAt(pos));
assert.equal(createObject.range.end.line, last);
assert.equal(createObject.range.end.character, lines[last].length);
assert.equal(offset.range.start.character, createObject.range.start.character);
// Simple-content children must be open/close pairs with a value
// placeholder, never a self-closing tag, and re-trigger value suggest.
assert.equal(createObject.insertText.value, "CreateObject>$1</CreateObject>");
assert.ok(createObject.command, "simple-content child re-triggers suggest");
const applied =
text.slice(0, document.offsetAt(createObject.range.start)) +
createObject.insertText.value +
text.slice(document.offsetAt(createObject.range.end));
assert.ok(!applied.includes("<<"), "no doubled angle bracket after accepting");
assert.match(
applied.split("\n")[last],
/<CreateObject>\$1<\/CreateObject>/,
);
});
test("no << when a closing tag follows the typed < (real file shape)", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject\n` +
` Options="IGNORE_ALL_OBJECTS"\n` +
` Disposition="RANDOM_FORCE RELATIVE_ANGLE ABSOLUTE_ANGLE">\n` +
` <Offset x="26.13" y="4.87" z="15.99"></Offset>\n` +
` <\n` +
` </CreateObject>\n` +
` </ObjectCreationList>\n` +
`</AssetDeclaration>`;
const lines = text.split("\n");
const last = 6;
const pos = new Position(last, lines[last].length);
const document = makeDocument(text);
const items = await providerNoIndex.provideCompletionItems(document, pos, token);
const createObject = items.find((i) => i.label === "CreateObject");
assert.ok(createObject, "child element still offered after a lone <");
assert.equal(document.offsetAt(createObject.range.start), document.offsetAt(pos));
assert.equal(createObject.insertText.value, "CreateObject>$1</CreateObject>");
const applied =
text.slice(0, document.offsetAt(createObject.range.start)) +
createObject.insertText.value +
text.slice(document.offsetAt(createObject.range.end));
assert.ok(!applied.includes("<<"), "no doubled angle bracket");
const appliedLines = applied.split("\n");
assert.match(appliedLines[6], /<CreateObject>\$1<\/CreateObject>/);
});
test("no << when a partial child name was typed before the closing tag", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject>\n` +
` <Cr\n` +
` </CreateObject>\n` +
` </ObjectCreationList>\n` +
`</AssetDeclaration>`;
const lines = text.split("\n");
const last = 3;
const pos = new Position(last, lines[last].length);
const document = makeDocument(text);
const items = await providerNoIndex.provideCompletionItems(document, pos, token);
const createObject = items.find((i) => i.label === "CreateObject");
assert.ok(createObject);
// The "<" stays; the range covers only the typed partial name "Cr", and
// the snippet has no leading "<".
const startOffset = document.offsetAt(createObject.range.start);
assert.equal(text[startOffset], "C");
assert.equal(createObject.insertText.value, "CreateObject>$1</CreateObject>");
const applied =
text.slice(0, startOffset) +
createObject.insertText.value +
text.slice(document.offsetAt(createObject.range.end));
assert.ok(!applied.includes("<<"), "no doubled angle bracket for partial names");
assert.match(applied.split("\n")[last], /<CreateObject>\$1<\/CreateObject>/);
});
test("content completion without a typed < inserts the full tag", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject>\n` +
` \n` +
` </CreateObject>\n` +
` </ObjectCreationList>\n` +
`</AssetDeclaration>`;
const pos = new Position(3, 6);
const items = await providerNoIndex.provideCompletionItems(
makeDocument(text),
pos,
token,
);
const createObject = items.find((i) => i.label === "CreateObject");
assert.ok(createObject);
// No "<" was typed: the snippet includes the opening bracket and the
// replacement range is empty at the cursor.
assert.equal(createObject.insertText.value, "<CreateObject>$1</CreateObject>");
assert.equal(createObject.range.start.character, 6);
assert.equal(createObject.range.end.character, 6);
});
test("simple-content element offers typed asset ids as the text value", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject>\n` +
` <CreateObject>C</CreateObject>\n` +
` </CreateObject>\n` +
` </ObjectCreationList>\n` +
`</AssetDeclaration>`;
const line = text.split("\n")[3];
const pos = new Position(3, line.indexOf(">C") + 2);
const go = {
type: "GameObject",
id: "CrateDebris_01",
file: "Crates.xml",
line: 1,
origin: "project",
};
const weapon = {
type: "WeaponTemplate",
id: "CrateWeapon_01",
file: "Weapons.xml",
line: 1,
origin: "project",
};
const idx = {
assets: new Map([
["GameObject", new Map([["cratedebris_01", [go]]])],
["WeaponTemplate", new Map([["crateweapon_01", [weapon]]])],
]),
assetsById: new Map([
["cratedebris_01", [go]],
["crateweapon_01", [weapon]],
]),
};
const items = await makeProvider(idx).provideCompletionItems(
makeDocument(text),
pos,
token,
);
const labels = items.map((i) => i.label);
assert.ok(labels.includes("CrateDebris_01"));
assert.ok(
!labels.includes("CrateWeapon_01"),
"content refs are filtered by the element's refType (GameObject)",
);
const item = items.find((i) => i.label === "CrateDebris_01");
assert.equal(item.range.start.character, line.indexOf(">C") + 1);
assert.equal(item.range.end.character, pos.character);
assert.equal(item.insertText, "CrateDebris_01");
});
test("simple-content value completion works before the closing tag is typed", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject>\n` +
` <CreateObject>C`;
const line = text.split("\n")[3];
const pos = new Position(3, line.length);
const go = {
type: "GameObject",
id: "CrateDebris_01",
file: "Crates.xml",
line: 1,
origin: "project",
};
const idx = {
assets: new Map([["GameObject", new Map([["cratedebris_01", [go]]])]]),
assetsById: new Map([["cratedebris_01", [go]]]),
};
const items = await makeProvider(idx).provideCompletionItems(
makeDocument(text),
pos,
token,
);
const labels = items.map((i) => i.label);
assert.ok(labels.includes("CrateDebris_01"));
const item = items.find((i) => i.label === "CrateDebris_01");
// The unclosed element's end is the document end, so the typed "C" is
// still a real token and the range covers it.
assert.equal(item.range.start.character, line.indexOf(">C") + 1);
assert.equal(item.range.end.character, pos.character);
});
test("content start after accepting a simple-content snippet offers values, not attributes", async () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject>\n` +
` <CreateObject></CreateObject>\n` +
` </CreateObject>\n` +
` </ObjectCreationList>\n` +
`</AssetDeclaration>`;
const line = text.split("\n")[3];
const pos = new Position(3, line.indexOf(">") + 1);
const go = {
type: "GameObject",
id: "CrateDebris_01",
file: "Crates.xml",
line: 1,
origin: "project",
};
const idx = {
assets: new Map([["GameObject", new Map([["cratedebris_01", [go]]])]]),
assetsById: new Map([["cratedebris_01", [go]]]),
};
const result = await makeProvider(idx).provideCompletionItems(
makeDocument(text),
pos,
token,
);
const labels = listItems(result).map((i) => i.label);
assert.ok(labels.includes("CrateDebris_01"));
assert.ok(!labels.includes("xai:joinAction"));
assert.ok(!labels.includes("xmlns:xai"));
});
test("large asset-id lists are incomplete so narrower prefixes can re-request", async () => {
const defs = [];
for (let i = 0; i < 450; i++) {
defs.push({
type: "GameObject",
id: `C${String(i).padStart(3, "0")}`,
file: `C${i}.xml`,
line: 1,
origin: "project",
});
}
defs.push({
type: "GameObject",
id: "CrateDebris_01",
file: "Crates.xml",
line: 1,
origin: "project",
});
const byId = new Map();
const gameObjects = new Map();
const assets = new Map([["GameObject", gameObjects]]);
for (const def of defs) {
const key = def.id.toLowerCase();
byId.set(key, [def]);
gameObjects.set(key, [def]);
}
const idx = { assets, assetsById: byId };
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject>\n` +
` <CreateObject>C</CreateObject>\n` +
` </CreateObject>\n` +
` </ObjectCreationList>\n` +
`</AssetDeclaration>`;
const line = text.split("\n")[3];
const pos = new Position(3, line.indexOf(">C") + 2);
const first = await makeProvider(idx).provideCompletionItems(
makeDocument(text),
pos,
token,
);
assert.equal(Array.isArray(first), false);
assert.equal(first.isIncomplete, true, "capped list asks VS Code to recompute");
assert.equal(first.items.length, 400);
assert.ok(
!first.items.some((i) => i.label === "CrateDebris_01"),
"the target is beyond the initial 400 and must be found by a re-request",
);
// isIncomplete makes VS Code call the provider again as the prefix narrows.
const text2 = text.replace(">C<", ">Cr<");
const line2 = text2.split("\n")[3];
const second = await makeProvider(idx).provideCompletionItems(
makeDocument(text2),
new Position(3, line2.indexOf(">Cr") + 3),
token,
);
const secondItems = listItems(second);
assert.ok(
secondItems.some((i) => i.label === "CrateDebris_01"),
"narrower prefix re-request reaches the previously cut-off id",
);
});
test("current-file local overlay assets survive the global 400 cap", async () => {
const defs = [];
for (let i = 0; i < 450; i++) {
defs.push({
type: "GameObject",
id: `C${String(i).padStart(3, "0")}`,
file: `C${i}.xml`,
line: 1,
origin: "project",
});
}
const byId = new Map();
const gameObjects = new Map();
const assets = new Map([["GameObject", gameObjects]]);
for (const def of defs) {
const key = def.id.toLowerCase();
byId.set(key, [def]);
gameObjects.set(key, [def]);
}
const localGo = {
type: "GameObject",
id: "CrateDebris_01",
file: "Crates.xml",
line: 1,
origin: "project",
stream: "local",
};
const idx = {
assets,
assetsById: byId,
local: {
assets: new Map([
["GameObject", new Map([["cratedebris_01", [localGo]]])],
]),
assetsById: new Map([["cratedebris_01", [localGo]]]),
defines: new Map(),
},
};
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject>\n` +
` <CreateObject>C</CreateObject>\n` +
` </CreateObject>\n` +
` </ObjectCreationList>\n` +
`</AssetDeclaration>`;
const line = text.split("\n")[3];
const result = await makeProvider(idx).provideCompletionItems(
makeDocument(text),
new Position(3, line.indexOf(">C") + 2),
token,
);
assert.equal(Array.isArray(result), false);
assert.equal(result.isIncomplete, true);
assert.ok(result.items.some((i) => i.label === "CrateDebris_01"));
});
+281
View File
@@ -0,0 +1,281 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { createRequire } from "node:module";
// Minimal vscode shim for hover / definition / diagnostics providers.
const CompletionItemKind = {};
class Position {
constructor(line, character) {
this.line = line;
this.character = character;
}
}
class Range {
constructor(start, end) {
this.start = start;
this.end = end;
}
}
class MarkdownString {
constructor(value) {
this.value = value ?? "";
}
appendMarkdown(text) {
this.value += text;
return this;
}
appendCodeblock(text) {
this.value += "\n```\n" + text + "\n```\n";
return this;
}
}
class Hover {
constructor(contents) {
this.contents = contents;
}
}
class Location {
constructor(uri, range) {
this.uri = uri;
this.range = range;
}
}
class Diagnostic {
constructor(range, message, severity) {
this.range = range;
this.message = message;
this.severity = severity;
}
}
class FakeDiagnosticCollection {
constructor() {
this.last = null;
}
set(uri, diags) {
this.last = { uri, diags };
}
delete() {}
dispose() {}
}
const Uri = {
file: (p) => ({ fsPath: p }),
};
const workspace = {
getWorkspaceFolder: (uri) => ({ uri: { fsPath: "C:/mod" } }),
};
const languages = {
createDiagnosticCollection: () => new FakeDiagnosticCollection(),
};
const require = createRequire(import.meta.url);
const Module = require("module");
const origResolve = Module._resolveFilename;
Module._resolveFilename = function (request, ...args) {
if (request === "vscode") return "vscode-stub";
return origResolve.call(this, request, ...args);
};
require.cache["vscode-stub"] = {
id: "vscode-stub",
filename: "vscode-stub",
loaded: true,
exports: {
CompletionItemKind,
Position,
Range,
MarkdownString,
Hover,
Location,
Diagnostic,
DiagnosticSeverity: { Error: 0, Warning: 1, Information: 2, Hint: 3 },
Uri,
workspace,
languages,
},
};
const { Ra3HoverProvider } = require("../out/features/hover.js");
const { Ra3DefinitionProvider } = require("../out/features/navigation.js");
const { Ra3Diagnostics } = require("../out/features/diagnostics.js");
const { parseXml, LineMap } = require("../out/language/xmlParser.js");
const { expandDocument } = require("../out/indexer/logicalTree.js");
const URI = "C:/mod/Data/Crates.xml";
function makeDocument(text, uri = URI) {
const lineStarts = [0];
for (let i = 0; i < text.length; i++) {
if (text.charCodeAt(i) === 10) lineStarts.push(i + 1);
}
return {
uri: { fsPath: uri },
getText: () => text,
offsetAt: (pos) => lineStarts[pos.line] + pos.character,
positionAt: (offset) => {
let lo = 0;
let hi = lineStarts.length - 1;
while (lo < hi) {
const mid = (lo + hi + 1) >> 1;
if (lineStarts[mid] <= offset) lo = mid;
else hi = mid - 1;
}
return new Position(lo, offset - lineStarts[lo]);
},
};
}
async function makeScope(text, idx) {
const lineMap = new LineMap(text);
const parse = parseXml(text);
const expanded = await expandDocument(URI, parse, {
resolve: () => null,
readDom: async () => null,
});
return {
uri: URI,
version: 1,
parse,
lineMap,
expanded,
lineMaps: new Map(),
overlay: {},
merged: idx,
};
}
function makeIdx(defs) {
const assets = new Map();
const assetsById = new Map();
for (const def of defs) {
const idKey = def.id.toLowerCase();
let byId = assets.get(def.type);
if (!byId) {
byId = new Map();
assets.set(def.type, byId);
}
byId.set(idKey, [def]);
assetsById.set(idKey, [def]);
}
return {
assets,
assetsById,
defines: new Map(),
projectDir: "C:/mod",
sdkDir: "C:/sdk",
files: new Map(),
streams: [],
manifests: new Map(),
sourceCandidates: [],
diagnostics: [],
stats: {},
};
}
const TEXT =
`<AssetDeclaration>\n` +
` <GameObject id="CrateDebris_01"/>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject>\n` +
` <CreateObject>CrateDebris_01</CreateObject>\n` +
` </CreateObject>\n` +
` </ObjectCreationList>\n` +
`</AssetDeclaration>`;
test("hover on simple-content text shows the referenced definition", async () => {
const def = {
type: "GameObject",
id: "CrateDebris_01",
file: URI,
line: 2,
origin: "project",
};
const idx = makeIdx([def]);
const scope = await makeScope(TEXT, idx);
const provider = new Ra3HoverProvider({
isRa3Workspace: () => true,
getScope: async () => scope,
searchPaths: () => null,
});
const line = TEXT.split("\n")[4];
const pos = new Position(4, line.indexOf("CrateDebris_01") + 3);
const hover = await provider.provideHover(makeDocument(TEXT), pos, {});
assert.ok(hover, "hover is returned for typed content text");
assert.match(hover.contents.value, /1 definition/);
assert.match(hover.contents.value, /GameObject/);
});
test("Ctrl+click on simple-content text jumps to the definition", async () => {
const def = {
type: "GameObject",
id: "CrateDebris_01",
file: URI,
line: 2,
origin: "project",
};
const scope = await makeScope(TEXT, makeIdx([def]));
const provider = new Ra3DefinitionProvider({
isRa3Workspace: () => true,
getScope: async () => scope,
settings: { definitionMode: "all" },
indexer: null,
});
const line = TEXT.split("\n")[4];
const pos = new Position(4, line.indexOf("CrateDebris_01") + 3);
const locations = await provider.provideDefinition(makeDocument(TEXT), pos, {});
assert.ok(locations && locations.length === 1, "content definition resolves");
const defStart = TEXT.indexOf('id="CrateDebris_01"') + 'id="'.length;
const defEnd = defStart + "CrateDebris_01".length;
assert.deepEqual(
{
start: locations[0].range.start,
end: locations[0].range.end,
},
{
start: makeDocument(TEXT).positionAt(defStart),
end: makeDocument(TEXT).positionAt(defEnd),
},
);
});
test("diagnostics report unresolved typed content references only", async () => {
const text =
`<AssetDeclaration>\n` +
` <GameObject id="CrateDebris_01"/>\n` +
` <ObjectFilter>\n` +
` <IncludeThing>SomeValue</IncludeThing>\n` +
` </ObjectFilter>\n` +
` <ObjectCreationList id="OCL">\n` +
` <CreateObject>\n` +
` <CreateObject>CrateDebris_01</CreateObject>\n` +
` <CreateObject>MissingThing</CreateObject>\n` +
` </CreateObject>\n` +
` </ObjectCreationList>\n` +
`</AssetDeclaration>`;
const def = {
type: "GameObject",
id: "CrateDebris_01",
file: URI,
line: 2,
origin: "project",
};
const scope = await makeScope(text, makeIdx([def]));
const collection = new FakeDiagnosticCollection();
const provider = new Ra3Diagnostics({
isRa3Workspace: () => true,
getScope: async () => scope,
settings: {
diagnoseUnknownElements: false,
reportUnresolvedReferences: "warning",
},
});
provider["collection"] = collection;
await provider.update(makeDocument(text));
const messages = collection.last.diags.map((d) => d.message);
assert.ok(
messages.some((m) => m.includes('Unresolved reference "MissingThing"')),
"typed content refs are diagnosed",
);
assert.ok(
!messages.some((m) => m.includes("SomeValue")),
"untyped WeakReference content is not diagnosed as a global ref",
);
});
+72
View File
@@ -34,6 +34,78 @@ test("closed quote still resolves value context", () => {
assert.equal(ctx.valuePrefix, "GROUND"); assert.equal(ctx.valuePrefix, "GROUND");
}); });
test("multi-line unterminated quote keeps attribute-value context", () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject\n` +
` Options="IGNORE_ALL_OBJECTS"\n` +
` Disposition="`;
const cursor = text.length;
const doc = parseXml(text);
const ctx = analyzeContext(doc, text, cursor);
assert.equal(ctx.kind, "attribute-value");
assert.equal(ctx.attr?.name, "Disposition");
assert.equal(ctx.valuePrefix, "");
assert.equal(ctx.element?.name, "CreateObject");
});
test("multi-line unterminated value prefix includes typed flags", () => {
const text =
`<AssetDeclaration>\n` +
` <ObjectCreationList id="OCL_CrateSpawn">\n` +
` <CreateObject\n` +
` Options="IGNORE_ALL_OBJECTS"\n` +
` Disposition="RANDOM_FORCE `;
const cursor = text.length;
const doc = parseXml(text);
const ctx = analyzeContext(doc, text, cursor);
assert.equal(ctx.kind, "attribute-value");
assert.equal(ctx.attr?.name, "Disposition");
assert.equal(ctx.valuePrefix, "RANDOM_FORCE ");
});
test("cursor after a closed quote is an attribute-name context", () => {
const text = `<Locomotor id="x" Surfaces="GROUND">\n</Locomotor>`;
const cursor = text.indexOf('GROUND"') + 'GROUND"'.length;
const doc = parseXml(text);
const ctx = analyzeContext(doc, text, cursor);
assert.equal(ctx.kind, "attribute-name");
assert.equal(ctx.attr, null);
});
test("cursor exactly after an opening tag with a closing tag is content", () => {
const text = `<CreateObject></CreateObject>`;
const cursor = text.indexOf(">") + 1;
const doc = parseXml(text);
const ctx = analyzeContext(doc, text, cursor);
assert.equal(ctx.kind, "content");
assert.equal(ctx.element?.name, "CreateObject");
});
test("cursor after a typed > but before the closing tag is content too", () => {
const text = `<CreateObject>`;
const cursor = text.length;
const doc = parseXml(text);
const ctx = analyzeContext(doc, text, cursor);
assert.equal(ctx.kind, "content");
assert.equal(ctx.element?.name, "CreateObject");
});
test("cursor after a closed child element belongs to the parent content", () => {
const text =
`<AssetDeclaration>` +
`<ObjectCreationList>` +
`<CreateObject>X</CreateObject>` +
`</ObjectCreationList>` +
`</AssetDeclaration>`;
const cursor = text.indexOf("</CreateObject>") + "</CreateObject>".length;
const doc = parseXml(text);
const ctx = analyzeContext(doc, text, cursor);
assert.equal(ctx.kind, "content");
assert.equal(ctx.element?.name, "ObjectCreationList");
});
test("splitListValuePrefix isolates the token being edited", () => { test("splitListValuePrefix isolates the token being edited", () => {
assert.deepEqual(splitListValuePrefix("GROUND WA"), { token: "WA", start: 7 }); assert.deepEqual(splitListValuePrefix("GROUND WA"), { token: "WA", start: 7 });
assert.deepEqual(splitListValuePrefix("GROUND "), { token: "", start: 7 }); assert.deepEqual(splitListValuePrefix("GROUND "), { token: "", start: 7 });
+132
View File
@@ -0,0 +1,132 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { join } from "node:path";
import fs from "node:fs";
import os from "node:os";
import { DiskRecordsCache, diskCacheKey } from "../out/indexer/diskCache.js";
const identity = {
projectDir: "C:/proj",
sdkDir: "C:/sdk",
indexSageXml: true,
additionalDataSearchPaths: [],
builtmodsDirs: ["C:/sdk/builtmods"],
};
const sampleRecords = {
assets: [{ type: "GameObject", id: "TankA", line: 3 }],
defines: [{ name: "HP", value: "100", line: 2 }],
includes: [{ type: "all", source: "Units.xml", line: 4 }],
rootXiIncludes: [],
nestedXiIncludes: [],
};
function stampOf(file) {
const s = fs.statSync(file);
return {
mtimeMs: s.mtimeMs,
size: s.size,
birthtimeMs: s.birthtimeMs,
ctimeMs: s.ctimeMs,
};
}
function makeTmp(t) {
const tmp = fs.mkdtempSync(join(os.tmpdir(), "ra3-diskcache-"));
t.after(() => fs.rmSync(tmp, { recursive: true, force: true }));
return tmp;
}
test("disk cache roundtrip keeps records and leaves no temp file", async (t) => {
const tmp = makeTmp(t);
const file = join(tmp, "a.xml");
fs.writeFileSync(file, "0123456789");
const filePath = join(tmp, "index-records.json.gz");
const cache = new DiskRecordsCache(filePath, identity);
await cache.save([
[
file.toLowerCase(),
{ stat: stampOf(file), records: sampleRecords, kind: "full" },
],
]);
assert.equal(fs.existsSync(`${filePath}.tmp`), false, "atomic write leaves no temp");
const { records, stats } = await cache.loadValidated();
assert.equal(stats.fileExists, true);
assert.equal(stats.keyMatched, true);
assert.equal(stats.loaded, 1);
assert.equal(stats.validated, 1);
assert.equal(stats.dropped, 0);
assert.equal(records.length, 1);
assert.deepEqual(records[0].records, sampleRecords);
});
test("stat mismatch drops the cached entry", async (t) => {
const tmp = makeTmp(t);
const file = join(tmp, "a.xml");
fs.writeFileSync(file, "0123456789");
const filePath = join(tmp, "index-records.json.gz");
const cache = new DiskRecordsCache(filePath, identity);
await cache.save([
[file.toLowerCase(), { stat: stampOf(file), records: sampleRecords, kind: "full" }],
]);
const past = new Date(Date.now() - 60000);
fs.utimesSync(file, past, past);
const { records, stats } = await cache.loadValidated();
assert.equal(stats.validated, 0);
assert.equal(stats.dropped, 1);
assert.equal(records.length, 0);
});
test("identity mismatch ignores the cache", async (t) => {
const tmp = makeTmp(t);
const file = join(tmp, "a.xml");
fs.writeFileSync(file, "0123456789");
const filePath = join(tmp, "index-records.json.gz");
const cache = new DiskRecordsCache(filePath, identity);
await cache.save([
[file.toLowerCase(), { stat: stampOf(file), records: sampleRecords, kind: "full" }],
]);
const other = new DiskRecordsCache(filePath, {
...identity,
sdkDir: "D:/other-sdk",
});
const { records, stats } = await other.loadValidated();
assert.equal(stats.fileExists, true);
assert.equal(stats.keyMatched, false);
assert.equal(records.length, 0);
});
test("corrupt cache file yields an empty result", async (t) => {
const tmp = makeTmp(t);
const filePath = join(tmp, "index-records.json.gz");
fs.writeFileSync(filePath, "this is not gzip json");
const cache = new DiskRecordsCache(filePath, identity);
const { records, stats } = await cache.loadValidated();
assert.equal(stats.fileExists, true);
assert.equal(records.length, 0);
});
test("clear removes the cache file", async (t) => {
const tmp = makeTmp(t);
const file = join(tmp, "a.xml");
fs.writeFileSync(file, "0123456789");
const filePath = join(tmp, "index-records.json.gz");
const cache = new DiskRecordsCache(filePath, identity);
await cache.save([
[file.toLowerCase(), { stat: stampOf(file), records: sampleRecords, kind: "full" }],
]);
assert.ok(fs.existsSync(filePath));
await cache.clear();
assert.equal(fs.existsSync(filePath), false);
});
test("diskCacheKey differs when the identity changes", () => {
const a = diskCacheKey(identity);
const b = diskCacheKey({ ...identity, indexSageXml: false });
assert.notEqual(a, b);
assert.equal(a, diskCacheKey(identity));
});
+102
View File
@@ -0,0 +1,102 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { join, parse, resolve } from "node:path";
import fs from "node:fs";
import os from "node:os";
import { resolveSource } from "../out/indexer/includeResolver.js";
import {
ExistenceSnapshot,
buildExistenceSnapshot,
isDriveRoot,
} from "../out/indexer/existence.js";
function makeTmp(t) {
const tmp = fs.mkdtempSync(join(os.tmpdir(), "ra3-existence-"));
t.after(() => fs.rmSync(tmp, { recursive: true, force: true }));
return tmp;
}
test("isDriveRoot detects filesystem roots", () => {
assert.equal(isDriveRoot(parse(process.cwd()).root), true);
assert.equal(isDriveRoot(process.cwd()), false);
});
test("ExistenceSnapshot answers covered paths and falls back outside roots", (t) => {
const tmp = makeTmp(t);
const dataDir = join(tmp, "data");
fs.mkdirSync(dataDir);
const existing = join(dataDir, "Units.xml");
fs.writeFileSync(existing, "<x/>");
const snap = new ExistenceSnapshot([dataDir]);
assert.equal(snap.has(existing), true);
assert.equal(snap.has(join(dataDir, "Missing.xml")), false);
assert.equal(snap.has(join(tmp, "outside.xml")), null, "outside roots is unknown");
assert.ok(snap.hits >= 2, "covered lookups counted as hits");
assert.equal(snap.fallbacks, 1);
if (process.platform === "win32") {
assert.equal(
snap.has(existing.toUpperCase()),
true,
"lookup is case-insensitive on Windows",
);
}
});
test("buildExistenceSnapshot covers bounded search bases lazily", async (t) => {
const tmp = makeTmp(t);
const dataDir = join(tmp, "data");
const artDir = join(tmp, "art");
fs.mkdirSync(join(dataDir, "sub"), { recursive: true });
fs.mkdirSync(artDir);
fs.writeFileSync(join(dataDir, "Units.xml"), "<x/>");
fs.writeFileSync(join(dataDir, "sub", "Nested.xml"), "<x/>");
fs.writeFileSync(join(artDir, "Tank.w3x"), "<x/>");
const snap = buildExistenceSnapshot({
DATA: [dataDir],
ART: [artDir],
AUDIO: [],
});
assert.equal(snap.has(join(dataDir, "Units.xml")), true);
assert.equal(snap.has(join(dataDir, "sub", "Nested.xml")), true);
assert.equal(snap.has(join(artDir, "Tank.w3x")), true);
assert.equal(snap.has(join(dataDir, "Missing.xml")), false);
});
test("resolveSource uses the snapshot and falls back to statSync outside it", async (t) => {
const tmp = makeTmp(t);
const dataDir = join(tmp, "data");
const outsideDir = join(tmp, "outside");
fs.mkdirSync(dataDir);
fs.mkdirSync(outsideDir);
const units = join(dataDir, "Units.xml");
const outside = join(outsideDir, "Extra.xml");
fs.writeFileSync(units, "<x/>");
fs.writeFileSync(outside, "<x/>");
const searchPaths = { DATA: [dataDir], ART: [], AUDIO: [] };
const snap = buildExistenceSnapshot(searchPaths);
const found = resolveSource("DATA:Units.xml", null, searchPaths, snap);
assert.equal(found.path, units);
assert.ok(snap.hits > 0, "covered lookup served by the snapshot");
const missing = resolveSource("DATA:Missing.xml", null, searchPaths, snap);
assert.equal(missing.path, null);
const emptySearchPaths = { DATA: [], ART: [], AUDIO: [] };
const outsideResolved = resolveSource(
outside,
outsideDir,
emptySearchPaths,
snap,
);
assert.equal(
resolve(outsideResolved.path ?? ""),
resolve(outside),
"uncovered path falls back to statSync",
);
assert.ok(snap.fallbacks > 0, "fallback counted");
});
+72
View File
@@ -0,0 +1,72 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { join } from "node:path";
import {
isContentRelevantPath,
isWatcherNoisePath,
} from "../out/indexer/fileScanner.js";
test("isWatcherNoisePath filters .git internals", () => {
assert.equal(
isWatcherNoisePath(join("C:/proj", ".git", "index")),
true,
".git files are noise",
);
assert.equal(
isWatcherNoisePath(join("C:/proj", "data", ".git", "FETCH_HEAD")),
true,
"nested .git directories are noise",
);
assert.equal(
isWatcherNoisePath(join("C:/proj", "Data", "Mod.xml")),
false,
"project XML is not noise",
);
assert.equal(
isWatcherNoisePath(join("C:/proj", ".gitignore")),
false,
".gitignore is a real project file",
);
assert.equal(
isWatcherNoisePath(join("C:/proj", "Data", "UnitCrate.xml.git")),
true,
"editor temp files ending in .git are noise",
);
assert.equal(
isWatcherNoisePath(join("C:/proj", "Data", "UnitCrate.xml.tmp")),
true,
".tmp files are noise",
);
assert.equal(
isWatcherNoisePath(join("C:/proj", "Data", "Mod.xml~")),
true,
"backup files ending in ~ are noise",
);
assert.equal(
isWatcherNoisePath(join("C:/proj", "Data", ".#Mod.xml")),
true,
"lock files starting with .# are noise",
);
});
test("isContentRelevantPath only reacts to XML-ish content", () => {
assert.equal(isContentRelevantPath("a.xml"), true);
assert.equal(isContentRelevantPath("a.w3x"), true);
assert.equal(
isContentRelevantPath("a.manifestxml"),
false,
"there is no .manifestxml source format",
);
assert.equal(
isContentRelevantPath("a.w3d"),
false,
".w3d is binary art, not text XML",
);
assert.equal(isContentRelevantPath("a.dds"), false);
assert.equal(isContentRelevantPath("a.xml.git"), false);
assert.equal(
isContentRelevantPath("a.lua"),
false,
"lua is not indexed yet",
);
});
@@ -0,0 +1,6 @@
<?xml version="1.0" encoding="utf-8"?>
<AssetDeclaration xmlns="uri:ea.com:eala:asset">
<HeadlightDraw2>
<TruckDraw id="ModuleTag_Headlight" />
</HeadlightDraw2>
</AssetDeclaration>
@@ -0,0 +1,2 @@
This is a fake DDS payload used to verify that binary assets included from
an XML hub are registered as files but never parsed as XML.
@@ -0,0 +1,4 @@
<?xml version="1.0" encoding="UTF-8"?>
<AssetDeclaration xmlns="uri:ea.com:eala:asset">
<W3DMesh id="Tank_FP" />
</AssetDeclaration>
@@ -0,0 +1,5 @@
<?xml version="1.0" encoding="UTF-8"?>
<AssetDeclaration xmlns="uri:ea.com:eala:asset">
<W3DContainer id="Tank_SKN" Hierarchy="Tank_SKL" />
<W3DHierarchy id="Tank_SKL" />
</AssetDeclaration>
@@ -0,0 +1,4 @@
<?xml version="1.0" encoding="utf-8"?>
<AssetDeclaration xmlns="uri:ea.com:eala:asset">
<GameObject id="StandaloneBase" Side="Allies" />
</AssetDeclaration>
+8
View File
@@ -0,0 +1,8 @@
<?xml version="1.0" encoding="utf-8"?>
<AssetDeclaration xmlns="uri:ea.com:eala:asset">
<Includes>
<Include type="all" source="Models/Tank_SKN.w3x" />
<Include type="all" source="Models/Tank_FP.dat" />
<Include type="all" source="Models/Tank_Damaged.dds" />
</Includes>
</AssetDeclaration>
+1
View File
@@ -9,5 +9,6 @@
<Include type="all" source="Includes/Weapons.xml" /> <Include type="all" source="Includes/Weapons.xml" />
<Include type="all" source="Includes/Shared.xml" /> <Include type="all" source="Includes/Shared.xml" />
<Include type="all" source="Includes/Refs.xml" /> <Include type="all" source="Includes/Refs.xml" />
<Include type="all" source="Includes/VehicleArt.xml" />
</Includes> </Includes>
</AssetDeclaration> </AssetDeclaration>
+20
View File
@@ -0,0 +1,20 @@
<?xml version="1.0" encoding="utf-8"?>
<AssetDeclaration xmlns="uri:ea.com:eala:asset">
<Includes>
<Include type="instance" source="Includes/StandaloneBase.xml" />
</Includes>
<Defines>
<Define name="STANDALONE_HEALTH" value="200.0" />
</Defines>
<GameObject id="StandaloneTank" inheritFrom="StandaloneBase">
<Draws>
<TruckDraw id="ModuleTag_Draw" />
<xi:include
href="DATA:Includes/HeadlightModules.xml"
xpointer="xmlns(n=uri:ea.com:eala:asset) xpointer(/n:HeadlightDraw2/child::*)" />
</Draws>
<BehaviorModules>
<ReconstituteStateSpecialAbility UpdateModuleId="ModuleTag_Headlight" />
</BehaviorModules>
</GameObject>
</AssetDeclaration>
+290
View File
@@ -2,8 +2,16 @@ import { test } from "node:test";
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { fileURLToPath } from "node:url"; import { fileURLToPath } from "node:url";
import { dirname, join } from "node:path"; import { dirname, join } from "node:path";
import fs from "node:fs";
import os from "node:os";
import { ModIndexer } from "../out/indexer/indexer.js"; import { ModIndexer } from "../out/indexer/indexer.js";
import { CachedDirectoryWalker } from "../out/indexer/fileScanner.js"; import { CachedDirectoryWalker } from "../out/indexer/fileScanner.js";
import {
DocumentCache,
IncludeResolveCache,
IndexRecordsCache,
} from "../out/indexer/caches.js";
import { resolveReferenceTargetsForType } from "../out/indexer/refs.js";
const root = dirname(dirname(fileURLToPath(import.meta.url))); const root = dirname(dirname(fileURLToPath(import.meta.url)));
const project = join(root, "test", "fixtures", "minimod"); const project = join(root, "test", "fixtures", "minimod");
@@ -70,3 +78,285 @@ test("provides include source candidates", async () => {
assert.ok(xml.some((c) => c.source === "Includes/Units.xml")); assert.ok(xml.some((c) => c.source === "Includes/Units.xml"));
assert.ok(xml.some((c) => c.source === "DATA:static.xml")); assert.ok(xml.some((c) => c.source === "DATA:static.xml"));
}); });
test("indexes art-asset XML (.w3x / sniffed unknown extension) via shallow scan and skips binary", async () => {
const idx = await buildIndex();
// The .w3x hub chain: Mod.xml -> VehicleArt.xml -> Models/Tank_SKN.w3x.
const skn = idx.assetsById.get("tank_skn");
assert.ok(skn?.some((d) => d.type === "W3DContainer"), "W3DContainer from .w3x indexed");
assert.ok(
idx.assetsById.get("tank_skl")?.some((d) => d.type === "W3DHierarchy"),
"W3DHierarchy from .w3x indexed",
);
const container = skn.find((d) => d.type === "W3DContainer");
assert.match(container.file, /Tank_SKN\.w3x$/);
assert.ok(container.line > 0, "definition line recorded from shallow scan");
// Unknown extension with XML content is sniffed and indexed.
assert.ok(
idx.assetsById.get("tank_fp")?.some((d) => d.type === "W3DMesh"),
"unknown-extension XML (.dat) sniffed and indexed",
);
// Binary content is registered as a file but never parsed.
assert.equal(idx.assetsById.get("tank_damaged"), undefined, "binary asset never indexed");
assert.ok(
idx.files.has(
join(project, "Data", "Includes", "Models", "Tank_Damaged.dds").toLowerCase(),
),
"binary include target registered as a file",
);
// The reported Harbinger scenario: <Model Name="Tank_SKN"/> (refType
// BaseRenderAssetType) must resolve to the W3DContainer defined in the w3x.
const targets = resolveReferenceTargetsForType(
idx,
"ScriptedModelDrawModel",
"Name",
"Tank_SKN",
);
assert.equal(targets.length, 1);
assert.equal(targets[0].def.type, "W3DContainer");
assert.match(targets[0].def.file, /Tank_SKN\.w3x$/);
});
test("w3x files appear in Include source completion candidates", async () => {
const idx = await buildIndex();
assert.ok(
idx.sourceCandidates.some((c) => c.source === "Includes/Models/Tank_SKN.w3x"),
"project-relative w3x candidate",
);
assert.ok(
idx.sourceCandidates.some((c) => c.source === "DATA:Includes/Models/Tank_SKN.w3x"),
"DATA: w3x candidate",
);
});
test("build publishes an immutable XML phase before art scanning", async () => {
let phaseA;
const indexer = new ModIndexer({
projectDir: project,
sdkDir: sdk,
builtmodsDirs: [join(sdk, "builtmods")],
indexSageXml: true,
additionalDataSearchPaths: [],
walker: new CachedDirectoryWalker(),
});
const idx = await indexer.build((p) => {
phaseA = p;
});
assert.ok(phaseA, "phase-A snapshot published");
assert.equal(phaseA.complete, false);
assert.equal(phaseA.phase, "xml");
assert.ok(phaseA.assetsById.has("testtank"), "XML assets available in phase A");
assert.ok(
phaseA.assetsById.has("vanillatank"),
"manifest assets available in phase A",
);
assert.equal(
phaseA.assetsById.has("tank_skn"),
false,
"art assets deferred in phase A",
);
assert.ok(phaseA.stats.deferredArtFiles >= 2, "deferred art queue recorded");
assert.equal(phaseA.stats.shallowScannedFiles, 0, "no art scanned during phase A");
assert.equal(idx.complete, true);
assert.equal(idx.phase, "art");
assert.ok(idx.assetsById.has("tank_skn"), "art assets present in the final index");
assert.equal(
phaseA.assetsById.has("tank_skn"),
false,
"phase-A snapshot is immutable (phase B did not mutate it)",
);
assert.equal(typeof idx.stats.artScanMs, "number");
assert.equal(
indexer.isIndexedFile(join(project, "Data", "Mod.xml")),
true,
"indexed files are recognized",
);
assert.equal(
indexer.isIndexedFile(join(project, "Data", "NotIndexed.xml")),
false,
"unrelated files are not recognized",
);
});
test("stat validation re-reads a file whose mtime changed", async (t) => {
const tmp = fs.mkdtempSync(join(os.tmpdir(), "ra3modxml-mtime-"));
t.after(() => fs.rmSync(tmp, { recursive: true, force: true }));
const projectDir = join(tmp, "project");
fs.mkdirSync(join(projectDir, "Data"), { recursive: true });
const modPath = join(projectDir, "Data", "Mod.xml");
fs.writeFileSync(
modPath,
`<?xml version="1.0"?>\n<AssetDeclaration>\n <GameObject id="TankA"/>\n</AssetDeclaration>\n`,
"utf8",
);
const documentCache = new DocumentCache();
const recordsCache = new IndexRecordsCache();
const resolveCache = new IncludeResolveCache();
const make = () =>
new ModIndexer({
projectDir,
sdkDir: sdk,
builtmodsDirs: [join(sdk, "builtmods")],
indexSageXml: false,
additionalDataSearchPaths: [],
walker: new CachedDirectoryWalker(),
documentCache,
recordsCache,
resolveCache,
trustUnchanged: false,
});
const first = await make().build();
assert.ok(first.assetsById.has("tanka"));
assert.equal(first.stats.recordsCacheHits, 0);
const past = new Date(Date.now() - 60000);
fs.utimesSync(modPath, past, past);
const second = await make().build();
assert.equal(
second.stats.recordsCacheHits,
0,
"mtime change invalidates the cached records",
);
assert.ok(second.assetsById.has("tanka"));
const third = await make().build();
assert.ok(
third.stats.recordsCacheHits > 0,
"unchanged files are served from the records cache",
);
});
test("shallow scans and full parses are cached across rebuilds", async () => {
const documentCache = new DocumentCache();
const recordsCache = new IndexRecordsCache();
const resolveCache = new IncludeResolveCache();
const makeIndexer = () =>
new ModIndexer({
projectDir: project,
sdkDir: sdk,
builtmodsDirs: [join(sdk, "builtmods")],
indexSageXml: true,
additionalDataSearchPaths: [],
walker: new CachedDirectoryWalker(),
documentCache,
recordsCache,
resolveCache,
});
const first = await makeIndexer().build();
const second = await makeIndexer().build();
assert.equal(first.stats.shallowScannedFiles, 2, "w3x + sniffed w3d scanned on first build");
assert.equal(second.stats.shallowScannedFiles, 0, "unchanged art assets are not re-scanned");
assert.equal(second.stats.shallowCacheHits, 2);
assert.ok(second.stats.recordsCacheHits > 0, "XML records served from cache");
assert.ok(second.stats.resolveCacheHits > 0, "include resolutions served from cache");
assert.equal(second.stats.resolveCalls, 0, "no include re-resolved on a trusted rebuild");
assert.equal(second.stats.assetCount, first.stats.assetCount);
assert.equal(second.stats.indexedFiles, first.stats.indexedFiles);
});
test("trusted rebuilds skip unchanged files; invalidation forces re-reads", async () => {
const documentCache = new DocumentCache();
const recordsCache = new IndexRecordsCache();
const resolveCache = new IncludeResolveCache();
const opts = () => ({
projectDir: project,
sdkDir: sdk,
builtmodsDirs: [join(sdk, "builtmods")],
indexSageXml: true,
additionalDataSearchPaths: [],
walker: new CachedDirectoryWalker(),
documentCache,
recordsCache,
resolveCache,
});
// Trusted rebuild: cached files are reused without any per-file stat/read.
const first = await new ModIndexer({ ...opts(), trustUnchanged: true }).build();
assert.equal(first.stats.shallowScannedFiles, 2);
const second = await new ModIndexer({ ...opts(), trustUnchanged: true }).build();
assert.equal(second.stats.shallowScannedFiles, 0);
assert.equal(second.stats.shallowCacheHits, 2);
assert.ok(second.stats.recordsCacheHits > 0);
// Simulate the file watcher / save handler: invalidate one art asset.
// The next trusted rebuild must re-scan exactly that file.
recordsCache.invalidate(join(project, "Data", "Includes", "Models", "Tank_SKN.w3x"));
const third = await new ModIndexer({ ...opts(), trustUnchanged: true }).build();
assert.equal(third.stats.shallowScannedFiles, 1);
assert.equal(third.stats.shallowCacheHits, 1);
assert.ok(
third.assetsById.get("tank_skn")?.some((d) => d.type === "W3DContainer"),
"re-scanned w3x asset present",
);
// Invalidate one XML file: exactly its records are re-extracted.
recordsCache.invalidate(join(project, "Data", "Includes", "Units.xml"));
const fourth = await new ModIndexer({ ...opts(), trustUnchanged: true }).build();
assert.equal(fourth.stats.recordsCacheHits, third.stats.recordsCacheHits - 1);
assert.ok(
fourth.assetsById.get("testtank")?.some((d) => d.type === "GameObject"),
"re-parsed XML asset present",
);
// A forced rebuild (ra3modxml.reindex) verifies stats but still reuses
// content caches for unchanged files.
const forced = await new ModIndexer({ ...opts(), trustUnchanged: false }).build();
assert.equal(forced.stats.shallowScannedFiles, 0);
assert.equal(forced.stats.shallowCacheHits, 2);
});
test("index stats include candidate/walk phase timings", async () => {
const idx = await buildIndex();
assert.equal(typeof idx.stats.candidatesMs, "number");
assert.equal(typeof idx.stats.walkMs, "number");
assert.ok(idx.stats.snapshotHits > 0, "existence snapshot answered lookups");
assert.equal(typeof idx.stats.snapshotFallbacks, "number");
});
test("w3x with a UTF-8 BOM is indexed with correct offsets", async (t) => {
const tmp = fs.mkdtempSync(join(os.tmpdir(), "ra3modxml-bom-"));
t.after(() => fs.rmSync(tmp, { recursive: true, force: true }));
const projectDir = join(tmp, "project");
fs.mkdirSync(join(projectDir, "Data", "Models"), { recursive: true });
fs.writeFileSync(
join(projectDir, "Data", "Mod.xml"),
`<?xml version="1.0" encoding="utf-8"?>
<AssetDeclaration>
<Includes>
<Include type="all" source="Models/Tank_BOM.w3x"/>
</Includes>
</AssetDeclaration>`,
"utf8",
);
fs.writeFileSync(
join(projectDir, "Data", "Models", "Tank_BOM.w3x"),
"\uFEFF<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n" +
"<AssetDeclaration>\n" +
" <W3DContainer id=\"Tank_BOM\" />\n" +
"</AssetDeclaration>\n",
"utf8",
);
const idx = await new ModIndexer({
projectDir,
sdkDir: sdk,
builtmodsDirs: [join(sdk, "builtmods")],
indexSageXml: false,
additionalDataSearchPaths: [],
walker: new CachedDirectoryWalker(),
}).build();
const def = idx.assetsById.get("tank_bom")?.find((d) => d.type === "W3DContainer");
assert.ok(def, "BOM-prefixed w3x asset indexed");
assert.equal(def.line, 3, "id line is correct despite the BOM");
});
+210
View File
@@ -0,0 +1,210 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { fileURLToPath } from "node:url";
import { dirname, join } from "node:path";
import { parseXml, LineMap, stripBom } from "../out/language/xmlParser.js";
import { extractIndexRecords } from "../out/indexer/records.js";
import { buildSearchPaths } from "../out/indexer/includeResolver.js";
import {
buildDocumentScope,
withLocalOverlay,
} from "../out/indexer/localScope.js";
import {
findContainingGameObject,
findLocalId,
collectLocalIds,
} from "../out/indexer/logicalTree.js";
import { resolveReferenceTargetsForType } from "../out/indexer/refs.js";
import { resolveElementType } from "../out/language/typeContext.js";
const root = dirname(dirname(fileURLToPath(import.meta.url)));
const project = join(root, "test", "fixtures", "minimod");
const sdk = join(root, "test", "fixtures", "fakesdk");
const standalonePath = join(project, "Data", "Standalone.xml");
async function readParsed(path) {
const text = stripBom(await readFile(path, "utf8"));
const lineMap = new LineMap(text);
const parse = parseXml(text);
return {
file: { path, stat: null },
parse,
records: extractIndexRecords(parse, lineMap),
lineMap,
};
}
async function makeScope() {
const searchPaths = buildSearchPaths(sdk, project);
const text = await readFile(standalonePath, "utf8");
return buildDocumentScope(standalonePath, text, 1, {
projectDir: project,
sdkDir: sdk,
searchPaths,
readRecords: readParsed,
readDom: readParsed,
});
}
test("local overlay resolves refs for a file outside every global stream", async () => {
const scope = await makeScope();
const merged = withLocalOverlay(null, scope.overlay, project, sdk);
assert.ok(merged.local.assetsById.has("standalonetank"));
assert.ok(merged.local.assetsById.has("standalonebase"));
assert.ok(merged.local.defines.has("standalone_health"));
const targets = resolveReferenceTargetsForType(
merged,
"GameObject",
"inheritFrom",
"StandaloneBase",
);
assert.equal(targets.length, 1);
assert.match(targets[0].def.file, /StandaloneBase\.xml$/);
assert.equal(targets[0].def.origin, "project");
});
test("logical xi:include expansion gives included modules their Draws context", async () => {
const scope = await makeScope();
const included = scope.expanded.elements.find(
(e) =>
e.name === "TruckDraw" &&
e.attrs.some((a) => a.name === "id" && a.value === "ModuleTag_Headlight"),
);
assert.ok(included, "xi:include target spliced into the logical tree");
assert.match(included.sourceFile, /HeadlightModules\.xml$/i);
assert.equal(
resolveElementType(included),
"W3DTruckDrawModuleData",
"included module resolves through the logical Draws parent",
);
const update = scope.expanded.elements.find(
(e) => e.name === "ReconstituteStateSpecialAbility",
);
assert.ok(update);
const gameObject = findContainingGameObject(update);
assert.equal(gameObject?.name, "GameObject");
assert.equal(
findLocalId(gameObject, "ModuleTag_Headlight"),
included,
"Poid reference can reach a module spliced in through xi:include",
);
const localIds = collectLocalIds(gameObject).map((i) => i.id);
assert.ok(localIds.includes("ModuleTag_Draw"));
assert.ok(localIds.includes("ModuleTag_Headlight"));
});
test("local overlay wins over a global definition with the same id", async () => {
const scope = await makeScope();
const global = {
assets: new Map(),
assetsById: new Map([
[
"standalonebase",
[
{
type: "GameObject",
id: "StandaloneBase",
file: join(sdk, "SageXml", "VanillaBase.xml"),
line: 1,
origin: "sdk",
},
],
],
]),
defines: new Map(),
};
const merged = withLocalOverlay(global, scope.overlay, project, sdk);
const targets = resolveReferenceTargetsForType(
merged,
"GameObject",
"inheritFrom",
"StandaloneBase",
);
assert.equal(targets.length, 2);
assert.match(targets[0].def.file, /StandaloneBase\.xml$/);
assert.equal(targets[0].def.origin, "project");
assert.match(targets[1].def.file, /VanillaBase\.xml$/);
});
test("logical expansion terminates on xi:include cycles", async (t) => {
const tmp = await mkdtemp(join(tmpdir(), "ra3-local-cycle-"));
t.after(() => rm(tmp, { recursive: true, force: true }));
const dataDir = join(tmp, "Data");
const includesDir = join(dataDir, "Includes");
await mkdir(includesDir, { recursive: true });
const aPath = join(dataDir, "A.xml");
const bPath = join(includesDir, "B.xml");
await writeFile(
aPath,
`<AssetDeclaration><GameObject id="A"><xi:include href="Includes/B.xml"/></GameObject></AssetDeclaration>`,
"utf8",
);
await writeFile(
bPath,
`<AssetDeclaration><GameObject id="B"><xi:include href="../A.xml"/></GameObject></AssetDeclaration>`,
"utf8",
);
const searchPaths = buildSearchPaths(sdk, tmp);
const text = await readFile(aPath, "utf8");
const scope = await buildDocumentScope(aPath, text, 1, {
projectDir: tmp,
sdkDir: sdk,
searchPaths,
readRecords: readParsed,
readDom: readParsed,
});
assert.ok(
scope.expanded.elements.some(
(e) =>
e.name === "GameObject" &&
e.attrs.some((a) => a.name === "id" && a.value === "A"),
),
"entry document survives a cycle",
);
});
test("the same xi:include target can expand under multiple parents", async (t) => {
const tmp = await mkdtemp(join(tmpdir(), "ra3-local-shared-"));
t.after(() => rm(tmp, { recursive: true, force: true }));
const dataDir = join(tmp, "Data");
const includesDir = join(dataDir, "Includes");
await mkdir(includesDir, { recursive: true });
const mainPath = join(dataDir, "Main.xml");
const fragmentPath = join(includesDir, "Fragment.xml");
await writeFile(
fragmentPath,
`<AssetDeclaration><HeadlightDraw2><TruckDraw id="ModuleTag_Shared"/></HeadlightDraw2></AssetDeclaration>`,
"utf8",
);
await writeFile(
mainPath,
`<AssetDeclaration>` +
`<GameObject id="A"><Draws><xi:include href="DATA:Includes/Fragment.xml" ` +
`xpointer="xmlns(n=uri:ea.com:eala:asset) xpointer(/n:HeadlightDraw2/child::*)"/></Draws></GameObject>` +
`<GameObject id="B"><Draws><xi:include href="DATA:Includes/Fragment.xml" ` +
`xpointer="xmlns(n=uri:ea.com:eala:asset) xpointer(/n:HeadlightDraw2/child::*)"/></Draws></GameObject>` +
`</AssetDeclaration>`,
"utf8",
);
const searchPaths = buildSearchPaths(sdk, tmp);
const text = await readFile(mainPath, "utf8");
const scope = await buildDocumentScope(mainPath, text, 1, {
projectDir: tmp,
sdkDir: sdk,
searchPaths,
readRecords: readParsed,
readDom: readParsed,
});
const shared = scope.expanded.elements.filter(
(e) =>
e.name === "TruckDraw" &&
e.attrs.some((a) => a.name === "id" && a.value === "ModuleTag_Shared"),
);
assert.equal(shared.length, 2, "same fragment expands once per parent");
});
+65
View File
@@ -0,0 +1,65 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { parseXml, LineMap } from "../out/language/xmlParser.js";
import { extractIndexRecords, recordsFromShallow } from "../out/indexer/records.js";
import { scanXmlShallow } from "../out/indexer/shallowScan.js";
test("extractIndexRecords mirrors the walk semantics", () => {
const text = `<?xml version="1.0"?>
<AssetDeclaration>
<Defines>
<Define name="HP" value="100"/>
</Defines>
<Includes>
<Include type="all" source="Units.xml"/>
<Include type="instance" source="Base.xml"/>
</Includes>
<GameObject id="Tank" CommandSet="TankCommandSet"/>
<WeaponTemplate id="TankGun"/>
<xi:include href="DATA:Extra.xml" xpointer="xmlns(n=uri:ea.com:eala:asset) xpointer(/n:Extra/child::*)"/>
<GameObject id="Tank2">
<Draws>
<xi:include href="DATA:Nested.xml"/>
</Draws>
</GameObject>
</AssetDeclaration>`;
const lineMap = new LineMap(text);
const records = extractIndexRecords(parseXml(text), lineMap);
assert.deepEqual(
records.assets.map((a) => [a.type, a.id, a.line]),
[
["GameObject", "Tank", 10],
["WeaponTemplate", "TankGun", 11],
["GameObject", "Tank2", 13],
],
);
assert.deepEqual(
records.defines.map((d) => [d.name, d.value]),
[["HP", "100"]],
);
assert.deepEqual(
records.includes.map((i) => [i.type, i.source]),
[
["all", "Units.xml"],
["instance", "Base.xml"],
],
);
assert.deepEqual(
records.rootXiIncludes.map((x) => [x.href, x.line]),
[["DATA:Extra.xml", 12]],
);
assert.deepEqual(
records.nestedXiIncludes.map((x) => [x.href, x.line]),
[["DATA:Nested.xml", 15]],
);
});
test("recordsFromShallow converts offsets to 1-based lines", () => {
const text = `<AssetDeclaration>\n <W3DContainer id="A"/>\n</AssetDeclaration>`;
const lineMap = new LineMap(text);
const records = recordsFromShallow(scanXmlShallow(text), lineMap);
assert.equal(records.assets.length, 1);
assert.equal(records.assets[0].type, "W3DContainer");
assert.equal(records.assets[0].id, "A");
assert.equal(records.assets[0].line, 2);
});
+58
View File
@@ -8,6 +8,8 @@ import {
isLocalReferenceAttribute, isLocalReferenceAttribute,
isReferenceAttribute, isReferenceAttribute,
isReferenceAttributeOfType, isReferenceAttributeOfType,
isReferenceContentType,
resolveContentReferenceTargets,
resolveReferenceTargets, resolveReferenceTargets,
resolveReferenceTargetsForType, resolveReferenceTargetsForType,
} from "../out/indexer/refs.js"; } from "../out/indexer/refs.js";
@@ -242,3 +244,59 @@ test("xi:include elements are outside the XSD model and unvalidated", () => {
assert.equal(model.isXsdAttributeName(a.name), true, a.name); assert.equal(model.isXsdAttributeName(a.name), true, a.name);
} }
}); });
test("typed simple content resolves like a typed attribute reference", () => {
// <CreateObject>CrateDebris_01</CreateObject> uses GameObjectWeakRef:
// the content is a GameObject reference, not a child element.
assert.equal(isReferenceContentType("GameObjectWeakRef"), true);
const idx = {
assetsById: new Map([
[
"cratedebris_01",
[
{
type: "GameObject",
id: "CrateDebris_01",
file: "Crates.xml",
line: 2,
origin: "project",
},
{
type: "WeaponTemplate",
id: "CrateDebris_01",
file: "Weapons.xml",
line: 4,
origin: "project",
},
],
],
]),
assets: new Map(),
projectDir: ".",
sdkDir: ".",
defines: new Map(),
files: new Map(),
streams: [],
manifests: new Map(),
sourceCandidates: [],
diagnostics: [],
stats: {},
};
const targets = resolveContentReferenceTargets(idx, "GameObjectWeakRef", "CrateDebris_01");
assert.equal(targets.length, 1);
assert.equal(targets[0].def.type, "GameObject");
});
test("untyped and pipeline-local content is not a global reference", () => {
// Generic AssetReference content is used for shader constants and model
// sub-object names, not global asset ids; Poid is pipeline-local.
assert.equal(isReferenceContentType("AssetReference"), false);
assert.equal(isReferenceContentType("Poid"), false);
assert.equal(isReferenceContentType("string"), false);
const idx = { assetsById: new Map(), assets: new Map(), defines: new Map() };
assert.equal(
resolveContentReferenceTargets(idx, "AssetReference", "Anything").length,
0,
);
assert.equal(resolveContentReferenceTargets(idx, "Poid", "Anything").length, 0);
});
+2 -2
View File
@@ -130,13 +130,13 @@ test("fallback tokens cover names, attributes and values", () => {
test("well-formed XML gets no semantic fallback tokens", async () => { test("well-formed XML gets no semantic fallback tokens", async () => {
const text = `<AssetDeclaration>\n <LocomotorTemplate id="x" Surfaces="GROUND"/>\n</AssetDeclaration>`; const text = `<AssetDeclaration>\n <LocomotorTemplate id="x" Surfaces="GROUND"/>\n</AssetDeclaration>`;
const provider = new Ra3SemanticTokensProvider(); const provider = new Ra3SemanticTokensProvider({ isRa3Workspace: () => true });
const result = await provider.provideDocumentSemanticTokens(makeDocument(text), {}); const result = await provider.provideDocumentSemanticTokens(makeDocument(text), {});
assert.equal(result.data.length, 0); assert.equal(result.data.length, 0);
}); });
test("malformed XML gets semantic fallback tokens", async () => { test("malformed XML gets semantic fallback tokens", async () => {
const provider = new Ra3SemanticTokensProvider(); const provider = new Ra3SemanticTokensProvider({ isRa3Workspace: () => true });
const result = await provider.provideDocumentSemanticTokens( const result = await provider.provideDocumentSemanticTokens(
makeDocument(MALFORMED), makeDocument(MALFORMED),
{}, {},
+122
View File
@@ -0,0 +1,122 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { scanXmlShallow } from "../out/indexer/shallowScan.js";
test("extracts top-level assets, includes, defines and xi:include", () => {
const text = `<?xml version="1.0"?>
<AssetDeclaration xmlns="uri:ea.com:eala:asset">
<Defines>
<Define name="MODEL_SCALE" value="1.0"/>
</Defines>
<Includes>
<Include type="all" source="Art/Model_SKN.w3x"/>
<Include type="reference" source="DATA:static.xml"/>
</Includes>
<W3DContainer id="Model_SKN" Hierarchy="Model_SKL">
<SubObject SubObjectID="GUN">
<RenderObject><Mesh>Model_SKN.GUN</Mesh></RenderObject>
</SubObject>
</W3DContainer>
<W3DMesh id="Model_SKN.GUN"/>
<xi:include href="DATA:Includes/Extra.xml" xpointer="xmlns(n=uri:ea.com:eala:asset) xpointer(/n:Extra/child::*)"/>
</AssetDeclaration>`;
const doc = scanXmlShallow(text);
assert.equal(doc.errors.length, 0);
assert.deepEqual(
doc.assets.map((a) => `${a.name}:${a.id}`),
["W3DContainer:Model_SKN", "W3DMesh:Model_SKN.GUN"],
);
assert.equal(doc.includes.length, 2);
assert.equal(doc.includes[0].type, "all");
assert.equal(doc.includes[0].source, "Art/Model_SKN.w3x");
assert.equal(doc.includes[1].type, "reference");
assert.equal(doc.defines.length, 1);
assert.equal(doc.defines[0].name, "MODEL_SCALE");
assert.equal(doc.defines[0].value, "1.0");
assert.equal(doc.rootXiIncludes.length, 1);
assert.equal(doc.rootXiIncludes[0].href, "DATA:Includes/Extra.xml");
assert.match(doc.rootXiIncludes[0].xpointer, /xpointer\(/);
assert.equal(doc.nestedXiIncludes.length, 0);
// Recorded offsets must slice the original text back out.
const container = doc.assets[0];
assert.equal(text.slice(container.idValueStart, container.idValueEnd), "Model_SKN");
assert.equal(text.slice(container.start, container.startTagEnd).startsWith("<W3DContainer"), true);
const mesh = doc.assets[1];
assert.equal(text.slice(mesh.idValueStart, mesh.idValueEnd), "Model_SKN.GUN");
});
test("handles > and / inside quoted values, self-closing tags and comments", () => {
const text = `<?xml version="1.0"?>
<!-- a > comment with < inside -->
<AssetDeclaration xmlns="uri:ea.com:eala:asset">
<W3DContainer id="Weird" Description="a > b < c" />
<W3DMesh id="X" VertexData="a / b"/>
</AssetDeclaration>`;
const doc = scanXmlShallow(text);
assert.equal(doc.errors.length, 0);
assert.deepEqual(
doc.assets.map((a) => a.id),
["Weird", "X"],
);
});
test("skips CDATA and processing instructions", () => {
const text = `<AssetDeclaration>
<!-- <W3DMesh id="FAKE"/> -->
<![CDATA[ <W3DMesh id="FAKE2"/> ]]>
<?xml-stylesheet href="x"?>
<W3DContainer id="REAL"/>
</AssetDeclaration>`;
const doc = scanXmlShallow(text);
assert.equal(doc.errors.length, 0);
assert.deepEqual(
doc.assets.map((a) => a.id),
["REAL"],
);
});
test("reports unterminated tags and keeps partial results", () => {
const doc = scanXmlShallow('<AssetDeclaration><W3DMesh id="A"/><W3DMesh id="B"');
assert.ok(doc.errors.length >= 1);
assert.deepEqual(
doc.assets.map((a) => a.id),
["A"],
);
});
test("nested module payload does not create asset records", () => {
// Only top-level elements with an id are assets; the hundreds of thousands
// of numeric V/T elements inside a mesh are not.
const doc = scanXmlShallow(`<AssetDeclaration>
<W3DMesh id="MESH">
<Vertices><V X="1" Y="2" Z="3"/><V X="4" Y="5" Z="6"/></Vertices>
<Triangles><T A="0" B="1" C="2"/></Triangles>
</W3DMesh>
</AssetDeclaration>`);
assert.deepEqual(
doc.assets.map((a) => a.id),
["MESH"],
);
});
test("distinguishes root-level and nested xi:include", () => {
const doc = scanXmlShallow(`<AssetDeclaration>
<xi:include href="DATA:Top.xml"/>
<W3DMesh id="MESH">
<SubObject>
<RenderObject>
<xi:include href="DATA:Nested.xml" xpointer="xmlns(n=uri:ea.com:eala:asset) xpointer(/n:X/child::*)"/>
</RenderObject>
</SubObject>
</W3DMesh>
</AssetDeclaration>`);
assert.deepEqual(
doc.rootXiIncludes.map((x) => x.href),
["DATA:Top.xml"],
);
assert.deepEqual(
doc.nestedXiIncludes.map((x) => x.href),
["DATA:Nested.xml"],
);
});
+112 -1
View File
@@ -1,6 +1,16 @@
import { test } from "node:test"; import { test } from "node:test";
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { parseXml, findElementAt } from "../out/language/xmlParser.js"; import {
parseXml,
findElementAt,
stripBom,
textContentTokenAt,
} from "../out/language/xmlParser.js";
test("stripBom removes a leading UTF-8 byte-order mark", () => {
assert.equal(stripBom("\uFEFF<A/>"), "<A/>");
assert.equal(stripBom("<A/>"), "<A/>");
});
test("parses elements, attributes and positions", () => { test("parses elements, attributes and positions", () => {
const text = `<AssetDeclaration xmlns="uri:ea.com:eala:asset">\n\t<GameObject id="X" KindOf="A B"/>\n</AssetDeclaration>`; const text = `<AssetDeclaration xmlns="uri:ea.com:eala:asset">\n\t<GameObject id="X" KindOf="A B"/>\n</AssetDeclaration>`;
@@ -39,6 +49,27 @@ test("findElementAt returns innermost element", () => {
assert.equal(at.name, "C"); assert.equal(at.name, "C");
}); });
test("findElementAt treats a completed element's end as exclusive", () => {
const text = `<A><B>X</B></A>`;
const doc = parseXml(text);
const b = doc.elements.find((e) => e.name === "B");
const at = findElementAt(doc, b.end);
assert.equal(at?.name, "A", "cursor after </B> belongs to the parent");
const self = `<A><B/></A>`;
const selfDoc = parseXml(self);
const b2 = selfDoc.elements.find((e) => e.name === "B");
const atSelf = findElementAt(selfDoc, b2.end);
assert.equal(atSelf?.name, "A", "cursor after <B/> belongs to the parent");
});
test("findElementAt still includes EOF inside an unclosed element", () => {
const text = `<A><B>C`;
const doc = parseXml(text);
const b = doc.elements.find((e) => e.name === "B");
assert.equal(findElementAt(doc, text.length)?.name, "B");
});
test("tolerates partial input while typing", () => { test("tolerates partial input while typing", () => {
const text = `<AssetDeclaration>\n\t<GameObject id="TestTank" Com`; const text = `<AssetDeclaration>\n\t<GameObject id="TestTank" Com`;
const doc = parseXml(text); const doc = parseXml(text);
@@ -62,6 +93,16 @@ test("recovers from an unterminated attribute value at end of line", () => {
assert.ok(b.start > a.start); assert.ok(b.start > a.start);
}); });
test("marks recovered start tags for partial completion re-parsing", () => {
const text = `<AssetDeclaration>\n <A x="1" y="abc\n <B/>\n </A>\n</AssetDeclaration>`;
const doc = parseXml(text);
const a = doc.elements.find((e) => e.name === "A");
assert.equal(a.recoveredStartTag, true);
// Well-formed elements parsed normally do not carry the marker.
const b = doc.elements.find((e) => e.name === "B");
assert.equal(b.recoveredStartTag, undefined);
});
test("reports an unterminated attribute value at EOF", () => { test("reports an unterminated attribute value at EOF", () => {
const text = `<A x="abc`; const text = `<A x="abc`;
const doc = parseXml(text); const doc = parseXml(text);
@@ -71,3 +112,73 @@ test("reports an unterminated attribute value at EOF", () => {
assert.equal(a.attrs[0].value, "abc"); assert.equal(a.attrs[0].value, "abc");
assert.equal(a.attrs[0].quoteEnd, -1); assert.equal(a.attrs[0].quoteEnd, -1);
}); });
test("textContentTokenAt returns the token inside element content", () => {
const text =
`<AssetDeclaration>` +
`<CreateObject> CrateDebris_01 </CreateObject>` +
`</AssetDeclaration>`;
const doc = parseXml(text);
const el = doc.elements.find((e) => e.name === "CreateObject");
const tokenStart = text.indexOf("Crate");
const cursor = tokenStart + 3;
const token = textContentTokenAt(text, el, cursor);
assert.deepEqual(token, {
value: "CrateDebris_01",
start: tokenStart,
end: tokenStart + "CrateDebris_01".length,
});
// The start tag, closing tag and whitespace-only content are not tokens.
assert.equal(textContentTokenAt(text, el, el.start + 1), null);
assert.equal(textContentTokenAt(text, el, text.indexOf("</CreateObject") + 1), null);
const empty = `<A><B/></A><C> </C>`;
const c = parseXml(empty).elements.find((e) => e.name === "C");
assert.equal(textContentTokenAt(empty, c, c.startTagEnd + 1), null);
});
test("textContentTokenAt works before the closing tag is typed", () => {
const text = `<A><B>C`;
const doc = parseXml(text);
const b = doc.elements.find((e) => e.name === "B");
assert.ok(b);
assert.equal(b.closeTagStart, -1);
const token = textContentTokenAt(text, b, text.length);
assert.deepEqual(token, {
value: "C",
start: text.indexOf("C"),
end: text.length,
});
});
test("a typed < in content followed by a closing tag does not swallow it", () => {
// In a real file the "<" the user just typed is followed by
// "</CreateObject>" on the next line. The parser must NOT treat that
// closing tag's ">" as the end of the malformed start tag (which would
// create a bogus empty-name element and break content completion).
const text =
`<AssetDeclaration>` +
`<ObjectCreationList>` +
`<CreateObject>\n\t<Offset/>\n\t<\n</CreateObject>` +
`</ObjectCreationList>` +
`</AssetDeclaration>`;
const doc = parseXml(text);
assert.ok(doc.errors.some((e) => /Unterminated start tag/.test(e.message)));
assert.ok(!doc.elements.some((e) => e.name === ""), "no bogus empty-name element");
const co = doc.elements.find((e) => e.name === "CreateObject");
assert.ok(co, "outer CreateObject still parsed");
assert.equal(co.end, text.indexOf("</CreateObject>") + "</CreateObject>".length);
});
test("a partial child name in content is recovered, not glued to the closing tag", () => {
const text = `<A><B>\n\t<Cr\n</B></A>`;
const doc = parseXml(text);
const cr = doc.elements.find((e) => e.name === "Cr");
assert.ok(cr, "partial name is recovered as an element shell");
assert.equal(cr.recoveredStartTag, true);
// The mismatched closing tag later closes the recovered shell (parser
// recovery), so the element stays a valid container for completion.
assert.equal(cr.closeTagStart, text.indexOf("</B>"));
assert.ok(cr.end > cr.startTagEnd);
const b = doc.elements.find((e) => e.name === "B");
assert.equal(b.end, text.length);
});