修代码补全

README.md

@@ -4,18 +4,19 @@
 
 ## 功能
 
-- **语法高亮**：在普通 XML 高亮之上叠加领域标记（`$DEFINE` 常量、`inheritFrom`、`xai:joinAction`、结构标签）。
+- **语法高亮**：在普通 XML 高亮之上叠加领域标记（`$DEFINE` 常量、`inheritFrom`、`xai:joinAction`、结构标签）；XML 语法异常（如未闭合引号）期间由语义 token 兜底，标签/属性/值着色不中断。
 - **自动补全**：
-  - 元素名：按当前父元素的 XSD 模型补全子元素；顶层资产（`AssetDeclaration` 内）补全 `GameObject`、`WeaponTemplate` 等 99+ 类型。
+  - 元素名：按当前父元素的 XSD 模型补全子元素；顶层资产（`AssetDeclaration` 内）补全 `GameObject`、`WeaponTemplate` 等 295 种类型。
   - 属性名：必填属性优先，附带类型/文档/默认值；自动提示 `xai:joinAction` 与 `xmlns:xai`。
   - 属性值：
     - 引用型属性（如 `CommandSet`、`Weapon`）按 `xas:refType` 补全对应类型的资产 ID（**同名 ID 只补全匹配类型**）；
     - `inheritFrom` 补全可继承的资产 ID；
-    - 枚举（如 `Include type`）、布尔值、`$DEFINE` 常量；
+    - 枚举与位标志列表（如 `Include type`、`LocomotorTemplate@Surfaces`、`KindOf`；列表值支持空格后继续补全下一项）；
+    - 布尔值、`$DEFINE` 常量；
     - `<Include source>` 补全可解析的 `DATA:` / `ART:` / `AUDIO:` 与项目相对路径。
-- **悬停提示**：元素/属性显示 XSD 文档、类型、必填/默认值；引用值显示定义位置；`$DEFINE` 显示值与定义位置。
+- **悬停提示**：元素/属性显示 XSD 文档、类型、必填/默认值；引用值显示定义位置；`$DEFINE` 显示值与定义位置；`Include source` / `xi:include href` 显示解析后的目标文件；`xi:include` 元素与属性给出 XInclude 说明。
-- **引用导航**：从引用值（`CommandSet="..."`、`Weapon="..."`、`inheritFrom`）跳转到定义（严格按引用类型过滤）；`Ctrl+点击` Include 打开目标文件；Find All References 搜索整个工作区；文档大纲列出顶层资产与 `$DEFINE`。
+- **引用导航**：从引用值（`CommandSet="..."`、`Weapon="..."`、`inheritFrom`）跳转到定义（严格按引用类型过滤，候选由 `ra3modxml.definitionMode` 控制：`all` 列出 mod + 原版、`project-only` 优先项目内定义）；`Ctrl+点击` Include / `xi:include href` 打开目标文件；Find All References 搜索整个工作区；文档大纲列出顶层资产与 `$DEFINE`。
-- **错误检查**：XML 格式错误、未知元素/属性、顶层资产缺 `id`、重复 ID、未解析引用（含类型不匹配）、Include 找不到、`$DEFINE` 未定义。
+- **错误检查**：XML 格式错误、未知元素/属性（`xi:` 等外来命名空间不误报）、顶层资产缺 `id`、重复 ID、未解析引用（含类型不匹配）、Include / 嵌套 `xi:include` 目标找不到、`$DEFINE` 未定义。
 - **manifest 支持**：`<Include type="reference">` 指向的 `static/global/audio.manifest`（SDK `builtmods`）会被解析，manifest 中的原版资产 ID 可用于补全/悬停/导航/诊断。
 
 ## 使用
@@ -32,6 +33,7 @@
 | `ra3modxml.indexSageXml` | `true` | 是否索引 SDK 的 `SageXml` 原版源码 |
 | `ra3modxml.reportUnresolvedReferences` | `warning` | 未解析引用诊断级别（`warning`/`information`/`none`） |
 | `ra3modxml.diagnoseUnknownElements` | `true` | 是否报告未知元素/属性（自定义 XSD 项目可关闭） |
+| `ra3modxml.definitionMode` | `all` | 跳转候选：`all` 列出 mod 定义与原版定义（mod 优先）；`project-only` 仅在项目内已有定义时直接跳转 mod 定义 |
 | `ra3modxml.additionalDataSearchPaths` | `[]` | 追加的 `DATA:` 搜索目录 |
 
 ### 命令
@@ -57,15 +59,21 @@ npm run package          # 生成可安装的 .vsix
 src/
   extension.ts            激活入口与 provider 注册
   workspace.ts            项目检测、索引生命周期、状态栏
-  language/xmlParser.ts   带源码偏移的轻量 XML 解析器
+  settings.ts             配置读取（sdkPath、definitionMode 等）
-  language/context.ts     补全上下文分析
+  language/
-  model/schemaModel.ts    XSD 模型运行时（由 tools 生成 JSON 驱动）
+    xmlParser.ts          带源码偏移的轻量 XML 解析器（容错、行尾恢复）
+    context.ts            补全上下文分析
+    typeContext.ts        上下文感知元素类型解析
+    semanticTokens.ts     语义 token 兜底高亮（纯 TS）
+  model/
+    schemaModel.ts        XSD 模型运行时（schema-model.json / asset-types.json 由 tools 生成）
   indexer/
     includeResolver.ts    Include 路径解析（纯 TS，移植 check_duplicate_ids.py）
     manifestParser.ts     .manifest 二进制解析（移植 OpenSAGE ManifestFile.cs）
+    fileScanner.ts        目录扫描与 Include source 候选
     refs.ts               引用目标解析（按引用类型过滤）
     indexer.ts            工作区索引器（后台、缓存、增量重建）
-  features/               completion / hover / navigation / diagnostics
+  features/               completion / hover / navigation / diagnostics / semanticTokens
 syntaxes/                 TextMate 注入语法
 tools/                    XSD → 模型、AssetType 枚举提取
 ```
@@ -76,4 +84,5 @@ tools/                    XSD → 模型、AssetType 枚举提取
 
 - 领域说明与需求：`docs/requirements.md`
 - 调研与设计决策：`docs/plan.md`
+- 问题分析与修复记录：`docs/analysis-issues.md`
 - Manifest 格式参考：OpenSAGE `src/OpenSage.Game/Data/StreamFS/ManifestFile.cs`（本仓库 `OpenSAGE/` 子目录，commit `d45d361`）

docs/analysis-issues.md

@@ -454,3 +454,139 @@ attr href: known=false -> would flag unknown-attribute: true
 > GameObject 连同 include 进来的兄弟模块都在同一棵树里，`AttachModuleId` 等模块引用才能
 > 静态判定。设计备忘（现状 / 逻辑树方案 / 展开范围 / 落地点 / 检查清单）已整理在
 > `docs/plan.md` 第六节，等待确认后作为下一阶段实现。
+
+---
+
+## 十一、问题分析（第六轮，2026-08-01）：`Surfaces="` 未闭合引号导致枚举补全失效
+
+### 现象
+
+在 `<Locomotor ...>` 的起始标签里输入 `Surfaces="`（引号尚未闭合）时：
+
+1. 光标处不出 `LocomotorSurfaceBitFlags` 的枚举补全（GROUND、WATER 等）；
+2. 整个文件高亮退化（XML 变成“不合法”的观感），直到补上第二个引号才恢复。
+
+### 根因（两个独立缺陷叠加）
+
+**A. 上下文分析不认未闭合的引号（`src/language/context.ts`）**
+
+复现证据（编译产物直接执行）：
+
+```
+闭合引号:  <Locomotor id="x" Surfaces="GROUND">…
+ctx.kind = attribute-value, attr = Surfaces, valuePrefix = "GROUND"
+
+未闭合引号: <Locomotor id="x" Surfaces="GROUND>…
+Surfaces: quoteStart=27, quoteEnd=-1   ← 解析器吞掉整个文件
+ctx.kind = attribute-name              ← 补全走错分支
+```
+
+`analyzeStartTag` 判断属性值上下文的条件是 `offset >= quoteStart && offset <= quoteEnd`，
+未闭合时 `quoteEnd = -1` 永远不成立，于是回退成 attribute-name。
+
+**B. 模型生成器不支持 `xs:list`（`tools/xsd-to-model.mjs`）**
+
+XSD 中 `LocomotorSurfaceBitFlags` 是：
+
+```xml
+<xs:simpleType name="LocomotorSurfaceBitFlags">
+  <xs:list itemType="Surface"></xs:list>
+</xs:simpleType>
+```
+
+而 `Surface` 才是真正带 11 个枚举值（GROUND、WATER、CLIFF、AIR…）的类型。生成器
+只读取 `restriction.enumeration`，list 层把枚举全部丢掉——所以**即使引号闭合，模型里
+该属性也没有任何候选值**。影响面：SDK XSD 共 79 个 `xs:list` 简单类型、317 处属性
+声明使用它们（`KindOfBitFlags`、`ObjectStatusBitFlags`、`WeaponFlagsBitFlags`、
+`ModelConditionBitFlags`、`BuildPlacementTypeBitFlags` 等），展开到继承后的模型条目
+共 890 个属性受影响。
+
+### 关于高亮丢失
+
+这是 TextMate XML 语法对“未闭合字符串”的正常行为：后续内容被当作字符串吞掉，直到
+遇到下一个引号或 EOF。与插件注入语法无关（注入部分只有 `$DEFINE`、`inheritFrom` 等
+少量规则），**即使改成 LSP 也不会自动消失**。真正的解法是语义 token
+（`DocumentSemanticTokensProvider`），本次未实施，列为可选后续。
+
+### 修复（第 1–4 项）
+
+1. **解析器行尾恢复**（`src/language/xmlParser.ts`）：起始标签扫描到 EOF 且引号仍未
+   闭合时，把标签在第一个换行处截断并继续解析主循环。未闭合引号只影响当前行，后面
+   的元素照常进入解析树，补全 / hover / 诊断不中断；解析错误仍照常上报
+   （`Unterminated start tag`）。
+2. **未闭合引号上下文**（`src/language/context.ts`）：`quoteEnd < 0` 时，
+   `offset >= quoteStart` 即视为 attribute-value 上下文，`valuePrefix` 照常取引号后
+   到光标处文本。
+3. **模型支持 `xs:list`**（`tools/xsd-to-model.mjs`）：`resolveTypeDescriptor` 解析
+   `xs:list` 的 `itemType`（属性形式或内联 simpleType），继承其枚举值 / refType /
+   isRef / allowsDefine，新增 `isList` 标记；重新生成 `schema-model.json`
+   （`LocomotorSurfaceBitFlags` 恢复 11 个枚举值，`ModelConditionBitFlags` 457 个
+   值与 XSD 一致）。`AttributeInfo` / `SimpleTypeInfo` 接口同步新增 `isList`。
+4. **多值补全按“最后一段”过滤**（`src/features/completion.ts` +
+   `context.splitListValuePrefix`）：list 属性只取当前空格段做前缀过滤，替换范围只
+   覆盖该段——`Surfaces="GROUND ` 之后输入 `W` 也能提示 WATER / WALL_RAILING，而不是
+   用整段前缀匹配失败。
+
+### 验证
+
+- 未闭合引号复现场景：仅报 1 条 `Unterminated start tag`，`<Other/>` 等后续元素仍被
+  解析；`ctx.kind = attribute-value`、`attr = Surfaces`、`valuePrefix = "GROUND"`。
+- 模拟补全过滤：前缀 `G` → `GROUND`；前缀 `W` → `WATER, WALL_RAILING`；
+  `GROUND ` 后输入 `W` → `WATER, WALL_RAILING`。
+- `GameObject@KindOf`：`isList=true`、284 个枚举值。
+- 单元测试 **39 → 50 全部通过**（新增 `test/context.test.mjs` 与带 vscode stub 的
+  `test/completion.test.mjs` 集成用例；xmlParser 新增未闭合引号恢复 / EOF 用例；
+  schemaModel 新增 list 枚举与 `isList` 用例）。
+- `tsc` / `esbuild` 构建通过。
+
+> 补充：真实文件中该场景的元素名是 `<LocomotorTemplate ...>`（AttachTest
+> `Locomotor.xml` 实测，`Surfaces="GROUND CRUSHABLE_OBSTACLE"` 正是多值 list）；
+> SDK XSD 中没有名为 `Locomotor` 的元素，补全集成测试按真实写法夹具。
+
+### 后续可做（未列入本次）
+
+- `AssetIdList` 等“任意资产 ID 列表”的引用语义建模（list 补全框架已就绪，但这类
+  属性在 XSD 里没有 refType，需要另行定义过滤规则）。
+
+> 语义 token 兜底高亮已在第七轮实现（见下节）。
+
+---
+
+## 十二、问题分析（第七轮，2026-08-01）：语义 token 兜底高亮
+
+### 目标
+
+未闭合引号期间 TextMate 把后续内容当字符串吞掉、整个文件高亮退化，这是 XML 语法
+固有的行为（任何 XML 编辑器皆然），也无法靠注入 grammar 修复。本轮的解法是语义
+token：文档出现解析错误时，由插件用自己的容错解析树继续给标签 / 属性 / 值着色。
+
+### 设计
+
+- **纯 TS 核心**（`src/language/semanticTokens.ts`）：`buildSemanticTokenRanges(doc, text)`
+  把解析树转换成按位置排序的 `{ line, startChar, length, tokenType }`；token 类型只用
+  标准 `type` / `property` / `string`，所有主题自带配色，无需额外贡献样式。
+  - 元素名：起始标签与闭合标签各一个 `type` token；
+  - 属性名：`property` token；
+  - 属性值：`string` token，闭合时含两端引号，未闭合时从开引号到行尾恢复点
+    （如 `"GROUND>`）。
+- **provider**（`src/features/semanticTokens.ts`）：`parseXml` 后若
+  `doc.errors.length === 0` 直接返回空——合法文件观感与纯 TextMate 完全一致；
+  有解析错误时才用 `SemanticTokensBuilder` 编码输出。
+- **注册**：`extension.ts` 对 `xml` 语言注册 `DocumentSemanticTokensProvider`，
+  legend 与 provider 共用同一实例。
+
+### 验证
+
+- malformed（`Surfaces="GROUND>` 未闭合）：`AssetDeclaration` / `LocomotorTemplate` /
+  `<Other/>` 标签名、`id` / `Surfaces` 属性名、`"x"` 与 `"GROUND>` 值均有 token，
+  且按位置升序排列；
+- 合法文档：返回空 token 数组；
+- 单元测试 **50 → 53 全部通过**（新增 `test/semanticTokens.test.mjs`，纯函数 + vscode
+  stub 的 provider 集成）；`tsc` 通过。
+
+### 边界与取舍
+
+- 语义 token 只在解析报错时启用，且使用主题对 `type` / `property` / `string` 的默认
+  配色，可能与 TextMate XML 配色略有差异——只在打字过程中出现，可接受；
+- 未实现 `provideDocumentSemanticTokensEdits`（delta 版本），每次全量计算；单文件
+  解析在 KB 级，开销可忽略。

docs/plan.md

@@ -61,7 +61,7 @@
 - **核心与编辑器解耦**：`language/`、`model/`、`indexer/` 为纯 TS 模块（不 import `vscode`），可单测与复用（呼应 P1 需求 7）。
 - **esbuild 打包**，产物 `dist/extension.js`。
 - **XSD → JSON 模型**：开发期工具 `tools/xsd-to-model.mjs` 把 821 个 XSD 解析成 `schema-model.json`（元素树、属性、文档、枚举、引用类型映射），随插件发布；运行时不再解析 XSD。
-- **运行时 XML 解析**：自研带源码偏移的轻量解析器 `language/xmlParser.ts`（标签/属性/值均记录起止偏移，容错解析以支持输入中的补全与诊断）。`fast-xml-parser` 仅用于开发期 XSD 生成。
+- **运行时 XML 解析**：自研带源码偏移的轻量解析器 `language/xmlParser.ts`（标签/属性/值均记录起止偏移，容错解析以支持输入中的补全与诊断；未闭合引号在行尾恢复，避免吞掉整个文档）。`fast-xml-parser` 仅用于开发期 XSD 生成。
 - **AssetType 哈希表**：`tools/extract-asset-types.mjs` 从 OpenSAGE `AssetType.cs` 提取 `asset-types.json`；哈希未知时以 manifest 名称前缀推导类型。
 
 ### 架构
@@ -75,6 +75,7 @@ src/
     xmlParser.ts       带源码偏移的轻量 XML 解析器（格式错误定位、容错）
     context.ts         补全上下文分析（元素名/属性名/属性值/内容）
     typeContext.ts     上下文感知元素类型解析（resolveElementType 沿解析树逐层解析）
+    semanticTokens.ts  语义 token 兜底高亮（纯 TS：标签/属性/值范围，仅 malformed 时启用）
   model/
     schemaModel.ts     schema-model.json 的类型/属性/子元素查询 + 类型名规范化（纯 TS）
     schema-model.json  由 tools/xsd-to-model.mjs 生成
@@ -87,19 +88,21 @@ src/
     indexer.ts         工作区索引器（资产/Define/流/manifest 合并，LRU 解析缓存）
     types.ts           共享类型
   features/
-    completion.ts      补全 provider（元素/属性/值，上下文感知）
+    completion.ts      补全 provider（元素/属性/值，上下文感知；xs:list 多值按当前段过滤）
     hover.ts           hover provider
     navigation.ts      定义/引用/文档链接/大纲
     diagnostics.ts     实时诊断
+    semanticTokens.ts  语义 token provider（文档有解析错误时接管着色）
 syntaxes/
   ra3modxml.tmLanguage.json   注入 source.xml 的领域高亮（纯注入，不替换 XML 主语法）
 tools/
-  xsd-to-model.mjs       XSD → schema-model.json
+  xsd-to-model.mjs       XSD → schema-model.json（含 xs:list：继承 item 枚举/引用语义，isList 标记）
   extract-asset-types.mjs OpenSAGE AssetType.cs → asset-types.json
 test/
   fixtures/minimod      样例 Mod（include 各种情形、同名 ID、嵌套 xi:include、manifest 回退）
-  *.test.mjs            8 个测试文件（xmlParser / includeResolver / manifestParser /
+  *.test.mjs            11 个测试文件（xmlParser / context / completion / semanticTokens /
-                        indexer / schemaModel / refs / typeContext / manifestTypes）
+                        includeResolver / manifestParser / indexer / schemaModel / refs /
+                        typeContext / manifestTypes）
 ```
 
 ### 关键设计决策
@@ -114,6 +117,17 @@ test/
 8. **跳转精度**：XML 定义跳转到 `id` 属性值的精确 Range；manifest 定义映射到源码文件（如 SageXml）时也在文件内精确定位；找不到再回退到记录行。
 9. **嵌套 `xi:include`**：任意层级处理——目标缺失产生诊断、目标存在则纳入索引；根级 `xpointer` 容器内容按顶层资产索引。
 10. **性能**：索引在后台执行；解析结果 LRU 缓存（约 64 个文档）；文件保存后防抖全量重建（1.5s），重建期间的新请求标记脏并在完成后重跑；状态栏显示进度与统计。
+11. **`xs:list` 建模与多值补全**：list 简单类型继承 itemType 的枚举 / refType / isRef /
+    allowsDefine 并标记 `isList`（`LocomotorSurfaceBitFlags`、`KindOfBitFlags` 等 79 个
+    类型、317 处属性声明受益）；补全只对“最后一个空格段”过滤，替换范围只覆盖当前段，
+    支持 `Surfaces="GROUND ` 之后继续输入 `W` 提示 `WATER`。
+12. **未闭合引号的行尾恢复**：起始标签扫描到 EOF 且引号未闭合时，在第一个换行处截断
+    标签并继续解析，未闭合只影响当前行（仍上报 `Unterminated start tag`），后续元素
+    的补全 / hover / 诊断不中断。
+13. **语义 token 兜底高亮**：TextMate 对未闭合引号会把后续内容当字符串吞掉（任何
+    XML 编辑器皆然）；扩展注册 `DocumentSemanticTokensProvider`，仅当解析报错时用
+    语义 token 覆盖标签名 / 属性名 / 属性值（标准 token 类型 `type` / `property` /
+    `string`，主题自带配色）。合法文件返回空，观感与纯 TextMate 完全一致。
 
 ## 三、实施步骤
 
@@ -122,8 +136,10 @@ test/
 3. [x] 生成模型：`schema-model.json`（XSD，295 顶层元素 / 1851 类型）与 `asset-types.json`（79 个 AssetType 哈希）。
 4. [x] 纯 TS 核心：include 解析、manifest 解析、XML 解析封装、索引器、引用解析。
 5. [x] 功能层：补全、hover、导航、诊断、大纲、高亮 grammar。
-6. [x] 单测（fixture Mod，31 个用例全绿）+ 编译 + `vsce package` 打包（ra3-mod-xml-0.1.0.vsix，约 259KB）。
+6. [x] 单测（fixture Mod，53 个用例全绿：含 xs:list 枚举、未闭合引号恢复、list 多值
-7. [x] 在 AttachTest / GenEvoTest / Corona 上做冒烟验证，并按真实项目反馈修复问题（详见 `docs/analysis-issues.md` 五轮分析）。
+   分段、带 vscode stub 的补全集成、语义 token 兜底）+ 编译 + `vsce package` 打包
+   （ra3-mod-xml-0.1.0.vsix，约 495KB）。
+7. [x] 在 AttachTest / GenEvoTest / Corona 上做冒烟验证，并按真实项目反馈修复问题（详见 `docs/analysis-issues.md` 六轮分析）。
 
 ## 四、验证结果（实测）
 
@@ -133,7 +149,7 @@ test/
 | GenEvoTest | 24 文件 | ~1.8s | 35,392（manifest 35,322） | 项目 ID（alliedmcv 等）正确收录 |
 | Corona | 3,448 文件（+ 非 XML 资产路径） | ~54.5s | 55,305（manifest 35,322） | 3 个流、183 个 Define、0 诊断 |
 
-单元测试覆盖：XML 解析（自闭合/容错/偏移）、include 解析（BAB 顺序、SDK 根优先于 SageXml）、manifest 二进制解析（合成 v5 样本、类型/ID 推导）、索引器（资产/Define/流/缺失 include/嵌套 xi:include）、XSD 模型（上下文类型、`childTypeOf`、大小写规范化、属性级 refType、外来命名空间判定）、引用过滤（`Weapon="X"` 只跳 `WeaponTemplate`、模块 `id` 定义点、Poid 局部引用、`xi:include` 不校验、`Side="Allies"` 命中 manifest 的 `PlayerTemplate`）。
+单元测试覆盖：XML 解析（自闭合/容错/偏移/未闭合引号行尾恢复）、补全上下文（未闭合引号仍为 attribute-value、list 多值分段）、补全集成（vscode stub 下 `LocomotorTemplate@Surfaces` 未闭合引号枚举补全、空格后第二段过滤与替换范围）、语义 token（标签/属性/值范围、合法文档返回空、malformed 返回兜底 token）、include 解析（BAB 顺序、SDK 根优先于 SageXml）、manifest 二进制解析（合成 v5 样本、类型/ID 推导）、索引器（资产/Define/流/缺失 include/嵌套 xi:include）、XSD 模型（上下文类型、`childTypeOf`、大小写规范化、属性级 refType、外来命名空间判定、`xs:list` 枚举继承与 `isList` 标记）、引用过滤（`Weapon="X"` 只跳 `WeaponTemplate`、模块 `id` 定义点、Poid 局部引用、`xi:include` 不校验、`Side="Allies"` 命中 manifest 的 `PlayerTemplate`）。
 
 > 注：`D:\Mods\CoronaMod` 位于移动硬盘，当前未连接；GenEvoTest / Corona 的回归需在 D: 盘可用时补跑（用户会另行通知）。
 

package.json

@@ -57,7 +57,7 @@
             "none"
           ],
           "default": "warning",
-          "description": "Severity for attribute references (CommandSet=..., inheritFrom=...) that cannot be resolved in the index. IDs that only exist in compiled manifests cannot be resolved yet."
+          "description": "Severity for attribute references (CommandSet=..., inheritFrom=...) that cannot be resolved in the index. The index covers mod XML sources, SDK SageXml sources and compiled manifests."
         },
         "ra3modxml.diagnoseUnknownElements": {
           "type": "boolean",

src/extension.ts

@@ -9,6 +9,10 @@ import {
   Ra3ReferenceProvider,
 } from "./features/navigation";
 import { Ra3Diagnostics } from "./features/diagnostics";
+import {
+  Ra3SemanticTokensProvider,
+  RA3_SEMANTIC_TOKENS_LEGEND,
+} from "./features/semanticTokens";
 
 const XML_SELECTOR: vscode.DocumentSelector = [{ language: "xml" }];
 
@@ -55,6 +59,13 @@ export function activate(context: vscode.ExtensionContext): void {
       new Ra3DocumentSymbolProvider(),
     ),
   );
+  context.subscriptions.push(
+    vscode.languages.registerDocumentSemanticTokensProvider(
+      XML_SELECTOR,
+      new Ra3SemanticTokensProvider(),
+      RA3_SEMANTIC_TOKENS_LEGEND,
+    ),
+  );
 
   const diagnostics = new Ra3Diagnostics(ws);
   context.subscriptions.push(diagnostics);

src/features/completion.ts

@@ -1,6 +1,10 @@
 import * as vscode from "vscode";
 import { parseXml, type XmlElement } from "../language/xmlParser";
-import { analyzeContext, type CompletionContext } from "../language/context";
+import {
+  analyzeContext,
+  splitListValuePrefix,
+  type CompletionContext,
+} from "../language/context";
 import { resolveElementType } from "../language/typeContext";
 import * as model from "../model/schemaModel";
 import { isLocalReferenceAttribute } from "../indexer/refs";
@@ -178,12 +182,31 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
     const el = ctx.element;
     const attr = ctx.attr;
     if (!el || !attr) return [];
-    const prefix = ctx.valuePrefix;
+    const rawPrefix = ctx.valuePrefix;
 
-    const endOffset = attr.quoteEnd > attr.valueEnd ? attr.valueEnd : document.offsetAt(position);
+    const attrName = attr.name.toLowerCase();
+    const elType = resolveElementType(el);
+    const attrInfo = model
+      .attributesOfType(elType)
+      .find((a) => a.name.toLowerCase() === attrName);
+
+    // xs:list values (bit flags such as Surfaces="GROUND WATER") are
+    // whitespace-separated: only the token currently being edited is used for
+    // filtering, and the replacement range covers that token instead of the
+    // whole value.
+    const seg = attrInfo?.isList
+      ? splitListValuePrefix(rawPrefix)
+      : { token: rawPrefix, start: 0 };
+    const prefix = seg.token;
+
+    const valueStartOffset =
+      attr.valueStart >= 0 ? attr.valueStart : document.offsetAt(position);
+    const endOffset =
+      attr.quoteEnd > attr.valueEnd ? attr.valueEnd : document.offsetAt(position);
+    const rangeStart = valueStartOffset + seg.start;
     const valueRange = new vscode.Range(
-      document.positionAt(attr.valueStart),
+      document.positionAt(rangeStart),
-      document.positionAt(Math.max(attr.valueStart, endOffset)),
+      document.positionAt(Math.max(rangeStart, endOffset)),
     );
 
     const make = (
@@ -201,7 +224,6 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
     };
 
     const isInclude = el.name === "Include";
-    const attrName = attr.name.toLowerCase();
 
     // Include type / source
     if (isInclude && attrName === "type") {
@@ -218,11 +240,6 @@ export class Ra3CompletionProvider implements vscode.CompletionItemProvider {
       );
     }
 
-    const elType = resolveElementType(el);
-    const attrInfo = model
-      .attributesOfType(elType)
-      .find((a) => a.name.toLowerCase() === attrName);
-
     // inheritFrom: same element type first, then everything.
     if (attrName === "inheritfrom") {
       return this.assetIdItems(idx, el.name, null, prefix, make);

src/features/semanticTokens.ts

@@ -0,0 +1,42 @@
+import * as vscode from "vscode";
+import { parseXml } from "../language/xmlParser";
+import { buildSemanticTokenRanges } from "../language/semanticTokens";
+
+const TOKEN_TYPES = ["type", "property", "string"] as const;
+
+export const RA3_SEMANTIC_TOKENS_LEGEND = new vscode.SemanticTokensLegend([
+  ...TOKEN_TYPES,
+]);
+
+/**
+ * Highlighting fallback for malformed XML.
+ *
+ * While the document is well-formed, the built-in TextMate XML grammar colors
+ * it as usual and this provider returns no tokens, so nothing changes. When
+ * parsing reports errors (e.g. an attribute value whose closing quote has not
+ * been typed yet), the TextMate structure is lost, and these semantic tokens
+ * keep element names, attribute names and values colored.
+ */
+export class Ra3SemanticTokensProvider
+  implements vscode.DocumentSemanticTokensProvider
+{
+  async provideDocumentSemanticTokens(
+    document: vscode.TextDocument,
+    _token: vscode.CancellationToken,
+  ): Promise<vscode.SemanticTokens> {
+    const text = document.getText();
+    const doc = parseXml(text);
+    if (doc.errors.length === 0) {
+      return new vscode.SemanticTokens(new Uint32Array(0));
+    }
+    const ranges = buildSemanticTokenRanges(doc, text);
+    const builder = new vscode.SemanticTokensBuilder(RA3_SEMANTIC_TOKENS_LEGEND);
+    for (const r of ranges) {
+      builder.push(
+        new vscode.Range(r.line, r.startChar, r.line, r.startChar + r.length),
+        r.tokenType,
+      );
+    }
+    return builder.build();
+  }
+}

src/language/context.ts

@@ -76,7 +76,15 @@ function analyzeStartTag(
 
   // Inside an attribute value?
   for (const attr of el.attrs) {
-    if (attr.hasValue && offset >= attr.quoteStart && offset <= attr.quoteEnd) {
+    // An unterminated value (quoteEnd < 0) happens while the user is typing
+    // the opening quote of a new attribute value; it must still be treated as
+    // an attribute-value context so enum/ref/define completions show up.
+    if (
+      attr.hasValue &&
+      attr.quoteStart >= 0 &&
+      offset >= attr.quoteStart &&
+      (attr.quoteEnd < 0 || offset <= attr.quoteEnd)
+    ) {
       const start = attr.valueStart;
       const prefix = offset > start ? text.slice(start, offset) : "";
       return {
@@ -128,3 +136,20 @@ function empty(kind: ContextKind): CompletionContext {
     existingAttrs: [],
   };
 }
+
+/**
+ * Splits the typed prefix of a whitespace-separated list value (xs:list, e.g.
+ * bit flags such as Surfaces="GROUND WATER") into the token being edited and
+ * the offset of that token inside the prefix.
+ *
+ * "GROUND WA" -> { token: "WA", start: 7 }
+ * "GROUND "   -> { token: "",  start: 7 }
+ * "WA"        -> { token: "WA", start: 0 }
+ */
+export function splitListValuePrefix(prefix: string): { token: string; start: number } {
+  let start = prefix.length;
+  while (start > 0 && !/\s/.test(prefix[start - 1])) {
+    start--;
+  }
+  return { token: prefix.slice(start), start };
+}

src/language/semanticTokens.ts

@@ -0,0 +1,63 @@
+import { LineMap, type XmlDocument } from "./xmlParser";
+
+/**
+ * Semantic token types used by the highlighting fallback. They are standard
+ * vscode token types, so every theme already has colors for them.
+ */
+export type SemanticTokenType = "type" | "property" | "string";
+
+export interface SemanticTokenRange {
+  line: number;
+  startChar: number;
+  length: number;
+  tokenType: SemanticTokenType;
+}
+
+/**
+ * Builds semantic token ranges from a tolerant parse tree.
+ *
+ * This is the highlighting fallback for malformed XML: while the TextMate
+ * grammar loses structure (e.g. an attribute value whose closing quote has
+ * not been typed yet turns the rest of the file into one string), semantic
+ * tokens keep element names, attribute names and attribute values colored.
+ * The ranges are sorted by position for the vscode encoder.
+ */
+export function buildSemanticTokenRanges(
+  doc: XmlDocument,
+  text: string,
+): SemanticTokenRange[] {
+  const lineMap = new LineMap(text);
+  const out: SemanticTokenRange[] = [];
+
+  const push = (offset: number, length: number, tokenType: SemanticTokenType) => {
+    if (length <= 0 || offset < 0 || offset + length > text.length) return;
+    const pos = lineMap.positionAt(offset);
+    out.push({
+      line: pos.line,
+      startChar: pos.character,
+      length,
+      tokenType,
+    });
+  };
+
+  for (const el of doc.elements) {
+    // Element name in the start tag.
+    push(el.start + 1, el.name.length, "type");
+    // Element name in the closing tag (when present).
+    if (el.closeTagStart >= 0) {
+      push(el.closeTagStart + 2, el.name.length, "type");
+    }
+    for (const attr of el.attrs) {
+      push(attr.nameStart, attr.name.length, "property");
+      if (!attr.hasValue) continue;
+      // Include the surrounding quotes when available; for an unterminated
+      // value quoteEnd is -1 and the token ends at the recovered value end.
+      const start = attr.quoteStart >= 0 ? attr.quoteStart : attr.valueStart;
+      const end = attr.quoteEnd >= 0 ? attr.quoteEnd : attr.valueEnd;
+      push(start, end - start, "string");
+    }
+  }
+
+  out.sort((a, b) => a.line - b.line || a.startChar - b.startChar);
+  return out;
+}

src/language/xmlParser.ts

@@ -336,7 +336,16 @@ export function parseXml(text: string): XmlDocument {
     const gt = findTagEnd(text, i + 1);
     if (gt < 0) {
       err("Unterminated start tag", i);
-      const content = text.slice(i + 1);
+      // Recovery while typing: an attribute value whose closing quote has not
+      // been typed yet makes the scanner run to EOF. End the malformed start
+      // tag at the first line break (or EOF) so the rest of the document is
+      // still parsed and completion/hover keep working for the elements after
+      // the broken tag. The missing quote/tag end is still reported above.
+      let recoverTo = i + 1;
+      while (recoverTo < text.length && text[recoverTo] !== "\n" && text[recoverTo] !== "\r") {
+        recoverTo++;
+      }
+      const content = text.slice(i + 1, recoverTo);
       const raw = parseTag(content, i + 1);
       if (raw.name) {
         const el = buildElement(raw, stack.length);
@@ -344,7 +353,8 @@ export function parseXml(text: string): XmlDocument {
         root = root ?? el;
         stack.push(el);
       }
-      break;
+      i = recoverTo + 1;
+      continue;
     }
     const content = text.slice(i + 1, gt);
     const raw = parseTag(content, i + 1);

src/model/schemaModel.ts

@@ -20,6 +20,8 @@ export interface AttributeInfo {
   /** True for reference-typed attributes whose simple type has no refType. */
   isRef: boolean;
   enumValues: string[];
+  /** True for xs:list types (whitespace-separated bit flags / lists). */
+  isList: boolean;
   allowsDefine: boolean;
   isBoolean: boolean;
   base: string | null;
@@ -39,6 +41,7 @@ export interface SimpleTypeInfo {
   refType: string | null;
   isRef: boolean;
   enumValues: string[];
+  isList: boolean;
   allowsDefine: boolean;
   doc: string;
 }

test/completion.test.mjs

@@ -0,0 +1,157 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import { createRequire } from "node:module";
+
+// Minimal vscode shim so the compiled completion provider can run under plain
+// node. Only the APIs used by the completion call path are implemented.
+const CompletionItemKind = {
+  Field: 1,
+  Property: 2,
+  EnumMember: 3,
+  Value: 4,
+  Constant: 5,
+  File: 6,
+};
+
+class CompletionItem {
+  constructor(label, kind) {
+    this.label = label;
+    this.kind = kind;
+  }
+}
+
+class Position {
+  constructor(line, character) {
+    this.line = line;
+    this.character = character;
+  }
+}
+
+class Range {
+  constructor(start, end) {
+    this.start = start;
+    this.end = end;
+  }
+}
+
+class MarkdownString {
+  constructor(value) {
+    this.value = value ?? "";
+  }
+  appendMarkdown(text) {
+    this.value += text;
+    return this;
+  }
+  appendCodeblock(text) {
+    this.value += "\n```\n" + text + "\n```\n";
+    return this;
+  }
+}
+
+class SnippetString {
+  constructor(value) {
+    this.value = value;
+  }
+}
+
+const require = createRequire(import.meta.url);
+const Module = require("module");
+const origResolve = Module._resolveFilename;
+Module._resolveFilename = function (request, ...args) {
+  if (request === "vscode") return "vscode-stub";
+  return origResolve.call(this, request, ...args);
+};
+require.cache["vscode-stub"] = {
+  id: "vscode-stub",
+  filename: "vscode-stub",
+  loaded: true,
+  exports: {
+    CompletionItem,
+    CompletionItemKind,
+    Position,
+    Range,
+    MarkdownString,
+    SnippetString,
+  },
+};
+
+const { Ra3CompletionProvider } = require("../out/features/completion.js");
+
+function makeDocument(text) {
+  const lineStarts = [0];
+  for (let i = 0; i < text.length; i++) {
+    if (text.charCodeAt(i) === 10) lineStarts.push(i + 1);
+  }
+  return {
+    getText: () => text,
+    offsetAt: (pos) => lineStarts[pos.line] + pos.character,
+    positionAt: (offset) => {
+      let lo = 0;
+      let hi = lineStarts.length - 1;
+      while (lo < hi) {
+        const mid = (lo + hi + 1) >> 1;
+        if (lineStarts[mid] <= offset) lo = mid;
+        else hi = mid - 1;
+      }
+      return new Position(lo, offset - lineStarts[lo]);
+    },
+  };
+}
+
+// Enum completions do not consult the index, but provideCompletionItems only
+// routes value contexts when the workspace index is present.
+const provider = new Ra3CompletionProvider({ index: {} });
+const token = { isCancellationRequested: false };
+
+test("Surfaces enum completion works with an unclosed quote", async () => {
+  const text =
+    `<AssetDeclaration>\n  <LocomotorTemplate id="x" Surfaces="G>\n  <Other/>\n</LocomotorTemplate>\n</AssetDeclaration>`;
+  const line1 = text.split("\n")[1];
+  const line1Start = text.indexOf("\n") + 1;
+  const valueStart = text.indexOf('Surfaces="') + 'Surfaces="'.length;
+  const pos = new Position(1, line1.indexOf("G") + 1);
+
+  const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
+  const labels = items.map((i) => i.label);
+  assert.ok(labels.includes("GROUND"));
+  assert.ok(!labels.includes("WATER"));
+  assert.ok(items.every((i) => i.kind === CompletionItemKind.EnumMember));
+
+  // Replacement range covers only the value (start..cursor).
+  const ground = items.find((i) => i.label === "GROUND");
+  assert.equal(ground.range.start.character, valueStart - line1Start);
+  assert.equal(ground.range.end.character, pos.character);
+});
+
+test("list values filter on the token after whitespace", async () => {
+  const text =
+    `<AssetDeclaration>\n  <LocomotorTemplate id="x" Surfaces="GROUND W>\n  <Other/>\n</LocomotorTemplate>\n</AssetDeclaration>`;
+  const line1 = text.split("\n")[1];
+  const line1Start = text.indexOf("\n") + 1;
+  const valueStart = text.indexOf('Surfaces="') + 'Surfaces="'.length;
+  const pos = new Position(1, line1.indexOf("W") + 1);
+
+  const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
+  const labels = items.map((i) => i.label);
+  assert.ok(labels.includes("WATER"));
+  assert.ok(labels.includes("WALL_RAILING"));
+  assert.ok(!labels.includes("GROUND"));
+
+  // Replacement range covers only the second token, not "GROUND ".
+  const water = items.find((i) => i.label === "WATER");
+  assert.equal(water.range.start.character, valueStart + "GROUND ".length - line1Start);
+});
+
+test("empty unterminated value offers all enum values", async () => {
+  const text =
+    `<AssetDeclaration>\n  <LocomotorTemplate id="x" Surfaces=">\n  <Other/>\n</LocomotorTemplate>\n</AssetDeclaration>`;
+  const line1 = text.split("\n")[1];
+  const pos = new Position(1, line1.indexOf('Surfaces="') + 'Surfaces="'.length);
+
+  const items = await provider.provideCompletionItems(makeDocument(text), pos, token);
+  const labels = items.map((i) => i.label);
+  assert.equal(items.length, 11);
+  assert.ok(labels.includes("GROUND"));
+  assert.ok(labels.includes("WATER"));
+  assert.ok(labels.includes("CRUSHABLE_WALL"));
+});

test/context.test.mjs

@@ -0,0 +1,42 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import { parseXml } from "../out/language/xmlParser.js";
+import { analyzeContext, splitListValuePrefix } from "../out/language/context.js";
+
+test("unterminated quote is still an attribute-value context", () => {
+  const text = `<Locomotor id="x" Surfaces="GROUND>\n  <Other/>\n</Locomotor>`;
+  const cursor = text.indexOf("GROUND") + 6;
+  const doc = parseXml(text);
+  const ctx = analyzeContext(doc, text, cursor);
+  assert.equal(ctx.kind, "attribute-value");
+  assert.equal(ctx.attr?.name, "Surfaces");
+  assert.equal(ctx.valuePrefix, "GROUND");
+  assert.equal(ctx.element?.name, "Locomotor");
+});
+
+test("empty unterminated value keeps attribute-value context", () => {
+  const text = `<Locomotor id="x" Surfaces="\n</Locomotor>`;
+  const cursor = text.indexOf('Surfaces="') + 'Surfaces="'.length;
+  const doc = parseXml(text);
+  const ctx = analyzeContext(doc, text, cursor);
+  assert.equal(ctx.kind, "attribute-value");
+  assert.equal(ctx.attr?.name, "Surfaces");
+  assert.equal(ctx.valuePrefix, "");
+});
+
+test("closed quote still resolves value context", () => {
+  const text = `<Locomotor id="x" Surfaces="GROUND">\n</Locomotor>`;
+  const cursor = text.indexOf("GROUND") + 6;
+  const doc = parseXml(text);
+  const ctx = analyzeContext(doc, text, cursor);
+  assert.equal(ctx.kind, "attribute-value");
+  assert.equal(ctx.attr?.name, "Surfaces");
+  assert.equal(ctx.valuePrefix, "GROUND");
+});
+
+test("splitListValuePrefix isolates the token being edited", () => {
+  assert.deepEqual(splitListValuePrefix("GROUND WA"), { token: "WA", start: 7 });
+  assert.deepEqual(splitListValuePrefix("GROUND "), { token: "", start: 7 });
+  assert.deepEqual(splitListValuePrefix("WA"), { token: "WA", start: 0 });
+  assert.deepEqual(splitListValuePrefix(""), { token: "", start: 0 });
+});

test/schemaModel.test.mjs

@@ -53,6 +53,31 @@ test("Include has reference/instance/all enum", () => {
   assert.deepEqual(type?.enumValues, ["reference", "instance", "all"]);
 });
 
+test("xs:list types expose their item enum and isList flag", () => {
+  const expected = [
+    "GROUND", "WATER", "CLIFF", "AIR", "RUBBLE", "OBSTACLE", "IMPASSABLE",
+    "DEEP_WATER", "WALL_RAILING", "CRUSHABLE_OBSTACLE", "CRUSHABLE_WALL",
+  ];
+  const flags = model.typeInfo("LocomotorSurfaceBitFlags");
+  assert.equal(flags.isList, true);
+  assert.deepEqual(flags.enumValues, expected);
+  const surfaces = model
+    .attributesOfType("LocomotorTemplate")
+    .find((a) => a.name === "Surfaces");
+  assert.equal(surfaces.isList, true);
+  assert.deepEqual(surfaces.enumValues, expected);
+});
+
+test("numeric list types do not invent enums", () => {
+  const list = model.typeInfo("SageUnsignedIntList");
+  assert.equal(list.isList, true);
+  assert.deepEqual(list.enumValues, []);
+  // Plain restriction enums keep isList=false.
+  const plain = model.typeInfo("Surface");
+  assert.equal(plain.isList, false);
+  assert.ok(plain.enumValues.length > 0);
+});
+
 test("foreign namespaces are excluded from XSD validation", () => {
   // XInclude elements (xi:include / xi:fallback) come from the W3C XInclude
   // namespace, not from the EA asset XSD.

test/semanticTokens.test.mjs

@@ -0,0 +1,145 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import { createRequire } from "node:module";
+
+// Minimal vscode shim so the compiled semantic tokens provider can run under
+// plain node. Only the APIs used by the provider call path are implemented.
+class Range {
+  constructor(startLine, startCharacter, endLine, endCharacter) {
+    this.start = { line: startLine, character: startCharacter };
+    this.end = { line: endLine, character: endCharacter };
+  }
+}
+
+class SemanticTokensLegend {
+  constructor(tokenTypes) {
+    this.tokenTypes = tokenTypes;
+  }
+}
+
+class SemanticTokens {
+  constructor(data) {
+    this.data = data;
+  }
+}
+
+class SemanticTokensBuilder {
+  constructor(legend) {
+    this.legend = legend;
+    this.pushed = [];
+  }
+  push(range, tokenType) {
+    this.pushed.push({ range, tokenType });
+  }
+  build() {
+    return new SemanticTokens(new Uint32Array(this.pushed.length * 5));
+  }
+}
+
+const require = createRequire(import.meta.url);
+const Module = require("module");
+const origResolve = Module._resolveFilename;
+Module._resolveFilename = function (request, ...args) {
+  if (request === "vscode") return "vscode-stub";
+  return origResolve.call(this, request, ...args);
+};
+require.cache["vscode-stub"] = {
+  id: "vscode-stub",
+  filename: "vscode-stub",
+  loaded: true,
+  exports: {
+    Range,
+    SemanticTokens,
+    SemanticTokensBuilder,
+    SemanticTokensLegend,
+  },
+};
+
+const { parseXml } = require("../out/language/xmlParser.js");
+const { buildSemanticTokenRanges } = require("../out/language/semanticTokens.js");
+const { Ra3SemanticTokensProvider } = require("../out/features/semanticTokens.js");
+
+function lineStartsOf(text) {
+  const starts = [0];
+  for (let i = 0; i < text.length; i++) {
+    if (text.charCodeAt(i) === 10) starts.push(i + 1);
+  }
+  return starts;
+}
+
+function sliceAt(text, token) {
+  const starts = lineStartsOf(text);
+  const offset = starts[token.line] + token.startChar;
+  return text.slice(offset, offset + token.length);
+}
+
+function makeDocument(text) {
+  const lineStarts = lineStartsOf(text);
+  return {
+    getText: () => text,
+    offsetAt: (pos) => lineStarts[pos.line] + pos.character,
+    positionAt: (offset) => {
+      let lo = 0;
+      let hi = lineStarts.length - 1;
+      while (lo < hi) {
+        const mid = (lo + hi + 1) >> 1;
+        if (lineStarts[mid] <= offset) lo = mid;
+        else hi = mid - 1;
+      }
+      return { line: lo, character: offset - lineStarts[lo] };
+    },
+  };
+}
+
+const MALFORMED =
+  `<AssetDeclaration>\n  <LocomotorTemplate id="x" Surfaces="GROUND>\n  <Other/>\n</LocomotorTemplate>\n</AssetDeclaration>`;
+
+test("fallback tokens cover names, attributes and values", () => {
+  const doc = parseXml(MALFORMED);
+  assert.ok(doc.errors.length > 0);
+  const tokens = buildSemanticTokenRanges(doc, MALFORMED);
+
+  // Element names (opening and closing tags).
+  assert.ok(tokens.some((t) => t.tokenType === "type" && sliceAt(MALFORMED, t) === "AssetDeclaration"));
+  assert.ok(tokens.some((t) => t.tokenType === "type" && sliceAt(MALFORMED, t) === "LocomotorTemplate"));
+  assert.ok(tokens.some((t) => t.tokenType === "type" && sliceAt(MALFORMED, t) === "Other"));
+  assert.ok(
+    tokens.some(
+      (t) => t.tokenType === "type" && sliceAt(MALFORMED, t) === "LocomotorTemplate" && t.line === 3,
+    ),
+  );
+
+  // Attribute names.
+  assert.ok(tokens.some((t) => t.tokenType === "property" && sliceAt(MALFORMED, t) === "Surfaces"));
+  assert.ok(tokens.some((t) => t.tokenType === "property" && sliceAt(MALFORMED, t) === "id"));
+
+  // Attribute values: closed quotes include both quotes; the unterminated
+  // Surfaces value keeps the opening quote and everything up to the recovered
+  // line end (the `>` is still inside the unclosed string).
+  const strings = tokens.filter((t) => t.tokenType === "string").map((t) => sliceAt(MALFORMED, t));
+  assert.ok(strings.includes('"x"'));
+  assert.ok(strings.includes('"GROUND>'));
+
+  // Tokens are sorted by position for the vscode encoder.
+  for (let i = 1; i < tokens.length; i++) {
+    const a = tokens[i - 1];
+    const b = tokens[i];
+    assert.ok(a.line < b.line || (a.line === b.line && a.startChar <= b.startChar));
+  }
+});
+
+test("well-formed XML gets no semantic fallback tokens", async () => {
+  const text = `<AssetDeclaration>\n  <LocomotorTemplate id="x" Surfaces="GROUND"/>\n</AssetDeclaration>`;
+  const provider = new Ra3SemanticTokensProvider();
+  const result = await provider.provideDocumentSemanticTokens(makeDocument(text), {});
+  assert.equal(result.data.length, 0);
+});
+
+test("malformed XML gets semantic fallback tokens", async () => {
+  const provider = new Ra3SemanticTokensProvider();
+  const result = await provider.provideDocumentSemanticTokens(
+    makeDocument(MALFORMED),
+    {},
+  );
+  assert.ok(result.data.length > 0);
+});

test/xmlParser.test.mjs

@@ -45,3 +45,29 @@ test("tolerates partial input while typing", () => {
   assert.ok(doc.errors.length >= 1); // unclosed
   assert.equal(doc.elements.length, 2);
 });
+
+test("recovers from an unterminated attribute value at end of line", () => {
+  // Typing an attribute value quote without its closing quote makes the tag
+  // malformed; the parser must stop the broken start tag at the line break so
+  // the rest of the document (and completion for it) keeps working.
+  const text = `<AssetDeclaration>\n  <A x="1" y="abc\n  <B/>\n  </A>\n</AssetDeclaration>`;
+  const doc = parseXml(text);
+  assert.ok(doc.errors.some((e) => e.message === "Unterminated start tag"));
+  assert.deepEqual(doc.elements.map((e) => e.name), ["AssetDeclaration", "A", "B"]);
+  const a = doc.elements.find((e) => e.name === "A");
+  const y = a.attrs.find((at) => at.name === "y");
+  assert.equal(y.quoteEnd, -1);
+  assert.equal(text.slice(y.valueStart, y.valueEnd), "abc");
+  const b = doc.elements.find((e) => e.name === "B");
+  assert.ok(b.start > a.start);
+});
+
+test("reports an unterminated attribute value at EOF", () => {
+  const text = `<A x="abc`;
+  const doc = parseXml(text);
+  assert.ok(doc.errors.some((e) => /Unterminated start tag/.test(e.message)));
+  assert.equal(doc.elements.length, 1);
+  const a = doc.elements[0];
+  assert.equal(a.attrs[0].value, "abc");
+  assert.equal(a.attrs[0].quoteEnd, -1);
+});

tools/xsd-to-model.mjs

@@ -168,6 +168,7 @@ function resolveTypeDescriptor(typeName, memo = new Map()) {
       name,
       refType: null,
       enumValues: [],
+      isList: false,
       allowsDefine: false,
       isBoolean: name === "boolean",
       base: null,
@@ -180,6 +181,37 @@ function resolveTypeDescriptor(typeName, memo = new Map()) {
   const st = simpleTypes.get(name);
   if (st) {
     memo.set(name, null); // guard against cycles
+    const listNode = st?.list;
+    if (listNode) {
+      // xs:list types (bit flags such as LocomotorSurfaceBitFlags, AssetIdList,
+      // PercentageList, ...) serialize as whitespace-separated items. The
+      // usable enum/ref semantics come from the item type, so inherit them.
+      let itemDesc = null;
+      let inlineEnum = [];
+      let inlineAllowsDefine = false;
+      const itemTypeName = normalizeTypeName(listNode?.["@_itemType"]);
+      if (itemTypeName) {
+        itemDesc = resolveTypeDescriptor(itemTypeName, memo);
+      } else if (listNode.simpleType) {
+        inlineEnum = enumOfSimpleType(listNode.simpleType);
+        const pats = patternOfSimpleType(listNode.simpleType);
+        inlineAllowsDefine = pats ? pats.some(patternAllowsDefine) : false;
+      }
+      const desc = {
+        kind: "simple",
+        name,
+        refType: itemDesc?.refType ?? null,
+        enumValues: itemDesc?.enumValues?.length ? itemDesc.enumValues : inlineEnum,
+        isList: true,
+        allowsDefine: (itemDesc?.allowsDefine ?? false) || inlineAllowsDefine,
+        isRef: itemDesc?.isRef ?? false,
+        isBoolean: false,
+        base: itemDesc?.base ?? null,
+        doc: docOf(st),
+      };
+      memo.set(name, desc);
+      return desc;
+    }
     const base = normalizeTypeName(st?.restriction?.["@_base"]);
     const baseDesc = base ? resolveTypeDescriptor(base, memo) : null;
     const enumValues = enumOfSimpleType(st);
@@ -192,6 +224,7 @@ function resolveTypeDescriptor(typeName, memo = new Map()) {
       name,
       refType: normalizeTypeName(refType) || baseDesc?.refType || null,
       enumValues: enumValues.length ? enumValues : baseDesc?.enumValues ?? [],
+      isList: false,
       allowsDefine:
         (patterns ? patterns.some(patternAllowsDefine) : false) ||
         baseDesc?.allowsDefine ||
@@ -215,6 +248,7 @@ function resolveTypeDescriptor(typeName, memo = new Map()) {
       name,
       refType: null,
       enumValues: [],
+      isList: false,
       allowsDefine: false,
       isRef: false,
       isBoolean: false,
@@ -230,6 +264,7 @@ function resolveTypeDescriptor(typeName, memo = new Map()) {
     name,
     refType: null,
     enumValues: [],
+    isList: false,
     allowsDefine: false,
     isRef: false,
     isBoolean: false,
@@ -287,6 +322,7 @@ function collectAttributes(node, out = []) {
         name: `@attr:${name}`,
         refType: attr.simpleType?.["@_refType"] ?? null,
         enumValues: inlineEnum,
+        isList: false,
         allowsDefine: pats ? pats.some(patternAllowsDefine) : false,
         isBoolean: false,
         base: normalizeTypeName(attr.simpleType?.restriction?.["@_base"]),
@@ -308,6 +344,7 @@ function collectAttributes(node, out = []) {
         normalizeTypeName(desc?.refType) ??
         null,
       enumValues: desc?.enumValues ?? inlineEnum ?? [],
+      isList: desc?.isList ?? false,
       allowsDefine: desc?.allowsDefine ?? false,
       isRef: desc?.isRef ?? false,
       isBoolean: desc?.isBoolean ?? false,
@@ -396,6 +433,7 @@ for (const [name, node] of simpleTypes) {
     refType: desc?.refType ?? node?.["@_refType"] ?? null,
     isRef: node?.["@_isRef"] === "true" || desc?.refType != null,
     enumValues: desc?.enumValues ?? [],
+    isList: desc?.isList ?? false,
     allowsDefine: desc?.allowsDefine ?? false,
     doc: docOf(node),
   };