deepseek wip

WIP.md

@@ -41,6 +41,29 @@ Examples discussed:
 - A UnitId claimed as an aircraft should be challenged if the same UnitId is observed using an unpack/deploy skill.
 - A UnitId claimed as a bomber should be challenged if it is operated before the player starts producing their first bomber.
 
+### Documented Example: MCV vs Miner Ambiguity (Allied)
+
+Observed replay sequence:
+1. UnitId 246 (confirmed main base) → `PackReplaceSelf`
+2. Player selects UnitId 587
+3. UnitId 587 → `UnpackReplaceSelf`
+4. AI claims: `587 = AlliedMCV`, evidenceLevel: `confirmed`
+
+**Why this cannot be definitively resolved:**
+- After base 246 packs, the engine creates a new MCV (UnitId A). The miner (UnitId B) also exists on the map.
+- When the player selects 587, we cannot prove 587 = A vs 587 = B.
+- After `UnpackReplaceSelf`, 587 is replaced by yet another UnitId (C if MCV→base, D if miner→command hub).
+- Even if we later see C building things (`开始建造建筑 [UnitId]C(建造者)`), there is no replay-observable link connecting C back to 587.
+- Allied MCV in mobile form has no unique observable ability that would distinguish it from a miner.
+
+**Conclusion:** There is **no deterministic validation rule** that can confirm an Allied MCV claim from replay operations alone. The upper bound for any such claim is `possible`, and an alternative (miner command hub) must always be listed.
+
+**Contrast with other factions:**
+- Soviet/Japan/神州 MCVs may have different observable behaviors (e.g., unique deploy animations, different upgrade paths) — each faction needs independent analysis.
+
+**Validation rule (negative check only):**
+- If a claim says `confirmed` or `highly likely` for AlliedMCV based only on `PackReplaceSelf → UnpackReplaceSelf` sequence, flag as **overconfident** (WeakEvidence). Downgrade recommendation: `possible` with miner command hub as alternative.
+
 ### Prompt Decisions
 
 The default prompt should explicitly require:
@@ -110,7 +133,23 @@ Reasoning:
 - A custom line format would be easier for a trivial parser but would become fragile once nested data is needed.
 - The app can tolerate partial or missing JSON by logging validation issues instead of failing the whole analysis.
 
-Current expected shape:
+### Evidence Format Decision (2026-07-06)
+
+We decided to move from free-form evidence text to a **structured pipe-delimited format**:
+
+```
+type|time|param1|param2|...
+```
+
+Supported types: `build`, `place`, `produce`, `sell`, `select`, `move`, `power`.
+
+Reasoning:
+- Free-form text could not be programmatically validated without NLP.
+- Structured evidence can be parsed deterministically with a simple regex.
+- Enables deterministic validation rules like unpack-ambiguity checking.
+- The format is simple enough for AI models to follow reliably.
+
+Current expected shape (all three claim types now have schema definitions in the prompt):
 
 ```json
 {
@@ -125,8 +164,20 @@ Current expected shape:
       "needsConfirmation": ["是否曾使用 SpecialPower_PackReplaceSelf", "后续是否作为建造者出现"]
     }
   ],
-  "eventClaims": [],
-  "timelineClaims": []
+  "eventClaims": [
+    {
+      "claim": "PlayerA 主基地打包并开始迁移",
+      "evidenceLevel": "confirmed",
+      "evidence": ["1:24.00 SpecialPower_PackReplaceSelf", "后续移动和展开操作"]
+    }
+  ],
+  "timelineClaims": [
+    {
+      "claim": "PlayerA 在开局 2 分钟内完成了基地迁移",
+      "evidenceLevel": "confirmed",
+      "evidence": ["1:24.00 打包", "1:41.00 展开"]
+    }
+  ]
 }
 ```
 
@@ -139,7 +190,13 @@ The prompt asks the AI to output:
 ```
 ```
 
-The parser first looks for the last fenced JSON block near `[机器可读声明]`, then falls back to the last `{...}` block.
+The parser first looks for the last fenced JSON block near `[机器可读声明]`, then falls back to the last `{...}`.
+
+**Known format issue (fixed):** The original prompt only defined `unitClaims` entries; `eventClaims` and `timelineClaims` were shown as empty arrays. The AI therefore invented its own fields (e.g., `"event"` / `"time"` instead of `"claim"`), which the parser silently ignored. Fixed by:
+1. Adding full schema definitions for all three claim types in the prompt.
+2. Making the parser accept `"event"` as a fallback for `"claim"` in `eventClaims`.
+
+**Claim count limits added:** Prompt instructs the AI to limit output (unitClaims ≤ 10, eventClaims ≤ 5, timelineClaims ≤ 3). The parser enforces these caps and emits Info-level issues if the AI exceeds them.
 
 ## Validation We Can Do
 
@@ -150,6 +207,9 @@ Format validation:
 - Missing machine-readable claims.
 - JSON parse failure.
 - Root value is not an object.
+- Claim count limits with truncation warnings.
+- `eventClaims` accepts both `"claim"` and `"event"` as field names.
+- Unknown `evidenceLevel` values logged as Info issue, fallback to `Uncertain`.
 
 Self-consistency validation:
 
@@ -158,15 +218,39 @@ Self-consistency validation:
 - High-confidence UnitId guess without evidence.
 - Low-confidence UnitId guess without alternatives or needed confirmation.
 
+### Evidence Format (Structured)
+
+The `evidence` field now uses a structured pipe-delimited format instead of free-form text:
+
+```
+build|time|assetName|builderUnitId
+place|time|assetName|builderUnitId|x,y,z
+produce|time|unitName|producerUnitId
+sell|time|unitId
+select|time|unitId
+move|time|x,y,z
+power|time|powerName|unitId
+```
+
+This format is parsed by `ParseStructuredEvidence()` into a `StructuredEvidence` record with typed `AIEvidenceType` enum. Parsing uses a single regex and is fully deterministic.
+
+**Backward compatibility:** The parser silently returns `Unknown` type for strings that don't match the structured format. No validation rules currently fire on unknown-typed evidence, so it degrades gracefully but invisibly.
+
 ### Near-Term Validations
 
 These need replay facts extracted from `CommandChunk` or an intermediate fact index:
 
-- UnitId special power contradictions.
-- UnitId production timeline contradictions.
-- UnitId used as builder vs claimed as non-builder unit.
+- UnitId production timeline contradictions (e.g., "bomber" claimed before first bomber production — needs game knowledge of which unit names are bombers).
 - Claims that use game knowledge not present in rules, such as transport/amphibious/building-placement abilities.
-- Missing alternatives for ambiguous skills such as Allied unpack.
+- **Overconfidence detection:** Claims with `confirmed`/`highly likely` that lack sufficient evidence given what is knowable from replay data alone (e.g., claiming AlliedMCV as `confirmed`).
+
+### Implemented via Fact Index
+
+The `ReplayFactIndex` now powers these checks:
+
+- **Special power contradiction:** Evidence `power|...|SomePower|unitId` is cross-checked against the actual special powers observed for that UnitId. If the power was never used, a `Contradiction` issue is emitted.
+- **UnitId existence:** Warns if a claim references a UnitId never seen in any replay command.
+- **Builder consistency:** If a claim describes a unit as MCV/builder but the UnitId was never observed as a builder, emits `WeakEvidence`.
 
 ### Suggested Fact Index
 
@@ -203,6 +287,37 @@ Implemented:
   - JSON extraction and parsing
   - initial self-consistency checks
 - Added per-segment validation logging in `AIChatPanel`.
+- Fixed inconsistent prompt ↔ parser schema for `eventClaims`/`timelineClaims`:
+  - Added full schema definitions for all three claim types in the system prompt.
+  - Parser now accepts `"event"` as fallback for `"claim"` in `eventClaims`.
+  - Both prompt and parser enforce claim count limits (10 unit, 5 event, 3 timeline) with truncation warnings.
+- Unknown `evidenceLevel` values now produce an Info-level validation issue (fallback to `Uncertain`).
+- Created ADR 0001 documenting the hidden revision pass design decision.
+- Recorded MCV vs Miner ambiguity as a documented validation scenario.
+- Evidence format changed from free-form text to structured pipe-delimited format:
+  - 7 evidence types: `build`, `place`, `produce`, `sell`, `select`, `move`, `power`.
+  - Prompt updated to require structured format only.
+  - Added `StructuredEvidence` record and `ParseStructuredEvidence()` parser.
+  - Added `ParseAllEvidence()` to convert all evidence strings for a claim.
+- Added first validation rule `ValidateUnpackAmbiguity()`:
+  - Flags `confirmed`/`highly likely` claims that use `UnpackReplaceSelf` without matching `PackReplaceSelf`.
+  - Emits `WeakEvidence`/`MissingAlternative` — the unpack could be MCV deploy or miner command hub deploy.
+  - If `PackReplaceSelf` IS present in the same claim's evidence, the chain is consistent and no flag.
+- Created `Utils/ReplayFactIndex.cs` — builds a fact index from raw `CommandChunk` data:
+  - `UnitIdFirstObservedTime`: first time each UnitId appears in any command.
+  - `UnitIdSpecialPowers`: set of special powers used by each UnitId.
+  - `BuilderUnitIds`: UnitIds that appeared as builder in construction commands.
+  - `ProducerUnitIds`: UnitIds that appeared as production structures.
+  - `PlayerFirstProductionTime`: per player, first production time for each unit asset name.
+  - `PlayerSelectedUnitIds`: which UnitIds each player has selected.
+- Plumbed `ReplayFactIndex` through the analysis pipeline:
+  - Built in `EventDump.ShowPlainText()` from `CommandChunk` + string hash table.
+  - Passed to `AIChatPanel.StartAnalysisAsync()` as new parameter.
+  - Forwarded to `AIAnalysisValidation.ValidateMachineReadableClaims()`.
+- Added `ValidateTimelineConsistency()` — three checks using fact index:
+  1. **UnitId existence check:** Warns if a claim references a UnitId never seen in the replay.
+  2. **Special power verification:** Cross-references `power|...` evidence entries against actual special powers observed for that UnitId; emits `Contradiction` if the claim says a UnitId used a power it never used.
+  3. **Builder consistency check:** If a claim describes a unit as MCV/builder/Nanocore but that UnitId was never observed as a builder, emits `WeakEvidence`.
 
 Build status:
 
@@ -219,15 +334,17 @@ Build status:
 - Should validation issues be visible by default, or only in an advanced/debug foldout?
 - How much game-unit knowledge should live in code versus prompt text?
 - Should the first verifier use hardcoded RA3/Corona knowledge, or should it load a small unit capability table from data files?
+- How should free-form evidence strings be programmatically validated? (See "Known Limitation" above.)
 
 ## Suggested Next Steps
 
-1. Build a replay fact index from `CommandChunk`.
-2. Add first deterministic validation rules:
-   - ambiguous Allied unpack
-   - pack/unpack consistency
-   - UnitId used as builder
-   - first production time vs first operation time
+1. ✅ Build a replay fact index from `CommandChunk` — done (`ReplayFactIndex`).
+2. ✅ Add first deterministic validation rules:
+   - ✅ ambiguous Allied unpack — done (`ValidateUnpackAmbiguity`).
+   - ✅ pack/unpack consistency — covered by unpack rule.
+   - ✅ UnitId used as builder — done (builder consistency check in `ValidateTimelineConsistency`).
+   - ✅ special power contradictions — done (special power verification in `ValidateTimelineConsistency`).
+   - ⬜ first production time vs first operation time — needs game knowledge of unit type names (e.g., "which names are bombers").
 3. Decide whether to buffer per-segment content before display.
 4. Add one hidden revision pass for `Contradiction` issues.
 5. Add validation summary UI, such as "验证器发现并修正 N 个问题".