45 lines
2.5 KiB
Markdown
45 lines
2.5 KiB
Markdown
# ADR 0001: Hidden Revision Pass for AI Analysis
|
|
|
|
**Date:** 2026-07-06
|
|
|
|
## Status
|
|
|
|
Proposed
|
|
|
|
## Context
|
|
|
|
The AI analysis feature sends player operation logs to an LLM and displays the analysis in `AIChatPanel`. LLM output is inherently unreliable — the model may make contradictory claims, use incorrect game knowledge, or miss alternative interpretations.
|
|
|
|
We considered several approaches to handle problematic output:
|
|
|
|
1. **Show raw output, let the user judge.** Simplest, but puts the burden on the user to spot errors.
|
|
2. **Show a corrected version alongside the original.** Transparent, but confusing — two competing analyses.
|
|
3. **Reject the entire segment and retry from scratch.** Wastes the prior reasoning; may produce similar mistakes.
|
|
4. **Hidden revision pass:** Send the draft, validation issues, and replay facts back to the AI, asking for a clean corrected version without apology text.
|
|
|
|
We chose option 4 because:
|
|
- The user sees only one coherent analysis.
|
|
- Prior reasoning is preserved and refined, not discarded.
|
|
- The user is not exposed to "sorry, my previous answer was wrong" chatter.
|
|
|
|
## Decision
|
|
|
|
- Implement a **hidden revision pass** for segments that have `Contradiction` or `Fatal` validation issues.
|
|
- The revision prompt includes: the original draft, the list of validation issues (with severity and kind), and relevant replay facts for the affected claims.
|
|
- The model is instructed to output a clean corrected analysis (natural language + machine-readable claims) **without** acknowledging the revision.
|
|
- Limit to **one revision pass per segment** to avoid infinite loops.
|
|
- If the revision still has `Fatal` issues, fall back to displaying the original with a warning.
|
|
- Per-segment content is **buffered** during validation — the user sees the final content only after validation and optional revision complete.
|
|
|
|
## Consequences
|
|
|
|
- Positive: User sees a single, cleaner analysis.
|
|
- Positive: Prior reasoning is reused, saving tokens vs. re-analyzing from scratch.
|
|
- Negative: Adds latency (one extra round trip) for segments that need revision.
|
|
- Negative: Increases token usage for revised segments (draft + revision prompt + corrected output).
|
|
- Negative: Hidden correction may reduce user trust if they discover it — consider a subtle indicator like "验证器发现并修正 N 个问题".
|
|
|
|
## Implementation Status
|
|
|
|
Not yet implemented. Validation issues are detected and logged in `AIChatPanel`, but no automatic revision pass is triggered. The revision prompt construction and retry orchestration still need to be wired in.
|