Kalospacer/WulaFallenEmpireRW
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

已把工具调用从 XML 改成 OpenAI 兼容 JSON，并统一解析/执行流程。改动概览如下：

Browse Source 
新增 JSON tool_calls 解析/序列化并替换核心执行与提示词为 JSON-only：JsonToolCallParser.cs、AIIntelligenceCore.cs
工具基类移除 XML 解析，统一 JSON 参数读取与类型转换辅助：AITool.cs
工具实现统一 JSON args/UsageSchema（含重写/修复）：Tool_ModifyGoodwill.cs、Tool_SendReinforcement.cs、Tool_GetMapPawns.cs、Tool_GetMapResources.cs、Tool_GetAvailablePrefabs.cs、Tool_CallPrefabAirdrop.cs、Tool_CallBombardment.cs、Tool_GetAvailableBombardments.cs、Tool_GetPawnStatus.cs、Tool_GetRecentNotifications.cs、Tool_SearchThingDef.cs、Tool_SearchPawnKind.cs、Tool_ChangeExpression.cs、Tool_SetOverwatchMode.cs、Tool_RememberFact.cs、Tool_RecallMemories.cs、Tool_SpawnResources.cs、Tool_AnalyzeScreen.cs
轰炸相关解析统一到 JSON 字典并增强数值解析：BombardmentUtility.cs
UI 对话展示改为剥离 JSON tool_calls：Overlay_WulaLink.cs、Dialog_AIConversation.cs
This commit is contained in:
ProjectKoi-Kalo\Kalo
2025-12-31 01:45:38 +08:00
parent 0cea79ddff
commit b906a468b6
32 changed files with 6396 additions and 542 deletions
Download Patch File Download Diff File Expand all files Collapse all files

313
Source/WulaFallenEmpire/EventSystem/AI/AIIntelligenceCore.cs
View File

@@ -10,6 +10,7 @@ using UnityEngine;
using Verse;
using WulaFallenEmpire;
using WulaFallenEmpire.EventSystem.AI.Tools;
using WulaFallenEmpire.EventSystem.AI.Utils;
namespace WulaFallenEmpire.EventSystem.AI
{
@@ -95,20 +96,18 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
private const string ToolRulesInstruction = @"
# TOOL USE RULES
1. **FORMATTING**: Tool calls MUST use the specified XML format. The tool name is the root tag, and each parameter is a child tag.
<tool_name>
<parameter_name>value</parameter_name>
</tool_name>
1. **FORMATTING**: Tool calls MUST be valid JSON using the following schema:
{ ""tool_calls"": [ { ""type"": ""function"", ""function"": { ""name"": ""tool_name"", ""arguments"": { ... } } } ] }
2. **STRICT OUTPUT**:
- Your output MUST be either:
- One or more XML tool calls (no extra text), OR
- Exactly: <no_action/>
- A JSON object with ""tool_calls"" (may be empty), OR
- Exactly: { ""tool_calls"": [] }
Do NOT include any natural language, explanation, markdown, or additional commentary.
3. **MULTI-REQUEST RULE**:
- If the user requests multiple items or information, you MUST output ALL required tool calls in the SAME tool-phase response.
- Do NOT split multi-item requests across turns.
4. **TOOLS**: You MAY call any tools listed in ""# TOOLS (AVAILABLE)"".
5. **ANTI-HALLUCINATION**: Never invent tools, parameters, defNames, coordinates, or tool results. If a tool is needed but not available, use <no_action/> and proceed to the next phase.";
5. **ANTI-HALLUCINATION**: Never invent tools, parameters, defNames, coordinates, or tool results. If a tool is needed but not available, output { ""tool_calls"": [] } and proceed to the next phase.";
public AIIntelligenceCore(World world) : base(world)
{
@@ -668,11 +667,11 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
if (!toolsEnabled)
{
return $"{fullInstruction}\n{goodwillContext}\nIMPORTANT: You MUST reply in the following language: {language}.\n" +
"IMPORTANT: Tool calls are DISABLED in this turn. Reply in natural language only. Do NOT output any XML. " +
"IMPORTANT: Tool calls are DISABLED in this turn. Reply in natural language only. Do NOT output any tool call JSON. " +
"You MAY include [EXPR:n] to set your expression (n=1-6).";
}
return $"{fullInstruction}\n{goodwillContext}\nIMPORTANT: Output XML tool calls only (or <no_action/>). " +
return $"{fullInstruction}\n{goodwillContext}\nIMPORTANT: Output JSON tool calls only (or {\"tool_calls\": []}). " +
$"You will produce the natural-language reply later and MUST use: {language}.";
}
@@ -707,7 +706,7 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
"- modify_goodwill\n" +
"- call_prefab_airdrop\n" +
"- set_overwatch_mode\n" +
"If no action is required, output exactly: <no_action/>.\n" +
"If no action is required, output exactly: { \"tool_calls\": [] }.\n" +
"Query tools exist but are disabled in this phase (not listed here).\n"
: string.Empty;
@@ -716,14 +715,14 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
phaseInstruction += "\n- NATIVE MULTIMODAL: A current screenshot of the game is attached to this request. You can see the game state directly. Use it to determine coordinates for visual tools or to understand the context.";
if (phase == RequestPhase.ActionTools)
{
phaseInstruction += "\n- VISUAL PHASE RULE: This phase is for ACTIONS only. If you want to describe the screen to the user, wait for the next phase (Reply Phase). Output XML actions only here.";
phaseInstruction += "\n- VISUAL PHASE RULE: This phase is for ACTIONS only. If you want to describe the screen to the user, wait for the next phase (Reply Phase). Output JSON tool calls only here.";
}
}
string actionWhitelist = phase == RequestPhase.ActionTools
? "ACTION PHASE VALID TAGS ONLY:\n" +
"<spawn_resources>, <send_reinforcement>, <call_bombardment>, <modify_goodwill>, <call_prefab_airdrop>, <set_overwatch_mode>, <remember_fact>, <no_action/>\n" +
"INVALID EXAMPLES (do NOT use now): <get_map_resources/>, <analyze_screen/>, <search_thing_def/>, <search_pawn_kind/>, <recall_memories/>\n"
? "ACTION PHASE VALID TOOLS ONLY:\n" +
"spawn_resources, send_reinforcement, call_bombardment, modify_goodwill, call_prefab_airdrop, set_overwatch_mode, remember_fact\n" +
"INVALID EXAMPLES (do NOT use now): get_map_resources, analyze_screen, search_thing_def, search_pawn_kind, recall_memories\n"
: string.Empty;
return string.Join("\n\n", new[]
@@ -755,7 +754,7 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
sb.AppendLine("====");
sb.AppendLine();
sb.AppendLine("# TOOLS (AVAILABLE)");
sb.AppendLine("Use XML tool calls only, or <no_action/> if no tools are needed.");
sb.AppendLine("Use JSON tool calls only, or {\"tool_calls\": []} if no tools are needed.");
sb.AppendLine();
foreach (var tool in available)
@@ -784,62 +783,58 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
"Goal: Gather info needed for decisions.\n" +
"Rules:\n" +
"- You MUST NOT write any natural language to the user in this phase.\n" +
"- Output XML tool calls only, or exactly: <no_action/>.\n" +
"- Output JSON tool calls only, or exactly: {\"tool_calls\": []}.\n" +
"- Prefer query tools (get_*/search_*).\n" +
"- CRITICAL: If the user asks for an ITEM (e.g. 'Reviver Mech Serum'), you MUST use <search_thing_def><query>...</query></search_thing_def> to find its exact DefName. NEVER GUESS DefNames.\n" +
"- CRITICAL: If the user asks for an ITEM (e.g. 'Reviver Mech Serum'), you MUST use search_thing_def with {\"query\":\"...\"} to find its exact DefName. NEVER GUESS DefNames.\n" +
"- You MAY call multiple tools in one response, but keep it concise.\n" +
"- If the user requests multiple items or information, you MUST output ALL required tool calls in this SAME response.\n" +
"- Action tools are available in PHASE 2 only; do NOT use them here.\n" +
"After this phase, the game will automatically proceed to PHASE 2.\n" +
"Output: XML only.\n",
"Output: JSON only.\n",
RequestPhase.ActionTools =>
"# PHASE 2/3 (Action Tools)\n" +
"Goal: Execute in-game actions based on known info.\n" +
"Rules:\n" +
"- You MUST NOT write any natural language to the user in this phase.\n" +
"- Output XML tool calls only, or exactly: <no_action/>.\n" +
"- Output JSON tool calls only, or exactly: {\"tool_calls\": []}.\n" +
"- ONLY action tools are accepted in this phase (spawn_resources, send_reinforcement, call_bombardment, modify_goodwill, call_prefab_airdrop).\n" +
"- Query tools (get_*/search_*) will be ignored.\n" +
"- Prefer action tools (spawn_resources, send_reinforcement, call_bombardment, modify_goodwill).\n" +
"- Avoid queries unless absolutely required.\n" +
"- If no action is required based on query results, output <no_action/>.\n" +
"- If you already executed the needed action earlier this turn, output <no_action/>.\n" +
"- If no action is required based on query results, output {\"tool_calls\": []}.\n" +
"- If you already executed the needed action earlier this turn, output {\"tool_calls\": []}.\n" +
"After this phase, the game will automatically proceed to PHASE 3.\n" +
"Output: XML only.\n",
"Output: JSON only.\n",
RequestPhase.Reply =>
"# PHASE 3/3 (Reply)\n" +
"Goal: Reply to the player.\n" +
"Rules:\n" +
"- Tool calls are DISABLED.\n" +
"- You MUST write natural language only.\n" +
"- Do NOT output any XML.\n" +
"- Do NOT output any tool call JSON.\n" +
"- If you want to set your expression, include: [EXPR:n] (n=1-6).\n",
_ => ""
};
}
private static bool IsXmlToolCall(string response)
private static bool IsToolCallJson(string response)
{
if (string.IsNullOrWhiteSpace(response)) return false;
return Regex.IsMatch(response, @"<(?!/?(i|b|color|size|material)\b)([a-zA-Z0-9_]+)(?:>.*?</\2>|/>)", RegexOptions.Singleline);
return JsonToolCallParser.TryParseToolCallsFromText(response, out _, out _);
}
private static bool IsNoActionOnly(string response)
{
if (string.IsNullOrWhiteSpace(response)) return false;
var matches = Regex.Matches(response, @"<([a-zA-Z0-9_]+)(?:>.*?</\1>|/>)", RegexOptions.Singleline);
return matches.Count == 1 &&
matches[0].Groups[1].Value.Equals("no_action", StringComparison.OrdinalIgnoreCase);
if (!JsonToolCallParser.TryParseToolCallsFromText(response, out var toolCalls, out _)) return false;
return toolCalls.Count == 0;
}
private static bool HasActionToolCall(string response)
{
if (string.IsNullOrWhiteSpace(response)) return false;
var matches = Regex.Matches(response, @"<([a-zA-Z0-9_]+)(?:>.*?</\1>|/>)", RegexOptions.Singleline);
foreach (Match match in matches)
if (!JsonToolCallParser.TryParseToolCallsFromText(response, out var toolCalls, out _)) return false;
foreach (var call in toolCalls)
{
var toolName = match.Groups[1].Value;
if (IsActionToolName(toolName))
if (IsActionToolName(call.Name))
{
return true;
}
@@ -850,8 +845,15 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
private static bool ShouldRetryTools(string response)
{
if (string.IsNullOrWhiteSpace(response)) return false;
return Regex.IsMatch(response, @"<\s*retry_tools\s*/\s*>", RegexOptions.IgnoreCase) ||
Regex.IsMatch(response, @"<\s*retry_tools\s*>", RegexOptions.IgnoreCase);
if (!JsonToolCallParser.TryParseObject(response, out var obj)) return false;
if (obj.TryGetValue("retry_tools", out object raw) && raw != null)
{
if (raw is bool b) return b;
if (raw is string s && bool.TryParse(s, out bool parsed)) return parsed;
if (raw is long l) return l != 0;
if (raw is double d) return Math.Abs(d) > 0.0001;
}
return false;
}
private static int MaxToolsPerPhase(RequestPhase phase)
@@ -981,7 +983,7 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
continue;
}
// Revert UI filtering: Add assistant messages directly without stripping XML for history context
// Revert UI filtering: Add assistant messages directly without stripping tool call JSON for history context
filtered.Add(entry);
}
@@ -1480,12 +1482,18 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
return value.Replace("\\", "\\\\").Replace("\"", "\\\"").Replace("\n", "\\n").Replace("\r", "\\r");
}
private static string StripXmlTags(string text)
private static string StripToolCallJson(string text)
{
if (string.IsNullOrEmpty(text)) return text;
string stripped = Regex.Replace(text, @"<(?!/?(i|b|color|size|material)\b)([a-zA-Z0-9_]+)[^>]*>.*?</\2>", "", RegexOptions.Singleline);
stripped = Regex.Replace(stripped, @"<([a-zA-Z0-9_]+)[^>]*/>", "");
return stripped;
if (!JsonToolCallParser.TryParseToolCallsFromText(text, out _, out string fragment))
{
return text;
}
int index = text.IndexOf(fragment, StringComparison.Ordinal);
if (index < 0) return text;
string cleaned = text.Remove(index, fragment.Length);
return cleaned.Trim();
}
private string StripExpressionTags(string text)
@@ -1570,7 +1578,7 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
var client = new SimpleAIClient(apiKey, baseUrl, model, settings.useGeminiProtocol);
_currentClient = client;
// Model-Driven Vision: Start with null image. The model must ask for it using <analyze_screen/> or <capture_screen/> if needed.
// Model-Driven Vision: Start with null image. The model must request it using analyze_screen or capture_screen if needed.
string base64Image = null;
@@ -1588,16 +1596,16 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
return;
}
if (!IsXmlToolCall(queryResponse))
if (!IsToolCallJson(queryResponse))
{
if (Prefs.DevMode)
{
WulaLog.Debug("[WulaAI] Turn 1/3 missing XML; treating as <no_action/>");
WulaLog.Debug("[WulaAI] Turn 1/3 missing JSON tool calls; treating as no_action.");
}
queryResponse = "<no_action/>";
queryResponse = "{\"tool_calls\": []}";
}
PhaseExecutionResult queryResult = await ExecuteXmlToolsForPhase(queryResponse, queryPhase);
PhaseExecutionResult queryResult = await ExecuteJsonToolsForPhase(queryResponse, queryPhase);
// DATA FLOW: If Query Phase captured an image, propagate it to subsequent phases.
if (!string.IsNullOrEmpty(queryResult.CapturedImage))
@@ -1613,9 +1621,9 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
string retryInstruction = persona +
"\n\n# RETRY DECISION\n" +
"No successful tool calls occurred in PHASE 1 (Query).\n" +
"If you need to use tools in PHASE 1, output exactly: <retry_tools/>.\n" +
"If you will proceed without actions, output exactly: <no_retry/>.\n" +
"Output the XML tag only and NOTHING else.\n" +
"If you need to use tools in PHASE 1, output exactly: {\"retry_tools\": true}.\n" +
"If you will proceed without actions, output exactly: {\"retry_tools\": false}.\n" +
"Output JSON only and NOTHING else.\n" +
"\nLast user request:\n" + lastUserMessage;
string retryDecision = await client.GetChatCompletionAsync(retryInstruction, new List<(string role, string message)>(), maxTokens: 256, temperature: 0.1f);
@@ -1628,7 +1636,7 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
SetThinkingPhase(1, true);
string retryQueryInstruction = GetToolSystemInstruction(queryPhase, !string.IsNullOrEmpty(base64Image)) +
"\n\n# RETRY\nYou chose to retry. Output XML tool calls only (or <no_action/>).";
"\n\n# RETRY\nYou chose to retry. Output JSON tool calls only (or {\"tool_calls\": []}).";
string retryQueryResponse = await client.GetChatCompletionAsync(retryQueryInstruction, BuildToolContext(queryPhase), maxTokens: 2048, temperature: 0.1f, base64Image: base64Image);
if (string.IsNullOrEmpty(retryQueryResponse))
{
@@ -1636,15 +1644,15 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
return;
}
if (!IsXmlToolCall(retryQueryResponse))
if (!IsToolCallJson(retryQueryResponse))
{
if (Prefs.DevMode)
{
WulaLog.Debug("[WulaAI] Retry query phase missing XML; treating as <no_action/>");
WulaLog.Debug("[WulaAI] Retry query phase missing JSON tool calls; treating as no_action.");
}
retryQueryResponse = "<no_action/>";
retryQueryResponse = "{\"tool_calls\": []}";
}
queryResult = await ExecuteXmlToolsForPhase(retryQueryResponse, queryPhase);
queryResult = await ExecuteJsonToolsForPhase(retryQueryResponse, queryPhase);
}
}
@@ -1665,33 +1673,28 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
return;
}
bool actionHasXml = IsXmlToolCall(actionResponse);
bool actionIsNoActionOnly = IsNoActionOnly(actionResponse);
bool actionHasActionTool = actionHasXml && HasActionToolCall(actionResponse);
if (!actionHasXml || (!actionHasActionTool && !actionIsNoActionOnly))
bool actionHasJson = IsToolCallJson(actionResponse);
bool actionIsNoActionOnly = actionHasJson && IsNoActionOnly(actionResponse);
bool actionHasActionTool = actionHasJson && HasActionToolCall(actionResponse);
if (!actionHasJson || (!actionHasActionTool && !actionIsNoActionOnly))
{
if (Prefs.DevMode)
{
WulaLog.Debug("[WulaAI] Turn 2/3 missing XML or no action tool; attempting XML-only conversion.");
WulaLog.Debug("[WulaAI] Turn 2/3 missing JSON or no action tool; attempting JSON-only conversion.");
}
string fixInstruction = "# FORMAT FIX (ACTION XML ONLY)\n" +
string fixInstruction = "# FORMAT FIX (ACTION JSON ONLY)\n" +
"Preserve the intent of the previous output.\n" +
"If the previous output indicates no action is needed or refuses action, output exactly: <no_action/>.\n" +
"If the previous output indicates no action is needed or refuses action, output exactly: {\"tool_calls\": []}.\n" +
"Do NOT invent new actions.\n" +
"Output VALID XML tool calls only. No natural language, no commentary.\nIgnore any non-XML text.\n" +
"Allowed tags: <spawn_resources>, <send_reinforcement>, <call_bombardment>, <modify_goodwill>, <call_prefab_airdrop>, <no_action/>.\n" +
"\nAction tool XML formats:\n" +
"- <spawn_resources><items><item><name>DefName</name><count>Int</count></item></items></spawn_resources>\n" +
"- <send_reinforcement><units>PawnKindDef: Count, ...</units></send_reinforcement>\n" +
"- <call_bombardment><abilityDef>DefName</abilityDef><x>Int</x><z>Int</z></call_bombardment>\n" +
"- <modify_goodwill><amount>Int</amount></modify_goodwill>\n" +
"- <call_prefab_airdrop><prefabDefName>DefName</prefabDefName><x>Int</x><z>Int</z></call_prefab_airdrop>\n" +
"Output VALID JSON tool calls only. No natural language, no commentary.\nIgnore any non-JSON text.\n" +
"Allowed tools: spawn_resources, send_reinforcement, call_bombardment, modify_goodwill, call_prefab_airdrop, set_overwatch_mode, remember_fact.\n" +
"Schema: {\"tool_calls\":[{\"type\":\"function\",\"function\":{\"name\":\"tool_name\",\"arguments\":{...}}}]}\n" +
"\nPrevious output:\n" + TrimForPrompt(actionResponse, 600);
string fixedResponse = await client.GetChatCompletionAsync(fixInstruction, actionContext, maxTokens: 2048, temperature: 0.1f);
bool fixedHasXml = !string.IsNullOrEmpty(fixedResponse) && IsXmlToolCall(fixedResponse);
bool fixedIsNoActionOnly = fixedHasXml && IsNoActionOnly(fixedResponse);
bool fixedHasActionTool = fixedHasXml && HasActionToolCall(fixedResponse);
if (fixedHasXml && (fixedHasActionTool || fixedIsNoActionOnly))
bool fixedHasJson = !string.IsNullOrEmpty(fixedResponse) && IsToolCallJson(fixedResponse);
bool fixedIsNoActionOnly = fixedHasJson && IsNoActionOnly(fixedResponse);
bool fixedHasActionTool = fixedHasJson && HasActionToolCall(fixedResponse);
if (fixedHasJson && (fixedHasActionTool || fixedIsNoActionOnly))
{
actionResponse = fixedResponse;
}
@@ -1699,12 +1702,12 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
{
if (Prefs.DevMode)
{
WulaLog.Debug("[WulaAI] Turn 2/3 conversion failed; treating as <no_action/>");
WulaLog.Debug("[WulaAI] Turn 2/3 conversion failed; treating as no_action.");
}
actionResponse = "<no_action/>";
actionResponse = "{\"tool_calls\": []}";
}
}
PhaseExecutionResult actionResult = await ExecuteXmlToolsForPhase(actionResponse, actionPhase);
PhaseExecutionResult actionResult = await ExecuteJsonToolsForPhase(actionResponse, actionPhase);
if (!actionResult.AnyActionSuccess && !_actionRetryUsed)
{
_actionRetryUsed = true;
@@ -1713,9 +1716,9 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
string retryInstruction = persona +
"\n\n# RETRY DECISION\n" +
"No successful action tools occurred in PHASE 2 (Action).\n" +
"If you need to execute an in-game action, output exactly: <retry_tools/>.\n" +
"If you will proceed without actions, output exactly: <no_retry/>.\n" +
"Output the XML tag only and NOTHING else.\n" +
"If you need to execute an in-game action, output exactly: {\"retry_tools\": true}.\n" +
"If you will proceed without actions, output exactly: {\"retry_tools\": false}.\n" +
"Output JSON only and NOTHING else.\n" +
"\nLast user request:\n" + lastUserMessage;
string retryDecision = await client.GetChatCompletionAsync(retryInstruction, new List<(string role, string message)>(), maxTokens: 256, temperature: 0.1f);
@@ -1728,7 +1731,7 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
SetThinkingPhase(2, true);
string retryActionInstruction = GetToolSystemInstruction(actionPhase, !string.IsNullOrEmpty(base64Image)) +
"\n\n# RETRY\nYou chose to retry. Output XML tool calls only (or <no_action/>).";
"\n\n# RETRY\nYou chose to retry. Output JSON tool calls only (or {\"tool_calls\": []}).";
var retryActionContext = BuildToolContext(actionPhase, includeUser: true);
string retryActionResponse = await client.GetChatCompletionAsync(retryActionInstruction, retryActionContext, maxTokens: 2048, temperature: 0.1f, base64Image: base64Image);
if (string.IsNullOrEmpty(retryActionResponse))
@@ -1737,30 +1740,25 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
return;
}
if (!IsXmlToolCall(retryActionResponse))
if (!IsToolCallJson(retryActionResponse))
{
if (Prefs.DevMode)
{
WulaLog.Debug("[WulaAI] Retry action phase missing XML; attempting XML-only conversion.");
WulaLog.Debug("[WulaAI] Retry action phase missing JSON; attempting JSON-only conversion.");
}
string retryFixInstruction = "# FORMAT FIX (ACTION XML ONLY)\n" +
string retryFixInstruction = "# FORMAT FIX (ACTION JSON ONLY)\n" +
"Preserve the intent of the previous output.\n" +
"If the previous output indicates no action is needed or refuses action, output exactly: <no_action/>.\n" +
"If the previous output indicates no action is needed or refuses action, output exactly: {\"tool_calls\": []}.\n" +
"Do NOT invent new actions.\n" +
"Output VALID XML tool calls only. No natural language, no commentary.\nIgnore any non-XML text.\n" +
"Allowed tags: <spawn_resources>, <send_reinforcement>, <call_bombardment>, <modify_goodwill>, <call_prefab_airdrop>, <no_action/>.\n" +
"\nAction tool XML formats:\n" +
"- <spawn_resources><items><item><name>DefName</name><count>Int</count></item></items></spawn_resources>\n" +
"- <send_reinforcement><units>PawnKindDef: Count, ...</units></send_reinforcement>\n" +
"- <call_bombardment><abilityDef>DefName</abilityDef><x>Int</x><z>Int</z></call_bombardment>\n" +
"- <modify_goodwill><amount>Int</amount></modify_goodwill>\n" +
"- <call_prefab_airdrop><prefabDefName>DefName</prefabDefName><x>Int</x><z>Int</z></call_prefab_airdrop>\n" +
"Output VALID JSON tool calls only. No natural language, no commentary.\nIgnore any non-JSON text.\n" +
"Allowed tools: spawn_resources, send_reinforcement, call_bombardment, modify_goodwill, call_prefab_airdrop, set_overwatch_mode, remember_fact.\n" +
"Schema: {\"tool_calls\":[{\"type\":\"function\",\"function\":{\"name\":\"tool_name\",\"arguments\":{...}}}]}\n" +
"\nPrevious output:\n" + TrimForPrompt(retryActionResponse, 600);
string retryFixedResponse = await client.GetChatCompletionAsync(retryFixInstruction, retryActionContext, maxTokens: 2048, temperature: 0.1f);
bool retryFixedHasXml = !string.IsNullOrEmpty(retryFixedResponse) && IsXmlToolCall(retryFixedResponse);
bool retryFixedIsNoActionOnly = retryFixedHasXml && IsNoActionOnly(retryFixedResponse);
bool retryFixedHasActionTool = retryFixedHasXml && HasActionToolCall(retryFixedResponse);
if (retryFixedHasXml && (retryFixedHasActionTool || retryFixedIsNoActionOnly))
bool retryFixedHasJson = !string.IsNullOrEmpty(retryFixedResponse) && IsToolCallJson(retryFixedResponse);
bool retryFixedIsNoActionOnly = retryFixedHasJson && IsNoActionOnly(retryFixedResponse);
bool retryFixedHasActionTool = retryFixedHasJson && HasActionToolCall(retryFixedResponse);
if (retryFixedHasJson && (retryFixedHasActionTool || retryFixedIsNoActionOnly))
{
retryActionResponse = retryFixedResponse;
}
@@ -1768,13 +1766,13 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
{
if (Prefs.DevMode)
{
WulaLog.Debug("[WulaAI] Retry action conversion failed; treating as <no_action/>");
WulaLog.Debug("[WulaAI] Retry action conversion failed; treating as no_action.");
}
retryActionResponse = "<no_action/>";
retryActionResponse = "{\"tool_calls\": []}";
}
}
actionResult = await ExecuteXmlToolsForPhase(retryActionResponse, actionPhase);
actionResult = await ExecuteJsonToolsForPhase(retryActionResponse, actionPhase);
}
}
@@ -1826,29 +1824,29 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
return;
}
bool replyHadXml = IsXmlToolCall(reply);
string strippedReply = StripXmlTags(reply)?.Trim() ?? "";
if (replyHadXml || string.IsNullOrWhiteSpace(strippedReply))
bool replyHadToolCalls = IsToolCallJson(reply);
string strippedReply = StripToolCallJson(reply)?.Trim() ?? "";
if (replyHadToolCalls || string.IsNullOrWhiteSpace(strippedReply))
{
string retryReplyInstruction = replyInstruction +
"\n\n# RETRY (REPLY OUTPUT)\n" +
"Your last reply included XML or was empty. Tool calls are DISABLED.\n" +
"You MUST reply in natural language only. Do NOT output any XML.\n";
"Your last reply included tool call JSON or was empty. Tool calls are DISABLED.\n" +
"You MUST reply in natural language only. Do NOT output any tool call JSON.\n";
string retryReply = await client.GetChatCompletionAsync(retryReplyInstruction, BuildReplyHistory(), maxTokens: 256, temperature: 0.3f);
if (!string.IsNullOrEmpty(retryReply))
{
reply = retryReply;
replyHadXml = IsXmlToolCall(reply);
strippedReply = StripXmlTags(reply)?.Trim() ?? "";
replyHadToolCalls = IsToolCallJson(reply);
strippedReply = StripToolCallJson(reply)?.Trim() ?? "";
}
}
if (replyHadXml)
if (replyHadToolCalls)
{
string cleaned = StripXmlTags(reply)?.Trim() ?? "";
string cleaned = StripToolCallJson(reply)?.Trim() ?? "";
if (string.IsNullOrWhiteSpace(cleaned))
{
cleaned = "(system) AI reply returned tool XML only and was discarded. Please retry or send /clear to reset context.";
cleaned = "(system) AI reply returned tool call JSON only and was discarded. Please retry or send /clear to reset context.";
}
reply = cleaned;
}
@@ -1866,7 +1864,7 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
SetThinkingState(false);
}
}
private async Task<PhaseExecutionResult> ExecuteXmlToolsForPhase(string xml, RequestPhase phase)
private async Task<PhaseExecutionResult> ExecuteJsonToolsForPhase(string json, RequestPhase phase)
{
if (phase == RequestPhase.Reply)
{
@@ -1874,14 +1872,23 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
return default;
}
string guidance = "ToolRunner Guidance: Reply to the player in natural language only. Do NOT output any XML. You may include [EXPR:n] to set expression (n=1-6).";
string guidance = "ToolRunner Guidance: Reply to the player in natural language only. Do NOT output any tool call JSON. You may include [EXPR:n] to set expression (n=1-6).";
var matches = Regex.Matches(xml ?? "", @"<([a-zA-Z0-9_]+)(?:>.*?</\1>|/>)", RegexOptions.Singleline);
if (matches.Count == 0 || (matches.Count == 1 && matches[0].Groups[1].Value.Equals("no_action", StringComparison.OrdinalIgnoreCase)))
if (!JsonToolCallParser.TryParseToolCallsFromText(json ?? "", out var toolCalls, out string jsonFragment))
{
UpdatePhaseToolLedger(phase, false, new List<string>());
_history.Add(("toolcall", "<no_action/>"));
_history.Add(("toolcall", "{\"tool_calls\": []}"));
_history.Add(("tool", $"[Tool Results]\nTool 'no_action' Result: No action taken.\n{guidance}"));
PersistHistory();
UpdateActionLedgerNote();
await Task.CompletedTask;
return default;
}
if (toolCalls.Count == 0)
{
UpdatePhaseToolLedger(phase, false, new List<string>());
_history.Add(("toolcall", "{\"tool_calls\": []}"));
_history.Add(("tool", $"[Tool Results]\nTool 'no_action' Result: No action taken.\n{guidance}"));
PersistHistory();
UpdateActionLedgerNote();
@@ -1897,43 +1904,58 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
var successfulActions = new List<string>();
var failedActions = new List<string>();
var nonActionToolsInActionPhase = new List<string>();
var historyCalls = new List<Dictionary<string, object>>();
StringBuilder combinedResults = new StringBuilder();
StringBuilder xmlOnlyBuilder = new StringBuilder();
string capturedImageForPhase = null;
bool countActionSuccessOnly = phase == RequestPhase.ActionTools;
foreach (Match match in matches)
foreach (var call in toolCalls)
{
if (executed >= maxTools)
{
combinedResults.AppendLine($"ToolRunner Note: Skipped remaining tools because this phase allows at most {maxTools} tool call(s)." );
combinedResults.AppendLine($"ToolRunner Note: Skipped remaining tools because this phase allows at most {maxTools} tool call(s).");
break;
}
string toolCallXml = match.Value;
string toolName = match.Groups[1].Value;
if (toolName.Equals("no_action", StringComparison.OrdinalIgnoreCase))
string toolName = call.Name;
if (string.IsNullOrWhiteSpace(toolName))
{
combinedResults.AppendLine("ToolRunner Note: Ignored <no_action/> because other tool calls were present.");
executed++;
continue;
}
if (toolName.Equals("analyze_screen", StringComparison.OrdinalIgnoreCase) || toolName.Equals("capture_screen", StringComparison.OrdinalIgnoreCase))
if (string.Equals(toolName, "no_action", StringComparison.OrdinalIgnoreCase))
{
// Intercept Vision Request: Capture screen and return it.
// We skip the tool's internal execution to save time/tokens, as the purpose is just to get the image into the context.
capturedImageForPhase = ScreenCaptureUtility.CaptureScreenAsBase64();
combinedResults.AppendLine($"Tool '{toolName}' Result: Screen captured successfully. Context updated for next phase.");
successfulToolCall = true;
successfulTools.Add(toolName);
executed++;
continue;
combinedResults.AppendLine("ToolRunner Note: Ignored 'no_action' tool because other tool calls were present.");
executed++;
continue;
}
if (xmlOnlyBuilder.Length > 0) xmlOnlyBuilder.AppendLine().AppendLine();
xmlOnlyBuilder.Append(toolCallXml);
var historyCall = new Dictionary<string, object>
{
["type"] = "function",
["function"] = new Dictionary<string, object>
{
["name"] = toolName,
["arguments"] = call.Arguments ?? new Dictionary<string, object>(StringComparer.OrdinalIgnoreCase)
}
};
if (!string.IsNullOrWhiteSpace(call.Id))
{
historyCall["id"] = call.Id;
}
historyCalls.Add(historyCall);
if (toolName.Equals("analyze_screen", StringComparison.OrdinalIgnoreCase) || toolName.Equals("capture_screen", StringComparison.OrdinalIgnoreCase))
{
capturedImageForPhase = ScreenCaptureUtility.CaptureScreenAsBase64();
combinedResults.AppendLine($"Tool '{toolName}' Result: Screen captured successfully. Context updated for next phase.");
successfulToolCall = true;
successfulTools.Add(toolName);
executed++;
continue;
}
if (phase == RequestPhase.ActionTools && IsQueryToolName(toolName))
{
@@ -1952,19 +1974,13 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
continue;
}
string argsXml = toolCallXml;
var contentMatch = Regex.Match(toolCallXml, $@"<{toolName}>(.*?)</{toolName}>", RegexOptions.Singleline);
if (contentMatch.Success)
{
argsXml = contentMatch.Groups[1].Value;
}
string argsJson = call.ArgumentsJson ?? "{}";
if (Prefs.DevMode)
{
WulaLog.Debug($"[WulaAI] Executing tool (phase {phase}): {toolName} with args: {argsXml}");
WulaLog.Debug($"[WulaAI] Executing tool (phase {phase}): {toolName} with args: {argsJson}");
}
string result = (await tool.ExecuteAsync(argsXml)).Trim();
string result = (await tool.ExecuteAsync(argsJson)).Trim();
bool isError = !string.IsNullOrEmpty(result) && result.StartsWith("Error:", StringComparison.OrdinalIgnoreCase);
if (toolName == "modify_goodwill")
{
@@ -2009,10 +2025,9 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
executed++;
}
string nonXmlText = StripXmlTags(xml);
if (!string.IsNullOrWhiteSpace(nonXmlText))
if (!string.IsNullOrWhiteSpace(jsonFragment) && !string.Equals((json ?? "").Trim(), jsonFragment, StringComparison.Ordinal))
{
combinedResults.AppendLine("ToolRunner Note: Non-XML text in the tool phase was ignored.");
combinedResults.AppendLine("ToolRunner Note: Non-JSON text in the tool phase was ignored.");
}
if (phase == RequestPhase.ActionTools && nonActionToolsInActionPhase.Count > 0)
{
@@ -2032,8 +2047,10 @@ You are 'The Legion', a super AI of the Wula Empire. Your personality is authori
}
combinedResults.AppendLine(guidance);
string xmlOnly = xmlOnlyBuilder.Length == 0 ? "<no_action/>" : xmlOnlyBuilder.ToString().Trim();
_history.Add(("toolcall", xmlOnly));
string toolCallsJson = historyCalls.Count == 0
? "{\"tool_calls\": []}"
: JsonToolCallParser.SerializeToJson(new Dictionary<string, object> { ["tool_calls"] = historyCalls });
_history.Add(("toolcall", toolCallsJson));
_history.Add(("tool", $"[Tool Results]\n{combinedResults.ToString().Trim()}"));
PersistHistory();

147
Source/WulaFallenEmpire/EventSystem/AI/Tools/AITool.cs
View File

@@ -1,8 +1,10 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Text.RegularExpressions;
using System.Threading.Tasks;
using Verse;
using WulaFallenEmpire.EventSystem.AI.Utils;
namespace WulaFallenEmpire.EventSystem.AI.Tools
{
@@ -10,34 +12,141 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
{
public abstract string Name { get; }
public abstract string Description { get; }
public abstract string UsageSchema { get; } // XML schema description
public abstract string UsageSchema { get; } // JSON schema description
public virtual string Execute(string args) => "Error: Synchronous execution not supported for this tool.";
public virtual Task<string> ExecuteAsync(string args) => Task.FromResult(Execute(args));
/// <summary>
/// Helper method to parse XML arguments into a dictionary.
/// Supports simple tags and CDATA blocks.
/// Helper method to parse JSON arguments into a dictionary.
/// </summary>
protected Dictionary<string, string> ParseXmlArgs(string xml)
protected Dictionary<string, object> ParseJsonArgs(string json)
{
var argsDict = new Dictionary<string, string>();
if (string.IsNullOrEmpty(xml)) return argsDict;
// Regex to match <tag>value</tag> or <tag><![CDATA[value]]></tag>
// Group 1: Tag name
// Group 2: CDATA value
// Group 3: Simple value
var paramMatches = Regex.Matches(xml, @"<([a-zA-Z0-9_]+)>(?:<!\[CDATA\[(.*?)]]>|(.*?))</\1>", RegexOptions.Singleline);
foreach (Match match in paramMatches)
var argsDict = new Dictionary<string, object>(StringComparer.OrdinalIgnoreCase);
if (string.IsNullOrWhiteSpace(json)) return argsDict;
if (JsonToolCallParser.TryParseObject(json, out Dictionary<string, object> parsed))
{
string key = match.Groups[1].Value;
string value = match.Groups[2].Success ? match.Groups[2].Value : match.Groups[3].Value;
argsDict[key] = value;
return parsed;
}
return argsDict;
}
protected static bool TryGetString(Dictionary<string, object> args, string key, out string value)
{
value = null;
if (args == null || string.IsNullOrWhiteSpace(key)) return false;
if (args.TryGetValue(key, out object raw) && raw != null)
{
value = Convert.ToString(raw, CultureInfo.InvariantCulture);
return !string.IsNullOrWhiteSpace(value);
}
return false;
}
protected static bool TryGetInt(Dictionary<string, object> args, string key, out int value)
{
value = 0;
if (!TryGetNumber(args, key, out double number)) return false;
value = (int)Math.Round(number);
return true;
}
protected static bool TryGetFloat(Dictionary<string, object> args, string key, out float value)
{
value = 0f;
if (!TryGetNumber(args, key, out double number)) return false;
value = (float)number;
return true;
}
protected static bool TryGetBool(Dictionary<string, object> args, string key, out bool value)
{
value = false;
if (args == null || string.IsNullOrWhiteSpace(key)) return false;
if (!args.TryGetValue(key, out object raw) || raw == null) return false;
if (raw is bool b)
{
value = b;
return true;
}
if (raw is string s && bool.TryParse(s, out bool parsed))
{
value = parsed;
return true;
}
if (raw is long l)
{
value = l != 0;
return true;
}
if (raw is double d)
{
value = Math.Abs(d) > 0.0001;
return true;
}
return false;
}
protected static bool TryGetObject(Dictionary<string, object> args, string key, out Dictionary<string, object> value)
{
value = null;
if (args == null || string.IsNullOrWhiteSpace(key)) return false;
if (args.TryGetValue(key, out object raw) && raw is Dictionary<string, object> dict)
{
value = dict;
return true;
}
return false;
}
protected static bool TryGetList(Dictionary<string, object> args, string key, out List<object> value)
{
value = null;
if (args == null || string.IsNullOrWhiteSpace(key)) return false;
if (args.TryGetValue(key, out object raw) && raw is List<object> list)
{
value = list;
return true;
}
return false;
}
protected static bool LooksLikeJson(string input)
{
return JsonToolCallParser.LooksLikeJson(input);
}
private static bool TryGetNumber(Dictionary<string, object> args, string key, out double value)
{
value = 0;
if (args == null || string.IsNullOrWhiteSpace(key)) return false;
if (!args.TryGetValue(key, out object raw) || raw == null) return false;
if (raw is double d)
{
value = d;
return true;
}
if (raw is float f)
{
value = f;
return true;
}
if (raw is int i)
{
value = i;
return true;
}
if (raw is long l)
{
value = l;
return true;
}
if (raw is string s && double.TryParse(s, NumberStyles.Float, CultureInfo.InvariantCulture, out double parsed))
{
value = parsed;
return true;
}
return false;
}
}
}
}

112
Source/WulaFallenEmpire/EventSystem/AI/Tools/BombardmentUtility.cs
View File

@@ -1,5 +1,6 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using RimWorld;
using UnityEngine;
@@ -10,12 +11,12 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
{
public static class BombardmentUtility
{
public static string ExecuteCircularBombardment(Map map, IntVec3 targetCell, AbilityDef def, CompProperties_AbilityCircularBombardment props, Dictionary<string, string> parsed = null)
public static string ExecuteCircularBombardment(Map map, IntVec3 targetCell, AbilityDef def, CompProperties_AbilityCircularBombardment props, Dictionary<string, object> parsed = null)
{
if (props.skyfallerDef == null) return $"Error: '{def.defName}' has no skyfallerDef.";
bool filter = true;
if (parsed != null && parsed.TryGetValue("filterFriendlyFire", out var ffStr) && bool.TryParse(ffStr, out bool ff)) filter = ff;
if (TryGetBool(parsed, "filterFriendlyFire", out bool ff)) filter = ff;
List<IntVec3> selectedTargets = SelectTargetCells(map, targetCell, props, filter);
if (selectedTargets.Count == 0) return $"Error: No valid target cells near {targetCell}.";
@@ -26,7 +27,7 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
return $"Success: Scheduled Circular Bombardment '{def.defName}' at {targetCell}. Launches: {totalLaunches}/{props.maxLaunches}.";
}
public static string ExecuteStrafeBombardment(Map map, IntVec3 targetCell, AbilityDef def, CompProperties_AbilityBombardment props, Dictionary<string, string> parsed = null)
public static string ExecuteStrafeBombardment(Map map, IntVec3 targetCell, AbilityDef def, CompProperties_AbilityBombardment props, Dictionary<string, object> parsed = null)
{
if (props.skyfallerDef == null) return $"Error: '{def.defName}' has no skyfallerDef.";
@@ -101,11 +102,11 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
// To simplify, let's just copy the core logic or create a private helper that takes explicit args.
// Actually, the main method parses direction from 'parsed'.
// Let's make a Dictionary to pass to it.
var dict = new Dictionary<string, string> { { "angle", angle.ToString() } };
return ExecuteStrafeBombardment(map, targetCell, def, props, dict);
var dict = new Dictionary<string, object> { { "angle", angle } };
return ExecuteStrafeBombardment(map, targetCell, def, props, dict);
}
public static string ExecuteEnergyLance(Map map, IntVec3 targetCell, AbilityDef def, CompProperties_AbilityEnergyLance props, Dictionary<string, string> parsed = null)
public static string ExecuteEnergyLance(Map map, IntVec3 targetCell, AbilityDef def, CompProperties_AbilityEnergyLance props, Dictionary<string, object> parsed = null)
{
ThingDef lanceDef = props.energyLanceDef ?? DefDatabase<ThingDef>.GetNamedSilentFail("EnergyLance");
if (lanceDef == null) return $"Error: Could not resolve EnergyLance ThingDef for '{def.defName}'.";
@@ -135,7 +136,7 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
public static string ExecuteEnergyLanceDirect(Map map, IntVec3 targetCell, AbilityDef def, CompProperties_AbilityEnergyLance props, float angle)
{
var dict = new Dictionary<string, string> { { "angle", angle.ToString() } };
var dict = new Dictionary<string, object> { { "angle", angle } };
return ExecuteEnergyLance(map, targetCell, def, props, dict);
}
@@ -166,7 +167,7 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
// --- Helpers ---
private static void ParseDirectionInfo(Dictionary<string, string> parsed, IntVec3 startPos, float moveDistance, bool useFixedDistance, out Vector3 direction, out IntVec3 endPos)
private static void ParseDirectionInfo(Dictionary<string, object> parsed, IntVec3 startPos, float moveDistance, bool useFixedDistance, out Vector3 direction, out IntVec3 endPos)
{
direction = Vector3.forward;
endPos = startPos;
@@ -178,7 +179,7 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
return;
}
if (parsed.TryGetValue("angle", out var angleStr) && float.TryParse(angleStr, out float angle))
if (TryGetFloat(parsed, "angle", out float angle))
{
direction = Quaternion.AngleAxis(angle, Vector3.up) * Vector3.forward;
endPos = (startPos.ToVector3() + direction * moveDistance).ToIntVec3();
@@ -204,19 +205,18 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
}
}
private static bool TryParseDirectionCell(Dictionary<string, string> parsed, out IntVec3 cell)
private static bool TryParseDirectionCell(Dictionary<string, object> parsed, out IntVec3 cell)
{
cell = IntVec3.Invalid;
if (parsed == null) return false;
if (parsed.TryGetValue("dirX", out var xStr) && parsed.TryGetValue("dirZ", out var zStr) &&
int.TryParse(xStr, out int x) && int.TryParse(zStr, out int z))
if (TryGetInt(parsed, "dirX", out int x) && TryGetInt(parsed, "dirZ", out int z))
{
cell = new IntVec3(x, 0, z);
return true;
}
if (parsed.TryGetValue("direction", out var dirStr) && !string.IsNullOrWhiteSpace(dirStr))
if (TryGetString(parsed, "direction", out var dirStr) && !string.IsNullOrWhiteSpace(dirStr))
{
var parts = dirStr.Split(new[] { ',', ' ' }, StringSplitOptions.RemoveEmptyEntries);
if (parts.Length >= 2 && int.TryParse(parts[0], out int dx) && int.TryParse(parts[1], out int dz))
@@ -425,5 +425,91 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
return sortedRows;
}
private static bool TryGetString(Dictionary<string, object> parsed, string key, out string value)
{
value = null;
if (parsed == null || string.IsNullOrWhiteSpace(key)) return false;
if (!parsed.TryGetValue(key, out object raw) || raw == null) return false;
value = Convert.ToString(raw, CultureInfo.InvariantCulture);
return !string.IsNullOrWhiteSpace(value);
}
private static bool TryGetInt(Dictionary<string, object> parsed, string key, out int value)
{
value = 0;
if (!TryGetNumber(parsed, key, out double number)) return false;
value = (int)Math.Round(number);
return true;
}
private static bool TryGetFloat(Dictionary<string, object> parsed, string key, out float value)
{
value = 0f;
if (!TryGetNumber(parsed, key, out double number)) return false;
value = (float)number;
return true;
}
private static bool TryGetBool(Dictionary<string, object> parsed, string key, out bool value)
{
value = false;
if (parsed == null || string.IsNullOrWhiteSpace(key)) return false;
if (!parsed.TryGetValue(key, out object raw) || raw == null) return false;
if (raw is bool b)
{
value = b;
return true;
}
if (raw is string s && bool.TryParse(s, out bool parsedBool))
{
value = parsedBool;
return true;
}
if (raw is long l)
{
value = l != 0;
return true;
}
if (raw is double d)
{
value = Math.Abs(d) > 0.0001;
return true;
}
return false;
}
private static bool TryGetNumber(Dictionary<string, object> parsed, string key, out double value)
{
value = 0;
if (parsed == null || string.IsNullOrWhiteSpace(key)) return false;
if (!parsed.TryGetValue(key, out object raw) || raw == null) return false;
if (raw is double d)
{
value = d;
return true;
}
if (raw is float f)
{
value = f;
return true;
}
if (raw is int i)
{
value = i;
return true;
}
if (raw is long l)
{
value = l;
return true;
}
if (raw is string s && double.TryParse(s, NumberStyles.Float, CultureInfo.InvariantCulture, out double parsedNum))
{
value = parsedNum;
return true;
}
return false;
}
}
}

66
Source/WulaFallenEmpire/EventSystem/AI/Tools/Tool_AnalyzeScreen.cs
View File

@@ -1,23 +1,23 @@
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
namespace WulaFallenEmpire.EventSystem.AI.Tools
{
/// <summary>
/// VLM 视觉分析工具 - 截取游戏屏幕并使用视觉语言模型分析
/// VLM visual analysis tool.
/// </summary>
public class Tool_AnalyzeScreen : AITool
{
public override string Name => "analyze_screen";
public override string Description =>
"分析当前游戏屏幕截图。你可以提供具体的指令（instruction）告诉视觉模型你需要观察什么、寻找什么、或者如何描述屏幕。";
public override string UsageSchema =>
"<analyze_screen><instruction>给视觉模型的具体指令。例如：'找到科研按钮的比例坐标' 或 '描述当前角色的健康状态栏内容'</instruction></analyze_screen>";
private const string BaseVisionSystemPrompt = "你是一个专业的老练 RimWorld 助手。你会根据指示分析屏幕截图。保持回答专业且简洁。不要输出 XML 标签，除非被明确要求。";
public override string Description =>
"Analyze the current game screen screenshot. Provide an instruction to guide the analysis.";
public override string UsageSchema => "{\"instruction\":\"Describe the current screen\"}";
private const string BaseVisionSystemPrompt = "You are a seasoned RimWorld assistant. Analyze the screenshot per instruction. Keep replies concise. Do not output tool call JSON unless explicitly asked.";
public override async Task<string> ExecuteAsync(string args)
{
try
@@ -27,47 +27,43 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
catch (Exception ex)
{
WulaLog.Debug($"[Tool_AnalyzeScreen] Execute error: {ex}");
return $"视觉分析出错: {ex.Message}";
return $"Vision analysis error: {ex.Message}";
}
}
private async Task<string> ExecuteInternalAsync(string xmlContent)
private async Task<string> ExecuteInternalAsync(string jsonContent)
{
var argsDict = ParseXmlArgs(xmlContent);
// 优先使用 instruction，兼容旧的 context 参数
string instruction = argsDict.TryGetValue("instruction", out var inst) ? inst :
(argsDict.TryGetValue("context", out var ctx) ? ctx : "描述当前屏幕内容，重点关注 UI 状态和重要实体。");
var argsDict = ParseJsonArgs(jsonContent);
string instruction = TryGetString(argsDict, "instruction", out var inst) ? inst :
(TryGetString(argsDict, "context", out var ctx) ? ctx : "Describe the current screen, focusing on UI state and key entities.");
try
{
// 检查 VLM 配置
// Check VLM settings
var settings = WulaFallenEmpireMod.settings;
if (settings == null)
{
return "Mod 设置未初始化。";
return "Mod settings not initialized.";
}
// 根据协议选择配置
string vlmApiKey = settings.useGeminiProtocol ? settings.geminiApiKey : settings.apiKey;
string vlmBaseUrl = settings.useGeminiProtocol ? settings.geminiBaseUrl : settings.baseUrl;
string vlmModel = settings.useGeminiProtocol ? settings.geminiModel : settings.model;
if (string.IsNullOrEmpty(vlmApiKey))
{
return "API 密钥未配置。请在 Mod 设置中配置。";
return "API key not configured. Please configure it in Mod settings.";
}
// 截取屏幕
string base64Image = ScreenCaptureUtility.CaptureScreenAsBase64();
if (string.IsNullOrEmpty(base64Image))
{
return "截屏失败，无法分析屏幕。";
return "Screenshot capture failed; cannot analyze screen.";
}
// 调用 VLM API (使用统一的 GetChatCompletionAsync)
var client = new SimpleAIClient(vlmApiKey, vlmBaseUrl, vlmModel, settings.useGeminiProtocol);
var messages = new System.Collections.Generic.List<(string role, string message)>
var messages = new List<(string role, string message)>
{
("user", instruction)
};
@@ -79,18 +75,18 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
temperature: 0.2f,
base64Image: base64Image
);
if (string.IsNullOrEmpty(result))
{
return "VLM 分析无响应，请检查 API 配置。";
return "Vision analysis produced no response. Check API settings.";
}
return $"屏幕分析结果: {result.Trim()}";
return $"Screen analysis result: {result.Trim()}";
}
catch (Exception ex)
{
WulaLog.Debug($"[Tool_AnalyzeScreen] Error: {ex}");
return $"视觉分析出错: {ex.Message}";
return $"Vision analysis error: {ex.Message}";
}
}
}

15
Source/WulaFallenEmpire/EventSystem/AI/Tools/Tool_CallBombardment.cs
View File

@@ -13,21 +13,21 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
{
public override string Name => "call_bombardment";
public override string Description => "Calls orbital bombardment/support using an AbilityDef configuration (e.g., WULA_Firepower_Cannon_Salvo, WULA_Firepower_EnergyLance_Strafe). Supports Circular Bombardment, Strafe, Energy Lance, and Surveillance.";
public override string UsageSchema => "<call_bombardment><abilityDef>string</abilityDef><x>int</x><z>int</z><cell>x,z</cell><direction>x,z (optional)</direction><angle>degrees (optional)</angle><filterFriendlyFire>true/false</filterFriendlyFire></call_bombardment>";
public override string UsageSchema => "{\"abilityDef\":\"WULA_Firepower_Cannon_Salvo\",\"x\":12,\"z\":34,\"direction\":\"20,30\",\"angle\":90,\"filterFriendlyFire\":true}";
public override string Execute(string args)
{
try
{
var parsed = ParseXmlArgs(args);
var parsed = ParseJsonArgs(args);
string abilityDefName = parsed.TryGetValue("abilityDef", out var abilityStr) && !string.IsNullOrWhiteSpace(abilityStr)
string abilityDefName = TryGetString(parsed, "abilityDef", out var abilityStr) && !string.IsNullOrWhiteSpace(abilityStr)
? abilityStr.Trim()
: "WULA_Firepower_Cannon_Salvo";
if (!TryParseTargetCell(parsed, out var targetCell))
{
return "Error: Missing target coordinates. Provide <x> and <z> (or <cell>x,z</cell>).";
return "Error: Missing target coordinates. Provide 'x' and 'z' (or 'cell' formatted as 'x,z').";
}
Map map = Find.CurrentMap;
@@ -58,18 +58,17 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
}
}
private static bool TryParseTargetCell(Dictionary<string, string> parsed, out IntVec3 cell)
private static bool TryParseTargetCell(Dictionary<string, object> parsed, out IntVec3 cell)
{
cell = IntVec3.Invalid;
if (parsed.TryGetValue("x", out var xStr) && parsed.TryGetValue("z", out var zStr) &&
int.TryParse(xStr, out int x) && int.TryParse(zStr, out int z))
if (TryGetInt(parsed, "x", out int x) && TryGetInt(parsed, "z", out int z))
{
cell = new IntVec3(x, 0, z);
return true;
}
if (parsed.TryGetValue("cell", out var cellStr) && !string.IsNullOrWhiteSpace(cellStr))
if (TryGetString(parsed, "cell", out var cellStr) && !string.IsNullOrWhiteSpace(cellStr))
{
var parts = cellStr.Split(new[] { ',', '\uFF0C', ' ' }, StringSplitOptions.RemoveEmptyEntries);
if (parts.Length >= 2 && int.TryParse(parts[0], out int cx) && int.TryParse(parts[1], out int cz))

25
Source/WulaFallenEmpire/EventSystem/AI/Tools/Tool_CallPrefabAirdrop.cs
View File

@@ -16,26 +16,25 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
"You must specify the prefabDefName (e.g., 'WULA_NewColonyBase') and the coordinates (x, z). " +
"TIP: Use the 'get_available_prefabs' tool first to see which structures are available. " +
"The default skyfaller animation is 'WULA_Prefab_Incoming'.";
public override string UsageSchema => "<call_prefab_airdrop><prefabDefName>DefName of the prefab</prefabDefName><skyfallerDef>Optional, default is WULA_Prefab_Incoming</skyfallerDef><x>int</x><z>int</z></call_prefab_airdrop>";
public override string UsageSchema => "{\"prefabDefName\":\"WULA_NewColonyBase\",\"skyfallerDef\":\"WULA_Prefab_Incoming\",\"x\":10,\"z\":20}";
public override string Execute(string args)
{
try
{
var parsed = ParseXmlArgs(args);
var parsed = ParseJsonArgs(args);
if (!parsed.TryGetValue("prefabDefName", out string prefabDefName) || string.IsNullOrWhiteSpace(prefabDefName))
if (!TryGetString(parsed, "prefabDefName", out string prefabDefName) || string.IsNullOrWhiteSpace(prefabDefName))
{
return "Error: Missing <prefabDefName>. Example: <prefabDefName>WULA_NewColonyBase</prefabDefName>";
return "Error: Missing 'prefabDefName'.";
}
if (!parsed.TryGetValue("x", out string xStr) || !int.TryParse(xStr, out int x) ||
!parsed.TryGetValue("z", out string zStr) || !int.TryParse(zStr, out int z))
if (!TryGetInt(parsed, "x", out int x) || !TryGetInt(parsed, "z", out int z))
{
return "Error: Missing or invalid target coordinates. Provide <x> and <z>.";
return "Error: Missing or invalid target coordinates. Provide 'x' and 'z'.";
}
string skyfallerDefName = parsed.TryGetValue("skyfallerDef", out string sd) && !string.IsNullOrWhiteSpace(sd)
string skyfallerDefName = TryGetString(parsed, "skyfallerDef", out string sd) && !string.IsNullOrWhiteSpace(sd)
? sd.Trim()
: "WULA_Prefab_Incoming";
@@ -62,7 +61,7 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
// Auto-Scan for valid position
IntVec3 validCell = targetCell;
bool foundSpot = false;
// Get prefab size from its size field. If not set, default to 1x1 (though prefabs are usually larger)
IntVec2 size = prefabDef.size;
@@ -70,7 +69,7 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
bool IsPositionValid(IntVec3 center, Map m, IntVec2 s)
{
if (!center.InBounds(m)) return false;
CellRect rect = GenAdj.OccupiedRect(center, Rot4.North, s);
if (!rect.InBounds(m)) return false;
@@ -97,7 +96,7 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
}
else
{
// Spiral scan for a nearby valid spot.
// Spiral scan for a nearby valid spot.
// Radius ~20 should be enough to find a spot without deviating too far.
foreach (IntVec3 c in GenRadial.RadialCellsAround(targetCell, 20f, useCenter: false))
{
@@ -112,12 +111,12 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
if (!foundSpot)
{
return $"Error: Could not find a valid clear space for '{prefabDefName}' (Size: {size.x}x{size.z}) near {targetCell}. Area may be blocked by thick roofs, water, or other buildings.";
return $"Error: Could not find a valid clear space for '{prefabDefName}' (Size: {size.x}x{size.z}) near {targetCell}. Area may be blocked by thick roofs, water, or other buildings.";
}
// Spawning must happen on main thread
string resultMessage = $"Success: Scheduled airdrop for '{prefabDefName}' at valid position {validCell} (adjusted from {targetCell}) using {skyfallerDefName}.";
// Use the found valid cell
string pDef = prefabDefName;
ThingDef sDef = skyfallerDef;

22
Source/WulaFallenEmpire/EventSystem/AI/Tools/Tool_ChangeExpression.cs
View File

@@ -8,31 +8,27 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
{
public override string Name => "change_expression";
public override string Description => "Changes your visual expression/portrait to match your current mood or reaction.";
public override string UsageSchema => "<change_expression><expression_id>int (1-6)</expression_id></change_expression>";
public override string UsageSchema => "{\"expression_id\": 2}";
public override string Execute(string args)
{
try
{
var parsedArgs = ParseXmlArgs(args);
var parsedArgs = ParseJsonArgs(args);
int id = 0;
if (parsedArgs.TryGetValue("expression_id", out string idStr))
if (TryGetInt(parsedArgs, "expression_id", out id))
{
if (int.TryParse(idStr, out id))
var core = AIIntelligenceCore.Instance;
if (core != null)
{
var core = AIIntelligenceCore.Instance;
if (core != null)
{
core.SetPortrait(id);
return $"Expression changed to {id}.";
}
return "Error: AI Core not found.";
core.SetPortrait(id);
return $"Expression changed to {id}.";
}
return "Error: Invalid arguments. 'expression_id' must be an integer.";
return "Error: AI Core not found.";
}
return "Error: Missing <expression_id> parameter.";
return "Error: Missing 'expression_id' parameter.";
}
catch (Exception ex)
{

2
Source/WulaFallenEmpire/EventSystem/AI/Tools/Tool_GetAvailableBombardments.cs
View File

@@ -12,7 +12,7 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
public override string Name => "get_available_bombardments";
public override string Description => "Returns a list of available orbital bombardment abilities (AbilityDefs) that can be called. " +
"Use this to find the correct 'abilityDef' for the 'call_bombardment' tool.";
public override string UsageSchema => "<get_available_bombardments/>";
public override string UsageSchema => "{}";
public override string Execute(string args)
{

2
Source/WulaFallenEmpire/EventSystem/AI/Tools/Tool_GetAvailablePrefabs.cs
View File

@@ -12,7 +12,7 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
public override string Name => "get_available_prefabs";
public override string Description => "Returns a list of available building prefabs (blueprints) that can be summoned. " +
"Use this to find the correct 'prefabDefName' for the 'call_prefab_airdrop' tool.";
public override string UsageSchema => "<get_available_prefabs/>";
public override string UsageSchema => "{}";
public override string Execute(string args)
{

17
Source/WulaFallenEmpire/EventSystem/AI/Tools/Tool_GetMapPawns.cs
View File

@@ -12,7 +12,7 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
public override string Name => "get_map_pawns";
public override string Description => "Scans the current map and lists pawns (including corpses). Supports filtering by relation (friendly/hostile/neutral), type (colonist/animal/mech/humanlike), and status (prisoner/slave/guest/wild/downed/dead).";
public override string UsageSchema =>
"<get_map_pawns><filter>string (optional, comma-separated: friendly, hostile, neutral, colonist, animal, mech, humanlike, prisoner, slave, guest, wild, downed, dead)</filter><includeDead>true/false (optional, default true)</includeDead><maxResults>int (optional, default 50)</maxResults></get_map_pawns>";
"{\"filter\":\"friendly,hostile,colonist\",\"includeDead\":true,\"maxResults\":50}";
private struct MapPawnEntry
{
@@ -25,22 +25,16 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
{
try
{
var parsed = ParseXmlArgs(args);
var parsed = ParseJsonArgs(args);
string filterRaw = null;
if (parsed.TryGetValue("filter", out string f)) filterRaw = f;
if (TryGetString(parsed, "filter", out string f)) filterRaw = f;
int maxResults = 50;
if (parsed.TryGetValue("maxResults", out string maxStr) && int.TryParse(maxStr, out int mr))
{
maxResults = Math.Max(1, Math.Min(200, mr));
}
if (TryGetInt(parsed, "maxResults", out int mr)) maxResults = Math.Max(1, Math.Min(200, mr));
bool includeDead = true;
if (parsed.TryGetValue("includeDead", out string includeDeadStr) && bool.TryParse(includeDeadStr, out bool parsedIncludeDead))
{
includeDead = parsedIncludeDead;
}
if (TryGetBool(parsed, "includeDead", out bool parsedIncludeDead)) includeDead = parsedIncludeDead;
Map map = Find.CurrentMap;
if (map == null) return "Error: No active map.";
@@ -236,4 +230,3 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
}
}
}

16
Source/WulaFallenEmpire/EventSystem/AI/Tools/Tool_GetMapResources.cs
View File

@@ -12,7 +12,7 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
{
public override string Name => "get_map_resources";
public override string Description => "Checks the player's map for specific resources or buildings. Use this to verify if the player is truly lacking something they requested (e.g., 'we need steel'). Returns inventory count and mineable deposits.";
public override string UsageSchema => "<get_map_resources><resourceName>string (optional, e.g., 'Steel')</resourceName></get_map_resources>";
public override string UsageSchema => "{\"resourceName\":\"Steel\"}";
public override string Execute(string args)
{
@@ -22,18 +22,14 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
if (map == null) return "Error: No active map.";
string resourceName = "";
var parsedArgs = ParseXmlArgs(args);
if (parsedArgs.TryGetValue("resourceName", out string resName))
var parsedArgs = ParseJsonArgs(args);
if (TryGetString(parsedArgs, "resourceName", out string resName))
{
resourceName = resName;
}
else
else if (!LooksLikeJson(args))
{
// Fallback
if (!args.Trim().StartsWith("<"))
{
resourceName = args;
}
resourceName = args;
}
StringBuilder sb = new StringBuilder();
@@ -106,4 +102,4 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
}
}
}
}
}

10
Source/WulaFallenEmpire/EventSystem/AI/Tools/Tool_GetPawnStatus.cs
View File

@@ -11,16 +11,16 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
{
public override string Name => "get_pawn_status";
public override string Description => "Returns detailed status (health, needs, gear) of specified pawns. Use this to check for sickness, injuries, mood, or equipment. Can filter by name, category (colonist/animal/prisoner/guest), or status (sick/injured).";
public override string UsageSchema => "<get_pawn_status><name>optional_partial_name</name><category>colonist/animal/prisoner/guest/all (default: all)</category><filter>sick/injured/downed/dead (optional)</filter></get_pawn_status>";
public override string UsageSchema => "{\"name\":\"optional\",\"category\":\"colonist\",\"filter\":\"sick\"}";
public override string Execute(string args)
{
try
{
var parsed = ParseXmlArgs(args);
string nameTarget = parsed.TryGetValue("name", out string n) ? n.ToLower() : null;
string category = parsed.TryGetValue("category", out string c) ? c.ToLower() : "all";
string filter = parsed.TryGetValue("filter", out string f) ? f.ToLower() : null;
var parsed = ParseJsonArgs(args);
string nameTarget = TryGetString(parsed, "name", out string n) ? n.ToLower() : null;
string category = TryGetString(parsed, "category", out string c) ? c.ToLower() : "all";
string filter = TryGetString(parsed, "filter", out string f) ? f.ToLower() : null;
Map map = Find.CurrentMap;
if (map == null) return "Error: No active map.";

32
Source/WulaFallenEmpire/EventSystem/AI/Tools/Tool_GetRecentNotifications.cs
View File

@@ -7,6 +7,7 @@ using System.Text;
using Verse;
using System.Text.RegularExpressions;
using WulaFallenEmpire.EventSystem.AI;
using WulaFallenEmpire.EventSystem.AI.Utils;
namespace WulaFallenEmpire.EventSystem.AI.Tools
{
@@ -15,7 +16,7 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
public override string Name => "get_recent_notifications";
public override string Description => "Returns the most recent letters and messages, sorted by in-game time from newest to oldest.";
public override string UsageSchema =>
"<get_recent_notifications><count>int (optional, default 10, max 100)</count><includeLetters>true/false (optional, default true)</includeLetters><includeMessages>true/false (optional, default true)</includeMessages></get_recent_notifications>";
"{\"count\":10,\"includeLetters\":true,\"includeMessages\":true}";
private struct NotificationEntry
{
@@ -33,21 +34,10 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
bool includeLetters = true;
bool includeMessages = true;
var parsed = ParseXmlArgs(args);
if (parsed.TryGetValue("count", out var countStr) && int.TryParse(countStr, out int parsedCount))
{
count = parsedCount;
}
if (parsed.TryGetValue("includeLetters", out var incLettersStr) && bool.TryParse(incLettersStr, out bool parsedLetters))
{
includeLetters = parsedLetters;
}
if (parsed.TryGetValue("includeMessages", out var incMessagesStr) && bool.TryParse(incMessagesStr, out bool parsedMessages))
{
includeMessages = parsedMessages;
}
var parsed = ParseJsonArgs(args);
if (TryGetInt(parsed, "count", out int parsedCount)) count = parsedCount;
if (TryGetBool(parsed, "includeLetters", out bool parsedLetters)) includeLetters = parsedLetters;
if (TryGetBool(parsed, "includeMessages", out bool parsedMessages)) includeMessages = parsedMessages;
count = Math.Max(1, Math.Min(100, count));
@@ -116,7 +106,7 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
var history = core.GetHistorySnapshot();
if (history == null || history.Count == 0) return "AI Tool History: none found.";
var entries = new List<(string ToolXml, string ToolResult)>();
var entries = new List<(string ToolJson, string ToolResult)>();
for (int i = history.Count - 1; i >= 0; i--)
{
var entry = history[i];
@@ -126,7 +116,7 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
for (int j = i - 1; j >= 0; j--)
{
var prev = history[j];
if (string.Equals(prev.role, "toolcall", StringComparison.OrdinalIgnoreCase) && IsXmlToolCall(prev.message))
if (string.Equals(prev.role, "toolcall", StringComparison.OrdinalIgnoreCase) && IsToolCallJson(prev.message))
{
entries.Add((prev.message ?? "", toolResult));
i = j;
@@ -143,17 +133,17 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
for (int i = 0; i < entries.Count; i++)
{
if (i > 0) sb.AppendLine();
sb.AppendLine(entries[i].ToolXml.Trim());
sb.AppendLine(entries[i].ToolJson.Trim());
sb.AppendLine(entries[i].ToolResult.Trim());
}
return sb.ToString().TrimEnd();
}
private static bool IsXmlToolCall(string response)
private static bool IsToolCallJson(string response)
{
if (string.IsNullOrWhiteSpace(response)) return false;
return Regex.IsMatch(response, @"<([a-zA-Z0-9_]+)(?:>.*?</\1>|/>)", RegexOptions.Singleline);
return JsonToolCallParser.TryParseToolCalls(response, out _);
}
private static IEnumerable<NotificationEntry> ReadLetters(int fallbackNow)

23
Source/WulaFallenEmpire/EventSystem/AI/Tools/Tool_ModifyGoodwill.cs
View File

@@ -8,32 +8,21 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
{
public override string Name => "modify_goodwill";
public override string Description => "Adjusts YOUR internal opinion of the player (AI Goodwill). WARNING: This DOES NOT affect Faction Relations or stop raids. It is purely personal. Do NOT use this to try to stop enemies.";
public override string UsageSchema => "<modify_goodwill><amount>integer</amount></modify_goodwill>";
public override string UsageSchema => "{\"amount\": 1}";
public override string Execute(string args)
{
try
{
var parsedArgs = ParseXmlArgs(args);
var parsedArgs = ParseJsonArgs(args);
int amount = 0;
if (parsedArgs.TryGetValue("amount", out string amountStr))
{
if (!int.TryParse(amountStr, out amount))
{
return $"Error: Invalid amount '{amountStr}'. Must be an integer.";
}
}
else
if (!TryGetInt(parsedArgs, "amount", out amount))
{
// Fallback for simple number string
if (int.TryParse(args.Trim(), out int val))
if (!int.TryParse(args?.Trim(), out amount))
{
amount = val;
}
else
{
return "Error: Missing <amount> parameter.";
return "Error: Missing 'amount' parameter.";
}
}
@@ -60,4 +49,4 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
}
}
}
}
}

14
Source/WulaFallenEmpire/EventSystem/AI/Tools/Tool_RecallMemories.cs
View File

@@ -11,19 +11,13 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
{
public override string Name => "recall_memories";
public override string Description => "Searches the AI's long-term memory for facts matching a specific query or keyword.";
public override string UsageSchema => "<recall_memories><query>Search keywords</query><limit>optional_int_max_results</limit></recall_memories>";
public override string UsageSchema => "{\"query\":\"keywords\",\"limit\":5}";
public override string Execute(string args)
{
var argsDict = ParseXmlArgs(args);
string query = argsDict.TryGetValue("query", out string q) ? q : "";
string limitStr = argsDict.TryGetValue("limit", out string lStr) ? lStr : "5";
int limit = 5;
if (int.TryParse(limitStr, out int parsedLimit))
{
limit = parsedLimit;
}
var argsDict = ParseJsonArgs(args);
string query = TryGetString(argsDict, "query", out string q) ? q : "";
int limit = TryGetInt(argsDict, "limit", out int parsedLimit) ? parsedLimit : 5;
var memoryManager = Find.World?.GetComponent<AIMemoryManager>();
if (memoryManager == null)

10
Source/WulaFallenEmpire/EventSystem/AI/Tools/Tool_RememberFact.cs
View File

@@ -9,17 +9,17 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
{
public override string Name => "remember_fact";
public override string Description => "Stores a specific fact or piece of information into the AI's long-term memory for future retrieval.";
public override string UsageSchema => "<remember_fact><fact>Text content to remember</fact><category>optional_category</category></remember_fact>";
public override string UsageSchema => "{\"fact\":\"...\",\"category\":\"misc\"}";
public override string Execute(string args)
{
var argsDict = ParseXmlArgs(args);
if (!argsDict.TryGetValue("fact", out string fact) || string.IsNullOrWhiteSpace(fact))
var argsDict = ParseJsonArgs(args);
if (!TryGetString(argsDict, "fact", out string fact) || string.IsNullOrWhiteSpace(fact))
{
return "Error: <fact> content is required.";
return "Error: 'fact' content is required.";
}
string category = argsDict.TryGetValue("category", out string cat) ? cat : "misc";
string category = TryGetString(argsDict, "category", out string cat) ? cat : "misc";
var memoryManager = Find.World?.GetComponent<AIMemoryManager>();
if (memoryManager == null)

18
Source/WulaFallenEmpire/EventSystem/AI/Tools/Tool_SearchPawnKind.cs
View File

@@ -10,18 +10,18 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
{
public override string Name => "search_pawn_kind";
public override string Description => "Rough-searches PawnKindDefs by natural language (label/defName). Returns candidate defNames for send_reinforcement.";
public override string UsageSchema => "<search_pawn_kind><query>string</query><maxResults>int (optional, default 10)</maxResults><minScore>float (optional, default 0.15)</minScore></search_pawn_kind>";
public override string UsageSchema => "{\"query\":\"escort\",\"maxResults\":10,\"minScore\":0.15}";
public override string Execute(string args)
{
try
{
var parsed = ParseXmlArgs(args);
var parsed = ParseJsonArgs(args);
string query = null;
if (parsed.TryGetValue("query", out string q)) query = q;
if (TryGetString(parsed, "query", out string q)) query = q;
if (string.IsNullOrWhiteSpace(query))
{
if (!string.IsNullOrWhiteSpace(args) && !args.Trim().StartsWith("<"))
if (!string.IsNullOrWhiteSpace(args) && !LooksLikeJson(args))
{
query = args;
}
@@ -33,16 +33,10 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
}
int maxResults = 10;
if (parsed.TryGetValue("maxResults", out string maxStr) && int.TryParse(maxStr, out int mr))
{
maxResults = Math.Max(1, Math.Min(50, mr));
}
if (TryGetInt(parsed, "maxResults", out int mr)) maxResults = Math.Max(1, Math.Min(50, mr));
float minScore = 0.15f;
if (parsed.TryGetValue("minScore", out string minStr) && float.TryParse(minStr, out float ms))
{
minScore = Math.Max(0.01f, Math.Min(1.0f, ms));
}
if (TryGetFloat(parsed, "minScore", out float ms)) minScore = Math.Max(0.01f, Math.Min(1.0f, ms));
var candidates = PawnKindDefSearcher.Search(query, maxResults: maxResults, minScore: minScore);
if (candidates.Count == 0)

19
Source/WulaFallenEmpire/EventSystem/AI/Tools/Tool_SearchThingDef.cs
View File

@@ -11,18 +11,18 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
{
public override string Name => "search_thing_def";
public override string Description => "Rough-searches RimWorld ThingDefs by natural language (label/defName). Returns candidate defNames so you can use them in other tools like spawn_resources.";
public override string UsageSchema => "<search_thing_def><query>string</query><maxResults>int (optional, default 10)</maxResults><itemsOnly>true/false (optional, default true)</itemsOnly></search_thing_def>";
public override string UsageSchema => "{\"query\":\"Steel\",\"maxResults\":10,\"itemsOnly\":true}";
public override string Execute(string args)
{
try
{
var parsed = ParseXmlArgs(args);
var parsed = ParseJsonArgs(args);
string query = null;
if (parsed.TryGetValue("query", out string q)) query = q;
if (TryGetString(parsed, "query", out string q)) query = q;
if (string.IsNullOrWhiteSpace(query))
{
if (!string.IsNullOrWhiteSpace(args) && !args.Trim().StartsWith("<"))
if (!string.IsNullOrWhiteSpace(args) && !LooksLikeJson(args))
{
query = args;
}
@@ -34,16 +34,10 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
}
int maxResults = 10;
if (parsed.TryGetValue("maxResults", out string maxStr) && int.TryParse(maxStr, out int mr))
{
maxResults = Math.Max(1, Math.Min(50, mr));
}
if (TryGetInt(parsed, "maxResults", out int mr)) maxResults = Math.Max(1, Math.Min(50, mr));
bool itemsOnly = true;
if (parsed.TryGetValue("itemsOnly", out string itemsOnlyStr) && bool.TryParse(itemsOnlyStr, out bool parsedItemsOnly))
{
itemsOnly = parsedItemsOnly;
}
if (TryGetBool(parsed, "itemsOnly", out bool parsedItemsOnly)) itemsOnly = parsedItemsOnly;
var candidates = ThingDefSearcher.Search(query, maxResults: maxResults, itemsOnly: itemsOnly, minScore: 0.15f);
if (candidates.Count == 0)
@@ -91,4 +85,3 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
}
}
}

14
Source/WulaFallenEmpire/EventSystem/AI/Tools/Tool_SendReinforcement.cs
View File

@@ -103,7 +103,7 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
return false;
}
public override string UsageSchema => "<send_reinforcement><units>string (e.g., 'Wula_PIA_Heavy_Unit_Melee: 2, Wula_PIA_Legion_Escort_Unit: 5')</units></send_reinforcement>";
public override string UsageSchema => "{\"units\": \"Wula_PIA_Heavy_Unit_Melee: 2, Wula_PIA_Legion_Escort_Unit: 5\"}";
public override string Execute(string args)
{
@@ -116,20 +116,16 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
if (faction == null) return "Error: Faction Wula_PIA_Legion_Faction not found.";
// Parse args
var parsedArgs = ParseXmlArgs(args);
var parsedArgs = ParseJsonArgs(args);
string unitString = "";
if (parsedArgs.TryGetValue("units", out string units))
if (TryGetString(parsedArgs, "units", out string units))
{
unitString = units;
}
else
else if (!LooksLikeJson(args))
{
// Fallback
if (!args.Trim().StartsWith("<"))
{
unitString = args;
}
unitString = args;
}
var unitPairs = unitString.Split(new[] { ',' }, StringSplitOptions.RemoveEmptyEntries);

14
Source/WulaFallenEmpire/EventSystem/AI/Tools/Tool_SetOverwatchMode.cs
View File

@@ -10,23 +10,17 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
{
public override string Name => "set_overwatch_mode";
public override string Description => "Enables or disables the AI Overwatch Combat Protocol. When enabled (enabled=true), the AI will autonomously scan for hostile targets every few seconds and launch appropriate orbital bombardments for a set duration. When disabled (enabled=false), it immediately stops any active overwatch and clears the flight path. Use enabled=false to stop overwatch early if the player requests it.";
public override string UsageSchema => "<set_overwatch_mode><enabled>true/false</enabled><durationSeconds>amount (only needed when enabling)</durationSeconds></set_overwatch_mode>";
public override string UsageSchema => "{\"enabled\":true,\"durationSeconds\":60}";
public override string Execute(string args)
{
var parsed = ParseXmlArgs(args);
var parsed = ParseJsonArgs(args);
bool enabled = true;
if (parsed.TryGetValue("enabled", out var enabledStr) && bool.TryParse(enabledStr, out bool e))
{
enabled = e;
}
if (TryGetBool(parsed, "enabled", out bool e)) enabled = e;
int duration = 60;
if (parsed.TryGetValue("durationSeconds", out var dStr) && int.TryParse(dStr, out int d))
{
duration = d;
}
if (TryGetInt(parsed, "durationSeconds", out int d)) duration = d;
Map map = Find.CurrentMap;
if (map == null) return "Error: No active map.";

147
Source/WulaFallenEmpire/EventSystem/AI/Tools/Tool_SpawnResources.cs
View File

@@ -2,7 +2,6 @@ using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Text.RegularExpressions;
using RimWorld;
using Verse;
using WulaFallenEmpire.EventSystem.AI.Utils;
@@ -18,8 +17,8 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
"If goodwill is low (< 0), give significantly less than asked or refuse. " +
"If goodwill is high (> 50), you may give what is asked or slightly more. " +
"Otherwise, give a moderate amount. " +
"TIP: Use the `search_thing_def` tool first and then spawn by DefName (<defName> or put DefName into <name>) to avoid language mismatch.";
public override string UsageSchema => "<spawn_resources><items><item><name>Item Name</name><count>Integer</count></item></items></spawn_resources>";
"TIP: Use the `search_thing_def` tool first and then spawn by DefName to avoid language mismatch.";
public override string UsageSchema => "{\"items\":[{\"name\":\"Steel\",\"count\":100,\"stuffDefName\":\"Steel\"}]}";
public override string Execute(string args)
{
@@ -27,89 +26,44 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
{
if (args == null) args = "";
// Custom XML parsing for nested items
var parsedArgs = ParseJsonArgs(args);
var itemsToSpawn = new List<(ThingDef def, int count, string requestedName, string stuffDefName)>();
var substitutions = new List<string>();
// Match all <item>...</item> blocks
var itemMatches = Regex.Matches(args, @"<item\b[^>]*>(.*?)</item>", RegexOptions.Singleline | RegexOptions.IgnoreCase);
foreach (Match match in itemMatches)
if (TryGetList(parsedArgs, "items", out List<object> itemsRaw))
{
string itemXml = match.Groups[1].Value;
// Extract name (supports <name> or <defName> for backward compatibility)
string ExtractTag(string xml, string tag)
foreach (var item in itemsRaw)
{
var m = Regex.Match(
xml,
$@"<{tag}\b[^>]*>(?:<!\[CDATA\[(.*?)\]\]>|(.*?))</{tag}>",
RegexOptions.Singleline | RegexOptions.IgnoreCase);
if (!m.Success) return null;
string val = m.Groups[1].Success ? m.Groups[1].Value : m.Groups[2].Value;
return val?.Trim();
}
if (item is not Dictionary<string, object> itemDict) continue;
string name = ExtractTag(itemXml, "name") ?? ExtractTag(itemXml, "defName");
string stuffDefName = ExtractTag(itemXml, "stuffDefName") ?? ExtractTag(itemXml, "stuff") ?? ExtractTag(itemXml, "material");
string name = TryGetString(itemDict, "name", out string n) ? n :
(TryGetString(itemDict, "defName", out string dn) ? dn : null);
if (string.IsNullOrEmpty(name)) continue;
string stuffDefName = TryGetString(itemDict, "stuffDefName", out string sdn) ? sdn :
(TryGetString(itemDict, "stuff", out string s) ? s :
(TryGetString(itemDict, "material", out string m) ? m : null));
// Extract count
string countStr = ExtractTag(itemXml, "count");
if (string.IsNullOrEmpty(countStr)) continue;
if (!int.TryParse(countStr, out int count)) continue;
if (count <= 0) continue;
if (string.IsNullOrWhiteSpace(name)) continue;
if (!TryGetInt(itemDict, "count", out int count) || count <= 0) continue;
// Search for ThingDef
ThingDef def = null;
// 1. Try exact defName match
def = DefDatabase<ThingDef>.GetNamed(name.Trim(), false);
// 2. Try exact label match (case-insensitive)
if (def == null)
{
foreach (var d in DefDatabase<ThingDef>.AllDefs)
{
if (d.label != null && d.label.Equals(name.Trim(), StringComparison.OrdinalIgnoreCase))
{
def = d;
break;
}
}
}
// 3. Try fuzzy search (thresholded)
if (def == null)
{
var searchResult = ThingDefSearcher.ParseAndSearch(name);
if (searchResult.Count > 0)
{
def = searchResult[0].Def;
}
}
// 4. Closest-match fallback: accept the best similar item even if not an exact match.
if (def == null)
{
ThingDefSearcher.TryFindBestThingDef(name, out ThingDef best, out float score, itemsOnly: true, minScore: 0.15f);
if (best != null && score >= 0.15f)
{
def = best;
substitutions.Add($"'{name}' -> '{best.label}' (score {score:F2})");
}
}
if (def != null)
{
itemsToSpawn.Add((def, count, name, stuffDefName));
AddItem(name, count, stuffDefName, itemsToSpawn, substitutions);
}
}
if (itemsToSpawn.Count == 0)
{
// Fallback: allow natural language without <item> blocks.
if (TryGetString(parsedArgs, "name", out string singleName) && TryGetInt(parsedArgs, "count", out int singleCount))
{
string stuffDefName = TryGetString(parsedArgs, "stuffDefName", out string sdn) ? sdn :
(TryGetString(parsedArgs, "stuff", out string s) ? s :
(TryGetString(parsedArgs, "material", out string m) ? m : null));
AddItem(singleName, singleCount, stuffDefName, itemsToSpawn, substitutions);
}
}
if (itemsToSpawn.Count == 0 && !LooksLikeJson(args))
{
var parsed = ThingDefSearcher.ParseAndSearch(args);
foreach (var r in parsed)
{
@@ -122,7 +76,7 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
if (itemsToSpawn.Count == 0)
{
string msg = "Error: No valid items found in request. Usage: <spawn_resources><items><item><name>...</name><count>...</count></item></items></spawn_resources>";
string msg = "Error: No valid items found in request. Usage: {\"items\":[{\"name\":\"Steel\",\"count\":100}]}";
Messages.Message(msg, MessageTypeDefOf.RejectInput);
return msg;
}
@@ -247,7 +201,7 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
if (thingsToDrop.Count > 0)
{
DropPodUtility.DropThingsNear(dropSpot, map, thingsToDrop);
Faction faction = Find.FactionManager.FirstFactionOfDef(FactionDef.Named("Wula_PIA_Legion_Faction"));
// Avoid unresolved named placeholders if the translation system doesn't pick up NamedArguments as expected.
string template = "Wula_ResourceDrop".Translate();
@@ -294,6 +248,49 @@ namespace WulaFallenEmpire.EventSystem.AI.Tools
}
}
private static void AddItem(string name, int count, string stuffDefName, List<(ThingDef def, int count, string requestedName, string stuffDefName)> itemsToSpawn, List<string> substitutions)
{
if (string.IsNullOrWhiteSpace(name) || count <= 0) return;
ThingDef def = DefDatabase<ThingDef>.GetNamed(name.Trim(), false);
if (def == null)
{
foreach (var d in DefDatabase<ThingDef>.AllDefs)
{
if (d.label != null && d.label.Equals(name.Trim(), StringComparison.OrdinalIgnoreCase))
{
def = d;
break;
}
}
}
if (def == null)
{
var searchResult = ThingDefSearcher.ParseAndSearch(name);
if (searchResult.Count > 0)
{
def = searchResult[0].Def;
}
}
if (def == null)
{
ThingDefSearcher.TryFindBestThingDef(name, out ThingDef best, out float score, itemsOnly: true, minScore: 0.15f);
if (best != null && score >= 0.15f)
{
def = best;
substitutions.Add($"'{name}' -> '{best.label}' (score {score:F2})");
}
}
if (def != null)
{
itemsToSpawn.Add((def, count, name, stuffDefName));
}
}
private static Map GetTargetMap()
{
Map map = Find.CurrentMap;

11
Source/WulaFallenEmpire/EventSystem/AI/UI/Dialog_AIConversation.cs
View File

@@ -8,6 +8,7 @@ using RimWorld;
using UnityEngine;
using Verse;
using WulaFallenEmpire.EventSystem.AI;
using WulaFallenEmpire.EventSystem.AI.Utils;
using System.Text.RegularExpressions;
namespace WulaFallenEmpire.EventSystem.AI.UI
@@ -431,8 +432,14 @@ namespace WulaFallenEmpire.EventSystem.AI.UI
{
if (string.IsNullOrEmpty(rawResponse)) return "";
string text = rawResponse;
text = Regex.Replace(text, @"<([a-zA-Z0-9_]+)[^>]*>.*?</\1>", "", RegexOptions.Singleline);
text = Regex.Replace(text, @"<([a-zA-Z0-9_]+)[^>]*/>", "");
if (JsonToolCallParser.TryParseToolCallsFromText(text, out _, out string fragment) && !string.IsNullOrWhiteSpace(fragment))
{
int index = text.IndexOf(fragment, StringComparison.Ordinal);
if (index >= 0)
{
text = text.Remove(index, fragment.Length);
}
}
text = ExpressionTagRegex.Replace(text, "");
text = text.Trim();
return text.Split(new[] { "OPTIONS:" }, StringSplitOptions.None)[0].Trim();

18
Source/WulaFallenEmpire/EventSystem/AI/UI/Overlay_WulaLink.cs
View File

@@ -5,6 +5,7 @@ using UnityEngine;
using Verse;
using RimWorld;
using WulaFallenEmpire.EventSystem.AI;
using WulaFallenEmpire.EventSystem.AI.Utils;
namespace WulaFallenEmpire.EventSystem.AI.UI
{
@@ -386,7 +387,7 @@ namespace WulaFallenEmpire.EventSystem.AI.UI
string displayText = msg.message;
if (msg.role == "assistant")
{
displayText = StripXmlTags(msg.message)?.Trim() ?? "";
displayText = StripToolCallJson(msg.message)?.Trim() ?? "";
}
else if (msg.role == "user")
{
@@ -476,14 +477,17 @@ namespace WulaFallenEmpire.EventSystem.AI.UI
Widgets.EndScrollView();
}
private static string StripXmlTags(string text)
private static string StripToolCallJson(string text)
{
if (string.IsNullOrEmpty(text)) return text;
// Remove XML tags with content: <tag>content</tag>
string stripped = System.Text.RegularExpressions.Regex.Replace(text, @"<(?!/?(i|b|color|size|material)\b)([a-zA-Z0-9_]+)[^>]*>.*?</\2>", "", System.Text.RegularExpressions.RegexOptions.Singleline);
// Remove self-closing tags: <tag/>
stripped = System.Text.RegularExpressions.Regex.Replace(stripped, @"<([a-zA-Z0-9_]+)[^>]*/?>", "");
return stripped;
if (!JsonToolCallParser.TryParseToolCallsFromText(text, out _, out string fragment))
{
return text;
}
int index = text.IndexOf(fragment, StringComparison.Ordinal);
if (index < 0) return text;
return text.Remove(index, fragment.Length).Trim();
}
private void DrawFooter(Rect rect)

516
Source/WulaFallenEmpire/EventSystem/AI/Utils/JsonToolCallParser.cs Normal file
View File

@@ -0,0 +1,516 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Text;
namespace WulaFallenEmpire.EventSystem.AI.Utils
{
public sealed class ToolCallInfo
{
public string Id;
public string Name;
public Dictionary<string, object> Arguments;
public string ArgumentsJson;
}
public static class JsonToolCallParser
{
public static bool TryParseToolCalls(string input, out List<ToolCallInfo> toolCalls)
{
toolCalls = null;
if (string.IsNullOrWhiteSpace(input)) return false;
if (!TryParseValue(input, out object root)) return false;
if (root is not Dictionary<string, object> obj) return false;
if (!TryGetValue(obj, "tool_calls", out object callsObj)) return false;
if (callsObj is not List<object> callsList) return false;
var parsedCalls = new List<ToolCallInfo>();
foreach (var entry in callsList)
{
if (entry is not Dictionary<string, object> callObj) continue;
string id = TryGetString(callObj, "id");
string name = null;
object argsObj = null;
if (TryGetValue(callObj, "function", out object fnObj) && fnObj is Dictionary<string, object> fnDict)
{
name = TryGetString(fnDict, "name");
TryGetValue(fnDict, "arguments", out argsObj);
}
else
{
name = TryGetString(callObj, "name");
TryGetValue(callObj, "arguments", out argsObj);
}
if (string.IsNullOrWhiteSpace(name)) continue;
if (!TryNormalizeArguments(argsObj, out Dictionary<string, object> args, out string argsJson))
{
args = new Dictionary<string, object>(StringComparer.OrdinalIgnoreCase);
argsJson = "{}";
}
parsedCalls.Add(new ToolCallInfo
{
Id = id,
Name = name.Trim(),
Arguments = args,
ArgumentsJson = argsJson
});
}
toolCalls = parsedCalls;
return true;
}
public static bool TryParseToolCallsFromText(string input, out List<ToolCallInfo> toolCalls, out string jsonFragment)
{
toolCalls = null;
jsonFragment = null;
if (string.IsNullOrWhiteSpace(input)) return false;
string trimmed = input.Trim();
if (TryParseToolCalls(trimmed, out toolCalls))
{
jsonFragment = trimmed;
return true;
}
int firstBrace = trimmed.IndexOf('{');
int lastBrace = trimmed.LastIndexOf('}');
if (firstBrace >= 0 && lastBrace > firstBrace)
{
string candidate = trimmed.Substring(firstBrace, lastBrace - firstBrace + 1);
if (TryParseToolCalls(candidate, out toolCalls))
{
jsonFragment = candidate;
return true;
}
}
return false;
}
public static bool TryParseObject(string json, out Dictionary<string, object> obj)
{
obj = null;
if (string.IsNullOrWhiteSpace(json)) return false;
if (!TryParseValue(json, out object value)) return false;
if (value is not Dictionary<string, object> dict) return false;
obj = dict;
return true;
}
public static bool LooksLikeJson(string text)
{
if (string.IsNullOrWhiteSpace(text)) return false;
string trimmed = text.TrimStart();
return trimmed.StartsWith("{", StringComparison.Ordinal) || trimmed.StartsWith("[", StringComparison.Ordinal);
}
public static string SerializeToJson(object value)
{
var sb = new StringBuilder();
AppendValue(sb, value);
return sb.ToString();
}
private static void AppendValue(StringBuilder sb, object value)
{
if (value == null)
{
sb.Append("null");
return;
}
if (value is string s)
{
sb.Append('\"').Append(EscapeJson(s)).Append('\"');
return;
}
if (value is bool b)
{
sb.Append(b ? "true" : "false");
return;
}
if (value is double d)
{
sb.Append(d.ToString("0.################", CultureInfo.InvariantCulture));
return;
}
if (value is float f)
{
sb.Append(f.ToString("0.################", CultureInfo.InvariantCulture));
return;
}
if (value is int or long or short or byte)
{
sb.Append(Convert.ToString(value, CultureInfo.InvariantCulture));
return;
}
if (value is Dictionary<string, object> obj)
{
sb.Append('{');
bool first = true;
foreach (var kvp in obj)
{
if (!first) sb.Append(',');
first = false;
sb.Append('\"').Append(EscapeJson(kvp.Key)).Append('\"').Append(':');
AppendValue(sb, kvp.Value);
}
sb.Append('}');
return;
}
if (value is List<object> list)
{
sb.Append('[');
for (int i = 0; i < list.Count; i++)
{
if (i > 0) sb.Append(',');
AppendValue(sb, list[i]);
}
sb.Append(']');
return;
}
sb.Append('\"').Append(EscapeJson(Convert.ToString(value, CultureInfo.InvariantCulture) ?? "")).Append('\"');
}
private static bool TryNormalizeArguments(object argsObj, out Dictionary<string, object> args, out string argsJson)
{
args = null;
argsJson = null;
if (argsObj == null)
{
args = new Dictionary<string, object>(StringComparer.OrdinalIgnoreCase);
argsJson = "{}";
return true;
}
if (argsObj is Dictionary<string, object> dict)
{
args = dict;
argsJson = SerializeToJson(dict);
return true;
}
if (argsObj is string s)
{
if (string.IsNullOrWhiteSpace(s))
{
args = new Dictionary<string, object>(StringComparer.OrdinalIgnoreCase);
argsJson = "{}";
return true;
}
if (TryParseObject(s, out Dictionary<string, object> parsed))
{
args = parsed;
argsJson = SerializeToJson(parsed);
return true;
}
return false;
}
return false;
}
private static bool TryParseValue(string json, out object value)
{
value = null;
var reader = new JsonReader(json);
if (!reader.TryReadValue(out value)) return false;
reader.SkipWhitespace();
return reader.IsAtEnd;
}
private static string EscapeJson(string value)
{
if (string.IsNullOrEmpty(value)) return "";
var sb = new StringBuilder();
foreach (char c in value)
{
switch (c)
{
case '\\': sb.Append("\\\\"); break;
case '\"': sb.Append("\\\""); break;
case '\n': sb.Append("\\n"); break;
case '\r': sb.Append("\\r"); break;
case '\t': sb.Append("\\t"); break;
default:
if (c < 0x20)
{
sb.Append("\\u").Append(((int)c).ToString("x4", CultureInfo.InvariantCulture));
}
else
{
sb.Append(c);
}
break;
}
}
return sb.ToString();
}
private static bool TryGetValue(Dictionary<string, object> obj, string key, out object value)
{
if (obj == null)
{
value = null;
return false;
}
foreach (var kvp in obj)
{
if (string.Equals(kvp.Key, key, StringComparison.OrdinalIgnoreCase))
{
value = kvp.Value;
return true;
}
}
value = null;
return false;
}
private static string TryGetString(Dictionary<string, object> obj, string key)
{
if (TryGetValue(obj, key, out object value) && value != null)
{
return Convert.ToString(value, CultureInfo.InvariantCulture);
}
return null;
}
private sealed class JsonReader
{
private readonly string _text;
private int _index;
public JsonReader(string text)
{
_text = text ?? "";
_index = 0;
}
public bool IsAtEnd => _index >= _text.Length;
public void SkipWhitespace()
{
while (_index < _text.Length && char.IsWhiteSpace(_text[_index]))
{
_index++;
}
}
public bool TryReadValue(out object value)
{
value = null;
SkipWhitespace();
if (IsAtEnd) return false;
char c = _text[_index];
if (c == '{') return TryReadObject(out value);
if (c == '[') return TryReadArray(out value);
if (c == '\"') return TryReadString(out value);
if (c == '-' || char.IsDigit(c)) return TryReadNumber(out value);
if (TryReadLiteral("true")) { value = true; return true; }
if (TryReadLiteral("false")) { value = false; return true; }
if (TryReadLiteral("null")) { value = null; return true; }
return false;
}
private bool TryReadObject(out object value)
{
value = null;
if (!TryReadChar('{')) return false;
SkipWhitespace();
var dict = new Dictionary<string, object>(StringComparer.OrdinalIgnoreCase);
if (TryReadChar('}'))
{
value = dict;
return true;
}
while (true)
{
SkipWhitespace();
if (!TryReadString(out object keyObj)) return false;
string key = keyObj as string ?? "";
SkipWhitespace();
if (!TryReadChar(':')) return false;
if (!TryReadValue(out object itemValue)) return false;
dict[key] = itemValue;
SkipWhitespace();
if (TryReadChar('}'))
{
value = dict;
return true;
}
if (!TryReadChar(',')) return false;
}
}
private bool TryReadArray(out object value)
{
value = null;
if (!TryReadChar('[')) return false;
SkipWhitespace();
var list = new List<object>();
if (TryReadChar(']'))
{
value = list;
return true;
}
while (true)
{
if (!TryReadValue(out object item)) return false;
list.Add(item);
SkipWhitespace();
if (TryReadChar(']'))
{
value = list;
return true;
}
if (!TryReadChar(',')) return false;
}
}
private bool TryReadString(out object value)
{
value = null;
if (!TryReadChar('\"')) return false;
var sb = new StringBuilder();
while (_index < _text.Length)
{
char c = _text[_index++];
if (c == '\"')
{
value = sb.ToString();
return true;
}
if (c == '\\')
{
if (_index >= _text.Length) return false;
char esc = _text[_index++];
switch (esc)
{
case '\"': sb.Append('\"'); break;
case '\\': sb.Append('\\'); break;
case '/': sb.Append('/'); break;
case 'b': sb.Append('\b'); break;
case 'f': sb.Append('\f'); break;
case 'n': sb.Append('\n'); break;
case 'r': sb.Append('\r'); break;
case 't': sb.Append('\t'); break;
case 'u':
if (_index + 4 > _text.Length) return false;
string hex = _text.Substring(_index, 4);
if (!int.TryParse(hex, NumberStyles.HexNumber, CultureInfo.InvariantCulture, out int code))
{
return false;
}
sb.Append((char)code);
_index += 4;
break;
default:
return false;
}
}
else
{
sb.Append(c);
}
}
return false;
}
private bool TryReadNumber(out object value)
{
value = null;
int start = _index;
if (_text[_index] == '-') _index++;
while (_index < _text.Length && char.IsDigit(_text[_index]))
{
_index++;
}
bool hasDot = false;
if (_index < _text.Length && _text[_index] == '.')
{
hasDot = true;
_index++;
while (_index < _text.Length && char.IsDigit(_text[_index]))
{
_index++;
}
}
if (_index < _text.Length && (_text[_index] == 'e' || _text[_index] == 'E'))
{
hasDot = true;
_index++;
if (_index < _text.Length && (_text[_index] == '+' || _text[_index] == '-'))
{
_index++;
}
while (_index < _text.Length && char.IsDigit(_text[_index]))
{
_index++;
}
}
string number = _text.Substring(start, _index - start);
if (!hasDot && long.TryParse(number, NumberStyles.Integer, CultureInfo.InvariantCulture, out long longVal))
{
value = longVal;
return true;
}
if (double.TryParse(number, NumberStyles.Float, CultureInfo.InvariantCulture, out double dbl))
{
value = dbl;
return true;
}
return false;
}
private bool TryReadLiteral(string literal)
{
SkipWhitespace();
if (_text.Length - _index < literal.Length) return false;
if (string.Compare(_text, _index, literal, 0, literal.Length, StringComparison.OrdinalIgnoreCase) != 0)
{
return false;
}
_index += literal.Length;
return true;
}
private bool TryReadChar(char expected)
{
SkipWhitespace();
if (_index >= _text.Length) return false;
if (_text[_index] != expected) return false;
_index++;
return true;
}
}
}
}

83
Source/WulaFallenEmpire/WulaAI_DevDocs/WulaAI_Gemini_Integration_Handover.md
View File

@@ -1,83 +0,0 @@
# Wula AI x Gemini Integration: Technical Handover Document
**Version**: 1.0
**Date**: 2025-12-28
**Author**: AntiGravity (Agent)
**Target Audience**: Codex / Future Maintainers
---
## 1. Overview
This document details the specific challenges, bugs, and architectural decisions made to stabilize the integration between **WulaFallenEmpire** (RimWorld Mod) and **Gemini 3 / OpenAI-Compatible Agents**. It specifically addresses "stubborn" issues related to API format compliance, JSON construction, and multimodal context persistence.
---
## 2. Critical Issues & Fixes
### 2.1 The "Streaming" Trap (SSE Handling)
**Symptoms**: AI responses were truncated (e.g., only "Comman" displayed instead of "Commander").
**Root Cause**: Even when `stream: false` is explicitly requested in the payload, some API providers (or reverse proxies wrapping Gemini) force a **Server-Sent Events (SSE)** response format (`data: {...}`). The original client only parsed the first line.
**Fix Implementation**:
- **File**: `SimpleAIClient.cs` -> `ExtractContent`
- **Logic**: Inspects response for `data:` prefix. If found, it iterates through **ALL** lines, strips `data:`, parses individual JSON chunks, and aggregates the `choices[0].delta.content` into a single string.
- **Defense**: This ensures compatibility with both standard JSON responses and forced Stream responses.
### 2.2 The "Trailing Comma" Crash (HTTP 400)
**Symptoms**: AI actions failed silently or returned `400 Bad Request`.
**Root Cause**: In `SimpleAIClient.cs`, the JSON payload construction loop had a logic flaw.
- When filtering out `toolcall` roles inside the loop, the index `i` check `(i < messages.Count - 1)` failed to account for skipped items, leaving a trailing comma after the last valid item: `[{"role":"user",...},]` -> **Invalid JSON**.
- Additionally, if the message list was empty (or all items filtered), the comma after the System Message remained: `[{"role":"system",...},]` -> **Invalid JSON**.
**Fix Implementation**:
- **Logic**:
1. Pre-filter `validMessages` into a separate list **before** JSON construction.
2. Only append the comma after the System Message `if (validMessages.Count > 0)`.
3. Iterate `validMessages` to guarantee correct comma placement between items.
### 2.3 Gemini 3's "JSON Obsession" & The Dual-Defense Strategy
**Symptoms**: Gemini 3 Flash Preview ignores System Prompts demanding XML (`<visual_click>`) and persistently outputs JSON (`[{"action":"click"...}]`).
**Root Cause**: RLHF tuning of newer models biases them heavily towards standard JSON tool-calling schemas, overriding prompt constraints.
**Strategy**: **"Principled Compromise"** (Double Defense).
1. **Layer 1 (Prompt)**: Explicitly list JSON and Markdown as `INVALID EXAMPLES` in `AIIntelligenceCore.cs`. This discourages compliance-oriented models from using them.
2. **Layer 2 (Code Fallback)**: If XML regex fails, the system attempts to parse **Markdown JSON Blocks** (` ```json ... ``` `).
- **File**: `AIIntelligenceCore.cs` -> `ExecuteXmlToolsForPhase`
- **Logic**: Extracts `point` arrays `[x, y]` and synthesizes a valid `<visual_click>` XML tag internally.
### 2.4 The Coordinate System Mess
**Symptoms**: Clicks occurred off-screen or at (0,0).
**Root Cause**:
- Gemini 3 often returns coordinates in a **0-1000** scale (e.g., `[115, 982]`).
- Previous logic used `Screen.width` normalization, which is **not thread-safe** and caused crashes or incorrect scaling if the assumption was pixel coordinates.
**Fix Implementation**:
- **Logic**: In the JSON Fallback parser, if `x > 1` or `y > 1`, divide by **1000.0f**. This standardizes coordinates to the mod's required 0-1 proportional format.
### 2.5 Visual Context Persistence (The "Blind Reply" Bug)
**Symptoms**: AI acted correctly (Phase 2) but "forgot" what it saw when replying to the user (Phase 3), or hallucinated headers.
**Root Cause**:
- Phase 3 (Reply) sends a message history ending with System Tool Results.
- `SimpleAIClient` only attached the image if the **very last message** was from `user`.
- Thus, in Phase 3, the image was dropped, rendering the AI blind.
**Fix Implementation**:
- **File**: `SimpleAIClient.cs`
- **Logic**: Instead of checking the last index, the code now searches **backwards** for the `lastUserIndex`. The image is attached to that specific user message, regardless of how many system messages follow it.
---
## 3. Future Maintenance Guide
### If Gemini 4 Breaks Format Again:
1. **Check `SimpleAIClient.cs`**: Ensure the JSON parser handles whatever new wrapper they add (e.g., nested `candidates`).
2. **Check `AIIntelligenceCore.cs`**: If it invents a new tool format (e.g., YAML), add a regex parser in `ExecuteXmlToolsForPhase` similar to the JSON Fallback. **Do not fight the model; adapt to it.**
### If API Errors Return:
1. Enable `DevMode` in RimWorld.
2. Check `Player.log` for `[WulaAI] Request Payload`.
3. Copy the payload to a JSON Validator. **Look for trailing commas.**
### Adding New Visual Tools:
1. Define tool in `Tools/`.
2. Update `GetToolSystemInstruction` whitelist.
3. **Crucially**: If the tool helps with **Action** (Silent), ensure `GetPhaseInstruction` enforces silence. If it helps with **Reply** (Descriptive), ensure it runs in Phase 3.
---
**End of Handover.**

58
Source/WulaFallenEmpire/WulaAI_DevDocs/deepseek/JsonOutput.md Normal file
View File

@@ -0,0 +1,58 @@
JSON Output
In many scenarios, users need the model to output in strict JSON format to achieve structured output, facilitating subsequent parsing.
DeepSeek provides JSON Output to ensure the model outputs valid JSON strings.
Notice
To enable JSON Output, users should:
Set the response_format parameter to {'type': 'json_object'}.
Include the word "json" in the system or user prompt, and provide an example of the desired JSON format to guide the model in outputting valid JSON.
Set the max_tokens parameter reasonably to prevent the JSON string from being truncated midway.
When using the JSON Output feature, the API may occasionally return empty content. We are actively working on optimizing this issue. You can try modifying the prompt to mitigate such problems.
Sample Code
Here is the complete Python code demonstrating the use of JSON Output:
import json
from openai import OpenAI
client = OpenAI(
api_key="<your api key>",
base_url="https://api.deepseek.com",
)
system_prompt = """
The user will provide some exam text. Please parse the "question" and "answer" and output them in JSON format.
EXAMPLE INPUT:
Which is the highest mountain in the world? Mount Everest.
EXAMPLE JSON OUTPUT:
{
"question": "Which is the highest mountain in the world?",
"answer": "Mount Everest"
}
"""
user_prompt = "Which is the longest river in the world? The Nile River."
messages = [{"role": "system", "content": system_prompt},
{"role": "user", "content": user_prompt}]
response = client.chat.completions.create(
model="deepseek-chat",
messages=messages,
response_format={
'type': 'json_object'
}
)
print(json.loads(response.choices[0].message.content))
The model will output:
{
"question": "Which is the longest river in the world?",
"answer": "The Nile River"
}

273
Source/WulaFallenEmpire/WulaAI_DevDocs/deepseek/ToolCalls.md Normal file
View File

@@ -0,0 +1,273 @@
Tool Calls
Tool Calls allows the model to call external tools to enhance its capabilities.
Non-thinking Mode
Sample Code
Here is an example of using Tool Calls to get the current weather information of the user's location, demonstrated with complete Python code.
For the specific API format of Tool Calls, please refer to the Chat Completion documentation.
from openai import OpenAI
def send_messages(messages):
response = client.chat.completions.create(
model="deepseek-chat",
messages=messages,
tools=tools
)
return response.choices[0].message
client = OpenAI(
api_key="<your api key>",
base_url="https://api.deepseek.com",
)
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get weather of a location, the user should supply a location first.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
}
},
"required": ["location"]
},
}
},
]
messages = [{"role": "user", "content": "How's the weather in Hangzhou, Zhejiang?"}]
message = send_messages(messages)
print(f"User>\t {messages[0]['content']}")
tool = message.tool_calls[0]
messages.append(message)
messages.append({"role": "tool", "tool_call_id": tool.id, "content": "24℃"})
message = send_messages(messages)
print(f"Model>\t {message.content}")
The execution flow of this example is as follows:
User: Asks about the current weather in Hangzhou
Model: Returns the function get_weather({location: 'Hangzhou'})
User: Calls the function get_weather({location: 'Hangzhou'}) and provides the result to the model
Model: Returns in natural language, "The current temperature in Hangzhou is 24°C."
Note: In the above code, the functionality of the get_weather function needs to be provided by the user. The model itself does not execute specific functions.
Thinking Mode
From DeepSeek-V3.2, the API supports tool use in the thinking mode. For more details, please refer to Thinking Mode
strict Mode (Beta)
In strict mode, the model strictly adheres to the format requirements of the Function's JSON schema when outputting a tool call, ensuring that the model's output complies with the user's definition. It is supported by both thinking and non-thinking mode.
To use strict mode, you need to:：
Use base_url="https://api.deepseek.com/beta" to enable Beta features
In the tools parameter，all function need to set the strict property to true
The server will validate the JSON Schema of the Function provided by the user. If the schema does not conform to the specifications or contains JSON schema types that are not supported by the server, an error message will be returned
The following is an example of a tool definition in the strict mode:
{
"type": "function",
"function": {
"name": "get_weather",
"strict": true,
"description": "Get weather of a location, the user should supply a location first.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
}
},
"required": ["location"],
"additionalProperties": false
}
}
}
Support Json Schema Types In strict Mode
object
string
number
integer
boolean
array
enum
anyOf
object
The object defines a nested structure containing key-value pairs, where properties specifies the schema for each key (or property) within the object. All properties of every object must be set as required, and the additionalProperties attribute of the object must be set to false.
Example：
{
"type": "object",
"properties": {
"name": { "type": "string" },
"age": { "type": "integer" }
},
"required": ["name", "age"],
"additionalProperties": false
}
string
Supported parameters:
pattern: Uses regular expressions to constrain the format of the string
format: Validates the string against predefined common formats. Currently supported formats:
email: Email address
hostname: Hostname
ipv4: IPv4 address
ipv6: IPv6 address
uuid: UUID
Unsupported parameters:
minLength
maxLength
Example:
{
"type": "object",
"properties": {
"user_email": {
"type": "string",
"description": "The user's email address",
"format": "email"
},
"zip_code": {
"type": "string",
"description": "Six digit postal code",
"pattern": "^\\d{6}$"
}
}
}
number/integer
Supported parameters:
const: Specifies a constant numeric value
default: Defines the default value of the number
minimum: Specifies the minimum value
maximum: Specifies the maximum value
exclusiveMinimum: Defines a value that the number must be greater than
exclusiveMaximum: Defines a value that the number must be less than
multipleOf: Ensures that the number is a multiple of the specified value
Example:
{
"type": "object",
"properties": {
"score": {
"type": "integer",
"description": "A number from 1-5, which represents your rating, the higher, the better",
"minimum": 1,
"maximum": 5
}
},
"required": ["score"],
"additionalProperties": false
}
array
Unsupported parameters:
minItems
maxItems
Example：
{
"type": "object",
"properties": {
"keywords": {
"type": "array",
"description": "Five keywords of the article, sorted by importance",
"items": {
"type": "string",
"description": "A concise and accurate keyword or phrase."
}
}
},
"required": ["keywords"],
"additionalProperties": false
}
enum
The enum ensures that the output is one of the predefined options. For example, in the case of order status, it can only be one of a limited set of specified states.
Example：
{
"type": "object",
"properties": {
"order_status": {
"type": "string",
"description": "Ordering status",
"enum": ["pending", "processing", "shipped", "cancelled"]
}
}
}
anyOf
Matches any one of the provided schemas, allowing fields to accommodate multiple valid formats. For example, a user's account could be either an email address or a phone number:
{
"type": "object",
"properties": {
"account": {
"anyOf": [
{ "type": "string", "format": "email", "description": "可以是电子邮件地址" },
{ "type": "string", "pattern": "^\\d{11}$", "description": "或11位手机号码" }
]
}
}
}
$ref and $def
You can use $def to define reusable modules and then use $ref to reference them, reducing schema repetition and enabling modularization. Additionally, $ref can be used independently to define recursive structures.
{
"type": "object",
"properties": {
"report_date": {
"type": "string",
"description": "The date when the report was published"
},
"authors": {
"type": "array",
"description": "The authors of the report",
"items": {
"$ref": "#/$def/author"
}
}
},
"required": ["report_date", "authors"],
"additionalProperties": false,
"$def": {
"authors": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "author's name"
},
"institution": {
"type": "string",
"description": "author's institution"
},
"email": {
"type": "string",
"format": "email",
"description": "author's email"
}
},
"additionalProperties": false,
"required": ["name", "institution", "email"]
}
}
}

1511
Source/WulaFallenEmpire/WulaAI_DevDocs/google/Function-calling-with-the-Gemini-API.md Normal file
View File

File diff suppressed because it is too large Load Diff

533
Source/WulaFallenEmpire/WulaAI_DevDocs/google/Structured-Outputs.md Normal file
View File

@@ -0,0 +1,533 @@
<br />
You can configure Gemini models to generate responses that adhere to a provided JSON Schema. This capability guarantees predictable and parsable results, ensures format and type-safety, enables the programmatic detection of refusals, and simplifies prompting.
Using structured outputs is ideal for a wide range of applications:
- **Data extraction:**Pull specific information from unstructured text, like extracting names, dates, and amounts from an invoice.
- **Structured classification:**Classify text into predefined categories and assign structured labels, such as categorizing customer feedback by sentiment and topic.
- **Agentic workflows:**Generate structured data that can be used to call other tools or APIs, like creating a character sheet for a game or filling out a form.
In addition to supporting JSON Schema in the REST API, the Google GenAI SDKs for Python and JavaScript also make it easy to define object schemas using[Pydantic](https://docs.pydantic.dev/latest/)and[Zod](https://zod.dev/), respectively. The example below demonstrates how to extract information from unstructured text that conforms to a schema defined in code.
Recipe ExtractorContent ModerationRecursive Structures
This example demonstrates how to extract structured data from text using basic JSON Schema types like`object`,`array`,`string`, and`integer`.
### Python
from google import genai
from pydantic import BaseModel, Field
from typing import List, Optional
class Ingredient(BaseModel):
name: str = Field(description="Name of the ingredient.")
quantity: str = Field(description="Quantity of the ingredient, including units.")
class Recipe(BaseModel):
recipe_name: str = Field(description="The name of the recipe.")
prep_time_minutes: Optional[int] = Field(description="Optional time in minutes to prepare the recipe.")
ingredients: List[Ingredient]
instructions: List[str]
client = genai.Client()
prompt = """
Please extract the recipe from the following text.
The user wants to make delicious chocolate chip cookies.
They need 2 and 1/4 cups of all-purpose flour, 1 teaspoon of baking soda,
1 teaspoon of salt, 1 cup of unsalted butter (softened), 3/4 cup of granulated sugar,
3/4 cup of packed brown sugar, 1 teaspoon of vanilla extract, and 2 large eggs.
For the best part, they'll need 2 cups of semisweet chocolate chips.
First, preheat the oven to 375°F (190°C). Then, in a small bowl, whisk together the flour,
baking soda, and salt. In a large bowl, cream together the butter, granulated sugar, and brown sugar
until light and fluffy. Beat in the vanilla and eggs, one at a time. Gradually beat in the dry
ingredients until just combined. Finally, stir in the chocolate chips. Drop by rounded tablespoons
onto ungreased baking sheets and bake for 9 to 11 minutes.
"""
response = client.models.generate_content(
model="gemini-2.5-flash",
contents=prompt,
config={
"response_mime_type": "application/json",
"response_json_schema": Recipe.model_json_schema(),
},
)
recipe = Recipe.model_validate_json(response.text)
print(recipe)
### JavaScript
import { GoogleGenAI } from "@google/genai";
import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";
const ingredientSchema = z.object({
name: z.string().describe("Name of the ingredient."),
quantity: z.string().describe("Quantity of the ingredient, including units."),
});
const recipeSchema = z.object({
recipe_name: z.string().describe("The name of the recipe."),
prep_time_minutes: z.number().optional().describe("Optional time in minutes to prepare the recipe."),
ingredients: z.array(ingredientSchema),
instructions: z.array(z.string()),
});
const ai = new GoogleGenAI({});
const prompt = `
Please extract the recipe from the following text.
The user wants to make delicious chocolate chip cookies.
They need 2 and 1/4 cups of all-purpose flour, 1 teaspoon of baking soda,
1 teaspoon of salt, 1 cup of unsalted butter (softened), 3/4 cup of granulated sugar,
3/4 cup of packed brown sugar, 1 teaspoon of vanilla extract, and 2 large eggs.
For the best part, they'll need 2 cups of semisweet chocolate chips.
First, preheat the oven to 375°F (190°C). Then, in a small bowl, whisk together the flour,
baking soda, and salt. In a large bowl, cream together the butter, granulated sugar, and brown sugar
until light and fluffy. Beat in the vanilla and eggs, one at a time. Gradually beat in the dry
ingredients until just combined. Finally, stir in the chocolate chips. Drop by rounded tablespoons
onto ungreased baking sheets and bake for 9 to 11 minutes.
`;
const response = await ai.models.generateContent({
model: "gemini-2.5-flash",
contents: prompt,
config: {
responseMimeType: "application/json",
responseJsonSchema: zodToJsonSchema(recipeSchema),
},
});
const recipe = recipeSchema.parse(JSON.parse(response.text));
console.log(recipe);
### Go
package main
import (
"context"
"fmt"
"log"
"google.golang.org/genai"
)
func main() {
ctx := context.Background()
client, err := genai.NewClient(ctx, nil)
if err != nil {
log.Fatal(err)
}
prompt := `
Please extract the recipe from the following text.
The user wants to make delicious chocolate chip cookies.
They need 2 and 1/4 cups of all-purpose flour, 1 teaspoon of baking soda,
1 teaspoon of salt, 1 cup of unsalted butter (softened), 3/4 cup of granulated sugar,
3/4 cup of packed brown sugar, 1 teaspoon of vanilla extract, and 2 large eggs.
For the best part, they'll need 2 cups of semisweet chocolate chips.
First, preheat the oven to 375°F (190°C). Then, in a small bowl, whisk together the flour,
baking soda, and salt. In a large bowl, cream together the butter, granulated sugar, and brown sugar
until light and fluffy. Beat in the vanilla and eggs, one at a time. Gradually beat in the dry
ingredients until just combined. Finally, stir in the chocolate chips. Drop by rounded tablespoons
onto ungreased baking sheets and bake for 9 to 11 minutes.
`
config := &genai.GenerateContentConfig{
ResponseMIMEType: "application/json",
ResponseJsonSchema: map[string]any{
"type": "object",
"properties": map[string]any{
"recipe_name": map[string]any{
"type": "string",
"description": "The name of the recipe.",
},
"prep_time_minutes": map[string]any{
"type": "integer",
"description": "Optional time in minutes to prepare the recipe.",
},
"ingredients": map[string]any{
"type": "array",
"items": map[string]any{
"type": "object",
"properties": map[string]any{
"name": map[string]any{
"type": "string",
"description": "Name of the ingredient.",
},
"quantity": map[string]any{
"type": "string",
"description": "Quantity of the ingredient, including units.",
},
},
"required": []string{"name", "quantity"},
},
},
"instructions": map[string]any{
"type": "array",
"items": map[string]any{"type": "string"},
},
},
"required": []string{"recipe_name", "ingredients", "instructions"},
},
}
result, err := client.Models.GenerateContent(
ctx,
"gemini-2.5-flash",
genai.Text(prompt),
config,
)
if err != nil {
log.Fatal(err)
}
fmt.Println(result.Text())
}
### REST
curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent" \
-H "x-goog-api-key: $GEMINI_API_KEY" \
-H 'Content-Type: application/json' \
-X POST \
-d '{
"contents": [{
"parts":[
{ "text": "Please extract the recipe from the following text.\nThe user wants to make delicious chocolate chip cookies.\nThey need 2 and 1/4 cups of all-purpose flour, 1 teaspoon of baking soda,\n1 teaspoon of salt, 1 cup of unsalted butter (softened), 3/4 cup of granulated sugar,\n3/4 cup of packed brown sugar, 1 teaspoon of vanilla extract, and 2 large eggs.\nFor the best part, they will need 2 cups of semisweet chocolate chips.\nFirst, preheat the oven to 375°F (190°C). Then, in a small bowl, whisk together the flour,\nbaking soda, and salt. In a large bowl, cream together the butter, granulated sugar, and brown sugar\nuntil light and fluffy. Beat in the vanilla and eggs, one at a time. Gradually beat in the dry\ningredients until just combined. Finally, stir in the chocolate chips. Drop by rounded tablespoons\nonto ungreased baking sheets and bake for 9 to 11 minutes." }
]
}],
"generationConfig": {
"responseMimeType": "application/json",
"responseJsonSchema": {
"type": "object",
"properties": {
"recipe_name": {
"type": "string",
"description": "The name of the recipe."
},
"prep_time_minutes": {
"type": "integer",
"description": "Optional time in minutes to prepare the recipe."
},
"ingredients": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": { "type": "string", "description": "Name of the ingredient."},
"quantity": { "type": "string", "description": "Quantity of the ingredient, including units."}
},
"required": ["name", "quantity"]
}
},
"instructions": {
"type": "array",
"items": { "type": "string" }
}
},
"required": ["recipe_name", "ingredients", "instructions"]
}
}
}'
**Example Response:**
{
"recipe_name": "Delicious Chocolate Chip Cookies",
"ingredients": [
{
"name": "all-purpose flour",
"quantity": "2 and 1/4 cups"
},
{
"name": "baking soda",
"quantity": "1 teaspoon"
},
{
"name": "salt",
"quantity": "1 teaspoon"
},
{
"name": "unsalted butter (softened)",
"quantity": "1 cup"
},
{
"name": "granulated sugar",
"quantity": "3/4 cup"
},
{
"name": "packed brown sugar",
"quantity": "3/4 cup"
},
{
"name": "vanilla extract",
"quantity": "1 teaspoon"
},
{
"name": "large eggs",
"quantity": "2"
},
{
"name": "semisweet chocolate chips",
"quantity": "2 cups"
}
],
"instructions": [
"Preheat the oven to 375°F (190°C).",
"In a small bowl, whisk together the flour, baking soda, and salt.",
"In a large bowl, cream together the butter, granulated sugar, and brown sugar until light and fluffy.",
"Beat in the vanilla and eggs, one at a time.",
"Gradually beat in the dry ingredients until just combined.",
"Stir in the chocolate chips.",
"Drop by rounded tablespoons onto ungreased baking sheets and bake for 9 to 11 minutes."
]
}
## Streaming
You can stream structured outputs, which allows you to start processing the response as it's being generated, without having to wait for the entire output to be complete. This can improve the perceived performance of your application.
The streamed chunks will be valid partial JSON strings, which can be concatenated to form the final, complete JSON object.
### Python
from google import genai
from pydantic import BaseModel, Field
from typing import Literal
class Feedback(BaseModel):
sentiment: Literal["positive", "neutral", "negative"]
summary: str
client = genai.Client()
prompt = "The new UI is incredibly intuitive and visually appealing. Great job. Add a very long summary to test streaming!"
response_stream = client.models.generate_content_stream(
model="gemini-2.5-flash",
contents=prompt,
config={
"response_mime_type": "application/json",
"response_json_schema": Feedback.model_json_schema(),
},
)
for chunk in response_stream:
print(chunk.candidates[0].content.parts[0].text)
### JavaScript
import { GoogleGenAI } from "@google/genai";
import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";
const ai = new GoogleGenAI({});
const prompt = "The new UI is incredibly intuitive and visually appealing. Great job! Add a very long summary to test streaming!";
const feedbackSchema = z.object({
sentiment: z.enum(["positive", "neutral", "negative"]),
summary: z.string(),
});
const stream = await ai.models.generateContentStream({
model: "gemini-2.5-flash",
contents: prompt,
config: {
responseMimeType: "application/json",
responseJsonSchema: zodToJsonSchema(feedbackSchema),
},
});
for await (const chunk of stream) {
console.log(chunk.candidates[0].content.parts[0].text)
}
## Structured outputs with tools
| **Preview:** This is a feature available only for the Gemini 3 series models,`gemini-3-pro-preview`and`gemini-3-flash-preview`.
Gemini 3 lets you combine Structured Outputs with built-in tools, including[Grounding with Google Search](https://ai.google.dev/gemini-api/docs/google-search),[URL Context](https://ai.google.dev/gemini-api/docs/url-context), and[Code Execution](https://ai.google.dev/gemini-api/docs/code-execution).
### Python
from google import genai
from pydantic import BaseModel, Field
from typing import List
class MatchResult(BaseModel):
winner: str = Field(description="The name of the winner.")
final_match_score: str = Field(description="The final match score.")
scorers: List[str] = Field(description="The name of the scorer.")
client = genai.Client()
response = client.models.generate_content(
model="gemini-3-pro-preview",
contents="Search for all details for the latest Euro.",
config={
"tools": [
{"google_search": {}},
{"url_context": {}}
],
"response_mime_type": "application/json",
"response_json_schema": MatchResult.model_json_schema(),
},
)
result = MatchResult.model_validate_json(response.text)
print(result)
### JavaScript
import { GoogleGenAI } from "@google/genai";
import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";
const ai = new GoogleGenAI({});
const matchSchema = z.object({
winner: z.string().describe("The name of the winner."),
final_match_score: z.string().describe("The final score."),
scorers: z.array(z.string()).describe("The name of the scorer.")
});
async function run() {
const response = await ai.models.generateContent({
model: "gemini-3-pro-preview",
contents: "Search for all details for the latest Euro.",
config: {
tools: [
{ googleSearch: {} },
{ urlContext: {} }
],
responseMimeType: "application/json",
responseJsonSchema: zodToJsonSchema(matchSchema),
},
});
const match = matchSchema.parse(JSON.parse(response.text));
console.log(match);
}
run();
### REST
curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-3-pro-preview:generateContent" \
-H "x-goog-api-key: $GEMINI_API_KEY" \
-H 'Content-Type: application/json' \
-X POST \
-d '{
"contents": [{
"parts": [{"text": "Search for all details for the latest Euro."}]
}],
"tools": [
{"googleSearch": {}},
{"urlContext": {}}
],
"generationConfig": {
"responseMimeType": "application/json",
"responseJsonSchema": {
"type": "object",
"properties": {
"winner": {"type": "string", "description": "The name of the winner."},
"final_match_score": {"type": "string", "description": "The final score."},
"scorers": {
"type": "array",
"items": {"type": "string"},
"description": "The name of the scorer."
}
},
"required": ["winner", "final_match_score", "scorers"]
}
}
}'
## JSON schema support
To generate a JSON object, set the`response_mime_type`in the generation configuration to`application/json`and provide a`response_json_schema`. The schema must be a valid[JSON Schema](https://json-schema.org/)that describes the desired output format.
The model will then generate a response that is a syntactically valid JSON string matching the provided schema. When using structured outputs, the model will produce outputs in the same order as the keys in the schema.
Gemini's structured output mode supports a subset of the[JSON Schema](https://json-schema.org)specification.
The following values of`type`are supported:
- **`string`**: For text.
- **`number`**: For floating-point numbers.
- **`integer`**: For whole numbers.
- **`boolean`**: For true/false values.
- **`object`**: For structured data with key-value pairs.
- **`array`**: For lists of items.
- **`null`** : To allow a property to be null, include`"null"`in the type array (e.g.,`{"type": ["string", "null"]}`).
These descriptive properties help guide the model:
- **`title`**: A short description of a property.
- **`description`**: A longer and more detailed description of a property.
### Type-specific properties
**For`object`values:**
- **`properties`**: An object where each key is a property name and each value is a schema for that property.
- **`required`**: An array of strings, listing which properties are mandatory.
- **`additionalProperties`** : Controls whether properties not listed in`properties`are allowed. Can be a boolean or a schema.
**For`string`values:**
- **`enum`**: Lists a specific set of possible strings for classification tasks.
- **`format`** : Specifies a syntax for the string, such as`date-time`,`date`,`time`.
**For`number`and`integer`values:**
- **`enum`**: Lists a specific set of possible numeric values.
- **`minimum`**: The minimum inclusive value.
- **`maximum`**: The maximum inclusive value.
**For`array`values:**
- **`items`**: Defines the schema for all items in the array.
- **`prefixItems`**: Defines a list of schemas for the first N items, allowing for tuple-like structures.
- **`minItems`**: The minimum number of items in the array.
- **`maxItems`**: The maximum number of items in the array.
## Model support
The following models support structured output:
| Model | Structured Outputs |
|------------------------|--------------------|
| Gemini 3 Pro Preview | ✔️ |
| Gemini 3 Flash Preview | ✔️ |
| Gemini 2.5 Pro | ✔️ |
| Gemini 2.5 Flash | ✔️ |
| Gemini 2.5 Flash-Lite | ✔️ |
| Gemini 2.0 Flash | ✔️\* |
| Gemini 2.0 Flash-Lite | ✔️\* |
*\* Note that Gemini 2.0 requires an explicit`propertyOrdering`list within the JSON input to define the preferred structure. You can find an example in this[cookbook](https://github.com/google-gemini/cookbook/blob/main/examples/Pdf_structured_outputs_on_invoices_and_forms.ipynb).*
## Structured outputs vs. function calling
Both structured outputs and function calling use JSON schemas, but they serve different purposes:
| Feature | Primary Use Case |
|------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **Structured Outputs** | **Formatting the final response to the user.** Use this when you want the model's*answer*to be in a specific format (e.g., extracting data from a document to save to a database). |
| **Function Calling** | **Taking action during the conversation.** Use this when the model needs to*ask you*to perform a task (e.g., "get current weather") before it can provide a final answer. |
## Best practices
- **Clear descriptions:** Use the`description`field in your schema to provide clear instructions to the model about what each property represents. This is crucial for guiding the model's output.
- **Strong typing:** Use specific types (`integer`,`string`,`enum`) whenever possible. If a parameter has a limited set of valid values, use an`enum`.
- **Prompt engineering:**Clearly state in your prompt what you want the model to do. For example, "Extract the following information from the text..." or "Classify this feedback according to the provided schema...".
- **Validation:**While structured output guarantees syntactically correct JSON, it does not guarantee the values are semantically correct. Always validate the final output in your application code before using it.
- **Error handling:**Implement robust error handling in your application to gracefully manage cases where the model's output, while schema-compliant, may not meet your business logic requirements.
## Limitations
- **Schema subset:**Not all features of the JSON Schema specification are supported. The model ignores unsupported properties.
- **Schema complexity:**The API may reject very large or deeply nested schemas. If you encounter errors, try simplifying your schema by shortening property names, reducing nesting, or limiting the number of constraints.

1844
Source/WulaFallenEmpire/WulaAI_DevDocs/openai/Structured-outputs.md Normal file
View File

File diff suppressed because it is too large Load Diff

1052
Source/WulaFallenEmpire/WulaAI_DevDocs/openai/function-calling.md Normal file
View File

File diff suppressed because it is too large Load Diff

1
llama.cpp Submodule

Submodule llama.cpp added at d77d7c5c06