diff --git a/CHANGELOG.md b/CHANGELOG.md index 147a1c9..10ae380 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,19 +2,28 @@ 格式遵循 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)，版本号遵循语义化版本。 -## [0.2.0] - 2026-07-10 +## [0.2.0] - 2026-07-12 ### Added -- AI 网关模型池自愈：同步与每日探测后自动逐个试调验证模型可用性（maxTokens=1），不可按需调用的模型（微调基座 / 区域未开放按需供给）自动标记并从模型列表、路由与探测候选中剔除；已标记模型每 20 小时复检，OCI 恢复供给后自动解除标记。`ai_model_caches` 新增 `unusable` / `unusable_reason` / `checked_at`（启动自动迁移），渠道详情接口保留标记行与原因便于排查 +- AI 网关模型黑名单：面板集中拉黑模型，全渠道剔除（模型列表 / 路由 / 探测候选不再出现，同步不入库），移除条目并重新同步后恢复；新表 `ai_model_blacklists`（启动自动迁移），管理接口 `/api/v1/ai-blacklist` +- AI 密钥模型白名单：密钥可限定放行的模型列表（空 = 不限），白名单外调用 404，模型列表只返回交集 +- 推理力度（effort）直通：Responses `reasoning.effort` 与 Anthropic Messages `output_config.effort` 原样透传上游、不做档位校验；grok-4.3 五档（`none` 完全关闭思考）、gpt-oss `minimal`-`high`、multi-agent `xhigh`（控制并行 agent 数）实测生效 +- xAI Grok 服务端工具：`/ai/v1/responses` 的 `tools` 支持 `{"type":"web_search"}` / `{"type":"x_search"}`（官方 Agent Tools 格式，可与 `function` 混用，仅非流式），响应保留 `web_search_call` 输出项与引用 ### Changed +- **AI 网关对话链路整体切换到 OCI OpenAI 兼容面**（`/actions/v1/responses`），typed chat 面全链路移除：`/ai/v1/responses` 请求体原样直通（仅强制 `store:false`），流式为上游原生事件流，gpt-oss 推理增量直达客户端；`/ai/v1/messages` 转换为 Responses 请求直通，流式经事件桥转回标准 Anthropic 事件序列，`stop_sequences` / `top_k` 等无兼容面对应物的字段改为静默忽略，思考输出不转 `thinking` 块；渠道探测改用兼容面最小请求；流式建立后绑定渠道，中断不重试不计熔断 - 后台任务「立即执行」改异步触发：接口立即返回 202，执行结果经任务日志轮询呈现；同一任务在途时重复触发返回 409「任务正在执行中」，与 cron 重叠触发静默跳过（此前同步阻塞数十秒且同一任务可并发重复执行） - 渠道探测候选跨厂商分散（上限 8，voice 等不可对话形态排除）：单一厂商在区域内全为微调基座时不再拖垮整个渠道的探测结论 -- 网关调用（对话 / 流式 / 向量化）遇「模型不可按需调用」类错误（微调基座 400 / 实体不存在 404）自动标记该模型并换渠道重试，且不计入渠道熔断；仅鉴权类错误才判定租户无配额 +- 网关调用遇「模型不可按需调用」类错误（微调基座 400 / 实体不存在 404）自动换渠道重试且不计入渠道熔断，错误信息带模型名便于加入黑名单；仅鉴权类错误才判定租户无配额 - 同名模型多条目去重优先保留非微调基座条目，降低缓存到不可调用 OCID 的概率 +### Removed + +- **`/ai/v1/chat/completions` 端点移除**（404）：OpenAI 官方新能力已集中到 Responses 且本面板真实流量为零，请迁移 `/ai/v1/responses` 或 `/ai/v1/messages` +- google / cohere 对话模型不再供给（OCI OpenAI 兼容面不承接，上游 400；cohere embed 向量模型不受影响），模型目录按 `xai.` / `meta.` / `openai.` 厂商前缀过滤 + ### Fixed - 渠道「同步模型成功但探测报错不可用」：法兰克福等区域的微调基座模型占满探测候选导致的误报（探测状态与真实可用性不符的根因） diff --git a/DASH_VERSION b/DASH_VERSION index b82608c..1474d00 100644 --- a/DASH_VERSION +++ b/DASH_VERSION @@ -1 +1 @@ -v0.1.0 +v0.2.0 diff --git a/README.md b/README.md index 5fb9cb3..decc223 100644 --- a/README.md +++ b/README.md @@ -34,7 +34,7 @@ - **资源管理**：实例（含创建/电源操作/更换公网 IP/IPv6/VNIC/引导卷）、VCN/子网/安全列表、块存储、限额与成本查询，多区域/多区间支持 - **抢机任务**：cron 周期尝试创建实例直到成功，支持熔断与通知 - **网页控制台**：实例串行控制台（xterm）与 VNC（noVNC），两跳 SSH 隧道 -- **AI 网关**：OpenAI / Anthropic 兼容端点转发 OCI GenAI（对话/Responses/Embeddings），号池渠道加权负载均衡、熔断探测、密钥管理与用量日志 +- **AI 网关**：OpenAI Responses / Anthropic Messages / Embeddings 兼容端点转发 OCI GenAI，号池渠道加权负载均衡、熔断探测、密钥管理与用量日志；对话统一走 OCI OpenAI 兼容面（xai / meta / openai 厂商模型，流式事件与推理增量原样保真），支持 xAI Grok 服务端工具 `web_search` / `x_search`（格式同 xAI 官方 Agent Tools，仅非流式）——兼容面属实测可用但未见 Oracle 文档承诺的能力，行为可能随上游调整 - **日志回传**：OCI Audit 事件经 Connector Hub → Notifications HTTPS 订阅回传入库，一键创建链路；自定义告警规则（事件类型/来源 IP 白名单/资源/频率阈值）命中即推送 - **租户治理**：IAM 用户/MFA/API Key 管理、密码策略、身份提供商（SAML）、通知收件人；多 Identity Domain 租户可按域切换管理 - **安全**：JWT + bcrypt（凭据变更旧令牌立即失效，可一键撤销全部会话）、TOTP 两步验证、OIDC/GitHub 外部登录、登录锁定、IP 限速、真实 IP 头可配、请求体/超时防护、系统操作审计 @@ -60,7 +60,7 @@ mkdir -p ./data && sudo chown 65532:65532 ./data DATA_KEY=$(openssl rand -hex 32) JWT_SECRET=$(openssl rand -hex 32) ADMIN_PASSWORD=<初始密码> docker compose up -d ``` -默认只监听 `127.0.0.1:18888`;公网访问须置于 TLS 反向代理之后,见下文「反向代理」。 +默认只监听 `127.0.0.1:18888`；公网访问须置于 TLS 反向代理之后，见下文「反向代理」。 ### 源码构建（单文件，含前端） @@ -148,6 +148,87 @@ services: - 反代默认追加的 `X-Forwarded-For` 用于还原真实客户端 IP，系统日志留痕、登录锁定与 IP 限速都依赖它 - 请求体上限建议与后端一致：`/api/*` 1MB、`/ai/*` 10MB（Caddy 用 `request_body` 按前缀分层） +## AI 网关接口 + +对外提供 OpenAI / Anthropic 兼容端点，转发 OCI GenAI On-Demand 推理。鉴权用面板创建的 AI 密钥，`Authorization: Bearer sk-...` 与 `x-api-key: sk-...` 双头均可；密钥可绑定渠道分组实现路由隔离，也可配置模型白名单（白名单外调用 404，模型列表只返回交集）。请求体上限 10MB。 + +| 端点 | 协议 | 流式 | +| --- | --- | --- | +| `POST /ai/v1/responses` | OpenAI Responses（无状态子集） | SSE 支持（服务端工具除外） | +| `POST /ai/v1/messages` | Anthropic Messages | SSE 支持 | +| `POST /ai/v1/embeddings` | OpenAI Embeddings | — | +| `GET /ai/v1/models` | OpenAI 模型列表（来自渠道模型目录） | — | + +对话端点统一转发 OCI OpenAI 兼容面（`/actions/v1/responses`），因此仅提供该面支持的 `xai.` / `meta.` / `openai.` 前缀模型；google / cohere 对话模型不在兼容面供给（上游 400），不再提供（cohere embed 模型不受影响）。兼容面属实测可用但无 Oracle 文档承诺的能力，行为可能随上游调整。**Chat Completions（`/ai/v1/chat/completions`）端点已移除**，请迁移到 Responses 或 Messages。 + +以下各表对照网关行为：✅ 转发上游；⚠️ 接受但忽略（静默丢弃，不影响请求）；❌ 拒绝（400，不发上游）。 + +### Responses 字段 + +请求体除下列例外**原样直通上游**（未列字段一并转发，效果由上游决定）： + +| 字段 | 支持 | 说明 | +| --- | --- | --- | +| `model`、`input` | ✅ | `input` 接受 string 或 item 数组 | +| `stream` | ✅ | SSE，上游原生事件流（含 gpt-oss 推理增量）；与服务端工具互斥 | +| `reasoning.effort` | ✅ | 直达上游不限档位，见下「推理力度」 | +| `store` | ⚠️ | 网关无状态，强制改写为 `false` | +| `previous_response_id`、`conversation`、`background` | ❌ | 网关不保存历史，请求需自带完整上下文 | +| `web_search` / `x_search` / `function` 之外的工具类型 | ❌ | 如 `file_search`、`code_interpreter`、`mcp` | + +**服务端工具**：`tools` 支持 `{"type":"web_search"}` 与 `{"type":"x_search"}`（xAI Grok 官方 Agent Tools 格式，可与 `function` 混用），响应保留 `web_search_call` 输出项与引用；**仅非流式**。响应 `usage.input_tokens_details.cached_tokens` 透传缓存命中量。 + +### Anthropic Messages 字段 + +网关把 Messages 请求转换为 Responses 请求送上游，响应（含流式事件序列）转回 Anthropic 形态： + +| 字段 | 支持 | 说明 | +| --- | --- | --- | +| `model`、`max_tokens`、`messages` | ✅ | `max_tokens` 必填；content 块支持 `text` / `image` / `tool_use` / `tool_result`，其余块类型 400 | +| `system` | ✅ | string 或块数组，映射为 `instructions` | +| `temperature`、`top_p` | ✅ | | +| `stream` | ✅ | SSE，标准 Anthropic 事件序列（`message_start` → `content_block_*` → `message_delta` → `message_stop`） | +| `tools`、`tool_choice` | ✅ | 映射为 Responses `function` 工具；`tool_choice` 支持 `auto` / `any` / `none` / `tool` | +| `output_config.effort` | ✅ | 直达上游不限档位，见下「推理力度」 | +| `stop_sequences`、`top_k`、`metadata`、`thinking`、`service_tier` | ⚠️ | Responses 面无对应物，静默忽略 | + +模型的思考输出（reasoning）不转为 `thinking` 块，静默丢弃；`usage.cache_read_input_tokens` 透传缓存命中量。 + +### Embeddings 字段 + +| 字段 | 支持 | 说明 | +| --- | --- | --- | +| `model`、`input` | ✅ | `input` 接受 string 或数组 | +| `dimensions` | ✅ | | +| `encoding_format`、`user` | ⚠️ | 输出恒为 float 数组 | + +### 推理力度（effort） + +两个对话协议各以原生字段控制推理深度，取值直达上游不做档位校验： + +| 协议 | 字段 | +| --- | --- | +| Responses | `reasoning.effort` | +| Anthropic Messages | `output_config.effort`（转小写后透传） | + +是否支持、可用档位与实际效果由模型决定（2026-07 兼容面实测）： + +| 模型 | 支持档位 | 实测行为 | +| --- | --- | --- | +| `xai.grok-4.3` | `none` / `minimal` / `low` / `medium` / `high` | 全部生效：`none` 完全关闭推理，默认 `low`，`high` 推理量最大 | +| `openai.gpt-oss-*` | `minimal` / `low` / `medium` / `high` | `none` 被上游 400（Harmony 格式不支持）；思考计入 completion tokens | +| `xai.grok-3-mini(-fast)` | 名义 `low` / `high` | 各档均被接受，但 `none` 并不关闭推理（仍产生思考） | +| `meta.llama-*` | 接受任意档 | 非推理模型，参数无实际效果 | +| `xai.grok-4.20-multi-agent*` | `low` / `medium` / `high` / `xhigh` | effort 控制并行 agent 数（4 / 16）而非推理深度 | +| 其余 grok（`grok-3`、`grok-4`、`grok-4-fast-*`、`grok-4-1-fast-*`、`grok-code-fast-1`、`grok-4.20-*` 单/双型号全系） | 不支持 | 携带即被上游 400（`does not support parameter reasoningEffort`） | + +不携带该字段时网关不下发，行为由模型默认档位决定。 + +### 通用行为 + +- 未知模型 404；无可用渠道 503；单次请求最多尝试 3 个渠道（429/5xx/网络错误自动换渠道并计入熔断，模型级 400/404 换渠道不计熔断，其余 4xx 原样透传）；流式建立后绑定渠道，中断不重试 +- 调用计量（渠道、token 用量、缓存命中、时延、重试数）写入面板「调用日志」；密钥可按需开启限时内容抓取 + ## 环境变量 | 变量 | 必填 | 默认值 | 说明 | diff --git a/docs/docs.go b/docs/docs.go index e27f341..52e4b7a 100644 --- a/docs/docs.go +++ b/docs/docs.go @@ -15,34 +15,6 @@ const docTemplate = `{ "host": "{{.Host}}", "basePath": "{{.BasePath}}", "paths": { - "/ai/v1/chat/completions": { - "post": { - "tags": [ - "AI 网关" - ], - "summary": "OpenAI 兼容对话补全", - "parameters": [ - { - "description": "OpenAI chat/completions 请求体(支持 stream)", - "name": "body", - "in": "body", - "required": true, - "schema": { - "type": "object" - } - } - ], - "responses": { - "200": { - "description": "OpenAI 兼容响应(流式为 SSE)", - "schema": { - "type": "object", - "additionalProperties": true - } - } - } - } - }, "/ai/v1/embeddings": { "post": { "tags": [ @@ -79,7 +51,7 @@ const docTemplate = `{ "summary": "Anthropic Messages 兼容端点", "parameters": [ { - "description": "Anthropic messages 请求体(支持 stream)", + "description": "Anthropic messages 请求体(支持 stream;经 OCI OpenAI 兼容面直通)", "name": "body", "in": "body", "required": true, @@ -124,7 +96,7 @@ const docTemplate = `{ "summary": "OpenAI Responses 兼容端点", "parameters": [ { - "description": "OpenAI responses 请求体(支持 stream)", + "description": "OpenAI responses 请求体(支持 stream;web_search/x_search 服务端工具仅非流式)", "name": "body", "in": "body", "required": true, @@ -166,6 +138,86 @@ const docTemplate = `{ } } }, + "/api/v1/ai-blacklist": { + "get": { + "security": [ + { + "BearerAuth": [] + } + ], + "tags": [ + "AI 管理" + ], + "summary": "模型黑名单列表", + "responses": { + "200": { + "description": "OK", + "schema": { + "type": "object", + "additionalProperties": true + } + } + } + }, + "post": { + "security": [ + { + "BearerAuth": [] + } + ], + "tags": [ + "AI 管理" + ], + "summary": "添加模型黑名单", + "parameters": [ + { + "description": "请求体:{name: 模型名}", + "name": "body", + "in": "body", + "required": true, + "schema": { + "type": "object" + } + } + ], + "responses": { + "201": { + "description": "Created", + "schema": { + "type": "object", + "additionalProperties": true + } + } + } + } + }, + "/api/v1/ai-blacklist/{id}": { + "delete": { + "security": [ + { + "BearerAuth": [] + } + ], + "tags": [ + "AI 管理" + ], + "summary": "移除模型黑名单(重新同步后模型恢复入池)", + "parameters": [ + { + "type": "integer", + "description": "黑名单条目 ID", + "name": "id", + "in": "path", + "required": true + } + ], + "responses": { + "204": { + "description": "无内容" + } + } + } + }, "/api/v1/ai-channels": { "get": { "security": [ diff --git a/docs/swagger.json b/docs/swagger.json index 6027c2f..58419e4 100644 --- a/docs/swagger.json +++ b/docs/swagger.json @@ -8,34 +8,6 @@ }, "basePath": "/", "paths": { - "/ai/v1/chat/completions": { - "post": { - "tags": [ - "AI 网关" - ], - "summary": "OpenAI 兼容对话补全", - "parameters": [ - { - "description": "OpenAI chat/completions 请求体(支持 stream)", - "name": "body", - "in": "body", - "required": true, - "schema": { - "type": "object" - } - } - ], - "responses": { - "200": { - "description": "OpenAI 兼容响应(流式为 SSE)", - "schema": { - "type": "object", - "additionalProperties": true - } - } - } - } - }, "/ai/v1/embeddings": { "post": { "tags": [ @@ -72,7 +44,7 @@ "summary": "Anthropic Messages 兼容端点", "parameters": [ { - "description": "Anthropic messages 请求体(支持 stream)", + "description": "Anthropic messages 请求体(支持 stream;经 OCI OpenAI 兼容面直通)", "name": "body", "in": "body", "required": true, @@ -117,7 +89,7 @@ "summary": "OpenAI Responses 兼容端点", "parameters": [ { - "description": "OpenAI responses 请求体(支持 stream)", + "description": "OpenAI responses 请求体(支持 stream;web_search/x_search 服务端工具仅非流式)", "name": "body", "in": "body", "required": true, @@ -159,6 +131,86 @@ } } }, + "/api/v1/ai-blacklist": { + "get": { + "security": [ + { + "BearerAuth": [] + } + ], + "tags": [ + "AI 管理" + ], + "summary": "模型黑名单列表", + "responses": { + "200": { + "description": "OK", + "schema": { + "type": "object", + "additionalProperties": true + } + } + } + }, + "post": { + "security": [ + { + "BearerAuth": [] + } + ], + "tags": [ + "AI 管理" + ], + "summary": "添加模型黑名单", + "parameters": [ + { + "description": "请求体:{name: 模型名}", + "name": "body", + "in": "body", + "required": true, + "schema": { + "type": "object" + } + } + ], + "responses": { + "201": { + "description": "Created", + "schema": { + "type": "object", + "additionalProperties": true + } + } + } + } + }, + "/api/v1/ai-blacklist/{id}": { + "delete": { + "security": [ + { + "BearerAuth": [] + } + ], + "tags": [ + "AI 管理" + ], + "summary": "移除模型黑名单(重新同步后模型恢复入池)", + "parameters": [ + { + "type": "integer", + "description": "黑名单条目 ID", + "name": "id", + "in": "path", + "required": true + } + ], + "responses": { + "204": { + "description": "无内容" + } + } + } + }, "/api/v1/ai-channels": { "get": { "security": [ diff --git a/docs/swagger.yaml b/docs/swagger.yaml index d739168..8c06b62 100644 --- a/docs/swagger.yaml +++ b/docs/swagger.yaml @@ -599,24 +599,6 @@ info: title: OCI Portal API version: 0.0.1 paths: - /ai/v1/chat/completions: - post: - parameters: - - description: OpenAI chat/completions 请求体(支持 stream) - in: body - name: body - required: true - schema: - type: object - responses: - "200": - description: OpenAI 兼容响应(流式为 SSE) - schema: - additionalProperties: true - type: object - summary: OpenAI 兼容对话补全 - tags: - - AI 网关 /ai/v1/embeddings: post: parameters: @@ -638,7 +620,7 @@ paths: /ai/v1/messages: post: parameters: - - description: Anthropic messages 请求体(支持 stream) + - description: Anthropic messages 请求体(支持 stream;经 OCI OpenAI 兼容面直通) in: body name: body required: true @@ -667,7 +649,7 @@ paths: /ai/v1/responses: post: parameters: - - description: OpenAI responses 请求体(支持 stream) + - description: OpenAI responses 请求体(支持 stream;web_search/x_search 服务端工具仅非流式) in: body name: body required: true @@ -695,6 +677,54 @@ paths: summary: 返回构建与运行环境信息,「设置 · 关于」页展示 tags: - 设置 + /api/v1/ai-blacklist: + get: + responses: + "200": + description: OK + schema: + additionalProperties: true + type: object + security: + - BearerAuth: [] + summary: 模型黑名单列表 + tags: + - AI 管理 + post: + parameters: + - description: '请求体:{name: 模型名}' + in: body + name: body + required: true + schema: + type: object + responses: + "201": + description: Created + schema: + additionalProperties: true + type: object + security: + - BearerAuth: [] + summary: 添加模型黑名单 + tags: + - AI 管理 + /api/v1/ai-blacklist/{id}: + delete: + parameters: + - description: 黑名单条目 ID + in: path + name: id + required: true + type: integer + responses: + "204": + description: 无内容 + security: + - BearerAuth: [] + summary: 移除模型黑名单(重新同步后模型恢复入池) + tags: + - AI 管理 /api/v1/ai-channels: get: responses: diff --git a/internal/aiwire/anthropic.go b/internal/aiwire/anthropic.go index 57e9823..16913f4 100644 --- a/internal/aiwire/anthropic.go +++ b/internal/aiwire/anthropic.go @@ -20,6 +20,13 @@ type MessagesRequest struct { ToolChoice json.RawMessage `json:"tool_choice,omitempty"` Metadata json.RawMessage `json:"metadata,omitempty"` Thinking json.RawMessage `json:"thinking,omitempty"` + // OutputConfig 是 Anthropic 官方输出控制(effort 等);网关映射 effort 到 OCI reasoningEffort。 + OutputConfig *AnthOutputConfig `json:"output_config,omitempty"` +} + +// AnthOutputConfig 是 Anthropic Messages 顶层 output_config 对象。 +type AnthOutputConfig struct { + Effort string `json:"effort,omitempty"` } // SystemText 把顶层 system(string 或块数组)拍平为文本。 diff --git a/internal/aiwire/openai.go b/internal/aiwire/openai.go index c986b7f..c3248a4 100644 --- a/internal/aiwire/openai.go +++ b/internal/aiwire/openai.go @@ -1,35 +1,11 @@ -// Package aiwire 定义 AI 网关的线格式类型:OpenAI Chat Completions(兼作网关内部 -// 中间表示 IR,因 OCI GENERIC 格式即其镜像)与 Anthropic Messages。零内部依赖, -// oci 层与 service 层共同引用。 +// Package aiwire 定义 AI 网关的线格式类型:OpenAI Responses / Anthropic Messages / +// Embeddings 与通用记账、错误、模型列表结构。上游统一为 OCI OpenAI 兼容面。 package aiwire import ( "encoding/json" - "strings" ) -// ChatRequest 是 OpenAI /v1/chat/completions 请求体;TopK 为 OCI/Anthropic 扩展字段。 -type ChatRequest struct { - Model string `json:"model"` - Messages []ChatMessage `json:"messages"` - MaxTokens *int `json:"max_tokens,omitempty"` - MaxCompletionTokens *int `json:"max_completion_tokens,omitempty"` - Temperature *float64 `json:"temperature,omitempty"` - TopP *float64 `json:"top_p,omitempty"` - TopK *int `json:"top_k,omitempty"` - FrequencyPenalty *float64 `json:"frequency_penalty,omitempty"` - PresencePenalty *float64 `json:"presence_penalty,omitempty"` - Stop StringList `json:"stop,omitempty"` - Seed *int `json:"seed,omitempty"` - N *int `json:"n,omitempty"` - Stream bool `json:"stream,omitempty"` - StreamOptions *StreamOptions `json:"stream_options,omitempty"` - Tools []Tool `json:"tools,omitempty"` - ToolChoice json.RawMessage `json:"tool_choice,omitempty"` - ResponseFormat *ResponseFormat `json:"response_format,omitempty"` - ReasoningEffort string `json:"reasoning_effort,omitempty"` -} - // StringList 兼容 OpenAI stop 的 string 与 []string 两种线格式。 type StringList []string @@ -50,151 +26,6 @@ func (s *StringList) UnmarshalJSON(b []byte) error { return nil } -// StreamOptions 控制流式末尾是否附带 usage 事件。 -type StreamOptions struct { - IncludeUsage bool `json:"include_usage"` -} - -// ChatMessage 是请求与非流式响应共用的消息结构。 -type ChatMessage struct { - Role string `json:"role"` - Content Content `json:"content"` - Name string `json:"name,omitempty"` - ToolCalls []ToolCall `json:"tool_calls,omitempty"` - ToolCallID string `json:"tool_call_id,omitempty"` -} - -// Content 兼容 string 与多模态块数组两种线格式;一期仅透传文本。 -type Content struct { - // Text 在原始值为字符串时填充 - Text string - // Parts 在原始值为数组时填充 - Parts []ContentPart - // isArray 记录原始形态,序列化时保形 - isArray bool -} - -// ContentPart 是块数组中的一段;text 与 image_url 被网关处理,其余类型拒绝。 -type ContentPart struct { - Type string `json:"type"` - Text string `json:"text,omitempty"` - ImageURL *ImageURL `json:"image_url,omitempty"` -} - -// ImageURL 是 image_url 块负载;url 为 data URI(base64)或公网地址,与 OCI ImageUrl 同构。 -type ImageURL struct { - URL string `json:"url"` - Detail string `json:"detail,omitempty"` -} - -func (c *Content) UnmarshalJSON(b []byte) error { - trimmed := strings.TrimSpace(string(b)) - if trimmed == "null" { - return nil - } - if len(trimmed) > 0 && trimmed[0] == '[' { - c.isArray = true - return json.Unmarshal(b, &c.Parts) - } - return json.Unmarshal(b, &c.Text) -} - -func (c Content) MarshalJSON() ([]byte, error) { - if c.isArray { - return json.Marshal(c.Parts) - } - return json.Marshal(c.Text) -} - -// NewTextContent 构造字符串形态的内容。 -func NewTextContent(text string) Content { - return Content{Text: text} -} - -// NewPartsContent 构造块数组形态的内容(多模态消息用)。 -func NewPartsContent(parts []ContentPart) Content { - return Content{Parts: parts, isArray: true} -} - -// JoinText 返回全部文本内容(数组形态拼接 text 块)。 -func (c Content) JoinText() string { - if !c.isArray { - return c.Text - } - var sb strings.Builder - for _, p := range c.Parts { - if p.Type == "" || p.Type == "text" { - sb.WriteString(p.Text) - } - } - return sb.String() -} - -// HasUnsupported 报告是否含网关无法承接的内容块(text / image_url 之外的类型)。 -func (c Content) HasUnsupported() bool { - for _, p := range c.Parts { - switch p.Type { - case "", "text": - case "image_url": - if p.ImageURL == nil || p.ImageURL.URL == "" { - return true - } - default: - return true - } - } - return false -} - -// Tool 是 function 工具定义。 -type Tool struct { - Type string `json:"type"` - Function FunctionDef `json:"function"` -} - -// FunctionDef 描述工具的名称与 JSON Schema 参数。 -type FunctionDef struct { - Name string `json:"name"` - Description string `json:"description,omitempty"` - Parameters json.RawMessage `json:"parameters,omitempty"` -} - -// ToolCall 是助手消息中的工具调用。 -type ToolCall struct { - ID string `json:"id"` - Type string `json:"type"` - Function FunctionCall `json:"function"` -} - -// FunctionCall 携带调用名与 JSON 编码的实参。 -type FunctionCall struct { - Name string `json:"name"` - Arguments string `json:"arguments"` -} - -// ResponseFormat 对应 response_format(json_object / json_schema)。 -type ResponseFormat struct { - Type string `json:"type"` - JSONSchema json.RawMessage `json:"json_schema,omitempty"` -} - -// ChatResponse 是非流式响应体。 -type ChatResponse struct { - ID string `json:"id"` - Object string `json:"object"` - Created int64 `json:"created"` - Model string `json:"model"` - Choices []Choice `json:"choices"` - Usage *Usage `json:"usage,omitempty"` -} - -// Choice 是一条候选回复。 -type Choice struct { - Index int `json:"index"` - Message ChatMessage `json:"message"` - FinishReason string `json:"finish_reason"` -} - // Usage 是 token 用量。 type Usage struct { PromptTokens int `json:"prompt_tokens"` @@ -217,44 +48,6 @@ func (u *Usage) CachedTokens() int { return u.PromptTokensDetails.CachedTokens } -// ChatChunk 是流式增量事件体(chat.completion.chunk)。 -type ChatChunk struct { - ID string `json:"id"` - Object string `json:"object"` - Created int64 `json:"created"` - Model string `json:"model"` - Choices []ChunkChoice `json:"choices"` - Usage *Usage `json:"usage,omitempty"` -} - -// ChunkChoice 是流式事件中的增量候选。 -type ChunkChoice struct { - Index int `json:"index"` - Delta Delta `json:"delta"` - FinishReason *string `json:"finish_reason"` -} - -// Delta 是流式增量内容;Content 为纯文本片段。 -type Delta struct { - Role string `json:"role,omitempty"` - Content string `json:"content,omitempty"` - ToolCalls []ToolCallDelta `json:"tool_calls,omitempty"` -} - -// ToolCallDelta 是工具调用的流式片段;Index 标识聚合目标。 -type ToolCallDelta struct { - Index int `json:"index"` - ID string `json:"id,omitempty"` - Type string `json:"type,omitempty"` - Function FunctionCallDelta `json:"function"` -} - -// FunctionCallDelta 是函数名 / 实参的增量片段。 -type FunctionCallDelta struct { - Name string `json:"name,omitempty"` - Arguments string `json:"arguments,omitempty"` -} - // Model 是 /v1/models 列表项。 type Model struct { ID string `json:"id"` diff --git a/internal/aiwire/responses.go b/internal/aiwire/responses.go index 8d92052..f735a24 100644 --- a/internal/aiwire/responses.go +++ b/internal/aiwire/responses.go @@ -156,39 +156,6 @@ type RespReasoning struct { Effort string `json:"effort,omitempty"` } -// Response 是 /responses 响应对象。 -type Response struct { - ID string `json:"id"` - Object string `json:"object"` - CreatedAt int64 `json:"created_at"` - Status string `json:"status"` - Model string `json:"model"` - Output []RespOutItem `json:"output"` - Usage *RespUsage `json:"usage,omitempty"` - Error *RespError `json:"error"` - IncompleteDetails *RespIncomplete `json:"incomplete_details"` - Store bool `json:"store"` -} - -// RespOutItem 是 output 数组项:message 或 function_call。 -type RespOutItem struct { - Type string `json:"type"` - ID string `json:"id"` - Status string `json:"status,omitempty"` - Role string `json:"role,omitempty"` - Content []RespOutPart `json:"content,omitempty"` - CallID string `json:"call_id,omitempty"` - Name string `json:"name,omitempty"` - Arguments string `json:"arguments,omitempty"` -} - -// RespOutPart 是 message item 的内容部件(output_text)。 -type RespOutPart struct { - Type string `json:"type"` - Text string `json:"text"` - Annotations []any `json:"annotations"` -} - // RespUsage 是 Responses 口径的用量(input/output 命名)。 type RespUsage struct { InputTokens int `json:"input_tokens"` diff --git a/internal/api/aiadmin.go b/internal/api/aiadmin.go index 3cd6154..19c8e05 100644 --- a/internal/api/aiadmin.go +++ b/internal/api/aiadmin.go @@ -40,15 +40,16 @@ func (h *aiAdminHandler) listKeys(c *gin.Context) { // @Router /api/v1/ai-keys [post] func (h *aiAdminHandler) createKey(c *gin.Context) { var req struct { - Name string `json:"name" binding:"required"` - Value string `json:"value"` - Group string `json:"group"` + Name string `json:"name" binding:"required"` + Value string `json:"value"` + Group string `json:"group"` + Models []string `json:"models"` } if err := c.ShouldBindJSON(&req); err != nil { c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()}) return } - plaintext, key, err := h.gw.CreateKey(c.Request.Context(), req.Name, req.Value, req.Group) + plaintext, key, err := h.gw.CreateKey(c.Request.Context(), req.Name, req.Value, req.Group, req.Models) if err != nil { c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()}) return @@ -69,15 +70,16 @@ func (h *aiAdminHandler) updateKey(c *gin.Context) { return } var req struct { - Name string `json:"name"` - Enabled *bool `json:"enabled"` - Group *string `json:"group"` + Name string `json:"name"` + Enabled *bool `json:"enabled"` + Group *string `json:"group"` + Models *[]string `json:"models"` } if err := c.ShouldBindJSON(&req); err != nil { c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()}) return } - if err := h.gw.UpdateKey(c.Request.Context(), id, req.Name, req.Enabled, req.Group); err != nil { + if err := h.gw.UpdateKey(c.Request.Context(), id, req.Name, req.Enabled, req.Group, req.Models); err != nil { respondError(c, err) return } @@ -285,6 +287,64 @@ func (h *aiAdminHandler) gatewayModels(c *gin.Context) { c.JSON(http.StatusOK, gin.H{"items": list.Data}) } +// ---- 模型黑名单 ---- + +// @Summary 模型黑名单列表 +// @Tags AI 管理 +// @Success 200 {object} map[string]any +// @Security BearerAuth +// @Router /api/v1/ai-blacklist [get] +func (h *aiAdminHandler) listBlacklist(c *gin.Context) { + items, err := h.gw.Blacklist(c.Request.Context()) + if err != nil { + respondError(c, err) + return + } + c.JSON(http.StatusOK, gin.H{"items": items}) +} + +// addBlacklist 拉黑模型:删除全部渠道缓存中的同名条目,后续同步 / 探测均过滤。 +// +// @Summary 添加模型黑名单 +// @Tags AI 管理 +// @Param body body object true "请求体:{name: 模型名}" +// @Success 201 {object} map[string]any +// @Security BearerAuth +// @Router /api/v1/ai-blacklist [post] +func (h *aiAdminHandler) addBlacklist(c *gin.Context) { + var req struct { + Name string `json:"name" binding:"required"` + } + if err := c.ShouldBindJSON(&req); err != nil { + c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()}) + return + } + item, err := h.gw.AddBlacklist(c.Request.Context(), req.Name) + if err != nil { + c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()}) + return + } + c.JSON(http.StatusCreated, gin.H{"item": item}) +} + +// @Summary 移除模型黑名单(重新同步后模型恢复入池) +// @Tags AI 管理 +// @Param id path int true "黑名单条目 ID" +// @Success 204 "无内容" +// @Security BearerAuth +// @Router /api/v1/ai-blacklist/{id} [delete] +func (h *aiAdminHandler) removeBlacklist(c *gin.Context) { + id, ok := aiPathID(c) + if !ok { + return + } + if err := h.gw.RemoveBlacklist(c.Request.Context(), id); err != nil { + c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()}) + return + } + c.Status(http.StatusNoContent) +} + // @Summary AI 调用日志列表 // @Tags AI 管理 // @Success 200 {object} map[string]any diff --git a/internal/api/aigateway.go b/internal/api/aigateway.go index 2ff6f8d..b6a3f07 100644 --- a/internal/api/aigateway.go +++ b/internal/api/aigateway.go @@ -1,13 +1,15 @@ package api import ( + "bufio" + "bytes" "crypto/rand" "encoding/hex" "encoding/json" "errors" - "fmt" "io" "net/http" + "slices" "strings" "time" @@ -86,6 +88,25 @@ func keyGroup(c *gin.Context) string { return "" } +// keyModels 取当前请求密钥的模型白名单(空 = 不限模型)。 +func keyModels(c *gin.Context) []string { + if key, ok := c.Get(aiKeyCtx); ok { + return key.(*model.AiKey).Models + } + return nil +} + +// checkKeyModel 校验请求模型是否在密钥白名单内;拒绝时按端点协议返回 +// 404 model_not_found(与未知模型同口径,不泄露密钥配置细节)并返回 false。 +func checkKeyModel(c *gin.Context, modelName string) bool { + allowed := keyModels(c) + if len(allowed) == 0 || slices.Contains(allowed, modelName) { + return true + } + aiError(c, http.StatusNotFound, "model_not_found", "模型不存在或无权访问: "+modelName) + return false +} + // logEntry 组装调用日志骨架;usage / status 由调用方补齐。 func (h *aiGatewayHandler) logEntry(c *gin.Context, endpoint, modelName string, stream bool, meta service.ChatMeta, start time.Time) model.AiCallLog { entry := model.AiCallLog{ @@ -102,21 +123,6 @@ func (h *aiGatewayHandler) logEntry(c *gin.Context, endpoint, modelName string, } // validateIR 做协议无关的入参检查(模型名、消息、不支持的内容块)。 -func validateIR(ir aiwire.ChatRequest) error { - if strings.TrimSpace(ir.Model) == "" { - return fmt.Errorf("model 不能为空") - } - if len(ir.Messages) == 0 { - return fmt.Errorf("messages 不能为空") - } - for _, m := range ir.Messages { - if m.Content.HasUnsupported() { - return service.ErrAiUnsupportedBlock - } - } - return nil -} - // maybeLogContent 在调用密钥显式开启内容日志且未过期时写正文(红线例外); // respBody 为 nil 时只记请求(流式响应与向量结果不记录);callLogID 关联同次调用日志。 func (h *aiGatewayHandler) maybeLogContent(c *gin.Context, callLogID uint, endpoint, modelName string, stream bool, reqBody, respBody any) { @@ -147,47 +153,6 @@ func (h *aiGatewayHandler) logFailure(c *gin.Context, entry model.AiCallLog, req h.maybeLogContent(c, callID, entry.Endpoint, entry.Model, entry.Stream, reqBody, nil) } -// chatCompletions 是 OpenAI /ai/v1/chat/completions 端点。 -// -// @Summary OpenAI 兼容对话补全 -// @Tags AI 网关 -// @Param body body object true "OpenAI chat/completions 请求体(支持 stream)" -// @Success 200 {object} map[string]any "OpenAI 兼容响应(流式为 SSE)" -// @Router /ai/v1/chat/completions [post] -func (h *aiGatewayHandler) chatCompletions(c *gin.Context) { - var ir aiwire.ChatRequest - if err := c.ShouldBindJSON(&ir); err != nil { - aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error()) - return - } - if err := validateIR(ir); err != nil { - aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error()) - return - } - if ir.Stream { - h.streamOpenAI(c, ir) - return - } - start := time.Now() - resp, meta, err := h.gw.Chat(c.Request.Context(), ir, keyGroup(c)) - entry := h.logEntry(c, "openai", ir.Model, false, meta, start) - if err != nil { - upstreamError(c, err) - entry.ErrMsg = err.Error() - h.logFailure(c, entry, ir) - return - } - resp.ID = aiRandID("chatcmpl-") - if resp.Created == 0 { - resp.Created = time.Now().Unix() - } - entry.Status = http.StatusOK - fillUsage(&entry, resp.Usage) - callID := h.gw.LogCall(entry) - h.maybeLogContent(c, callID, "openai", ir.Model, false, ir, resp) - c.JSON(http.StatusOK, resp) -} - func fillUsage(entry *model.AiCallLog, u *aiwire.Usage) { if u == nil { return @@ -217,6 +182,9 @@ func (h *aiGatewayHandler) embeddings(c *gin.Context) { aiError(c, http.StatusBadRequest, "invalid_request_error", "仅支持 encoding_format=float") return } + if !checkKeyModel(c, req.Model) { + return + } start := time.Now() resp, meta, err := h.gw.Embeddings(c.Request.Context(), req, keyGroup(c)) entry := h.logEntry(c, "embeddings", req.Model, false, meta, start) @@ -235,44 +203,6 @@ func (h *aiGatewayHandler) embeddings(c *gin.Context) { c.JSON(http.StatusOK, resp) } -// streamOpenAI 以 OpenAI SSE 直通流式响应,末尾发 [DONE]。 -func (h *aiGatewayHandler) streamOpenAI(c *gin.Context, ir aiwire.ChatRequest) { - start := time.Now() - stream, meta, err := h.gw.OpenStream(c.Request.Context(), ir, keyGroup(c)) - entry := h.logEntry(c, "openai", ir.Model, true, meta, start) - if err != nil { - upstreamError(c, err) - entry.ErrMsg = err.Error() - h.logFailure(c, entry, ir) - return - } - defer stream.Close() - sseHeaders(c) - id, created := aiRandID("chatcmpl-"), time.Now().Unix() - var usage *aiwire.Usage - for { - chunk, err := stream.Next() - if err != nil { - if !errors.Is(err, io.EOF) { - entry.ErrMsg = err.Error() - } - break - } - chunk.ID, chunk.Created = id, created - if chunk.Usage != nil { - usage = chunk.Usage - } - writeSSEData(c, chunk) - } - c.Writer.WriteString("data: [DONE]\n\n") - c.Writer.Flush() - entry.Status = http.StatusOK - entry.LatencyMs = time.Since(start).Milliseconds() - fillUsage(&entry, usage) - callID := h.gw.LogCall(entry) - h.maybeLogContent(c, callID, "openai", ir.Model, true, ir, nil) -} - func sseHeaders(c *gin.Context) { c.Header("Content-Type", "text/event-stream") c.Header("Cache-Control", "no-cache") @@ -280,22 +210,11 @@ func sseHeaders(c *gin.Context) { c.Writer.Flush() } -func writeSSEData(c *gin.Context, v any) { - b, err := json.Marshal(v) - if err != nil { - return - } - c.Writer.WriteString("data: ") - c.Writer.Write(b) - c.Writer.WriteString("\n\n") - c.Writer.Flush() -} - // messages 是 Anthropic /ai/v1/messages 端点。 // // @Summary Anthropic Messages 兼容端点 // @Tags AI 网关 -// @Param body body object true "Anthropic messages 请求体(支持 stream)" +// @Param body body object true "Anthropic messages 请求体(支持 stream;经 OCI OpenAI 兼容面直通)" // @Success 200 {object} map[string]any "Anthropic 兼容响应(流式为 SSE)" // @Router /ai/v1/messages [post] func (h *aiGatewayHandler) messages(c *gin.Context) { @@ -308,69 +227,92 @@ func (h *aiGatewayHandler) messages(c *gin.Context) { aiError(c, http.StatusBadRequest, "invalid_request_error", "max_tokens 必填且需大于 0") return } - ir, err := service.AnthropicToIR(req) + if len(req.Messages) == 0 { + aiError(c, http.StatusBadRequest, "invalid_request_error", "messages 不能为空") + return + } + if !checkKeyModel(c, req.Model) { + return + } + body, err := service.AnthropicToResponsesBody(req) if err != nil { aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error()) return } - if err := validateIR(ir); err != nil { - aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error()) - return - } if req.Stream { - h.streamAnthropic(c, ir) + h.streamAnthropic(c, body, req) return } start := time.Now() - resp, meta, err := h.gw.Chat(c.Request.Context(), ir, keyGroup(c)) - entry := h.logEntry(c, "anthropic", ir.Model, false, meta, start) + payload, meta, err := h.gw.RespPassthrough(c.Request.Context(), body, req.Model, keyGroup(c)) + entry := h.logEntry(c, "anthropic", req.Model, false, meta, start) if err != nil { upstreamError(c, err) entry.ErrMsg = err.Error() h.logFailure(c, entry, req) return } + out, err := service.ResponsesToAnthropic(payload, aiRandID("msg_")) + if err != nil { + aiError(c, http.StatusBadGateway, "api_error", err.Error()) + entry.ErrMsg = err.Error() + h.logFailure(c, entry, req) + return + } entry.Status = http.StatusOK - fillUsage(&entry, resp.Usage) + fillUsage(&entry, service.RespPassthroughUsage(payload)) callID := h.gw.LogCall(entry) - out := service.IRRespToAnthropic(resp, aiRandID("msg_")) - h.maybeLogContent(c, callID, "anthropic", ir.Model, false, req, out) + h.maybeLogContent(c, callID, "anthropic", req.Model, false, req, out) c.JSON(http.StatusOK, out) } -// streamAnthropic 经状态机把 IR chunk 流聚合为 Anthropic SSE 事件流。 -func (h *aiGatewayHandler) streamAnthropic(c *gin.Context, ir aiwire.ChatRequest) { +// streamAnthropic 流式直通并桥接:上游 Responses SSE 事件经 AnthRespBridge +// 转为 Anthropic 事件序列逐块写出;usage 从 completed 事件记账。 +func (h *aiGatewayHandler) streamAnthropic(c *gin.Context, body []byte, req aiwire.MessagesRequest) { start := time.Now() - stream, meta, err := h.gw.OpenStream(c.Request.Context(), ir, keyGroup(c)) - entry := h.logEntry(c, "anthropic", ir.Model, true, meta, start) + upstream, meta, err := h.gw.RespPassthroughStream(c.Request.Context(), body, req.Model, keyGroup(c)) + entry := h.logEntry(c, "anthropic", req.Model, true, meta, start) if err != nil { upstreamError(c, err) entry.ErrMsg = err.Error() - h.logFailure(c, entry, ir) + h.logFailure(c, entry, req) return } - defer stream.Close() + defer upstream.Close() sseHeaders(c) - st := service.NewAnthStream(aiRandID("msg_"), ir.Model) - for { - chunk, err := stream.Next() - if err != nil { - if !errors.Is(err, io.EOF) { - entry.ErrMsg = err.Error() - } - break - } - writeAnthEvents(c, st.Feed(chunk)) + bridge := service.NewAnthRespBridge(aiRandID("msg_"), req.Model) + if err := forwardAnthSSE(c, upstream, bridge); err != nil { + entry.ErrMsg = err.Error() } - writeAnthEvents(c, st.Finish()) + writeAnthEvents(c, bridge.Finish()) entry.Status = http.StatusOK entry.LatencyMs = time.Since(start).Milliseconds() - usage := st.Usage() + usage := bridge.Usage() entry.PromptTokens, entry.CompletionTokens = usage.InputTokens, usage.OutputTokens entry.TotalTokens = usage.InputTokens + usage.OutputTokens entry.CachedTokens = usage.CacheReadInputTokens callID := h.gw.LogCall(entry) - h.maybeLogContent(c, callID, "anthropic", ir.Model, true, ir, nil) + h.maybeLogContent(c, callID, "anthropic", req.Model, true, req, nil) +} + +// forwardAnthSSE 逐行读上游 SSE,把 data 行喂给桥并即时写出转换后的事件。 +func forwardAnthSSE(c *gin.Context, upstream io.Reader, bridge *service.AnthRespBridge) error { + reader := bufio.NewReader(upstream) + for { + line, err := reader.ReadBytes('\n') + if len(line) > 0 { + trimmed := bytes.TrimSpace(line) + if data, ok := bytes.CutPrefix(trimmed, []byte("data: ")); ok { + writeAnthEvents(c, bridge.Feed(data)) + } + } + if err != nil { + if errors.Is(err, io.EOF) { + return nil + } + return err + } + } } func writeAnthEvents(c *gin.Context, events []service.AnthEvent) { @@ -398,6 +340,15 @@ func (h *aiGatewayHandler) listModels(c *gin.Context) { aiError(c, http.StatusInternalServerError, "api_error", "查询模型列表失败") return } + if allowed := keyModels(c); len(allowed) > 0 { + kept := make([]aiwire.Model, 0, len(list.Data)) + for _, m := range list.Data { + if slices.Contains(allowed, m.ID) { + kept = append(kept, m) + } + } + list.Data = kept + } c.JSON(http.StatusOK, list) } @@ -405,31 +356,45 @@ func (h *aiGatewayHandler) listModels(c *gin.Context) { // // @Summary OpenAI Responses 兼容端点 // @Tags AI 网关 -// @Param body body object true "OpenAI responses 请求体(支持 stream)" +// @Param body body object true "OpenAI responses 请求体(支持 stream;web_search/x_search 服务端工具仅非流式)" // @Success 200 {object} map[string]any "OpenAI 兼容响应(流式为 SSE)" // @Router /ai/v1/responses [post] func (h *aiGatewayHandler) responses(c *gin.Context) { - var req aiwire.RespRequest - if err := c.ShouldBindJSON(&req); err != nil { - aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error()) - return - } - ir, err := service.ResponsesToIR(req) + raw, err := c.GetRawData() if err != nil { aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error()) return } - if err := validateIR(ir); err != nil { + var req aiwire.RespRequest + if err := json.Unmarshal(raw, &req); err != nil { + aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error()) + return + } + if !checkKeyModel(c, req.Model) { + return + } + h.responsesPassthrough(c, raw, req) +} + +// responsesPassthrough 直通链路(唯一上游):原始 body 改写后直达 +// OCI /actions/v1/responses,响应原样透传。 +func (h *aiGatewayHandler) responsesPassthrough(c *gin.Context, raw []byte, req aiwire.RespRequest) { + if err := service.RespPassthroughValidate(req); err != nil { + aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error()) + return + } + body, err := service.RespPassthroughBody(raw) + if err != nil { aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error()) return } if req.Stream { - h.streamResponses(c, ir) + h.responsesPassthroughStream(c, body, req) return } start := time.Now() - resp, meta, err := h.gw.Chat(c.Request.Context(), ir, keyGroup(c)) - entry := h.logEntry(c, "responses", ir.Model, false, meta, start) + payload, meta, err := h.gw.RespPassthrough(c.Request.Context(), body, req.Model, keyGroup(c)) + entry := h.logEntry(c, "responses", req.Model, false, meta, start) if err != nil { upstreamError(c, err) entry.ErrMsg = err.Error() @@ -437,54 +402,61 @@ func (h *aiGatewayHandler) responses(c *gin.Context) { return } entry.Status = http.StatusOK - fillUsage(&entry, resp.Usage) + fillUsage(&entry, service.RespPassthroughUsage(payload)) callID := h.gw.LogCall(entry) - out := service.IRRespToResponses(resp, aiRandID("resp_"), time.Now().Unix()) - h.maybeLogContent(c, callID, "responses", ir.Model, false, req, out) - c.JSON(http.StatusOK, out) + h.maybeLogContent(c, callID, "responses", req.Model, false, req, json.RawMessage(payload)) + c.Data(http.StatusOK, "application/json; charset=utf-8", payload) } -// streamResponses 经状态机把 IR chunk 流聚合为 Responses 语义事件流。 -func (h *aiGatewayHandler) streamResponses(c *gin.Context, ir aiwire.ChatRequest) { +// responsesPassthroughStream 流式直通:SSE 事件原样转发(推理增量等直达客户端), +// 逐行扫描 completed 事件提取 usage 记账。 +func (h *aiGatewayHandler) responsesPassthroughStream(c *gin.Context, body []byte, req aiwire.RespRequest) { start := time.Now() - stream, meta, err := h.gw.OpenStream(c.Request.Context(), ir, keyGroup(c)) - entry := h.logEntry(c, "responses", ir.Model, true, meta, start) + upstream, meta, err := h.gw.RespPassthroughStream(c.Request.Context(), body, req.Model, keyGroup(c)) + entry := h.logEntry(c, "responses", req.Model, true, meta, start) if err != nil { upstreamError(c, err) entry.ErrMsg = err.Error() - h.logFailure(c, entry, ir) + h.logFailure(c, entry, req) return } - defer stream.Close() + defer upstream.Close() sseHeaders(c) - st := service.NewRespStream(aiRandID("resp_"), ir.Model, time.Now().Unix()) - for { - chunk, err := stream.Next() - if err != nil { - if !errors.Is(err, io.EOF) { - entry.ErrMsg = err.Error() - } - break - } - writeRespEvents(c, st.Feed(chunk)) + usage, err := forwardSSE(c, upstream) + if err != nil { + entry.ErrMsg = err.Error() } - writeRespEvents(c, st.Finish()) entry.Status = http.StatusOK entry.LatencyMs = time.Since(start).Milliseconds() - fillUsage(&entry, st.Usage()) + fillUsage(&entry, usage) callID := h.gw.LogCall(entry) - h.maybeLogContent(c, callID, "responses", ir.Model, true, ir, nil) + h.maybeLogContent(c, callID, "responses", req.Model, true, req, nil) } -func writeRespEvents(c *gin.Context, events []service.RespEvent) { - for _, ev := range events { - b, err := json.Marshal(ev.Data) +// forwardSSE 把上游 SSE 逐行转发给客户端,空行(事件边界)即 flush; +// 顺带从 data 行提取 response.completed 的 usage。 +func forwardSSE(c *gin.Context, upstream io.Reader) (*aiwire.Usage, error) { + reader := bufio.NewReader(upstream) + var usage *aiwire.Usage + for { + line, err := reader.ReadBytes('\n') + if len(line) > 0 { + c.Writer.Write(line) + trimmed := bytes.TrimSpace(line) + if len(trimmed) == 0 { + c.Writer.Flush() + } else if data, ok := bytes.CutPrefix(trimmed, []byte("data: ")); ok { + if u := service.RespStreamCompletedUsage(data); u != nil { + usage = u + } + } + } if err != nil { - continue + c.Writer.Flush() + if errors.Is(err, io.EOF) { + return usage, nil + } + return usage, err } - c.Writer.WriteString("event: " + ev.Event + "\ndata: ") - c.Writer.Write(b) - c.Writer.WriteString("\n\n") } - c.Writer.Flush() } diff --git a/internal/api/aigateway_test.go b/internal/api/aigateway_test.go new file mode 100644 index 0000000..aa0dec8 --- /dev/null +++ b/internal/api/aigateway_test.go @@ -0,0 +1,206 @@ +package api + +import ( + "context" + "encoding/json" + "net/http" + "strconv" + "strings" + "testing" + "time" + + "gorm.io/gorm" + + "oci-portal/internal/model" +) + +// seedGatewayModel 直插启用渠道与模型缓存,绕过云同步。 +func seedGatewayModel(t *testing.T, db *gorm.DB, name string) { + t.Helper() + ch := &model.AiChannel{Name: "t-" + name, OciConfigID: 1, Region: "r-" + name, Enabled: true, Priority: 1, Weight: 1} + if err := db.Create(ch).Error; err != nil { + t.Fatalf("seed channel: %v", err) + } + cache := &model.AiModelCache{ChannelID: ch.ID, ModelOcid: "ocid1.." + name, Name: name, Vendor: "v", SyncedAt: time.Now()} + if err := db.Create(cache).Error; err != nil { + t.Fatalf("seed model cache: %v", err) + } +} + +func TestAiGatewayKeyModelRestrict(t *testing.T) { + r, auth, _, db := newTestRouterDB(t) + token, _, err := auth.Login(context.Background(), "admin", "pass123", "127.0.0.1", "") + if err != nil { + t.Fatalf("login: %v", err) + } + + // 受限密钥:白名单含脏输入,入库应规范化为 2 项(cohere + ghost) + w := doRequest(t, r, http.MethodPost, "/api/v1/ai-keys", token, + `{"name":"limited","value":"limited-key-1234","models":[" cohere.command-a-03-2025 ","cohere.command-a-03-2025","","ghost-model"]}`) + if w.Code != http.StatusCreated { + t.Fatalf("创建受限密钥 status = %d, body %s", w.Code, w.Body.String()) + } + var created struct { + Item model.AiKey `json:"item"` + } + if err := json.Unmarshal(w.Body.Bytes(), &created); err != nil { + t.Fatalf("decode: %v", err) + } + if len(created.Item.Models) != 2 || created.Item.Models[0] != "cohere.command-a-03-2025" { + t.Fatalf("创建后 models 未规范化: %v", created.Item.Models) + } + // 不限密钥对照 + w = doRequest(t, r, http.MethodPost, "/api/v1/ai-keys", token, `{"name":"open","value":"open-key-12345"}`) + if w.Code != http.StatusCreated { + t.Fatalf("创建不限密钥 status = %d, body %s", w.Code, w.Body.String()) + } + + seedGatewayModel(t, db, "cohere.command-a-03-2025") + seedGatewayModel(t, db, "meta.llama-3.3-70b-instruct") + + deny := []string{"model_not_found", "无权访问"} + pass := []string{"未知模型"} // 穿过白名单闸门,由编排层以「未知模型」拒绝(池内无 ghost-model) + tests := []struct { + name string + key string + path string + body string + wantCode int + wantSub []string + }{ + {"responses 流式同样拦截", "limited-key-1234", "/ai/v1/responses", + `{"model":"meta.llama-3.3-70b-instruct","stream":true,"input":"hi"}`, 404, deny}, + {"responses 白名单内穿透", "limited-key-1234", "/ai/v1/responses", + `{"model":"ghost-model","input":"hi"}`, 404, pass}, + {"embeddings 白名单外拦截", "limited-key-1234", "/ai/v1/embeddings", + `{"model":"meta.llama-3.3-70b-instruct","input":["hi"]}`, 404, deny}, + {"messages 白名单外拦截为 Anthropic 体", "limited-key-1234", "/ai/v1/messages", + `{"model":"meta.llama-3.3-70b-instruct","max_tokens":16,"messages":[{"role":"user","content":"hi"}]}`, 404, + []string{`"type":"error"`, "model_not_found", "无权访问"}}, + {"responses 白名单外拦截", "limited-key-1234", "/ai/v1/responses", + `{"model":"meta.llama-3.3-70b-instruct","input":"hi"}`, 404, deny}, + {"不限密钥穿透闸门", "open-key-12345", "/ai/v1/responses", + `{"model":"ghost-model","input":"hi"}`, 404, pass}, + {"chat completions 端点已删除", "open-key-12345", "/ai/v1/chat/completions", + `{"model":"ghost-model","messages":[{"role":"user","content":"hi"}]}`, 404, []string{"not found"}}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + w := doRequest(t, r, http.MethodPost, tt.path, tt.key, tt.body) + if w.Code != tt.wantCode { + t.Fatalf("status = %d, want %d, body %s", w.Code, tt.wantCode, w.Body.String()) + } + for _, sub := range tt.wantSub { + if !strings.Contains(w.Body.String(), sub) { + t.Errorf("body 缺少 %q: %s", sub, w.Body.String()) + } + } + }) + } +} + +func TestAiGatewayModelsFilteredByKey(t *testing.T) { + r, auth, _, db := newTestRouterDB(t) + token, _, err := auth.Login(context.Background(), "admin", "pass123", "127.0.0.1", "") + if err != nil { + t.Fatalf("login: %v", err) + } + seedGatewayModel(t, db, "cohere.command-a-03-2025") + seedGatewayModel(t, db, "meta.llama-3.3-70b-instruct") + w := doRequest(t, r, http.MethodPost, "/api/v1/ai-keys", token, + `{"name":"limited","value":"limited-key-1234","models":["cohere.command-a-03-2025"]}`) + if w.Code != http.StatusCreated { + t.Fatalf("创建密钥 status = %d", w.Code) + } + w = doRequest(t, r, http.MethodPost, "/api/v1/ai-keys", token, `{"name":"open","value":"open-key-12345"}`) + if w.Code != http.StatusCreated { + t.Fatalf("创建密钥 status = %d", w.Code) + } + + tests := []struct { + name string + key string + want []string + notWant []string + }{ + {"受限密钥仅见交集", "limited-key-1234", []string{"cohere.command-a-03-2025"}, []string{"meta.llama-3.3-70b-instruct"}}, + {"不限密钥全量可见", "open-key-12345", []string{"cohere.command-a-03-2025", "meta.llama-3.3-70b-instruct"}, nil}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + w := doRequest(t, r, http.MethodGet, "/ai/v1/models", tt.key, "") + if w.Code != http.StatusOK { + t.Fatalf("status = %d, body %s", w.Code, w.Body.String()) + } + for _, sub := range tt.want { + if !strings.Contains(w.Body.String(), sub) { + t.Errorf("应包含 %q: %s", sub, w.Body.String()) + } + } + for _, sub := range tt.notWant { + if strings.Contains(w.Body.String(), sub) { + t.Errorf("不应包含 %q: %s", sub, w.Body.String()) + } + } + }) + } +} + +func TestAiKeyModelsUpdateRoundTrip(t *testing.T) { + r, auth, _, _ := newTestRouterDB(t) + token, _, err := auth.Login(context.Background(), "admin", "pass123", "127.0.0.1", "") + if err != nil { + t.Fatalf("login: %v", err) + } + w := doRequest(t, r, http.MethodPost, "/api/v1/ai-keys", token, `{"name":"k","value":"round-key-1234"}`) + if w.Code != http.StatusCreated { + t.Fatalf("创建 status = %d", w.Code) + } + var created struct { + Item model.AiKey `json:"item"` + } + _ = json.Unmarshal(w.Body.Bytes(), &created) + id := created.Item.ID + + listModels := func() []string { + w := doRequest(t, r, http.MethodGet, "/api/v1/ai-keys", token, "") + var resp struct { + Items []model.AiKey `json:"items"` + } + _ = json.Unmarshal(w.Body.Bytes(), &resp) + for _, k := range resp.Items { + if k.ID == id { + return k.Models + } + } + t.Fatalf("密钥 %d 不在列表中", id) + return nil + } + + tests := []struct { + name string + body string + want []string + }{ + {"设置白名单", `{"models":[" a-model ","a-model","b-model"]}`, []string{"a-model", "b-model"}}, + {"不传 models 保持不变", `{"name":"k2"}`, []string{"a-model", "b-model"}}, + {"空数组清空恢复不限", `{"models":[]}`, nil}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + w := doRequest(t, r, http.MethodPut, "/api/v1/ai-keys/"+strconv.Itoa(int(id)), token, tt.body) + if w.Code != http.StatusNoContent { + t.Fatalf("update status = %d, body %s", w.Code, w.Body.String()) + } + got := listModels() + if len(got) != len(tt.want) { + t.Fatalf("models = %v, want %v", got, tt.want) + } + for i := range tt.want { + if got[i] != tt.want[i] { + t.Fatalf("models = %v, want %v", got, tt.want) + } + } + }) + } +} diff --git a/internal/api/router_test.go b/internal/api/router_test.go index b110225..049e15e 100644 --- a/internal/api/router_test.go +++ b/internal/api/router_test.go @@ -57,6 +57,13 @@ func (nullClient) GetSubscription(context.Context, oci.Credentials, string) (oci } func newTestRouter(t *testing.T) (*gin.Engine, *service.AuthService, *service.SystemLogService) { + t.Helper() + r, auth, systemLogs, _ := newTestRouterDB(t) + return r, auth, systemLogs +} + +// newTestRouterDB 额外回传 db 句柄,供需要直插数据(如网关模型缓存)的测试使用。 +func newTestRouterDB(t *testing.T) (*gin.Engine, *service.AuthService, *service.SystemLogService, *gorm.DB) { t.Helper() gin.SetMode(gin.TestMode) db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{ @@ -90,7 +97,7 @@ func newTestRouter(t *testing.T) (*gin.Engine, *service.AuthService, *service.Sy logEvents := service.NewLogEventService(db) oauth := service.NewOAuthService(db, settings, auth) r := NewRouter(auth, oauth, ociConfigs, tasks, service.NewConsoleService(ociConfigs), settings, notifier, systemLogs, logEvents, service.NewProxyService(db, cipher), service.NewAiGatewayService(db, ociConfigs, nullClient{})) - return r, auth, systemLogs + return r, auth, systemLogs, db } func doRequest(t *testing.T, r *gin.Engine, method, path, token, body string) *httptest.ResponseRecorder { diff --git a/internal/api/routes_ai.go b/internal/api/routes_ai.go index 8738129..2643858 100644 --- a/internal/api/routes_ai.go +++ b/internal/api/routes_ai.go @@ -11,7 +11,6 @@ import ( func registerAiGateway(r *gin.Engine, aiGateway *service.AiGatewayService) { aih := &aiGatewayHandler{gw: aiGateway} ai := r.Group("/ai/v1", bodyLimit(10<<20), aih.auth) - ai.POST("/chat/completions", aih.chatCompletions) ai.POST("/responses", aih.responses) ai.POST("/messages", aih.messages) ai.POST("/embeddings", aih.embeddings) @@ -33,6 +32,9 @@ func registerAiAdmin(secured *gin.RouterGroup, aiGateway *service.AiGatewayServi secured.POST("/ai-channels/:id/probe", aiadmin.probeChannel) secured.POST("/ai-channels/:id/sync-models", aiadmin.syncChannelModels) secured.GET("/ai-models", aiadmin.gatewayModels) + secured.GET("/ai-blacklist", aiadmin.listBlacklist) + secured.POST("/ai-blacklist", aiadmin.addBlacklist) + secured.DELETE("/ai-blacklist/:id", aiadmin.removeBlacklist) secured.GET("/ai-logs", aiadmin.listLogs) secured.GET("/ai-content-logs", aiadmin.listContentLogs) } diff --git a/internal/database/database.go b/internal/database/database.go index 9b351ee..1d81741 100644 --- a/internal/database/database.go +++ b/internal/database/database.go @@ -68,7 +68,7 @@ func autoMigrate(db *gorm.DB) error { &model.CheckSnapshot{}, &model.CostSnapshot{}, &model.RegionCache{}, &model.CompartmentCache{}, &model.Setting{}, &model.SystemLog{}, &model.LogEvent{}, &model.AlertRule{}, &model.AlertRuleHit{}, &model.Proxy{}, - &model.AiKey{}, &model.AiChannel{}, &model.AiModelCache{}, &model.AiCallLog{}, + &model.AiKey{}, &model.AiChannel{}, &model.AiModelCache{}, &model.AiModelBlacklist{}, &model.AiCallLog{}, &model.AiContentLog{}, ) } diff --git a/internal/model/models.go b/internal/model/models.go index 11f038e..6dabf68 100644 --- a/internal/model/models.go +++ b/internal/model/models.go @@ -261,11 +261,13 @@ type CompartmentCache struct { // AiKey 是 AI 网关的访问密钥;明文不落库,仅存 SHA-256 与尾 4 位。 // Group 非空时该密钥只在同分组渠道内负载均衡(列名避开 SQL 保留字 group)。 type AiKey struct { - ID uint `gorm:"primaryKey" json:"id"` - Name string `gorm:"uniqueIndex;size:64" json:"name"` - KeyHash string `gorm:"uniqueIndex;size:64" json:"-"` - Tail string `gorm:"size:4" json:"tail"` - Group string `gorm:"column:key_group;size:32" json:"group"` + ID uint `gorm:"primaryKey" json:"id"` + Name string `gorm:"uniqueIndex;size:64" json:"name"` + KeyHash string `gorm:"uniqueIndex;size:64" json:"-"` + Tail string `gorm:"size:4" json:"tail"` + Group string `gorm:"column:key_group;size:32" json:"group"` + // Models 是模型白名单(精确匹配模型名);空 = 不限,非空时网关仅放行列表内模型 + Models []string `gorm:"serializer:json;type:text" json:"models"` Enabled bool `json:"enabled"` LastUsedAt *time.Time `json:"lastUsedAt"` // ContentLogUntil 是内容日志开启截止时间;nil = 永关(默认,红线), @@ -309,13 +311,14 @@ type AiModelCache struct { // 弃用后仍可调用,直到 RetiredAt(按需推理退役,同步层已剔除过期项)。 DeprecatedAt *time.Time `json:"deprecatedAt"` RetiredAt *time.Time `json:"retiredAt"` - // Unusable 标记该模型在此渠道不可按需调用(微调基座/区域未开放按需), - // 由探测、真实调用或同步后的后台验证习得:不参与网关列表/路由/探测候选; - // 任何同步都按 OCID 结转标记,已标记模型定期复检、恢复供给自动解除。 - Unusable bool `json:"unusable"` - UnusableReason string `gorm:"size:200" json:"unusableReason,omitempty"` - // CheckedAt 是最近一次可用性验证时间;NULL 表示从未验证(同步后的后台验证以此为队列)。 - CheckedAt *time.Time `json:"checkedAt"` +} + +// AiModelBlacklist 是全局模型黑名单(按模型名):加入即从全部渠道缓存删除该模型, +// 同步与探测过滤不再入库;适用于微调基座等实际不可按需调用的模型,由用户手动维护。 +type AiModelBlacklist struct { + ID uint `gorm:"primaryKey" json:"id"` + Name string `gorm:"size:96;uniqueIndex" json:"name"` + CreatedAt time.Time `json:"createdAt"` } // AiCallLog 是网关调用日志:仅元数据与 token 用量,绝不记录 prompt / 响应正文。 diff --git a/internal/oci/client.go b/internal/oci/client.go index efece79..f559b58 100644 --- a/internal/oci/client.go +++ b/internal/oci/client.go @@ -6,6 +6,7 @@ import ( "context" "errors" "fmt" + "io" "strings" "sync" "time" @@ -99,10 +100,12 @@ type Client interface { AddVnicIpv6(ctx context.Context, cred Credentials, region, vnicID, address string) (string, error) // GenAI 网关:区域模型列表、聊天(非流式/流式)、配额探测、文本向量化。 ListGenAiModels(ctx context.Context, cred Credentials, region string) ([]GenAiModel, error) - GenAiChat(ctx context.Context, cred Credentials, region, modelOcid string, ir aiwire.ChatRequest) (*aiwire.ChatResponse, error) - GenAiChatStream(ctx context.Context, cred Credentials, region, modelOcid string, ir aiwire.ChatRequest) (GenAiStream, error) GenAiProbeChat(ctx context.Context, cred Credentials, region, modelOcid, modelName string) (int, error) GenAiEmbed(ctx context.Context, cred Credentials, region, modelOcid string, inputs []string, dimensions *int) ([][]float32, *aiwire.Usage, error) + // GenAiCompatResponses 直通 OpenAI Responses 请求体到 /actions/v1/responses(xAI 服务端工具通路)。 + GenAiCompatResponses(ctx context.Context, cred Credentials, region string, body []byte) ([]byte, error) + // GenAiCompatResponsesStream 流式直通 /actions/v1/responses,建立成功返回 SSE body。 + GenAiCompatResponsesStream(ctx context.Context, cred Credentials, region string, body []byte) (io.ReadCloser, error) // 控制台连接：创建（VNC/串口连接串）、列出、删除。 CreateConsoleConnection(ctx context.Context, cred Credentials, region, instanceID, sshPublicKey string) (ConsoleConnection, error) ListConsoleConnections(ctx context.Context, cred Credentials, region, instanceID string) ([]ConsoleConnection, error) diff --git a/internal/oci/genai.go b/internal/oci/genai.go index 81d77e2..7bab6a2 100644 --- a/internal/oci/genai.go +++ b/internal/oci/genai.go @@ -2,13 +2,11 @@ package oci import ( "context" + "encoding/json" "fmt" - "io" "net/http" - "strings" "time" - "github.com/oracle/oci-go-sdk/v65/common" "github.com/oracle/oci-go-sdk/v65/generativeai" "github.com/oracle/oci-go-sdk/v65/generativeaiinference" @@ -30,12 +28,6 @@ type GenAiModel struct { Retired *time.Time `json:"retired"` } -// GenAiStream 逐事件读取流式聊天;Next 在流结束时返回 io.EOF。 -type GenAiStream interface { - Next() (aiwire.ChatChunk, error) - Close() error -} - func (c *RealClient) genAiClient(cred Credentials, region string) (generativeai.GenerativeAiClient, error) { gc, err := generativeai.NewGenerativeAiClientWithConfigurationProvider(provider(cred)) if err != nil { @@ -152,121 +144,15 @@ func capStrings(caps []generativeai.ModelCapabilityEnum) []string { return out } -// GenAiChat 实现 Client:非流式聊天;modelOcid 走 on-demand serving, -// cohere.* 模型走 COHERE 请求格式,其余走 GENERIC。 -func (c *RealClient) GenAiChat(ctx context.Context, cred Credentials, region, modelOcid string, ir aiwire.ChatRequest) (*aiwire.ChatResponse, error) { - ic, err := c.genAiInferenceClient(cred, region) - if err != nil { - return nil, err - } - req, err := buildChatRequest(ir, false) - if err != nil { - return nil, err - } - resp, err := ic.Chat(ctx, chatRequest(cred, modelOcid, req)) - if err != nil { - return nil, fmt.Errorf("genai chat: %w", err) - } - return chatResponseToIR(resp.ChatResult.ChatResponse, ir.Model) -} - -// buildChatRequest 按模型 vendor 组装底层请求体。 -func buildChatRequest(ir aiwire.ChatRequest, stream bool) (generativeaiinference.BaseChatRequest, error) { - if isCohereModel(ir.Model) { - return irToCohereSDK(ir, stream) - } - return irToSDK(ir, stream), nil -} - -// chatResponseToIR 按响应实际形态(GENERIC / COHERE)转回 IR。 -func chatResponseToIR(resp generativeaiinference.BaseChatResponse, model string) (*aiwire.ChatResponse, error) { - switch r := resp.(type) { - case generativeaiinference.GenericChatResponse: - return sdkToIR(r, model), nil - case generativeaiinference.CohereChatResponse: - return cohereSDKToIR(r, model), nil - default: - return nil, fmt.Errorf("genai chat: unexpected response format %T", resp) - } -} - -func chatRequest(cred Credentials, modelOcid string, req generativeaiinference.BaseChatRequest) generativeaiinference.ChatRequest { - return generativeaiinference.ChatRequest{ - ChatDetails: generativeaiinference.ChatDetails{ - CompartmentId: &cred.TenancyOCID, - ServingMode: generativeaiinference.OnDemandServingMode{ModelId: &modelOcid}, - ChatRequest: req, - }, - } -} - -// GenAiChatStream 实现 Client:流式聊天,返回逐事件读取器。 -// SDK 对 text/event-stream 跳过 unmarshal,原始流经 RawResponse 交由 SSEReader 消费。 -func (c *RealClient) GenAiChatStream(ctx context.Context, cred Credentials, region, modelOcid string, ir aiwire.ChatRequest) (GenAiStream, error) { - ic, err := c.genAiInferenceClient(cred, region) - if err != nil { - return nil, err - } - req, err := buildChatRequest(ir, true) - if err != nil { - return nil, err - } - resp, err := ic.Chat(ctx, chatRequest(cred, modelOcid, req)) - if err != nil { - return nil, fmt.Errorf("genai chat stream: %w", err) - } - reader, err := common.NewSSEReader(resp.RawResponse) - if err != nil { - return nil, fmt.Errorf("genai chat stream: sse reader: %w", err) - } - return &genAiSSEStream{reader: reader, body: resp.RawResponse.Body, model: ir.Model}, nil -} - -// genAiSSEStream 把 OCI SSE 事件流适配为 IR chunk 流。 -type genAiSSEStream struct { - reader *common.SseReader - body io.ReadCloser - model string -} - -// Next 读取下一条有效事件;空事件与 [DONE] 哨兵跳过,流尽返回 io.EOF。 -func (s *genAiSSEStream) Next() (aiwire.ChatChunk, error) { - for { - data, err := s.reader.ReadNextEvent() - if err != nil { - return aiwire.ChatChunk{}, err - } - text := strings.TrimSpace(string(data)) - if text == "" { - continue - } - if text == "[DONE]" { - return aiwire.ChatChunk{}, io.EOF - } - if chunk, ok := parseGenAiEvent([]byte(text), s.model); ok { - return chunk, nil - } - } -} - -func (s *genAiSSEStream) Close() error { - if s.body != nil { - return s.body.Close() - } - return nil -} - -// GenAiProbeChat 实现 Client:配额探测专用的最小聊天(maxTokens=1),返回 HTTP 状态码; +// GenAiProbeChat 实现 Client:经 OpenAI 兼容面(直通同链路)发一次极小请求探测渠道;配额探测专用的最小聊天(maxTokens=1),返回 HTTP 状态码; // modelName 决定请求格式(cohere.* 走 COHERE)。 func (c *RealClient) GenAiProbeChat(ctx context.Context, cred Credentials, region, modelOcid, modelName string) (int, error) { - one := 1 - ir := aiwire.ChatRequest{ - Model: modelName, - Messages: []aiwire.ChatMessage{{Role: "user", Content: aiwire.NewTextContent("hi")}}, - MaxTokens: &one, + body, err := json.Marshal(map[string]any{"model": modelName, "input": "hi", + "max_output_tokens": 1, "store": false}) + if err != nil { + return 0, err } - _, err := c.GenAiChat(ctx, cred, region, modelOcid, ir) - if err == nil { + if _, err = c.GenAiCompatResponses(ctx, cred, region, body); err == nil { return http.StatusOK, nil } if status, ok := ServiceStatus(err); ok { @@ -275,6 +161,27 @@ func (c *RealClient) GenAiProbeChat(ctx context.Context, cred Credentials, regio return 0, err } +// sdkUsageToIR 把 SDK 用量转为内部记账结构(缓存命中挂 details,仅命中时出现)。 +func sdkUsageToIR(u *generativeaiinference.Usage) *aiwire.Usage { + if u == nil { + return nil + } + out := &aiwire.Usage{} + if u.PromptTokens != nil { + out.PromptTokens = *u.PromptTokens + } + if u.CompletionTokens != nil { + out.CompletionTokens = *u.CompletionTokens + } + if u.TotalTokens != nil { + out.TotalTokens = *u.TotalTokens + } + if u.PromptTokensDetails != nil && u.PromptTokensDetails.CachedTokens != nil && *u.PromptTokensDetails.CachedTokens > 0 { + out.PromptTokensDetails = &aiwire.PromptTokensDetails{CachedTokens: *u.PromptTokensDetails.CachedTokens} + } + return out +} + // GenAiEmbed 实现 Client:文本向量化(on-demand serving);dimensions 透传 OutputDimensions。 func (c *RealClient) GenAiEmbed(ctx context.Context, cred Credentials, region, modelOcid string, inputs []string, dimensions *int) ([][]float32, *aiwire.Usage, error) { ic, err := c.genAiInferenceClient(cred, region) diff --git a/internal/oci/genai_cohere.go b/internal/oci/genai_cohere.go deleted file mode 100644 index a4bc4c5..0000000 --- a/internal/oci/genai_cohere.go +++ /dev/null @@ -1,266 +0,0 @@ -package oci - -import ( - "encoding/json" - "fmt" - "strings" - - "github.com/oracle/oci-go-sdk/v65/generativeaiinference" - - "oci-portal/internal/aiwire" -) - -// isCohereModel 依据模型名前缀判定 COHERE 系(走专属请求格式)。 -func isCohereModel(name string) bool { - return strings.HasPrefix(strings.ToLower(name), "cohere.") -} - -// irToCohereSDK 把 IR 转为 COHERE 聊天请求;工具与结构化输出做有损降级 -// (参数仅取 JSON Schema 顶层 properties,tool_choice / json_schema 的 name、strict 无对应)。 -func irToCohereSDK(ir aiwire.ChatRequest, stream bool) (generativeaiinference.CohereChatRequest, error) { - var req generativeaiinference.CohereChatRequest - if err := cohereRejectUnsupported(ir); err != nil { - return req, err - } - p, err := cohereSplitMessages(ir.Messages) - if err != nil { - return req, err - } - req = generativeaiinference.CohereChatRequest{ - Message: &p.message, - ChatHistory: p.history, - ToolResults: p.toolResults, - Tools: cohereTools(ir.Tools), - ResponseFormat: cohereResponseFormat(ir.ResponseFormat), - MaxTokens: ir.MaxTokens, - Temperature: ir.Temperature, - TopP: ir.TopP, - TopK: ir.TopK, - FrequencyPenalty: ir.FrequencyPenalty, - PresencePenalty: ir.PresencePenalty, - Seed: ir.Seed, - } - if p.preamble != "" { - req.PreambleOverride = &p.preamble - } - if len(ir.Stop) > 0 { - req.StopSequences = ir.Stop - } - if stream { - req.IsStream = &stream - includeUsage := ir.StreamOptions == nil || ir.StreamOptions.IncludeUsage - req.StreamOptions = &generativeaiinference.StreamOptions{IsIncludeUsage: &includeUsage} - } - return req, nil -} - -// cohereRejectUnsupported 拒绝 COHERE 无法承接的能力(多模态图片)。 -func cohereRejectUnsupported(ir aiwire.ChatRequest) error { - for _, m := range ir.Messages { - for _, p := range m.Content.Parts { - if p.Type == "image_url" { - return fmt.Errorf("cohere 系模型暂不支持图片输入,请改用 meta/google 等多模态模型") - } - } - } - return nil -} - -// cohereParts 是 IR 消息拆装为 COHERE 请求的中间结果。 -type cohereParts struct { - message string - history []generativeaiinference.CohereMessage - preamble string - toolResults []generativeaiinference.CohereToolResult -} - -// cohereSplitMessages 拆装消息:system 拼 preamble,末尾 user 抽为 message, -// assistant(含 toolCalls)与其余 user 进 history,tool 结果转顶层 toolResults -// (工具结果在末尾时 message 留空,COHERE 以 toolResults 续跑)。 -func cohereSplitMessages(msgs []aiwire.ChatMessage) (cohereParts, error) { - lastUser, lastTool := -1, -1 - for i, m := range msgs { - switch m.Role { - case "user": - lastUser = i - case "tool": - lastTool = i - } - } - if lastUser == -1 && lastTool == -1 { - return cohereParts{}, fmt.Errorf("cohere 系模型至少需要一条 user 消息") - } - var p cohereParts - var preamble []string - calls := map[string]generativeaiinference.CohereToolCall{} - for i, m := range msgs { - text := m.Content.JoinText() - switch m.Role { - case "system", "developer": - preamble = append(preamble, text) - case "assistant": - p.history = append(p.history, cohereBotMessage(text, m.ToolCalls, calls)) - case "tool": - p.toolResults = append(p.toolResults, cohereToolResult(m, calls)) - default: // user - if i == lastUser && lastUser > lastTool { - p.message = text - continue - } - p.history = append(p.history, generativeaiinference.CohereUserMessage{Message: &text}) - } - } - p.preamble = strings.Join(preamble, "\n") - return p, nil -} - -// cohereBotMessage 转 assistant 消息;toolCalls 同步登记进 id→call 映射供 tool 结果回查。 -func cohereBotMessage(text string, tcs []aiwire.ToolCall, calls map[string]generativeaiinference.CohereToolCall) generativeaiinference.CohereChatBotMessage { - msg := generativeaiinference.CohereChatBotMessage{} - if text != "" { - msg.Message = &text - } - for _, tc := range tcs { - name := tc.Function.Name - var params interface{} - if json.Unmarshal([]byte(tc.Function.Arguments), ¶ms) != nil || params == nil { - params = map[string]any{} - } - call := generativeaiinference.CohereToolCall{Name: &name, Parameters: ¶ms} - calls[tc.ID] = call - msg.ToolCalls = append(msg.ToolCalls, call) - } - return msg -} - -// cohereToolResult 把 IR tool 消息转顶层工具结果;COHERE 工具调用无 id, -// 靠 assistant 历史登记的映射回查,缺失时以 tool_call_id 名义调用兜底。 -func cohereToolResult(m aiwire.ChatMessage, calls map[string]generativeaiinference.CohereToolCall) generativeaiinference.CohereToolResult { - call, ok := calls[m.ToolCallID] - if !ok { - name := m.ToolCallID - var params interface{} = map[string]any{} - call = generativeaiinference.CohereToolCall{Name: &name, Parameters: ¶ms} - } - text := m.Content.JoinText() - var output interface{} - if json.Unmarshal([]byte(text), &output) != nil || output == nil { - output = map[string]any{"output": text} - } - if arr, isArr := output.([]interface{}); isArr { - return generativeaiinference.CohereToolResult{Call: &call, Outputs: arr} - } - if _, isMap := output.(map[string]interface{}); !isMap { - output = map[string]any{"output": output} - } - return generativeaiinference.CohereToolResult{Call: &call, Outputs: []interface{}{output}} -} - -// cohereTools 把 JSON Schema 工具定义降级为 COHERE 扁平参数表(嵌套结构有损:仅取顶层)。 -func cohereTools(tools []aiwire.Tool) []generativeaiinference.CohereTool { - if len(tools) == 0 { - return nil - } - out := make([]generativeaiinference.CohereTool, 0, len(tools)) - for _, t := range tools { - name, desc := t.Function.Name, t.Function.Description - if desc == "" { - desc = name // Description 为 COHERE 必填 - } - out = append(out, generativeaiinference.CohereTool{ - Name: &name, Description: &desc, - ParameterDefinitions: cohereParams(t.Function.Parameters), - }) - } - return out -} - -// cohereParams 取 JSON Schema 顶层 properties 转扁平参数定义;嵌套 schema 只保留类型名。 -func cohereParams(schema json.RawMessage) map[string]generativeaiinference.CohereParameterDefinition { - var s struct { - Properties map[string]struct { - Type string `json:"type"` - Description string `json:"description"` - } `json:"properties"` - Required []string `json:"required"` - } - if json.Unmarshal(schema, &s) != nil || len(s.Properties) == 0 { - return nil - } - req := map[string]bool{} - for _, r := range s.Required { - req[r] = true - } - out := make(map[string]generativeaiinference.CohereParameterDefinition, len(s.Properties)) - for k, v := range s.Properties { - typ, desc := v.Type, v.Description - if typ == "" { - typ = "string" - } - pd := generativeaiinference.CohereParameterDefinition{Type: &typ} - if desc != "" { - pd.Description = &desc - } - if req[k] { - t := true - pd.IsRequired = &t - } - out[k] = pd - } - return out -} - -// cohereResponseFormat 映射 json_object / json_schema(name、strict 无对应,有损降级)。 -func cohereResponseFormat(rf *aiwire.ResponseFormat) generativeaiinference.CohereResponseFormat { - if rf == nil { - return nil - } - switch rf.Type { - case "json_object", "json_schema": - out := generativeaiinference.CohereResponseJsonFormat{} - var in struct { - Schema json.RawMessage `json:"schema"` - } - if json.Unmarshal(rf.JSONSchema, &in) == nil && len(in.Schema) > 0 { - var schema interface{} - if json.Unmarshal(in.Schema, &schema) == nil { - out.Schema = &schema - } - } - return out - } - return nil -} - -// cohereSDKToIR 把 COHERE 非流式响应转回 IR;工具调用补派生 ID,存在时结束原因归 tool_calls。 -func cohereSDKToIR(resp generativeaiinference.CohereChatResponse, model string) *aiwire.ChatResponse { - msg := aiwire.ChatMessage{Role: "assistant", Content: aiwire.NewTextContent(deref(resp.Text))} - for i, tc := range resp.ToolCalls { - msg.ToolCalls = append(msg.ToolCalls, cohereCallToIR(deref(tc.Name), tc.Parameters, i)) - } - finish := mapFinishReason(string(resp.FinishReason)) - if len(msg.ToolCalls) > 0 { - finish = "tool_calls" - } - return &aiwire.ChatResponse{ - Object: "chat.completion", - Model: model, - Choices: []aiwire.Choice{{Message: msg, FinishReason: finish}}, - Usage: sdkUsageToIR(resp.Usage), - } -} - -// cohereCallToIR 生成 IR 工具调用;COHERE 无调用 id,按名称+序号派生(回传时按此回查)。 -func cohereCallToIR(name string, params *interface{}, idx int) aiwire.ToolCall { - args := "{}" - if params != nil && *params != nil { - if b, err := json.Marshal(*params); err == nil { - args = string(b) - } - } - return aiwire.ToolCall{ - ID: fmt.Sprintf("call_%s_%d", name, idx), - Type: "function", - Function: aiwire.FunctionCall{Name: name, Arguments: args}, - } -} diff --git a/internal/oci/genai_convert.go b/internal/oci/genai_convert.go deleted file mode 100644 index c1f9cdc..0000000 --- a/internal/oci/genai_convert.go +++ /dev/null @@ -1,367 +0,0 @@ -package oci - -import ( - "encoding/json" - "strings" - - "github.com/oracle/oci-go-sdk/v65/generativeaiinference" - - "oci-portal/internal/aiwire" -) - -// irToSDK 把 IR(OpenAI 线格式)转为 OCI GENERIC 聊天请求;字段近一一镜像。 -func irToSDK(ir aiwire.ChatRequest, stream bool) generativeaiinference.GenericChatRequest { - req := generativeaiinference.GenericChatRequest{ - Messages: irMessages(ir.Messages), - MaxTokens: ir.MaxTokens, - MaxCompletionTokens: ir.MaxCompletionTokens, - Temperature: ir.Temperature, - TopP: ir.TopP, - TopK: ir.TopK, - FrequencyPenalty: ir.FrequencyPenalty, - PresencePenalty: ir.PresencePenalty, - Seed: ir.Seed, - NumGenerations: ir.N, - Tools: irTools(ir.Tools), - ToolChoice: irToolChoice(ir.ToolChoice), - ResponseFormat: irResponseFormat(ir.ResponseFormat), - } - if len(ir.Stop) > 0 { - req.Stop = ir.Stop - } - if ir.ReasoningEffort != "" { - req.ReasoningEffort = generativeaiinference.GenericChatRequestReasoningEffortEnum(strings.ToUpper(ir.ReasoningEffort)) - } - if stream { - req.IsStream = &stream - includeUsage := ir.StreamOptions == nil || ir.StreamOptions.IncludeUsage - req.StreamOptions = &generativeaiinference.StreamOptions{IsIncludeUsage: &includeUsage} - } - return req -} - -// irMessages 按 role 拆装消息;user 消息保留图文块,其余角色只取文本。 -func irMessages(msgs []aiwire.ChatMessage) []generativeaiinference.Message { - out := make([]generativeaiinference.Message, 0, len(msgs)) - for _, m := range msgs { - switch m.Role { - case "system", "developer": - out = append(out, generativeaiinference.SystemMessage{Content: textContents(m.Content.JoinText())}) - case "assistant": - out = append(out, generativeaiinference.AssistantMessage{ - Content: textContents(m.Content.JoinText()), - ToolCalls: irToolCalls(m.ToolCalls), - }) - case "tool": - tm := generativeaiinference.ToolMessage{Content: textContents(m.Content.JoinText())} - if m.ToolCallID != "" { - id := m.ToolCallID - tm.ToolCallId = &id - } - out = append(out, tm) - default: // user - out = append(out, generativeaiinference.UserMessage{Content: chatContents(m.Content)}) - } - } - return out -} - -// chatContents 把 IR 内容转 SDK 内容块;文本直通,image_url 转 IMAGE 块(data URI / 公网地址)。 -func chatContents(c aiwire.Content) []generativeaiinference.ChatContent { - if len(c.Parts) == 0 { - return textContents(c.Text) - } - var out []generativeaiinference.ChatContent - for _, p := range c.Parts { - switch { - case p.Type == "image_url" && p.ImageURL != nil: - img := generativeaiinference.ImageUrl{Url: &p.ImageURL.URL} - if d, ok := generativeaiinference.GetMappingImageUrlDetailEnum(p.ImageURL.Detail); ok { - img.Detail = d - } - out = append(out, generativeaiinference.ImageContent{ImageUrl: &img}) - case p.Text != "": - t := p.Text - out = append(out, generativeaiinference.TextContent{Text: &t}) - } - } - return out -} - -// textContents 把纯文本包装为单元素 TextContent 数组;空文本返回 nil。 -func textContents(text string) []generativeaiinference.ChatContent { - if text == "" { - return nil - } - return []generativeaiinference.ChatContent{generativeaiinference.TextContent{Text: &text}} -} - -func irToolCalls(calls []aiwire.ToolCall) []generativeaiinference.ToolCall { - if len(calls) == 0 { - return nil - } - out := make([]generativeaiinference.ToolCall, 0, len(calls)) - for _, c := range calls { - id, name, args := c.ID, c.Function.Name, c.Function.Arguments - out = append(out, generativeaiinference.FunctionCall{Id: &id, Name: &name, Arguments: &args}) - } - return out -} - -func irTools(tools []aiwire.Tool) []generativeaiinference.ToolDefinition { - if len(tools) == 0 { - return nil - } - out := make([]generativeaiinference.ToolDefinition, 0, len(tools)) - for _, t := range tools { - name, desc := t.Function.Name, t.Function.Description - fd := generativeaiinference.FunctionDefinition{Name: &name} - if desc != "" { - fd.Description = &desc - } - if len(t.Function.Parameters) > 0 { - var params interface{} - if json.Unmarshal(t.Function.Parameters, ¶ms) == nil { - fd.Parameters = ¶ms - } - } - out = append(out, fd) - } - return out -} - -// irToolChoice 解析 "auto"/"none"/"required" 或 {type:function,function:{name}}。 -func irToolChoice(raw json.RawMessage) generativeaiinference.ToolChoice { - if len(raw) == 0 { - return nil - } - var s string - if json.Unmarshal(raw, &s) == nil { - switch s { - case "none": - return generativeaiinference.ToolChoiceNone{} - case "required": - return generativeaiinference.ToolChoiceRequired{} - case "auto": - return generativeaiinference.ToolChoiceAuto{} - } - return nil - } - var obj struct { - Function struct { - Name string `json:"name"` - } `json:"function"` - } - if json.Unmarshal(raw, &obj) == nil && obj.Function.Name != "" { - return generativeaiinference.ToolChoiceFunction{Name: &obj.Function.Name} - } - return nil -} - -func irResponseFormat(rf *aiwire.ResponseFormat) generativeaiinference.ResponseFormat { - if rf == nil { - return nil - } - switch rf.Type { - case "json_object": - return generativeaiinference.JsonObjectResponseFormat{} - case "json_schema": - return irJSONSchemaFormat(rf.JSONSchema) - default: - return nil - } -} - -// irJSONSchemaFormat 映射 OpenAI json_schema{name,description,schema,strict}。 -func irJSONSchemaFormat(raw json.RawMessage) generativeaiinference.ResponseFormat { - var in struct { - Name string `json:"name"` - Description string `json:"description"` - Schema json.RawMessage `json:"schema"` - Strict *bool `json:"strict"` - } - if json.Unmarshal(raw, &in) != nil || in.Name == "" { - return generativeaiinference.JsonObjectResponseFormat{} - } - js := generativeaiinference.ResponseJsonSchema{Name: &in.Name, IsStrict: in.Strict} - if in.Description != "" { - js.Description = &in.Description - } - if len(in.Schema) > 0 { - var schema interface{} - if json.Unmarshal(in.Schema, &schema) == nil { - js.Schema = &schema - } - } - return generativeaiinference.JsonSchemaResponseFormat{JsonSchema: &js} -} - -// sdkToIR 把非流式 GenericChatResponse 转回 IR 响应;id/created 由调用方补。 -func sdkToIR(resp generativeaiinference.GenericChatResponse, model string) *aiwire.ChatResponse { - out := &aiwire.ChatResponse{Object: "chat.completion", Model: model} - if resp.TimeCreated != nil { - out.Created = resp.TimeCreated.Unix() - } - for i, ch := range resp.Choices { - idx := i - if ch.Index != nil { - idx = *ch.Index - } - out.Choices = append(out.Choices, aiwire.Choice{ - Index: idx, - Message: sdkMessageToIR(ch.Message), - FinishReason: mapFinishReason(deref(ch.FinishReason)), - }) - } - out.Usage = sdkUsageToIR(resp.Usage) - return out -} - -func sdkUsageToIR(u *generativeaiinference.Usage) *aiwire.Usage { - if u == nil { - return nil - } - out := &aiwire.Usage{} - if u.PromptTokens != nil { - out.PromptTokens = *u.PromptTokens - } - if u.CompletionTokens != nil { - out.CompletionTokens = *u.CompletionTokens - } - if u.TotalTokens != nil { - out.TotalTokens = *u.TotalTokens - } - if d := u.PromptTokensDetails; d != nil && d.CachedTokens != nil && *d.CachedTokens > 0 { - out.PromptTokensDetails = &aiwire.PromptTokensDetails{CachedTokens: *d.CachedTokens} - } - return out -} - -// sdkMessageToIR 提取助手消息文本与工具调用。 -func sdkMessageToIR(m generativeaiinference.Message) aiwire.ChatMessage { - out := aiwire.ChatMessage{Role: "assistant"} - am, ok := m.(generativeaiinference.AssistantMessage) - if !ok { - return out - } - var sb strings.Builder - for _, c := range am.Content { - if tc, ok := c.(generativeaiinference.TextContent); ok && tc.Text != nil { - sb.WriteString(*tc.Text) - } - } - out.Content = aiwire.NewTextContent(sb.String()) - for _, call := range am.ToolCalls { - if fc, ok := call.(generativeaiinference.FunctionCall); ok { - out.ToolCalls = append(out.ToolCalls, aiwire.ToolCall{ - ID: deref(fc.Id), - Type: "function", - Function: aiwire.FunctionCall{ - Name: deref(fc.Name), - Arguments: deref(fc.Arguments), - }, - }) - } - } - return out -} - -// mapFinishReason 归一化结束原因;OCI 与 OpenAI 取值同名,做小写与别名兜底。 -func mapFinishReason(reason string) string { - switch strings.ToLower(reason) { - case "", "null": - return "" - case "stop", "completed", "end_turn": - return "stop" - case "length", "max_tokens": - return "length" - case "tool_calls", "tool_call", "tool_use": - return "tool_calls" - case "complete": - return "stop" - case "error_toxic": - return "content_filter" - default: - return strings.ToLower(reason) - } -} - -// genAiStreamEvent 是 GENERIC 流式事件的宽容解析结构(字段按增量 choice 形状)。 -type genAiStreamEvent struct { - Index *int `json:"index"` - Message struct { - Role string `json:"role"` - Content []struct { - Type string `json:"type"` - Text string `json:"text"` - } `json:"content"` - ToolCalls []struct { - ID string `json:"id"` - Name string `json:"name"` - Arguments string `json:"arguments"` - } `json:"toolCalls"` - } `json:"message"` - FinishReason *string `json:"finishReason"` - Usage *generativeaiinference.Usage `json:"usage"` - Choices []json.RawMessage `json:"choices"` // 兼容整包 choices 形态 - Text string `json:"text"` // COHERE 流事件的顶层增量文本 - // CohereCalls 是 COHERE 流事件顶层的工具调用(与 GENERIC 的 message.toolCalls 不同层级) - CohereCalls []struct { - Name string `json:"name"` - Parameters interface{} `json:"parameters"` - } `json:"toolCalls"` -} - -// parseGenAiEvent 把一条 SSE 事件 JSON 解析为 IR chunk;无有效负载时 ok=false。 -func parseGenAiEvent(data []byte, model string) (aiwire.ChatChunk, bool) { - var ev genAiStreamEvent - if err := json.Unmarshal(data, &ev); err != nil { - return aiwire.ChatChunk{}, false - } - if len(ev.Choices) > 0 { // choices 包裹形态:取首个展开重解析 - var inner genAiStreamEvent - if json.Unmarshal(ev.Choices[0], &inner) == nil { - inner.Usage = ev.Usage - ev = inner - } - } - chunk := aiwire.ChatChunk{Object: "chat.completion.chunk", Model: model} - choice := aiwire.ChunkChoice{} - if ev.Index != nil { - choice.Index = *ev.Index - } - choice.Delta.Role = strings.ToLower(ev.Message.Role) - var sb strings.Builder - for _, c := range ev.Message.Content { - sb.WriteString(c.Text) - } - if sb.Len() == 0 && ev.Text != "" { // COHERE 事件形态 - sb.WriteString(ev.Text) - } - choice.Delta.Content = sb.String() - for i, tc := range ev.Message.ToolCalls { - d := aiwire.ToolCallDelta{Index: i, ID: tc.ID} - if tc.ID != "" { - d.Type = "function" - } - d.Function.Name = tc.Name - d.Function.Arguments = tc.Arguments - choice.Delta.ToolCalls = append(choice.Delta.ToolCalls, d) - } - for i, tc := range ev.CohereCalls { // COHERE 顶层工具调用:整包出现,补派生 ID - call := cohereCallToIR(tc.Name, &tc.Parameters, i) - d := aiwire.ToolCallDelta{Index: len(choice.Delta.ToolCalls) + i, ID: call.ID, Type: "function"} - d.Function.Name = call.Function.Name - d.Function.Arguments = call.Function.Arguments - choice.Delta.ToolCalls = append(choice.Delta.ToolCalls, d) - } - if r := mapFinishReason(deref(ev.FinishReason)); r != "" { - choice.FinishReason = &r - } - hasPayload := choice.Delta.Content != "" || len(choice.Delta.ToolCalls) > 0 || choice.FinishReason != nil - if hasPayload { - chunk.Choices = []aiwire.ChunkChoice{choice} - } - chunk.Usage = sdkUsageToIR(ev.Usage) - return chunk, hasPayload || chunk.Usage != nil -} diff --git a/internal/oci/genai_effort_integration_test.go b/internal/oci/genai_effort_integration_test.go new file mode 100644 index 0000000..fcf66fe --- /dev/null +++ b/internal/oci/genai_effort_integration_test.go @@ -0,0 +1,496 @@ +package oci + +// 真实 OCI GenAI effort 探针(重建版)。 +// +// 前身 genai_cache_integration_test.go 承载缓存/亲和/服务端工具等历轮调研模式, +// 调研结论均已归档(.trellis/tasks/archive/2026-07/*/research/),该文件已删除; +// 本文件重建共享基础设施,保留 effort 支持矩阵探测,并新增 responses 端点探测 +// (multi-agent 模型仅允许走 Responses 面)。 +// +// 运行(默认跳过,不触网): +// +// OCI_GENAI_CACHE_PROBE=effort-matrix OCI_CACHE_DB= DATA_KEY=<主密钥> \ +// OCI_CACHE_RUN_ID=<唯一标识> go test ./internal/oci/ -run TestRealGenAiEffortMatrix -v +// +// 可选:OCI_CACHE_CONFIG_ID(默认 6)、OCI_CACHE_REGION(默认 us-chicago-1)、 +// OCI_EFFORT_CASES 自定义用例("model=effort" 逗号分隔;effort 后缀 "@responses" +// 表示走 /actions/v1/responses 端点,如 "xai.grok-4.20-multi-agent=low@responses")。 +// +// 纪律:数据库只读打开;OCI_GO_SDK_DEBUG 必须为空;日志不落原始请求体与凭据, +// 错误消息经 OCID 遮掩后截断。 + +import ( + "context" + "encoding/json" + "fmt" + "io" + "net/http" + "os" + "regexp" + "strings" + "testing" + "time" + + "github.com/glebarez/sqlite" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/generativeaiinference" + "gorm.io/gorm" + "gorm.io/gorm/logger" + + projectcrypto "oci-portal/internal/crypto" + "oci-portal/internal/model" +) + +const ( + effortProbeRegion = "us-chicago-1" + effortProbeLimit = int64(4 << 20) +) + +var effortRunIDPattern = regexp.MustCompile(`^[A-Za-z0-9._-]{1,64}$`) + +type effortProbeCase struct { + model string + effort string + responses bool // true 走 /actions/v1/responses,否则 typed /actions/chat + stream bool // 仅 responses 面:请求 SSE 流,观测事件类型序列 +} + +type effortProbeResources struct { + cred Credentials + region string + modelOCID map[string]string + inference generativeaiinference.GenerativeAiInferenceClient +} + +type effortProbeResult struct { + status int + requestRef string + duration time.Duration + errorCode string + errorMessage string + completionTokens int + reasoningTokens int + answerLen int + outputTypes []string +} + +// effortMatrixCases 默认批为 grok-4.3 五档基线;全模型矩阵结论见任务归档, +// 复测用 OCI_EFFORT_CASES 自定义。 +func effortMatrixCases() []effortProbeCase { + if raw := os.Getenv("OCI_EFFORT_CASES"); raw != "" { + return parseEffortCases(raw) + } + return []effortProbeCase{ + {model: "xai.grok-4.3", effort: "none", responses: true}, + {model: "xai.grok-4.3", effort: "low", responses: true}, + {model: "xai.grok-4.3", effort: "high", responses: true}, + } +} + +// parseEffortCases 解析 "model=effort[@responses|@responses-stream]" 逗号分隔用例。 +func parseEffortCases(raw string) []effortProbeCase { + var out []effortProbeCase + for _, item := range strings.Split(raw, ",") { + parts := strings.SplitN(strings.TrimSpace(item), "=", 2) + if len(parts) != 2 || parts[0] == "" || parts[1] == "" { + continue + } + effort, streaming := strings.CutSuffix(parts[1], "@responses-stream") + if streaming { + out = append(out, effortProbeCase{model: parts[0], effort: effort, responses: true, stream: true}) + continue + } + effort, _ = strings.CutSuffix(effort, "@responses") // 后缀兼容保留;所有用例均走 responses 面 + out = append(out, effortProbeCase{model: parts[0], effort: effort, responses: true}) + } + return out +} + +func TestRealGenAiEffortMatrix(t *testing.T) { + requireEffortProbeMode(t, "effort-matrix") + ctx, cancel := context.WithTimeout(context.Background(), 20*time.Minute) + defer cancel() + resources := newEffortProbeResources(t, ctx) + for _, tc := range effortMatrixCases() { + runEffortCase(t, ctx, resources, tc) + } +} + +func requireEffortProbeMode(t *testing.T, wanted string) { + t.Helper() + if os.Getenv("OCI_GENAI_CACHE_PROBE") != wanted { + t.Skipf("set OCI_GENAI_CACHE_PROBE=%s to run", wanted) + } + if os.Getenv("OCI_GO_SDK_DEBUG") != "" { + t.Fatal("OCI_GO_SDK_DEBUG must be unset to protect signed requests") + } + if !effortRunIDPattern.MatchString(os.Getenv("OCI_CACHE_RUN_ID")) { + t.Fatal("set a unique OCI_CACHE_RUN_ID (1-64 safe characters)") + } +} + +func runEffortCase(t *testing.T, ctx context.Context, resources effortProbeResources, tc effortProbeCase) { + t.Helper() + ocid, ok := resources.modelOCID[tc.model] + if !ok { + t.Logf("model=%s effort=%s resolve-failed: model not in on-demand catalog", tc.model, tc.effort) + return + } + _ = ocid + body, path, err := effortProbeBody(tc) + if err != nil { + t.Fatalf("build effort body: %v", err) + } + result := executeEffortProbe(ctx, resources, path, body) + logEffortResult(t, tc, result) +} + +// effortProbeBody 构造 OpenAI Responses 直通请求体(typed chat 面已随网关剔除, +// 探针仅保留 responses 端点;历史 typed 结论见任务归档)。 +func effortProbeBody(tc effortProbeCase) ([]byte, string, error) { + question := "How many positive divisors does 360 have? Answer with the number only." + body := map[string]interface{}{"model": tc.model, "input": question, + "max_output_tokens": 2048, "store": false} + if tc.effort != "" { + body["reasoning"] = map[string]string{"effort": tc.effort} + } + if tc.stream { + body["stream"] = true + } + payload, err := json.Marshal(body) + return payload, "/actions/v1/responses", err +} + +// executeEffortProbe 以 IAM 签名裸 POST 指定路径;responses 面按生产实现补 compartment 头。 +func executeEffortProbe(ctx context.Context, resources effortProbeResources, path string, body []byte) effortProbeResult { + client := resources.inference.BaseClient + client.Configuration.CircuitBreaker = nil + common.UpdateEndpointTemplateForOptions(&client) + common.SetMissingTemplateParams(&client) + request, err := http.NewRequestWithContext(ctx, http.MethodPost, path, strings.NewReader(string(body))) + if err != nil { + return effortProbeResult{errorCode: "BuildRequest", errorMessage: err.Error()} + } + request.Header.Set("Content-Type", "application/json") + if path != "/actions/chat" { + request.Header.Set("CompartmentId", resources.cred.TenancyOCID) + request.Header.Set("opc-compartment-id", resources.cred.TenancyOCID) + } + return effortCallProbe(ctx, client, request) +} + +func effortCallProbe(ctx context.Context, client common.BaseClient, request *http.Request) effortProbeResult { + started := time.Now() + response, err := client.Call(ctx, request) + result := effortProbeResult{duration: time.Since(started)} + if response != nil { + result.status = response.StatusCode + result.requestRef = shortEffortRef(response.Header.Get("opc-request-id")) + defer response.Body.Close() + } + if err != nil { + result.errorCode, result.errorMessage = effortProbeError(err) + return result + } + payload, readErr := io.ReadAll(io.LimitReader(response.Body, effortProbeLimit)) + if readErr != nil { + result.errorCode, result.errorMessage = "ReadResponse", readErr.Error() + return result + } + parseEffortPayload(&result, payload) + return result +} + +func effortProbeError(err error) (string, string) { + if serviceErr, ok := common.IsServiceError(err); ok { + return fmt.Sprintf("%d", serviceErr.GetHTTPStatusCode()), sanitizeEffortText(serviceErr.GetMessage()) + } + return "CallError", sanitizeEffortText(err.Error()) +} + +// parseEffortPayload 兼容三种 usage 形态:typed 驼峰(chatResponse.usage.completionTokens)、 +// chat 兼容面下划线(usage.completion_tokens)、responses 面(usage.output_tokens)。 +func parseEffortPayload(result *effortProbeResult, payload []byte) { + if text := strings.TrimSpace(string(payload)); strings.HasPrefix(text, "event:") || strings.HasPrefix(text, "data:") { + parseEffortSSE(result, text) + return + } + var root map[string]interface{} + if json.Unmarshal(payload, &root) != nil { + return + } + result.answerLen = len(parseEffortAnswer(root)) + result.outputTypes = parseEffortOutputTypes(root) + if typed, ok := root["chatResponse"].(map[string]interface{}); ok { + root = typed + } + usage, _ := root["usage"].(map[string]interface{}) + for _, key := range []string{"completionTokens", "completion_tokens", "output_tokens"} { + if v, ok := effortInt(usage, key); ok { + result.completionTokens = v + break + } + } + for _, detailsKey := range []string{"completionTokensDetails", "completion_tokens_details", "output_tokens_details"} { + details, _ := usage[detailsKey].(map[string]interface{}) + for _, key := range []string{"reasoningTokens", "reasoning_tokens"} { + if v, ok := effortInt(details, key); ok { + result.reasoningTokens = v + } + } + } +} + +// parseEffortAnswer 提取正文文本:typed 取 chatResponse.choices[].message.content[].text, +// responses 取 output[] 里 message 项的 content[].text。仅在内存中量长度,不落日志。 +func parseEffortAnswer(root map[string]interface{}) string { + var sb strings.Builder + if typed, ok := root["chatResponse"].(map[string]interface{}); ok { + choices, _ := typed["choices"].([]interface{}) + for _, c := range choices { + cm, _ := c.(map[string]interface{}) + msg, _ := cm["message"].(map[string]interface{}) + collectEffortText(&sb, msg["content"]) + } + return sb.String() + } + output, _ := root["output"].([]interface{}) + for _, item := range output { + im, _ := item.(map[string]interface{}) + if im["type"] == "message" { + collectEffortText(&sb, im["content"]) + } + } + return sb.String() +} + +func collectEffortText(sb *strings.Builder, content interface{}) { + parts, _ := content.([]interface{}) + for _, p := range parts { + pm, _ := p.(map[string]interface{}) + if text, ok := pm["text"].(string); ok { + sb.WriteString(text) + } + } +} + +// parseEffortSSE 汇总 SSE 流:事件类型去重序列进 outputTypes,正文增量长度进 answerLen, +// 末尾 completed 事件里的 usage 进 tokens 字段。 +func parseEffortSSE(result *effortProbeResult, text string) { + seen := map[string]bool{} + for _, line := range strings.Split(text, "\n") { + if name, ok := strings.CutPrefix(line, "event: "); ok { + if name = strings.TrimSpace(name); !seen[name] { + seen[name] = true + result.outputTypes = append(result.outputTypes, name) + } + continue + } + data, ok := strings.CutPrefix(line, "data: ") + if !ok { + continue + } + var ev map[string]interface{} + if json.Unmarshal([]byte(data), &ev) != nil { + continue + } + if name, ok := ev["type"].(string); ok && !seen[name] { // 纯 data: 行流(无 event: 行)从事件体取类型 + seen[name] = true + result.outputTypes = append(result.outputTypes, name) + } + if delta, ok := ev["delta"].(string); ok && strings.Contains(fmt.Sprint(ev["type"]), "output_text") { + result.answerLen += len(delta) + } + if resp, ok := ev["response"].(map[string]interface{}); ok { + usage, _ := resp["usage"].(map[string]interface{}) + if v, ok := effortInt(usage, "output_tokens"); ok { + result.completionTokens = v + } + } + } +} + +// parseEffortOutputTypes 收集 responses 面 output 项类型序列(multi-agent 行为观测)。 +func parseEffortOutputTypes(root map[string]interface{}) []string { + output, _ := root["output"].([]interface{}) + var types []string + for _, item := range output { + im, _ := item.(map[string]interface{}) + if s, ok := im["type"].(string); ok { + types = append(types, s) + } + } + return types +} + +func effortInt(root map[string]interface{}, key string) (int, bool) { + if root == nil { + return 0, false + } + if value, ok := root[key].(float64); ok { + return int(value), true + } + return 0, false +} + +func logEffortResult(t *testing.T, tc effortProbeCase, result effortProbeResult) { + t.Helper() + endpoint := "typed-chat" + if tc.responses { + endpoint = "compat-responses" + } + t.Logf("endpoint=%s model=%s effort=%s status=%d completion=%d reasoning=%d answer_len=%d output_types=%v request=%s duration_ms=%d error=%s message=%q", + endpoint, tc.model, tc.effort, result.status, result.completionTokens, result.reasoningTokens, + result.answerLen, result.outputTypes, result.requestRef, result.duration.Milliseconds(), + result.errorCode, result.errorMessage) +} + +// sanitizeEffortText 遮掩 OCID 并截断,避免错误消息携带租户可定位信息。 +func sanitizeEffortText(text string) string { + text = regexp.MustCompile(`ocid1\.[a-z0-9._-]+`).ReplaceAllString(text, "ocid1.***") + if len(text) > 300 { + text = text[:300] + "..." + } + return text +} + +func shortEffortRef(ref string) string { + if len(ref) > 12 { + return ref[:12] + } + return ref +} + +func effortEnvOr(key, fallback string) string { + if value := os.Getenv(key); value != "" { + return value + } + return fallback +} + +// newEffortProbeResources 装配凭据、区域、inference 客户端与按需模型目录。 +func newEffortProbeResources(t *testing.T, ctx context.Context) effortProbeResources { + t.Helper() + cred := effortProbeCredentials(t) + region := effortEnvOr("OCI_CACHE_REGION", effortProbeRegion) + cred.Region = region + real := &RealClient{} + ic, err := real.genAiInferenceClient(cred, region) + if err != nil { + t.Fatalf("new inference client: %v", err) + } + models, err := real.ListGenAiModels(ctx, cred, region) + if err != nil { + t.Fatalf("list models: %s", sanitizeEffortText(err.Error())) + } + index := make(map[string]string, len(models)) + for _, m := range models { + index[m.Name] = m.Ocid + } + return effortProbeResources{cred: cred, region: region, modelOCID: index, inference: ic} +} + +// effortProbeCredentials 优先从面板数据库(只读)取渠道凭据,否则退回本地 ini 测试凭据。 +func effortProbeCredentials(t *testing.T) Credentials { + t.Helper() + if dbPath := os.Getenv("OCI_CACHE_DB"); dbPath != "" { + return effortDBCredentials(t, dbPath) + } + return loadTestCredentials(t, effortEnvOr("OCI_TEST_KEY", "试用期")) +} + +func effortDBCredentials(t *testing.T, dbPath string) Credentials { + t.Helper() + dsn := fmt.Sprintf("file:%s?mode=ro", dbPath) + db, err := gorm.Open(sqlite.Open(dsn), &gorm.Config{Logger: logger.Default.LogMode(logger.Silent)}) + if err != nil { + t.Fatalf("open read-only probe database: %v", err) + } + cipher, err := projectcrypto.NewCipher(os.Getenv("DATA_KEY")) + if err != nil { + t.Fatalf("create data cipher: %v", err) + } + var config model.OciConfig + if err := db.First(&config, effortEnvOr("OCI_CACHE_CONFIG_ID", "6")).Error; err != nil { + t.Fatalf("load OCI config: %v", err) + } + return decryptEffortCredentials(t, db, cipher, config) +} + +func decryptEffortCredentials(t *testing.T, db *gorm.DB, cipher *projectcrypto.Cipher, config model.OciConfig) Credentials { + t.Helper() + privateKey, err := cipher.DecryptString(config.PrivateKeyEnc) + if err != nil { + t.Fatalf("decrypt OCI config %d private key: %v", config.ID, err) + } + passphrase := "" + if config.PassphraseEnc != "" { + if passphrase, err = cipher.DecryptString(config.PassphraseEnc); err != nil { + t.Fatalf("decrypt OCI config %d passphrase: %v", config.ID, err) + } + } + return Credentials{TenancyOCID: config.TenancyOCID, UserOCID: config.UserOCID, + Fingerprint: config.Fingerprint, Region: config.Region, PrivateKey: privateKey, + Passphrase: passphrase, Proxy: loadEffortProxy(t, db, cipher, config.ProxyID)} +} + +func loadEffortProxy(t *testing.T, db *gorm.DB, cipher *projectcrypto.Cipher, proxyID *uint) *ProxySpec { + t.Helper() + if proxyID == nil { + return nil + } + var proxy model.Proxy + if err := db.First(&proxy, *proxyID).Error; err != nil { + t.Fatalf("load proxy %d: %v", *proxyID, err) + } + password := "" + if proxy.PasswordEnc != "" { + decrypted, err := cipher.DecryptString(proxy.PasswordEnc) + if err != nil { + t.Fatalf("decrypt proxy %d password: %v", *proxyID, err) + } + password = decrypted + } + return &ProxySpec{Type: proxy.Type, Host: proxy.Host, Port: proxy.Port, + Username: proxy.Username, Password: password} +} + +// TestParseEffortCases 断言自定义用例解析:端点后缀、空项与畸形项跳过。 +func TestParseEffortCases(t *testing.T) { + got := parseEffortCases("a=low, b=high@responses ,bad,=x,c=") + want := []effortProbeCase{{model: "a", effort: "low", responses: true}, {model: "b", effort: "high", responses: true}} + if len(got) != len(want) { + t.Fatalf("parseEffortCases = %+v, want %+v", got, want) + } + for i := range want { + if got[i] != want[i] { + t.Fatalf("case %d = %+v, want %+v", i, got[i], want[i]) + } + } +} + +// TestParseEffortPayload 断言三种 usage 形态与 output 类型序列的解析。 +func TestParseEffortPayload(t *testing.T) { + cases := []struct { + name string + payload string + completion int + reasoning int + types int + }{ + {"typed 驼峰", `{"chatResponse":{"choices":[{"message":{"content":[{"type":"TEXT","text":"24"}]}}],"usage":{"completionTokens":5,"completionTokensDetails":{"reasoningTokens":9}}}}`, 5, 9, 0}, + {"responses 面", `{"output":[{"type":"reasoning"},{"type":"message","content":[{"type":"output_text","text":"24"}]}],"usage":{"output_tokens":7,"output_tokens_details":{"reasoning_tokens":3}}}`, 7, 3, 2}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + var result effortProbeResult + parseEffortPayload(&result, []byte(tc.payload)) + if result.completionTokens != tc.completion || result.reasoningTokens != tc.reasoning || len(result.outputTypes) != tc.types { + t.Fatalf("parse = %+v, want completion=%d reasoning=%d types=%d", result, tc.completion, tc.reasoning, tc.types) + } + if result.answerLen != 2 { + t.Fatalf("answerLen = %d, want 2", result.answerLen) + } + }) + } +} diff --git a/internal/oci/genai_responses.go b/internal/oci/genai_responses.go new file mode 100644 index 0000000..a1e9516 --- /dev/null +++ b/internal/oci/genai_responses.go @@ -0,0 +1,73 @@ +package oci + +import ( + "bytes" + "context" + "fmt" + "io" + "net/http" + + "github.com/oracle/oci-go-sdk/v65/common" +) + +// compatResponsesLimit 限制直通响应体大小;web_search 输出含多段引用,给足余量。 +const compatResponsesLimit = int64(8 << 20) + +// GenAiCompatResponses 实现 Client:把 OpenAI Responses 请求体直通到 OCI +// `/20231130/actions/v1/responses`(IAM 签名)。该端点实测可执行 xAI 服务端工具 +// (web_search / x_search),但不在 Oracle 文档化工具白名单内,行为可能随服务 +// 版本、模型或区域变化;调用方须自行校验并改写请求体(store/stream)。 +func (c *RealClient) GenAiCompatResponses(ctx context.Context, cred Credentials, region string, body []byte) ([]byte, error) { + ic, err := c.genAiInferenceClient(cred, region) + if err != nil { + return nil, err + } + client := ic.BaseClient + common.UpdateEndpointTemplateForOptions(&client) + common.SetMissingTemplateParams(&client) + request, err := http.NewRequestWithContext(ctx, http.MethodPost, "/actions/v1/responses", bytes.NewReader(body)) + if err != nil { + return nil, fmt.Errorf("build compat responses request: %w", err) + } + request.Header.Set("Content-Type", "application/json") + request.Header.Set("CompartmentId", cred.TenancyOCID) + request.Header.Set("opc-compartment-id", cred.TenancyOCID) + response, err := client.Call(ctx, request) + if err != nil { + return nil, err + } + defer response.Body.Close() + payload, err := io.ReadAll(io.LimitReader(response.Body, compatResponsesLimit)) + if err != nil { + return nil, fmt.Errorf("read compat responses body: %w", err) + } + return payload, nil +} + +// GenAiCompatResponsesStream 实现 Client:以流式直通 OCI `/actions/v1/responses`, +// 建立成功(2xx)返回 SSE body(调用方负责 Close);建立失败返回 SDK ServiceError, +// 与既有渠道切换/熔断错误分类兼容。请求体须由调用方置 stream:true。 +func (c *RealClient) GenAiCompatResponsesStream(ctx context.Context, cred Credentials, region string, body []byte) (io.ReadCloser, error) { + ic, err := c.genAiInferenceClient(cred, region) + if err != nil { + return nil, err + } + client := ic.BaseClient + common.UpdateEndpointTemplateForOptions(&client) + common.SetMissingTemplateParams(&client) + request, err := http.NewRequestWithContext(ctx, http.MethodPost, "/actions/v1/responses", bytes.NewReader(body)) + if err != nil { + return nil, fmt.Errorf("build compat responses stream request: %w", err) + } + request.Header.Set("Content-Type", "application/json") + request.Header.Set("CompartmentId", cred.TenancyOCID) + request.Header.Set("opc-compartment-id", cred.TenancyOCID) + response, err := client.Call(ctx, request) + if err != nil { + if response != nil && response.Body != nil { + response.Body.Close() + } + return nil, err + } + return response.Body, nil +} diff --git a/internal/oci/genai_test.go b/internal/oci/genai_test.go index 336c474..aecdfda 100644 --- a/internal/oci/genai_test.go +++ b/internal/oci/genai_test.go @@ -1,193 +1,27 @@ package oci import ( - "encoding/json" "testing" "github.com/oracle/oci-go-sdk/v65/generativeaiinference" - - "oci-portal/internal/aiwire" ) -// TestIrToSDK 断言 IR→GENERIC 的角色拆装、采样参数与工具映射。 -func TestIrToSDK(t *testing.T) { - temp, mt := 0.7, 100 - ir := aiwire.ChatRequest{ - Model: "meta.llama-3.3-70b-instruct", - Messages: []aiwire.ChatMessage{ - {Role: "system", Content: aiwire.NewTextContent("你是助手")}, - {Role: "user", Content: aiwire.NewTextContent("你好")}, - {Role: "assistant", ToolCalls: []aiwire.ToolCall{{ID: "c1", Type: "function", Function: aiwire.FunctionCall{Name: "get_weather", Arguments: `{"city":"东京"}`}}}}, - {Role: "tool", ToolCallID: "c1", Content: aiwire.NewTextContent("晴")}, - }, - Temperature: &temp, - MaxTokens: &mt, - Stop: aiwire.StringList{"END"}, - Tools: []aiwire.Tool{{Type: "function", Function: aiwire.FunctionDef{Name: "get_weather", Parameters: json.RawMessage(`{"type":"object"}`)}}}, - ToolChoice: json.RawMessage(`"auto"`), - } - req := irToSDK(ir, true) - if len(req.Messages) != 4 { - t.Fatalf("messages = %d, want 4", len(req.Messages)) - } - if _, ok := req.Messages[0].(generativeaiinference.SystemMessage); !ok { - t.Fatalf("messages[0] = %T, want SystemMessage", req.Messages[0]) - } - am, ok := req.Messages[2].(generativeaiinference.AssistantMessage) - if !ok || len(am.ToolCalls) != 1 { - t.Fatalf("assistant tool calls 未映射: %T %+v", req.Messages[2], am) - } - tm, ok := req.Messages[3].(generativeaiinference.ToolMessage) - if !ok || deref(tm.ToolCallId) != "c1" { - t.Fatalf("tool message 未映射 toolCallId: %+v", tm) - } - if req.Temperature == nil || *req.Temperature != 0.7 { - t.Fatalf("temperature 未直通") - } - if req.MaxTokens == nil || *req.MaxTokens != 100 || len(req.Stop) != 1 { - t.Fatalf("maxTokens/stop 未直通") - } - if _, ok := req.ToolChoice.(generativeaiinference.ToolChoiceAuto); !ok { - t.Fatalf("toolChoice = %T, want auto", req.ToolChoice) - } - if len(req.Tools) != 1 || req.IsStream == nil || !*req.IsStream || req.StreamOptions == nil { - t.Fatalf("tools/stream 选项未映射") - } -} - -// TestSdkToIR 断言响应文本、工具调用与用量的反向映射。 -func TestSdkToIR(t *testing.T) { - text, fr := "东京晴", "tool_calls" - idx, pt, ct, tt := 0, 10, 5, 15 - id, name, args := "c1", "get_weather", `{"city":"东京"}` - resp := generativeaiinference.GenericChatResponse{ - Choices: []generativeaiinference.ChatChoice{{ - Index: &idx, - Message: generativeaiinference.AssistantMessage{ - Content: []generativeaiinference.ChatContent{generativeaiinference.TextContent{Text: &text}}, - ToolCalls: []generativeaiinference.ToolCall{generativeaiinference.FunctionCall{Id: &id, Name: &name, Arguments: &args}}, - }, - FinishReason: &fr, - }}, - Usage: &generativeaiinference.Usage{PromptTokens: &pt, CompletionTokens: &ct, TotalTokens: &tt}, - } - out := sdkToIR(resp, "m1") - if len(out.Choices) != 1 || out.Choices[0].Message.Content.JoinText() != "东京晴" { - t.Fatalf("文本未映射: %+v", out) - } - if out.Choices[0].FinishReason != "tool_calls" || len(out.Choices[0].Message.ToolCalls) != 1 { - t.Fatalf("finishReason/toolCalls 未映射: %+v", out.Choices[0]) - } - if out.Usage == nil || out.Usage.TotalTokens != 15 { - t.Fatalf("usage 未映射: %+v", out.Usage) - } -} - -// TestParseGenAiEvent 断言流事件宽容解析:文本增量、结束原因与 usage 事件。 -func TestParseGenAiEvent(t *testing.T) { - chunk, ok := parseGenAiEvent([]byte(`{"index":0,"message":{"role":"ASSISTANT","content":[{"type":"TEXT","text":"你"}]}}`), "m1") - if !ok || len(chunk.Choices) != 1 || chunk.Choices[0].Delta.Content != "你" { - t.Fatalf("文本增量解析失败: %+v", chunk) - } - chunk, ok = parseGenAiEvent([]byte(`{"finishReason":"stop","usage":{"promptTokens":3,"completionTokens":2,"totalTokens":5}}`), "m1") - if !ok || chunk.Choices[0].FinishReason == nil || *chunk.Choices[0].FinishReason != "stop" { - t.Fatalf("finishReason 解析失败: %+v", chunk) - } - if chunk.Usage == nil || chunk.Usage.TotalTokens != 5 { - t.Fatalf("usage 解析失败: %+v", chunk.Usage) - } - if _, ok := parseGenAiEvent([]byte(`{}`), "m1"); ok { - t.Fatal("空事件应返回 ok=false") - } -} - +// TestSdkUsageCachedTokens 断言 embed 用量映射:缓存命中挂 details,仅命中 >0 时出现。 func TestSdkUsageCachedTokens(t *testing.T) { p, c, tot, cached := 10, 5, 15, 8 u := sdkUsageToIR(&generativeaiinference.Usage{PromptTokens: &p, CompletionTokens: &c, TotalTokens: &tot, PromptTokensDetails: &generativeaiinference.PromptTokensDetails{CachedTokens: &cached}}) - if u.CachedTokens() != 8 { - t.Errorf("CachedTokens = %d, want 8", u.CachedTokens()) + if u.PromptTokens != 10 || u.CompletionTokens != 5 || u.TotalTokens != 15 { + t.Fatalf("usage 映射错误: %+v", u) } - if sdkUsageToIR(&generativeaiinference.Usage{PromptTokens: &p}).CachedTokens() != 0 { - t.Error("无细分时 CachedTokens 应为 0") + if u.PromptTokensDetails == nil || u.PromptTokensDetails.CachedTokens != 8 { + t.Fatalf("cachedTokens 未映射: %+v", u.PromptTokensDetails) } -} - -func TestIrToCohereSDK(t *testing.T) { - ir := aiwire.ChatRequest{ - Model: "cohere.command-r-plus", - Messages: []aiwire.ChatMessage{ - {Role: "system", Content: aiwire.NewTextContent("你是助手")}, - {Role: "user", Content: aiwire.NewTextContent("第一问")}, - {Role: "assistant", Content: aiwire.NewTextContent("第一答")}, - {Role: "user", Content: aiwire.NewTextContent("第二问")}, - }, + zero := 0 + if got := sdkUsageToIR(&generativeaiinference.Usage{PromptTokens: &p, PromptTokensDetails: &generativeaiinference.PromptTokensDetails{CachedTokens: &zero}}); got.PromptTokensDetails != nil { + t.Fatalf("零命中不应带 details: %+v", got.PromptTokensDetails) } - req, err := irToCohereSDK(ir, true) - if err != nil { - t.Fatalf("irToCohereSDK: %v", err) - } - if *req.Message != "第二问" || *req.PreambleOverride != "你是助手" || len(req.ChatHistory) != 2 { - t.Errorf("拆装错误: msg=%q preamble=%v history=%d", *req.Message, req.PreambleOverride, len(req.ChatHistory)) - } - if _, ok := req.ChatHistory[0].(generativeaiinference.CohereUserMessage); !ok { - t.Errorf("history[0] 应为 user: %T", req.ChatHistory[0]) - } - if _, ok := req.ChatHistory[1].(generativeaiinference.CohereChatBotMessage); !ok { - t.Errorf("history[1] 应为 chatbot: %T", req.ChatHistory[1]) - } - if req.IsStream == nil || !*req.IsStream { - t.Error("IsStream 未设置") - } - if req.StreamOptions == nil || req.StreamOptions.IsIncludeUsage == nil || !*req.StreamOptions.IsIncludeUsage { - t.Error("流式应默认开启 usage 回传(StreamOptions.IsIncludeUsage)") - } - // 工具定义降级为扁平参数表(仅取 JSON Schema 顶层 properties) - ir.Tools = []aiwire.Tool{{Type: "function", Function: aiwire.FunctionDef{ - Name: "get_weather", Parameters: json.RawMessage(`{"type":"object","properties":{"city":{"type":"string","description":"城市"}},"required":["city"]}`)}}} - req2, err := irToCohereSDK(ir, false) - if err != nil { - t.Fatalf("工具请求应支持: %v", err) - } - tool := req2.Tools[0] - if *tool.Name != "get_weather" || *tool.Description != "get_weather" { - t.Errorf("tool = %+v", tool) - } - if pd, ok := tool.ParameterDefinitions["city"]; !ok || *pd.Type != "string" || pd.IsRequired == nil || !*pd.IsRequired { - t.Errorf("param city = %+v", pd) - } - // 图片输入仍拒绝 - ir.Tools = nil - ir.Messages = append(ir.Messages, aiwire.ChatMessage{Role: "user", Content: aiwire.NewPartsContent([]aiwire.ContentPart{ - {Type: "image_url", ImageURL: &aiwire.ImageURL{URL: "data:image/png;base64,xx"}}})}) - if _, err := irToCohereSDK(ir, false); err == nil { - t.Error("图片输入应被拒绝") - } - // 无 user 消息被拒 - if _, err := irToCohereSDK(aiwire.ChatRequest{Model: "cohere.x", Messages: []aiwire.ChatMessage{{Role: "system", Content: aiwire.NewTextContent("s")}}}, false); err == nil { - t.Error("无 user 消息应被拒绝") - } -} - -func TestCohereSDKToIR(t *testing.T) { - text := "回答" - resp := generativeaiinference.CohereChatResponse{ - Text: &text, - FinishReason: generativeaiinference.CohereChatResponseFinishReasonComplete, - } - out := cohereSDKToIR(resp, "cohere.command-r-plus") - if out.Choices[0].Message.Content.JoinText() != "回答" || out.Choices[0].FinishReason != "stop" { - t.Errorf("cohereSDKToIR = %+v", out.Choices[0]) - } -} - -func TestParseGenAiEventCohere(t *testing.T) { - chunk, ok := parseGenAiEvent([]byte(`{"apiFormat":"COHERE","text":"你好"}`), "cohere.command-r") - if !ok || chunk.Choices[0].Delta.Content != "你好" { - t.Errorf("cohere text 事件解析 = %+v, %v", chunk, ok) - } - chunk, ok = parseGenAiEvent([]byte(`{"apiFormat":"COHERE","finishReason":"COMPLETE","usage":{"promptTokens":3,"completionTokens":5,"totalTokens":8}}`), "cohere.command-r") - if !ok || *chunk.Choices[0].FinishReason != "stop" || chunk.Usage.TotalTokens != 8 { - t.Errorf("cohere 终帧解析 = %+v, %v", chunk, ok) + if sdkUsageToIR(nil) != nil { + t.Fatal("nil usage 应返回 nil") } } diff --git a/internal/service/aiconvert.go b/internal/service/aiconvert.go deleted file mode 100644 index cf012f7..0000000 --- a/internal/service/aiconvert.go +++ /dev/null @@ -1,332 +0,0 @@ -package service - -import ( - "encoding/json" - "fmt" - "strings" - - "oci-portal/internal/aiwire" -) - -// ErrAiUnsupportedBlock 表示请求含网关无法承接的内容块(仅支持文本与图片)。 -var ErrAiUnsupportedBlock = fmt.Errorf("暂不支持文本与图片以外的内容块") - -// AnthropicToIR 把 Anthropic Messages 请求转为 IR(OpenAI 线格式)。 -// tool_result 块拆为独立 tool 角色消息(一拆多),顶层 system 变首条 system 消息。 -func AnthropicToIR(req aiwire.MessagesRequest) (aiwire.ChatRequest, error) { - ir := aiwire.ChatRequest{ - Model: req.Model, - Temperature: req.Temperature, - TopP: req.TopP, - TopK: req.TopK, - Stop: req.StopSequences, - Stream: req.Stream, - Tools: anthTools(req.Tools), - ToolChoice: anthToolChoice(req.ToolChoice), - } - if req.MaxTokens > 0 { - mt := req.MaxTokens - ir.MaxTokens = &mt - } - if sys := req.SystemText(); sys != "" { - ir.Messages = append(ir.Messages, aiwire.ChatMessage{Role: "system", Content: aiwire.NewTextContent(sys)}) - } - for _, m := range req.Messages { - msgs, err := anthMessageToIR(m) - if err != nil { - return ir, err - } - ir.Messages = append(ir.Messages, msgs...) - } - return ir, nil -} - -// anthMessageToIR 拆解单条 Anthropic 消息;user 消息里的 tool_result 前置为独立 tool 消息, -// image 块转为 IR image_url 部件。 -func anthMessageToIR(m aiwire.AnthMessage) ([]aiwire.ChatMessage, error) { - var out []aiwire.ChatMessage - var parts []aiwire.ContentPart - var toolCalls []aiwire.ToolCall - hasImage := false - for _, b := range m.Content.AllBlocks() { - switch b.Type { - case "text": - parts = append(parts, aiwire.ContentPart{Type: "text", Text: b.Text}) - case "image": - p, err := anthImagePart(b.Source) - if err != nil { - return nil, err - } - parts, hasImage = append(parts, p), true - case "tool_use": - toolCalls = append(toolCalls, aiwire.ToolCall{ID: b.ID, Type: "function", - Function: aiwire.FunctionCall{Name: b.Name, Arguments: string(b.Input)}}) - case "tool_result": - out = append(out, aiwire.ChatMessage{Role: "tool", ToolCallID: b.ToolUseID, - Content: aiwire.NewTextContent(b.ResultText())}) - case "thinking", "redacted_thinking": - // 一期忽略 thinking 块 - default: - return nil, ErrAiUnsupportedBlock - } - } - content := partsContent(parts, hasImage) - if hasImage || content.JoinText() != "" || len(toolCalls) > 0 { - out = append(out, aiwire.ChatMessage{Role: m.Role, Content: content, ToolCalls: toolCalls}) - } - return out, nil -} - -// anthImagePart 把 Anthropic image 块转为 IR image_url 部件(base64 → data URI)。 -func anthImagePart(source json.RawMessage) (aiwire.ContentPart, error) { - var src struct { - Type string `json:"type"` - MediaType string `json:"media_type"` - Data string `json:"data"` - URL string `json:"url"` - } - if err := json.Unmarshal(source, &src); err != nil { - return aiwire.ContentPart{}, fmt.Errorf("image source 解析失败: %w", err) - } - switch src.Type { - case "base64": - if src.MediaType == "" || src.Data == "" { - return aiwire.ContentPart{}, fmt.Errorf("image source 缺少 media_type 或 data") - } - url := "data:" + src.MediaType + ";base64," + src.Data - return aiwire.ContentPart{Type: "image_url", ImageURL: &aiwire.ImageURL{URL: url}}, nil - case "url": - if src.URL == "" { - return aiwire.ContentPart{}, fmt.Errorf("image source 缺少 url") - } - return aiwire.ContentPart{Type: "image_url", ImageURL: &aiwire.ImageURL{URL: src.URL}}, nil - } - return aiwire.ContentPart{}, fmt.Errorf("不支持的 image source 类型 %q", src.Type) -} - -// partsContent 无图片时退回字符串形态(与上游线格式习惯一致),含图片时保留块数组。 -func partsContent(parts []aiwire.ContentPart, hasImage bool) aiwire.Content { - if !hasImage { - var sb strings.Builder - for _, p := range parts { - sb.WriteString(p.Text) - } - return aiwire.NewTextContent(sb.String()) - } - return aiwire.NewPartsContent(parts) -} - -func anthTools(tools []aiwire.AnthTool) []aiwire.Tool { - if len(tools) == 0 { - return nil - } - out := make([]aiwire.Tool, 0, len(tools)) - for _, t := range tools { - out = append(out, aiwire.Tool{Type: "function", Function: aiwire.FunctionDef{ - Name: t.Name, Description: t.Description, Parameters: t.InputSchema}}) - } - return out -} - -// anthToolChoice 映射 {type:auto|any|tool,name} → OpenAI 形态。 -func anthToolChoice(raw json.RawMessage) json.RawMessage { - if len(raw) == 0 { - return nil - } - var tc struct { - Type string `json:"type"` - Name string `json:"name"` - } - if json.Unmarshal(raw, &tc) != nil { - return nil - } - switch tc.Type { - case "auto": - return json.RawMessage(`"auto"`) - case "any": - return json.RawMessage(`"required"`) - case "tool": - b, _ := json.Marshal(map[string]any{"type": "function", "function": map[string]string{"name": tc.Name}}) - return b - case "none": - return json.RawMessage(`"none"`) - } - return nil -} - -// IRRespToAnthropic 把 IR 非流式响应转为 Anthropic Messages 响应。 -func IRRespToAnthropic(resp *aiwire.ChatResponse, id string) aiwire.MessagesResponse { - out := aiwire.MessagesResponse{ID: id, Type: "message", Role: "assistant", Model: resp.Model, Content: []aiwire.AnthBlock{}} - if len(resp.Choices) > 0 { - choice := resp.Choices[0] - if text := choice.Message.Content.JoinText(); text != "" { - out.Content = append(out.Content, aiwire.AnthBlock{Type: "text", Text: text}) - } - for _, tc := range choice.Message.ToolCalls { - out.Content = append(out.Content, aiwire.AnthBlock{Type: "tool_use", ID: tc.ID, - Name: tc.Function.Name, Input: argsToJSON(tc.Function.Arguments)}) - } - out.StopReason = anthStopReason(choice.FinishReason) - } - if resp.Usage != nil { - out.Usage = aiwire.AnthUsage{InputTokens: resp.Usage.PromptTokens, OutputTokens: resp.Usage.CompletionTokens, - CacheReadInputTokens: resp.Usage.CachedTokens()} - } - return out -} - -// argsToJSON 保证 tool_use.input 是合法 JSON 对象(模型可能产出非法片段)。 -func argsToJSON(args string) json.RawMessage { - trimmed := strings.TrimSpace(args) - if trimmed == "" { - return json.RawMessage(`{}`) - } - if json.Valid([]byte(trimmed)) { - return json.RawMessage(trimmed) - } - b, _ := json.Marshal(map[string]string{"_raw": args}) - return b -} - -func anthStopReason(finish string) string { - switch finish { - case "length": - return "max_tokens" - case "tool_calls": - return "tool_use" - default: - return "end_turn" - } -} - -// ---- Anthropic 流式状态机 ---- - -// AnthEvent 是一条待写出的 Anthropic SSE 事件。 -type AnthEvent struct { - Event string - Data any -} - -// AnthStream 把 IR chunk 流聚合为 Anthropic 事件序列: -// message_start → content_block_start/delta/stop(text 与 tool_use 分块)→ message_delta → message_stop。 -type AnthStream struct { - id, model string - started bool - blockOpen bool - blockIsTool bool - toolID string - blockIndex int - stopReason string - usage aiwire.AnthUsage -} - -// NewAnthStream 构造状态机;id 为响应消息 ID。 -func NewAnthStream(id, model string) *AnthStream { - return &AnthStream{id: id, model: model, blockIndex: -1, stopReason: "end_turn"} -} - -// Feed 消费一个 IR chunk,返回应立即写出的事件。 -func (st *AnthStream) Feed(chunk aiwire.ChatChunk) []AnthEvent { - var events []AnthEvent - if !st.started { - st.started = true - events = append(events, st.startEvent()) - } - if chunk.Usage != nil { - st.usage = aiwire.AnthUsage{InputTokens: chunk.Usage.PromptTokens, OutputTokens: chunk.Usage.CompletionTokens, - CacheReadInputTokens: chunk.Usage.CachedTokens()} - } - for _, choice := range chunk.Choices { - events = append(events, st.feedDelta(choice.Delta)...) - if choice.FinishReason != nil && *choice.FinishReason != "" { - st.stopReason = anthStopReason(*choice.FinishReason) - } - } - return events -} - -func (st *AnthStream) startEvent() AnthEvent { - return AnthEvent{Event: "message_start", Data: map[string]any{ - "type": "message_start", - "message": map[string]any{ - "id": st.id, "type": "message", "role": "assistant", "model": st.model, - "content": []any{}, "stop_reason": nil, - "usage": map[string]int{"input_tokens": 0, "output_tokens": 0}, - }, - }} -} - -// feedDelta 处理文本与工具调用增量,必要时切块。 -func (st *AnthStream) feedDelta(d aiwire.Delta) []AnthEvent { - var events []AnthEvent - if d.Content != "" { - if !st.blockOpen || st.blockIsTool { - events = append(events, st.openBlock(false, "", "")...) - } - events = append(events, AnthEvent{Event: "content_block_delta", Data: map[string]any{ - "type": "content_block_delta", "index": st.blockIndex, - "delta": map[string]string{"type": "text_delta", "text": d.Content}, - }}) - } - for _, tc := range d.ToolCalls { - if tc.ID != "" && (!st.blockOpen || !st.blockIsTool || st.toolID != tc.ID) { - events = append(events, st.openBlock(true, tc.ID, tc.Function.Name)...) - } - if tc.Function.Arguments != "" && st.blockOpen && st.blockIsTool { - events = append(events, AnthEvent{Event: "content_block_delta", Data: map[string]any{ - "type": "content_block_delta", "index": st.blockIndex, - "delta": map[string]string{"type": "input_json_delta", "partial_json": tc.Function.Arguments}, - }}) - } - } - return events -} - -// openBlock 关闭当前块并打开新块(text 或 tool_use)。 -func (st *AnthStream) openBlock(isTool bool, toolID, toolName string) []AnthEvent { - var events []AnthEvent - if st.blockOpen { - events = append(events, st.closeBlockEvent()) - } - st.blockOpen, st.blockIsTool, st.toolID = true, isTool, toolID - st.blockIndex++ - block := map[string]any{"type": "text", "text": ""} - if isTool { - block = map[string]any{"type": "tool_use", "id": toolID, "name": toolName, "input": map[string]any{}} - } - events = append(events, AnthEvent{Event: "content_block_start", Data: map[string]any{ - "type": "content_block_start", "index": st.blockIndex, "content_block": block, - }}) - return events -} - -func (st *AnthStream) closeBlockEvent() AnthEvent { - return AnthEvent{Event: "content_block_stop", Data: map[string]any{ - "type": "content_block_stop", "index": st.blockIndex, - }} -} - -// Finish 在上游流结束后收尾:关块 → message_delta(stop_reason+usage)→ message_stop。 -func (st *AnthStream) Finish() []AnthEvent { - var events []AnthEvent - if !st.started { - events = append(events, st.startEvent()) - st.started = true - } - if st.blockOpen { - events = append(events, st.closeBlockEvent()) - st.blockOpen = false - } - events = append(events, - AnthEvent{Event: "message_delta", Data: map[string]any{ - "type": "message_delta", - "delta": map[string]any{"stop_reason": st.stopReason, "stop_sequence": nil}, - "usage": map[string]int{"output_tokens": st.usage.OutputTokens}, - }}, - AnthEvent{Event: "message_stop", Data: map[string]any{"type": "message_stop"}}, - ) - return events -} - -// Usage 返回聚合到的用量(供调用日志)。 -func (st *AnthStream) Usage() aiwire.AnthUsage { return st.usage } diff --git a/internal/service/aiconvert_test.go b/internal/service/aiconvert_test.go deleted file mode 100644 index 540028c..0000000 --- a/internal/service/aiconvert_test.go +++ /dev/null @@ -1,136 +0,0 @@ -package service - -import ( - "encoding/json" - "strings" - "testing" - - "oci-portal/internal/aiwire" -) - -func mustAnthReq(t *testing.T, body string) aiwire.MessagesRequest { - t.Helper() - var req aiwire.MessagesRequest - if err := json.Unmarshal([]byte(body), &req); err != nil { - t.Fatalf("unmarshal anthropic request: %v", err) - } - return req -} - -func TestAnthropicToIR(t *testing.T) { - req := mustAnthReq(t, `{ - "model": "meta.llama-3.3-70b-instruct", "max_tokens": 128, "system": "你是助手", - "messages": [ - {"role": "user", "content": "东京天气?"}, - {"role": "assistant", "content": [ - {"type": "text", "text": "查询中"}, - {"type": "tool_use", "id": "t1", "name": "get_weather", "input": {"city": "东京"}} - ]}, - {"role": "user", "content": [ - {"type": "tool_result", "tool_use_id": "t1", "content": "晴 25 度"}, - {"type": "text", "text": "继续"} - ]} - ], - "tools": [{"name": "get_weather", "input_schema": {"type": "object"}}], - "tool_choice": {"type": "any"} - }`) - ir, err := AnthropicToIR(req) - if err != nil { - t.Fatalf("AnthropicToIR: %v", err) - } - roles := make([]string, 0, len(ir.Messages)) - for _, m := range ir.Messages { - roles = append(roles, m.Role) - } - want := []string{"system", "user", "assistant", "tool", "user"} - if strings.Join(roles, ",") != strings.Join(want, ",") { - t.Fatalf("roles = %v, want %v", roles, want) - } - if ir.Messages[2].ToolCalls[0].Function.Name != "get_weather" { - t.Errorf("tool_use 未映射: %+v", ir.Messages[2]) - } - if ir.Messages[3].ToolCallID != "t1" || ir.Messages[3].Content.JoinText() != "晴 25 度" { - t.Errorf("tool_result 未拆为 tool 消息: %+v", ir.Messages[3]) - } - if ir.MaxTokens == nil || *ir.MaxTokens != 128 || len(ir.Tools) != 1 { - t.Errorf("max_tokens/tools 未直通") - } - if string(ir.ToolChoice) != `"required"` { - t.Errorf("tool_choice any → %s, want required", ir.ToolChoice) - } - // image 块拒绝 - bad := mustAnthReq(t, `{"model":"m","max_tokens":1,"messages":[{"role":"user","content":[{"type":"image","source":{}}]}]}`) - if _, err := AnthropicToIR(bad); err == nil { - t.Error("image 块应被拒绝") - } -} - -func TestIRRespToAnthropic(t *testing.T) { - resp := &aiwire.ChatResponse{ - Model: "m1", - Choices: []aiwire.Choice{{ - Message: aiwire.ChatMessage{ - Role: "assistant", - Content: aiwire.NewTextContent("查到了"), - ToolCalls: []aiwire.ToolCall{{ID: "t1", Type: "function", - Function: aiwire.FunctionCall{Name: "get_weather", Arguments: `{"city":"东京"}`}}}, - }, - FinishReason: "tool_calls", - }}, - Usage: &aiwire.Usage{PromptTokens: 10, CompletionTokens: 5}, - } - out := IRRespToAnthropic(resp, "msg_1") - if len(out.Content) != 2 || out.Content[0].Type != "text" || out.Content[1].Type != "tool_use" { - t.Fatalf("content 块 = %+v", out.Content) - } - if out.StopReason != "tool_use" || out.Usage.InputTokens != 10 || out.Usage.OutputTokens != 5 { - t.Errorf("stop_reason/usage 未映射: %+v", out) - } - if string(out.Content[1].Input) != `{"city":"东京"}` { - t.Errorf("tool input = %s", out.Content[1].Input) - } -} - -// eventTypes 提取事件类型序列便于断言。 -func eventTypes(events []AnthEvent) string { - types := make([]string, 0, len(events)) - for _, e := range events { - types = append(types, e.Event) - } - return strings.Join(types, ",") -} - -func TestAnthStreamTextAndTool(t *testing.T) { - st := NewAnthStream("msg_1", "m1") - fr := "tool_calls" - var events []AnthEvent - events = append(events, st.Feed(aiwire.ChatChunk{Choices: []aiwire.ChunkChoice{{Delta: aiwire.Delta{Content: "你"}}}})...) - events = append(events, st.Feed(aiwire.ChatChunk{Choices: []aiwire.ChunkChoice{{Delta: aiwire.Delta{Content: "好"}}}})...) - events = append(events, st.Feed(aiwire.ChatChunk{Choices: []aiwire.ChunkChoice{{Delta: aiwire.Delta{ - ToolCalls: []aiwire.ToolCallDelta{{ID: "t1", Function: aiwire.FunctionCallDelta{Name: "get_weather", Arguments: `{"ci`}}}, - }}}})...) - events = append(events, st.Feed(aiwire.ChatChunk{ - Choices: []aiwire.ChunkChoice{{Delta: aiwire.Delta{ToolCalls: []aiwire.ToolCallDelta{{ID: "t1", Function: aiwire.FunctionCallDelta{Arguments: `ty":"东京"}`}}}}, FinishReason: &fr}}, - Usage: &aiwire.Usage{PromptTokens: 8, CompletionTokens: 4}, - })...) - events = append(events, st.Finish()...) - - got := eventTypes(events) - want := "message_start,content_block_start,content_block_delta,content_block_delta," + - "content_block_stop,content_block_start,content_block_delta,content_block_delta," + - "content_block_stop,message_delta,message_stop" - if got != want { - t.Fatalf("事件序列:\n got %s\nwant %s", got, want) - } - if st.Usage().OutputTokens != 4 { - t.Errorf("usage 未聚合: %+v", st.Usage()) - } -} - -func TestAnthStreamEmpty(t *testing.T) { - st := NewAnthStream("msg_1", "m1") - events := st.Finish() - if got := eventTypes(events); got != "message_start,message_delta,message_stop" { - t.Errorf("空流事件序列 = %s", got) - } -} diff --git a/internal/service/aigateway.go b/internal/service/aigateway.go index 6ea075c..4762a36 100644 --- a/internal/service/aigateway.go +++ b/internal/service/aigateway.go @@ -5,6 +5,7 @@ import ( "crypto/rand" "crypto/sha256" "encoding/hex" + "encoding/json" "errors" "fmt" "log" @@ -30,10 +31,6 @@ const ( aiBackoffCap = 30 * time.Minute // aiKeyTouchGap 是 LastUsedAt 的最小写库间隔,避免高频调用刷库 aiKeyTouchGap = time.Minute - // modelRecheckGap 是已标记不可用模型的复检间隔(恢复供给自动解除标记); - // validateBatchCap 限制单渠道单轮验证的试调次数 - modelRecheckGap = 20 * time.Hour - validateBatchCap = 32 // 内容日志(红线例外)约束:开启必须限时(上限 7 天),正文截断,短保留 aiContentLogMaxHours = 168 aiContentLogRetention = 7 * 24 * time.Hour @@ -59,17 +56,13 @@ type AiGatewayService struct { // touchMu 保护各密钥的最近触达时间(内存节流,不追求跨实例精确) touchMu sync.Mutex lastTouch map[uint]time.Time - // validateMu 保护 validating:同一渠道的模型验证不并发 - validateMu sync.Mutex - validating map[uint]bool // onChannelsChanged 在渠道增删后触发,由 main 装配为探测任务同步钩子 onChannelsChanged func(context.Context) } // NewAiGatewayService 组装依赖;调用 StartCleanup 后开始调用日志周期清理。 func NewAiGatewayService(db *gorm.DB, configs *OciConfigService, client oci.Client) *AiGatewayService { - return &AiGatewayService{db: db, configs: configs, client: client, - lastTouch: map[uint]time.Time{}, validating: map[uint]bool{}} + return &AiGatewayService{db: db, configs: configs, client: client, lastTouch: map[uint]time.Time{}} } // SetOnChannelsChanged 注册渠道数量变化钩子(渠道创建/删除成功后调用)。 @@ -87,7 +80,7 @@ func (s *AiGatewayService) fireChannelsChanged(ctx context.Context) { // CreateKey 生成网关密钥并返回明文(仅此一次);customValue 非空时使用给定值, // group 非空时该密钥只在同分组渠道内路由。 -func (s *AiGatewayService) CreateKey(ctx context.Context, name, customValue, group string) (string, *model.AiKey, error) { +func (s *AiGatewayService) CreateKey(ctx context.Context, name, customValue, group string, models []string) (string, *model.AiKey, error) { name = strings.TrimSpace(name) if name == "" { return "", nil, fmt.Errorf("密钥名称不能为空") @@ -103,13 +96,28 @@ func (s *AiGatewayService) CreateKey(ctx context.Context, name, customValue, gro if len(raw) < 8 { return "", nil, fmt.Errorf("自定义密钥至少 8 个字符") } - key := &model.AiKey{Name: name, KeyHash: hashKey(raw), Tail: raw[len(raw)-4:], Group: strings.TrimSpace(group), Enabled: true} + key := &model.AiKey{Name: name, KeyHash: hashKey(raw), Tail: raw[len(raw)-4:], Group: strings.TrimSpace(group), Models: normalizeKeyModels(models), Enabled: true} if err := s.db.WithContext(ctx).Create(key).Error; err != nil { return "", nil, fmt.Errorf("密钥名称或取值与现有密钥重复") } return raw, key, nil } +// normalizeKeyModels 规范化模型白名单:trim、剔空串、保序去重;结果为空返回 nil(= 不限)。 +func normalizeKeyModels(models []string) []string { + var out []string + seen := map[string]bool{} + for _, m := range models { + m = strings.TrimSpace(m) + if m == "" || seen[m] { + continue + } + seen[m] = true + out = append(out, m) + } + return out +} + func hashKey(raw string) string { sum := sha256.Sum256([]byte(raw)) return hex.EncodeToString(sum[:]) @@ -122,8 +130,8 @@ func (s *AiGatewayService) Keys(ctx context.Context) ([]model.AiKey, error) { return keys, err } -// UpdateKey 修改密钥名称 / 启用状态 / 分组(group 指针非空即覆盖,可置空)。 -func (s *AiGatewayService) UpdateKey(ctx context.Context, id uint, name string, enabled *bool, group *string) error { +// UpdateKey 修改密钥名称 / 启用状态 / 分组 / 模型白名单(指针非空即覆盖,可置空)。 +func (s *AiGatewayService) UpdateKey(ctx context.Context, id uint, name string, enabled *bool, group *string, models *[]string) error { updates := map[string]any{} if name = strings.TrimSpace(name); name != "" { updates["name"] = name @@ -134,6 +142,14 @@ func (s *AiGatewayService) UpdateKey(ctx context.Context, id uint, name string, if group != nil { updates["key_group"] = strings.TrimSpace(*group) } + if models != nil { + // 手动序列化走 map 更新,不依赖 GORM map 路径对 serializer 的支持 + b, err := json.Marshal(normalizeKeyModels(*models)) + if err != nil { + return fmt.Errorf("serialize models: %w", err) + } + updates["models"] = string(b) + } if len(updates) == 0 { return nil } @@ -298,12 +314,17 @@ func (s *AiGatewayService) ProbeChannel(ctx context.Context, id uint) (*model.Ai return &fresh, nil } -// probe 执行探测并返回 (状态, 错误摘要);同时完成模型缓存同步(保留不可用标记)。 +// probe 执行探测并返回 (状态, 错误摘要);同时完成模型缓存同步(黑名单模型不入库)。 func (s *AiGatewayService) probe(ctx context.Context, cred oci.Credentials, ch *model.AiChannel) (string, string) { models, err := s.client.ListGenAiModels(ctx, cred, ch.Region) if err != nil { return classifyProbeErr(err), truncateErr(oci.CompactError(err)) } + models = supportedGatewayModels(models) + models, err = s.withoutBlacklisted(ctx, models) + if err != nil { + return "error", truncateErr(err.Error()) + } if len(models) == 0 { _ = s.replaceModels(ctx, ch.ID, nil) return "no_service", "区域无可用模型(GenAI 服务不可用或未开放)" @@ -311,31 +332,12 @@ func (s *AiGatewayService) probe(ctx context.Context, cred oci.Credentials, ch * if err := s.replaceModels(ctx, ch.ID, models); err != nil { return "error", truncateErr(err.Error()) } - usable, err := s.usableModels(ctx, ch.ID, models) - if err != nil { - return "error", truncateErr(err.Error()) - } - return s.probeChat(ctx, cred, ch, usable) + return s.probeChat(ctx, cred, ch, models) } -// usableModels 过滤掉缓存中已标记不可按需调用的模型,探测候选不再反复踩坑。 -func (s *AiGatewayService) usableModels(ctx context.Context, channelID uint, models []oci.GenAiModel) ([]oci.GenAiModel, error) { - marks, err := s.loadModelMarks(ctx, channelID) - if err != nil { - return nil, err - } - out := make([]oci.GenAiModel, 0, len(models)) - for _, m := range models { - if !marks[m.Ocid].Unusable { - out = append(out, m) - } - } - return out, nil -} - -// probeChat 按候选顺序试调(上限 8):遇「模型不可按需调用」标记剔除并换下一个; -// 401/403 与鉴权类 404 属租户级直接定论 no_quota;其余错误(部分模型元数据标 CHAT -// 但实际不可对话,如 voice agent)累计 3 次止损。 +// probeChat 按候选顺序试调(上限 8):遇「模型不可按需调用」(微调基座 400 / 实体 +// 不存在 404)换下一个候选,错误信息带模型名供用户加入黑名单;401/403 与鉴权类 404 +// 属租户级直接定论 no_quota;其余错误(元数据标 CHAT 但实际不可对话等)累计 3 次止损。 func (s *AiGatewayService) probeChat(ctx context.Context, cred oci.Credentials, ch *model.AiChannel, models []oci.GenAiModel) (string, string) { status, detail := "error", "无可试调对话模型" errBudget := 3 @@ -345,8 +347,7 @@ func (s *AiGatewayService) probeChat(ctx context.Context, cred oci.Credentials, case code == 200 || code == 429: return "ok", "" case oci.IsModelUnavailable(err): - s.markModelUnusable(ctx, ch.ID, m.Ocid, oci.CompactError(err)) - status, detail = "error", truncateErr(fmt.Sprintf("%s: 不可按需调用,已从模型池剔除", m.Name)) + status, detail = "error", truncateErr(fmt.Sprintf("%s: 不可按需调用,建议加入模型黑名单", m.Name)) case code == 401 || code == 403 || code == 404: return "no_quota", truncateErr(oci.CompactError(err)) default: @@ -359,106 +360,8 @@ func (s *AiGatewayService) probeChat(ctx context.Context, cred oci.Credentials, return status, detail } -// markModelUnusable 把 (渠道, 模型OCID) 标记为不可按需调用;失败仅记日志不阻断主流程。 -func (s *AiGatewayService) markModelUnusable(ctx context.Context, channelID uint, ocid, reason string) { - err := s.db.WithContext(ctx).Model(&model.AiModelCache{}). - Where("channel_id = ? AND model_ocid = ?", channelID, ocid). - Updates(map[string]any{"unusable": true, "unusable_reason": shortReason(reason), "checked_at": time.Now()}).Error - if err != nil { - log.Printf("mark model unusable: %v", err) - } -} - -func shortReason(reason string) string { - if len(reason) > 200 { - return reason[:200] - } - return reason -} - -// beginValidate 抢占渠道的验证执行权,同渠道同时只跑一轮。 -func (s *AiGatewayService) beginValidate(id uint) bool { - s.validateMu.Lock() - defer s.validateMu.Unlock() - if s.validating[id] { - return false - } - s.validating[id] = true - return true -} - -func (s *AiGatewayService) endValidate(id uint) { - s.validateMu.Lock() - delete(s.validating, id) - s.validateMu.Unlock() -} - -// validateTargets 取待验证行:从未验证的新模型,以及标记超过复检间隔的模型(仅对话能力, -// 兼容存量空串);单轮上限 validateBatchCap 控制调用量。 -func (s *AiGatewayService) validateTargets(ctx context.Context, channelID uint) ([]model.AiModelCache, error) { - stale := time.Now().Add(-modelRecheckGap) - var rows []model.AiModelCache - err := s.db.WithContext(ctx). - Where("channel_id = ? AND capability IN ?", channelID, []string{"CHAT", ""}). - Where("checked_at IS NULL OR (unusable = ? AND checked_at < ?)", true, stale). - Order("id ASC").Limit(validateBatchCap).Find(&rows).Error - return rows, err -} - -// validateChannelModels 逐个试调渠道内待验证模型并落结论,把不可按需调用的模型 -// 从池中剔除、把恢复供给的解除标记;ListModels 无字段可事先判别,只能试调习得。 -func (s *AiGatewayService) validateChannelModels(ctx context.Context, channelID uint) { - if !s.beginValidate(channelID) { - return - } - defer s.endValidate(channelID) - var ch model.AiChannel - if err := s.db.WithContext(ctx).First(&ch, channelID).Error; err != nil { - return - } - cred, err := s.configs.credentialsByID(ctx, ch.OciConfigID) - if err != nil { - return - } - rows, err := s.validateTargets(ctx, channelID) - if err != nil { - log.Printf("validate channel %d models: %v", channelID, err) - return - } - for _, r := range rows { - s.validateOne(ctx, cred, &ch, r) - } -} - -// validateOne 试调单个模型并落库:可用(200/429)清除标记;模型级不可用标记剔除; -// 其余 4xx 记录已检不改状态;5xx/网络类瞬态不落 checked_at,下轮再验。 -func (s *AiGatewayService) validateOne(ctx context.Context, cred oci.Credentials, ch *model.AiChannel, r model.AiModelCache) { - code, err := s.client.GenAiProbeChat(ctx, cred, ch.Region, r.ModelOcid, r.Name) - updates := map[string]any{"checked_at": time.Now()} - switch { - case code == 200 || code == 429: - updates["unusable"], updates["unusable_reason"] = false, "" - case oci.IsModelUnavailable(err): - updates["unusable"], updates["unusable_reason"] = true, shortReason(oci.CompactError(err)) - case code == 0 || code >= 500: - return - } - s.db.WithContext(ctx).Model(&model.AiModelCache{}).Where("id = ?", r.ID).Updates(updates) -} - -// validateModelsAsync 后台验证渠道模型池(手动同步后触发,不阻塞接口响应)。 -func (s *AiGatewayService) validateModelsAsync(ctx context.Context, channelID uint) { - s.wg.Add(1) - go func() { - defer s.wg.Done() - vctx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 3*time.Minute) - defer cancel() - s.validateChannelModels(vctx, channelID) - }() -} - -// probeCandidateCap 是单次探测的候选上限:模型级不可用会被标记剔除不再重试, -// 放宽到 8 让一次探测有机会越过整批坏模型找到可用者;其他错误另有 3 次止损预算。 +// probeCandidateCap 是单次探测的候选上限:放宽到 8 让一次探测有机会越过整批 +// 不可按需调用的坏模型找到可用者;其他错误另有 3 次止损预算。 const probeCandidateCap = 8 // probeCandidates 只取对话模型,按可靠度排序后跨厂商取候选:主流文本模型优先; @@ -551,8 +454,7 @@ func truncateErr(msg string) string { return msg } -// SyncModels 重新拉取渠道区域的模型列表并覆盖缓存(标记按 OCID 结转), -// 随后触发后台验证:新模型逐个试调,不可按需调用的数十秒内从池中剔除。 +// SyncModels 重新拉取渠道区域的模型列表并覆盖缓存,黑名单中的模型不入库。 func (s *AiGatewayService) SyncModels(ctx context.Context, id uint) ([]model.AiModelCache, error) { var ch model.AiChannel if err := s.db.WithContext(ctx).First(&ch, id).Error; err != nil { @@ -566,29 +468,23 @@ func (s *AiGatewayService) SyncModels(ctx context.Context, id uint) ([]model.AiM if err != nil { return nil, fmt.Errorf("同步模型失败:%s", oci.CompactError(err)) } + models = supportedGatewayModels(models) + if models, err = s.withoutBlacklisted(ctx, models); err != nil { + return nil, err + } if err := s.replaceModels(ctx, id, models); err != nil { return nil, err } - s.validateModelsAsync(ctx, id) return s.channelModels(ctx, id) } -// replaceModels 以事务整组覆盖渠道模型缓存,按 OCID 结转不可用标记与验证时间 -// (OCID 变化视为新条目,自然回到待验证状态)。 +// replaceModels 以事务整组覆盖渠道模型缓存。 func (s *AiGatewayService) replaceModels(ctx context.Context, channelID uint, models []oci.GenAiModel) error { - marks, err := s.loadModelMarks(ctx, channelID) - if err != nil { - return err - } rows := make([]model.AiModelCache, 0, len(models)) now := time.Now() for _, m := range models { - row := model.AiModelCache{ChannelID: channelID, ModelOcid: m.Ocid, Name: m.Name, Vendor: m.Vendor, - Capability: m.Capability, SyncedAt: now, DeprecatedAt: m.Deprecated, RetiredAt: m.Retired} - if prev, ok := marks[m.Ocid]; ok { - row.Unusable, row.UnusableReason, row.CheckedAt = prev.Unusable, prev.UnusableReason, prev.CheckedAt - } - rows = append(rows, row) + rows = append(rows, model.AiModelCache{ChannelID: channelID, ModelOcid: m.Ocid, Name: m.Name, Vendor: m.Vendor, + Capability: m.Capability, SyncedAt: now, DeprecatedAt: m.Deprecated, RetiredAt: m.Retired}) } return s.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error { if err := tx.Where("channel_id = ?", channelID).Delete(&model.AiModelCache{}).Error; err != nil { @@ -601,33 +497,17 @@ func (s *AiGatewayService) replaceModels(ctx context.Context, channelID uint, mo }) } -// loadModelMarks 取渠道内带标记或已验证过的行(OCID → 行),供同步结转与候选过滤。 -func (s *AiGatewayService) loadModelMarks(ctx context.Context, channelID uint) (map[string]model.AiModelCache, error) { - var rows []model.AiModelCache - err := s.db.WithContext(ctx).Select("model_ocid", "unusable", "unusable_reason", "checked_at"). - Where("channel_id = ? AND (unusable = ? OR checked_at IS NOT NULL)", channelID, true).Find(&rows).Error - if err != nil { - return nil, fmt.Errorf("load model marks: %w", err) - } - marks := make(map[string]model.AiModelCache, len(rows)) - for _, r := range rows { - marks[r.ModelOcid] = r - } - return marks, nil -} - func (s *AiGatewayService) channelModels(ctx context.Context, channelID uint) ([]model.AiModelCache, error) { var rows []model.AiModelCache err := s.db.WithContext(ctx).Where("channel_id = ?", channelID).Order("name ASC").Find(&rows).Error return rows, err } -// GatewayModels 聚合启用渠道的可用模型(按名称去重,剔除不可按需调用标记), -// 供 /ai/v1/models;group 非空时仅聚合该分组渠道(与密钥分组路由口径一致)。 +// GatewayModels 聚合启用渠道的可用模型(按名称去重),供 /ai/v1/models; +// group 非空时仅聚合该分组渠道(与密钥分组路由口径一致)。 func (s *AiGatewayService) GatewayModels(ctx context.Context, group string) (aiwire.ModelList, error) { q := s.db.WithContext(ctx). - Joins("JOIN ai_channels ON ai_channels.id = ai_model_caches.channel_id AND ai_channels.enabled = ?", true). - Where("(ai_model_caches.unusable = ? OR ai_model_caches.unusable IS NULL)", false) + Joins("JOIN ai_channels ON ai_channels.id = ai_model_caches.channel_id AND ai_channels.enabled = ?", true) if group != "" { q = q.Where("ai_channels.channel_group = ?", group) } @@ -658,7 +538,6 @@ func (s *AiGatewayService) DeprecatingModels(ctx context.Context, within time.Du err := s.db.WithContext(ctx). Where("(retired_at IS NOT NULL AND retired_at > ? AND retired_at <= ?) OR (deprecated_at IS NOT NULL AND deprecated_at >= ? AND deprecated_at <= ?)", now, deadline, now, deadline). - Where("(unusable = ? OR unusable IS NULL)", false). Order("name ASC").Find(&rows).Error if err != nil { return nil, err @@ -679,7 +558,7 @@ func (s *AiGatewayService) DeprecatingModels(ctx context.Context, within time.Du return out, nil } -// ProbeAll 逐个探测全部渠道并顺带验证模型池,返回状态汇总;供 AI 探测后台任务调用。 +// ProbeAll 逐个探测全部渠道,返回状态汇总;供 AI 探测后台任务调用。 func (s *AiGatewayService) ProbeAll(ctx context.Context) (string, error) { chs, err := s.Channels(ctx) if err != nil { @@ -695,8 +574,6 @@ func (s *AiGatewayService) ProbeAll(ctx context.Context) (string, error) { continue } counts[fresh.ProbeStatus]++ - // 后台任务里同步执行:新模型验证入池、坏模型定期复检,零人工收敛 - s.validateChannelModels(ctx, ch.ID) } msg := fmt.Sprintf("probed %d: %d ok, %d no_service, %d no_quota, %d error", len(chs), counts["ok"], counts["no_service"], counts["no_quota"], counts["error"]) @@ -706,6 +583,99 @@ func (s *AiGatewayService) ProbeAll(ctx context.Context) (string, error) { return msg, nil } +// ---- 模型黑名单 ---- + +// Blacklist 列出全部黑名单模型(按名称排序)。 +func (s *AiGatewayService) Blacklist(ctx context.Context) ([]model.AiModelBlacklist, error) { + var rows []model.AiModelBlacklist + err := s.db.WithContext(ctx).Order("name ASC").Find(&rows).Error + return rows, err +} + +// AddBlacklist 把模型名加入黑名单并删除全部渠道缓存中的同名条目; +// 该模型此后同步 / 探测均被过滤,直到移出黑名单后重新同步。 +func (s *AiGatewayService) AddBlacklist(ctx context.Context, name string) (*model.AiModelBlacklist, error) { + name = strings.TrimSpace(name) + if name == "" { + return nil, fmt.Errorf("模型名不能为空") + } + row := model.AiModelBlacklist{Name: name} + err := s.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error { + var n int64 + if err := tx.Model(&model.AiModelBlacklist{}).Where("name = ?", name).Count(&n).Error; err != nil { + return err + } + if n > 0 { + return fmt.Errorf("模型已在黑名单中") + } + if err := tx.Create(&row).Error; err != nil { + return err + } + return tx.Where("name = ?", name).Delete(&model.AiModelCache{}).Error + }) + if err != nil { + return nil, err + } + return &row, nil +} + +// RemoveBlacklist 把模型移出黑名单;缓存不回填,下次同步 / 探测自然恢复入池。 +func (s *AiGatewayService) RemoveBlacklist(ctx context.Context, id uint) error { + res := s.db.WithContext(ctx).Delete(&model.AiModelBlacklist{}, id) + if res.Error != nil { + return res.Error + } + if res.RowsAffected == 0 { + return fmt.Errorf("黑名单条目不存在") + } + return nil +} + +// withoutBlacklisted 过滤掉黑名单中的模型,同步与探测入库前统一经此收口。 +// supportedGatewayModels 过滤模型目录:对话模型仅保留实测支持 OpenAI 兼容面的 +// 厂商(xai / meta / openai)——typed chat 面已剔除,google / cohere 对话模型无 +// 上游通路,不入目录(不出现在列表、路由与探测候选);EMBEDDING 模型不受影响。 +func supportedGatewayModels(models []oci.GenAiModel) []oci.GenAiModel { + out := make([]oci.GenAiModel, 0, len(models)) + for _, m := range models { + if m.Capability == "CHAT" && !compatChatVendor(m.Name) { + continue + } + out = append(out, m) + } + return out +} + +func compatChatVendor(model string) bool { + for _, prefix := range []string{"xai.", "meta.", "openai."} { + if strings.HasPrefix(model, prefix) { + return true + } + } + return false +} + +func (s *AiGatewayService) withoutBlacklisted(ctx context.Context, models []oci.GenAiModel) ([]oci.GenAiModel, error) { + var names []string + if err := s.db.WithContext(ctx).Model(&model.AiModelBlacklist{}).Pluck("name", &names).Error; err != nil { + return nil, fmt.Errorf("读取模型黑名单: %w", err) + } + if len(names) == 0 { + return models, nil + } + banned := make(map[string]bool, len(names)) + for _, n := range names { + banned[n] = true + } + out := make([]oci.GenAiModel, 0, len(models)) + for _, m := range models { + if !banned[m.Name] { + out = append(out, m) + } + } + return out, nil +} + // ---- 调用日志 ---- // LogCall 落一条调用日志(仅元数据与用量,永不含请求 / 响应正文),返回落库 ID 供内容日志关联(失败为 0)。 diff --git a/internal/service/aigateway_chat.go b/internal/service/aigateway_chat.go index 28aea9c..1f4e738 100644 --- a/internal/service/aigateway_chat.go +++ b/internal/service/aigateway_chat.go @@ -3,6 +3,7 @@ package service import ( "context" "errors" + "io" "math/rand" "time" @@ -26,24 +27,25 @@ type aiCandidate struct { modelOcid string } -// Chat 编排非流式调用:选渠道 → 调用 → 可重试错误换渠道(整请求上限 3 次)。 -// group 非空时只在同分组渠道内路由(取自调用密钥)。 -func (s *AiGatewayService) Chat(ctx context.Context, ir aiwire.ChatRequest, group string) (*aiwire.ChatResponse, ChatMeta, error) { +// RespPassthrough 编排一次非流式直通调用:选渠道(priority→加权随机)→ 调用 → +// 可重试错误换渠道(整请求上限 3 次)并维护熔断;group 非空时只在同分组渠道内路由。 +// 上游为 OpenAI-compatible /actions/v1/responses(实测可用,无 Oracle 文档合同)。 +func (s *AiGatewayService) RespPassthrough(ctx context.Context, raw []byte, modelName, group string) ([]byte, ChatMeta, error) { meta := ChatMeta{} excluded := map[uint]bool{} var lastErr error for attempt := 0; attempt < 3; attempt++ { - cand, err := s.pick(ctx, ir.Model, group, "CHAT", excluded) + cand, err := s.pick(ctx, modelName, group, "CHAT", excluded) if err != nil { return nil, meta, firstErr(lastErr, err) } meta.ChannelID, meta.ChannelName = cand.ch.ID, cand.ch.Name - resp, err := s.callOnce(ctx, cand, ir) + payload, err := s.passthroughOnce(ctx, cand, raw) if err == nil { s.markSuccess(ctx, cand.ch.ID) - return resp, meta, nil + return payload, meta, nil } - retry, penalize := s.noteCallErr(ctx, cand, err) + retry, penalize := switchable(err) if !retry { return nil, meta, err } @@ -57,22 +59,22 @@ func (s *AiGatewayService) Chat(ctx context.Context, ir aiwire.ChatRequest, grou return nil, meta, lastErr } -func (s *AiGatewayService) callOnce(ctx context.Context, cand *aiCandidate, ir aiwire.ChatRequest) (*aiwire.ChatResponse, error) { +func (s *AiGatewayService) passthroughOnce(ctx context.Context, cand *aiCandidate, raw []byte) ([]byte, error) { cred, err := s.configs.credentialsByID(ctx, cand.ch.OciConfigID) if err != nil { return nil, err } - return s.client.GenAiChat(ctx, cred, cand.ch.Region, cand.modelOcid, ir) + return s.client.GenAiCompatResponses(ctx, cred, cand.ch.Region, raw) } -// OpenStream 编排流式调用:流建立成功后即绑定渠道,建立失败可换渠道重试。 -// group 语义与 Chat 相同。 -func (s *AiGatewayService) OpenStream(ctx context.Context, ir aiwire.ChatRequest, group string) (oci.GenAiStream, ChatMeta, error) { +// RespPassthroughStream 编排流式直通:流建立成功即绑定渠道,建立失败按 switchable +// 换渠道重试;建立后的中断不重试、不计熔断(与 OpenStream 语义一致)。 +func (s *AiGatewayService) RespPassthroughStream(ctx context.Context, raw []byte, modelName, group string) (io.ReadCloser, ChatMeta, error) { meta := ChatMeta{} excluded := map[uint]bool{} var lastErr error for attempt := 0; attempt < 3; attempt++ { - cand, err := s.pick(ctx, ir.Model, group, "CHAT", excluded) + cand, err := s.pick(ctx, modelName, group, "CHAT", excluded) if err != nil { return nil, meta, firstErr(lastErr, err) } @@ -81,12 +83,12 @@ func (s *AiGatewayService) OpenStream(ctx context.Context, ir aiwire.ChatRequest if err != nil { return nil, meta, err } - stream, err := s.client.GenAiChatStream(ctx, cred, cand.ch.Region, cand.modelOcid, ir) + stream, err := s.client.GenAiCompatResponsesStream(ctx, cred, cand.ch.Region, raw) if err == nil { s.markSuccess(ctx, cand.ch.ID) return stream, meta, nil } - retry, penalize := s.noteCallErr(ctx, cand, err) + retry, penalize := switchable(err) if !retry { return nil, meta, err } @@ -108,12 +110,11 @@ func firstErr(lastErr, pickErr error) error { return pickErr } -// noteCallErr 汇总一次调用失败:模型级不可用(微调基座 400 / 实体不存在 404)先标记 -// 剔除该 (渠道, 模型),换渠道重试且不计熔断——这是模型×区域供给问题而非渠道健康问题; -// 429 / 5xx / 网络错误换渠道并计熔断;其余 4xx 直接透传。 -func (s *AiGatewayService) noteCallErr(ctx context.Context, cand *aiCandidate, err error) (retry, penalize bool) { +// switchable 判断调用失败是否换渠道重试、是否计入熔断:模型级不可用(微调基座 +// 400 / 实体不存在 404)换渠道但不计熔断——这是模型×区域供给问题而非渠道健康 +// 问题,持久解决靠加入模型黑名单;429 / 5xx / 网络错误换渠道并计熔断;其余 4xx 直接透传。 +func switchable(err error) (retry, penalize bool) { if oci.IsModelUnavailable(err) { - s.markModelUnusable(ctx, cand.ch.ID, cand.modelOcid, oci.CompactError(err)) return true, false } if status, ok := oci.ServiceStatus(err); ok { @@ -154,11 +155,10 @@ func (s *AiGatewayService) pick(ctx context.Context, modelName, group, capabilit return &aiCandidate{ch: chosen, modelOcid: ocids[chosen.ID]}, nil } -// modelChannels 查出提供该模型的渠道 ID 及各自的模型 OCID(剔除不可按需调用标记); -// capability=CHAT 时兼容存量空串(加列前只同步对话模型);unusable 需兼容 NULL -// (AutoMigrate 加列后、首次重同步前的存量行)。 +// modelChannels 查出提供该模型的渠道 ID 及各自的模型 OCID; +// capability=CHAT 时兼容存量空串(加列前只同步对话模型)。 func (s *AiGatewayService) modelChannels(ctx context.Context, modelName, capability string) (map[uint]string, []uint, error) { - q := s.db.WithContext(ctx).Where("name = ? AND (unusable = ? OR unusable IS NULL)", modelName, false) + q := s.db.WithContext(ctx).Where("name = ?", modelName) if capability == "CHAT" { q = q.Where("capability IN ?", []string{"CHAT", ""}) } else { @@ -234,7 +234,7 @@ func (s *AiGatewayService) Embeddings(ctx context.Context, req aiwire.Embeddings s.markSuccess(ctx, cand.ch.ID) return resp, meta, nil } - retry, penalize := s.noteCallErr(ctx, cand, err) + retry, penalize := switchable(err) if !retry { return nil, meta, err } diff --git a/internal/service/aigateway_models_test.go b/internal/service/aigateway_models_test.go new file mode 100644 index 0000000..b80d6d7 --- /dev/null +++ b/internal/service/aigateway_models_test.go @@ -0,0 +1,29 @@ +package service + +import ( + "strings" + "testing" + + "oci-portal/internal/oci" +) + +// TestSupportedGatewayModels 断言 CHAT 目录仅保留兼容面厂商,EMBEDDING 不受影响。 +func TestSupportedGatewayModels(t *testing.T) { + in := []oci.GenAiModel{ + {Name: "xai.grok-4.3", Capability: "CHAT"}, + {Name: "meta.llama-4-maverick-17b-128e", Capability: "CHAT"}, + {Name: "openai.gpt-oss-120b", Capability: "CHAT"}, + {Name: "google.gemini-2.5-flash", Capability: "CHAT"}, + {Name: "cohere.command-a-03-2025", Capability: "CHAT"}, + {Name: "cohere.embed-v4.0", Capability: "EMBEDDING"}, + } + out := supportedGatewayModels(in) + names := make([]string, 0, len(out)) + for _, m := range out { + names = append(names, m.Name) + } + want := "xai.grok-4.3,meta.llama-4-maverick-17b-128e,openai.gpt-oss-120b,cohere.embed-v4.0" + if got := strings.Join(names, ","); got != want { + t.Fatalf("过滤结果 = %s, want %s", got, want) + } +} diff --git a/internal/service/aigateway_test.go b/internal/service/aigateway_test.go index ba98d74..8e90d4d 100644 --- a/internal/service/aigateway_test.go +++ b/internal/service/aigateway_test.go @@ -1,9 +1,11 @@ package service import ( + "bytes" "context" "errors" "fmt" + "io" "strings" "testing" "time" @@ -45,15 +47,46 @@ type gatewayStubClient struct { probeCode int probeErr error // probeSeq 非空时逐次弹出,模拟按候选依次试调;弹尽后回落 probeCode/probeErr - probeSeq []probeResult - chatResp *aiwire.ChatResponse - chatErrs []error - chatCalls int - regions []string + probeSeq []probeResult + // probedNames 记录试调过的模型名,供断言候选过滤 + probedNames []string + chatCalls int + regions []string embedVecs [][]float32 embedUsage *aiwire.Usage embedErr error + + passPayload []byte + passErrs []error + passCalls int + passRegions []string +} + +func (f *gatewayStubClient) GenAiCompatResponses(ctx context.Context, cred oci.Credentials, region string, body []byte) ([]byte, error) { + f.passCalls++ + f.passRegions = append(f.passRegions, region) + if len(f.passErrs) > 0 { + err := f.passErrs[0] + f.passErrs = f.passErrs[1:] + if err != nil { + return nil, err + } + } + return f.passPayload, nil +} + +func (f *gatewayStubClient) GenAiCompatResponsesStream(ctx context.Context, cred oci.Credentials, region string, body []byte) (io.ReadCloser, error) { + f.passCalls++ + f.passRegions = append(f.passRegions, region) + if len(f.passErrs) > 0 { + err := f.passErrs[0] + f.passErrs = f.passErrs[1:] + if err != nil { + return nil, err + } + } + return io.NopCloser(bytes.NewReader(f.passPayload)), nil } func (f *gatewayStubClient) GenAiEmbed(ctx context.Context, cred oci.Credentials, region, modelOcid string, inputs []string, dimensions *int) ([][]float32, *aiwire.Usage, error) { @@ -71,6 +104,7 @@ type probeResult struct { } func (f *gatewayStubClient) GenAiProbeChat(ctx context.Context, cred oci.Credentials, region, modelOcid, modelName string) (int, error) { + f.probedNames = append(f.probedNames, modelName) if len(f.probeSeq) > 0 { r := f.probeSeq[0] f.probeSeq = f.probeSeq[1:] @@ -79,23 +113,10 @@ func (f *gatewayStubClient) GenAiProbeChat(ctx context.Context, cred oci.Credent return f.probeCode, f.probeErr } -func (f *gatewayStubClient) GenAiChat(ctx context.Context, cred oci.Credentials, region, modelOcid string, ir aiwire.ChatRequest) (*aiwire.ChatResponse, error) { - f.chatCalls++ - f.regions = append(f.regions, region) - if len(f.chatErrs) > 0 { - err := f.chatErrs[0] - f.chatErrs = f.chatErrs[1:] - if err != nil { - return nil, err - } - } - return f.chatResp, nil -} - func newTestGateway(t *testing.T, client oci.Client) (*AiGatewayService, *OciConfigService) { t.Helper() svc := newTestService(t, client) - if err := svc.db.AutoMigrate(&model.AiKey{}, &model.AiChannel{}, &model.AiModelCache{}, &model.AiCallLog{}); err != nil { + if err := svc.db.AutoMigrate(&model.AiKey{}, &model.AiChannel{}, &model.AiModelCache{}, &model.AiModelBlacklist{}, &model.AiCallLog{}); err != nil { t.Fatalf("auto migrate ai tables: %v", err) } return NewAiGatewayService(svc.db, svc, client), svc @@ -105,14 +126,14 @@ func TestAiKeyLifecycle(t *testing.T) { gw, _ := newTestGateway(t, &gatewayStubClient{fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}}) ctx := context.Background() - raw, key, err := gw.CreateKey(ctx, "免费-ai-api-key", "test-api-key", "") + raw, key, err := gw.CreateKey(ctx, "免费-ai-api-key", "test-api-key", "", nil) if err != nil || raw != "test-api-key" || key.Tail != "-key" { t.Fatalf("CreateKey 自定义值 = %q %+v, %v", raw, key, err) } - if _, _, err := gw.CreateKey(ctx, "short", "abc", ""); err == nil { + if _, _, err := gw.CreateKey(ctx, "short", "abc", "", nil); err == nil { t.Error("过短自定义密钥应被拒绝") } - auto, _, err := gw.CreateKey(ctx, "auto", "", "") + auto, _, err := gw.CreateKey(ctx, "auto", "", "", nil) if err != nil || len(auto) < 40 || auto[:3] != "sk-" { t.Fatalf("CreateKey 随机值 = %q, %v", auto, err) } @@ -124,7 +145,7 @@ func TestAiKeyLifecycle(t *testing.T) { t.Errorf("错误密钥 err = %v", err) } off := false - _ = gw.UpdateKey(ctx, key.ID, "", &off, nil) + _ = gw.UpdateKey(ctx, key.ID, "", &off, nil, nil) if _, err := gw.VerifyKey(ctx, "test-api-key"); !errors.Is(err, ErrAiKeyInvalid) { t.Errorf("禁用后 VerifyKey err = %v", err) } @@ -187,51 +208,6 @@ func seedChannel(t *testing.T, gw *AiGatewayService, cfgID uint, region string, return ch } -func TestAiChatRetrySwitchesChannel(t *testing.T) { - client := &gatewayStubClient{ - fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}, - chatResp: &aiwire.ChatResponse{Model: "meta.llama-3.3-70b-instruct", Choices: []aiwire.Choice{{Message: aiwire.ChatMessage{Role: "assistant", Content: aiwire.NewTextContent("hi")}, FinishReason: "stop"}}}, - chatErrs: []error{stubServiceError{status: 429}}, - } - gw, svc := newTestGateway(t, client) - cfg := importAliveConfig(t, svc) - // 两个同优先级渠道 - seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1) - seedChannel(t, gw, cfg.ID, "us-chicago-1", 1, 1) - ctx := context.Background() - - resp, meta, err := gw.Chat(ctx, aiwire.ChatRequest{Model: "meta.llama-3.3-70b-instruct", Messages: []aiwire.ChatMessage{{Role: "user", Content: aiwire.NewTextContent("你好")}}}, "") - if err != nil || resp == nil { - t.Fatalf("Chat = %v, %v", resp, err) - } - if meta.Retries != 1 || client.chatCalls != 2 { - t.Errorf("应换渠道重试一次: retries=%d calls=%d", meta.Retries, client.chatCalls) - } - if len(client.regions) != 2 && client.regions[0] == client.regions[1] { - t.Errorf("重试未换渠道: %v", client.regions) - } - // 未知模型 - if _, _, err := gw.Chat(ctx, aiwire.ChatRequest{Model: "no-such-model"}, ""); !errors.Is(err, ErrAiUnknownModel) { - t.Errorf("未知模型 err = %v", err) - } -} - -func TestAiChatNonRetryablePassThrough(t *testing.T) { - client := &gatewayStubClient{ - fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}, - chatErrs: []error{stubServiceError{status: 400}}, - } - gw, svc := newTestGateway(t, client) - cfg := importAliveConfig(t, svc) - seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1) - seedChannel(t, gw, cfg.ID, "us-chicago-1", 1, 1) - - _, meta, err := gw.Chat(context.Background(), aiwire.ChatRequest{Model: "meta.llama-3.3-70b-instruct"}, "") - if err == nil || meta.Retries != 0 || client.chatCalls != 1 { - t.Errorf("400 应直接透传不重试: err=%v retries=%d calls=%d", err, meta.Retries, client.chatCalls) - } -} - func TestPickPriorityAndBreaker(t *testing.T) { gw, svc := newTestGateway(t, &gatewayStubClient{fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}}) cfg := importAliveConfig(t, svc) @@ -282,8 +258,8 @@ func TestMarkFailureBackoff(t *testing.T) { func TestAiGroupRouting(t *testing.T) { client := &gatewayStubClient{ - fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}, - chatResp: &aiwire.ChatResponse{Model: "meta.llama-3.3-70b-instruct", Choices: []aiwire.Choice{{Message: aiwire.ChatMessage{Role: "assistant", Content: aiwire.NewTextContent("hi")}, FinishReason: "stop"}}}, + fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}, + passPayload: []byte(`{"id":"resp_1","usage":{"input_tokens":3,"output_tokens":1}}`), } gw, svc := newTestGateway(t, client) cfg := importAliveConfig(t, svc) @@ -292,17 +268,17 @@ func TestAiGroupRouting(t *testing.T) { gw.db.Model(vip).Update("channel_group", "vip") gw.db.Create(&model.AiModelCache{ChannelID: other.ID, ModelOcid: "ocid1..cr", Name: "cohere.command-r", Vendor: "cohere", SyncedAt: time.Now()}) ctx := context.Background() - req := aiwire.ChatRequest{Model: "meta.llama-3.3-70b-instruct", Messages: []aiwire.ChatMessage{{Role: "user", Content: aiwire.NewTextContent("你好")}}} + body := []byte(`{"model":"meta.llama-3.3-70b-instruct","input":"你好"}`) // 分组密钥只落同分组渠道 for i := 0; i < 5; i++ { - _, meta, err := gw.Chat(ctx, req, "vip") + _, meta, err := gw.RespPassthrough(ctx, body, "meta.llama-3.3-70b-instruct", "vip") if err != nil || meta.ChannelID != vip.ID { t.Fatalf("vip 组第 %d 次落点 = %d, err %v, want %d", i, meta.ChannelID, err, vip.ID) } } // 分组内无渠道 → 无可用渠道 - if _, _, err := gw.Chat(ctx, req, "nope"); !errors.Is(err, ErrAiNoChannel) { + if _, _, err := gw.RespPassthrough(ctx, body, "meta.llama-3.3-70b-instruct", "nope"); !errors.Is(err, ErrAiNoChannel) { t.Errorf("空分组 err = %v, want ErrAiNoChannel", err) } // 模型列表按分组过滤 @@ -315,12 +291,12 @@ func TestAiGroupRouting(t *testing.T) { t.Errorf("不限组模型数 = %d, want 2", len(all.Data)) } // 密钥分组落库 - _, key, err := gw.CreateKey(ctx, "vip-key", "vip-secret-1234", "vip") + _, key, err := gw.CreateKey(ctx, "vip-key", "vip-secret-1234", "vip", nil) if err != nil || key.Group != "vip" { t.Fatalf("CreateKey group = %+v, %v", key, err) } empty := "" - _ = gw.UpdateKey(ctx, key.ID, "", nil, &empty) + _ = gw.UpdateKey(ctx, key.ID, "", nil, &empty, nil) var fresh model.AiKey gw.db.First(&fresh, key.ID) if fresh.Group != "" { @@ -465,7 +441,8 @@ func entityNotFoundErr() stubServiceError { } func TestProbeSkipsUnavailableModels(t *testing.T) { - // 首选 llama 不可按需调用(基座 400 / 实体 404)→ 标记剔除 → 换候选成功 → 渠道判可用 + // 首选 llama 不可按需调用(基座 400 / 实体 404)→ 换候选成功 → 渠道判可用; + // 模型全量入库不再自动标记,持久剔除交由用户手动拉黑 tests := []struct { name string bad probeResult @@ -496,112 +473,16 @@ func TestProbeSkipsUnavailableModels(t *testing.T) { if err != nil || probed.ProbeStatus != "ok" { t.Fatalf("坏候选后应换候选并判可用: %+v, %v", probed, err) } - var row model.AiModelCache - if err := gw.db.Where("channel_id = ? AND model_ocid = ?", ch.ID, "m1").First(&row).Error; err != nil || !row.Unusable { - t.Fatalf("首个坏候选应被标记不可用: %+v, %v", row, err) - } - // 再次探测:同步保留标记,m1 不再进候选(probeSeq 只需一次 200) - client.probeSeq = []probeResult{{200, nil}} - probed, err = gw.ProbeChannel(ctx, ch.ID) - if err != nil || probed.ProbeStatus != "ok" { - t.Fatalf("复测应跳过已标记模型: %+v, %v", probed, err) - } - var again model.AiModelCache - gw.db.Where("channel_id = ? AND model_ocid = ?", ch.ID, "m1").First(&again) - if !again.Unusable { - t.Error("探测触发的同步应保留不可用标记") - } - // 手动同步同样保留标记(坏模型不随重新同步复活);近期已检的标记不被后台验证翻转 - client.probeCode = 200 - if _, err := gw.SyncModels(ctx, ch.ID); err != nil { - t.Fatalf("SyncModels: %v", err) - } - gw.Wait() - var kept model.AiModelCache - gw.db.Where("channel_id = ? AND model_ocid = ?", ch.ID, "m1").First(&kept) - if !kept.Unusable { - t.Error("手动同步不应清除不可用标记") + rows, _ := gw.channelModels(ctx, ch.ID) + if len(rows) != 3 { + t.Errorf("模型应全量入库, got %d", len(rows)) } }) } } -func TestValidateModelsAfterSync(t *testing.T) { - // 同步后后台验证:坏模型标记剔除、好模型记录已检、其他 4xx 不改状态、非对话模型不试调 - client := &gatewayStubClient{ - fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}, - models: []oci.GenAiModel{ - {Ocid: "v1", Name: "meta.llama-4-maverick", Vendor: "meta", Capability: "CHAT"}, - {Ocid: "v2", Name: "xai.grok-4", Vendor: "xai", Capability: "CHAT"}, - {Ocid: "v3", Name: "xai.grok-voice-agent", Vendor: "xai", Capability: "CHAT"}, - {Ocid: "v4", Name: "cohere.embed-v4.0", Vendor: "cohere", Capability: "EMBEDDING"}, - }, - probeSeq: []probeResult{{404, entityNotFoundErr()}, {200, nil}, {400, stubServiceError{status: 400}}}, - } - gw, svc := newTestGateway(t, client) - cfg := importAliveConfig(t, svc) - ctx := context.Background() - ch, err := gw.CreateChannel(ctx, ChannelInput{OciConfigID: cfg.ID, Region: "us-ashburn-1"}) - if err != nil { - t.Fatalf("CreateChannel: %v", err) - } - if _, err := gw.SyncModels(ctx, ch.ID); err != nil { - t.Fatalf("SyncModels: %v", err) - } - gw.Wait() - - want := map[string]struct { - unusable bool - checked bool - }{ - "v1": {true, true}, // 实体 404 → 标记剔除 - "v2": {false, true}, // 200 → 可用已检 - "v3": {false, true}, // 普通 400 → 已检不标记 - "v4": {false, false}, // EMBEDDING 不试调 - } - rows, _ := gw.channelModels(ctx, ch.ID) - for _, r := range rows { - w := want[r.ModelOcid] - if r.Unusable != w.unusable || (r.CheckedAt != nil) != w.checked { - t.Errorf("%s: unusable=%v checked=%v, want %+v", r.ModelOcid, r.Unusable, r.CheckedAt != nil, w) - } - } - list, err := gw.GatewayModels(ctx, "") - if err != nil || len(list.Data) != 3 { - t.Errorf("网关列表应只剔除被标记的坏模型(余 v2/v3/v4), got %+v, %v", list.Data, err) - } -} - -func TestValidateRecheckUnmarksRecovered(t *testing.T) { - // 已标记模型超过复检间隔后重验:恢复供给(200)自动解除标记 - client := &gatewayStubClient{fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}, probeCode: 200} - gw, svc := newTestGateway(t, client) - cfg := importAliveConfig(t, svc) - ch := seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1) - ctx := context.Background() - old := time.Now().Add(-25 * time.Hour) - gw.db.Model(&model.AiModelCache{}).Where("channel_id = ?", ch.ID). - Updates(map[string]any{"unusable": true, "unusable_reason": "x", "checked_at": old}) - - gw.validateChannelModels(ctx, ch.ID) - var row model.AiModelCache - gw.db.Where("channel_id = ?", ch.ID).First(&row) - if row.Unusable || row.UnusableReason != "" || row.CheckedAt == nil || !row.CheckedAt.After(old) { - t.Errorf("超期复检应解除标记并刷新已检时间: %+v", row) - } - // 未超期的标记不复检(probeSeq 为空、fallback 200 也不会被消费) - fresh := time.Now() - gw.db.Model(&model.AiModelCache{}).Where("channel_id = ?", ch.ID). - Updates(map[string]any{"unusable": true, "checked_at": fresh}) - gw.validateChannelModels(ctx, ch.ID) - gw.db.Where("channel_id = ?", ch.ID).First(&row) - if !row.Unusable { - t.Error("未超期的标记不应被复检翻转") - } -} - func TestProbeAuth404StillNoQuota(t *testing.T) { - // 鉴权类 404(NotAuthorizedOrNotFound)仍属租户级,直接定论 no_quota 且不标记模型 + // 鉴权类 404(NotAuthorizedOrNotFound)仍属租户级,直接定论 no_quota client := &gatewayStubClient{ fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}, models: []oci.GenAiModel{{Ocid: "m1", Name: "meta.llama-3.3-70b-instruct", Vendor: "meta"}}, @@ -617,31 +498,26 @@ func TestProbeAuth404StillNoQuota(t *testing.T) { if probed.ProbeStatus != "no_quota" { t.Errorf("鉴权 404 status = %q, want no_quota", probed.ProbeStatus) } - var marked int64 - gw.db.Model(&model.AiModelCache{}).Where("unusable = ?", true).Count(&marked) - if marked != 0 { - t.Errorf("鉴权 404 不应标记模型, marked=%d", marked) - } } func TestAiChatFinetuneSwitchesChannelWithoutPenalty(t *testing.T) { // 微调基座 400 换渠道重试成功,且不计入熔断失败 client := &gatewayStubClient{ - fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}, - chatResp: &aiwire.ChatResponse{Model: "meta.llama-3.3-70b-instruct", Choices: []aiwire.Choice{{Message: aiwire.ChatMessage{Role: "assistant", Content: aiwire.NewTextContent("hi")}, FinishReason: "stop"}}}, - chatErrs: []error{finetuneBaseErr()}, + fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}, + passPayload: []byte(`{"id":"resp_1","usage":{"input_tokens":3,"output_tokens":1}}`), + passErrs: []error{finetuneBaseErr()}, } gw, svc := newTestGateway(t, client) cfg := importAliveConfig(t, svc) seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1) seedChannel(t, gw, cfg.ID, "us-chicago-1", 1, 1) - resp, meta, err := gw.Chat(context.Background(), aiwire.ChatRequest{Model: "meta.llama-3.3-70b-instruct", Messages: []aiwire.ChatMessage{{Role: "user", Content: aiwire.NewTextContent("你好")}}}, "") + resp, meta, err := gw.RespPassthrough(context.Background(), []byte(`{}`), "meta.llama-3.3-70b-instruct", "") if err != nil || resp == nil { - t.Fatalf("Chat = %v, %v", resp, err) + t.Fatalf("RespPassthrough = %v, %v", resp, err) } - if meta.Retries != 1 || client.chatCalls != 2 { - t.Errorf("应换渠道重试一次: retries=%d calls=%d", meta.Retries, client.chatCalls) + if meta.Retries != 1 || client.passCalls != 2 { + t.Errorf("应换渠道重试一次: retries=%d calls=%d", meta.Retries, client.passCalls) } var chs []model.AiChannel gw.db.Find(&chs) @@ -650,38 +526,93 @@ func TestAiChatFinetuneSwitchesChannelWithoutPenalty(t *testing.T) { t.Errorf("微调基座 400 不应计入熔断: 渠道 %s failCount=%d", ch.Name, ch.FailCount) } } - // 失败渠道的该模型被标记,不再参与路由;成功渠道不受影响 - region := client.regions[0] - var row model.AiModelCache - gw.db.Where("model_ocid = ?", "ocid1..m-"+region).First(&row) - if !row.Unusable { - t.Errorf("失败渠道的模型应被标记不可用: %+v", row) +} + +func TestBlacklistLifecycle(t *testing.T) { + // 拉黑:全渠道同名缓存删除、列表与路由立即不可见;重复/空名被拒; + // 移出黑名单后重新同步恢复入池 + client := &gatewayStubClient{ + fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}, + models: []oci.GenAiModel{{Ocid: "ocid1..m-eu-frankfurt-1", Name: "meta.llama-3.3-70b-instruct", Vendor: "meta"}}, } - var usable int64 - gw.db.Model(&model.AiModelCache{}).Where("unusable = ?", false).Count(&usable) - if usable != 1 { - t.Errorf("成功渠道模型应保持可用, usable=%d", usable) + gw, svc := newTestGateway(t, client) + cfg := importAliveConfig(t, svc) + ch := seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1) + seedChannel(t, gw, cfg.ID, "us-chicago-1", 1, 1) + ctx := context.Background() + + item, err := gw.AddBlacklist(ctx, " meta.llama-3.3-70b-instruct ") + if err != nil || item.Name != "meta.llama-3.3-70b-instruct" { + t.Fatalf("AddBlacklist = %+v, %v", item, err) + } + var left int64 + gw.db.Model(&model.AiModelCache{}).Count(&left) + if left != 0 { + t.Errorf("拉黑应删除全部渠道的同名缓存, 剩 %d", left) + } + if _, _, err := gw.RespPassthrough(ctx, []byte(`{}`), "meta.llama-3.3-70b-instruct", ""); !errors.Is(err, ErrAiUnknownModel) { + t.Errorf("拉黑后路由应按未知模型拒绝: %v", err) + } + if _, err := gw.AddBlacklist(ctx, "meta.llama-3.3-70b-instruct"); err == nil { + t.Error("重复拉黑应被拒绝") + } + if _, err := gw.AddBlacklist(ctx, " "); err == nil { + t.Error("空模型名应被拒绝") + } + rows, err := gw.Blacklist(ctx) + if err != nil || len(rows) != 1 { + t.Fatalf("Blacklist = %+v, %v", rows, err) + } + if err := gw.RemoveBlacklist(ctx, rows[0].ID); err != nil { + t.Fatalf("RemoveBlacklist: %v", err) + } + if err := gw.RemoveBlacklist(ctx, rows[0].ID); err == nil { + t.Error("移除不存在的条目应报错") + } + // 移出后重新同步:模型恢复入池 + if _, err := gw.SyncModels(ctx, ch.ID); err != nil { + t.Fatalf("SyncModels: %v", err) + } + list, _ := gw.GatewayModels(ctx, "") + if len(list.Data) != 1 { + t.Errorf("移出黑名单并同步后应恢复入池: %+v", list.Data) } } -func TestUnusableModelExcludedFromPoolAndRouting(t *testing.T) { - // 唯一渠道的模型被标记后:网关列表不再展示,路由按未知模型拒绝,渠道详情仍可见标记 - gw, svc := newTestGateway(t, &gatewayStubClient{fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}}) +func TestSyncAndProbeFilterBlacklisted(t *testing.T) { + // 同步与探测都过滤黑名单模型:不入库、不进试调候选 + client := &gatewayStubClient{ + fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}, + models: []oci.GenAiModel{ + {Ocid: "m1", Name: "meta.llama-3-70b-instruct", Vendor: "meta"}, + {Ocid: "m2", Name: "cohere.command-a-03-2025", Vendor: "cohere"}, + }, + probeCode: 200, + } + gw, svc := newTestGateway(t, client) cfg := importAliveConfig(t, svc) - ch := seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1) ctx := context.Background() - - gw.markModelUnusable(ctx, ch.ID, "ocid1..m-eu-frankfurt-1", "Entity with key … not found") - list, err := gw.GatewayModels(ctx, "") - if err != nil || len(list.Data) != 0 { - t.Errorf("已标记模型不应出现在网关列表: %+v, %v", list.Data, err) + if _, err := gw.AddBlacklist(ctx, "meta.llama-3-70b-instruct"); err != nil { + t.Fatalf("AddBlacklist: %v", err) } - if _, _, err := gw.Chat(ctx, aiwire.ChatRequest{Model: "meta.llama-3.3-70b-instruct"}, ""); !errors.Is(err, ErrAiUnknownModel) { - t.Errorf("已标记模型路由应拒绝: %v", err) + ch, err := gw.CreateChannel(ctx, ChannelInput{OciConfigID: cfg.ID, Region: "eu-frankfurt-1"}) + if err != nil { + t.Fatalf("CreateChannel: %v", err) } - rows, err := gw.channelModels(ctx, ch.ID) - if err != nil || len(rows) != 1 || !rows[0].Unusable || rows[0].UnusableReason == "" { - t.Errorf("渠道详情应保留标记行便于排查: %+v, %v", rows, err) + probed, err := gw.ProbeChannel(ctx, ch.ID) + if err != nil || probed.ProbeStatus != "ok" { + t.Fatalf("ProbeChannel = %+v, %v", probed, err) + } + rows, _ := gw.channelModels(ctx, ch.ID) + if len(rows) != 1 || rows[0].Name != "cohere.command-a-03-2025" { + t.Errorf("探测同步应过滤黑名单模型: %+v", rows) + } + if len(client.probedNames) != 1 || client.probedNames[0] != "cohere.command-a-03-2025" { + t.Errorf("试调候选不应包含黑名单模型: %v", client.probedNames) + } + models, err := gw.SyncModels(ctx, ch.ID) + if err != nil || len(models) != 1 || models[0].Name != "cohere.command-a-03-2025" { + t.Errorf("SyncModels 应过滤黑名单模型: %+v, %v", models, err) } } @@ -745,7 +676,7 @@ func TestAiEmbeddings(t *testing.T) { t.Errorf("chat 模型走 embeddings err = %v, want ErrAiUnknownModel", err) } // embedding 模型名打 chat:同样未知模型 - if _, _, err := gw.Chat(ctx, aiwire.ChatRequest{Model: "cohere.embed-v4.0", Messages: []aiwire.ChatMessage{{Role: "user", Content: aiwire.NewTextContent("x")}}}, ""); !errors.Is(err, ErrAiUnknownModel) { + if _, _, err := gw.RespPassthrough(ctx, []byte(`{}`), "cohere.embed-v4.0", ""); !errors.Is(err, ErrAiUnknownModel) { t.Errorf("embedding 模型走 chat err = %v, want ErrAiUnknownModel", err) } } @@ -756,7 +687,7 @@ func TestAiContentLogSwitch(t *testing.T) { t.Fatalf("migrate content log: %v", err) } ctx := context.Background() - _, key, err := gw.CreateKey(ctx, "k1", "content-key-1234", "") + _, key, err := gw.CreateKey(ctx, "k1", "content-key-1234", "", nil) if err != nil { t.Fatalf("CreateKey: %v", err) } @@ -793,23 +724,130 @@ func TestAiContentLogSwitch(t *testing.T) { } } -func TestLegacyNullUnusableRowsStayVisible(t *testing.T) { - // 升级路径回归:AutoMigrate 加列后、首次重同步前,存量行 unusable 为 NULL, - // 网关列表与路由必须照常包含这些行,不能因 unusable = false 过滤而整池消失 - gw, svc := newTestGateway(t, &gatewayStubClient{fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}}) +func TestRespPassthroughSwitchesChannel(t *testing.T) { + client := &gatewayStubClient{ + fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}, + passPayload: []byte(`{"id":"resp_1","usage":{"input_tokens":9,"output_tokens":3}}`), + passErrs: []error{stubServiceError{status: 429}}, + } + gw, svc := newTestGateway(t, client) cfg := importAliveConfig(t, svc) - ch := seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1) - ctx := context.Background() - if err := gw.db.Exec("UPDATE ai_model_caches SET unusable = NULL WHERE channel_id = ?", ch.ID).Error; err != nil { - t.Fatalf("set legacy null: %v", err) - } + seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1) + seedChannel(t, gw, cfg.ID, "us-chicago-1", 1, 1) - list, err := gw.GatewayModels(ctx, "") - if err != nil || len(list.Data) != 1 { - t.Errorf("存量 NULL 行应仍在网关列表: %+v, %v", list.Data, err) + payload, meta, err := gw.RespPassthrough(context.Background(), []byte(`{}`), "meta.llama-3.3-70b-instruct", "") + if err != nil || len(payload) == 0 { + t.Fatalf("RespPassthrough = %v, %v", payload, err) } - _, ids, err := gw.modelChannels(ctx, "meta.llama-3.3-70b-instruct", "CHAT") - if err != nil || len(ids) != 1 { - t.Errorf("存量 NULL 行应仍参与路由: %v, %v", ids, err) + if meta.Retries != 1 || client.passCalls != 2 { + t.Errorf("应换渠道重试一次: retries=%d calls=%d", meta.Retries, client.passCalls) + } + if len(client.passRegions) == 2 && client.passRegions[0] == client.passRegions[1] { + t.Errorf("重试未换渠道: %v", client.passRegions) + } +} + +func TestRespPassthroughNonRetryable(t *testing.T) { + client := &gatewayStubClient{ + fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}, + passErrs: []error{stubServiceError{status: 400}}, + } + gw, svc := newTestGateway(t, client) + cfg := importAliveConfig(t, svc) + seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1) + seedChannel(t, gw, cfg.ID, "us-chicago-1", 1, 1) + + _, meta, err := gw.RespPassthrough(context.Background(), []byte(`{}`), "meta.llama-3.3-70b-instruct", "") + if err == nil || meta.Retries != 0 || client.passCalls != 1 { + t.Errorf("400 应直接透传不重试: err=%v retries=%d calls=%d", err, meta.Retries, client.passCalls) + } +} + +func TestNormalizeKeyModels(t *testing.T) { + tests := []struct { + name string + in []string + want []string + }{ + {"nil 输入", nil, nil}, + {"全空串", []string{"", " "}, nil}, + {"trim 与保序去重", []string{" a ", "b", "a", "", "b "}, []string{"a", "b"}}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got := normalizeKeyModels(tt.in) + if len(got) != len(tt.want) { + t.Fatalf("normalizeKeyModels(%v) = %v, want %v", tt.in, got, tt.want) + } + for i := range tt.want { + if got[i] != tt.want[i] { + t.Fatalf("normalizeKeyModels(%v) = %v, want %v", tt.in, got, tt.want) + } + } + }) + } +} + +// TestAiKeyModelsPersistence 钉住 serializer:json 字段经 Create 与 Updates(map) 两条路径的往返。 +func TestAiKeyModelsPersistence(t *testing.T) { + gw, _ := newTestGateway(t, &gatewayStubClient{fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}}) + ctx := context.Background() + + _, key, err := gw.CreateKey(ctx, "m-key", "model-key-1234", "", []string{" x-model ", "x-model", "y-model", ""}) + if err != nil { + t.Fatalf("CreateKey: %v", err) + } + fresh, err := gw.VerifyKey(ctx, "model-key-1234") + if err != nil || len(fresh.Models) != 2 || fresh.Models[0] != "x-model" || fresh.Models[1] != "y-model" { + t.Fatalf("创建后读回 models = %v, %v", fresh.Models, err) + } + set := []string{"z-model"} + if err := gw.UpdateKey(ctx, key.ID, "", nil, nil, &set); err != nil { + t.Fatalf("UpdateKey 覆盖: %v", err) + } + fresh, _ = gw.VerifyKey(ctx, "model-key-1234") + if len(fresh.Models) != 1 || fresh.Models[0] != "z-model" { + t.Fatalf("覆盖后 models = %v", fresh.Models) + } + if err := gw.UpdateKey(ctx, key.ID, "renamed", nil, nil, nil); err != nil { + t.Fatalf("UpdateKey 不传 models: %v", err) + } + fresh, _ = gw.VerifyKey(ctx, "model-key-1234") + if len(fresh.Models) != 1 { + t.Fatalf("未传 models 却被改动: %v", fresh.Models) + } + empty := []string{} + if err := gw.UpdateKey(ctx, key.ID, "", nil, nil, &empty); err != nil { + t.Fatalf("UpdateKey 清空: %v", err) + } + fresh, _ = gw.VerifyKey(ctx, "model-key-1234") + if len(fresh.Models) != 0 { + t.Fatalf("清空后 models = %v, want 空", fresh.Models) + } +} + +// TestRespPassthroughStreamSwitchesChannel 断言流式直通建立失败按 switchable 换渠道。 +func TestRespPassthroughStreamSwitchesChannel(t *testing.T) { + client := &gatewayStubClient{ + fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}, + passPayload: []byte("data: {\"type\":\"response.completed\"}\n\n"), + passErrs: []error{stubServiceError{status: 503}}, + } + gw, svc := newTestGateway(t, client) + cfg := importAliveConfig(t, svc) + seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1) + seedChannel(t, gw, cfg.ID, "us-chicago-1", 1, 1) + + stream, meta, err := gw.RespPassthroughStream(context.Background(), []byte(`{}`), "meta.llama-3.3-70b-instruct", "") + if err != nil || stream == nil { + t.Fatalf("RespPassthroughStream = %v", err) + } + defer stream.Close() + if meta.Retries != 1 || client.passCalls != 2 { + t.Errorf("应换渠道重试一次: retries=%d calls=%d", meta.Retries, client.passCalls) + } + payload, _ := io.ReadAll(stream) + if !bytes.Contains(payload, []byte("response.completed")) { + t.Errorf("流内容未透传: %s", payload) } } diff --git a/internal/service/airesponses.go b/internal/service/airesponses.go index 6c03b4d..cdcbfe3 100644 --- a/internal/service/airesponses.go +++ b/internal/service/airesponses.go @@ -8,38 +8,8 @@ import ( "oci-portal/internal/aiwire" ) -// ResponsesToIR 把 Responses API 请求转换为网关 IR(OpenAI Chat 形态)。 -// 有状态特性与内置工具不支持,直接报错(API 层 400)。 -func ResponsesToIR(req aiwire.RespRequest) (aiwire.ChatRequest, error) { - if err := respRejectUnsupported(req); err != nil { - return aiwire.ChatRequest{}, err - } - ir := aiwire.ChatRequest{ - Model: req.Model, - MaxTokens: req.MaxOutputTokens, - Temperature: req.Temperature, - TopP: req.TopP, - Stream: req.Stream, - ToolChoice: respToolChoice(req.ToolChoice), - } - if req.Reasoning != nil { - ir.ReasoningEffort = req.Reasoning.Effort - } - if req.Instructions != "" { - ir.Messages = append(ir.Messages, aiwire.ChatMessage{Role: "system", Content: aiwire.NewTextContent(req.Instructions)}) - } - msgs, err := respInputToMessages(req.Input) - if err != nil { - return aiwire.ChatRequest{}, err - } - ir.Messages = append(ir.Messages, msgs...) - ir.Tools = respTools(req.Tools) - ir.ResponseFormat = respFormat(req.Text) - return ir, nil -} - -// respRejectUnsupported 拒绝无状态网关无法承接的请求特性。 -func respRejectUnsupported(req aiwire.RespRequest) error { +// respRejectStateful 拒绝有状态特性(网关无状态)。 +func respRejectStateful(req aiwire.RespRequest) error { if req.PreviousResponseID != "" { return fmt.Errorf("previous_response_id 不支持:网关不保存历史响应,请在 input 中自带完整上下文(store:false 模式)") } @@ -49,195 +19,80 @@ func respRejectUnsupported(req aiwire.RespRequest) error { if req.Background != nil && *req.Background { return fmt.Errorf("background 模式不支持") } + return nil +} + +// RespServerTools 报告工具列表是否含 xAI 服务端工具(web_search / x_search)。 +func RespServerTools(tools []aiwire.RespTool) bool { + for _, t := range tools { + if t.Type == "web_search" || t.Type == "x_search" { + return true + } + } + return false +} + +// RespPassthroughValidate 校验直通请求:模型必填,有状态特性不支持,工具类型 +// 只放行 function 与已实测的 web_search / x_search;流式仅在含服务端工具时拒绝 +// (工具流式事件形态未实测,不放开)。 +func RespPassthroughValidate(req aiwire.RespRequest) error { + if strings.TrimSpace(req.Model) == "" { + return fmt.Errorf("model 不能为空") + } + if req.Stream && RespServerTools(req.Tools) { + return fmt.Errorf("服务端工具暂不支持流式:请去掉 stream 或改用 function 工具") + } + if err := respRejectStateful(req); err != nil { + return err + } for _, t := range req.Tools { - if t.Type != "function" { - return fmt.Errorf("不支持的工具类型 %q:仅支持 function 工具", t.Type) + switch t.Type { + case "function", "web_search", "x_search": + default: + return fmt.Errorf("不支持的工具类型 %q:服务端工具仅支持 web_search / x_search", t.Type) } } return nil } -// respInputToMessages 把 input(string 或 item 数组)展开为 IR 消息序列。 -func respInputToMessages(in aiwire.RespInput) ([]aiwire.ChatMessage, error) { - if !in.IsArray { - if strings.TrimSpace(in.Text) == "" { - return nil, fmt.Errorf("input 不能为空") - } - return []aiwire.ChatMessage{{Role: "user", Content: aiwire.NewTextContent(in.Text)}}, nil +// RespPassthroughBody 以原始请求体为基构造上游 body:强制 store:false(禁上游 +// 存态),stream 原样保留(流式直通);用 json.Number 保真未知字段与数值。 +func RespPassthroughBody(raw []byte) ([]byte, error) { + dec := json.NewDecoder(strings.NewReader(string(raw))) + dec.UseNumber() + var body map[string]any + if err := dec.Decode(&body); err != nil { + return nil, fmt.Errorf("解析请求体: %w", err) } - var msgs []aiwire.ChatMessage - for _, it := range in.Items { - out, err := respItemToMessages(it, msgs) - if err != nil { - return nil, err - } - msgs = out - } - return msgs, nil + body["store"] = false + return json.Marshal(body) } -// respItemToMessages 追加一个 input item;连续 function_call 合并进同一 assistant 消息。 -func respItemToMessages(it aiwire.RespItem, msgs []aiwire.ChatMessage) ([]aiwire.ChatMessage, error) { - switch it.Type { - case "", "message": - if it.Content.HasUnsupported() { - return nil, ErrAiUnsupportedBlock - } - role := respRole(it.Role) - return append(msgs, aiwire.ChatMessage{Role: role, Content: respContentToIR(it.Content)}), nil - case "function_call": - call := aiwire.ToolCall{ID: it.CallID, Type: "function", - Function: aiwire.FunctionCall{Name: it.Name, Arguments: it.Arguments}} - if n := len(msgs); n > 0 && msgs[n-1].Role == "assistant" && len(msgs[n-1].ToolCalls) > 0 { - msgs[n-1].ToolCalls = append(msgs[n-1].ToolCalls, call) - return msgs, nil - } - return append(msgs, aiwire.ChatMessage{Role: "assistant", ToolCalls: []aiwire.ToolCall{call}}), nil - case "function_call_output": - return append(msgs, aiwire.ChatMessage{Role: "tool", ToolCallID: it.CallID, - Content: aiwire.NewTextContent(it.OutputText())}), nil - case "reasoning": - return msgs, nil // 推理块不回灌上游 - default: - return nil, fmt.Errorf("不支持的 input item 类型 %q", it.Type) +// RespPassthroughUsage 从直通响应提取用量;缺失时返回 nil(日志记零)。 +func RespPassthroughUsage(payload []byte) *aiwire.Usage { + var root struct { + Usage *aiwire.RespUsage `json:"usage"` } -} - -// respRole 归一 item 角色;developer 视为 system,缺省 user。 -func respRole(role string) string { - switch role { - case "system", "developer": - return "system" - case "assistant": - return "assistant" - default: - return "user" - } -} - -// respContentToIR 把 Responses 内容转 IR;纯文本拍平,含图片时保留部件顺序。 -func respContentToIR(c aiwire.RespContent) aiwire.Content { - var parts []aiwire.ContentPart - hasImage := false - for _, p := range c.Parts { - switch p.Type { - case "input_image": - parts = append(parts, aiwire.ContentPart{Type: "image_url", - ImageURL: &aiwire.ImageURL{URL: p.ImageURL, Detail: p.Detail}}) - hasImage = true - default: - if p.Text != "" { - parts = append(parts, aiwire.ContentPart{Type: "text", Text: p.Text}) - } - } - } - if !c.IsArray || !hasImage { - return aiwire.NewTextContent(c.JoinText()) - } - return aiwire.NewPartsContent(parts) -} - -// respTools 把扁平工具定义还原为嵌套 Chat 形态;strict 无 OCI 对应,忽略。 -func respTools(tools []aiwire.RespTool) []aiwire.Tool { - if len(tools) == 0 { + if json.Unmarshal(payload, &root) != nil || root.Usage == nil { return nil } - out := make([]aiwire.Tool, 0, len(tools)) - for _, t := range tools { - out = append(out, aiwire.Tool{Type: "function", - Function: aiwire.FunctionDef{Name: t.Name, Description: t.Description, Parameters: t.Parameters}}) + usage := &aiwire.Usage{PromptTokens: root.Usage.InputTokens, + CompletionTokens: root.Usage.OutputTokens, TotalTokens: root.Usage.TotalTokens} + if cached := root.Usage.InputTokensDetails.CachedTokens; cached > 0 { + usage.PromptTokensDetails = &aiwire.PromptTokensDetails{CachedTokens: cached} } - return out + return usage } -// respToolChoice 转换 tool_choice:字符串直通;{type:function,name} 转嵌套; -// allowed_tools 等对象形态降级 auto。 -func respToolChoice(raw json.RawMessage) json.RawMessage { - if len(raw) == 0 { +// RespStreamCompletedUsage 从一行 SSE data JSON 中提取 response.completed 事件的 +// usage;非 completed 事件或解析失败返回 nil。流式直通逐行喂入,最后一次非 nil 生效。 +func RespStreamCompletedUsage(data []byte) *aiwire.Usage { + var ev struct { + Type string `json:"type"` + Response json.RawMessage `json:"response"` + } + if json.Unmarshal(data, &ev) != nil || ev.Type != "response.completed" || len(ev.Response) == 0 { return nil } - t := strings.TrimSpace(string(raw)) - if strings.HasPrefix(t, "\"") { - return raw - } - var obj struct { - Type string `json:"type"` - Name string `json:"name"` - } - if err := json.Unmarshal(raw, &obj); err != nil || obj.Type != "function" || obj.Name == "" { - return json.RawMessage(`"auto"`) - } - out, _ := json.Marshal(map[string]any{"type": "function", "function": map[string]string{"name": obj.Name}}) - return out -} - -// respFormat 把 text.format 转成 response_format;text 形态无需显式指定。 -func respFormat(text *aiwire.RespText) *aiwire.ResponseFormat { - if text == nil || text.Format == nil { - return nil - } - switch text.Format.Type { - case "json_object": - return &aiwire.ResponseFormat{Type: "json_object"} - case "json_schema": - spec, _ := json.Marshal(map[string]any{ - "name": text.Format.Name, "schema": json.RawMessage(text.Format.Schema), "strict": text.Format.Strict, - }) - return &aiwire.ResponseFormat{Type: "json_schema", JSONSchema: spec} - default: - return nil - } -} - -// IRRespToResponses 把 IR 非流式响应装配为 Response 对象。 -func IRRespToResponses(resp *aiwire.ChatResponse, id string, created int64) aiwire.Response { - out := aiwire.Response{ID: id, Object: "response", CreatedAt: created, - Status: "completed", Model: resp.Model, Output: []aiwire.RespOutItem{}, Store: false} - if len(resp.Choices) == 0 { - return out - } - choice := resp.Choices[0] - if text := choice.Message.Content.JoinText(); text != "" { - out.Output = append(out.Output, respMessageItem(id, 0, text)) - } - for i, tc := range choice.Message.ToolCalls { - out.Output = append(out.Output, aiwire.RespOutItem{Type: "function_call", - ID: respItemID(id, "fc", len(out.Output)+i), Status: "completed", - CallID: tc.ID, Name: tc.Function.Name, Arguments: respArgs(tc.Function.Arguments)}) - } - if choice.FinishReason == "length" { - out.Status = "incomplete" - out.IncompleteDetails = &aiwire.RespIncomplete{Reason: "max_output_tokens"} - } - out.Usage = respUsage(resp.Usage) - return out -} - -func respMessageItem(respID string, idx int, text string) aiwire.RespOutItem { - return aiwire.RespOutItem{Type: "message", ID: respItemID(respID, "msg", idx), - Status: "completed", Role: "assistant", - Content: []aiwire.RespOutPart{{Type: "output_text", Text: text, Annotations: []any{}}}} -} - -// respItemID 从响应 ID 派生确定性 item ID。 -func respItemID(respID, kind string, idx int) string { - return fmt.Sprintf("%s_%s_%d", kind, strings.TrimPrefix(respID, "resp_"), idx) -} - -// respArgs 保证 arguments 是合法 JSON 字符串(空实参回退 {})。 -func respArgs(args string) string { - if strings.TrimSpace(args) == "" { - return "{}" - } - return args -} - -// respUsage 把 IR 用量改写为 Responses 命名口径。 -func respUsage(u *aiwire.Usage) *aiwire.RespUsage { - if u == nil { - return nil - } - return &aiwire.RespUsage{InputTokens: u.PromptTokens, OutputTokens: u.CompletionTokens, - TotalTokens: u.TotalTokens, - InputTokensDetails: aiwire.RespInDetails{CachedTokens: u.CachedTokens()}} + return RespPassthroughUsage(ev.Response) } diff --git a/internal/service/airesponses_stream.go b/internal/service/airesponses_stream.go deleted file mode 100644 index 351f66a..0000000 --- a/internal/service/airesponses_stream.go +++ /dev/null @@ -1,199 +0,0 @@ -package service - -import ( - "strings" - - "oci-portal/internal/aiwire" -) - -// RespEvent 是一条 Responses SSE 语义事件(event 名 + data 载荷)。 -type RespEvent struct { - Event string - Data map[string]any -} - -// RespStream 把 IR chunk 流聚合为 Responses 语义事件序列。 -// 与 AnthStream 同构,但终态事件须携带完整 Response 快照,故全程缓冲文本与实参。 -type RespStream struct { - id string - model string - created int64 - seq int - started bool - items []aiwire.RespOutItem - msgOpen bool - text strings.Builder - tool aiwire.RespOutItem - toolIdx int - tOpen bool - args strings.Builder - finish string - usage *aiwire.Usage -} - -// NewRespStream 构造状态机;id 形如 resp_*,created 为响应时间戳。 -func NewRespStream(id, model string, created int64) *RespStream { - return &RespStream{id: id, model: model, created: created} -} - -// Feed 消化一个上游 chunk,返回应立即下发的事件。 -func (s *RespStream) Feed(chunk aiwire.ChatChunk) []RespEvent { - var evs []RespEvent - if !s.started { - s.started = true - evs = append(evs, s.respEvent("response.created", "in_progress"), - s.respEvent("response.in_progress", "in_progress")) - } - if chunk.Usage != nil { - s.usage = chunk.Usage - } - if len(chunk.Choices) == 0 { - return evs - } - choice := chunk.Choices[0] - if choice.Delta.Content != "" { - evs = append(evs, s.feedText(choice.Delta.Content)...) - } - for _, tc := range choice.Delta.ToolCalls { - evs = append(evs, s.feedTool(tc)...) - } - if choice.FinishReason != nil { - s.finish = *choice.FinishReason - } - return evs -} - -// Finish 关闭未闭合的块并产出终态事件(带完整 Response 快照)。 -func (s *RespStream) Finish() []RespEvent { - var evs []RespEvent - if !s.started { - s.started = true - evs = append(evs, s.respEvent("response.created", "in_progress")) - } - evs = append(evs, s.closeMsg()...) - evs = append(evs, s.closeTool()...) - status := "completed" - if s.finish == "length" { - status = "incomplete" - } - return append(evs, s.respEvent("response."+status, status)) -} - -// Usage 返回聚合到的用量(可能为 nil),供调用日志。 -func (s *RespStream) Usage() *aiwire.Usage { return s.usage } - -// ev 构造带自增 sequence_number 的事件。 -func (s *RespStream) ev(typ string, kv map[string]any) RespEvent { - s.seq++ - data := map[string]any{"type": typ, "sequence_number": s.seq} - for k, v := range kv { - data[k] = v - } - return RespEvent{Event: typ, Data: data} -} - -// respEvent 构造携带 Response 快照的生命周期事件。 -func (s *RespStream) respEvent(typ, status string) RespEvent { - return s.ev(typ, map[string]any{"response": s.snapshot(status)}) -} - -// snapshot 组装当前累计状态的 Response 对象。 -func (s *RespStream) snapshot(status string) aiwire.Response { - resp := aiwire.Response{ID: s.id, Object: "response", CreatedAt: s.created, - Status: status, Model: s.model, Store: false, - Output: append([]aiwire.RespOutItem{}, s.items...)} - resp.Usage = respUsage(s.usage) - if status == "incomplete" { - resp.IncompleteDetails = &aiwire.RespIncomplete{Reason: "max_output_tokens"} - } - return resp -} - -// feedText 处理文本增量:必要时开 message item 与 content part。 -func (s *RespStream) feedText(delta string) []RespEvent { - evs := s.closeTool() - if !s.msgOpen { - s.msgOpen = true - s.text.Reset() - item := aiwire.RespOutItem{Type: "message", ID: s.itemID("msg"), Status: "in_progress", - Role: "assistant", Content: []aiwire.RespOutPart{}} - evs = append(evs, s.ev("response.output_item.added", map[string]any{ - "output_index": len(s.items), "item": item})) - evs = append(evs, s.ev("response.content_part.added", map[string]any{ - "item_id": item.ID, "output_index": len(s.items), "content_index": 0, - "part": aiwire.RespOutPart{Type: "output_text", Text: "", Annotations: []any{}}})) - } - s.text.WriteString(delta) - return append(evs, s.ev("response.output_text.delta", map[string]any{ - "item_id": s.itemID("msg"), "output_index": len(s.items), "content_index": 0, "delta": delta})) -} - -// closeMsg 闭合当前 message item(text done → part done → item done)。 -func (s *RespStream) closeMsg() []RespEvent { - if !s.msgOpen { - return nil - } - s.msgOpen = false - id, idx, full := s.itemID("msg"), len(s.items), s.text.String() - part := aiwire.RespOutPart{Type: "output_text", Text: full, Annotations: []any{}} - item := aiwire.RespOutItem{Type: "message", ID: id, Status: "completed", - Role: "assistant", Content: []aiwire.RespOutPart{part}} - evs := []RespEvent{ - s.ev("response.output_text.done", map[string]any{"item_id": id, "output_index": idx, "content_index": 0, "text": full}), - s.ev("response.content_part.done", map[string]any{"item_id": id, "output_index": idx, "content_index": 0, "part": part}), - s.ev("response.output_item.done", map[string]any{"output_index": idx, "item": item}), - } - s.items = append(s.items, item) - return evs -} - -// feedTool 处理工具调用增量:新 Index 先闭合旧调用再开新 item。 -func (s *RespStream) feedTool(tc aiwire.ToolCallDelta) []RespEvent { - evs := s.closeMsg() - if s.tOpen && tc.Index != s.toolIdx { - evs = append(evs, s.closeTool()...) - } - if !s.tOpen { - s.tOpen, s.toolIdx = true, tc.Index - s.args.Reset() - s.tool = aiwire.RespOutItem{Type: "function_call", ID: s.itemID("fc"), Status: "in_progress", - CallID: tc.ID, Name: tc.Function.Name} - evs = append(evs, s.ev("response.output_item.added", map[string]any{ - "output_index": len(s.items), "item": s.tool})) - } - if tc.ID != "" && s.tool.CallID == "" { - s.tool.CallID = tc.ID - } - if tc.Function.Name != "" && s.tool.Name == "" { - s.tool.Name = tc.Function.Name - } - if tc.Function.Arguments == "" { - return evs - } - s.args.WriteString(tc.Function.Arguments) - return append(evs, s.ev("response.function_call_arguments.delta", map[string]any{ - "item_id": s.tool.ID, "output_index": len(s.items), "delta": tc.Function.Arguments})) -} - -// closeTool 闭合当前 function_call item(arguments done → item done)。 -func (s *RespStream) closeTool() []RespEvent { - if !s.tOpen { - return nil - } - s.tOpen = false - s.tool.Arguments = respArgs(s.args.String()) - s.tool.Status = "completed" - idx := len(s.items) - evs := []RespEvent{ - s.ev("response.function_call_arguments.done", map[string]any{ - "item_id": s.tool.ID, "output_index": idx, "arguments": s.tool.Arguments}), - s.ev("response.output_item.done", map[string]any{"output_index": idx, "item": s.tool}), - } - s.items = append(s.items, s.tool) - return evs -} - -// itemID 按当前 output 序号派生确定性 item ID。 -func (s *RespStream) itemID(kind string) string { - return respItemID(s.id, kind, len(s.items)) -} diff --git a/internal/service/airesponses_test.go b/internal/service/airesponses_test.go index 208114a..33acfc2 100644 --- a/internal/service/airesponses_test.go +++ b/internal/service/airesponses_test.go @@ -17,47 +17,6 @@ func respReq(t *testing.T, raw string) aiwire.RespRequest { return req } -func TestResponsesToIR(t *testing.T) { - req := respReq(t, `{ - "model": "meta.llama-3.3-70b-instruct", - "instructions": "你是助手", - "max_output_tokens": 128, - "input": [ - {"role": "user", "content": "查天气"}, - {"type": "function_call", "call_id": "call_1", "name": "get_weather", "arguments": "{\"city\":\"上海\"}"}, - {"type": "function_call_output", "call_id": "call_1", "output": "{\"temp\":31}"}, - {"type": "message", "role": "assistant", "content": [{"type": "output_text", "text": "31 度"}]} - ], - "tools": [{"type": "function", "name": "get_weather", "description": "查天气", "parameters": {"type": "object"}, "strict": true}], - "tool_choice": {"type": "function", "name": "get_weather"}, - "text": {"format": {"type": "json_schema", "name": "out", "schema": {"type": "object"}}} - }`) - ir, err := ResponsesToIR(req) - if err != nil { - t.Fatalf("ResponsesToIR: %v", err) - } - roles := make([]string, 0, len(ir.Messages)) - for _, m := range ir.Messages { - roles = append(roles, m.Role) - } - want := []string{"system", "user", "assistant", "tool", "assistant"} - if strings.Join(roles, ",") != strings.Join(want, ",") { - t.Errorf("roles = %v, want %v", roles, want) - } - if ir.Messages[2].ToolCalls[0].ID != "call_1" || ir.Messages[3].ToolCallID != "call_1" { - t.Errorf("tool call 链接错误: %+v", ir.Messages) - } - if deref(ir.MaxTokens) != 128 || len(ir.Tools) != 1 || ir.Tools[0].Function.Name != "get_weather" { - t.Errorf("参数映射错误: max=%v tools=%+v", ir.MaxTokens, ir.Tools) - } - if !strings.Contains(string(ir.ToolChoice), `"function"`) || !strings.Contains(string(ir.ToolChoice), "get_weather") { - t.Errorf("tool_choice = %s", ir.ToolChoice) - } - if ir.ResponseFormat == nil || ir.ResponseFormat.Type != "json_schema" { - t.Errorf("response_format = %+v", ir.ResponseFormat) - } -} - func deref(p *int) int { if p == nil { return 0 @@ -65,112 +24,124 @@ func deref(p *int) int { return *p } -func TestResponsesToIRRejects(t *testing.T) { - cases := []struct{ name, raw string }{ - {"previous_response_id", `{"model":"m","input":"hi","previous_response_id":"resp_x"}`}, - {"conversation", `{"model":"m","input":"hi","conversation":{"id":"conv_1"}}`}, - {"background", `{"model":"m","input":"hi","background":true}`}, - {"builtin tool", `{"model":"m","input":"hi","tools":[{"type":"web_search"}]}`}, - {"file_id image", `{"model":"m","input":[{"role":"user","content":[{"type":"input_image","file_id":"file_1"}]}]}`}, - {"unknown item", `{"model":"m","input":[{"type":"item_reference","id":"x"}]}`}, +func TestRespServerTools(t *testing.T) { + tests := []struct { + name string + tools []aiwire.RespTool + want bool + }{ + {"web_search", []aiwire.RespTool{{Type: "web_search"}}, true}, + {"x_search混用", []aiwire.RespTool{{Type: "function", Name: "f"}, {Type: "x_search"}}, true}, + {"仅function", []aiwire.RespTool{{Type: "function", Name: "f"}}, false}, + {"空", nil, false}, + {"其他类型", []aiwire.RespTool{{Type: "code_interpreter"}}, false}, } - for _, c := range cases { - if _, err := ResponsesToIR(respReq(t, c.raw)); err == nil { - t.Errorf("%s 应被拒绝", c.name) - } + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + if got := RespServerTools(test.tools); got != test.want { + t.Fatalf("got %v, want %v", got, test.want) + } + }) } - // input 字符串形态 + store/reasoning 忽略项不报错 - req := respReq(t, `{"model":"m","input":"你好","store":false,"reasoning":{"effort":"low"},"parallel_tool_calls":true}`) - ir, err := ResponsesToIR(req) - if err != nil || len(ir.Messages) != 1 || ir.Messages[0].Role != "user" || ir.ReasoningEffort != "low" { - t.Errorf("字符串 input = %+v, %v", ir.Messages, err) +} + +func TestRespPassthroughValidate(t *testing.T) { + prev := "resp_1" + bg := true + tests := []struct { + name string + req aiwire.RespRequest + wantErr bool + }{ + {"合法", aiwire.RespRequest{Model: "m", Tools: []aiwire.RespTool{{Type: "web_search"}}}, false}, + {"混用function", aiwire.RespRequest{Model: "m", Tools: []aiwire.RespTool{{Type: "web_search"}, {Type: "function", Name: "f"}}}, false}, + {"缺model", aiwire.RespRequest{Tools: []aiwire.RespTool{{Type: "web_search"}}}, true}, + {"工具加流式拒绝", aiwire.RespRequest{Model: "m", Stream: true, Tools: []aiwire.RespTool{{Type: "web_search"}}}, true}, + {"无工具流式放行", aiwire.RespRequest{Model: "m", Stream: true}, false}, + {"function工具流式放行", aiwire.RespRequest{Model: "m", Stream: true, Tools: []aiwire.RespTool{{Type: "function", Name: "f"}}}, false}, + {"有状态拒绝", aiwire.RespRequest{Model: "m", PreviousResponseID: prev}, true}, + {"background拒绝", aiwire.RespRequest{Model: "m", Background: &bg}, true}, + {"未知工具拒绝", aiwire.RespRequest{Model: "m", Tools: []aiwire.RespTool{{Type: "web_search"}, {Type: "mcp"}}}, true}, } - // input_image(url 形态)放行并保留图文顺序 - req2 := respReq(t, `{"model":"m","input":[{"role":"user","content":[{"type":"input_text","text":"看图"},{"type":"input_image","image_url":"https://x/1.png","detail":"low"}]}]}`) - ir2, err := ResponsesToIR(req2) + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + err := RespPassthroughValidate(test.req) + if (err != nil) != test.wantErr { + t.Fatalf("err = %v, wantErr %v", err, test.wantErr) + } + }) + } +} + +func TestRespPassthroughBody(t *testing.T) { + raw := []byte(`{"model":"m","input":"hi","stream":true,"store":true,"max_output_tokens":128,"custom_field":{"a":1.5}}`) + out, err := RespPassthroughBody(raw) if err != nil { - t.Fatalf("input_image 应放行: %v", err) + t.Fatalf("RespPassthroughBody: %v", err) } - parts := ir2.Messages[0].Content.Parts - if len(parts) != 2 || parts[1].Type != "image_url" || parts[1].ImageURL == nil || parts[1].ImageURL.URL != "https://x/1.png" { - t.Errorf("parts = %+v", parts) + var body map[string]any + if err := json.Unmarshal(out, &body); err != nil { + t.Fatalf("unmarshal out: %v", err) + } + if body["store"] != false { + t.Errorf("store 应强制 false, got %v", body["store"]) + } + if body["stream"] != true { + t.Errorf("stream 应原样保留, got %v", body["stream"]) + } + if string(out) == "" || !strings.Contains(string(out), `"max_output_tokens":128`) { + t.Errorf("数值字段应保真: %s", out) + } + if !strings.Contains(string(out), `"custom_field"`) { + t.Errorf("未知字段应保留: %s", out) + } + if _, err := RespPassthroughBody([]byte("not-json")); err == nil { + t.Error("非法 JSON 应报错") } } -func TestIRRespToResponses(t *testing.T) { - resp := &aiwire.ChatResponse{Model: "m", Choices: []aiwire.Choice{{ - Message: aiwire.ChatMessage{Role: "assistant", Content: aiwire.NewTextContent("你好"), - ToolCalls: []aiwire.ToolCall{{ID: "call_9", Type: "function", - Function: aiwire.FunctionCall{Name: "f", Arguments: ""}}}}, - FinishReason: "length", - }}, Usage: &aiwire.Usage{PromptTokens: 10, CompletionTokens: 5, TotalTokens: 15}} - out := IRRespToResponses(resp, "resp_abc", 1751966400) - if out.Status != "incomplete" || out.IncompleteDetails == nil || out.IncompleteDetails.Reason != "max_output_tokens" { - t.Errorf("length 应映射 incomplete: %+v", out) +func TestRespPassthroughUsage(t *testing.T) { + tests := []struct { + name string + payload string + wantNil bool + prompt, cached int + }{ + {"完整", `{"usage":{"input_tokens":9,"output_tokens":3,"total_tokens":12,"input_tokens_details":{"cached_tokens":5}}}`, false, 9, 5}, + {"无细分", `{"usage":{"input_tokens":9,"output_tokens":3,"total_tokens":12}}`, false, 9, 0}, + {"无usage", `{"id":"resp_1"}`, true, 0, 0}, + {"非法JSON", `xx`, true, 0, 0}, } - if len(out.Output) != 2 || out.Output[0].Type != "message" || out.Output[1].Type != "function_call" { - t.Fatalf("output = %+v", out.Output) - } - if out.Output[0].Content[0].Text != "你好" || out.Output[1].CallID != "call_9" || out.Output[1].Arguments != "{}" { - t.Errorf("item 装配错误: %+v", out.Output) - } - if out.Usage.InputTokens != 10 || out.Usage.OutputTokens != 5 || out.Store { - t.Errorf("usage/store 错误: %+v", out) + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + usage := RespPassthroughUsage([]byte(test.payload)) + if (usage == nil) != test.wantNil { + t.Fatalf("usage = %v, wantNil %v", usage, test.wantNil) + } + if usage == nil { + return + } + if usage.PromptTokens != test.prompt || usage.CachedTokens() != test.cached { + t.Fatalf("prompt=%d cached=%d, want %d/%d", usage.PromptTokens, usage.CachedTokens(), test.prompt, test.cached) + } + }) } } -func chunkText(text string) aiwire.ChatChunk { - return aiwire.ChatChunk{Choices: []aiwire.ChunkChoice{{Delta: aiwire.Delta{Content: text}}}} -} - -func TestRespStreamTextAndTool(t *testing.T) { - st := NewRespStream("resp_x", "m", 1751966400) - var types []string - collect := func(evs []RespEvent) { - for _, ev := range evs { - types = append(types, ev.Event) +// TestRespStreamCompletedUsage 断言流式 usage 只从 completed 事件提取。 +func TestRespStreamCompletedUsage(t *testing.T) { + completed := []byte(`{"type":"response.completed","response":{"usage":{"input_tokens":10,"output_tokens":5,"total_tokens":15,"input_tokens_details":{"cached_tokens":4}}}}`) + if u := RespStreamCompletedUsage(completed); u == nil || u.PromptTokens != 10 || u.CompletionTokens != 5 || + u.PromptTokensDetails == nil || u.PromptTokensDetails.CachedTokens != 4 { + t.Fatalf("completed 事件 usage 解析失败: %+v", u) + } + for _, data := range []string{ + `{"type":"response.output_text.delta","delta":"hi"}`, + `{"type":"response.completed"}`, + `not-json`, + } { + if u := RespStreamCompletedUsage([]byte(data)); u != nil { + t.Fatalf("非 completed 或缺 response 应返回 nil: %s", data) } } - collect(st.Feed(chunkText("你"))) - collect(st.Feed(chunkText("好"))) - collect(st.Feed(aiwire.ChatChunk{Choices: []aiwire.ChunkChoice{{Delta: aiwire.Delta{ - ToolCalls: []aiwire.ToolCallDelta{{Index: 0, ID: "call_1", Function: aiwire.FunctionCallDelta{Name: "f", Arguments: "{\"a\""}}}}}}})) - collect(st.Feed(aiwire.ChatChunk{Choices: []aiwire.ChunkChoice{{Delta: aiwire.Delta{ - ToolCalls: []aiwire.ToolCallDelta{{Index: 0, Function: aiwire.FunctionCallDelta{Arguments: ":1}"}}}}}}, - Usage: &aiwire.Usage{PromptTokens: 3, CompletionTokens: 7, TotalTokens: 10}})) - finish := st.Finish() - collect(finish) - want := []string{ - "response.created", "response.in_progress", - "response.output_item.added", "response.content_part.added", "response.output_text.delta", - "response.output_text.delta", - "response.output_text.done", "response.content_part.done", "response.output_item.done", - "response.output_item.added", "response.function_call_arguments.delta", - "response.function_call_arguments.delta", - "response.function_call_arguments.done", "response.output_item.done", - "response.completed", - } - if strings.Join(types, "\n") != strings.Join(want, "\n") { - t.Errorf("事件序列 =\n%s\nwant\n%s", strings.Join(types, "\n"), strings.Join(want, "\n")) - } - last := finish[len(finish)-1].Data - resp := last["response"].(aiwire.Response) - if len(resp.Output) != 2 || resp.Output[0].Content[0].Text != "你好" || resp.Output[1].Arguments != "{\"a\":1}" { - t.Errorf("终态快照 = %+v", resp.Output) - } - if resp.Usage == nil || resp.Usage.TotalTokens != 10 { - t.Errorf("终态 usage = %+v", resp.Usage) - } - // sequence_number 严格递增 - if seq, ok := last["sequence_number"].(int); !ok || seq != len(want) { - t.Errorf("最终 sequence_number = %v, want %d", last["sequence_number"], len(want)) - } -} - -func TestRespStreamEmpty(t *testing.T) { - st := NewRespStream("resp_e", "m", 1) - evs := st.Finish() - if len(evs) != 2 || evs[0].Event != "response.created" || evs[1].Event != "response.completed" { - t.Errorf("空流事件 = %+v", evs) - } } diff --git a/internal/service/anthresponses.go b/internal/service/anthresponses.go new file mode 100644 index 0000000..0f8c4fa --- /dev/null +++ b/internal/service/anthresponses.go @@ -0,0 +1,411 @@ +package service + +import ( + "encoding/json" + "fmt" + "strings" + + "oci-portal/internal/aiwire" +) + +// Anthropic Messages ↔ OCI OpenAI 兼容面(/actions/v1/responses)直通转换。 +// typed chat 面剔除后 Messages 入口的唯一上游通路。语义损失(README 已披露): +// stop_sequences / top_k / metadata / thinking 无对应字段,忽略;上游 reasoning +// 输出项与增量事件丢弃(Anthropic thinking 块含签名语义,不伪造)。 + +// ErrAiUnsupportedBlock 表示请求含网关无法承接的内容块(仅支持文本/图片/工具块)。 +var ErrAiUnsupportedBlock = fmt.Errorf("暂不支持文本与图片以外的内容块") + +// AnthropicToResponsesBody 把 Messages 请求转为直通 body(强制 store:false)。 +func AnthropicToResponsesBody(req aiwire.MessagesRequest) ([]byte, error) { + input, err := anthInputItems(req.Messages) + if err != nil { + return nil, err + } + body := map[string]any{"model": req.Model, "max_output_tokens": req.MaxTokens, + "input": input, "store": false} + if sys := req.SystemText(); sys != "" { + body["instructions"] = sys + } + if req.Temperature != nil { + body["temperature"] = *req.Temperature + } + if req.TopP != nil { + body["top_p"] = *req.TopP + } + if tools := anthRespTools(req.Tools); tools != nil { + body["tools"] = tools + } + if tc := anthRespToolChoice(req.ToolChoice); tc != nil { + body["tool_choice"] = tc + } + if req.OutputConfig != nil && req.OutputConfig.Effort != "" { + body["reasoning"] = map[string]string{"effort": strings.ToLower(req.OutputConfig.Effort)} + } + if req.Stream { + body["stream"] = true + } + return json.Marshal(body) +} + +// anthInputItems 把消息序列展开为 Responses input 项:tool_use / tool_result 为 +// 独立 function_call / function_call_output 项,其余聚合为 message 项;遇独立项时 +// 先冲刷已聚合部件,保持块间相对顺序。 +func anthInputItems(messages []aiwire.AnthMessage) ([]any, error) { + var items []any + for _, m := range messages { + var parts []map[string]any + flush := func() { + if len(parts) > 0 { + items = append(items, map[string]any{"role": m.Role, "content": parts}) + parts = nil + } + } + for _, b := range m.Content.AllBlocks() { + item, part, err := anthBlockItem(m.Role, b) + if err != nil { + return nil, err + } + if part != nil { + parts = append(parts, part) + } + if item != nil { + flush() + items = append(items, item) + } + } + flush() + } + return items, nil +} + +// anthBlockItem 把单个内容块转为独立项或消息部件(thinking 忽略,未知块拒绝)。 +func anthBlockItem(role string, b aiwire.AnthBlock) (item, part map[string]any, err error) { + switch b.Type { + case "text": + return nil, anthTextPart(role, b.Text), nil + case "image": + part, err = anthImageInput(b.Source) + return nil, part, err + case "tool_use": + return map[string]any{"type": "function_call", "call_id": b.ID, + "name": b.Name, "arguments": string(b.Input)}, nil, nil + case "tool_result": + return map[string]any{"type": "function_call_output", + "call_id": b.ToolUseID, "output": b.ResultText()}, nil, nil + case "thinking", "redacted_thinking": + return nil, nil, nil + default: + return nil, nil, ErrAiUnsupportedBlock + } +} + +// anthTextPart 按角色选择 Responses 文本部件类型(assistant 历史为 output_text)。 +func anthTextPart(role, text string) map[string]any { + if role == "assistant" { + return map[string]any{"type": "output_text", "text": text} + } + return map[string]any{"type": "input_text", "text": text} +} + +// anthImageInput 把 Anthropic image source 转为 Responses input_image 部件。 +func anthImageInput(source json.RawMessage) (map[string]any, error) { + var src struct { + Type string `json:"type"` + MediaType string `json:"media_type"` + Data string `json:"data"` + URL string `json:"url"` + } + if err := json.Unmarshal(source, &src); err != nil { + return nil, fmt.Errorf("image source 解析失败: %w", err) + } + switch src.Type { + case "base64": + if src.MediaType == "" || src.Data == "" { + return nil, fmt.Errorf("image source 缺少 media_type 或 data") + } + return map[string]any{"type": "input_image", + "image_url": "data:" + src.MediaType + ";base64," + src.Data}, nil + case "url": + if src.URL == "" { + return nil, fmt.Errorf("image source 缺少 url") + } + return map[string]any{"type": "input_image", "image_url": src.URL}, nil + } + return nil, fmt.Errorf("不支持的 image source 类型 %q", src.Type) +} + +func anthRespTools(tools []aiwire.AnthTool) []map[string]any { + if len(tools) == 0 { + return nil + } + out := make([]map[string]any, 0, len(tools)) + for _, t := range tools { + out = append(out, map[string]any{"type": "function", "name": t.Name, + "description": t.Description, "parameters": t.InputSchema}) + } + return out +} + +// anthRespToolChoice 映射 {type:auto|any|tool|none,name} → Responses 形态。 +func anthRespToolChoice(raw json.RawMessage) any { + if len(raw) == 0 { + return nil + } + var tc struct { + Type string `json:"type"` + Name string `json:"name"` + } + if json.Unmarshal(raw, &tc) != nil { + return nil + } + switch tc.Type { + case "auto": + return "auto" + case "any": + return "required" + case "none": + return "none" + case "tool": + return map[string]string{"type": "function", "name": tc.Name} + } + return nil +} + +// respPayload 是直通响应中本转换关心的子集(未知字段忽略)。 +type respPayload struct { + Model string `json:"model"` + Status string `json:"status"` + IncompleteDetails *respIncomplete `json:"incomplete_details"` + Output []respOutputItem `json:"output"` + Usage *aiwire.RespUsage `json:"usage"` + Error *map[string]string `json:"error"` +} + +type respIncomplete struct { + Reason string `json:"reason"` +} + +type respOutputItem struct { + Type string `json:"type"` + CallID string `json:"call_id"` + Name string `json:"name"` + Arguments string `json:"arguments"` + Content []struct { + Type string `json:"type"` + Text string `json:"text"` + } `json:"content"` +} + +// ResponsesToAnthropic 把直通非流式响应转为 Anthropic Messages 响应。 +func ResponsesToAnthropic(payload []byte, msgID string) (*aiwire.MessagesResponse, error) { + var resp respPayload + if err := json.Unmarshal(payload, &resp); err != nil { + return nil, fmt.Errorf("解析上游响应: %w", err) + } + out := &aiwire.MessagesResponse{ID: msgID, Type: "message", Role: "assistant", + Model: resp.Model, Content: []aiwire.AnthBlock{}, StopReason: "end_turn"} + for _, item := range resp.Output { + switch item.Type { + case "message": + for _, part := range item.Content { + if part.Type == "output_text" && part.Text != "" { + out.Content = append(out.Content, aiwire.AnthBlock{Type: "text", Text: part.Text}) + } + } + case "function_call": + out.Content = append(out.Content, aiwire.AnthBlock{Type: "tool_use", ID: item.CallID, + Name: item.Name, Input: argsToJSON(item.Arguments)}) + out.StopReason = "tool_use" + } + } + if resp.Status == "incomplete" && resp.IncompleteDetails != nil && + resp.IncompleteDetails.Reason == "max_output_tokens" { + out.StopReason = "max_tokens" + } + out.Usage = anthUsageFromResp(resp.Usage) + return out, nil +} + +func anthUsageFromResp(u *aiwire.RespUsage) aiwire.AnthUsage { + if u == nil { + return aiwire.AnthUsage{} + } + return aiwire.AnthUsage{InputTokens: u.InputTokens, OutputTokens: u.OutputTokens, + CacheReadInputTokens: u.InputTokensDetails.CachedTokens} +} + +// argsToJSON 保证 tool_use.input 是合法 JSON 对象(模型可能产出非法片段)。 +func argsToJSON(args string) json.RawMessage { + trimmed := strings.TrimSpace(args) + if trimmed == "" { + return json.RawMessage(`{}`) + } + if json.Valid([]byte(trimmed)) { + return json.RawMessage(trimmed) + } + b, _ := json.Marshal(map[string]string{"_raw": args}) + return b +} + +// ---- Anthropic 流式桥:Responses SSE 事件 → Anthropic 事件序列 ---- + +// AnthEvent 是一条待写出的 Anthropic SSE 事件。 +type AnthEvent struct { + Event string + Data any +} + +// AnthRespBridge 把直通 SSE 事件流桥接为 Anthropic 事件序列: +// message_start → content_block_start/delta/stop(text 与 tool_use 分块)→ message_delta → message_stop。 +// reasoning 系列事件丢弃。 +type AnthRespBridge struct { + id, model string + started bool + blockOpen bool + blockIsTool bool + blockIndex int + stopReason string + usage aiwire.AnthUsage +} + +// NewAnthRespBridge 构造桥;id 为响应消息 ID。 +func NewAnthRespBridge(id, model string) *AnthRespBridge { + return &AnthRespBridge{id: id, model: model, blockIndex: -1, stopReason: "end_turn"} +} + +// respStreamEvent 是直通 SSE data JSON 中桥关心的子集。 +type respStreamEvent struct { + Type string `json:"type"` + Delta string `json:"delta"` + Item *respOutputItem `json:"item"` + Response *respPayload `json:"response"` +} + +// Feed 消费一行 SSE data JSON,返回应立即写出的事件。 +func (st *AnthRespBridge) Feed(data []byte) []AnthEvent { + var ev respStreamEvent + if json.Unmarshal(data, &ev) != nil { + return nil + } + var events []AnthEvent + if !st.started { + st.started = true + events = append(events, st.startEvent()) + } + switch ev.Type { + case "response.output_item.added": + if ev.Item != nil && ev.Item.Type == "function_call" { + events = append(events, st.openBlock(true, ev.Item.CallID, ev.Item.Name)...) + st.stopReason = "tool_use" + } + case "response.output_text.delta": + events = append(events, st.textDelta(ev.Delta)...) + case "response.function_call_arguments.delta": + events = append(events, st.argsDelta(ev.Delta)...) + case "response.completed", "response.incomplete", "response.failed": + st.finishFrom(ev.Response) + } + return events +} + +func (st *AnthRespBridge) textDelta(delta string) []AnthEvent { + var events []AnthEvent + if !st.blockOpen || st.blockIsTool { + events = append(events, st.openBlock(false, "", "")...) + } + events = append(events, AnthEvent{Event: "content_block_delta", Data: map[string]any{ + "type": "content_block_delta", "index": st.blockIndex, + "delta": map[string]string{"type": "text_delta", "text": delta}, + }}) + return events +} + +func (st *AnthRespBridge) argsDelta(delta string) []AnthEvent { + if !st.blockOpen || !st.blockIsTool { + return nil + } + return []AnthEvent{{Event: "content_block_delta", Data: map[string]any{ + "type": "content_block_delta", "index": st.blockIndex, + "delta": map[string]string{"type": "input_json_delta", "partial_json": delta}, + }}} +} + +// finishFrom 记录终态:usage 与 stop_reason(max_output_tokens 截断 → max_tokens)。 +func (st *AnthRespBridge) finishFrom(resp *respPayload) { + if resp == nil { + return + } + st.usage = anthUsageFromResp(resp.Usage) + if resp.Status == "incomplete" && resp.IncompleteDetails != nil && + resp.IncompleteDetails.Reason == "max_output_tokens" { + st.stopReason = "max_tokens" + } +} + +func (st *AnthRespBridge) startEvent() AnthEvent { + return AnthEvent{Event: "message_start", Data: map[string]any{ + "type": "message_start", + "message": map[string]any{ + "id": st.id, "type": "message", "role": "assistant", "model": st.model, + "content": []any{}, "stop_reason": nil, + "usage": map[string]int{"input_tokens": 0, "output_tokens": 0}, + }, + }} +} + +// openBlock 关闭当前块并打开新块(text 或 tool_use)。 +func (st *AnthRespBridge) openBlock(isTool bool, toolID, toolName string) []AnthEvent { + var events []AnthEvent + if st.blockOpen { + events = append(events, st.closeBlockEvent()) + } + st.blockOpen, st.blockIsTool = true, isTool + st.blockIndex++ + block := map[string]any{"type": "text", "text": ""} + if isTool { + block = map[string]any{"type": "tool_use", "id": toolID, "name": toolName, "input": map[string]any{}} + } + events = append(events, AnthEvent{Event: "content_block_start", Data: map[string]any{ + "type": "content_block_start", "index": st.blockIndex, "content_block": block, + }}) + return events +} + +func (st *AnthRespBridge) closeBlockEvent() AnthEvent { + return AnthEvent{Event: "content_block_stop", Data: map[string]any{ + "type": "content_block_stop", "index": st.blockIndex, + }} +} + +// Finish 在上游流结束后收尾:关块 → message_delta(stop_reason+usage)→ message_stop。 +func (st *AnthRespBridge) Finish() []AnthEvent { + var events []AnthEvent + if !st.started { + st.started = true + events = append(events, st.startEvent()) + } + if st.blockOpen { + events = append(events, st.closeBlockEvent()) + st.blockOpen = false + } + usage := map[string]int{"output_tokens": st.usage.OutputTokens} + if st.usage.InputTokens > 0 { + usage["input_tokens"] = st.usage.InputTokens + } + if st.usage.CacheReadInputTokens > 0 { + usage["cache_read_input_tokens"] = st.usage.CacheReadInputTokens + } + events = append(events, + AnthEvent{Event: "message_delta", Data: map[string]any{ + "type": "message_delta", + "delta": map[string]any{"stop_reason": st.stopReason, "stop_sequence": nil}, + "usage": usage, + }}, + AnthEvent{Event: "message_stop", Data: map[string]any{"type": "message_stop"}}, + ) + return events +} + +// Usage 返回聚合到的用量(供调用日志)。 +func (st *AnthRespBridge) Usage() aiwire.AnthUsage { return st.usage } diff --git a/internal/service/anthresponses_test.go b/internal/service/anthresponses_test.go new file mode 100644 index 0000000..328a725 --- /dev/null +++ b/internal/service/anthresponses_test.go @@ -0,0 +1,173 @@ +package service + +import ( + "encoding/json" + "strings" + "testing" + + "oci-portal/internal/aiwire" +) + +func mustAnthReq(t *testing.T, raw string) aiwire.MessagesRequest { + t.Helper() + var req aiwire.MessagesRequest + if err := json.Unmarshal([]byte(raw), &req); err != nil { + t.Fatalf("解析请求: %v", err) + } + return req +} + +// TestAnthropicToResponsesBody 断言 system/消息/工具/effort 的直通装配与 store 强制。 +func TestAnthropicToResponsesBody(t *testing.T) { + req := mustAnthReq(t, `{ + "model": "xai.grok-4.3", "max_tokens": 128, "system": "你是助手", "stream": true, + "temperature": 0.5, + "messages": [ + {"role": "user", "content": "东京天气?"}, + {"role": "assistant", "content": [ + {"type": "text", "text": "查询中"}, + {"type": "tool_use", "id": "t1", "name": "get_weather", "input": {"city": "东京"}} + ]}, + {"role": "user", "content": [ + {"type": "tool_result", "tool_use_id": "t1", "content": "晴 25 度"}, + {"type": "text", "text": "继续"} + ]} + ], + "tools": [{"name": "get_weather", "description": "查天气", "input_schema": {"type": "object"}}], + "tool_choice": {"type": "any"}, + "output_config": {"effort": "HIGH"} + }`) + payload, err := AnthropicToResponsesBody(req) + if err != nil { + t.Fatalf("AnthropicToResponsesBody: %v", err) + } + var body map[string]any + if err := json.Unmarshal(payload, &body); err != nil { + t.Fatalf("unmarshal: %v", err) + } + if body["model"] != "xai.grok-4.3" || body["max_output_tokens"] != float64(128) || + body["instructions"] != "你是助手" || body["store"] != false || body["stream"] != true { + t.Fatalf("顶层字段装配错误: %v", body) + } + if body["tool_choice"] != "required" { + t.Fatalf("tool_choice = %v, want required", body["tool_choice"]) + } + reasoning, _ := body["reasoning"].(map[string]any) + if reasoning["effort"] != "high" { + t.Fatalf("effort = %v, want high(小写透传)", reasoning) + } + input, _ := body["input"].([]any) + // user 消息、assistant 文本消息、function_call、function_call_output、末条 user 文本 + if len(input) != 5 { + t.Fatalf("input 项数 = %d, want 5: %s", len(input), payload) + } + kinds := make([]string, 0, len(input)) + for _, it := range input { + m := it.(map[string]any) + if ty, ok := m["type"].(string); ok { + kinds = append(kinds, ty) + } else { + kinds = append(kinds, "message/"+m["role"].(string)) + } + } + want := "message/user,message/assistant,function_call,function_call_output,message/user" + if got := strings.Join(kinds, ","); got != want { + t.Fatalf("input 顺序 = %s, want %s", got, want) + } + if !strings.Contains(string(payload), `"output_text"`) { + t.Fatalf("assistant 历史文本应为 output_text 部件: %s", payload) + } +} + +// TestAnthropicToResponsesBodyRejects 断言不支持的内容块拒绝与图片装配。 +func TestAnthropicToResponsesBodyRejects(t *testing.T) { + bad := mustAnthReq(t, `{"model":"m","max_tokens":8,"messages":[ + {"role":"user","content":[{"type":"document","source":{}}]}]}`) + if _, err := AnthropicToResponsesBody(bad); err == nil { + t.Fatal("document 块应拒绝") + } + img := mustAnthReq(t, `{"model":"m","max_tokens":8,"messages":[ + {"role":"user","content":[{"type":"image","source":{"type":"base64","media_type":"image/png","data":"QUJD"}}]}]}`) + payload, err := AnthropicToResponsesBody(img) + if err != nil || !strings.Contains(string(payload), "data:image/png;base64,QUJD") { + t.Fatalf("图片应转 data URI: %v %s", err, payload) + } +} + +// TestResponsesToAnthropic 断言输出块、stop_reason 与 usage 的回转。 +func TestResponsesToAnthropic(t *testing.T) { + payload := []byte(`{"model":"xai.grok-4.3","status":"completed","output":[ + {"type":"reasoning","summary":[]}, + {"type":"message","content":[{"type":"output_text","text":"你好"}]}, + {"type":"function_call","call_id":"c1","name":"get_weather","arguments":"{\"city\":\"东京\"}"}], + "usage":{"input_tokens":10,"output_tokens":5,"total_tokens":15,"input_tokens_details":{"cached_tokens":4}}}`) + out, err := ResponsesToAnthropic(payload, "msg_1") + if err != nil { + t.Fatalf("ResponsesToAnthropic: %v", err) + } + if len(out.Content) != 2 || out.Content[0].Type != "text" || out.Content[0].Text != "你好" { + t.Fatalf("content 装配错误: %+v", out.Content) + } + if out.Content[1].Type != "tool_use" || out.Content[1].ID != "c1" || string(out.Content[1].Input) != `{"city":"东京"}` { + t.Fatalf("tool_use 装配错误: %+v", out.Content[1]) + } + if out.StopReason != "tool_use" { + t.Fatalf("stop_reason = %s, want tool_use", out.StopReason) + } + if out.Usage.InputTokens != 10 || out.Usage.OutputTokens != 5 || out.Usage.CacheReadInputTokens != 4 { + t.Fatalf("usage = %+v", out.Usage) + } + + trunc := []byte(`{"model":"m","status":"incomplete","incomplete_details":{"reason":"max_output_tokens"}, + "output":[{"type":"message","content":[{"type":"output_text","text":"半"}]}]}`) + out2, err := ResponsesToAnthropic(trunc, "msg_2") + if err != nil || out2.StopReason != "max_tokens" { + t.Fatalf("截断 stop_reason = %v, %v", out2, err) + } +} + +func bridgeEventTypes(events []AnthEvent) string { + kinds := make([]string, 0, len(events)) + for _, ev := range events { + kinds = append(kinds, ev.Event) + } + return strings.Join(kinds, ",") +} + +// TestAnthRespBridge 断言流桥:文本增量、工具调用切块、reasoning 丢弃与收尾事件。 +func TestAnthRespBridge(t *testing.T) { + st := NewAnthRespBridge("msg_1", "m1") + var events []AnthEvent + feed := func(lines ...string) { + for _, l := range lines { + events = append(events, st.Feed([]byte(l))...) + } + } + feed(`{"type":"response.created","response":{"model":"m1"}}`, + `{"type":"response.reasoning_text.delta","delta":"思考中"}`, + `{"type":"response.output_text.delta","delta":"你"}`, + `{"type":"response.output_text.delta","delta":"好"}`, + `{"type":"response.output_item.added","item":{"type":"function_call","call_id":"c1","name":"f"}}`, + `{"type":"response.function_call_arguments.delta","delta":"{\"a\":1}"}`, + `{"type":"response.completed","response":{"status":"completed","usage":{"input_tokens":8,"output_tokens":4}}}`) + events = append(events, st.Finish()...) + + got := bridgeEventTypes(events) + want := "message_start,content_block_start,content_block_delta,content_block_delta," + + "content_block_stop,content_block_start,content_block_delta,content_block_stop," + + "message_delta,message_stop" + if got != want { + t.Fatalf("事件序列:\n got %s\nwant %s", got, want) + } + if st.Usage().InputTokens != 8 || st.Usage().OutputTokens != 4 { + t.Fatalf("usage = %+v", st.Usage()) + } +} + +// TestAnthRespBridgeEmpty 断言空流也产出完整事件骨架。 +func TestAnthRespBridgeEmpty(t *testing.T) { + st := NewAnthRespBridge("msg_1", "m1") + if got := bridgeEventTypes(st.Finish()); got != "message_start,message_delta,message_stop" { + t.Errorf("空流事件序列 = %s", got) + } +}