AI 网关切换 OpenAI 兼容面并移除 chat 端点,新增模型黑白名单
CI / test (push) Successful in 31s
Release / release (push) Successful in 54s

This commit is contained in:
2026-07-12 17:48:28 +08:00
parent 7706f59549
commit 489cb49cb3
34 changed files with 2602 additions and 2901 deletions
+12 -3
View File
@@ -2,19 +2,28 @@
格式遵循 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/),版本号遵循语义化版本。 格式遵循 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/),版本号遵循语义化版本。
## [0.2.0] - 2026-07-10 ## [0.2.0] - 2026-07-12
### Added ### Added
- AI 网关模型池自愈:同步与每日探测后自动逐个试调验证模型可用性(maxTokens=1),不可按需调用的模型(微调基座 / 区域未开放按需供给)自动标记并从模型列表、路由与探测候选中剔除;已标记模型每 20 小时复检,OCI 恢复供给后自动解除标记。`ai_model_caches` 新增 `unusable` / `unusable_reason` / `checked_at`(启动自动迁移),渠道详情接口保留标记行与原因便于排查 - AI 网关模型黑名单:面板集中拉黑模型,全渠道剔除(模型列表 / 路由 / 探测候选不再出现,同步不入库),移除条目并重新同步后恢复;新表 `ai_model_blacklists`(启动自动迁移),管理接口 `/api/v1/ai-blacklist`
- AI 密钥模型白名单:密钥可限定放行的模型列表(空 = 不限),白名单外调用 404,模型列表只返回交集
- 推理力度(effort)直通:Responses `reasoning.effort` 与 Anthropic Messages `output_config.effort` 原样透传上游、不做档位校验;grok-4.3 五档(`none` 完全关闭思考)、gpt-oss `minimal`-`high`、multi-agent `xhigh`(控制并行 agent 数)实测生效
- xAI Grok 服务端工具:`/ai/v1/responses``tools` 支持 `{"type":"web_search"}` / `{"type":"x_search"}`(官方 Agent Tools 格式,可与 `function` 混用,仅非流式),响应保留 `web_search_call` 输出项与引用
### Changed ### Changed
- **AI 网关对话链路整体切换到 OCI OpenAI 兼容面**`/actions/v1/responses`),typed chat 面全链路移除:`/ai/v1/responses` 请求体原样直通(仅强制 `store:false`),流式为上游原生事件流,gpt-oss 推理增量直达客户端;`/ai/v1/messages` 转换为 Responses 请求直通,流式经事件桥转回标准 Anthropic 事件序列,`stop_sequences` / `top_k` 等无兼容面对应物的字段改为静默忽略,思考输出不转 `thinking` 块;渠道探测改用兼容面最小请求;流式建立后绑定渠道,中断不重试不计熔断
- 后台任务「立即执行」改异步触发:接口立即返回 202,执行结果经任务日志轮询呈现;同一任务在途时重复触发返回 409「任务正在执行中」,与 cron 重叠触发静默跳过(此前同步阻塞数十秒且同一任务可并发重复执行) - 后台任务「立即执行」改异步触发:接口立即返回 202,执行结果经任务日志轮询呈现;同一任务在途时重复触发返回 409「任务正在执行中」,与 cron 重叠触发静默跳过(此前同步阻塞数十秒且同一任务可并发重复执行)
- 渠道探测候选跨厂商分散(上限 8,voice 等不可对话形态排除):单一厂商在区域内全为微调基座时不再拖垮整个渠道的探测结论 - 渠道探测候选跨厂商分散(上限 8,voice 等不可对话形态排除):单一厂商在区域内全为微调基座时不再拖垮整个渠道的探测结论
- 网关调用(对话 / 流式 / 向量化)遇「模型不可按需调用」类错误(微调基座 400 / 实体不存在 404)自动标记该模型并换渠道重试且不计入渠道熔断;仅鉴权类错误才判定租户无配额 - 网关调用遇「模型不可按需调用」类错误(微调基座 400 / 实体不存在 404)自动换渠道重试且不计入渠道熔断,错误信息带模型名便于加入黑名单;仅鉴权类错误才判定租户无配额
- 同名模型多条目去重优先保留非微调基座条目,降低缓存到不可调用 OCID 的概率 - 同名模型多条目去重优先保留非微调基座条目,降低缓存到不可调用 OCID 的概率
### Removed
- **`/ai/v1/chat/completions` 端点移除**(404):OpenAI 官方新能力已集中到 Responses 且本面板真实流量为零,请迁移 `/ai/v1/responses``/ai/v1/messages`
- google / cohere 对话模型不再供给(OCI OpenAI 兼容面不承接,上游 400cohere embed 向量模型不受影响),模型目录按 `xai.` / `meta.` / `openai.` 厂商前缀过滤
### Fixed ### Fixed
- 渠道「同步模型成功但探测报错不可用」:法兰克福等区域的微调基座模型占满探测候选导致的误报(探测状态与真实可用性不符的根因) - 渠道「同步模型成功但探测报错不可用」:法兰克福等区域的微调基座模型占满探测候选导致的误报(探测状态与真实可用性不符的根因)
+1 -1
View File
@@ -1 +1 @@
v0.1.0 v0.2.0
+83 -2
View File
@@ -34,7 +34,7 @@
- **资源管理**:实例(含创建/电源操作/更换公网 IP/IPv6/VNIC/引导卷)、VCN/子网/安全列表、块存储、限额与成本查询,多区域/多区间支持 - **资源管理**:实例(含创建/电源操作/更换公网 IP/IPv6/VNIC/引导卷)、VCN/子网/安全列表、块存储、限额与成本查询,多区域/多区间支持
- **抢机任务**:cron 周期尝试创建实例直到成功,支持熔断与通知 - **抢机任务**:cron 周期尝试创建实例直到成功,支持熔断与通知
- **网页控制台**:实例串行控制台(xterm)与 VNC(noVNC),两跳 SSH 隧道 - **网页控制台**:实例串行控制台(xterm)与 VNC(noVNC),两跳 SSH 隧道
- **AI 网关**OpenAI / Anthropic 兼容端点转发 OCI GenAI(对话/Responses/Embeddings),号池渠道加权负载均衡、熔断探测、密钥管理与用量日志 - **AI 网关**OpenAI Responses / Anthropic Messages / Embeddings 兼容端点转发 OCI GenAI,号池渠道加权负载均衡、熔断探测、密钥管理与用量日志;对话统一走 OCI OpenAI 兼容面(xai / meta / openai 厂商模型,流式事件与推理增量原样保真),支持 xAI Grok 服务端工具 `web_search` / `x_search`(格式同 xAI 官方 Agent Tools,仅非流式)——兼容面属实测可用但未见 Oracle 文档承诺的能力,行为可能随上游调整
- **日志回传**OCI Audit 事件经 Connector Hub → Notifications HTTPS 订阅回传入库,一键创建链路;自定义告警规则(事件类型/来源 IP 白名单/资源/频率阈值)命中即推送 - **日志回传**OCI Audit 事件经 Connector Hub → Notifications HTTPS 订阅回传入库,一键创建链路;自定义告警规则(事件类型/来源 IP 白名单/资源/频率阈值)命中即推送
- **租户治理**IAM 用户/MFA/API Key 管理、密码策略、身份提供商(SAML)、通知收件人;多 Identity Domain 租户可按域切换管理 - **租户治理**IAM 用户/MFA/API Key 管理、密码策略、身份提供商(SAML)、通知收件人;多 Identity Domain 租户可按域切换管理
- **安全**JWT + bcrypt(凭据变更旧令牌立即失效,可一键撤销全部会话)、TOTP 两步验证、OIDC/GitHub 外部登录、登录锁定、IP 限速、真实 IP 头可配、请求体/超时防护、系统操作审计 - **安全**JWT + bcrypt(凭据变更旧令牌立即失效,可一键撤销全部会话)、TOTP 两步验证、OIDC/GitHub 外部登录、登录锁定、IP 限速、真实 IP 头可配、请求体/超时防护、系统操作审计
@@ -60,7 +60,7 @@ mkdir -p ./data && sudo chown 65532:65532 ./data
DATA_KEY=$(openssl rand -hex 32) JWT_SECRET=$(openssl rand -hex 32) ADMIN_PASSWORD=<初始密码> docker compose up -d DATA_KEY=$(openssl rand -hex 32) JWT_SECRET=$(openssl rand -hex 32) ADMIN_PASSWORD=<初始密码> docker compose up -d
``` ```
默认只监听 `127.0.0.1:18888`;公网访问须置于 TLS 反向代理之后,见下文「反向代理」。 默认只监听 `127.0.0.1:18888`公网访问须置于 TLS 反向代理之后见下文「反向代理」。
### 源码构建(单文件,含前端) ### 源码构建(单文件,含前端)
@@ -148,6 +148,87 @@ services:
- 反代默认追加的 `X-Forwarded-For` 用于还原真实客户端 IP,系统日志留痕、登录锁定与 IP 限速都依赖它 - 反代默认追加的 `X-Forwarded-For` 用于还原真实客户端 IP,系统日志留痕、登录锁定与 IP 限速都依赖它
- 请求体上限建议与后端一致:`/api/*` 1MB、`/ai/*` 10MBCaddy 用 `request_body` 按前缀分层) - 请求体上限建议与后端一致:`/api/*` 1MB、`/ai/*` 10MBCaddy 用 `request_body` 按前缀分层)
## AI 网关接口
对外提供 OpenAI / Anthropic 兼容端点,转发 OCI GenAI On-Demand 推理。鉴权用面板创建的 AI 密钥,`Authorization: Bearer sk-...``x-api-key: sk-...` 双头均可;密钥可绑定渠道分组实现路由隔离,也可配置模型白名单(白名单外调用 404,模型列表只返回交集)。请求体上限 10MB。
| 端点 | 协议 | 流式 |
| --- | --- | --- |
| `POST /ai/v1/responses` | OpenAI Responses(无状态子集) | SSE 支持(服务端工具除外) |
| `POST /ai/v1/messages` | Anthropic Messages | SSE 支持 |
| `POST /ai/v1/embeddings` | OpenAI Embeddings | — |
| `GET /ai/v1/models` | OpenAI 模型列表(来自渠道模型目录) | — |
对话端点统一转发 OCI OpenAI 兼容面(`/actions/v1/responses`),因此仅提供该面支持的 `xai.` / `meta.` / `openai.` 前缀模型;google / cohere 对话模型不在兼容面供给(上游 400),不再提供(cohere embed 模型不受影响)。兼容面属实测可用但无 Oracle 文档承诺的能力,行为可能随上游调整。**Chat Completions`/ai/v1/chat/completions`)端点已移除**,请迁移到 Responses 或 Messages。
以下各表对照网关行为:✅ 转发上游;⚠️ 接受但忽略(静默丢弃,不影响请求);❌ 拒绝(400,不发上游)。
### Responses 字段
请求体除下列例外**原样直通上游**(未列字段一并转发,效果由上游决定):
| 字段 | 支持 | 说明 |
| --- | --- | --- |
| `model``input` | ✅ | `input` 接受 string 或 item 数组 |
| `stream` | ✅ | SSE,上游原生事件流(含 gpt-oss 推理增量);与服务端工具互斥 |
| `reasoning.effort` | ✅ | 直达上游不限档位,见下「推理力度」 |
| `store` | ⚠️ | 网关无状态,强制改写为 `false` |
| `previous_response_id``conversation``background` | ❌ | 网关不保存历史,请求需自带完整上下文 |
| `web_search` / `x_search` / `function` 之外的工具类型 | ❌ | 如 `file_search``code_interpreter``mcp` |
**服务端工具**`tools` 支持 `{"type":"web_search"}``{"type":"x_search"}`xAI Grok 官方 Agent Tools 格式,可与 `function` 混用),响应保留 `web_search_call` 输出项与引用;**仅非流式**。响应 `usage.input_tokens_details.cached_tokens` 透传缓存命中量。
### Anthropic Messages 字段
网关把 Messages 请求转换为 Responses 请求送上游,响应(含流式事件序列)转回 Anthropic 形态:
| 字段 | 支持 | 说明 |
| --- | --- | --- |
| `model``max_tokens``messages` | ✅ | `max_tokens` 必填;content 块支持 `text` / `image` / `tool_use` / `tool_result`,其余块类型 400 |
| `system` | ✅ | string 或块数组,映射为 `instructions` |
| `temperature``top_p` | ✅ | |
| `stream` | ✅ | SSE,标准 Anthropic 事件序列(`message_start``content_block_*``message_delta``message_stop` |
| `tools``tool_choice` | ✅ | 映射为 Responses `function` 工具;`tool_choice` 支持 `auto` / `any` / `none` / `tool` |
| `output_config.effort` | ✅ | 直达上游不限档位,见下「推理力度」 |
| `stop_sequences``top_k``metadata``thinking``service_tier` | ⚠️ | Responses 面无对应物,静默忽略 |
模型的思考输出(reasoning)不转为 `thinking` 块,静默丢弃;`usage.cache_read_input_tokens` 透传缓存命中量。
### Embeddings 字段
| 字段 | 支持 | 说明 |
| --- | --- | --- |
| `model``input` | ✅ | `input` 接受 string 或数组 |
| `dimensions` | ✅ | |
| `encoding_format``user` | ⚠️ | 输出恒为 float 数组 |
### 推理力度(effort
两个对话协议各以原生字段控制推理深度,取值直达上游不做档位校验:
| 协议 | 字段 |
| --- | --- |
| Responses | `reasoning.effort` |
| Anthropic Messages | `output_config.effort`(转小写后透传) |
是否支持、可用档位与实际效果由模型决定(2026-07 兼容面实测):
| 模型 | 支持档位 | 实测行为 |
| --- | --- | --- |
| `xai.grok-4.3` | `none` / `minimal` / `low` / `medium` / `high` | 全部生效:`none` 完全关闭推理,默认 `low``high` 推理量最大 |
| `openai.gpt-oss-*` | `minimal` / `low` / `medium` / `high` | `none` 被上游 400(Harmony 格式不支持);思考计入 completion tokens |
| `xai.grok-3-mini(-fast)` | 名义 `low` / `high` | 各档均被接受,但 `none` 并不关闭推理(仍产生思考) |
| `meta.llama-*` | 接受任意档 | 非推理模型,参数无实际效果 |
| `xai.grok-4.20-multi-agent*` | `low` / `medium` / `high` / `xhigh` | effort 控制并行 agent 数(4 / 16)而非推理深度 |
| 其余 grok`grok-3``grok-4``grok-4-fast-*``grok-4-1-fast-*``grok-code-fast-1``grok-4.20-*` 单/双型号全系) | 不支持 | 携带即被上游 400(`does not support parameter reasoningEffort` |
不携带该字段时网关不下发,行为由模型默认档位决定。
### 通用行为
- 未知模型 404;无可用渠道 503;单次请求最多尝试 3 个渠道(429/5xx/网络错误自动换渠道并计入熔断,模型级 400/404 换渠道不计熔断,其余 4xx 原样透传);流式建立后绑定渠道,中断不重试
- 调用计量(渠道、token 用量、缓存命中、时延、重试数)写入面板「调用日志」;密钥可按需开启限时内容抓取
## 环境变量 ## 环境变量
| 变量 | 必填 | 默认值 | 说明 | | 变量 | 必填 | 默认值 | 说明 |
+82 -30
View File
@@ -15,34 +15,6 @@ const docTemplate = `{
"host": "{{.Host}}", "host": "{{.Host}}",
"basePath": "{{.BasePath}}", "basePath": "{{.BasePath}}",
"paths": { "paths": {
"/ai/v1/chat/completions": {
"post": {
"tags": [
"AI 网关"
],
"summary": "OpenAI 兼容对话补全",
"parameters": [
{
"description": "OpenAI chat/completions 请求体(支持 stream)",
"name": "body",
"in": "body",
"required": true,
"schema": {
"type": "object"
}
}
],
"responses": {
"200": {
"description": "OpenAI 兼容响应(流式为 SSE)",
"schema": {
"type": "object",
"additionalProperties": true
}
}
}
}
},
"/ai/v1/embeddings": { "/ai/v1/embeddings": {
"post": { "post": {
"tags": [ "tags": [
@@ -79,7 +51,7 @@ const docTemplate = `{
"summary": "Anthropic Messages 兼容端点", "summary": "Anthropic Messages 兼容端点",
"parameters": [ "parameters": [
{ {
"description": "Anthropic messages 请求体(支持 stream)", "description": "Anthropic messages 请求体(支持 stream;经 OCI OpenAI 兼容面直通)",
"name": "body", "name": "body",
"in": "body", "in": "body",
"required": true, "required": true,
@@ -124,7 +96,7 @@ const docTemplate = `{
"summary": "OpenAI Responses 兼容端点", "summary": "OpenAI Responses 兼容端点",
"parameters": [ "parameters": [
{ {
"description": "OpenAI responses 请求体(支持 stream)", "description": "OpenAI responses 请求体(支持 stream;web_search/x_search 服务端工具仅非流式)",
"name": "body", "name": "body",
"in": "body", "in": "body",
"required": true, "required": true,
@@ -166,6 +138,86 @@ const docTemplate = `{
} }
} }
}, },
"/api/v1/ai-blacklist": {
"get": {
"security": [
{
"BearerAuth": []
}
],
"tags": [
"AI 管理"
],
"summary": "模型黑名单列表",
"responses": {
"200": {
"description": "OK",
"schema": {
"type": "object",
"additionalProperties": true
}
}
}
},
"post": {
"security": [
{
"BearerAuth": []
}
],
"tags": [
"AI 管理"
],
"summary": "添加模型黑名单",
"parameters": [
{
"description": "请求体:{name: 模型名}",
"name": "body",
"in": "body",
"required": true,
"schema": {
"type": "object"
}
}
],
"responses": {
"201": {
"description": "Created",
"schema": {
"type": "object",
"additionalProperties": true
}
}
}
}
},
"/api/v1/ai-blacklist/{id}": {
"delete": {
"security": [
{
"BearerAuth": []
}
],
"tags": [
"AI 管理"
],
"summary": "移除模型黑名单(重新同步后模型恢复入池)",
"parameters": [
{
"type": "integer",
"description": "黑名单条目 ID",
"name": "id",
"in": "path",
"required": true
}
],
"responses": {
"204": {
"description": "无内容"
}
}
}
},
"/api/v1/ai-channels": { "/api/v1/ai-channels": {
"get": { "get": {
"security": [ "security": [
+82 -30
View File
@@ -8,34 +8,6 @@
}, },
"basePath": "/", "basePath": "/",
"paths": { "paths": {
"/ai/v1/chat/completions": {
"post": {
"tags": [
"AI 网关"
],
"summary": "OpenAI 兼容对话补全",
"parameters": [
{
"description": "OpenAI chat/completions 请求体(支持 stream)",
"name": "body",
"in": "body",
"required": true,
"schema": {
"type": "object"
}
}
],
"responses": {
"200": {
"description": "OpenAI 兼容响应(流式为 SSE)",
"schema": {
"type": "object",
"additionalProperties": true
}
}
}
}
},
"/ai/v1/embeddings": { "/ai/v1/embeddings": {
"post": { "post": {
"tags": [ "tags": [
@@ -72,7 +44,7 @@
"summary": "Anthropic Messages 兼容端点", "summary": "Anthropic Messages 兼容端点",
"parameters": [ "parameters": [
{ {
"description": "Anthropic messages 请求体(支持 stream)", "description": "Anthropic messages 请求体(支持 stream;经 OCI OpenAI 兼容面直通)",
"name": "body", "name": "body",
"in": "body", "in": "body",
"required": true, "required": true,
@@ -117,7 +89,7 @@
"summary": "OpenAI Responses 兼容端点", "summary": "OpenAI Responses 兼容端点",
"parameters": [ "parameters": [
{ {
"description": "OpenAI responses 请求体(支持 stream)", "description": "OpenAI responses 请求体(支持 stream;web_search/x_search 服务端工具仅非流式)",
"name": "body", "name": "body",
"in": "body", "in": "body",
"required": true, "required": true,
@@ -159,6 +131,86 @@
} }
} }
}, },
"/api/v1/ai-blacklist": {
"get": {
"security": [
{
"BearerAuth": []
}
],
"tags": [
"AI 管理"
],
"summary": "模型黑名单列表",
"responses": {
"200": {
"description": "OK",
"schema": {
"type": "object",
"additionalProperties": true
}
}
}
},
"post": {
"security": [
{
"BearerAuth": []
}
],
"tags": [
"AI 管理"
],
"summary": "添加模型黑名单",
"parameters": [
{
"description": "请求体:{name: 模型名}",
"name": "body",
"in": "body",
"required": true,
"schema": {
"type": "object"
}
}
],
"responses": {
"201": {
"description": "Created",
"schema": {
"type": "object",
"additionalProperties": true
}
}
}
}
},
"/api/v1/ai-blacklist/{id}": {
"delete": {
"security": [
{
"BearerAuth": []
}
],
"tags": [
"AI 管理"
],
"summary": "移除模型黑名单(重新同步后模型恢复入池)",
"parameters": [
{
"type": "integer",
"description": "黑名单条目 ID",
"name": "id",
"in": "path",
"required": true
}
],
"responses": {
"204": {
"description": "无内容"
}
}
}
},
"/api/v1/ai-channels": { "/api/v1/ai-channels": {
"get": { "get": {
"security": [ "security": [
+50 -20
View File
@@ -599,24 +599,6 @@ info:
title: OCI Portal API title: OCI Portal API
version: 0.0.1 version: 0.0.1
paths: paths:
/ai/v1/chat/completions:
post:
parameters:
- description: OpenAI chat/completions 请求体(支持 stream)
in: body
name: body
required: true
schema:
type: object
responses:
"200":
description: OpenAI 兼容响应(流式为 SSE)
schema:
additionalProperties: true
type: object
summary: OpenAI 兼容对话补全
tags:
- AI 网关
/ai/v1/embeddings: /ai/v1/embeddings:
post: post:
parameters: parameters:
@@ -638,7 +620,7 @@ paths:
/ai/v1/messages: /ai/v1/messages:
post: post:
parameters: parameters:
- description: Anthropic messages 请求体(支持 stream) - description: Anthropic messages 请求体(支持 stream;经 OCI OpenAI 兼容面直通)
in: body in: body
name: body name: body
required: true required: true
@@ -667,7 +649,7 @@ paths:
/ai/v1/responses: /ai/v1/responses:
post: post:
parameters: parameters:
- description: OpenAI responses 请求体(支持 stream) - description: OpenAI responses 请求体(支持 stream;web_search/x_search 服务端工具仅非流式)
in: body in: body
name: body name: body
required: true required: true
@@ -695,6 +677,54 @@ paths:
summary: 返回构建与运行环境信息,「设置 · 关于」页展示 summary: 返回构建与运行环境信息,「设置 · 关于」页展示
tags: tags:
- 设置 - 设置
/api/v1/ai-blacklist:
get:
responses:
"200":
description: OK
schema:
additionalProperties: true
type: object
security:
- BearerAuth: []
summary: 模型黑名单列表
tags:
- AI 管理
post:
parameters:
- description: '请求体:{name: 模型名}'
in: body
name: body
required: true
schema:
type: object
responses:
"201":
description: Created
schema:
additionalProperties: true
type: object
security:
- BearerAuth: []
summary: 添加模型黑名单
tags:
- AI 管理
/api/v1/ai-blacklist/{id}:
delete:
parameters:
- description: 黑名单条目 ID
in: path
name: id
required: true
type: integer
responses:
"204":
description: 无内容
security:
- BearerAuth: []
summary: 移除模型黑名单(重新同步后模型恢复入池)
tags:
- AI 管理
/api/v1/ai-channels: /api/v1/ai-channels:
get: get:
responses: responses:
+7
View File
@@ -20,6 +20,13 @@ type MessagesRequest struct {
ToolChoice json.RawMessage `json:"tool_choice,omitempty"` ToolChoice json.RawMessage `json:"tool_choice,omitempty"`
Metadata json.RawMessage `json:"metadata,omitempty"` Metadata json.RawMessage `json:"metadata,omitempty"`
Thinking json.RawMessage `json:"thinking,omitempty"` Thinking json.RawMessage `json:"thinking,omitempty"`
// OutputConfig 是 Anthropic 官方输出控制(effort 等);网关映射 effort 到 OCI reasoningEffort。
OutputConfig *AnthOutputConfig `json:"output_config,omitempty"`
}
// AnthOutputConfig 是 Anthropic Messages 顶层 output_config 对象。
type AnthOutputConfig struct {
Effort string `json:"effort,omitempty"`
} }
// SystemText 把顶层 system(string 或块数组)拍平为文本。 // SystemText 把顶层 system(string 或块数组)拍平为文本。
+2 -209
View File
@@ -1,35 +1,11 @@
// Package aiwire 定义 AI 网关的线格式类型:OpenAI Chat Completions(兼作网关内部 // Package aiwire 定义 AI 网关的线格式类型:OpenAI Responses / Anthropic Messages /
// 中间表示 IR,因 OCI GENERIC 格式即其镜像)与 Anthropic Messages。零内部依赖, // Embeddings 与通用记账、错误、模型列表结构。上游统一为 OCI OpenAI 兼容面。
// oci 层与 service 层共同引用。
package aiwire package aiwire
import ( import (
"encoding/json" "encoding/json"
"strings"
) )
// ChatRequest 是 OpenAI /v1/chat/completions 请求体;TopK 为 OCI/Anthropic 扩展字段。
type ChatRequest struct {
Model string `json:"model"`
Messages []ChatMessage `json:"messages"`
MaxTokens *int `json:"max_tokens,omitempty"`
MaxCompletionTokens *int `json:"max_completion_tokens,omitempty"`
Temperature *float64 `json:"temperature,omitempty"`
TopP *float64 `json:"top_p,omitempty"`
TopK *int `json:"top_k,omitempty"`
FrequencyPenalty *float64 `json:"frequency_penalty,omitempty"`
PresencePenalty *float64 `json:"presence_penalty,omitempty"`
Stop StringList `json:"stop,omitempty"`
Seed *int `json:"seed,omitempty"`
N *int `json:"n,omitempty"`
Stream bool `json:"stream,omitempty"`
StreamOptions *StreamOptions `json:"stream_options,omitempty"`
Tools []Tool `json:"tools,omitempty"`
ToolChoice json.RawMessage `json:"tool_choice,omitempty"`
ResponseFormat *ResponseFormat `json:"response_format,omitempty"`
ReasoningEffort string `json:"reasoning_effort,omitempty"`
}
// StringList 兼容 OpenAI stop 的 string 与 []string 两种线格式。 // StringList 兼容 OpenAI stop 的 string 与 []string 两种线格式。
type StringList []string type StringList []string
@@ -50,151 +26,6 @@ func (s *StringList) UnmarshalJSON(b []byte) error {
return nil return nil
} }
// StreamOptions 控制流式末尾是否附带 usage 事件。
type StreamOptions struct {
IncludeUsage bool `json:"include_usage"`
}
// ChatMessage 是请求与非流式响应共用的消息结构。
type ChatMessage struct {
Role string `json:"role"`
Content Content `json:"content"`
Name string `json:"name,omitempty"`
ToolCalls []ToolCall `json:"tool_calls,omitempty"`
ToolCallID string `json:"tool_call_id,omitempty"`
}
// Content 兼容 string 与多模态块数组两种线格式;一期仅透传文本。
type Content struct {
// Text 在原始值为字符串时填充
Text string
// Parts 在原始值为数组时填充
Parts []ContentPart
// isArray 记录原始形态,序列化时保形
isArray bool
}
// ContentPart 是块数组中的一段;text 与 image_url 被网关处理,其余类型拒绝。
type ContentPart struct {
Type string `json:"type"`
Text string `json:"text,omitempty"`
ImageURL *ImageURL `json:"image_url,omitempty"`
}
// ImageURL 是 image_url 块负载;url 为 data URI(base64)或公网地址,与 OCI ImageUrl 同构。
type ImageURL struct {
URL string `json:"url"`
Detail string `json:"detail,omitempty"`
}
func (c *Content) UnmarshalJSON(b []byte) error {
trimmed := strings.TrimSpace(string(b))
if trimmed == "null" {
return nil
}
if len(trimmed) > 0 && trimmed[0] == '[' {
c.isArray = true
return json.Unmarshal(b, &c.Parts)
}
return json.Unmarshal(b, &c.Text)
}
func (c Content) MarshalJSON() ([]byte, error) {
if c.isArray {
return json.Marshal(c.Parts)
}
return json.Marshal(c.Text)
}
// NewTextContent 构造字符串形态的内容。
func NewTextContent(text string) Content {
return Content{Text: text}
}
// NewPartsContent 构造块数组形态的内容(多模态消息用)。
func NewPartsContent(parts []ContentPart) Content {
return Content{Parts: parts, isArray: true}
}
// JoinText 返回全部文本内容(数组形态拼接 text 块)。
func (c Content) JoinText() string {
if !c.isArray {
return c.Text
}
var sb strings.Builder
for _, p := range c.Parts {
if p.Type == "" || p.Type == "text" {
sb.WriteString(p.Text)
}
}
return sb.String()
}
// HasUnsupported 报告是否含网关无法承接的内容块(text / image_url 之外的类型)。
func (c Content) HasUnsupported() bool {
for _, p := range c.Parts {
switch p.Type {
case "", "text":
case "image_url":
if p.ImageURL == nil || p.ImageURL.URL == "" {
return true
}
default:
return true
}
}
return false
}
// Tool 是 function 工具定义。
type Tool struct {
Type string `json:"type"`
Function FunctionDef `json:"function"`
}
// FunctionDef 描述工具的名称与 JSON Schema 参数。
type FunctionDef struct {
Name string `json:"name"`
Description string `json:"description,omitempty"`
Parameters json.RawMessage `json:"parameters,omitempty"`
}
// ToolCall 是助手消息中的工具调用。
type ToolCall struct {
ID string `json:"id"`
Type string `json:"type"`
Function FunctionCall `json:"function"`
}
// FunctionCall 携带调用名与 JSON 编码的实参。
type FunctionCall struct {
Name string `json:"name"`
Arguments string `json:"arguments"`
}
// ResponseFormat 对应 response_format(json_object / json_schema)。
type ResponseFormat struct {
Type string `json:"type"`
JSONSchema json.RawMessage `json:"json_schema,omitempty"`
}
// ChatResponse 是非流式响应体。
type ChatResponse struct {
ID string `json:"id"`
Object string `json:"object"`
Created int64 `json:"created"`
Model string `json:"model"`
Choices []Choice `json:"choices"`
Usage *Usage `json:"usage,omitempty"`
}
// Choice 是一条候选回复。
type Choice struct {
Index int `json:"index"`
Message ChatMessage `json:"message"`
FinishReason string `json:"finish_reason"`
}
// Usage 是 token 用量。 // Usage 是 token 用量。
type Usage struct { type Usage struct {
PromptTokens int `json:"prompt_tokens"` PromptTokens int `json:"prompt_tokens"`
@@ -217,44 +48,6 @@ func (u *Usage) CachedTokens() int {
return u.PromptTokensDetails.CachedTokens return u.PromptTokensDetails.CachedTokens
} }
// ChatChunk 是流式增量事件体(chat.completion.chunk)。
type ChatChunk struct {
ID string `json:"id"`
Object string `json:"object"`
Created int64 `json:"created"`
Model string `json:"model"`
Choices []ChunkChoice `json:"choices"`
Usage *Usage `json:"usage,omitempty"`
}
// ChunkChoice 是流式事件中的增量候选。
type ChunkChoice struct {
Index int `json:"index"`
Delta Delta `json:"delta"`
FinishReason *string `json:"finish_reason"`
}
// Delta 是流式增量内容;Content 为纯文本片段。
type Delta struct {
Role string `json:"role,omitempty"`
Content string `json:"content,omitempty"`
ToolCalls []ToolCallDelta `json:"tool_calls,omitempty"`
}
// ToolCallDelta 是工具调用的流式片段;Index 标识聚合目标。
type ToolCallDelta struct {
Index int `json:"index"`
ID string `json:"id,omitempty"`
Type string `json:"type,omitempty"`
Function FunctionCallDelta `json:"function"`
}
// FunctionCallDelta 是函数名 / 实参的增量片段。
type FunctionCallDelta struct {
Name string `json:"name,omitempty"`
Arguments string `json:"arguments,omitempty"`
}
// Model 是 /v1/models 列表项。 // Model 是 /v1/models 列表项。
type Model struct { type Model struct {
ID string `json:"id"` ID string `json:"id"`
-33
View File
@@ -156,39 +156,6 @@ type RespReasoning struct {
Effort string `json:"effort,omitempty"` Effort string `json:"effort,omitempty"`
} }
// Response 是 /responses 响应对象。
type Response struct {
ID string `json:"id"`
Object string `json:"object"`
CreatedAt int64 `json:"created_at"`
Status string `json:"status"`
Model string `json:"model"`
Output []RespOutItem `json:"output"`
Usage *RespUsage `json:"usage,omitempty"`
Error *RespError `json:"error"`
IncompleteDetails *RespIncomplete `json:"incomplete_details"`
Store bool `json:"store"`
}
// RespOutItem 是 output 数组项:message 或 function_call。
type RespOutItem struct {
Type string `json:"type"`
ID string `json:"id"`
Status string `json:"status,omitempty"`
Role string `json:"role,omitempty"`
Content []RespOutPart `json:"content,omitempty"`
CallID string `json:"call_id,omitempty"`
Name string `json:"name,omitempty"`
Arguments string `json:"arguments,omitempty"`
}
// RespOutPart 是 message item 的内容部件(output_text)。
type RespOutPart struct {
Type string `json:"type"`
Text string `json:"text"`
Annotations []any `json:"annotations"`
}
// RespUsage 是 Responses 口径的用量(input/output 命名)。 // RespUsage 是 Responses 口径的用量(input/output 命名)。
type RespUsage struct { type RespUsage struct {
InputTokens int `json:"input_tokens"` InputTokens int `json:"input_tokens"`
+62 -2
View File
@@ -43,12 +43,13 @@ func (h *aiAdminHandler) createKey(c *gin.Context) {
Name string `json:"name" binding:"required"` Name string `json:"name" binding:"required"`
Value string `json:"value"` Value string `json:"value"`
Group string `json:"group"` Group string `json:"group"`
Models []string `json:"models"`
} }
if err := c.ShouldBindJSON(&req); err != nil { if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()}) c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return return
} }
plaintext, key, err := h.gw.CreateKey(c.Request.Context(), req.Name, req.Value, req.Group) plaintext, key, err := h.gw.CreateKey(c.Request.Context(), req.Name, req.Value, req.Group, req.Models)
if err != nil { if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()}) c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return return
@@ -72,12 +73,13 @@ func (h *aiAdminHandler) updateKey(c *gin.Context) {
Name string `json:"name"` Name string `json:"name"`
Enabled *bool `json:"enabled"` Enabled *bool `json:"enabled"`
Group *string `json:"group"` Group *string `json:"group"`
Models *[]string `json:"models"`
} }
if err := c.ShouldBindJSON(&req); err != nil { if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()}) c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return return
} }
if err := h.gw.UpdateKey(c.Request.Context(), id, req.Name, req.Enabled, req.Group); err != nil { if err := h.gw.UpdateKey(c.Request.Context(), id, req.Name, req.Enabled, req.Group, req.Models); err != nil {
respondError(c, err) respondError(c, err)
return return
} }
@@ -285,6 +287,64 @@ func (h *aiAdminHandler) gatewayModels(c *gin.Context) {
c.JSON(http.StatusOK, gin.H{"items": list.Data}) c.JSON(http.StatusOK, gin.H{"items": list.Data})
} }
// ---- 模型黑名单 ----
// @Summary 模型黑名单列表
// @Tags AI 管理
// @Success 200 {object} map[string]any
// @Security BearerAuth
// @Router /api/v1/ai-blacklist [get]
func (h *aiAdminHandler) listBlacklist(c *gin.Context) {
items, err := h.gw.Blacklist(c.Request.Context())
if err != nil {
respondError(c, err)
return
}
c.JSON(http.StatusOK, gin.H{"items": items})
}
// addBlacklist 拉黑模型:删除全部渠道缓存中的同名条目,后续同步 / 探测均过滤。
//
// @Summary 添加模型黑名单
// @Tags AI 管理
// @Param body body object true "请求体:{name: 模型名}"
// @Success 201 {object} map[string]any
// @Security BearerAuth
// @Router /api/v1/ai-blacklist [post]
func (h *aiAdminHandler) addBlacklist(c *gin.Context) {
var req struct {
Name string `json:"name" binding:"required"`
}
if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
item, err := h.gw.AddBlacklist(c.Request.Context(), req.Name)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
c.JSON(http.StatusCreated, gin.H{"item": item})
}
// @Summary 移除模型黑名单(重新同步后模型恢复入池)
// @Tags AI 管理
// @Param id path int true "黑名单条目 ID"
// @Success 204 "无内容"
// @Security BearerAuth
// @Router /api/v1/ai-blacklist/{id} [delete]
func (h *aiAdminHandler) removeBlacklist(c *gin.Context) {
id, ok := aiPathID(c)
if !ok {
return
}
if err := h.gw.RemoveBlacklist(c.Request.Context(), id); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
c.Status(http.StatusNoContent)
}
// @Summary AI 调用日志列表 // @Summary AI 调用日志列表
// @Tags AI 管理 // @Tags AI 管理
// @Success 200 {object} map[string]any // @Success 200 {object} map[string]any
+150 -178
View File
@@ -1,13 +1,15 @@
package api package api
import ( import (
"bufio"
"bytes"
"crypto/rand" "crypto/rand"
"encoding/hex" "encoding/hex"
"encoding/json" "encoding/json"
"errors" "errors"
"fmt"
"io" "io"
"net/http" "net/http"
"slices"
"strings" "strings"
"time" "time"
@@ -86,6 +88,25 @@ func keyGroup(c *gin.Context) string {
return "" return ""
} }
// keyModels 取当前请求密钥的模型白名单(空 = 不限模型)。
func keyModels(c *gin.Context) []string {
if key, ok := c.Get(aiKeyCtx); ok {
return key.(*model.AiKey).Models
}
return nil
}
// checkKeyModel 校验请求模型是否在密钥白名单内;拒绝时按端点协议返回
// 404 model_not_found(与未知模型同口径,不泄露密钥配置细节)并返回 false。
func checkKeyModel(c *gin.Context, modelName string) bool {
allowed := keyModels(c)
if len(allowed) == 0 || slices.Contains(allowed, modelName) {
return true
}
aiError(c, http.StatusNotFound, "model_not_found", "模型不存在或无权访问: "+modelName)
return false
}
// logEntry 组装调用日志骨架;usage / status 由调用方补齐。 // logEntry 组装调用日志骨架;usage / status 由调用方补齐。
func (h *aiGatewayHandler) logEntry(c *gin.Context, endpoint, modelName string, stream bool, meta service.ChatMeta, start time.Time) model.AiCallLog { func (h *aiGatewayHandler) logEntry(c *gin.Context, endpoint, modelName string, stream bool, meta service.ChatMeta, start time.Time) model.AiCallLog {
entry := model.AiCallLog{ entry := model.AiCallLog{
@@ -102,21 +123,6 @@ func (h *aiGatewayHandler) logEntry(c *gin.Context, endpoint, modelName string,
} }
// validateIR 做协议无关的入参检查(模型名、消息、不支持的内容块)。 // validateIR 做协议无关的入参检查(模型名、消息、不支持的内容块)。
func validateIR(ir aiwire.ChatRequest) error {
if strings.TrimSpace(ir.Model) == "" {
return fmt.Errorf("model 不能为空")
}
if len(ir.Messages) == 0 {
return fmt.Errorf("messages 不能为空")
}
for _, m := range ir.Messages {
if m.Content.HasUnsupported() {
return service.ErrAiUnsupportedBlock
}
}
return nil
}
// maybeLogContent 在调用密钥显式开启内容日志且未过期时写正文(红线例外); // maybeLogContent 在调用密钥显式开启内容日志且未过期时写正文(红线例外);
// respBody 为 nil 时只记请求(流式响应与向量结果不记录);callLogID 关联同次调用日志。 // respBody 为 nil 时只记请求(流式响应与向量结果不记录);callLogID 关联同次调用日志。
func (h *aiGatewayHandler) maybeLogContent(c *gin.Context, callLogID uint, endpoint, modelName string, stream bool, reqBody, respBody any) { func (h *aiGatewayHandler) maybeLogContent(c *gin.Context, callLogID uint, endpoint, modelName string, stream bool, reqBody, respBody any) {
@@ -147,47 +153,6 @@ func (h *aiGatewayHandler) logFailure(c *gin.Context, entry model.AiCallLog, req
h.maybeLogContent(c, callID, entry.Endpoint, entry.Model, entry.Stream, reqBody, nil) h.maybeLogContent(c, callID, entry.Endpoint, entry.Model, entry.Stream, reqBody, nil)
} }
// chatCompletions 是 OpenAI /ai/v1/chat/completions 端点。
//
// @Summary OpenAI 兼容对话补全
// @Tags AI 网关
// @Param body body object true "OpenAI chat/completions 请求体(支持 stream)"
// @Success 200 {object} map[string]any "OpenAI 兼容响应(流式为 SSE)"
// @Router /ai/v1/chat/completions [post]
func (h *aiGatewayHandler) chatCompletions(c *gin.Context) {
var ir aiwire.ChatRequest
if err := c.ShouldBindJSON(&ir); err != nil {
aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error())
return
}
if err := validateIR(ir); err != nil {
aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error())
return
}
if ir.Stream {
h.streamOpenAI(c, ir)
return
}
start := time.Now()
resp, meta, err := h.gw.Chat(c.Request.Context(), ir, keyGroup(c))
entry := h.logEntry(c, "openai", ir.Model, false, meta, start)
if err != nil {
upstreamError(c, err)
entry.ErrMsg = err.Error()
h.logFailure(c, entry, ir)
return
}
resp.ID = aiRandID("chatcmpl-")
if resp.Created == 0 {
resp.Created = time.Now().Unix()
}
entry.Status = http.StatusOK
fillUsage(&entry, resp.Usage)
callID := h.gw.LogCall(entry)
h.maybeLogContent(c, callID, "openai", ir.Model, false, ir, resp)
c.JSON(http.StatusOK, resp)
}
func fillUsage(entry *model.AiCallLog, u *aiwire.Usage) { func fillUsage(entry *model.AiCallLog, u *aiwire.Usage) {
if u == nil { if u == nil {
return return
@@ -217,6 +182,9 @@ func (h *aiGatewayHandler) embeddings(c *gin.Context) {
aiError(c, http.StatusBadRequest, "invalid_request_error", "仅支持 encoding_format=float") aiError(c, http.StatusBadRequest, "invalid_request_error", "仅支持 encoding_format=float")
return return
} }
if !checkKeyModel(c, req.Model) {
return
}
start := time.Now() start := time.Now()
resp, meta, err := h.gw.Embeddings(c.Request.Context(), req, keyGroup(c)) resp, meta, err := h.gw.Embeddings(c.Request.Context(), req, keyGroup(c))
entry := h.logEntry(c, "embeddings", req.Model, false, meta, start) entry := h.logEntry(c, "embeddings", req.Model, false, meta, start)
@@ -235,44 +203,6 @@ func (h *aiGatewayHandler) embeddings(c *gin.Context) {
c.JSON(http.StatusOK, resp) c.JSON(http.StatusOK, resp)
} }
// streamOpenAI 以 OpenAI SSE 直通流式响应,末尾发 [DONE]。
func (h *aiGatewayHandler) streamOpenAI(c *gin.Context, ir aiwire.ChatRequest) {
start := time.Now()
stream, meta, err := h.gw.OpenStream(c.Request.Context(), ir, keyGroup(c))
entry := h.logEntry(c, "openai", ir.Model, true, meta, start)
if err != nil {
upstreamError(c, err)
entry.ErrMsg = err.Error()
h.logFailure(c, entry, ir)
return
}
defer stream.Close()
sseHeaders(c)
id, created := aiRandID("chatcmpl-"), time.Now().Unix()
var usage *aiwire.Usage
for {
chunk, err := stream.Next()
if err != nil {
if !errors.Is(err, io.EOF) {
entry.ErrMsg = err.Error()
}
break
}
chunk.ID, chunk.Created = id, created
if chunk.Usage != nil {
usage = chunk.Usage
}
writeSSEData(c, chunk)
}
c.Writer.WriteString("data: [DONE]\n\n")
c.Writer.Flush()
entry.Status = http.StatusOK
entry.LatencyMs = time.Since(start).Milliseconds()
fillUsage(&entry, usage)
callID := h.gw.LogCall(entry)
h.maybeLogContent(c, callID, "openai", ir.Model, true, ir, nil)
}
func sseHeaders(c *gin.Context) { func sseHeaders(c *gin.Context) {
c.Header("Content-Type", "text/event-stream") c.Header("Content-Type", "text/event-stream")
c.Header("Cache-Control", "no-cache") c.Header("Cache-Control", "no-cache")
@@ -280,22 +210,11 @@ func sseHeaders(c *gin.Context) {
c.Writer.Flush() c.Writer.Flush()
} }
func writeSSEData(c *gin.Context, v any) {
b, err := json.Marshal(v)
if err != nil {
return
}
c.Writer.WriteString("data: ")
c.Writer.Write(b)
c.Writer.WriteString("\n\n")
c.Writer.Flush()
}
// messages 是 Anthropic /ai/v1/messages 端点。 // messages 是 Anthropic /ai/v1/messages 端点。
// //
// @Summary Anthropic Messages 兼容端点 // @Summary Anthropic Messages 兼容端点
// @Tags AI 网关 // @Tags AI 网关
// @Param body body object true "Anthropic messages 请求体(支持 stream)" // @Param body body object true "Anthropic messages 请求体(支持 stream;经 OCI OpenAI 兼容面直通)"
// @Success 200 {object} map[string]any "Anthropic 兼容响应(流式为 SSE)" // @Success 200 {object} map[string]any "Anthropic 兼容响应(流式为 SSE)"
// @Router /ai/v1/messages [post] // @Router /ai/v1/messages [post]
func (h *aiGatewayHandler) messages(c *gin.Context) { func (h *aiGatewayHandler) messages(c *gin.Context) {
@@ -308,69 +227,92 @@ func (h *aiGatewayHandler) messages(c *gin.Context) {
aiError(c, http.StatusBadRequest, "invalid_request_error", "max_tokens 必填且需大于 0") aiError(c, http.StatusBadRequest, "invalid_request_error", "max_tokens 必填且需大于 0")
return return
} }
ir, err := service.AnthropicToIR(req) if len(req.Messages) == 0 {
aiError(c, http.StatusBadRequest, "invalid_request_error", "messages 不能为空")
return
}
if !checkKeyModel(c, req.Model) {
return
}
body, err := service.AnthropicToResponsesBody(req)
if err != nil { if err != nil {
aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error()) aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error())
return return
} }
if err := validateIR(ir); err != nil {
aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error())
return
}
if req.Stream { if req.Stream {
h.streamAnthropic(c, ir) h.streamAnthropic(c, body, req)
return return
} }
start := time.Now() start := time.Now()
resp, meta, err := h.gw.Chat(c.Request.Context(), ir, keyGroup(c)) payload, meta, err := h.gw.RespPassthrough(c.Request.Context(), body, req.Model, keyGroup(c))
entry := h.logEntry(c, "anthropic", ir.Model, false, meta, start) entry := h.logEntry(c, "anthropic", req.Model, false, meta, start)
if err != nil { if err != nil {
upstreamError(c, err) upstreamError(c, err)
entry.ErrMsg = err.Error() entry.ErrMsg = err.Error()
h.logFailure(c, entry, req) h.logFailure(c, entry, req)
return return
} }
out, err := service.ResponsesToAnthropic(payload, aiRandID("msg_"))
if err != nil {
aiError(c, http.StatusBadGateway, "api_error", err.Error())
entry.ErrMsg = err.Error()
h.logFailure(c, entry, req)
return
}
entry.Status = http.StatusOK entry.Status = http.StatusOK
fillUsage(&entry, resp.Usage) fillUsage(&entry, service.RespPassthroughUsage(payload))
callID := h.gw.LogCall(entry) callID := h.gw.LogCall(entry)
out := service.IRRespToAnthropic(resp, aiRandID("msg_")) h.maybeLogContent(c, callID, "anthropic", req.Model, false, req, out)
h.maybeLogContent(c, callID, "anthropic", ir.Model, false, req, out)
c.JSON(http.StatusOK, out) c.JSON(http.StatusOK, out)
} }
// streamAnthropic 经状态机把 IR chunk 流聚合为 Anthropic SSE 事件流。 // streamAnthropic 流式直通并桥接:上游 Responses SSE 事件经 AnthRespBridge
func (h *aiGatewayHandler) streamAnthropic(c *gin.Context, ir aiwire.ChatRequest) { // 转为 Anthropic 事件序列逐块写出;usage 从 completed 事件记账。
func (h *aiGatewayHandler) streamAnthropic(c *gin.Context, body []byte, req aiwire.MessagesRequest) {
start := time.Now() start := time.Now()
stream, meta, err := h.gw.OpenStream(c.Request.Context(), ir, keyGroup(c)) upstream, meta, err := h.gw.RespPassthroughStream(c.Request.Context(), body, req.Model, keyGroup(c))
entry := h.logEntry(c, "anthropic", ir.Model, true, meta, start) entry := h.logEntry(c, "anthropic", req.Model, true, meta, start)
if err != nil { if err != nil {
upstreamError(c, err) upstreamError(c, err)
entry.ErrMsg = err.Error() entry.ErrMsg = err.Error()
h.logFailure(c, entry, ir) h.logFailure(c, entry, req)
return return
} }
defer stream.Close() defer upstream.Close()
sseHeaders(c) sseHeaders(c)
st := service.NewAnthStream(aiRandID("msg_"), ir.Model) bridge := service.NewAnthRespBridge(aiRandID("msg_"), req.Model)
for { if err := forwardAnthSSE(c, upstream, bridge); err != nil {
chunk, err := stream.Next()
if err != nil {
if !errors.Is(err, io.EOF) {
entry.ErrMsg = err.Error() entry.ErrMsg = err.Error()
} }
break writeAnthEvents(c, bridge.Finish())
}
writeAnthEvents(c, st.Feed(chunk))
}
writeAnthEvents(c, st.Finish())
entry.Status = http.StatusOK entry.Status = http.StatusOK
entry.LatencyMs = time.Since(start).Milliseconds() entry.LatencyMs = time.Since(start).Milliseconds()
usage := st.Usage() usage := bridge.Usage()
entry.PromptTokens, entry.CompletionTokens = usage.InputTokens, usage.OutputTokens entry.PromptTokens, entry.CompletionTokens = usage.InputTokens, usage.OutputTokens
entry.TotalTokens = usage.InputTokens + usage.OutputTokens entry.TotalTokens = usage.InputTokens + usage.OutputTokens
entry.CachedTokens = usage.CacheReadInputTokens entry.CachedTokens = usage.CacheReadInputTokens
callID := h.gw.LogCall(entry) callID := h.gw.LogCall(entry)
h.maybeLogContent(c, callID, "anthropic", ir.Model, true, ir, nil) h.maybeLogContent(c, callID, "anthropic", req.Model, true, req, nil)
}
// forwardAnthSSE 逐行读上游 SSE,把 data 行喂给桥并即时写出转换后的事件。
func forwardAnthSSE(c *gin.Context, upstream io.Reader, bridge *service.AnthRespBridge) error {
reader := bufio.NewReader(upstream)
for {
line, err := reader.ReadBytes('\n')
if len(line) > 0 {
trimmed := bytes.TrimSpace(line)
if data, ok := bytes.CutPrefix(trimmed, []byte("data: ")); ok {
writeAnthEvents(c, bridge.Feed(data))
}
}
if err != nil {
if errors.Is(err, io.EOF) {
return nil
}
return err
}
}
} }
func writeAnthEvents(c *gin.Context, events []service.AnthEvent) { func writeAnthEvents(c *gin.Context, events []service.AnthEvent) {
@@ -398,6 +340,15 @@ func (h *aiGatewayHandler) listModels(c *gin.Context) {
aiError(c, http.StatusInternalServerError, "api_error", "查询模型列表失败") aiError(c, http.StatusInternalServerError, "api_error", "查询模型列表失败")
return return
} }
if allowed := keyModels(c); len(allowed) > 0 {
kept := make([]aiwire.Model, 0, len(list.Data))
for _, m := range list.Data {
if slices.Contains(allowed, m.ID) {
kept = append(kept, m)
}
}
list.Data = kept
}
c.JSON(http.StatusOK, list) c.JSON(http.StatusOK, list)
} }
@@ -405,31 +356,45 @@ func (h *aiGatewayHandler) listModels(c *gin.Context) {
// //
// @Summary OpenAI Responses 兼容端点 // @Summary OpenAI Responses 兼容端点
// @Tags AI 网关 // @Tags AI 网关
// @Param body body object true "OpenAI responses 请求体(支持 stream)" // @Param body body object true "OpenAI responses 请求体(支持 stream;web_search/x_search 服务端工具仅非流式)"
// @Success 200 {object} map[string]any "OpenAI 兼容响应(流式为 SSE)" // @Success 200 {object} map[string]any "OpenAI 兼容响应(流式为 SSE)"
// @Router /ai/v1/responses [post] // @Router /ai/v1/responses [post]
func (h *aiGatewayHandler) responses(c *gin.Context) { func (h *aiGatewayHandler) responses(c *gin.Context) {
var req aiwire.RespRequest raw, err := c.GetRawData()
if err := c.ShouldBindJSON(&req); err != nil {
aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error())
return
}
ir, err := service.ResponsesToIR(req)
if err != nil { if err != nil {
aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error()) aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error())
return return
} }
if err := validateIR(ir); err != nil { var req aiwire.RespRequest
if err := json.Unmarshal(raw, &req); err != nil {
aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error())
return
}
if !checkKeyModel(c, req.Model) {
return
}
h.responsesPassthrough(c, raw, req)
}
// responsesPassthrough 直通链路(唯一上游):原始 body 改写后直达
// OCI /actions/v1/responses,响应原样透传。
func (h *aiGatewayHandler) responsesPassthrough(c *gin.Context, raw []byte, req aiwire.RespRequest) {
if err := service.RespPassthroughValidate(req); err != nil {
aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error())
return
}
body, err := service.RespPassthroughBody(raw)
if err != nil {
aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error()) aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error())
return return
} }
if req.Stream { if req.Stream {
h.streamResponses(c, ir) h.responsesPassthroughStream(c, body, req)
return return
} }
start := time.Now() start := time.Now()
resp, meta, err := h.gw.Chat(c.Request.Context(), ir, keyGroup(c)) payload, meta, err := h.gw.RespPassthrough(c.Request.Context(), body, req.Model, keyGroup(c))
entry := h.logEntry(c, "responses", ir.Model, false, meta, start) entry := h.logEntry(c, "responses", req.Model, false, meta, start)
if err != nil { if err != nil {
upstreamError(c, err) upstreamError(c, err)
entry.ErrMsg = err.Error() entry.ErrMsg = err.Error()
@@ -437,54 +402,61 @@ func (h *aiGatewayHandler) responses(c *gin.Context) {
return return
} }
entry.Status = http.StatusOK entry.Status = http.StatusOK
fillUsage(&entry, resp.Usage) fillUsage(&entry, service.RespPassthroughUsage(payload))
callID := h.gw.LogCall(entry) callID := h.gw.LogCall(entry)
out := service.IRRespToResponses(resp, aiRandID("resp_"), time.Now().Unix()) h.maybeLogContent(c, callID, "responses", req.Model, false, req, json.RawMessage(payload))
h.maybeLogContent(c, callID, "responses", ir.Model, false, req, out) c.Data(http.StatusOK, "application/json; charset=utf-8", payload)
c.JSON(http.StatusOK, out)
} }
// streamResponses 经状态机把 IR chunk 流聚合为 Responses 语义事件流。 // responsesPassthroughStream 流式直通:SSE 事件原样转发(推理增量等直达客户端),
func (h *aiGatewayHandler) streamResponses(c *gin.Context, ir aiwire.ChatRequest) { // 逐行扫描 completed 事件提取 usage 记账。
func (h *aiGatewayHandler) responsesPassthroughStream(c *gin.Context, body []byte, req aiwire.RespRequest) {
start := time.Now() start := time.Now()
stream, meta, err := h.gw.OpenStream(c.Request.Context(), ir, keyGroup(c)) upstream, meta, err := h.gw.RespPassthroughStream(c.Request.Context(), body, req.Model, keyGroup(c))
entry := h.logEntry(c, "responses", ir.Model, true, meta, start) entry := h.logEntry(c, "responses", req.Model, true, meta, start)
if err != nil { if err != nil {
upstreamError(c, err) upstreamError(c, err)
entry.ErrMsg = err.Error() entry.ErrMsg = err.Error()
h.logFailure(c, entry, ir) h.logFailure(c, entry, req)
return return
} }
defer stream.Close() defer upstream.Close()
sseHeaders(c) sseHeaders(c)
st := service.NewRespStream(aiRandID("resp_"), ir.Model, time.Now().Unix()) usage, err := forwardSSE(c, upstream)
for {
chunk, err := stream.Next()
if err != nil { if err != nil {
if !errors.Is(err, io.EOF) {
entry.ErrMsg = err.Error() entry.ErrMsg = err.Error()
} }
break
}
writeRespEvents(c, st.Feed(chunk))
}
writeRespEvents(c, st.Finish())
entry.Status = http.StatusOK entry.Status = http.StatusOK
entry.LatencyMs = time.Since(start).Milliseconds() entry.LatencyMs = time.Since(start).Milliseconds()
fillUsage(&entry, st.Usage()) fillUsage(&entry, usage)
callID := h.gw.LogCall(entry) callID := h.gw.LogCall(entry)
h.maybeLogContent(c, callID, "responses", ir.Model, true, ir, nil) h.maybeLogContent(c, callID, "responses", req.Model, true, req, nil)
} }
func writeRespEvents(c *gin.Context, events []service.RespEvent) { // forwardSSE 把上游 SSE 逐行转发给客户端,空行(事件边界)即 flush;
for _, ev := range events { // 顺带从 data 行提取 response.completed 的 usage。
b, err := json.Marshal(ev.Data) func forwardSSE(c *gin.Context, upstream io.Reader) (*aiwire.Usage, error) {
if err != nil { reader := bufio.NewReader(upstream)
continue var usage *aiwire.Usage
} for {
c.Writer.WriteString("event: " + ev.Event + "\ndata: ") line, err := reader.ReadBytes('\n')
c.Writer.Write(b) if len(line) > 0 {
c.Writer.WriteString("\n\n") c.Writer.Write(line)
} trimmed := bytes.TrimSpace(line)
if len(trimmed) == 0 {
c.Writer.Flush() c.Writer.Flush()
} else if data, ok := bytes.CutPrefix(trimmed, []byte("data: ")); ok {
if u := service.RespStreamCompletedUsage(data); u != nil {
usage = u
}
}
}
if err != nil {
c.Writer.Flush()
if errors.Is(err, io.EOF) {
return usage, nil
}
return usage, err
}
}
} }
+206
View File
@@ -0,0 +1,206 @@
package api
import (
"context"
"encoding/json"
"net/http"
"strconv"
"strings"
"testing"
"time"
"gorm.io/gorm"
"oci-portal/internal/model"
)
// seedGatewayModel 直插启用渠道与模型缓存,绕过云同步。
func seedGatewayModel(t *testing.T, db *gorm.DB, name string) {
t.Helper()
ch := &model.AiChannel{Name: "t-" + name, OciConfigID: 1, Region: "r-" + name, Enabled: true, Priority: 1, Weight: 1}
if err := db.Create(ch).Error; err != nil {
t.Fatalf("seed channel: %v", err)
}
cache := &model.AiModelCache{ChannelID: ch.ID, ModelOcid: "ocid1.." + name, Name: name, Vendor: "v", SyncedAt: time.Now()}
if err := db.Create(cache).Error; err != nil {
t.Fatalf("seed model cache: %v", err)
}
}
func TestAiGatewayKeyModelRestrict(t *testing.T) {
r, auth, _, db := newTestRouterDB(t)
token, _, err := auth.Login(context.Background(), "admin", "pass123", "127.0.0.1", "")
if err != nil {
t.Fatalf("login: %v", err)
}
// 受限密钥:白名单含脏输入,入库应规范化为 2 项(cohere + ghost)
w := doRequest(t, r, http.MethodPost, "/api/v1/ai-keys", token,
`{"name":"limited","value":"limited-key-1234","models":[" cohere.command-a-03-2025 ","cohere.command-a-03-2025","","ghost-model"]}`)
if w.Code != http.StatusCreated {
t.Fatalf("创建受限密钥 status = %d, body %s", w.Code, w.Body.String())
}
var created struct {
Item model.AiKey `json:"item"`
}
if err := json.Unmarshal(w.Body.Bytes(), &created); err != nil {
t.Fatalf("decode: %v", err)
}
if len(created.Item.Models) != 2 || created.Item.Models[0] != "cohere.command-a-03-2025" {
t.Fatalf("创建后 models 未规范化: %v", created.Item.Models)
}
// 不限密钥对照
w = doRequest(t, r, http.MethodPost, "/api/v1/ai-keys", token, `{"name":"open","value":"open-key-12345"}`)
if w.Code != http.StatusCreated {
t.Fatalf("创建不限密钥 status = %d, body %s", w.Code, w.Body.String())
}
seedGatewayModel(t, db, "cohere.command-a-03-2025")
seedGatewayModel(t, db, "meta.llama-3.3-70b-instruct")
deny := []string{"model_not_found", "无权访问"}
pass := []string{"未知模型"} // 穿过白名单闸门,由编排层以「未知模型」拒绝(池内无 ghost-model)
tests := []struct {
name string
key string
path string
body string
wantCode int
wantSub []string
}{
{"responses 流式同样拦截", "limited-key-1234", "/ai/v1/responses",
`{"model":"meta.llama-3.3-70b-instruct","stream":true,"input":"hi"}`, 404, deny},
{"responses 白名单内穿透", "limited-key-1234", "/ai/v1/responses",
`{"model":"ghost-model","input":"hi"}`, 404, pass},
{"embeddings 白名单外拦截", "limited-key-1234", "/ai/v1/embeddings",
`{"model":"meta.llama-3.3-70b-instruct","input":["hi"]}`, 404, deny},
{"messages 白名单外拦截为 Anthropic 体", "limited-key-1234", "/ai/v1/messages",
`{"model":"meta.llama-3.3-70b-instruct","max_tokens":16,"messages":[{"role":"user","content":"hi"}]}`, 404,
[]string{`"type":"error"`, "model_not_found", "无权访问"}},
{"responses 白名单外拦截", "limited-key-1234", "/ai/v1/responses",
`{"model":"meta.llama-3.3-70b-instruct","input":"hi"}`, 404, deny},
{"不限密钥穿透闸门", "open-key-12345", "/ai/v1/responses",
`{"model":"ghost-model","input":"hi"}`, 404, pass},
{"chat completions 端点已删除", "open-key-12345", "/ai/v1/chat/completions",
`{"model":"ghost-model","messages":[{"role":"user","content":"hi"}]}`, 404, []string{"not found"}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
w := doRequest(t, r, http.MethodPost, tt.path, tt.key, tt.body)
if w.Code != tt.wantCode {
t.Fatalf("status = %d, want %d, body %s", w.Code, tt.wantCode, w.Body.String())
}
for _, sub := range tt.wantSub {
if !strings.Contains(w.Body.String(), sub) {
t.Errorf("body 缺少 %q: %s", sub, w.Body.String())
}
}
})
}
}
func TestAiGatewayModelsFilteredByKey(t *testing.T) {
r, auth, _, db := newTestRouterDB(t)
token, _, err := auth.Login(context.Background(), "admin", "pass123", "127.0.0.1", "")
if err != nil {
t.Fatalf("login: %v", err)
}
seedGatewayModel(t, db, "cohere.command-a-03-2025")
seedGatewayModel(t, db, "meta.llama-3.3-70b-instruct")
w := doRequest(t, r, http.MethodPost, "/api/v1/ai-keys", token,
`{"name":"limited","value":"limited-key-1234","models":["cohere.command-a-03-2025"]}`)
if w.Code != http.StatusCreated {
t.Fatalf("创建密钥 status = %d", w.Code)
}
w = doRequest(t, r, http.MethodPost, "/api/v1/ai-keys", token, `{"name":"open","value":"open-key-12345"}`)
if w.Code != http.StatusCreated {
t.Fatalf("创建密钥 status = %d", w.Code)
}
tests := []struct {
name string
key string
want []string
notWant []string
}{
{"受限密钥仅见交集", "limited-key-1234", []string{"cohere.command-a-03-2025"}, []string{"meta.llama-3.3-70b-instruct"}},
{"不限密钥全量可见", "open-key-12345", []string{"cohere.command-a-03-2025", "meta.llama-3.3-70b-instruct"}, nil},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
w := doRequest(t, r, http.MethodGet, "/ai/v1/models", tt.key, "")
if w.Code != http.StatusOK {
t.Fatalf("status = %d, body %s", w.Code, w.Body.String())
}
for _, sub := range tt.want {
if !strings.Contains(w.Body.String(), sub) {
t.Errorf("应包含 %q: %s", sub, w.Body.String())
}
}
for _, sub := range tt.notWant {
if strings.Contains(w.Body.String(), sub) {
t.Errorf("不应包含 %q: %s", sub, w.Body.String())
}
}
})
}
}
func TestAiKeyModelsUpdateRoundTrip(t *testing.T) {
r, auth, _, _ := newTestRouterDB(t)
token, _, err := auth.Login(context.Background(), "admin", "pass123", "127.0.0.1", "")
if err != nil {
t.Fatalf("login: %v", err)
}
w := doRequest(t, r, http.MethodPost, "/api/v1/ai-keys", token, `{"name":"k","value":"round-key-1234"}`)
if w.Code != http.StatusCreated {
t.Fatalf("创建 status = %d", w.Code)
}
var created struct {
Item model.AiKey `json:"item"`
}
_ = json.Unmarshal(w.Body.Bytes(), &created)
id := created.Item.ID
listModels := func() []string {
w := doRequest(t, r, http.MethodGet, "/api/v1/ai-keys", token, "")
var resp struct {
Items []model.AiKey `json:"items"`
}
_ = json.Unmarshal(w.Body.Bytes(), &resp)
for _, k := range resp.Items {
if k.ID == id {
return k.Models
}
}
t.Fatalf("密钥 %d 不在列表中", id)
return nil
}
tests := []struct {
name string
body string
want []string
}{
{"设置白名单", `{"models":[" a-model ","a-model","b-model"]}`, []string{"a-model", "b-model"}},
{"不传 models 保持不变", `{"name":"k2"}`, []string{"a-model", "b-model"}},
{"空数组清空恢复不限", `{"models":[]}`, nil},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
w := doRequest(t, r, http.MethodPut, "/api/v1/ai-keys/"+strconv.Itoa(int(id)), token, tt.body)
if w.Code != http.StatusNoContent {
t.Fatalf("update status = %d, body %s", w.Code, w.Body.String())
}
got := listModels()
if len(got) != len(tt.want) {
t.Fatalf("models = %v, want %v", got, tt.want)
}
for i := range tt.want {
if got[i] != tt.want[i] {
t.Fatalf("models = %v, want %v", got, tt.want)
}
}
})
}
}
+8 -1
View File
@@ -57,6 +57,13 @@ func (nullClient) GetSubscription(context.Context, oci.Credentials, string) (oci
} }
func newTestRouter(t *testing.T) (*gin.Engine, *service.AuthService, *service.SystemLogService) { func newTestRouter(t *testing.T) (*gin.Engine, *service.AuthService, *service.SystemLogService) {
t.Helper()
r, auth, systemLogs, _ := newTestRouterDB(t)
return r, auth, systemLogs
}
// newTestRouterDB 额外回传 db 句柄,供需要直插数据(如网关模型缓存)的测试使用。
func newTestRouterDB(t *testing.T) (*gin.Engine, *service.AuthService, *service.SystemLogService, *gorm.DB) {
t.Helper() t.Helper()
gin.SetMode(gin.TestMode) gin.SetMode(gin.TestMode)
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{ db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{
@@ -90,7 +97,7 @@ func newTestRouter(t *testing.T) (*gin.Engine, *service.AuthService, *service.Sy
logEvents := service.NewLogEventService(db) logEvents := service.NewLogEventService(db)
oauth := service.NewOAuthService(db, settings, auth) oauth := service.NewOAuthService(db, settings, auth)
r := NewRouter(auth, oauth, ociConfigs, tasks, service.NewConsoleService(ociConfigs), settings, notifier, systemLogs, logEvents, service.NewProxyService(db, cipher), service.NewAiGatewayService(db, ociConfigs, nullClient{})) r := NewRouter(auth, oauth, ociConfigs, tasks, service.NewConsoleService(ociConfigs), settings, notifier, systemLogs, logEvents, service.NewProxyService(db, cipher), service.NewAiGatewayService(db, ociConfigs, nullClient{}))
return r, auth, systemLogs return r, auth, systemLogs, db
} }
func doRequest(t *testing.T, r *gin.Engine, method, path, token, body string) *httptest.ResponseRecorder { func doRequest(t *testing.T, r *gin.Engine, method, path, token, body string) *httptest.ResponseRecorder {
+3 -1
View File
@@ -11,7 +11,6 @@ import (
func registerAiGateway(r *gin.Engine, aiGateway *service.AiGatewayService) { func registerAiGateway(r *gin.Engine, aiGateway *service.AiGatewayService) {
aih := &aiGatewayHandler{gw: aiGateway} aih := &aiGatewayHandler{gw: aiGateway}
ai := r.Group("/ai/v1", bodyLimit(10<<20), aih.auth) ai := r.Group("/ai/v1", bodyLimit(10<<20), aih.auth)
ai.POST("/chat/completions", aih.chatCompletions)
ai.POST("/responses", aih.responses) ai.POST("/responses", aih.responses)
ai.POST("/messages", aih.messages) ai.POST("/messages", aih.messages)
ai.POST("/embeddings", aih.embeddings) ai.POST("/embeddings", aih.embeddings)
@@ -33,6 +32,9 @@ func registerAiAdmin(secured *gin.RouterGroup, aiGateway *service.AiGatewayServi
secured.POST("/ai-channels/:id/probe", aiadmin.probeChannel) secured.POST("/ai-channels/:id/probe", aiadmin.probeChannel)
secured.POST("/ai-channels/:id/sync-models", aiadmin.syncChannelModels) secured.POST("/ai-channels/:id/sync-models", aiadmin.syncChannelModels)
secured.GET("/ai-models", aiadmin.gatewayModels) secured.GET("/ai-models", aiadmin.gatewayModels)
secured.GET("/ai-blacklist", aiadmin.listBlacklist)
secured.POST("/ai-blacklist", aiadmin.addBlacklist)
secured.DELETE("/ai-blacklist/:id", aiadmin.removeBlacklist)
secured.GET("/ai-logs", aiadmin.listLogs) secured.GET("/ai-logs", aiadmin.listLogs)
secured.GET("/ai-content-logs", aiadmin.listContentLogs) secured.GET("/ai-content-logs", aiadmin.listContentLogs)
} }
+1 -1
View File
@@ -68,7 +68,7 @@ func autoMigrate(db *gorm.DB) error {
&model.CheckSnapshot{}, &model.CostSnapshot{}, &model.CheckSnapshot{}, &model.CostSnapshot{},
&model.RegionCache{}, &model.CompartmentCache{}, &model.Setting{}, &model.RegionCache{}, &model.CompartmentCache{}, &model.Setting{},
&model.SystemLog{}, &model.LogEvent{}, &model.AlertRule{}, &model.AlertRuleHit{}, &model.Proxy{}, &model.SystemLog{}, &model.LogEvent{}, &model.AlertRule{}, &model.AlertRuleHit{}, &model.Proxy{},
&model.AiKey{}, &model.AiChannel{}, &model.AiModelCache{}, &model.AiCallLog{}, &model.AiKey{}, &model.AiChannel{}, &model.AiModelCache{}, &model.AiModelBlacklist{}, &model.AiCallLog{},
&model.AiContentLog{}, &model.AiContentLog{},
) )
} }
+10 -7
View File
@@ -266,6 +266,8 @@ type AiKey struct {
KeyHash string `gorm:"uniqueIndex;size:64" json:"-"` KeyHash string `gorm:"uniqueIndex;size:64" json:"-"`
Tail string `gorm:"size:4" json:"tail"` Tail string `gorm:"size:4" json:"tail"`
Group string `gorm:"column:key_group;size:32" json:"group"` Group string `gorm:"column:key_group;size:32" json:"group"`
// Models 是模型白名单(精确匹配模型名);空 = 不限,非空时网关仅放行列表内模型
Models []string `gorm:"serializer:json;type:text" json:"models"`
Enabled bool `json:"enabled"` Enabled bool `json:"enabled"`
LastUsedAt *time.Time `json:"lastUsedAt"` LastUsedAt *time.Time `json:"lastUsedAt"`
// ContentLogUntil 是内容日志开启截止时间;nil = 永关(默认,红线), // ContentLogUntil 是内容日志开启截止时间;nil = 永关(默认,红线),
@@ -309,13 +311,14 @@ type AiModelCache struct {
// 弃用后仍可调用,直到 RetiredAt(按需推理退役,同步层已剔除过期项)。 // 弃用后仍可调用,直到 RetiredAt(按需推理退役,同步层已剔除过期项)。
DeprecatedAt *time.Time `json:"deprecatedAt"` DeprecatedAt *time.Time `json:"deprecatedAt"`
RetiredAt *time.Time `json:"retiredAt"` RetiredAt *time.Time `json:"retiredAt"`
// Unusable 标记该模型在此渠道不可按需调用(微调基座/区域未开放按需), }
// 由探测、真实调用或同步后的后台验证习得:不参与网关列表/路由/探测候选;
// 任何同步都按 OCID 结转标记,已标记模型定期复检、恢复供给自动解除。 // AiModelBlacklist 是全局模型黑名单(按模型名):加入即从全部渠道缓存删除该模型,
Unusable bool `json:"unusable"` // 同步与探测过滤不再入库;适用于微调基座等实际不可按需调用的模型,由用户手动维护。
UnusableReason string `gorm:"size:200" json:"unusableReason,omitempty"` type AiModelBlacklist struct {
// CheckedAt 是最近一次可用性验证时间;NULL 表示从未验证(同步后的后台验证以此为队列)。 ID uint `gorm:"primaryKey" json:"id"`
CheckedAt *time.Time `json:"checkedAt"` Name string `gorm:"size:96;uniqueIndex" json:"name"`
CreatedAt time.Time `json:"createdAt"`
} }
// AiCallLog 是网关调用日志:仅元数据与 token 用量,绝不记录 prompt / 响应正文。 // AiCallLog 是网关调用日志:仅元数据与 token 用量,绝不记录 prompt / 响应正文。
+5 -2
View File
@@ -6,6 +6,7 @@ import (
"context" "context"
"errors" "errors"
"fmt" "fmt"
"io"
"strings" "strings"
"sync" "sync"
"time" "time"
@@ -99,10 +100,12 @@ type Client interface {
AddVnicIpv6(ctx context.Context, cred Credentials, region, vnicID, address string) (string, error) AddVnicIpv6(ctx context.Context, cred Credentials, region, vnicID, address string) (string, error)
// GenAI 网关:区域模型列表、聊天(非流式/流式)、配额探测、文本向量化。 // GenAI 网关:区域模型列表、聊天(非流式/流式)、配额探测、文本向量化。
ListGenAiModels(ctx context.Context, cred Credentials, region string) ([]GenAiModel, error) ListGenAiModels(ctx context.Context, cred Credentials, region string) ([]GenAiModel, error)
GenAiChat(ctx context.Context, cred Credentials, region, modelOcid string, ir aiwire.ChatRequest) (*aiwire.ChatResponse, error)
GenAiChatStream(ctx context.Context, cred Credentials, region, modelOcid string, ir aiwire.ChatRequest) (GenAiStream, error)
GenAiProbeChat(ctx context.Context, cred Credentials, region, modelOcid, modelName string) (int, error) GenAiProbeChat(ctx context.Context, cred Credentials, region, modelOcid, modelName string) (int, error)
GenAiEmbed(ctx context.Context, cred Credentials, region, modelOcid string, inputs []string, dimensions *int) ([][]float32, *aiwire.Usage, error) GenAiEmbed(ctx context.Context, cred Credentials, region, modelOcid string, inputs []string, dimensions *int) ([][]float32, *aiwire.Usage, error)
// GenAiCompatResponses 直通 OpenAI Responses 请求体到 /actions/v1/responses(xAI 服务端工具通路)。
GenAiCompatResponses(ctx context.Context, cred Credentials, region string, body []byte) ([]byte, error)
// GenAiCompatResponsesStream 流式直通 /actions/v1/responses,建立成功返回 SSE body。
GenAiCompatResponsesStream(ctx context.Context, cred Credentials, region string, body []byte) (io.ReadCloser, error)
// 控制台连接:创建(VNC/串口连接串)、列出、删除。 // 控制台连接:创建(VNC/串口连接串)、列出、删除。
CreateConsoleConnection(ctx context.Context, cred Credentials, region, instanceID, sshPublicKey string) (ConsoleConnection, error) CreateConsoleConnection(ctx context.Context, cred Credentials, region, instanceID, sshPublicKey string) (ConsoleConnection, error)
ListConsoleConnections(ctx context.Context, cred Credentials, region, instanceID string) ([]ConsoleConnection, error) ListConsoleConnections(ctx context.Context, cred Credentials, region, instanceID string) ([]ConsoleConnection, error)
+28 -121
View File
@@ -2,13 +2,11 @@ package oci
import ( import (
"context" "context"
"encoding/json"
"fmt" "fmt"
"io"
"net/http" "net/http"
"strings"
"time" "time"
"github.com/oracle/oci-go-sdk/v65/common"
"github.com/oracle/oci-go-sdk/v65/generativeai" "github.com/oracle/oci-go-sdk/v65/generativeai"
"github.com/oracle/oci-go-sdk/v65/generativeaiinference" "github.com/oracle/oci-go-sdk/v65/generativeaiinference"
@@ -30,12 +28,6 @@ type GenAiModel struct {
Retired *time.Time `json:"retired"` Retired *time.Time `json:"retired"`
} }
// GenAiStream 逐事件读取流式聊天;Next 在流结束时返回 io.EOF。
type GenAiStream interface {
Next() (aiwire.ChatChunk, error)
Close() error
}
func (c *RealClient) genAiClient(cred Credentials, region string) (generativeai.GenerativeAiClient, error) { func (c *RealClient) genAiClient(cred Credentials, region string) (generativeai.GenerativeAiClient, error) {
gc, err := generativeai.NewGenerativeAiClientWithConfigurationProvider(provider(cred)) gc, err := generativeai.NewGenerativeAiClientWithConfigurationProvider(provider(cred))
if err != nil { if err != nil {
@@ -152,121 +144,15 @@ func capStrings(caps []generativeai.ModelCapabilityEnum) []string {
return out return out
} }
// GenAiChat 实现 Client:非流式聊天;modelOcid 走 on-demand serving, // GenAiProbeChat 实现 Client:经 OpenAI 兼容面(直通同链路)发一次极小请求探测渠道;配额探测专用的最小聊天(maxTokens=1),返回 HTTP 状态码;
// cohere.* 模型走 COHERE 请求格式,其余走 GENERIC。
func (c *RealClient) GenAiChat(ctx context.Context, cred Credentials, region, modelOcid string, ir aiwire.ChatRequest) (*aiwire.ChatResponse, error) {
ic, err := c.genAiInferenceClient(cred, region)
if err != nil {
return nil, err
}
req, err := buildChatRequest(ir, false)
if err != nil {
return nil, err
}
resp, err := ic.Chat(ctx, chatRequest(cred, modelOcid, req))
if err != nil {
return nil, fmt.Errorf("genai chat: %w", err)
}
return chatResponseToIR(resp.ChatResult.ChatResponse, ir.Model)
}
// buildChatRequest 按模型 vendor 组装底层请求体。
func buildChatRequest(ir aiwire.ChatRequest, stream bool) (generativeaiinference.BaseChatRequest, error) {
if isCohereModel(ir.Model) {
return irToCohereSDK(ir, stream)
}
return irToSDK(ir, stream), nil
}
// chatResponseToIR 按响应实际形态(GENERIC / COHERE)转回 IR。
func chatResponseToIR(resp generativeaiinference.BaseChatResponse, model string) (*aiwire.ChatResponse, error) {
switch r := resp.(type) {
case generativeaiinference.GenericChatResponse:
return sdkToIR(r, model), nil
case generativeaiinference.CohereChatResponse:
return cohereSDKToIR(r, model), nil
default:
return nil, fmt.Errorf("genai chat: unexpected response format %T", resp)
}
}
func chatRequest(cred Credentials, modelOcid string, req generativeaiinference.BaseChatRequest) generativeaiinference.ChatRequest {
return generativeaiinference.ChatRequest{
ChatDetails: generativeaiinference.ChatDetails{
CompartmentId: &cred.TenancyOCID,
ServingMode: generativeaiinference.OnDemandServingMode{ModelId: &modelOcid},
ChatRequest: req,
},
}
}
// GenAiChatStream 实现 Client:流式聊天,返回逐事件读取器。
// SDK 对 text/event-stream 跳过 unmarshal,原始流经 RawResponse 交由 SSEReader 消费。
func (c *RealClient) GenAiChatStream(ctx context.Context, cred Credentials, region, modelOcid string, ir aiwire.ChatRequest) (GenAiStream, error) {
ic, err := c.genAiInferenceClient(cred, region)
if err != nil {
return nil, err
}
req, err := buildChatRequest(ir, true)
if err != nil {
return nil, err
}
resp, err := ic.Chat(ctx, chatRequest(cred, modelOcid, req))
if err != nil {
return nil, fmt.Errorf("genai chat stream: %w", err)
}
reader, err := common.NewSSEReader(resp.RawResponse)
if err != nil {
return nil, fmt.Errorf("genai chat stream: sse reader: %w", err)
}
return &genAiSSEStream{reader: reader, body: resp.RawResponse.Body, model: ir.Model}, nil
}
// genAiSSEStream 把 OCI SSE 事件流适配为 IR chunk 流。
type genAiSSEStream struct {
reader *common.SseReader
body io.ReadCloser
model string
}
// Next 读取下一条有效事件;空事件与 [DONE] 哨兵跳过,流尽返回 io.EOF。
func (s *genAiSSEStream) Next() (aiwire.ChatChunk, error) {
for {
data, err := s.reader.ReadNextEvent()
if err != nil {
return aiwire.ChatChunk{}, err
}
text := strings.TrimSpace(string(data))
if text == "" {
continue
}
if text == "[DONE]" {
return aiwire.ChatChunk{}, io.EOF
}
if chunk, ok := parseGenAiEvent([]byte(text), s.model); ok {
return chunk, nil
}
}
}
func (s *genAiSSEStream) Close() error {
if s.body != nil {
return s.body.Close()
}
return nil
}
// GenAiProbeChat 实现 Client:配额探测专用的最小聊天(maxTokens=1),返回 HTTP 状态码;
// modelName 决定请求格式(cohere.* 走 COHERE)。 // modelName 决定请求格式(cohere.* 走 COHERE)。
func (c *RealClient) GenAiProbeChat(ctx context.Context, cred Credentials, region, modelOcid, modelName string) (int, error) { func (c *RealClient) GenAiProbeChat(ctx context.Context, cred Credentials, region, modelOcid, modelName string) (int, error) {
one := 1 body, err := json.Marshal(map[string]any{"model": modelName, "input": "hi",
ir := aiwire.ChatRequest{ "max_output_tokens": 1, "store": false})
Model: modelName, if err != nil {
Messages: []aiwire.ChatMessage{{Role: "user", Content: aiwire.NewTextContent("hi")}}, return 0, err
MaxTokens: &one,
} }
_, err := c.GenAiChat(ctx, cred, region, modelOcid, ir) if _, err = c.GenAiCompatResponses(ctx, cred, region, body); err == nil {
if err == nil {
return http.StatusOK, nil return http.StatusOK, nil
} }
if status, ok := ServiceStatus(err); ok { if status, ok := ServiceStatus(err); ok {
@@ -275,6 +161,27 @@ func (c *RealClient) GenAiProbeChat(ctx context.Context, cred Credentials, regio
return 0, err return 0, err
} }
// sdkUsageToIR 把 SDK 用量转为内部记账结构(缓存命中挂 details,仅命中时出现)。
func sdkUsageToIR(u *generativeaiinference.Usage) *aiwire.Usage {
if u == nil {
return nil
}
out := &aiwire.Usage{}
if u.PromptTokens != nil {
out.PromptTokens = *u.PromptTokens
}
if u.CompletionTokens != nil {
out.CompletionTokens = *u.CompletionTokens
}
if u.TotalTokens != nil {
out.TotalTokens = *u.TotalTokens
}
if u.PromptTokensDetails != nil && u.PromptTokensDetails.CachedTokens != nil && *u.PromptTokensDetails.CachedTokens > 0 {
out.PromptTokensDetails = &aiwire.PromptTokensDetails{CachedTokens: *u.PromptTokensDetails.CachedTokens}
}
return out
}
// GenAiEmbed 实现 Client:文本向量化(on-demand serving);dimensions 透传 OutputDimensions。 // GenAiEmbed 实现 Client:文本向量化(on-demand serving);dimensions 透传 OutputDimensions。
func (c *RealClient) GenAiEmbed(ctx context.Context, cred Credentials, region, modelOcid string, inputs []string, dimensions *int) ([][]float32, *aiwire.Usage, error) { func (c *RealClient) GenAiEmbed(ctx context.Context, cred Credentials, region, modelOcid string, inputs []string, dimensions *int) ([][]float32, *aiwire.Usage, error) {
ic, err := c.genAiInferenceClient(cred, region) ic, err := c.genAiInferenceClient(cred, region)
-266
View File
@@ -1,266 +0,0 @@
package oci
import (
"encoding/json"
"fmt"
"strings"
"github.com/oracle/oci-go-sdk/v65/generativeaiinference"
"oci-portal/internal/aiwire"
)
// isCohereModel 依据模型名前缀判定 COHERE 系(走专属请求格式)。
func isCohereModel(name string) bool {
return strings.HasPrefix(strings.ToLower(name), "cohere.")
}
// irToCohereSDK 把 IR 转为 COHERE 聊天请求;工具与结构化输出做有损降级
// (参数仅取 JSON Schema 顶层 properties,tool_choice / json_schema 的 name、strict 无对应)。
func irToCohereSDK(ir aiwire.ChatRequest, stream bool) (generativeaiinference.CohereChatRequest, error) {
var req generativeaiinference.CohereChatRequest
if err := cohereRejectUnsupported(ir); err != nil {
return req, err
}
p, err := cohereSplitMessages(ir.Messages)
if err != nil {
return req, err
}
req = generativeaiinference.CohereChatRequest{
Message: &p.message,
ChatHistory: p.history,
ToolResults: p.toolResults,
Tools: cohereTools(ir.Tools),
ResponseFormat: cohereResponseFormat(ir.ResponseFormat),
MaxTokens: ir.MaxTokens,
Temperature: ir.Temperature,
TopP: ir.TopP,
TopK: ir.TopK,
FrequencyPenalty: ir.FrequencyPenalty,
PresencePenalty: ir.PresencePenalty,
Seed: ir.Seed,
}
if p.preamble != "" {
req.PreambleOverride = &p.preamble
}
if len(ir.Stop) > 0 {
req.StopSequences = ir.Stop
}
if stream {
req.IsStream = &stream
includeUsage := ir.StreamOptions == nil || ir.StreamOptions.IncludeUsage
req.StreamOptions = &generativeaiinference.StreamOptions{IsIncludeUsage: &includeUsage}
}
return req, nil
}
// cohereRejectUnsupported 拒绝 COHERE 无法承接的能力(多模态图片)。
func cohereRejectUnsupported(ir aiwire.ChatRequest) error {
for _, m := range ir.Messages {
for _, p := range m.Content.Parts {
if p.Type == "image_url" {
return fmt.Errorf("cohere 系模型暂不支持图片输入,请改用 meta/google 等多模态模型")
}
}
}
return nil
}
// cohereParts 是 IR 消息拆装为 COHERE 请求的中间结果。
type cohereParts struct {
message string
history []generativeaiinference.CohereMessage
preamble string
toolResults []generativeaiinference.CohereToolResult
}
// cohereSplitMessages 拆装消息:system 拼 preamble,末尾 user 抽为 message,
// assistant(含 toolCalls)与其余 user 进 history,tool 结果转顶层 toolResults
// (工具结果在末尾时 message 留空,COHERE 以 toolResults 续跑)。
func cohereSplitMessages(msgs []aiwire.ChatMessage) (cohereParts, error) {
lastUser, lastTool := -1, -1
for i, m := range msgs {
switch m.Role {
case "user":
lastUser = i
case "tool":
lastTool = i
}
}
if lastUser == -1 && lastTool == -1 {
return cohereParts{}, fmt.Errorf("cohere 系模型至少需要一条 user 消息")
}
var p cohereParts
var preamble []string
calls := map[string]generativeaiinference.CohereToolCall{}
for i, m := range msgs {
text := m.Content.JoinText()
switch m.Role {
case "system", "developer":
preamble = append(preamble, text)
case "assistant":
p.history = append(p.history, cohereBotMessage(text, m.ToolCalls, calls))
case "tool":
p.toolResults = append(p.toolResults, cohereToolResult(m, calls))
default: // user
if i == lastUser && lastUser > lastTool {
p.message = text
continue
}
p.history = append(p.history, generativeaiinference.CohereUserMessage{Message: &text})
}
}
p.preamble = strings.Join(preamble, "\n")
return p, nil
}
// cohereBotMessage 转 assistant 消息;toolCalls 同步登记进 id→call 映射供 tool 结果回查。
func cohereBotMessage(text string, tcs []aiwire.ToolCall, calls map[string]generativeaiinference.CohereToolCall) generativeaiinference.CohereChatBotMessage {
msg := generativeaiinference.CohereChatBotMessage{}
if text != "" {
msg.Message = &text
}
for _, tc := range tcs {
name := tc.Function.Name
var params interface{}
if json.Unmarshal([]byte(tc.Function.Arguments), &params) != nil || params == nil {
params = map[string]any{}
}
call := generativeaiinference.CohereToolCall{Name: &name, Parameters: &params}
calls[tc.ID] = call
msg.ToolCalls = append(msg.ToolCalls, call)
}
return msg
}
// cohereToolResult 把 IR tool 消息转顶层工具结果;COHERE 工具调用无 id,
// 靠 assistant 历史登记的映射回查,缺失时以 tool_call_id 名义调用兜底。
func cohereToolResult(m aiwire.ChatMessage, calls map[string]generativeaiinference.CohereToolCall) generativeaiinference.CohereToolResult {
call, ok := calls[m.ToolCallID]
if !ok {
name := m.ToolCallID
var params interface{} = map[string]any{}
call = generativeaiinference.CohereToolCall{Name: &name, Parameters: &params}
}
text := m.Content.JoinText()
var output interface{}
if json.Unmarshal([]byte(text), &output) != nil || output == nil {
output = map[string]any{"output": text}
}
if arr, isArr := output.([]interface{}); isArr {
return generativeaiinference.CohereToolResult{Call: &call, Outputs: arr}
}
if _, isMap := output.(map[string]interface{}); !isMap {
output = map[string]any{"output": output}
}
return generativeaiinference.CohereToolResult{Call: &call, Outputs: []interface{}{output}}
}
// cohereTools 把 JSON Schema 工具定义降级为 COHERE 扁平参数表(嵌套结构有损:仅取顶层)。
func cohereTools(tools []aiwire.Tool) []generativeaiinference.CohereTool {
if len(tools) == 0 {
return nil
}
out := make([]generativeaiinference.CohereTool, 0, len(tools))
for _, t := range tools {
name, desc := t.Function.Name, t.Function.Description
if desc == "" {
desc = name // Description 为 COHERE 必填
}
out = append(out, generativeaiinference.CohereTool{
Name: &name, Description: &desc,
ParameterDefinitions: cohereParams(t.Function.Parameters),
})
}
return out
}
// cohereParams 取 JSON Schema 顶层 properties 转扁平参数定义;嵌套 schema 只保留类型名。
func cohereParams(schema json.RawMessage) map[string]generativeaiinference.CohereParameterDefinition {
var s struct {
Properties map[string]struct {
Type string `json:"type"`
Description string `json:"description"`
} `json:"properties"`
Required []string `json:"required"`
}
if json.Unmarshal(schema, &s) != nil || len(s.Properties) == 0 {
return nil
}
req := map[string]bool{}
for _, r := range s.Required {
req[r] = true
}
out := make(map[string]generativeaiinference.CohereParameterDefinition, len(s.Properties))
for k, v := range s.Properties {
typ, desc := v.Type, v.Description
if typ == "" {
typ = "string"
}
pd := generativeaiinference.CohereParameterDefinition{Type: &typ}
if desc != "" {
pd.Description = &desc
}
if req[k] {
t := true
pd.IsRequired = &t
}
out[k] = pd
}
return out
}
// cohereResponseFormat 映射 json_object / json_schema(name、strict 无对应,有损降级)。
func cohereResponseFormat(rf *aiwire.ResponseFormat) generativeaiinference.CohereResponseFormat {
if rf == nil {
return nil
}
switch rf.Type {
case "json_object", "json_schema":
out := generativeaiinference.CohereResponseJsonFormat{}
var in struct {
Schema json.RawMessage `json:"schema"`
}
if json.Unmarshal(rf.JSONSchema, &in) == nil && len(in.Schema) > 0 {
var schema interface{}
if json.Unmarshal(in.Schema, &schema) == nil {
out.Schema = &schema
}
}
return out
}
return nil
}
// cohereSDKToIR 把 COHERE 非流式响应转回 IR;工具调用补派生 ID,存在时结束原因归 tool_calls。
func cohereSDKToIR(resp generativeaiinference.CohereChatResponse, model string) *aiwire.ChatResponse {
msg := aiwire.ChatMessage{Role: "assistant", Content: aiwire.NewTextContent(deref(resp.Text))}
for i, tc := range resp.ToolCalls {
msg.ToolCalls = append(msg.ToolCalls, cohereCallToIR(deref(tc.Name), tc.Parameters, i))
}
finish := mapFinishReason(string(resp.FinishReason))
if len(msg.ToolCalls) > 0 {
finish = "tool_calls"
}
return &aiwire.ChatResponse{
Object: "chat.completion",
Model: model,
Choices: []aiwire.Choice{{Message: msg, FinishReason: finish}},
Usage: sdkUsageToIR(resp.Usage),
}
}
// cohereCallToIR 生成 IR 工具调用;COHERE 无调用 id,按名称+序号派生(回传时按此回查)。
func cohereCallToIR(name string, params *interface{}, idx int) aiwire.ToolCall {
args := "{}"
if params != nil && *params != nil {
if b, err := json.Marshal(*params); err == nil {
args = string(b)
}
}
return aiwire.ToolCall{
ID: fmt.Sprintf("call_%s_%d", name, idx),
Type: "function",
Function: aiwire.FunctionCall{Name: name, Arguments: args},
}
}
-367
View File
@@ -1,367 +0,0 @@
package oci
import (
"encoding/json"
"strings"
"github.com/oracle/oci-go-sdk/v65/generativeaiinference"
"oci-portal/internal/aiwire"
)
// irToSDK 把 IR(OpenAI 线格式)转为 OCI GENERIC 聊天请求;字段近一一镜像。
func irToSDK(ir aiwire.ChatRequest, stream bool) generativeaiinference.GenericChatRequest {
req := generativeaiinference.GenericChatRequest{
Messages: irMessages(ir.Messages),
MaxTokens: ir.MaxTokens,
MaxCompletionTokens: ir.MaxCompletionTokens,
Temperature: ir.Temperature,
TopP: ir.TopP,
TopK: ir.TopK,
FrequencyPenalty: ir.FrequencyPenalty,
PresencePenalty: ir.PresencePenalty,
Seed: ir.Seed,
NumGenerations: ir.N,
Tools: irTools(ir.Tools),
ToolChoice: irToolChoice(ir.ToolChoice),
ResponseFormat: irResponseFormat(ir.ResponseFormat),
}
if len(ir.Stop) > 0 {
req.Stop = ir.Stop
}
if ir.ReasoningEffort != "" {
req.ReasoningEffort = generativeaiinference.GenericChatRequestReasoningEffortEnum(strings.ToUpper(ir.ReasoningEffort))
}
if stream {
req.IsStream = &stream
includeUsage := ir.StreamOptions == nil || ir.StreamOptions.IncludeUsage
req.StreamOptions = &generativeaiinference.StreamOptions{IsIncludeUsage: &includeUsage}
}
return req
}
// irMessages 按 role 拆装消息;user 消息保留图文块,其余角色只取文本。
func irMessages(msgs []aiwire.ChatMessage) []generativeaiinference.Message {
out := make([]generativeaiinference.Message, 0, len(msgs))
for _, m := range msgs {
switch m.Role {
case "system", "developer":
out = append(out, generativeaiinference.SystemMessage{Content: textContents(m.Content.JoinText())})
case "assistant":
out = append(out, generativeaiinference.AssistantMessage{
Content: textContents(m.Content.JoinText()),
ToolCalls: irToolCalls(m.ToolCalls),
})
case "tool":
tm := generativeaiinference.ToolMessage{Content: textContents(m.Content.JoinText())}
if m.ToolCallID != "" {
id := m.ToolCallID
tm.ToolCallId = &id
}
out = append(out, tm)
default: // user
out = append(out, generativeaiinference.UserMessage{Content: chatContents(m.Content)})
}
}
return out
}
// chatContents 把 IR 内容转 SDK 内容块;文本直通,image_url 转 IMAGE 块(data URI / 公网地址)。
func chatContents(c aiwire.Content) []generativeaiinference.ChatContent {
if len(c.Parts) == 0 {
return textContents(c.Text)
}
var out []generativeaiinference.ChatContent
for _, p := range c.Parts {
switch {
case p.Type == "image_url" && p.ImageURL != nil:
img := generativeaiinference.ImageUrl{Url: &p.ImageURL.URL}
if d, ok := generativeaiinference.GetMappingImageUrlDetailEnum(p.ImageURL.Detail); ok {
img.Detail = d
}
out = append(out, generativeaiinference.ImageContent{ImageUrl: &img})
case p.Text != "":
t := p.Text
out = append(out, generativeaiinference.TextContent{Text: &t})
}
}
return out
}
// textContents 把纯文本包装为单元素 TextContent 数组;空文本返回 nil。
func textContents(text string) []generativeaiinference.ChatContent {
if text == "" {
return nil
}
return []generativeaiinference.ChatContent{generativeaiinference.TextContent{Text: &text}}
}
func irToolCalls(calls []aiwire.ToolCall) []generativeaiinference.ToolCall {
if len(calls) == 0 {
return nil
}
out := make([]generativeaiinference.ToolCall, 0, len(calls))
for _, c := range calls {
id, name, args := c.ID, c.Function.Name, c.Function.Arguments
out = append(out, generativeaiinference.FunctionCall{Id: &id, Name: &name, Arguments: &args})
}
return out
}
func irTools(tools []aiwire.Tool) []generativeaiinference.ToolDefinition {
if len(tools) == 0 {
return nil
}
out := make([]generativeaiinference.ToolDefinition, 0, len(tools))
for _, t := range tools {
name, desc := t.Function.Name, t.Function.Description
fd := generativeaiinference.FunctionDefinition{Name: &name}
if desc != "" {
fd.Description = &desc
}
if len(t.Function.Parameters) > 0 {
var params interface{}
if json.Unmarshal(t.Function.Parameters, &params) == nil {
fd.Parameters = &params
}
}
out = append(out, fd)
}
return out
}
// irToolChoice 解析 "auto"/"none"/"required" 或 {type:function,function:{name}}。
func irToolChoice(raw json.RawMessage) generativeaiinference.ToolChoice {
if len(raw) == 0 {
return nil
}
var s string
if json.Unmarshal(raw, &s) == nil {
switch s {
case "none":
return generativeaiinference.ToolChoiceNone{}
case "required":
return generativeaiinference.ToolChoiceRequired{}
case "auto":
return generativeaiinference.ToolChoiceAuto{}
}
return nil
}
var obj struct {
Function struct {
Name string `json:"name"`
} `json:"function"`
}
if json.Unmarshal(raw, &obj) == nil && obj.Function.Name != "" {
return generativeaiinference.ToolChoiceFunction{Name: &obj.Function.Name}
}
return nil
}
func irResponseFormat(rf *aiwire.ResponseFormat) generativeaiinference.ResponseFormat {
if rf == nil {
return nil
}
switch rf.Type {
case "json_object":
return generativeaiinference.JsonObjectResponseFormat{}
case "json_schema":
return irJSONSchemaFormat(rf.JSONSchema)
default:
return nil
}
}
// irJSONSchemaFormat 映射 OpenAI json_schema{name,description,schema,strict}。
func irJSONSchemaFormat(raw json.RawMessage) generativeaiinference.ResponseFormat {
var in struct {
Name string `json:"name"`
Description string `json:"description"`
Schema json.RawMessage `json:"schema"`
Strict *bool `json:"strict"`
}
if json.Unmarshal(raw, &in) != nil || in.Name == "" {
return generativeaiinference.JsonObjectResponseFormat{}
}
js := generativeaiinference.ResponseJsonSchema{Name: &in.Name, IsStrict: in.Strict}
if in.Description != "" {
js.Description = &in.Description
}
if len(in.Schema) > 0 {
var schema interface{}
if json.Unmarshal(in.Schema, &schema) == nil {
js.Schema = &schema
}
}
return generativeaiinference.JsonSchemaResponseFormat{JsonSchema: &js}
}
// sdkToIR 把非流式 GenericChatResponse 转回 IR 响应;id/created 由调用方补。
func sdkToIR(resp generativeaiinference.GenericChatResponse, model string) *aiwire.ChatResponse {
out := &aiwire.ChatResponse{Object: "chat.completion", Model: model}
if resp.TimeCreated != nil {
out.Created = resp.TimeCreated.Unix()
}
for i, ch := range resp.Choices {
idx := i
if ch.Index != nil {
idx = *ch.Index
}
out.Choices = append(out.Choices, aiwire.Choice{
Index: idx,
Message: sdkMessageToIR(ch.Message),
FinishReason: mapFinishReason(deref(ch.FinishReason)),
})
}
out.Usage = sdkUsageToIR(resp.Usage)
return out
}
func sdkUsageToIR(u *generativeaiinference.Usage) *aiwire.Usage {
if u == nil {
return nil
}
out := &aiwire.Usage{}
if u.PromptTokens != nil {
out.PromptTokens = *u.PromptTokens
}
if u.CompletionTokens != nil {
out.CompletionTokens = *u.CompletionTokens
}
if u.TotalTokens != nil {
out.TotalTokens = *u.TotalTokens
}
if d := u.PromptTokensDetails; d != nil && d.CachedTokens != nil && *d.CachedTokens > 0 {
out.PromptTokensDetails = &aiwire.PromptTokensDetails{CachedTokens: *d.CachedTokens}
}
return out
}
// sdkMessageToIR 提取助手消息文本与工具调用。
func sdkMessageToIR(m generativeaiinference.Message) aiwire.ChatMessage {
out := aiwire.ChatMessage{Role: "assistant"}
am, ok := m.(generativeaiinference.AssistantMessage)
if !ok {
return out
}
var sb strings.Builder
for _, c := range am.Content {
if tc, ok := c.(generativeaiinference.TextContent); ok && tc.Text != nil {
sb.WriteString(*tc.Text)
}
}
out.Content = aiwire.NewTextContent(sb.String())
for _, call := range am.ToolCalls {
if fc, ok := call.(generativeaiinference.FunctionCall); ok {
out.ToolCalls = append(out.ToolCalls, aiwire.ToolCall{
ID: deref(fc.Id),
Type: "function",
Function: aiwire.FunctionCall{
Name: deref(fc.Name),
Arguments: deref(fc.Arguments),
},
})
}
}
return out
}
// mapFinishReason 归一化结束原因;OCI 与 OpenAI 取值同名,做小写与别名兜底。
func mapFinishReason(reason string) string {
switch strings.ToLower(reason) {
case "", "null":
return ""
case "stop", "completed", "end_turn":
return "stop"
case "length", "max_tokens":
return "length"
case "tool_calls", "tool_call", "tool_use":
return "tool_calls"
case "complete":
return "stop"
case "error_toxic":
return "content_filter"
default:
return strings.ToLower(reason)
}
}
// genAiStreamEvent 是 GENERIC 流式事件的宽容解析结构(字段按增量 choice 形状)。
type genAiStreamEvent struct {
Index *int `json:"index"`
Message struct {
Role string `json:"role"`
Content []struct {
Type string `json:"type"`
Text string `json:"text"`
} `json:"content"`
ToolCalls []struct {
ID string `json:"id"`
Name string `json:"name"`
Arguments string `json:"arguments"`
} `json:"toolCalls"`
} `json:"message"`
FinishReason *string `json:"finishReason"`
Usage *generativeaiinference.Usage `json:"usage"`
Choices []json.RawMessage `json:"choices"` // 兼容整包 choices 形态
Text string `json:"text"` // COHERE 流事件的顶层增量文本
// CohereCalls 是 COHERE 流事件顶层的工具调用(与 GENERIC 的 message.toolCalls 不同层级)
CohereCalls []struct {
Name string `json:"name"`
Parameters interface{} `json:"parameters"`
} `json:"toolCalls"`
}
// parseGenAiEvent 把一条 SSE 事件 JSON 解析为 IR chunk;无有效负载时 ok=false。
func parseGenAiEvent(data []byte, model string) (aiwire.ChatChunk, bool) {
var ev genAiStreamEvent
if err := json.Unmarshal(data, &ev); err != nil {
return aiwire.ChatChunk{}, false
}
if len(ev.Choices) > 0 { // choices 包裹形态:取首个展开重解析
var inner genAiStreamEvent
if json.Unmarshal(ev.Choices[0], &inner) == nil {
inner.Usage = ev.Usage
ev = inner
}
}
chunk := aiwire.ChatChunk{Object: "chat.completion.chunk", Model: model}
choice := aiwire.ChunkChoice{}
if ev.Index != nil {
choice.Index = *ev.Index
}
choice.Delta.Role = strings.ToLower(ev.Message.Role)
var sb strings.Builder
for _, c := range ev.Message.Content {
sb.WriteString(c.Text)
}
if sb.Len() == 0 && ev.Text != "" { // COHERE 事件形态
sb.WriteString(ev.Text)
}
choice.Delta.Content = sb.String()
for i, tc := range ev.Message.ToolCalls {
d := aiwire.ToolCallDelta{Index: i, ID: tc.ID}
if tc.ID != "" {
d.Type = "function"
}
d.Function.Name = tc.Name
d.Function.Arguments = tc.Arguments
choice.Delta.ToolCalls = append(choice.Delta.ToolCalls, d)
}
for i, tc := range ev.CohereCalls { // COHERE 顶层工具调用:整包出现,补派生 ID
call := cohereCallToIR(tc.Name, &tc.Parameters, i)
d := aiwire.ToolCallDelta{Index: len(choice.Delta.ToolCalls) + i, ID: call.ID, Type: "function"}
d.Function.Name = call.Function.Name
d.Function.Arguments = call.Function.Arguments
choice.Delta.ToolCalls = append(choice.Delta.ToolCalls, d)
}
if r := mapFinishReason(deref(ev.FinishReason)); r != "" {
choice.FinishReason = &r
}
hasPayload := choice.Delta.Content != "" || len(choice.Delta.ToolCalls) > 0 || choice.FinishReason != nil
if hasPayload {
chunk.Choices = []aiwire.ChunkChoice{choice}
}
chunk.Usage = sdkUsageToIR(ev.Usage)
return chunk, hasPayload || chunk.Usage != nil
}
@@ -0,0 +1,496 @@
package oci
// 真实 OCI GenAI effort 探针(重建版)。
//
// 前身 genai_cache_integration_test.go 承载缓存/亲和/服务端工具等历轮调研模式,
// 调研结论均已归档(.trellis/tasks/archive/2026-07/*/research/),该文件已删除;
// 本文件重建共享基础设施,保留 effort 支持矩阵探测,并新增 responses 端点探测
// (multi-agent 模型仅允许走 Responses 面)。
//
// 运行(默认跳过,不触网):
//
// OCI_GENAI_CACHE_PROBE=effort-matrix OCI_CACHE_DB=<db 绝对路径> DATA_KEY=<主密钥> \
// OCI_CACHE_RUN_ID=<唯一标识> go test ./internal/oci/ -run TestRealGenAiEffortMatrix -v
//
// 可选:OCI_CACHE_CONFIG_ID(默认 6)、OCI_CACHE_REGION(默认 us-chicago-1)、
// OCI_EFFORT_CASES 自定义用例("model=effort" 逗号分隔;effort 后缀 "@responses"
// 表示走 /actions/v1/responses 端点,如 "xai.grok-4.20-multi-agent=low@responses")。
//
// 纪律:数据库只读打开;OCI_GO_SDK_DEBUG 必须为空;日志不落原始请求体与凭据,
// 错误消息经 OCID 遮掩后截断。
import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"regexp"
"strings"
"testing"
"time"
"github.com/glebarez/sqlite"
"github.com/oracle/oci-go-sdk/v65/common"
"github.com/oracle/oci-go-sdk/v65/generativeaiinference"
"gorm.io/gorm"
"gorm.io/gorm/logger"
projectcrypto "oci-portal/internal/crypto"
"oci-portal/internal/model"
)
const (
effortProbeRegion = "us-chicago-1"
effortProbeLimit = int64(4 << 20)
)
var effortRunIDPattern = regexp.MustCompile(`^[A-Za-z0-9._-]{1,64}$`)
type effortProbeCase struct {
model string
effort string
responses bool // true 走 /actions/v1/responses,否则 typed /actions/chat
stream bool // 仅 responses 面:请求 SSE 流,观测事件类型序列
}
type effortProbeResources struct {
cred Credentials
region string
modelOCID map[string]string
inference generativeaiinference.GenerativeAiInferenceClient
}
type effortProbeResult struct {
status int
requestRef string
duration time.Duration
errorCode string
errorMessage string
completionTokens int
reasoningTokens int
answerLen int
outputTypes []string
}
// effortMatrixCases 默认批为 grok-4.3 五档基线;全模型矩阵结论见任务归档,
// 复测用 OCI_EFFORT_CASES 自定义。
func effortMatrixCases() []effortProbeCase {
if raw := os.Getenv("OCI_EFFORT_CASES"); raw != "" {
return parseEffortCases(raw)
}
return []effortProbeCase{
{model: "xai.grok-4.3", effort: "none", responses: true},
{model: "xai.grok-4.3", effort: "low", responses: true},
{model: "xai.grok-4.3", effort: "high", responses: true},
}
}
// parseEffortCases 解析 "model=effort[@responses|@responses-stream]" 逗号分隔用例。
func parseEffortCases(raw string) []effortProbeCase {
var out []effortProbeCase
for _, item := range strings.Split(raw, ",") {
parts := strings.SplitN(strings.TrimSpace(item), "=", 2)
if len(parts) != 2 || parts[0] == "" || parts[1] == "" {
continue
}
effort, streaming := strings.CutSuffix(parts[1], "@responses-stream")
if streaming {
out = append(out, effortProbeCase{model: parts[0], effort: effort, responses: true, stream: true})
continue
}
effort, _ = strings.CutSuffix(effort, "@responses") // 后缀兼容保留;所有用例均走 responses 面
out = append(out, effortProbeCase{model: parts[0], effort: effort, responses: true})
}
return out
}
func TestRealGenAiEffortMatrix(t *testing.T) {
requireEffortProbeMode(t, "effort-matrix")
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Minute)
defer cancel()
resources := newEffortProbeResources(t, ctx)
for _, tc := range effortMatrixCases() {
runEffortCase(t, ctx, resources, tc)
}
}
func requireEffortProbeMode(t *testing.T, wanted string) {
t.Helper()
if os.Getenv("OCI_GENAI_CACHE_PROBE") != wanted {
t.Skipf("set OCI_GENAI_CACHE_PROBE=%s to run", wanted)
}
if os.Getenv("OCI_GO_SDK_DEBUG") != "" {
t.Fatal("OCI_GO_SDK_DEBUG must be unset to protect signed requests")
}
if !effortRunIDPattern.MatchString(os.Getenv("OCI_CACHE_RUN_ID")) {
t.Fatal("set a unique OCI_CACHE_RUN_ID (1-64 safe characters)")
}
}
func runEffortCase(t *testing.T, ctx context.Context, resources effortProbeResources, tc effortProbeCase) {
t.Helper()
ocid, ok := resources.modelOCID[tc.model]
if !ok {
t.Logf("model=%s effort=%s resolve-failed: model not in on-demand catalog", tc.model, tc.effort)
return
}
_ = ocid
body, path, err := effortProbeBody(tc)
if err != nil {
t.Fatalf("build effort body: %v", err)
}
result := executeEffortProbe(ctx, resources, path, body)
logEffortResult(t, tc, result)
}
// effortProbeBody 构造 OpenAI Responses 直通请求体(typed chat 面已随网关剔除,
// 探针仅保留 responses 端点;历史 typed 结论见任务归档)。
func effortProbeBody(tc effortProbeCase) ([]byte, string, error) {
question := "How many positive divisors does 360 have? Answer with the number only."
body := map[string]interface{}{"model": tc.model, "input": question,
"max_output_tokens": 2048, "store": false}
if tc.effort != "" {
body["reasoning"] = map[string]string{"effort": tc.effort}
}
if tc.stream {
body["stream"] = true
}
payload, err := json.Marshal(body)
return payload, "/actions/v1/responses", err
}
// executeEffortProbe 以 IAM 签名裸 POST 指定路径;responses 面按生产实现补 compartment 头。
func executeEffortProbe(ctx context.Context, resources effortProbeResources, path string, body []byte) effortProbeResult {
client := resources.inference.BaseClient
client.Configuration.CircuitBreaker = nil
common.UpdateEndpointTemplateForOptions(&client)
common.SetMissingTemplateParams(&client)
request, err := http.NewRequestWithContext(ctx, http.MethodPost, path, strings.NewReader(string(body)))
if err != nil {
return effortProbeResult{errorCode: "BuildRequest", errorMessage: err.Error()}
}
request.Header.Set("Content-Type", "application/json")
if path != "/actions/chat" {
request.Header.Set("CompartmentId", resources.cred.TenancyOCID)
request.Header.Set("opc-compartment-id", resources.cred.TenancyOCID)
}
return effortCallProbe(ctx, client, request)
}
func effortCallProbe(ctx context.Context, client common.BaseClient, request *http.Request) effortProbeResult {
started := time.Now()
response, err := client.Call(ctx, request)
result := effortProbeResult{duration: time.Since(started)}
if response != nil {
result.status = response.StatusCode
result.requestRef = shortEffortRef(response.Header.Get("opc-request-id"))
defer response.Body.Close()
}
if err != nil {
result.errorCode, result.errorMessage = effortProbeError(err)
return result
}
payload, readErr := io.ReadAll(io.LimitReader(response.Body, effortProbeLimit))
if readErr != nil {
result.errorCode, result.errorMessage = "ReadResponse", readErr.Error()
return result
}
parseEffortPayload(&result, payload)
return result
}
func effortProbeError(err error) (string, string) {
if serviceErr, ok := common.IsServiceError(err); ok {
return fmt.Sprintf("%d", serviceErr.GetHTTPStatusCode()), sanitizeEffortText(serviceErr.GetMessage())
}
return "CallError", sanitizeEffortText(err.Error())
}
// parseEffortPayload 兼容三种 usage 形态:typed 驼峰(chatResponse.usage.completionTokens)、
// chat 兼容面下划线(usage.completion_tokens)、responses 面(usage.output_tokens)。
func parseEffortPayload(result *effortProbeResult, payload []byte) {
if text := strings.TrimSpace(string(payload)); strings.HasPrefix(text, "event:") || strings.HasPrefix(text, "data:") {
parseEffortSSE(result, text)
return
}
var root map[string]interface{}
if json.Unmarshal(payload, &root) != nil {
return
}
result.answerLen = len(parseEffortAnswer(root))
result.outputTypes = parseEffortOutputTypes(root)
if typed, ok := root["chatResponse"].(map[string]interface{}); ok {
root = typed
}
usage, _ := root["usage"].(map[string]interface{})
for _, key := range []string{"completionTokens", "completion_tokens", "output_tokens"} {
if v, ok := effortInt(usage, key); ok {
result.completionTokens = v
break
}
}
for _, detailsKey := range []string{"completionTokensDetails", "completion_tokens_details", "output_tokens_details"} {
details, _ := usage[detailsKey].(map[string]interface{})
for _, key := range []string{"reasoningTokens", "reasoning_tokens"} {
if v, ok := effortInt(details, key); ok {
result.reasoningTokens = v
}
}
}
}
// parseEffortAnswer 提取正文文本:typed 取 chatResponse.choices[].message.content[].text,
// responses 取 output[] 里 message 项的 content[].text。仅在内存中量长度,不落日志。
func parseEffortAnswer(root map[string]interface{}) string {
var sb strings.Builder
if typed, ok := root["chatResponse"].(map[string]interface{}); ok {
choices, _ := typed["choices"].([]interface{})
for _, c := range choices {
cm, _ := c.(map[string]interface{})
msg, _ := cm["message"].(map[string]interface{})
collectEffortText(&sb, msg["content"])
}
return sb.String()
}
output, _ := root["output"].([]interface{})
for _, item := range output {
im, _ := item.(map[string]interface{})
if im["type"] == "message" {
collectEffortText(&sb, im["content"])
}
}
return sb.String()
}
func collectEffortText(sb *strings.Builder, content interface{}) {
parts, _ := content.([]interface{})
for _, p := range parts {
pm, _ := p.(map[string]interface{})
if text, ok := pm["text"].(string); ok {
sb.WriteString(text)
}
}
}
// parseEffortSSE 汇总 SSE 流:事件类型去重序列进 outputTypes,正文增量长度进 answerLen,
// 末尾 completed 事件里的 usage 进 tokens 字段。
func parseEffortSSE(result *effortProbeResult, text string) {
seen := map[string]bool{}
for _, line := range strings.Split(text, "\n") {
if name, ok := strings.CutPrefix(line, "event: "); ok {
if name = strings.TrimSpace(name); !seen[name] {
seen[name] = true
result.outputTypes = append(result.outputTypes, name)
}
continue
}
data, ok := strings.CutPrefix(line, "data: ")
if !ok {
continue
}
var ev map[string]interface{}
if json.Unmarshal([]byte(data), &ev) != nil {
continue
}
if name, ok := ev["type"].(string); ok && !seen[name] { // 纯 data: 行流(无 event: 行)从事件体取类型
seen[name] = true
result.outputTypes = append(result.outputTypes, name)
}
if delta, ok := ev["delta"].(string); ok && strings.Contains(fmt.Sprint(ev["type"]), "output_text") {
result.answerLen += len(delta)
}
if resp, ok := ev["response"].(map[string]interface{}); ok {
usage, _ := resp["usage"].(map[string]interface{})
if v, ok := effortInt(usage, "output_tokens"); ok {
result.completionTokens = v
}
}
}
}
// parseEffortOutputTypes 收集 responses 面 output 项类型序列(multi-agent 行为观测)。
func parseEffortOutputTypes(root map[string]interface{}) []string {
output, _ := root["output"].([]interface{})
var types []string
for _, item := range output {
im, _ := item.(map[string]interface{})
if s, ok := im["type"].(string); ok {
types = append(types, s)
}
}
return types
}
func effortInt(root map[string]interface{}, key string) (int, bool) {
if root == nil {
return 0, false
}
if value, ok := root[key].(float64); ok {
return int(value), true
}
return 0, false
}
func logEffortResult(t *testing.T, tc effortProbeCase, result effortProbeResult) {
t.Helper()
endpoint := "typed-chat"
if tc.responses {
endpoint = "compat-responses"
}
t.Logf("endpoint=%s model=%s effort=%s status=%d completion=%d reasoning=%d answer_len=%d output_types=%v request=%s duration_ms=%d error=%s message=%q",
endpoint, tc.model, tc.effort, result.status, result.completionTokens, result.reasoningTokens,
result.answerLen, result.outputTypes, result.requestRef, result.duration.Milliseconds(),
result.errorCode, result.errorMessage)
}
// sanitizeEffortText 遮掩 OCID 并截断,避免错误消息携带租户可定位信息。
func sanitizeEffortText(text string) string {
text = regexp.MustCompile(`ocid1\.[a-z0-9._-]+`).ReplaceAllString(text, "ocid1.***")
if len(text) > 300 {
text = text[:300] + "..."
}
return text
}
func shortEffortRef(ref string) string {
if len(ref) > 12 {
return ref[:12]
}
return ref
}
func effortEnvOr(key, fallback string) string {
if value := os.Getenv(key); value != "" {
return value
}
return fallback
}
// newEffortProbeResources 装配凭据、区域、inference 客户端与按需模型目录。
func newEffortProbeResources(t *testing.T, ctx context.Context) effortProbeResources {
t.Helper()
cred := effortProbeCredentials(t)
region := effortEnvOr("OCI_CACHE_REGION", effortProbeRegion)
cred.Region = region
real := &RealClient{}
ic, err := real.genAiInferenceClient(cred, region)
if err != nil {
t.Fatalf("new inference client: %v", err)
}
models, err := real.ListGenAiModels(ctx, cred, region)
if err != nil {
t.Fatalf("list models: %s", sanitizeEffortText(err.Error()))
}
index := make(map[string]string, len(models))
for _, m := range models {
index[m.Name] = m.Ocid
}
return effortProbeResources{cred: cred, region: region, modelOCID: index, inference: ic}
}
// effortProbeCredentials 优先从面板数据库(只读)取渠道凭据,否则退回本地 ini 测试凭据。
func effortProbeCredentials(t *testing.T) Credentials {
t.Helper()
if dbPath := os.Getenv("OCI_CACHE_DB"); dbPath != "" {
return effortDBCredentials(t, dbPath)
}
return loadTestCredentials(t, effortEnvOr("OCI_TEST_KEY", "试用期"))
}
func effortDBCredentials(t *testing.T, dbPath string) Credentials {
t.Helper()
dsn := fmt.Sprintf("file:%s?mode=ro", dbPath)
db, err := gorm.Open(sqlite.Open(dsn), &gorm.Config{Logger: logger.Default.LogMode(logger.Silent)})
if err != nil {
t.Fatalf("open read-only probe database: %v", err)
}
cipher, err := projectcrypto.NewCipher(os.Getenv("DATA_KEY"))
if err != nil {
t.Fatalf("create data cipher: %v", err)
}
var config model.OciConfig
if err := db.First(&config, effortEnvOr("OCI_CACHE_CONFIG_ID", "6")).Error; err != nil {
t.Fatalf("load OCI config: %v", err)
}
return decryptEffortCredentials(t, db, cipher, config)
}
func decryptEffortCredentials(t *testing.T, db *gorm.DB, cipher *projectcrypto.Cipher, config model.OciConfig) Credentials {
t.Helper()
privateKey, err := cipher.DecryptString(config.PrivateKeyEnc)
if err != nil {
t.Fatalf("decrypt OCI config %d private key: %v", config.ID, err)
}
passphrase := ""
if config.PassphraseEnc != "" {
if passphrase, err = cipher.DecryptString(config.PassphraseEnc); err != nil {
t.Fatalf("decrypt OCI config %d passphrase: %v", config.ID, err)
}
}
return Credentials{TenancyOCID: config.TenancyOCID, UserOCID: config.UserOCID,
Fingerprint: config.Fingerprint, Region: config.Region, PrivateKey: privateKey,
Passphrase: passphrase, Proxy: loadEffortProxy(t, db, cipher, config.ProxyID)}
}
func loadEffortProxy(t *testing.T, db *gorm.DB, cipher *projectcrypto.Cipher, proxyID *uint) *ProxySpec {
t.Helper()
if proxyID == nil {
return nil
}
var proxy model.Proxy
if err := db.First(&proxy, *proxyID).Error; err != nil {
t.Fatalf("load proxy %d: %v", *proxyID, err)
}
password := ""
if proxy.PasswordEnc != "" {
decrypted, err := cipher.DecryptString(proxy.PasswordEnc)
if err != nil {
t.Fatalf("decrypt proxy %d password: %v", *proxyID, err)
}
password = decrypted
}
return &ProxySpec{Type: proxy.Type, Host: proxy.Host, Port: proxy.Port,
Username: proxy.Username, Password: password}
}
// TestParseEffortCases 断言自定义用例解析:端点后缀、空项与畸形项跳过。
func TestParseEffortCases(t *testing.T) {
got := parseEffortCases("a=low, b=high@responses ,bad,=x,c=")
want := []effortProbeCase{{model: "a", effort: "low", responses: true}, {model: "b", effort: "high", responses: true}}
if len(got) != len(want) {
t.Fatalf("parseEffortCases = %+v, want %+v", got, want)
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("case %d = %+v, want %+v", i, got[i], want[i])
}
}
}
// TestParseEffortPayload 断言三种 usage 形态与 output 类型序列的解析。
func TestParseEffortPayload(t *testing.T) {
cases := []struct {
name string
payload string
completion int
reasoning int
types int
}{
{"typed 驼峰", `{"chatResponse":{"choices":[{"message":{"content":[{"type":"TEXT","text":"24"}]}}],"usage":{"completionTokens":5,"completionTokensDetails":{"reasoningTokens":9}}}}`, 5, 9, 0},
{"responses 面", `{"output":[{"type":"reasoning"},{"type":"message","content":[{"type":"output_text","text":"24"}]}],"usage":{"output_tokens":7,"output_tokens_details":{"reasoning_tokens":3}}}`, 7, 3, 2},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
var result effortProbeResult
parseEffortPayload(&result, []byte(tc.payload))
if result.completionTokens != tc.completion || result.reasoningTokens != tc.reasoning || len(result.outputTypes) != tc.types {
t.Fatalf("parse = %+v, want completion=%d reasoning=%d types=%d", result, tc.completion, tc.reasoning, tc.types)
}
if result.answerLen != 2 {
t.Fatalf("answerLen = %d, want 2", result.answerLen)
}
})
}
}
+73
View File
@@ -0,0 +1,73 @@
package oci
import (
"bytes"
"context"
"fmt"
"io"
"net/http"
"github.com/oracle/oci-go-sdk/v65/common"
)
// compatResponsesLimit 限制直通响应体大小;web_search 输出含多段引用,给足余量。
const compatResponsesLimit = int64(8 << 20)
// GenAiCompatResponses 实现 Client:把 OpenAI Responses 请求体直通到 OCI
// `/20231130/actions/v1/responses`(IAM 签名)。该端点实测可执行 xAI 服务端工具
// (web_search / x_search),但不在 Oracle 文档化工具白名单内,行为可能随服务
// 版本、模型或区域变化;调用方须自行校验并改写请求体(store/stream)。
func (c *RealClient) GenAiCompatResponses(ctx context.Context, cred Credentials, region string, body []byte) ([]byte, error) {
ic, err := c.genAiInferenceClient(cred, region)
if err != nil {
return nil, err
}
client := ic.BaseClient
common.UpdateEndpointTemplateForOptions(&client)
common.SetMissingTemplateParams(&client)
request, err := http.NewRequestWithContext(ctx, http.MethodPost, "/actions/v1/responses", bytes.NewReader(body))
if err != nil {
return nil, fmt.Errorf("build compat responses request: %w", err)
}
request.Header.Set("Content-Type", "application/json")
request.Header.Set("CompartmentId", cred.TenancyOCID)
request.Header.Set("opc-compartment-id", cred.TenancyOCID)
response, err := client.Call(ctx, request)
if err != nil {
return nil, err
}
defer response.Body.Close()
payload, err := io.ReadAll(io.LimitReader(response.Body, compatResponsesLimit))
if err != nil {
return nil, fmt.Errorf("read compat responses body: %w", err)
}
return payload, nil
}
// GenAiCompatResponsesStream 实现 Client:以流式直通 OCI `/actions/v1/responses`,
// 建立成功(2xx)返回 SSE body(调用方负责 Close);建立失败返回 SDK ServiceError,
// 与既有渠道切换/熔断错误分类兼容。请求体须由调用方置 stream:true。
func (c *RealClient) GenAiCompatResponsesStream(ctx context.Context, cred Credentials, region string, body []byte) (io.ReadCloser, error) {
ic, err := c.genAiInferenceClient(cred, region)
if err != nil {
return nil, err
}
client := ic.BaseClient
common.UpdateEndpointTemplateForOptions(&client)
common.SetMissingTemplateParams(&client)
request, err := http.NewRequestWithContext(ctx, http.MethodPost, "/actions/v1/responses", bytes.NewReader(body))
if err != nil {
return nil, fmt.Errorf("build compat responses stream request: %w", err)
}
request.Header.Set("Content-Type", "application/json")
request.Header.Set("CompartmentId", cred.TenancyOCID)
request.Header.Set("opc-compartment-id", cred.TenancyOCID)
response, err := client.Call(ctx, request)
if err != nil {
if response != nil && response.Body != nil {
response.Body.Close()
}
return nil, err
}
return response.Body, nil
}
+10 -176
View File
@@ -1,193 +1,27 @@
package oci package oci
import ( import (
"encoding/json"
"testing" "testing"
"github.com/oracle/oci-go-sdk/v65/generativeaiinference" "github.com/oracle/oci-go-sdk/v65/generativeaiinference"
"oci-portal/internal/aiwire"
) )
// TestIrToSDK 断言 IR→GENERIC 的角色拆装、采样参数与工具映射 // TestSdkUsageCachedTokens 断言 embed 用量映射:缓存命中挂 details,仅命中 >0 时出现
func TestIrToSDK(t *testing.T) {
temp, mt := 0.7, 100
ir := aiwire.ChatRequest{
Model: "meta.llama-3.3-70b-instruct",
Messages: []aiwire.ChatMessage{
{Role: "system", Content: aiwire.NewTextContent("你是助手")},
{Role: "user", Content: aiwire.NewTextContent("你好")},
{Role: "assistant", ToolCalls: []aiwire.ToolCall{{ID: "c1", Type: "function", Function: aiwire.FunctionCall{Name: "get_weather", Arguments: `{"city":"东京"}`}}}},
{Role: "tool", ToolCallID: "c1", Content: aiwire.NewTextContent("晴")},
},
Temperature: &temp,
MaxTokens: &mt,
Stop: aiwire.StringList{"END"},
Tools: []aiwire.Tool{{Type: "function", Function: aiwire.FunctionDef{Name: "get_weather", Parameters: json.RawMessage(`{"type":"object"}`)}}},
ToolChoice: json.RawMessage(`"auto"`),
}
req := irToSDK(ir, true)
if len(req.Messages) != 4 {
t.Fatalf("messages = %d, want 4", len(req.Messages))
}
if _, ok := req.Messages[0].(generativeaiinference.SystemMessage); !ok {
t.Fatalf("messages[0] = %T, want SystemMessage", req.Messages[0])
}
am, ok := req.Messages[2].(generativeaiinference.AssistantMessage)
if !ok || len(am.ToolCalls) != 1 {
t.Fatalf("assistant tool calls 未映射: %T %+v", req.Messages[2], am)
}
tm, ok := req.Messages[3].(generativeaiinference.ToolMessage)
if !ok || deref(tm.ToolCallId) != "c1" {
t.Fatalf("tool message 未映射 toolCallId: %+v", tm)
}
if req.Temperature == nil || *req.Temperature != 0.7 {
t.Fatalf("temperature 未直通")
}
if req.MaxTokens == nil || *req.MaxTokens != 100 || len(req.Stop) != 1 {
t.Fatalf("maxTokens/stop 未直通")
}
if _, ok := req.ToolChoice.(generativeaiinference.ToolChoiceAuto); !ok {
t.Fatalf("toolChoice = %T, want auto", req.ToolChoice)
}
if len(req.Tools) != 1 || req.IsStream == nil || !*req.IsStream || req.StreamOptions == nil {
t.Fatalf("tools/stream 选项未映射")
}
}
// TestSdkToIR 断言响应文本、工具调用与用量的反向映射。
func TestSdkToIR(t *testing.T) {
text, fr := "东京晴", "tool_calls"
idx, pt, ct, tt := 0, 10, 5, 15
id, name, args := "c1", "get_weather", `{"city":"东京"}`
resp := generativeaiinference.GenericChatResponse{
Choices: []generativeaiinference.ChatChoice{{
Index: &idx,
Message: generativeaiinference.AssistantMessage{
Content: []generativeaiinference.ChatContent{generativeaiinference.TextContent{Text: &text}},
ToolCalls: []generativeaiinference.ToolCall{generativeaiinference.FunctionCall{Id: &id, Name: &name, Arguments: &args}},
},
FinishReason: &fr,
}},
Usage: &generativeaiinference.Usage{PromptTokens: &pt, CompletionTokens: &ct, TotalTokens: &tt},
}
out := sdkToIR(resp, "m1")
if len(out.Choices) != 1 || out.Choices[0].Message.Content.JoinText() != "东京晴" {
t.Fatalf("文本未映射: %+v", out)
}
if out.Choices[0].FinishReason != "tool_calls" || len(out.Choices[0].Message.ToolCalls) != 1 {
t.Fatalf("finishReason/toolCalls 未映射: %+v", out.Choices[0])
}
if out.Usage == nil || out.Usage.TotalTokens != 15 {
t.Fatalf("usage 未映射: %+v", out.Usage)
}
}
// TestParseGenAiEvent 断言流事件宽容解析:文本增量、结束原因与 usage 事件。
func TestParseGenAiEvent(t *testing.T) {
chunk, ok := parseGenAiEvent([]byte(`{"index":0,"message":{"role":"ASSISTANT","content":[{"type":"TEXT","text":"你"}]}}`), "m1")
if !ok || len(chunk.Choices) != 1 || chunk.Choices[0].Delta.Content != "你" {
t.Fatalf("文本增量解析失败: %+v", chunk)
}
chunk, ok = parseGenAiEvent([]byte(`{"finishReason":"stop","usage":{"promptTokens":3,"completionTokens":2,"totalTokens":5}}`), "m1")
if !ok || chunk.Choices[0].FinishReason == nil || *chunk.Choices[0].FinishReason != "stop" {
t.Fatalf("finishReason 解析失败: %+v", chunk)
}
if chunk.Usage == nil || chunk.Usage.TotalTokens != 5 {
t.Fatalf("usage 解析失败: %+v", chunk.Usage)
}
if _, ok := parseGenAiEvent([]byte(`{}`), "m1"); ok {
t.Fatal("空事件应返回 ok=false")
}
}
func TestSdkUsageCachedTokens(t *testing.T) { func TestSdkUsageCachedTokens(t *testing.T) {
p, c, tot, cached := 10, 5, 15, 8 p, c, tot, cached := 10, 5, 15, 8
u := sdkUsageToIR(&generativeaiinference.Usage{PromptTokens: &p, CompletionTokens: &c, TotalTokens: &tot, u := sdkUsageToIR(&generativeaiinference.Usage{PromptTokens: &p, CompletionTokens: &c, TotalTokens: &tot,
PromptTokensDetails: &generativeaiinference.PromptTokensDetails{CachedTokens: &cached}}) PromptTokensDetails: &generativeaiinference.PromptTokensDetails{CachedTokens: &cached}})
if u.CachedTokens() != 8 { if u.PromptTokens != 10 || u.CompletionTokens != 5 || u.TotalTokens != 15 {
t.Errorf("CachedTokens = %d, want 8", u.CachedTokens()) t.Fatalf("usage 映射错误: %+v", u)
} }
if sdkUsageToIR(&generativeaiinference.Usage{PromptTokens: &p}).CachedTokens() != 0 { if u.PromptTokensDetails == nil || u.PromptTokensDetails.CachedTokens != 8 {
t.Error("无细分时 CachedTokens 应为 0") t.Fatalf("cachedTokens 未映射: %+v", u.PromptTokensDetails)
} }
zero := 0
if got := sdkUsageToIR(&generativeaiinference.Usage{PromptTokens: &p, PromptTokensDetails: &generativeaiinference.PromptTokensDetails{CachedTokens: &zero}}); got.PromptTokensDetails != nil {
t.Fatalf("零命中不应带 details: %+v", got.PromptTokensDetails)
} }
if sdkUsageToIR(nil) != nil {
func TestIrToCohereSDK(t *testing.T) { t.Fatal("nil usage 应返回 nil")
ir := aiwire.ChatRequest{
Model: "cohere.command-r-plus",
Messages: []aiwire.ChatMessage{
{Role: "system", Content: aiwire.NewTextContent("你是助手")},
{Role: "user", Content: aiwire.NewTextContent("第一问")},
{Role: "assistant", Content: aiwire.NewTextContent("第一答")},
{Role: "user", Content: aiwire.NewTextContent("第二问")},
},
}
req, err := irToCohereSDK(ir, true)
if err != nil {
t.Fatalf("irToCohereSDK: %v", err)
}
if *req.Message != "第二问" || *req.PreambleOverride != "你是助手" || len(req.ChatHistory) != 2 {
t.Errorf("拆装错误: msg=%q preamble=%v history=%d", *req.Message, req.PreambleOverride, len(req.ChatHistory))
}
if _, ok := req.ChatHistory[0].(generativeaiinference.CohereUserMessage); !ok {
t.Errorf("history[0] 应为 user: %T", req.ChatHistory[0])
}
if _, ok := req.ChatHistory[1].(generativeaiinference.CohereChatBotMessage); !ok {
t.Errorf("history[1] 应为 chatbot: %T", req.ChatHistory[1])
}
if req.IsStream == nil || !*req.IsStream {
t.Error("IsStream 未设置")
}
if req.StreamOptions == nil || req.StreamOptions.IsIncludeUsage == nil || !*req.StreamOptions.IsIncludeUsage {
t.Error("流式应默认开启 usage 回传(StreamOptions.IsIncludeUsage)")
}
// 工具定义降级为扁平参数表(仅取 JSON Schema 顶层 properties)
ir.Tools = []aiwire.Tool{{Type: "function", Function: aiwire.FunctionDef{
Name: "get_weather", Parameters: json.RawMessage(`{"type":"object","properties":{"city":{"type":"string","description":"城市"}},"required":["city"]}`)}}}
req2, err := irToCohereSDK(ir, false)
if err != nil {
t.Fatalf("工具请求应支持: %v", err)
}
tool := req2.Tools[0]
if *tool.Name != "get_weather" || *tool.Description != "get_weather" {
t.Errorf("tool = %+v", tool)
}
if pd, ok := tool.ParameterDefinitions["city"]; !ok || *pd.Type != "string" || pd.IsRequired == nil || !*pd.IsRequired {
t.Errorf("param city = %+v", pd)
}
// 图片输入仍拒绝
ir.Tools = nil
ir.Messages = append(ir.Messages, aiwire.ChatMessage{Role: "user", Content: aiwire.NewPartsContent([]aiwire.ContentPart{
{Type: "image_url", ImageURL: &aiwire.ImageURL{URL: "data:image/png;base64,xx"}}})})
if _, err := irToCohereSDK(ir, false); err == nil {
t.Error("图片输入应被拒绝")
}
// 无 user 消息被拒
if _, err := irToCohereSDK(aiwire.ChatRequest{Model: "cohere.x", Messages: []aiwire.ChatMessage{{Role: "system", Content: aiwire.NewTextContent("s")}}}, false); err == nil {
t.Error("无 user 消息应被拒绝")
}
}
func TestCohereSDKToIR(t *testing.T) {
text := "回答"
resp := generativeaiinference.CohereChatResponse{
Text: &text,
FinishReason: generativeaiinference.CohereChatResponseFinishReasonComplete,
}
out := cohereSDKToIR(resp, "cohere.command-r-plus")
if out.Choices[0].Message.Content.JoinText() != "回答" || out.Choices[0].FinishReason != "stop" {
t.Errorf("cohereSDKToIR = %+v", out.Choices[0])
}
}
func TestParseGenAiEventCohere(t *testing.T) {
chunk, ok := parseGenAiEvent([]byte(`{"apiFormat":"COHERE","text":"你好"}`), "cohere.command-r")
if !ok || chunk.Choices[0].Delta.Content != "你好" {
t.Errorf("cohere text 事件解析 = %+v, %v", chunk, ok)
}
chunk, ok = parseGenAiEvent([]byte(`{"apiFormat":"COHERE","finishReason":"COMPLETE","usage":{"promptTokens":3,"completionTokens":5,"totalTokens":8}}`), "cohere.command-r")
if !ok || *chunk.Choices[0].FinishReason != "stop" || chunk.Usage.TotalTokens != 8 {
t.Errorf("cohere 终帧解析 = %+v, %v", chunk, ok)
} }
} }
-332
View File
@@ -1,332 +0,0 @@
package service
import (
"encoding/json"
"fmt"
"strings"
"oci-portal/internal/aiwire"
)
// ErrAiUnsupportedBlock 表示请求含网关无法承接的内容块(仅支持文本与图片)。
var ErrAiUnsupportedBlock = fmt.Errorf("暂不支持文本与图片以外的内容块")
// AnthropicToIR 把 Anthropic Messages 请求转为 IR(OpenAI 线格式)。
// tool_result 块拆为独立 tool 角色消息(一拆多),顶层 system 变首条 system 消息。
func AnthropicToIR(req aiwire.MessagesRequest) (aiwire.ChatRequest, error) {
ir := aiwire.ChatRequest{
Model: req.Model,
Temperature: req.Temperature,
TopP: req.TopP,
TopK: req.TopK,
Stop: req.StopSequences,
Stream: req.Stream,
Tools: anthTools(req.Tools),
ToolChoice: anthToolChoice(req.ToolChoice),
}
if req.MaxTokens > 0 {
mt := req.MaxTokens
ir.MaxTokens = &mt
}
if sys := req.SystemText(); sys != "" {
ir.Messages = append(ir.Messages, aiwire.ChatMessage{Role: "system", Content: aiwire.NewTextContent(sys)})
}
for _, m := range req.Messages {
msgs, err := anthMessageToIR(m)
if err != nil {
return ir, err
}
ir.Messages = append(ir.Messages, msgs...)
}
return ir, nil
}
// anthMessageToIR 拆解单条 Anthropic 消息;user 消息里的 tool_result 前置为独立 tool 消息,
// image 块转为 IR image_url 部件。
func anthMessageToIR(m aiwire.AnthMessage) ([]aiwire.ChatMessage, error) {
var out []aiwire.ChatMessage
var parts []aiwire.ContentPart
var toolCalls []aiwire.ToolCall
hasImage := false
for _, b := range m.Content.AllBlocks() {
switch b.Type {
case "text":
parts = append(parts, aiwire.ContentPart{Type: "text", Text: b.Text})
case "image":
p, err := anthImagePart(b.Source)
if err != nil {
return nil, err
}
parts, hasImage = append(parts, p), true
case "tool_use":
toolCalls = append(toolCalls, aiwire.ToolCall{ID: b.ID, Type: "function",
Function: aiwire.FunctionCall{Name: b.Name, Arguments: string(b.Input)}})
case "tool_result":
out = append(out, aiwire.ChatMessage{Role: "tool", ToolCallID: b.ToolUseID,
Content: aiwire.NewTextContent(b.ResultText())})
case "thinking", "redacted_thinking":
// 一期忽略 thinking 块
default:
return nil, ErrAiUnsupportedBlock
}
}
content := partsContent(parts, hasImage)
if hasImage || content.JoinText() != "" || len(toolCalls) > 0 {
out = append(out, aiwire.ChatMessage{Role: m.Role, Content: content, ToolCalls: toolCalls})
}
return out, nil
}
// anthImagePart 把 Anthropic image 块转为 IR image_url 部件(base64 → data URI)。
func anthImagePart(source json.RawMessage) (aiwire.ContentPart, error) {
var src struct {
Type string `json:"type"`
MediaType string `json:"media_type"`
Data string `json:"data"`
URL string `json:"url"`
}
if err := json.Unmarshal(source, &src); err != nil {
return aiwire.ContentPart{}, fmt.Errorf("image source 解析失败: %w", err)
}
switch src.Type {
case "base64":
if src.MediaType == "" || src.Data == "" {
return aiwire.ContentPart{}, fmt.Errorf("image source 缺少 media_type 或 data")
}
url := "data:" + src.MediaType + ";base64," + src.Data
return aiwire.ContentPart{Type: "image_url", ImageURL: &aiwire.ImageURL{URL: url}}, nil
case "url":
if src.URL == "" {
return aiwire.ContentPart{}, fmt.Errorf("image source 缺少 url")
}
return aiwire.ContentPart{Type: "image_url", ImageURL: &aiwire.ImageURL{URL: src.URL}}, nil
}
return aiwire.ContentPart{}, fmt.Errorf("不支持的 image source 类型 %q", src.Type)
}
// partsContent 无图片时退回字符串形态(与上游线格式习惯一致),含图片时保留块数组。
func partsContent(parts []aiwire.ContentPart, hasImage bool) aiwire.Content {
if !hasImage {
var sb strings.Builder
for _, p := range parts {
sb.WriteString(p.Text)
}
return aiwire.NewTextContent(sb.String())
}
return aiwire.NewPartsContent(parts)
}
func anthTools(tools []aiwire.AnthTool) []aiwire.Tool {
if len(tools) == 0 {
return nil
}
out := make([]aiwire.Tool, 0, len(tools))
for _, t := range tools {
out = append(out, aiwire.Tool{Type: "function", Function: aiwire.FunctionDef{
Name: t.Name, Description: t.Description, Parameters: t.InputSchema}})
}
return out
}
// anthToolChoice 映射 {type:auto|any|tool,name} → OpenAI 形态。
func anthToolChoice(raw json.RawMessage) json.RawMessage {
if len(raw) == 0 {
return nil
}
var tc struct {
Type string `json:"type"`
Name string `json:"name"`
}
if json.Unmarshal(raw, &tc) != nil {
return nil
}
switch tc.Type {
case "auto":
return json.RawMessage(`"auto"`)
case "any":
return json.RawMessage(`"required"`)
case "tool":
b, _ := json.Marshal(map[string]any{"type": "function", "function": map[string]string{"name": tc.Name}})
return b
case "none":
return json.RawMessage(`"none"`)
}
return nil
}
// IRRespToAnthropic 把 IR 非流式响应转为 Anthropic Messages 响应。
func IRRespToAnthropic(resp *aiwire.ChatResponse, id string) aiwire.MessagesResponse {
out := aiwire.MessagesResponse{ID: id, Type: "message", Role: "assistant", Model: resp.Model, Content: []aiwire.AnthBlock{}}
if len(resp.Choices) > 0 {
choice := resp.Choices[0]
if text := choice.Message.Content.JoinText(); text != "" {
out.Content = append(out.Content, aiwire.AnthBlock{Type: "text", Text: text})
}
for _, tc := range choice.Message.ToolCalls {
out.Content = append(out.Content, aiwire.AnthBlock{Type: "tool_use", ID: tc.ID,
Name: tc.Function.Name, Input: argsToJSON(tc.Function.Arguments)})
}
out.StopReason = anthStopReason(choice.FinishReason)
}
if resp.Usage != nil {
out.Usage = aiwire.AnthUsage{InputTokens: resp.Usage.PromptTokens, OutputTokens: resp.Usage.CompletionTokens,
CacheReadInputTokens: resp.Usage.CachedTokens()}
}
return out
}
// argsToJSON 保证 tool_use.input 是合法 JSON 对象(模型可能产出非法片段)。
func argsToJSON(args string) json.RawMessage {
trimmed := strings.TrimSpace(args)
if trimmed == "" {
return json.RawMessage(`{}`)
}
if json.Valid([]byte(trimmed)) {
return json.RawMessage(trimmed)
}
b, _ := json.Marshal(map[string]string{"_raw": args})
return b
}
func anthStopReason(finish string) string {
switch finish {
case "length":
return "max_tokens"
case "tool_calls":
return "tool_use"
default:
return "end_turn"
}
}
// ---- Anthropic 流式状态机 ----
// AnthEvent 是一条待写出的 Anthropic SSE 事件。
type AnthEvent struct {
Event string
Data any
}
// AnthStream 把 IR chunk 流聚合为 Anthropic 事件序列:
// message_start → content_block_start/delta/stop(text 与 tool_use 分块)→ message_delta → message_stop。
type AnthStream struct {
id, model string
started bool
blockOpen bool
blockIsTool bool
toolID string
blockIndex int
stopReason string
usage aiwire.AnthUsage
}
// NewAnthStream 构造状态机;id 为响应消息 ID。
func NewAnthStream(id, model string) *AnthStream {
return &AnthStream{id: id, model: model, blockIndex: -1, stopReason: "end_turn"}
}
// Feed 消费一个 IR chunk,返回应立即写出的事件。
func (st *AnthStream) Feed(chunk aiwire.ChatChunk) []AnthEvent {
var events []AnthEvent
if !st.started {
st.started = true
events = append(events, st.startEvent())
}
if chunk.Usage != nil {
st.usage = aiwire.AnthUsage{InputTokens: chunk.Usage.PromptTokens, OutputTokens: chunk.Usage.CompletionTokens,
CacheReadInputTokens: chunk.Usage.CachedTokens()}
}
for _, choice := range chunk.Choices {
events = append(events, st.feedDelta(choice.Delta)...)
if choice.FinishReason != nil && *choice.FinishReason != "" {
st.stopReason = anthStopReason(*choice.FinishReason)
}
}
return events
}
func (st *AnthStream) startEvent() AnthEvent {
return AnthEvent{Event: "message_start", Data: map[string]any{
"type": "message_start",
"message": map[string]any{
"id": st.id, "type": "message", "role": "assistant", "model": st.model,
"content": []any{}, "stop_reason": nil,
"usage": map[string]int{"input_tokens": 0, "output_tokens": 0},
},
}}
}
// feedDelta 处理文本与工具调用增量,必要时切块。
func (st *AnthStream) feedDelta(d aiwire.Delta) []AnthEvent {
var events []AnthEvent
if d.Content != "" {
if !st.blockOpen || st.blockIsTool {
events = append(events, st.openBlock(false, "", "")...)
}
events = append(events, AnthEvent{Event: "content_block_delta", Data: map[string]any{
"type": "content_block_delta", "index": st.blockIndex,
"delta": map[string]string{"type": "text_delta", "text": d.Content},
}})
}
for _, tc := range d.ToolCalls {
if tc.ID != "" && (!st.blockOpen || !st.blockIsTool || st.toolID != tc.ID) {
events = append(events, st.openBlock(true, tc.ID, tc.Function.Name)...)
}
if tc.Function.Arguments != "" && st.blockOpen && st.blockIsTool {
events = append(events, AnthEvent{Event: "content_block_delta", Data: map[string]any{
"type": "content_block_delta", "index": st.blockIndex,
"delta": map[string]string{"type": "input_json_delta", "partial_json": tc.Function.Arguments},
}})
}
}
return events
}
// openBlock 关闭当前块并打开新块(text 或 tool_use)。
func (st *AnthStream) openBlock(isTool bool, toolID, toolName string) []AnthEvent {
var events []AnthEvent
if st.blockOpen {
events = append(events, st.closeBlockEvent())
}
st.blockOpen, st.blockIsTool, st.toolID = true, isTool, toolID
st.blockIndex++
block := map[string]any{"type": "text", "text": ""}
if isTool {
block = map[string]any{"type": "tool_use", "id": toolID, "name": toolName, "input": map[string]any{}}
}
events = append(events, AnthEvent{Event: "content_block_start", Data: map[string]any{
"type": "content_block_start", "index": st.blockIndex, "content_block": block,
}})
return events
}
func (st *AnthStream) closeBlockEvent() AnthEvent {
return AnthEvent{Event: "content_block_stop", Data: map[string]any{
"type": "content_block_stop", "index": st.blockIndex,
}}
}
// Finish 在上游流结束后收尾:关块 → message_delta(stop_reason+usage)→ message_stop。
func (st *AnthStream) Finish() []AnthEvent {
var events []AnthEvent
if !st.started {
events = append(events, st.startEvent())
st.started = true
}
if st.blockOpen {
events = append(events, st.closeBlockEvent())
st.blockOpen = false
}
events = append(events,
AnthEvent{Event: "message_delta", Data: map[string]any{
"type": "message_delta",
"delta": map[string]any{"stop_reason": st.stopReason, "stop_sequence": nil},
"usage": map[string]int{"output_tokens": st.usage.OutputTokens},
}},
AnthEvent{Event: "message_stop", Data: map[string]any{"type": "message_stop"}},
)
return events
}
// Usage 返回聚合到的用量(供调用日志)。
func (st *AnthStream) Usage() aiwire.AnthUsage { return st.usage }
-136
View File
@@ -1,136 +0,0 @@
package service
import (
"encoding/json"
"strings"
"testing"
"oci-portal/internal/aiwire"
)
func mustAnthReq(t *testing.T, body string) aiwire.MessagesRequest {
t.Helper()
var req aiwire.MessagesRequest
if err := json.Unmarshal([]byte(body), &req); err != nil {
t.Fatalf("unmarshal anthropic request: %v", err)
}
return req
}
func TestAnthropicToIR(t *testing.T) {
req := mustAnthReq(t, `{
"model": "meta.llama-3.3-70b-instruct", "max_tokens": 128, "system": "你是助手",
"messages": [
{"role": "user", "content": "东京天气?"},
{"role": "assistant", "content": [
{"type": "text", "text": "查询中"},
{"type": "tool_use", "id": "t1", "name": "get_weather", "input": {"city": "东京"}}
]},
{"role": "user", "content": [
{"type": "tool_result", "tool_use_id": "t1", "content": "晴 25 度"},
{"type": "text", "text": "继续"}
]}
],
"tools": [{"name": "get_weather", "input_schema": {"type": "object"}}],
"tool_choice": {"type": "any"}
}`)
ir, err := AnthropicToIR(req)
if err != nil {
t.Fatalf("AnthropicToIR: %v", err)
}
roles := make([]string, 0, len(ir.Messages))
for _, m := range ir.Messages {
roles = append(roles, m.Role)
}
want := []string{"system", "user", "assistant", "tool", "user"}
if strings.Join(roles, ",") != strings.Join(want, ",") {
t.Fatalf("roles = %v, want %v", roles, want)
}
if ir.Messages[2].ToolCalls[0].Function.Name != "get_weather" {
t.Errorf("tool_use 未映射: %+v", ir.Messages[2])
}
if ir.Messages[3].ToolCallID != "t1" || ir.Messages[3].Content.JoinText() != "晴 25 度" {
t.Errorf("tool_result 未拆为 tool 消息: %+v", ir.Messages[3])
}
if ir.MaxTokens == nil || *ir.MaxTokens != 128 || len(ir.Tools) != 1 {
t.Errorf("max_tokens/tools 未直通")
}
if string(ir.ToolChoice) != `"required"` {
t.Errorf("tool_choice any → %s, want required", ir.ToolChoice)
}
// image 块拒绝
bad := mustAnthReq(t, `{"model":"m","max_tokens":1,"messages":[{"role":"user","content":[{"type":"image","source":{}}]}]}`)
if _, err := AnthropicToIR(bad); err == nil {
t.Error("image 块应被拒绝")
}
}
func TestIRRespToAnthropic(t *testing.T) {
resp := &aiwire.ChatResponse{
Model: "m1",
Choices: []aiwire.Choice{{
Message: aiwire.ChatMessage{
Role: "assistant",
Content: aiwire.NewTextContent("查到了"),
ToolCalls: []aiwire.ToolCall{{ID: "t1", Type: "function",
Function: aiwire.FunctionCall{Name: "get_weather", Arguments: `{"city":"东京"}`}}},
},
FinishReason: "tool_calls",
}},
Usage: &aiwire.Usage{PromptTokens: 10, CompletionTokens: 5},
}
out := IRRespToAnthropic(resp, "msg_1")
if len(out.Content) != 2 || out.Content[0].Type != "text" || out.Content[1].Type != "tool_use" {
t.Fatalf("content 块 = %+v", out.Content)
}
if out.StopReason != "tool_use" || out.Usage.InputTokens != 10 || out.Usage.OutputTokens != 5 {
t.Errorf("stop_reason/usage 未映射: %+v", out)
}
if string(out.Content[1].Input) != `{"city":"东京"}` {
t.Errorf("tool input = %s", out.Content[1].Input)
}
}
// eventTypes 提取事件类型序列便于断言。
func eventTypes(events []AnthEvent) string {
types := make([]string, 0, len(events))
for _, e := range events {
types = append(types, e.Event)
}
return strings.Join(types, ",")
}
func TestAnthStreamTextAndTool(t *testing.T) {
st := NewAnthStream("msg_1", "m1")
fr := "tool_calls"
var events []AnthEvent
events = append(events, st.Feed(aiwire.ChatChunk{Choices: []aiwire.ChunkChoice{{Delta: aiwire.Delta{Content: "你"}}}})...)
events = append(events, st.Feed(aiwire.ChatChunk{Choices: []aiwire.ChunkChoice{{Delta: aiwire.Delta{Content: "好"}}}})...)
events = append(events, st.Feed(aiwire.ChatChunk{Choices: []aiwire.ChunkChoice{{Delta: aiwire.Delta{
ToolCalls: []aiwire.ToolCallDelta{{ID: "t1", Function: aiwire.FunctionCallDelta{Name: "get_weather", Arguments: `{"ci`}}},
}}}})...)
events = append(events, st.Feed(aiwire.ChatChunk{
Choices: []aiwire.ChunkChoice{{Delta: aiwire.Delta{ToolCalls: []aiwire.ToolCallDelta{{ID: "t1", Function: aiwire.FunctionCallDelta{Arguments: `ty":"东京"}`}}}}, FinishReason: &fr}},
Usage: &aiwire.Usage{PromptTokens: 8, CompletionTokens: 4},
})...)
events = append(events, st.Finish()...)
got := eventTypes(events)
want := "message_start,content_block_start,content_block_delta,content_block_delta," +
"content_block_stop,content_block_start,content_block_delta,content_block_delta," +
"content_block_stop,message_delta,message_stop"
if got != want {
t.Fatalf("事件序列:\n got %s\nwant %s", got, want)
}
if st.Usage().OutputTokens != 4 {
t.Errorf("usage 未聚合: %+v", st.Usage())
}
}
func TestAnthStreamEmpty(t *testing.T) {
st := NewAnthStream("msg_1", "m1")
events := st.Finish()
if got := eventTypes(events); got != "message_start,message_delta,message_stop" {
t.Errorf("空流事件序列 = %s", got)
}
}
+147 -177
View File
@@ -5,6 +5,7 @@ import (
"crypto/rand" "crypto/rand"
"crypto/sha256" "crypto/sha256"
"encoding/hex" "encoding/hex"
"encoding/json"
"errors" "errors"
"fmt" "fmt"
"log" "log"
@@ -30,10 +31,6 @@ const (
aiBackoffCap = 30 * time.Minute aiBackoffCap = 30 * time.Minute
// aiKeyTouchGap 是 LastUsedAt 的最小写库间隔,避免高频调用刷库 // aiKeyTouchGap 是 LastUsedAt 的最小写库间隔,避免高频调用刷库
aiKeyTouchGap = time.Minute aiKeyTouchGap = time.Minute
// modelRecheckGap 是已标记不可用模型的复检间隔(恢复供给自动解除标记);
// validateBatchCap 限制单渠道单轮验证的试调次数
modelRecheckGap = 20 * time.Hour
validateBatchCap = 32
// 内容日志(红线例外)约束:开启必须限时(上限 7 天),正文截断,短保留 // 内容日志(红线例外)约束:开启必须限时(上限 7 天),正文截断,短保留
aiContentLogMaxHours = 168 aiContentLogMaxHours = 168
aiContentLogRetention = 7 * 24 * time.Hour aiContentLogRetention = 7 * 24 * time.Hour
@@ -59,17 +56,13 @@ type AiGatewayService struct {
// touchMu 保护各密钥的最近触达时间(内存节流,不追求跨实例精确) // touchMu 保护各密钥的最近触达时间(内存节流,不追求跨实例精确)
touchMu sync.Mutex touchMu sync.Mutex
lastTouch map[uint]time.Time lastTouch map[uint]time.Time
// validateMu 保护 validating:同一渠道的模型验证不并发
validateMu sync.Mutex
validating map[uint]bool
// onChannelsChanged 在渠道增删后触发,由 main 装配为探测任务同步钩子 // onChannelsChanged 在渠道增删后触发,由 main 装配为探测任务同步钩子
onChannelsChanged func(context.Context) onChannelsChanged func(context.Context)
} }
// NewAiGatewayService 组装依赖;调用 StartCleanup 后开始调用日志周期清理。 // NewAiGatewayService 组装依赖;调用 StartCleanup 后开始调用日志周期清理。
func NewAiGatewayService(db *gorm.DB, configs *OciConfigService, client oci.Client) *AiGatewayService { func NewAiGatewayService(db *gorm.DB, configs *OciConfigService, client oci.Client) *AiGatewayService {
return &AiGatewayService{db: db, configs: configs, client: client, return &AiGatewayService{db: db, configs: configs, client: client, lastTouch: map[uint]time.Time{}}
lastTouch: map[uint]time.Time{}, validating: map[uint]bool{}}
} }
// SetOnChannelsChanged 注册渠道数量变化钩子(渠道创建/删除成功后调用)。 // SetOnChannelsChanged 注册渠道数量变化钩子(渠道创建/删除成功后调用)。
@@ -87,7 +80,7 @@ func (s *AiGatewayService) fireChannelsChanged(ctx context.Context) {
// CreateKey 生成网关密钥并返回明文(仅此一次);customValue 非空时使用给定值, // CreateKey 生成网关密钥并返回明文(仅此一次);customValue 非空时使用给定值,
// group 非空时该密钥只在同分组渠道内路由。 // group 非空时该密钥只在同分组渠道内路由。
func (s *AiGatewayService) CreateKey(ctx context.Context, name, customValue, group string) (string, *model.AiKey, error) { func (s *AiGatewayService) CreateKey(ctx context.Context, name, customValue, group string, models []string) (string, *model.AiKey, error) {
name = strings.TrimSpace(name) name = strings.TrimSpace(name)
if name == "" { if name == "" {
return "", nil, fmt.Errorf("密钥名称不能为空") return "", nil, fmt.Errorf("密钥名称不能为空")
@@ -103,13 +96,28 @@ func (s *AiGatewayService) CreateKey(ctx context.Context, name, customValue, gro
if len(raw) < 8 { if len(raw) < 8 {
return "", nil, fmt.Errorf("自定义密钥至少 8 个字符") return "", nil, fmt.Errorf("自定义密钥至少 8 个字符")
} }
key := &model.AiKey{Name: name, KeyHash: hashKey(raw), Tail: raw[len(raw)-4:], Group: strings.TrimSpace(group), Enabled: true} key := &model.AiKey{Name: name, KeyHash: hashKey(raw), Tail: raw[len(raw)-4:], Group: strings.TrimSpace(group), Models: normalizeKeyModels(models), Enabled: true}
if err := s.db.WithContext(ctx).Create(key).Error; err != nil { if err := s.db.WithContext(ctx).Create(key).Error; err != nil {
return "", nil, fmt.Errorf("密钥名称或取值与现有密钥重复") return "", nil, fmt.Errorf("密钥名称或取值与现有密钥重复")
} }
return raw, key, nil return raw, key, nil
} }
// normalizeKeyModels 规范化模型白名单:trim、剔空串、保序去重;结果为空返回 nil(= 不限)。
func normalizeKeyModels(models []string) []string {
var out []string
seen := map[string]bool{}
for _, m := range models {
m = strings.TrimSpace(m)
if m == "" || seen[m] {
continue
}
seen[m] = true
out = append(out, m)
}
return out
}
func hashKey(raw string) string { func hashKey(raw string) string {
sum := sha256.Sum256([]byte(raw)) sum := sha256.Sum256([]byte(raw))
return hex.EncodeToString(sum[:]) return hex.EncodeToString(sum[:])
@@ -122,8 +130,8 @@ func (s *AiGatewayService) Keys(ctx context.Context) ([]model.AiKey, error) {
return keys, err return keys, err
} }
// UpdateKey 修改密钥名称 / 启用状态 / 分组(group 指针非空即覆盖,可置空)。 // UpdateKey 修改密钥名称 / 启用状态 / 分组 / 模型白名单(指针非空即覆盖,可置空)。
func (s *AiGatewayService) UpdateKey(ctx context.Context, id uint, name string, enabled *bool, group *string) error { func (s *AiGatewayService) UpdateKey(ctx context.Context, id uint, name string, enabled *bool, group *string, models *[]string) error {
updates := map[string]any{} updates := map[string]any{}
if name = strings.TrimSpace(name); name != "" { if name = strings.TrimSpace(name); name != "" {
updates["name"] = name updates["name"] = name
@@ -134,6 +142,14 @@ func (s *AiGatewayService) UpdateKey(ctx context.Context, id uint, name string,
if group != nil { if group != nil {
updates["key_group"] = strings.TrimSpace(*group) updates["key_group"] = strings.TrimSpace(*group)
} }
if models != nil {
// 手动序列化走 map 更新,不依赖 GORM map 路径对 serializer 的支持
b, err := json.Marshal(normalizeKeyModels(*models))
if err != nil {
return fmt.Errorf("serialize models: %w", err)
}
updates["models"] = string(b)
}
if len(updates) == 0 { if len(updates) == 0 {
return nil return nil
} }
@@ -298,12 +314,17 @@ func (s *AiGatewayService) ProbeChannel(ctx context.Context, id uint) (*model.Ai
return &fresh, nil return &fresh, nil
} }
// probe 执行探测并返回 (状态, 错误摘要);同时完成模型缓存同步(保留不可用标记)。 // probe 执行探测并返回 (状态, 错误摘要);同时完成模型缓存同步(黑名单模型不入库)。
func (s *AiGatewayService) probe(ctx context.Context, cred oci.Credentials, ch *model.AiChannel) (string, string) { func (s *AiGatewayService) probe(ctx context.Context, cred oci.Credentials, ch *model.AiChannel) (string, string) {
models, err := s.client.ListGenAiModels(ctx, cred, ch.Region) models, err := s.client.ListGenAiModels(ctx, cred, ch.Region)
if err != nil { if err != nil {
return classifyProbeErr(err), truncateErr(oci.CompactError(err)) return classifyProbeErr(err), truncateErr(oci.CompactError(err))
} }
models = supportedGatewayModels(models)
models, err = s.withoutBlacklisted(ctx, models)
if err != nil {
return "error", truncateErr(err.Error())
}
if len(models) == 0 { if len(models) == 0 {
_ = s.replaceModels(ctx, ch.ID, nil) _ = s.replaceModels(ctx, ch.ID, nil)
return "no_service", "区域无可用模型(GenAI 服务不可用或未开放)" return "no_service", "区域无可用模型(GenAI 服务不可用或未开放)"
@@ -311,31 +332,12 @@ func (s *AiGatewayService) probe(ctx context.Context, cred oci.Credentials, ch *
if err := s.replaceModels(ctx, ch.ID, models); err != nil { if err := s.replaceModels(ctx, ch.ID, models); err != nil {
return "error", truncateErr(err.Error()) return "error", truncateErr(err.Error())
} }
usable, err := s.usableModels(ctx, ch.ID, models) return s.probeChat(ctx, cred, ch, models)
if err != nil {
return "error", truncateErr(err.Error())
}
return s.probeChat(ctx, cred, ch, usable)
} }
// usableModels 过滤掉缓存中已标记不可按需调用的模型,探测候选不再反复踩坑。 // probeChat 按候选顺序试调(上限 8):遇「模型不可按需调用」(微调基座 400 / 实体
func (s *AiGatewayService) usableModels(ctx context.Context, channelID uint, models []oci.GenAiModel) ([]oci.GenAiModel, error) { // 不存在 404)换下一个候选,错误信息带模型名供用户加入黑名单;401/403 与鉴权类 404
marks, err := s.loadModelMarks(ctx, channelID) // 属租户级直接定论 no_quota;其余错误(元数据标 CHAT 但实际不可对话等)累计 3 次止损。
if err != nil {
return nil, err
}
out := make([]oci.GenAiModel, 0, len(models))
for _, m := range models {
if !marks[m.Ocid].Unusable {
out = append(out, m)
}
}
return out, nil
}
// probeChat 按候选顺序试调(上限 8):遇「模型不可按需调用」标记剔除并换下一个;
// 401/403 与鉴权类 404 属租户级直接定论 no_quota;其余错误(部分模型元数据标 CHAT
// 但实际不可对话,如 voice agent)累计 3 次止损。
func (s *AiGatewayService) probeChat(ctx context.Context, cred oci.Credentials, ch *model.AiChannel, models []oci.GenAiModel) (string, string) { func (s *AiGatewayService) probeChat(ctx context.Context, cred oci.Credentials, ch *model.AiChannel, models []oci.GenAiModel) (string, string) {
status, detail := "error", "无可试调对话模型" status, detail := "error", "无可试调对话模型"
errBudget := 3 errBudget := 3
@@ -345,8 +347,7 @@ func (s *AiGatewayService) probeChat(ctx context.Context, cred oci.Credentials,
case code == 200 || code == 429: case code == 200 || code == 429:
return "ok", "" return "ok", ""
case oci.IsModelUnavailable(err): case oci.IsModelUnavailable(err):
s.markModelUnusable(ctx, ch.ID, m.Ocid, oci.CompactError(err)) status, detail = "error", truncateErr(fmt.Sprintf("%s: 不可按需调用,建议加入模型黑名单", m.Name))
status, detail = "error", truncateErr(fmt.Sprintf("%s: 不可按需调用,已从模型池剔除", m.Name))
case code == 401 || code == 403 || code == 404: case code == 401 || code == 403 || code == 404:
return "no_quota", truncateErr(oci.CompactError(err)) return "no_quota", truncateErr(oci.CompactError(err))
default: default:
@@ -359,106 +360,8 @@ func (s *AiGatewayService) probeChat(ctx context.Context, cred oci.Credentials,
return status, detail return status, detail
} }
// markModelUnusable 把 (渠道, 模型OCID) 标记为不可按需调用;失败仅记日志不阻断主流程。 // probeCandidateCap 是单次探测的候选上限:放宽到 8 让一次探测有机会越过整批
func (s *AiGatewayService) markModelUnusable(ctx context.Context, channelID uint, ocid, reason string) { // 不可按需调用的坏模型找到可用者;其他错误另有 3 次止损预算。
err := s.db.WithContext(ctx).Model(&model.AiModelCache{}).
Where("channel_id = ? AND model_ocid = ?", channelID, ocid).
Updates(map[string]any{"unusable": true, "unusable_reason": shortReason(reason), "checked_at": time.Now()}).Error
if err != nil {
log.Printf("mark model unusable: %v", err)
}
}
func shortReason(reason string) string {
if len(reason) > 200 {
return reason[:200]
}
return reason
}
// beginValidate 抢占渠道的验证执行权,同渠道同时只跑一轮。
func (s *AiGatewayService) beginValidate(id uint) bool {
s.validateMu.Lock()
defer s.validateMu.Unlock()
if s.validating[id] {
return false
}
s.validating[id] = true
return true
}
func (s *AiGatewayService) endValidate(id uint) {
s.validateMu.Lock()
delete(s.validating, id)
s.validateMu.Unlock()
}
// validateTargets 取待验证行:从未验证的新模型,以及标记超过复检间隔的模型(仅对话能力,
// 兼容存量空串);单轮上限 validateBatchCap 控制调用量。
func (s *AiGatewayService) validateTargets(ctx context.Context, channelID uint) ([]model.AiModelCache, error) {
stale := time.Now().Add(-modelRecheckGap)
var rows []model.AiModelCache
err := s.db.WithContext(ctx).
Where("channel_id = ? AND capability IN ?", channelID, []string{"CHAT", ""}).
Where("checked_at IS NULL OR (unusable = ? AND checked_at < ?)", true, stale).
Order("id ASC").Limit(validateBatchCap).Find(&rows).Error
return rows, err
}
// validateChannelModels 逐个试调渠道内待验证模型并落结论,把不可按需调用的模型
// 从池中剔除、把恢复供给的解除标记;ListModels 无字段可事先判别,只能试调习得。
func (s *AiGatewayService) validateChannelModels(ctx context.Context, channelID uint) {
if !s.beginValidate(channelID) {
return
}
defer s.endValidate(channelID)
var ch model.AiChannel
if err := s.db.WithContext(ctx).First(&ch, channelID).Error; err != nil {
return
}
cred, err := s.configs.credentialsByID(ctx, ch.OciConfigID)
if err != nil {
return
}
rows, err := s.validateTargets(ctx, channelID)
if err != nil {
log.Printf("validate channel %d models: %v", channelID, err)
return
}
for _, r := range rows {
s.validateOne(ctx, cred, &ch, r)
}
}
// validateOne 试调单个模型并落库:可用(200/429)清除标记;模型级不可用标记剔除;
// 其余 4xx 记录已检不改状态;5xx/网络类瞬态不落 checked_at,下轮再验。
func (s *AiGatewayService) validateOne(ctx context.Context, cred oci.Credentials, ch *model.AiChannel, r model.AiModelCache) {
code, err := s.client.GenAiProbeChat(ctx, cred, ch.Region, r.ModelOcid, r.Name)
updates := map[string]any{"checked_at": time.Now()}
switch {
case code == 200 || code == 429:
updates["unusable"], updates["unusable_reason"] = false, ""
case oci.IsModelUnavailable(err):
updates["unusable"], updates["unusable_reason"] = true, shortReason(oci.CompactError(err))
case code == 0 || code >= 500:
return
}
s.db.WithContext(ctx).Model(&model.AiModelCache{}).Where("id = ?", r.ID).Updates(updates)
}
// validateModelsAsync 后台验证渠道模型池(手动同步后触发,不阻塞接口响应)。
func (s *AiGatewayService) validateModelsAsync(ctx context.Context, channelID uint) {
s.wg.Add(1)
go func() {
defer s.wg.Done()
vctx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 3*time.Minute)
defer cancel()
s.validateChannelModels(vctx, channelID)
}()
}
// probeCandidateCap 是单次探测的候选上限:模型级不可用会被标记剔除不再重试,
// 放宽到 8 让一次探测有机会越过整批坏模型找到可用者;其他错误另有 3 次止损预算。
const probeCandidateCap = 8 const probeCandidateCap = 8
// probeCandidates 只取对话模型,按可靠度排序后跨厂商取候选:主流文本模型优先; // probeCandidates 只取对话模型,按可靠度排序后跨厂商取候选:主流文本模型优先;
@@ -551,8 +454,7 @@ func truncateErr(msg string) string {
return msg return msg
} }
// SyncModels 重新拉取渠道区域的模型列表并覆盖缓存(标记按 OCID 结转), // SyncModels 重新拉取渠道区域的模型列表并覆盖缓存,黑名单中的模型不入库。
// 随后触发后台验证:新模型逐个试调,不可按需调用的数十秒内从池中剔除。
func (s *AiGatewayService) SyncModels(ctx context.Context, id uint) ([]model.AiModelCache, error) { func (s *AiGatewayService) SyncModels(ctx context.Context, id uint) ([]model.AiModelCache, error) {
var ch model.AiChannel var ch model.AiChannel
if err := s.db.WithContext(ctx).First(&ch, id).Error; err != nil { if err := s.db.WithContext(ctx).First(&ch, id).Error; err != nil {
@@ -566,29 +468,23 @@ func (s *AiGatewayService) SyncModels(ctx context.Context, id uint) ([]model.AiM
if err != nil { if err != nil {
return nil, fmt.Errorf("同步模型失败:%s", oci.CompactError(err)) return nil, fmt.Errorf("同步模型失败:%s", oci.CompactError(err))
} }
models = supportedGatewayModels(models)
if models, err = s.withoutBlacklisted(ctx, models); err != nil {
return nil, err
}
if err := s.replaceModels(ctx, id, models); err != nil { if err := s.replaceModels(ctx, id, models); err != nil {
return nil, err return nil, err
} }
s.validateModelsAsync(ctx, id)
return s.channelModels(ctx, id) return s.channelModels(ctx, id)
} }
// replaceModels 以事务整组覆盖渠道模型缓存,按 OCID 结转不可用标记与验证时间 // replaceModels 以事务整组覆盖渠道模型缓存
// (OCID 变化视为新条目,自然回到待验证状态)。
func (s *AiGatewayService) replaceModels(ctx context.Context, channelID uint, models []oci.GenAiModel) error { func (s *AiGatewayService) replaceModels(ctx context.Context, channelID uint, models []oci.GenAiModel) error {
marks, err := s.loadModelMarks(ctx, channelID)
if err != nil {
return err
}
rows := make([]model.AiModelCache, 0, len(models)) rows := make([]model.AiModelCache, 0, len(models))
now := time.Now() now := time.Now()
for _, m := range models { for _, m := range models {
row := model.AiModelCache{ChannelID: channelID, ModelOcid: m.Ocid, Name: m.Name, Vendor: m.Vendor, rows = append(rows, model.AiModelCache{ChannelID: channelID, ModelOcid: m.Ocid, Name: m.Name, Vendor: m.Vendor,
Capability: m.Capability, SyncedAt: now, DeprecatedAt: m.Deprecated, RetiredAt: m.Retired} Capability: m.Capability, SyncedAt: now, DeprecatedAt: m.Deprecated, RetiredAt: m.Retired})
if prev, ok := marks[m.Ocid]; ok {
row.Unusable, row.UnusableReason, row.CheckedAt = prev.Unusable, prev.UnusableReason, prev.CheckedAt
}
rows = append(rows, row)
} }
return s.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error { return s.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
if err := tx.Where("channel_id = ?", channelID).Delete(&model.AiModelCache{}).Error; err != nil { if err := tx.Where("channel_id = ?", channelID).Delete(&model.AiModelCache{}).Error; err != nil {
@@ -601,33 +497,17 @@ func (s *AiGatewayService) replaceModels(ctx context.Context, channelID uint, mo
}) })
} }
// loadModelMarks 取渠道内带标记或已验证过的行(OCID → 行),供同步结转与候选过滤。
func (s *AiGatewayService) loadModelMarks(ctx context.Context, channelID uint) (map[string]model.AiModelCache, error) {
var rows []model.AiModelCache
err := s.db.WithContext(ctx).Select("model_ocid", "unusable", "unusable_reason", "checked_at").
Where("channel_id = ? AND (unusable = ? OR checked_at IS NOT NULL)", channelID, true).Find(&rows).Error
if err != nil {
return nil, fmt.Errorf("load model marks: %w", err)
}
marks := make(map[string]model.AiModelCache, len(rows))
for _, r := range rows {
marks[r.ModelOcid] = r
}
return marks, nil
}
func (s *AiGatewayService) channelModels(ctx context.Context, channelID uint) ([]model.AiModelCache, error) { func (s *AiGatewayService) channelModels(ctx context.Context, channelID uint) ([]model.AiModelCache, error) {
var rows []model.AiModelCache var rows []model.AiModelCache
err := s.db.WithContext(ctx).Where("channel_id = ?", channelID).Order("name ASC").Find(&rows).Error err := s.db.WithContext(ctx).Where("channel_id = ?", channelID).Order("name ASC").Find(&rows).Error
return rows, err return rows, err
} }
// GatewayModels 聚合启用渠道的可用模型(按名称去重,剔除不可按需调用标记), // GatewayModels 聚合启用渠道的可用模型(按名称去重),供 /ai/v1/models;
// 供 /ai/v1/models;group 非空时仅聚合该分组渠道(与密钥分组路由口径一致)。 // group 非空时仅聚合该分组渠道(与密钥分组路由口径一致)。
func (s *AiGatewayService) GatewayModels(ctx context.Context, group string) (aiwire.ModelList, error) { func (s *AiGatewayService) GatewayModels(ctx context.Context, group string) (aiwire.ModelList, error) {
q := s.db.WithContext(ctx). q := s.db.WithContext(ctx).
Joins("JOIN ai_channels ON ai_channels.id = ai_model_caches.channel_id AND ai_channels.enabled = ?", true). Joins("JOIN ai_channels ON ai_channels.id = ai_model_caches.channel_id AND ai_channels.enabled = ?", true)
Where("(ai_model_caches.unusable = ? OR ai_model_caches.unusable IS NULL)", false)
if group != "" { if group != "" {
q = q.Where("ai_channels.channel_group = ?", group) q = q.Where("ai_channels.channel_group = ?", group)
} }
@@ -658,7 +538,6 @@ func (s *AiGatewayService) DeprecatingModels(ctx context.Context, within time.Du
err := s.db.WithContext(ctx). err := s.db.WithContext(ctx).
Where("(retired_at IS NOT NULL AND retired_at > ? AND retired_at <= ?) OR (deprecated_at IS NOT NULL AND deprecated_at >= ? AND deprecated_at <= ?)", Where("(retired_at IS NOT NULL AND retired_at > ? AND retired_at <= ?) OR (deprecated_at IS NOT NULL AND deprecated_at >= ? AND deprecated_at <= ?)",
now, deadline, now, deadline). now, deadline, now, deadline).
Where("(unusable = ? OR unusable IS NULL)", false).
Order("name ASC").Find(&rows).Error Order("name ASC").Find(&rows).Error
if err != nil { if err != nil {
return nil, err return nil, err
@@ -679,7 +558,7 @@ func (s *AiGatewayService) DeprecatingModels(ctx context.Context, within time.Du
return out, nil return out, nil
} }
// ProbeAll 逐个探测全部渠道并顺带验证模型池,返回状态汇总;供 AI 探测后台任务调用。 // ProbeAll 逐个探测全部渠道,返回状态汇总;供 AI 探测后台任务调用。
func (s *AiGatewayService) ProbeAll(ctx context.Context) (string, error) { func (s *AiGatewayService) ProbeAll(ctx context.Context) (string, error) {
chs, err := s.Channels(ctx) chs, err := s.Channels(ctx)
if err != nil { if err != nil {
@@ -695,8 +574,6 @@ func (s *AiGatewayService) ProbeAll(ctx context.Context) (string, error) {
continue continue
} }
counts[fresh.ProbeStatus]++ counts[fresh.ProbeStatus]++
// 后台任务里同步执行:新模型验证入池、坏模型定期复检,零人工收敛
s.validateChannelModels(ctx, ch.ID)
} }
msg := fmt.Sprintf("probed %d: %d ok, %d no_service, %d no_quota, %d error", msg := fmt.Sprintf("probed %d: %d ok, %d no_service, %d no_quota, %d error",
len(chs), counts["ok"], counts["no_service"], counts["no_quota"], counts["error"]) len(chs), counts["ok"], counts["no_service"], counts["no_quota"], counts["error"])
@@ -706,6 +583,99 @@ func (s *AiGatewayService) ProbeAll(ctx context.Context) (string, error) {
return msg, nil return msg, nil
} }
// ---- 模型黑名单 ----
// Blacklist 列出全部黑名单模型(按名称排序)。
func (s *AiGatewayService) Blacklist(ctx context.Context) ([]model.AiModelBlacklist, error) {
var rows []model.AiModelBlacklist
err := s.db.WithContext(ctx).Order("name ASC").Find(&rows).Error
return rows, err
}
// AddBlacklist 把模型名加入黑名单并删除全部渠道缓存中的同名条目;
// 该模型此后同步 / 探测均被过滤,直到移出黑名单后重新同步。
func (s *AiGatewayService) AddBlacklist(ctx context.Context, name string) (*model.AiModelBlacklist, error) {
name = strings.TrimSpace(name)
if name == "" {
return nil, fmt.Errorf("模型名不能为空")
}
row := model.AiModelBlacklist{Name: name}
err := s.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
var n int64
if err := tx.Model(&model.AiModelBlacklist{}).Where("name = ?", name).Count(&n).Error; err != nil {
return err
}
if n > 0 {
return fmt.Errorf("模型已在黑名单中")
}
if err := tx.Create(&row).Error; err != nil {
return err
}
return tx.Where("name = ?", name).Delete(&model.AiModelCache{}).Error
})
if err != nil {
return nil, err
}
return &row, nil
}
// RemoveBlacklist 把模型移出黑名单;缓存不回填,下次同步 / 探测自然恢复入池。
func (s *AiGatewayService) RemoveBlacklist(ctx context.Context, id uint) error {
res := s.db.WithContext(ctx).Delete(&model.AiModelBlacklist{}, id)
if res.Error != nil {
return res.Error
}
if res.RowsAffected == 0 {
return fmt.Errorf("黑名单条目不存在")
}
return nil
}
// withoutBlacklisted 过滤掉黑名单中的模型,同步与探测入库前统一经此收口。
// supportedGatewayModels 过滤模型目录:对话模型仅保留实测支持 OpenAI 兼容面的
// 厂商(xai / meta / openai)——typed chat 面已剔除,google / cohere 对话模型无
// 上游通路,不入目录(不出现在列表、路由与探测候选);EMBEDDING 模型不受影响。
func supportedGatewayModels(models []oci.GenAiModel) []oci.GenAiModel {
out := make([]oci.GenAiModel, 0, len(models))
for _, m := range models {
if m.Capability == "CHAT" && !compatChatVendor(m.Name) {
continue
}
out = append(out, m)
}
return out
}
func compatChatVendor(model string) bool {
for _, prefix := range []string{"xai.", "meta.", "openai."} {
if strings.HasPrefix(model, prefix) {
return true
}
}
return false
}
func (s *AiGatewayService) withoutBlacklisted(ctx context.Context, models []oci.GenAiModel) ([]oci.GenAiModel, error) {
var names []string
if err := s.db.WithContext(ctx).Model(&model.AiModelBlacklist{}).Pluck("name", &names).Error; err != nil {
return nil, fmt.Errorf("读取模型黑名单: %w", err)
}
if len(names) == 0 {
return models, nil
}
banned := make(map[string]bool, len(names))
for _, n := range names {
banned[n] = true
}
out := make([]oci.GenAiModel, 0, len(models))
for _, m := range models {
if !banned[m.Name] {
out = append(out, m)
}
}
return out, nil
}
// ---- 调用日志 ---- // ---- 调用日志 ----
// LogCall 落一条调用日志(仅元数据与用量,永不含请求 / 响应正文),返回落库 ID 供内容日志关联(失败为 0)。 // LogCall 落一条调用日志(仅元数据与用量,永不含请求 / 响应正文),返回落库 ID 供内容日志关联(失败为 0)。
+25 -25
View File
@@ -3,6 +3,7 @@ package service
import ( import (
"context" "context"
"errors" "errors"
"io"
"math/rand" "math/rand"
"time" "time"
@@ -26,24 +27,25 @@ type aiCandidate struct {
modelOcid string modelOcid string
} }
// Chat 编排非流式调用:选渠道 → 调用 → 可重试错误换渠道(整请求上限 3 次)。 // RespPassthrough 编排一次非流式直通调用:选渠道(priority→加权随机)→ 调用 →
// group 非空时只在同分组渠道内路由(取自调用密钥) // 可重试错误换渠道(整请求上限 3 次)并维护熔断;group 非空时只在同分组渠道内路由。
func (s *AiGatewayService) Chat(ctx context.Context, ir aiwire.ChatRequest, group string) (*aiwire.ChatResponse, ChatMeta, error) { // 上游为 OpenAI-compatible /actions/v1/responses(实测可用,无 Oracle 文档合同)。
func (s *AiGatewayService) RespPassthrough(ctx context.Context, raw []byte, modelName, group string) ([]byte, ChatMeta, error) {
meta := ChatMeta{} meta := ChatMeta{}
excluded := map[uint]bool{} excluded := map[uint]bool{}
var lastErr error var lastErr error
for attempt := 0; attempt < 3; attempt++ { for attempt := 0; attempt < 3; attempt++ {
cand, err := s.pick(ctx, ir.Model, group, "CHAT", excluded) cand, err := s.pick(ctx, modelName, group, "CHAT", excluded)
if err != nil { if err != nil {
return nil, meta, firstErr(lastErr, err) return nil, meta, firstErr(lastErr, err)
} }
meta.ChannelID, meta.ChannelName = cand.ch.ID, cand.ch.Name meta.ChannelID, meta.ChannelName = cand.ch.ID, cand.ch.Name
resp, err := s.callOnce(ctx, cand, ir) payload, err := s.passthroughOnce(ctx, cand, raw)
if err == nil { if err == nil {
s.markSuccess(ctx, cand.ch.ID) s.markSuccess(ctx, cand.ch.ID)
return resp, meta, nil return payload, meta, nil
} }
retry, penalize := s.noteCallErr(ctx, cand, err) retry, penalize := switchable(err)
if !retry { if !retry {
return nil, meta, err return nil, meta, err
} }
@@ -57,22 +59,22 @@ func (s *AiGatewayService) Chat(ctx context.Context, ir aiwire.ChatRequest, grou
return nil, meta, lastErr return nil, meta, lastErr
} }
func (s *AiGatewayService) callOnce(ctx context.Context, cand *aiCandidate, ir aiwire.ChatRequest) (*aiwire.ChatResponse, error) { func (s *AiGatewayService) passthroughOnce(ctx context.Context, cand *aiCandidate, raw []byte) ([]byte, error) {
cred, err := s.configs.credentialsByID(ctx, cand.ch.OciConfigID) cred, err := s.configs.credentialsByID(ctx, cand.ch.OciConfigID)
if err != nil { if err != nil {
return nil, err return nil, err
} }
return s.client.GenAiChat(ctx, cred, cand.ch.Region, cand.modelOcid, ir) return s.client.GenAiCompatResponses(ctx, cred, cand.ch.Region, raw)
} }
// OpenStream 编排流式调用:流建立成功即绑定渠道,建立失败可换渠道重试。 // RespPassthroughStream 编排流式直通:流建立成功即绑定渠道,建立失败按 switchable
// group 语义与 Chat 相同 // 换渠道重试;建立后的中断不重试、不计熔断(与 OpenStream 语义一致)
func (s *AiGatewayService) OpenStream(ctx context.Context, ir aiwire.ChatRequest, group string) (oci.GenAiStream, ChatMeta, error) { func (s *AiGatewayService) RespPassthroughStream(ctx context.Context, raw []byte, modelName, group string) (io.ReadCloser, ChatMeta, error) {
meta := ChatMeta{} meta := ChatMeta{}
excluded := map[uint]bool{} excluded := map[uint]bool{}
var lastErr error var lastErr error
for attempt := 0; attempt < 3; attempt++ { for attempt := 0; attempt < 3; attempt++ {
cand, err := s.pick(ctx, ir.Model, group, "CHAT", excluded) cand, err := s.pick(ctx, modelName, group, "CHAT", excluded)
if err != nil { if err != nil {
return nil, meta, firstErr(lastErr, err) return nil, meta, firstErr(lastErr, err)
} }
@@ -81,12 +83,12 @@ func (s *AiGatewayService) OpenStream(ctx context.Context, ir aiwire.ChatRequest
if err != nil { if err != nil {
return nil, meta, err return nil, meta, err
} }
stream, err := s.client.GenAiChatStream(ctx, cred, cand.ch.Region, cand.modelOcid, ir) stream, err := s.client.GenAiCompatResponsesStream(ctx, cred, cand.ch.Region, raw)
if err == nil { if err == nil {
s.markSuccess(ctx, cand.ch.ID) s.markSuccess(ctx, cand.ch.ID)
return stream, meta, nil return stream, meta, nil
} }
retry, penalize := s.noteCallErr(ctx, cand, err) retry, penalize := switchable(err)
if !retry { if !retry {
return nil, meta, err return nil, meta, err
} }
@@ -108,12 +110,11 @@ func firstErr(lastErr, pickErr error) error {
return pickErr return pickErr
} }
// noteCallErr 汇总一次调用失败:模型级不可用(微调基座 400 / 实体不存在 404)先标记 // switchable 判断调用失败是否换渠道重试、是否计入熔断:模型级不可用(微调基座
// 剔除该 (渠道, 模型),换渠道重试且不计熔断——这是模型×区域供给问题而非渠道健康问题; // 400 / 实体不存在 404)换渠道但不计熔断——这是模型×区域供给问题而非渠道健康
// 429 / 5xx / 网络错误换渠道并计熔断;其余 4xx 直接透传。 // 问题,持久解决靠加入模型黑名单;429 / 5xx / 网络错误换渠道并计熔断;其余 4xx 直接透传。
func (s *AiGatewayService) noteCallErr(ctx context.Context, cand *aiCandidate, err error) (retry, penalize bool) { func switchable(err error) (retry, penalize bool) {
if oci.IsModelUnavailable(err) { if oci.IsModelUnavailable(err) {
s.markModelUnusable(ctx, cand.ch.ID, cand.modelOcid, oci.CompactError(err))
return true, false return true, false
} }
if status, ok := oci.ServiceStatus(err); ok { if status, ok := oci.ServiceStatus(err); ok {
@@ -154,11 +155,10 @@ func (s *AiGatewayService) pick(ctx context.Context, modelName, group, capabilit
return &aiCandidate{ch: chosen, modelOcid: ocids[chosen.ID]}, nil return &aiCandidate{ch: chosen, modelOcid: ocids[chosen.ID]}, nil
} }
// modelChannels 查出提供该模型的渠道 ID 及各自的模型 OCID(剔除不可按需调用标记); // modelChannels 查出提供该模型的渠道 ID 及各自的模型 OCID;
// capability=CHAT 时兼容存量空串(加列前只同步对话模型);unusable 需兼容 NULL // capability=CHAT 时兼容存量空串(加列前只同步对话模型)
// (AutoMigrate 加列后、首次重同步前的存量行)。
func (s *AiGatewayService) modelChannels(ctx context.Context, modelName, capability string) (map[uint]string, []uint, error) { func (s *AiGatewayService) modelChannels(ctx context.Context, modelName, capability string) (map[uint]string, []uint, error) {
q := s.db.WithContext(ctx).Where("name = ? AND (unusable = ? OR unusable IS NULL)", modelName, false) q := s.db.WithContext(ctx).Where("name = ?", modelName)
if capability == "CHAT" { if capability == "CHAT" {
q = q.Where("capability IN ?", []string{"CHAT", ""}) q = q.Where("capability IN ?", []string{"CHAT", ""})
} else { } else {
@@ -234,7 +234,7 @@ func (s *AiGatewayService) Embeddings(ctx context.Context, req aiwire.Embeddings
s.markSuccess(ctx, cand.ch.ID) s.markSuccess(ctx, cand.ch.ID)
return resp, meta, nil return resp, meta, nil
} }
retry, penalize := s.noteCallErr(ctx, cand, err) retry, penalize := switchable(err)
if !retry { if !retry {
return nil, meta, err return nil, meta, err
} }
+29
View File
@@ -0,0 +1,29 @@
package service
import (
"strings"
"testing"
"oci-portal/internal/oci"
)
// TestSupportedGatewayModels 断言 CHAT 目录仅保留兼容面厂商,EMBEDDING 不受影响。
func TestSupportedGatewayModels(t *testing.T) {
in := []oci.GenAiModel{
{Name: "xai.grok-4.3", Capability: "CHAT"},
{Name: "meta.llama-4-maverick-17b-128e", Capability: "CHAT"},
{Name: "openai.gpt-oss-120b", Capability: "CHAT"},
{Name: "google.gemini-2.5-flash", Capability: "CHAT"},
{Name: "cohere.command-a-03-2025", Capability: "CHAT"},
{Name: "cohere.embed-v4.0", Capability: "EMBEDDING"},
}
out := supportedGatewayModels(in)
names := make([]string, 0, len(out))
for _, m := range out {
names = append(names, m.Name)
}
want := "xai.grok-4.3,meta.llama-4-maverick-17b-128e,openai.gpt-oss-120b,cohere.embed-v4.0"
if got := strings.Join(names, ","); got != want {
t.Fatalf("过滤结果 = %s, want %s", got, want)
}
}
+261 -223
View File
@@ -1,9 +1,11 @@
package service package service
import ( import (
"bytes"
"context" "context"
"errors" "errors"
"fmt" "fmt"
"io"
"strings" "strings"
"testing" "testing"
"time" "time"
@@ -46,14 +48,45 @@ type gatewayStubClient struct {
probeErr error probeErr error
// probeSeq 非空时逐次弹出,模拟按候选依次试调;弹尽后回落 probeCode/probeErr // probeSeq 非空时逐次弹出,模拟按候选依次试调;弹尽后回落 probeCode/probeErr
probeSeq []probeResult probeSeq []probeResult
chatResp *aiwire.ChatResponse // probedNames 记录试调过的模型名,供断言候选过滤
chatErrs []error probedNames []string
chatCalls int chatCalls int
regions []string regions []string
embedVecs [][]float32 embedVecs [][]float32
embedUsage *aiwire.Usage embedUsage *aiwire.Usage
embedErr error embedErr error
passPayload []byte
passErrs []error
passCalls int
passRegions []string
}
func (f *gatewayStubClient) GenAiCompatResponses(ctx context.Context, cred oci.Credentials, region string, body []byte) ([]byte, error) {
f.passCalls++
f.passRegions = append(f.passRegions, region)
if len(f.passErrs) > 0 {
err := f.passErrs[0]
f.passErrs = f.passErrs[1:]
if err != nil {
return nil, err
}
}
return f.passPayload, nil
}
func (f *gatewayStubClient) GenAiCompatResponsesStream(ctx context.Context, cred oci.Credentials, region string, body []byte) (io.ReadCloser, error) {
f.passCalls++
f.passRegions = append(f.passRegions, region)
if len(f.passErrs) > 0 {
err := f.passErrs[0]
f.passErrs = f.passErrs[1:]
if err != nil {
return nil, err
}
}
return io.NopCloser(bytes.NewReader(f.passPayload)), nil
} }
func (f *gatewayStubClient) GenAiEmbed(ctx context.Context, cred oci.Credentials, region, modelOcid string, inputs []string, dimensions *int) ([][]float32, *aiwire.Usage, error) { func (f *gatewayStubClient) GenAiEmbed(ctx context.Context, cred oci.Credentials, region, modelOcid string, inputs []string, dimensions *int) ([][]float32, *aiwire.Usage, error) {
@@ -71,6 +104,7 @@ type probeResult struct {
} }
func (f *gatewayStubClient) GenAiProbeChat(ctx context.Context, cred oci.Credentials, region, modelOcid, modelName string) (int, error) { func (f *gatewayStubClient) GenAiProbeChat(ctx context.Context, cred oci.Credentials, region, modelOcid, modelName string) (int, error) {
f.probedNames = append(f.probedNames, modelName)
if len(f.probeSeq) > 0 { if len(f.probeSeq) > 0 {
r := f.probeSeq[0] r := f.probeSeq[0]
f.probeSeq = f.probeSeq[1:] f.probeSeq = f.probeSeq[1:]
@@ -79,23 +113,10 @@ func (f *gatewayStubClient) GenAiProbeChat(ctx context.Context, cred oci.Credent
return f.probeCode, f.probeErr return f.probeCode, f.probeErr
} }
func (f *gatewayStubClient) GenAiChat(ctx context.Context, cred oci.Credentials, region, modelOcid string, ir aiwire.ChatRequest) (*aiwire.ChatResponse, error) {
f.chatCalls++
f.regions = append(f.regions, region)
if len(f.chatErrs) > 0 {
err := f.chatErrs[0]
f.chatErrs = f.chatErrs[1:]
if err != nil {
return nil, err
}
}
return f.chatResp, nil
}
func newTestGateway(t *testing.T, client oci.Client) (*AiGatewayService, *OciConfigService) { func newTestGateway(t *testing.T, client oci.Client) (*AiGatewayService, *OciConfigService) {
t.Helper() t.Helper()
svc := newTestService(t, client) svc := newTestService(t, client)
if err := svc.db.AutoMigrate(&model.AiKey{}, &model.AiChannel{}, &model.AiModelCache{}, &model.AiCallLog{}); err != nil { if err := svc.db.AutoMigrate(&model.AiKey{}, &model.AiChannel{}, &model.AiModelCache{}, &model.AiModelBlacklist{}, &model.AiCallLog{}); err != nil {
t.Fatalf("auto migrate ai tables: %v", err) t.Fatalf("auto migrate ai tables: %v", err)
} }
return NewAiGatewayService(svc.db, svc, client), svc return NewAiGatewayService(svc.db, svc, client), svc
@@ -105,14 +126,14 @@ func TestAiKeyLifecycle(t *testing.T) {
gw, _ := newTestGateway(t, &gatewayStubClient{fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}}) gw, _ := newTestGateway(t, &gatewayStubClient{fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}})
ctx := context.Background() ctx := context.Background()
raw, key, err := gw.CreateKey(ctx, "免费-ai-api-key", "test-api-key", "") raw, key, err := gw.CreateKey(ctx, "免费-ai-api-key", "test-api-key", "", nil)
if err != nil || raw != "test-api-key" || key.Tail != "-key" { if err != nil || raw != "test-api-key" || key.Tail != "-key" {
t.Fatalf("CreateKey 自定义值 = %q %+v, %v", raw, key, err) t.Fatalf("CreateKey 自定义值 = %q %+v, %v", raw, key, err)
} }
if _, _, err := gw.CreateKey(ctx, "short", "abc", ""); err == nil { if _, _, err := gw.CreateKey(ctx, "short", "abc", "", nil); err == nil {
t.Error("过短自定义密钥应被拒绝") t.Error("过短自定义密钥应被拒绝")
} }
auto, _, err := gw.CreateKey(ctx, "auto", "", "") auto, _, err := gw.CreateKey(ctx, "auto", "", "", nil)
if err != nil || len(auto) < 40 || auto[:3] != "sk-" { if err != nil || len(auto) < 40 || auto[:3] != "sk-" {
t.Fatalf("CreateKey 随机值 = %q, %v", auto, err) t.Fatalf("CreateKey 随机值 = %q, %v", auto, err)
} }
@@ -124,7 +145,7 @@ func TestAiKeyLifecycle(t *testing.T) {
t.Errorf("错误密钥 err = %v", err) t.Errorf("错误密钥 err = %v", err)
} }
off := false off := false
_ = gw.UpdateKey(ctx, key.ID, "", &off, nil) _ = gw.UpdateKey(ctx, key.ID, "", &off, nil, nil)
if _, err := gw.VerifyKey(ctx, "test-api-key"); !errors.Is(err, ErrAiKeyInvalid) { if _, err := gw.VerifyKey(ctx, "test-api-key"); !errors.Is(err, ErrAiKeyInvalid) {
t.Errorf("禁用后 VerifyKey err = %v", err) t.Errorf("禁用后 VerifyKey err = %v", err)
} }
@@ -187,51 +208,6 @@ func seedChannel(t *testing.T, gw *AiGatewayService, cfgID uint, region string,
return ch return ch
} }
func TestAiChatRetrySwitchesChannel(t *testing.T) {
client := &gatewayStubClient{
fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}},
chatResp: &aiwire.ChatResponse{Model: "meta.llama-3.3-70b-instruct", Choices: []aiwire.Choice{{Message: aiwire.ChatMessage{Role: "assistant", Content: aiwire.NewTextContent("hi")}, FinishReason: "stop"}}},
chatErrs: []error{stubServiceError{status: 429}},
}
gw, svc := newTestGateway(t, client)
cfg := importAliveConfig(t, svc)
// 两个同优先级渠道
seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1)
seedChannel(t, gw, cfg.ID, "us-chicago-1", 1, 1)
ctx := context.Background()
resp, meta, err := gw.Chat(ctx, aiwire.ChatRequest{Model: "meta.llama-3.3-70b-instruct", Messages: []aiwire.ChatMessage{{Role: "user", Content: aiwire.NewTextContent("你好")}}}, "")
if err != nil || resp == nil {
t.Fatalf("Chat = %v, %v", resp, err)
}
if meta.Retries != 1 || client.chatCalls != 2 {
t.Errorf("应换渠道重试一次: retries=%d calls=%d", meta.Retries, client.chatCalls)
}
if len(client.regions) != 2 && client.regions[0] == client.regions[1] {
t.Errorf("重试未换渠道: %v", client.regions)
}
// 未知模型
if _, _, err := gw.Chat(ctx, aiwire.ChatRequest{Model: "no-such-model"}, ""); !errors.Is(err, ErrAiUnknownModel) {
t.Errorf("未知模型 err = %v", err)
}
}
func TestAiChatNonRetryablePassThrough(t *testing.T) {
client := &gatewayStubClient{
fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}},
chatErrs: []error{stubServiceError{status: 400}},
}
gw, svc := newTestGateway(t, client)
cfg := importAliveConfig(t, svc)
seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1)
seedChannel(t, gw, cfg.ID, "us-chicago-1", 1, 1)
_, meta, err := gw.Chat(context.Background(), aiwire.ChatRequest{Model: "meta.llama-3.3-70b-instruct"}, "")
if err == nil || meta.Retries != 0 || client.chatCalls != 1 {
t.Errorf("400 应直接透传不重试: err=%v retries=%d calls=%d", err, meta.Retries, client.chatCalls)
}
}
func TestPickPriorityAndBreaker(t *testing.T) { func TestPickPriorityAndBreaker(t *testing.T) {
gw, svc := newTestGateway(t, &gatewayStubClient{fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}}) gw, svc := newTestGateway(t, &gatewayStubClient{fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}})
cfg := importAliveConfig(t, svc) cfg := importAliveConfig(t, svc)
@@ -283,7 +259,7 @@ func TestMarkFailureBackoff(t *testing.T) {
func TestAiGroupRouting(t *testing.T) { func TestAiGroupRouting(t *testing.T) {
client := &gatewayStubClient{ client := &gatewayStubClient{
fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}, fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}},
chatResp: &aiwire.ChatResponse{Model: "meta.llama-3.3-70b-instruct", Choices: []aiwire.Choice{{Message: aiwire.ChatMessage{Role: "assistant", Content: aiwire.NewTextContent("hi")}, FinishReason: "stop"}}}, passPayload: []byte(`{"id":"resp_1","usage":{"input_tokens":3,"output_tokens":1}}`),
} }
gw, svc := newTestGateway(t, client) gw, svc := newTestGateway(t, client)
cfg := importAliveConfig(t, svc) cfg := importAliveConfig(t, svc)
@@ -292,17 +268,17 @@ func TestAiGroupRouting(t *testing.T) {
gw.db.Model(vip).Update("channel_group", "vip") gw.db.Model(vip).Update("channel_group", "vip")
gw.db.Create(&model.AiModelCache{ChannelID: other.ID, ModelOcid: "ocid1..cr", Name: "cohere.command-r", Vendor: "cohere", SyncedAt: time.Now()}) gw.db.Create(&model.AiModelCache{ChannelID: other.ID, ModelOcid: "ocid1..cr", Name: "cohere.command-r", Vendor: "cohere", SyncedAt: time.Now()})
ctx := context.Background() ctx := context.Background()
req := aiwire.ChatRequest{Model: "meta.llama-3.3-70b-instruct", Messages: []aiwire.ChatMessage{{Role: "user", Content: aiwire.NewTextContent("你好")}}} body := []byte(`{"model":"meta.llama-3.3-70b-instruct","input":"你好"}`)
// 分组密钥只落同分组渠道 // 分组密钥只落同分组渠道
for i := 0; i < 5; i++ { for i := 0; i < 5; i++ {
_, meta, err := gw.Chat(ctx, req, "vip") _, meta, err := gw.RespPassthrough(ctx, body, "meta.llama-3.3-70b-instruct", "vip")
if err != nil || meta.ChannelID != vip.ID { if err != nil || meta.ChannelID != vip.ID {
t.Fatalf("vip 组第 %d 次落点 = %d, err %v, want %d", i, meta.ChannelID, err, vip.ID) t.Fatalf("vip 组第 %d 次落点 = %d, err %v, want %d", i, meta.ChannelID, err, vip.ID)
} }
} }
// 分组内无渠道 → 无可用渠道 // 分组内无渠道 → 无可用渠道
if _, _, err := gw.Chat(ctx, req, "nope"); !errors.Is(err, ErrAiNoChannel) { if _, _, err := gw.RespPassthrough(ctx, body, "meta.llama-3.3-70b-instruct", "nope"); !errors.Is(err, ErrAiNoChannel) {
t.Errorf("空分组 err = %v, want ErrAiNoChannel", err) t.Errorf("空分组 err = %v, want ErrAiNoChannel", err)
} }
// 模型列表按分组过滤 // 模型列表按分组过滤
@@ -315,12 +291,12 @@ func TestAiGroupRouting(t *testing.T) {
t.Errorf("不限组模型数 = %d, want 2", len(all.Data)) t.Errorf("不限组模型数 = %d, want 2", len(all.Data))
} }
// 密钥分组落库 // 密钥分组落库
_, key, err := gw.CreateKey(ctx, "vip-key", "vip-secret-1234", "vip") _, key, err := gw.CreateKey(ctx, "vip-key", "vip-secret-1234", "vip", nil)
if err != nil || key.Group != "vip" { if err != nil || key.Group != "vip" {
t.Fatalf("CreateKey group = %+v, %v", key, err) t.Fatalf("CreateKey group = %+v, %v", key, err)
} }
empty := "" empty := ""
_ = gw.UpdateKey(ctx, key.ID, "", nil, &empty) _ = gw.UpdateKey(ctx, key.ID, "", nil, &empty, nil)
var fresh model.AiKey var fresh model.AiKey
gw.db.First(&fresh, key.ID) gw.db.First(&fresh, key.ID)
if fresh.Group != "" { if fresh.Group != "" {
@@ -465,7 +441,8 @@ func entityNotFoundErr() stubServiceError {
} }
func TestProbeSkipsUnavailableModels(t *testing.T) { func TestProbeSkipsUnavailableModels(t *testing.T) {
// 首选 llama 不可按需调用(基座 400 / 实体 404)→ 标记剔除 → 换候选成功 → 渠道判可用 // 首选 llama 不可按需调用(基座 400 / 实体 404)→ 换候选成功 → 渠道判可用;
// 模型全量入库不再自动标记,持久剔除交由用户手动拉黑
tests := []struct { tests := []struct {
name string name string
bad probeResult bad probeResult
@@ -496,112 +473,16 @@ func TestProbeSkipsUnavailableModels(t *testing.T) {
if err != nil || probed.ProbeStatus != "ok" { if err != nil || probed.ProbeStatus != "ok" {
t.Fatalf("坏候选后应换候选并判可用: %+v, %v", probed, err) t.Fatalf("坏候选后应换候选并判可用: %+v, %v", probed, err)
} }
var row model.AiModelCache rows, _ := gw.channelModels(ctx, ch.ID)
if err := gw.db.Where("channel_id = ? AND model_ocid = ?", ch.ID, "m1").First(&row).Error; err != nil || !row.Unusable { if len(rows) != 3 {
t.Fatalf("首个坏候选应被标记不可用: %+v, %v", row, err) t.Errorf("模型应全量入库, got %d", len(rows))
}
// 再次探测:同步保留标记,m1 不再进候选(probeSeq 只需一次 200)
client.probeSeq = []probeResult{{200, nil}}
probed, err = gw.ProbeChannel(ctx, ch.ID)
if err != nil || probed.ProbeStatus != "ok" {
t.Fatalf("复测应跳过已标记模型: %+v, %v", probed, err)
}
var again model.AiModelCache
gw.db.Where("channel_id = ? AND model_ocid = ?", ch.ID, "m1").First(&again)
if !again.Unusable {
t.Error("探测触发的同步应保留不可用标记")
}
// 手动同步同样保留标记(坏模型不随重新同步复活);近期已检的标记不被后台验证翻转
client.probeCode = 200
if _, err := gw.SyncModels(ctx, ch.ID); err != nil {
t.Fatalf("SyncModels: %v", err)
}
gw.Wait()
var kept model.AiModelCache
gw.db.Where("channel_id = ? AND model_ocid = ?", ch.ID, "m1").First(&kept)
if !kept.Unusable {
t.Error("手动同步不应清除不可用标记")
} }
}) })
} }
} }
func TestValidateModelsAfterSync(t *testing.T) {
// 同步后后台验证:坏模型标记剔除、好模型记录已检、其他 4xx 不改状态、非对话模型不试调
client := &gatewayStubClient{
fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}},
models: []oci.GenAiModel{
{Ocid: "v1", Name: "meta.llama-4-maverick", Vendor: "meta", Capability: "CHAT"},
{Ocid: "v2", Name: "xai.grok-4", Vendor: "xai", Capability: "CHAT"},
{Ocid: "v3", Name: "xai.grok-voice-agent", Vendor: "xai", Capability: "CHAT"},
{Ocid: "v4", Name: "cohere.embed-v4.0", Vendor: "cohere", Capability: "EMBEDDING"},
},
probeSeq: []probeResult{{404, entityNotFoundErr()}, {200, nil}, {400, stubServiceError{status: 400}}},
}
gw, svc := newTestGateway(t, client)
cfg := importAliveConfig(t, svc)
ctx := context.Background()
ch, err := gw.CreateChannel(ctx, ChannelInput{OciConfigID: cfg.ID, Region: "us-ashburn-1"})
if err != nil {
t.Fatalf("CreateChannel: %v", err)
}
if _, err := gw.SyncModels(ctx, ch.ID); err != nil {
t.Fatalf("SyncModels: %v", err)
}
gw.Wait()
want := map[string]struct {
unusable bool
checked bool
}{
"v1": {true, true}, // 实体 404 → 标记剔除
"v2": {false, true}, // 200 → 可用已检
"v3": {false, true}, // 普通 400 → 已检不标记
"v4": {false, false}, // EMBEDDING 不试调
}
rows, _ := gw.channelModels(ctx, ch.ID)
for _, r := range rows {
w := want[r.ModelOcid]
if r.Unusable != w.unusable || (r.CheckedAt != nil) != w.checked {
t.Errorf("%s: unusable=%v checked=%v, want %+v", r.ModelOcid, r.Unusable, r.CheckedAt != nil, w)
}
}
list, err := gw.GatewayModels(ctx, "")
if err != nil || len(list.Data) != 3 {
t.Errorf("网关列表应只剔除被标记的坏模型(余 v2/v3/v4), got %+v, %v", list.Data, err)
}
}
func TestValidateRecheckUnmarksRecovered(t *testing.T) {
// 已标记模型超过复检间隔后重验:恢复供给(200)自动解除标记
client := &gatewayStubClient{fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}, probeCode: 200}
gw, svc := newTestGateway(t, client)
cfg := importAliveConfig(t, svc)
ch := seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1)
ctx := context.Background()
old := time.Now().Add(-25 * time.Hour)
gw.db.Model(&model.AiModelCache{}).Where("channel_id = ?", ch.ID).
Updates(map[string]any{"unusable": true, "unusable_reason": "x", "checked_at": old})
gw.validateChannelModels(ctx, ch.ID)
var row model.AiModelCache
gw.db.Where("channel_id = ?", ch.ID).First(&row)
if row.Unusable || row.UnusableReason != "" || row.CheckedAt == nil || !row.CheckedAt.After(old) {
t.Errorf("超期复检应解除标记并刷新已检时间: %+v", row)
}
// 未超期的标记不复检(probeSeq 为空、fallback 200 也不会被消费)
fresh := time.Now()
gw.db.Model(&model.AiModelCache{}).Where("channel_id = ?", ch.ID).
Updates(map[string]any{"unusable": true, "checked_at": fresh})
gw.validateChannelModels(ctx, ch.ID)
gw.db.Where("channel_id = ?", ch.ID).First(&row)
if !row.Unusable {
t.Error("未超期的标记不应被复检翻转")
}
}
func TestProbeAuth404StillNoQuota(t *testing.T) { func TestProbeAuth404StillNoQuota(t *testing.T) {
// 鉴权类 404(NotAuthorizedOrNotFound)仍属租户级,直接定论 no_quota 且不标记模型 // 鉴权类 404(NotAuthorizedOrNotFound)仍属租户级,直接定论 no_quota
client := &gatewayStubClient{ client := &gatewayStubClient{
fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}, fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}},
models: []oci.GenAiModel{{Ocid: "m1", Name: "meta.llama-3.3-70b-instruct", Vendor: "meta"}}, models: []oci.GenAiModel{{Ocid: "m1", Name: "meta.llama-3.3-70b-instruct", Vendor: "meta"}},
@@ -617,31 +498,26 @@ func TestProbeAuth404StillNoQuota(t *testing.T) {
if probed.ProbeStatus != "no_quota" { if probed.ProbeStatus != "no_quota" {
t.Errorf("鉴权 404 status = %q, want no_quota", probed.ProbeStatus) t.Errorf("鉴权 404 status = %q, want no_quota", probed.ProbeStatus)
} }
var marked int64
gw.db.Model(&model.AiModelCache{}).Where("unusable = ?", true).Count(&marked)
if marked != 0 {
t.Errorf("鉴权 404 不应标记模型, marked=%d", marked)
}
} }
func TestAiChatFinetuneSwitchesChannelWithoutPenalty(t *testing.T) { func TestAiChatFinetuneSwitchesChannelWithoutPenalty(t *testing.T) {
// 微调基座 400 换渠道重试成功,且不计入熔断失败 // 微调基座 400 换渠道重试成功,且不计入熔断失败
client := &gatewayStubClient{ client := &gatewayStubClient{
fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}, fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}},
chatResp: &aiwire.ChatResponse{Model: "meta.llama-3.3-70b-instruct", Choices: []aiwire.Choice{{Message: aiwire.ChatMessage{Role: "assistant", Content: aiwire.NewTextContent("hi")}, FinishReason: "stop"}}}, passPayload: []byte(`{"id":"resp_1","usage":{"input_tokens":3,"output_tokens":1}}`),
chatErrs: []error{finetuneBaseErr()}, passErrs: []error{finetuneBaseErr()},
} }
gw, svc := newTestGateway(t, client) gw, svc := newTestGateway(t, client)
cfg := importAliveConfig(t, svc) cfg := importAliveConfig(t, svc)
seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1) seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1)
seedChannel(t, gw, cfg.ID, "us-chicago-1", 1, 1) seedChannel(t, gw, cfg.ID, "us-chicago-1", 1, 1)
resp, meta, err := gw.Chat(context.Background(), aiwire.ChatRequest{Model: "meta.llama-3.3-70b-instruct", Messages: []aiwire.ChatMessage{{Role: "user", Content: aiwire.NewTextContent("你好")}}}, "") resp, meta, err := gw.RespPassthrough(context.Background(), []byte(`{}`), "meta.llama-3.3-70b-instruct", "")
if err != nil || resp == nil { if err != nil || resp == nil {
t.Fatalf("Chat = %v, %v", resp, err) t.Fatalf("RespPassthrough = %v, %v", resp, err)
} }
if meta.Retries != 1 || client.chatCalls != 2 { if meta.Retries != 1 || client.passCalls != 2 {
t.Errorf("应换渠道重试一次: retries=%d calls=%d", meta.Retries, client.chatCalls) t.Errorf("应换渠道重试一次: retries=%d calls=%d", meta.Retries, client.passCalls)
} }
var chs []model.AiChannel var chs []model.AiChannel
gw.db.Find(&chs) gw.db.Find(&chs)
@@ -650,38 +526,93 @@ func TestAiChatFinetuneSwitchesChannelWithoutPenalty(t *testing.T) {
t.Errorf("微调基座 400 不应计入熔断: 渠道 %s failCount=%d", ch.Name, ch.FailCount) t.Errorf("微调基座 400 不应计入熔断: 渠道 %s failCount=%d", ch.Name, ch.FailCount)
} }
} }
// 失败渠道的该模型被标记,不再参与路由;成功渠道不受影响
region := client.regions[0]
var row model.AiModelCache
gw.db.Where("model_ocid = ?", "ocid1..m-"+region).First(&row)
if !row.Unusable {
t.Errorf("失败渠道的模型应被标记不可用: %+v", row)
}
var usable int64
gw.db.Model(&model.AiModelCache{}).Where("unusable = ?", false).Count(&usable)
if usable != 1 {
t.Errorf("成功渠道模型应保持可用, usable=%d", usable)
}
} }
func TestUnusableModelExcludedFromPoolAndRouting(t *testing.T) { func TestBlacklistLifecycle(t *testing.T) {
// 唯一渠道的模型被标记后:网关列表不再展示,路由按未知模型拒绝,渠道详情仍可见标记 // 拉黑:全渠道同名缓存删除、列表与路由立即不可见;重复/空名被拒;
gw, svc := newTestGateway(t, &gatewayStubClient{fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}}) // 移出黑名单后重新同步恢复入池
client := &gatewayStubClient{
fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}},
models: []oci.GenAiModel{{Ocid: "ocid1..m-eu-frankfurt-1", Name: "meta.llama-3.3-70b-instruct", Vendor: "meta"}},
}
gw, svc := newTestGateway(t, client)
cfg := importAliveConfig(t, svc) cfg := importAliveConfig(t, svc)
ch := seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1) ch := seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1)
seedChannel(t, gw, cfg.ID, "us-chicago-1", 1, 1)
ctx := context.Background() ctx := context.Background()
gw.markModelUnusable(ctx, ch.ID, "ocid1..m-eu-frankfurt-1", "Entity with key … not found") item, err := gw.AddBlacklist(ctx, " meta.llama-3.3-70b-instruct ")
list, err := gw.GatewayModels(ctx, "") if err != nil || item.Name != "meta.llama-3.3-70b-instruct" {
if err != nil || len(list.Data) != 0 { t.Fatalf("AddBlacklist = %+v, %v", item, err)
t.Errorf("已标记模型不应出现在网关列表: %+v, %v", list.Data, err)
} }
if _, _, err := gw.Chat(ctx, aiwire.ChatRequest{Model: "meta.llama-3.3-70b-instruct"}, ""); !errors.Is(err, ErrAiUnknownModel) { var left int64
t.Errorf("已标记模型路由应拒绝: %v", err) gw.db.Model(&model.AiModelCache{}).Count(&left)
if left != 0 {
t.Errorf("拉黑应删除全部渠道的同名缓存, 剩 %d", left)
} }
rows, err := gw.channelModels(ctx, ch.ID) if _, _, err := gw.RespPassthrough(ctx, []byte(`{}`), "meta.llama-3.3-70b-instruct", ""); !errors.Is(err, ErrAiUnknownModel) {
if err != nil || len(rows) != 1 || !rows[0].Unusable || rows[0].UnusableReason == "" { t.Errorf("拉黑后路由应按未知模型拒绝: %v", err)
t.Errorf("渠道详情应保留标记行便于排查: %+v, %v", rows, err) }
if _, err := gw.AddBlacklist(ctx, "meta.llama-3.3-70b-instruct"); err == nil {
t.Error("重复拉黑应被拒绝")
}
if _, err := gw.AddBlacklist(ctx, " "); err == nil {
t.Error("空模型名应被拒绝")
}
rows, err := gw.Blacklist(ctx)
if err != nil || len(rows) != 1 {
t.Fatalf("Blacklist = %+v, %v", rows, err)
}
if err := gw.RemoveBlacklist(ctx, rows[0].ID); err != nil {
t.Fatalf("RemoveBlacklist: %v", err)
}
if err := gw.RemoveBlacklist(ctx, rows[0].ID); err == nil {
t.Error("移除不存在的条目应报错")
}
// 移出后重新同步:模型恢复入池
if _, err := gw.SyncModels(ctx, ch.ID); err != nil {
t.Fatalf("SyncModels: %v", err)
}
list, _ := gw.GatewayModels(ctx, "")
if len(list.Data) != 1 {
t.Errorf("移出黑名单并同步后应恢复入池: %+v", list.Data)
}
}
func TestSyncAndProbeFilterBlacklisted(t *testing.T) {
// 同步与探测都过滤黑名单模型:不入库、不进试调候选
client := &gatewayStubClient{
fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}},
models: []oci.GenAiModel{
{Ocid: "m1", Name: "meta.llama-3-70b-instruct", Vendor: "meta"},
{Ocid: "m2", Name: "cohere.command-a-03-2025", Vendor: "cohere"},
},
probeCode: 200,
}
gw, svc := newTestGateway(t, client)
cfg := importAliveConfig(t, svc)
ctx := context.Background()
if _, err := gw.AddBlacklist(ctx, "meta.llama-3-70b-instruct"); err != nil {
t.Fatalf("AddBlacklist: %v", err)
}
ch, err := gw.CreateChannel(ctx, ChannelInput{OciConfigID: cfg.ID, Region: "eu-frankfurt-1"})
if err != nil {
t.Fatalf("CreateChannel: %v", err)
}
probed, err := gw.ProbeChannel(ctx, ch.ID)
if err != nil || probed.ProbeStatus != "ok" {
t.Fatalf("ProbeChannel = %+v, %v", probed, err)
}
rows, _ := gw.channelModels(ctx, ch.ID)
if len(rows) != 1 || rows[0].Name != "cohere.command-a-03-2025" {
t.Errorf("探测同步应过滤黑名单模型: %+v", rows)
}
if len(client.probedNames) != 1 || client.probedNames[0] != "cohere.command-a-03-2025" {
t.Errorf("试调候选不应包含黑名单模型: %v", client.probedNames)
}
models, err := gw.SyncModels(ctx, ch.ID)
if err != nil || len(models) != 1 || models[0].Name != "cohere.command-a-03-2025" {
t.Errorf("SyncModels 应过滤黑名单模型: %+v, %v", models, err)
} }
} }
@@ -745,7 +676,7 @@ func TestAiEmbeddings(t *testing.T) {
t.Errorf("chat 模型走 embeddings err = %v, want ErrAiUnknownModel", err) t.Errorf("chat 模型走 embeddings err = %v, want ErrAiUnknownModel", err)
} }
// embedding 模型名打 chat:同样未知模型 // embedding 模型名打 chat:同样未知模型
if _, _, err := gw.Chat(ctx, aiwire.ChatRequest{Model: "cohere.embed-v4.0", Messages: []aiwire.ChatMessage{{Role: "user", Content: aiwire.NewTextContent("x")}}}, ""); !errors.Is(err, ErrAiUnknownModel) { if _, _, err := gw.RespPassthrough(ctx, []byte(`{}`), "cohere.embed-v4.0", ""); !errors.Is(err, ErrAiUnknownModel) {
t.Errorf("embedding 模型走 chat err = %v, want ErrAiUnknownModel", err) t.Errorf("embedding 模型走 chat err = %v, want ErrAiUnknownModel", err)
} }
} }
@@ -756,7 +687,7 @@ func TestAiContentLogSwitch(t *testing.T) {
t.Fatalf("migrate content log: %v", err) t.Fatalf("migrate content log: %v", err)
} }
ctx := context.Background() ctx := context.Background()
_, key, err := gw.CreateKey(ctx, "k1", "content-key-1234", "") _, key, err := gw.CreateKey(ctx, "k1", "content-key-1234", "", nil)
if err != nil { if err != nil {
t.Fatalf("CreateKey: %v", err) t.Fatalf("CreateKey: %v", err)
} }
@@ -793,23 +724,130 @@ func TestAiContentLogSwitch(t *testing.T) {
} }
} }
func TestLegacyNullUnusableRowsStayVisible(t *testing.T) { func TestRespPassthroughSwitchesChannel(t *testing.T) {
// 升级路径回归:AutoMigrate 加列后、首次重同步前,存量行 unusable 为 NULL, client := &gatewayStubClient{
// 网关列表与路由必须照常包含这些行,不能因 unusable = false 过滤而整池消失 fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}},
gw, svc := newTestGateway(t, &gatewayStubClient{fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}}) passPayload: []byte(`{"id":"resp_1","usage":{"input_tokens":9,"output_tokens":3}}`),
passErrs: []error{stubServiceError{status: 429}},
}
gw, svc := newTestGateway(t, client)
cfg := importAliveConfig(t, svc) cfg := importAliveConfig(t, svc)
ch := seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1) seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1)
ctx := context.Background() seedChannel(t, gw, cfg.ID, "us-chicago-1", 1, 1)
if err := gw.db.Exec("UPDATE ai_model_caches SET unusable = NULL WHERE channel_id = ?", ch.ID).Error; err != nil {
t.Fatalf("set legacy null: %v", err) payload, meta, err := gw.RespPassthrough(context.Background(), []byte(`{}`), "meta.llama-3.3-70b-instruct", "")
if err != nil || len(payload) == 0 {
t.Fatalf("RespPassthrough = %v, %v", payload, err)
}
if meta.Retries != 1 || client.passCalls != 2 {
t.Errorf("应换渠道重试一次: retries=%d calls=%d", meta.Retries, client.passCalls)
}
if len(client.passRegions) == 2 && client.passRegions[0] == client.passRegions[1] {
t.Errorf("重试未换渠道: %v", client.passRegions)
}
} }
list, err := gw.GatewayModels(ctx, "") func TestRespPassthroughNonRetryable(t *testing.T) {
if err != nil || len(list.Data) != 1 { client := &gatewayStubClient{
t.Errorf("存量 NULL 行应仍在网关列表: %+v, %v", list.Data, err) fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}},
passErrs: []error{stubServiceError{status: 400}},
} }
_, ids, err := gw.modelChannels(ctx, "meta.llama-3.3-70b-instruct", "CHAT") gw, svc := newTestGateway(t, client)
if err != nil || len(ids) != 1 { cfg := importAliveConfig(t, svc)
t.Errorf("存量 NULL 行应仍参与路由: %v, %v", ids, err) seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1)
seedChannel(t, gw, cfg.ID, "us-chicago-1", 1, 1)
_, meta, err := gw.RespPassthrough(context.Background(), []byte(`{}`), "meta.llama-3.3-70b-instruct", "")
if err == nil || meta.Retries != 0 || client.passCalls != 1 {
t.Errorf("400 应直接透传不重试: err=%v retries=%d calls=%d", err, meta.Retries, client.passCalls)
}
}
func TestNormalizeKeyModels(t *testing.T) {
tests := []struct {
name string
in []string
want []string
}{
{"nil 输入", nil, nil},
{"全空串", []string{"", " "}, nil},
{"trim 与保序去重", []string{" a ", "b", "a", "", "b "}, []string{"a", "b"}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := normalizeKeyModels(tt.in)
if len(got) != len(tt.want) {
t.Fatalf("normalizeKeyModels(%v) = %v, want %v", tt.in, got, tt.want)
}
for i := range tt.want {
if got[i] != tt.want[i] {
t.Fatalf("normalizeKeyModels(%v) = %v, want %v", tt.in, got, tt.want)
}
}
})
}
}
// TestAiKeyModelsPersistence 钉住 serializer:json 字段经 Create 与 Updates(map) 两条路径的往返。
func TestAiKeyModelsPersistence(t *testing.T) {
gw, _ := newTestGateway(t, &gatewayStubClient{fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}})
ctx := context.Background()
_, key, err := gw.CreateKey(ctx, "m-key", "model-key-1234", "", []string{" x-model ", "x-model", "y-model", ""})
if err != nil {
t.Fatalf("CreateKey: %v", err)
}
fresh, err := gw.VerifyKey(ctx, "model-key-1234")
if err != nil || len(fresh.Models) != 2 || fresh.Models[0] != "x-model" || fresh.Models[1] != "y-model" {
t.Fatalf("创建后读回 models = %v, %v", fresh.Models, err)
}
set := []string{"z-model"}
if err := gw.UpdateKey(ctx, key.ID, "", nil, nil, &set); err != nil {
t.Fatalf("UpdateKey 覆盖: %v", err)
}
fresh, _ = gw.VerifyKey(ctx, "model-key-1234")
if len(fresh.Models) != 1 || fresh.Models[0] != "z-model" {
t.Fatalf("覆盖后 models = %v", fresh.Models)
}
if err := gw.UpdateKey(ctx, key.ID, "renamed", nil, nil, nil); err != nil {
t.Fatalf("UpdateKey 不传 models: %v", err)
}
fresh, _ = gw.VerifyKey(ctx, "model-key-1234")
if len(fresh.Models) != 1 {
t.Fatalf("未传 models 却被改动: %v", fresh.Models)
}
empty := []string{}
if err := gw.UpdateKey(ctx, key.ID, "", nil, nil, &empty); err != nil {
t.Fatalf("UpdateKey 清空: %v", err)
}
fresh, _ = gw.VerifyKey(ctx, "model-key-1234")
if len(fresh.Models) != 0 {
t.Fatalf("清空后 models = %v, want 空", fresh.Models)
}
}
// TestRespPassthroughStreamSwitchesChannel 断言流式直通建立失败按 switchable 换渠道。
func TestRespPassthroughStreamSwitchesChannel(t *testing.T) {
client := &gatewayStubClient{
fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}},
passPayload: []byte("data: {\"type\":\"response.completed\"}\n\n"),
passErrs: []error{stubServiceError{status: 503}},
}
gw, svc := newTestGateway(t, client)
cfg := importAliveConfig(t, svc)
seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1)
seedChannel(t, gw, cfg.ID, "us-chicago-1", 1, 1)
stream, meta, err := gw.RespPassthroughStream(context.Background(), []byte(`{}`), "meta.llama-3.3-70b-instruct", "")
if err != nil || stream == nil {
t.Fatalf("RespPassthroughStream = %v", err)
}
defer stream.Close()
if meta.Retries != 1 || client.passCalls != 2 {
t.Errorf("应换渠道重试一次: retries=%d calls=%d", meta.Retries, client.passCalls)
}
payload, _ := io.ReadAll(stream)
if !bytes.Contains(payload, []byte("response.completed")) {
t.Errorf("流内容未透传: %s", payload)
} }
} }
+60 -205
View File
@@ -8,38 +8,8 @@ import (
"oci-portal/internal/aiwire" "oci-portal/internal/aiwire"
) )
// ResponsesToIR 把 Responses API 请求转换为网关 IR(OpenAI Chat 形态)。 // respRejectStateful 拒绝有状态特性(网关无状态)。
// 有状态特性与内置工具不支持,直接报错(API 层 400)。 func respRejectStateful(req aiwire.RespRequest) error {
func ResponsesToIR(req aiwire.RespRequest) (aiwire.ChatRequest, error) {
if err := respRejectUnsupported(req); err != nil {
return aiwire.ChatRequest{}, err
}
ir := aiwire.ChatRequest{
Model: req.Model,
MaxTokens: req.MaxOutputTokens,
Temperature: req.Temperature,
TopP: req.TopP,
Stream: req.Stream,
ToolChoice: respToolChoice(req.ToolChoice),
}
if req.Reasoning != nil {
ir.ReasoningEffort = req.Reasoning.Effort
}
if req.Instructions != "" {
ir.Messages = append(ir.Messages, aiwire.ChatMessage{Role: "system", Content: aiwire.NewTextContent(req.Instructions)})
}
msgs, err := respInputToMessages(req.Input)
if err != nil {
return aiwire.ChatRequest{}, err
}
ir.Messages = append(ir.Messages, msgs...)
ir.Tools = respTools(req.Tools)
ir.ResponseFormat = respFormat(req.Text)
return ir, nil
}
// respRejectUnsupported 拒绝无状态网关无法承接的请求特性。
func respRejectUnsupported(req aiwire.RespRequest) error {
if req.PreviousResponseID != "" { if req.PreviousResponseID != "" {
return fmt.Errorf("previous_response_id 不支持:网关不保存历史响应,请在 input 中自带完整上下文(store:false 模式)") return fmt.Errorf("previous_response_id 不支持:网关不保存历史响应,请在 input 中自带完整上下文(store:false 模式)")
} }
@@ -49,195 +19,80 @@ func respRejectUnsupported(req aiwire.RespRequest) error {
if req.Background != nil && *req.Background { if req.Background != nil && *req.Background {
return fmt.Errorf("background 模式不支持") return fmt.Errorf("background 模式不支持")
} }
for _, t := range req.Tools {
if t.Type != "function" {
return fmt.Errorf("不支持的工具类型 %q:仅支持 function 工具", t.Type)
}
}
return nil return nil
} }
// respInputToMessages 把 input(string 或 item 数组)展开为 IR 消息序列 // RespServerTools 报告工具列表是否含 xAI 服务端工具(web_search / x_search)
func respInputToMessages(in aiwire.RespInput) ([]aiwire.ChatMessage, error) { func RespServerTools(tools []aiwire.RespTool) bool {
if !in.IsArray {
if strings.TrimSpace(in.Text) == "" {
return nil, fmt.Errorf("input 不能为空")
}
return []aiwire.ChatMessage{{Role: "user", Content: aiwire.NewTextContent(in.Text)}}, nil
}
var msgs []aiwire.ChatMessage
for _, it := range in.Items {
out, err := respItemToMessages(it, msgs)
if err != nil {
return nil, err
}
msgs = out
}
return msgs, nil
}
// respItemToMessages 追加一个 input item;连续 function_call 合并进同一 assistant 消息。
func respItemToMessages(it aiwire.RespItem, msgs []aiwire.ChatMessage) ([]aiwire.ChatMessage, error) {
switch it.Type {
case "", "message":
if it.Content.HasUnsupported() {
return nil, ErrAiUnsupportedBlock
}
role := respRole(it.Role)
return append(msgs, aiwire.ChatMessage{Role: role, Content: respContentToIR(it.Content)}), nil
case "function_call":
call := aiwire.ToolCall{ID: it.CallID, Type: "function",
Function: aiwire.FunctionCall{Name: it.Name, Arguments: it.Arguments}}
if n := len(msgs); n > 0 && msgs[n-1].Role == "assistant" && len(msgs[n-1].ToolCalls) > 0 {
msgs[n-1].ToolCalls = append(msgs[n-1].ToolCalls, call)
return msgs, nil
}
return append(msgs, aiwire.ChatMessage{Role: "assistant", ToolCalls: []aiwire.ToolCall{call}}), nil
case "function_call_output":
return append(msgs, aiwire.ChatMessage{Role: "tool", ToolCallID: it.CallID,
Content: aiwire.NewTextContent(it.OutputText())}), nil
case "reasoning":
return msgs, nil // 推理块不回灌上游
default:
return nil, fmt.Errorf("不支持的 input item 类型 %q", it.Type)
}
}
// respRole 归一 item 角色;developer 视为 system,缺省 user。
func respRole(role string) string {
switch role {
case "system", "developer":
return "system"
case "assistant":
return "assistant"
default:
return "user"
}
}
// respContentToIR 把 Responses 内容转 IR;纯文本拍平,含图片时保留部件顺序。
func respContentToIR(c aiwire.RespContent) aiwire.Content {
var parts []aiwire.ContentPart
hasImage := false
for _, p := range c.Parts {
switch p.Type {
case "input_image":
parts = append(parts, aiwire.ContentPart{Type: "image_url",
ImageURL: &aiwire.ImageURL{URL: p.ImageURL, Detail: p.Detail}})
hasImage = true
default:
if p.Text != "" {
parts = append(parts, aiwire.ContentPart{Type: "text", Text: p.Text})
}
}
}
if !c.IsArray || !hasImage {
return aiwire.NewTextContent(c.JoinText())
}
return aiwire.NewPartsContent(parts)
}
// respTools 把扁平工具定义还原为嵌套 Chat 形态;strict 无 OCI 对应,忽略。
func respTools(tools []aiwire.RespTool) []aiwire.Tool {
if len(tools) == 0 {
return nil
}
out := make([]aiwire.Tool, 0, len(tools))
for _, t := range tools { for _, t := range tools {
out = append(out, aiwire.Tool{Type: "function", if t.Type == "web_search" || t.Type == "x_search" {
Function: aiwire.FunctionDef{Name: t.Name, Description: t.Description, Parameters: t.Parameters}}) return true
} }
return out }
return false
} }
// respToolChoice 转换 tool_choice:字符串直通;{type:function,name} 转嵌套; // RespPassthroughValidate 校验直通请求:模型必填,有状态特性不支持,工具类型
// allowed_tools 等对象形态降级 auto。 // 只放行 function 与已实测的 web_search / x_search;流式仅在含服务端工具时拒绝
func respToolChoice(raw json.RawMessage) json.RawMessage { // (工具流式事件形态未实测,不放开)。
if len(raw) == 0 { func RespPassthroughValidate(req aiwire.RespRequest) error {
return nil if strings.TrimSpace(req.Model) == "" {
return fmt.Errorf("model 不能为空")
} }
t := strings.TrimSpace(string(raw)) if req.Stream && RespServerTools(req.Tools) {
if strings.HasPrefix(t, "\"") { return fmt.Errorf("服务端工具暂不支持流式:请去掉 stream 或改用 function 工具")
return raw
} }
var obj struct { if err := respRejectStateful(req); err != nil {
Type string `json:"type"` return err
Name string `json:"name"`
} }
if err := json.Unmarshal(raw, &obj); err != nil || obj.Type != "function" || obj.Name == "" { for _, t := range req.Tools {
return json.RawMessage(`"auto"`) switch t.Type {
} case "function", "web_search", "x_search":
out, _ := json.Marshal(map[string]any{"type": "function", "function": map[string]string{"name": obj.Name}})
return out
}
// respFormat 把 text.format 转成 response_format;text 形态无需显式指定。
func respFormat(text *aiwire.RespText) *aiwire.ResponseFormat {
if text == nil || text.Format == nil {
return nil
}
switch text.Format.Type {
case "json_object":
return &aiwire.ResponseFormat{Type: "json_object"}
case "json_schema":
spec, _ := json.Marshal(map[string]any{
"name": text.Format.Name, "schema": json.RawMessage(text.Format.Schema), "strict": text.Format.Strict,
})
return &aiwire.ResponseFormat{Type: "json_schema", JSONSchema: spec}
default: default:
return fmt.Errorf("不支持的工具类型 %q:服务端工具仅支持 web_search / x_search", t.Type)
}
}
return nil return nil
} }
// RespPassthroughBody 以原始请求体为基构造上游 body:强制 store:false(禁上游
// 存态),stream 原样保留(流式直通);用 json.Number 保真未知字段与数值。
func RespPassthroughBody(raw []byte) ([]byte, error) {
dec := json.NewDecoder(strings.NewReader(string(raw)))
dec.UseNumber()
var body map[string]any
if err := dec.Decode(&body); err != nil {
return nil, fmt.Errorf("解析请求体: %w", err)
}
body["store"] = false
return json.Marshal(body)
} }
// IRRespToResponses 把 IR 非流式响应装配为 Response 对象 // RespPassthroughUsage 从直通响应提取用量;缺失时返回 nil(日志记零)
func IRRespToResponses(resp *aiwire.ChatResponse, id string, created int64) aiwire.Response { func RespPassthroughUsage(payload []byte) *aiwire.Usage {
out := aiwire.Response{ID: id, Object: "response", CreatedAt: created, var root struct {
Status: "completed", Model: resp.Model, Output: []aiwire.RespOutItem{}, Store: false} Usage *aiwire.RespUsage `json:"usage"`
if len(resp.Choices) == 0 {
return out
} }
choice := resp.Choices[0] if json.Unmarshal(payload, &root) != nil || root.Usage == nil {
if text := choice.Message.Content.JoinText(); text != "" {
out.Output = append(out.Output, respMessageItem(id, 0, text))
}
for i, tc := range choice.Message.ToolCalls {
out.Output = append(out.Output, aiwire.RespOutItem{Type: "function_call",
ID: respItemID(id, "fc", len(out.Output)+i), Status: "completed",
CallID: tc.ID, Name: tc.Function.Name, Arguments: respArgs(tc.Function.Arguments)})
}
if choice.FinishReason == "length" {
out.Status = "incomplete"
out.IncompleteDetails = &aiwire.RespIncomplete{Reason: "max_output_tokens"}
}
out.Usage = respUsage(resp.Usage)
return out
}
func respMessageItem(respID string, idx int, text string) aiwire.RespOutItem {
return aiwire.RespOutItem{Type: "message", ID: respItemID(respID, "msg", idx),
Status: "completed", Role: "assistant",
Content: []aiwire.RespOutPart{{Type: "output_text", Text: text, Annotations: []any{}}}}
}
// respItemID 从响应 ID 派生确定性 item ID。
func respItemID(respID, kind string, idx int) string {
return fmt.Sprintf("%s_%s_%d", kind, strings.TrimPrefix(respID, "resp_"), idx)
}
// respArgs 保证 arguments 是合法 JSON 字符串(空实参回退 {})。
func respArgs(args string) string {
if strings.TrimSpace(args) == "" {
return "{}"
}
return args
}
// respUsage 把 IR 用量改写为 Responses 命名口径。
func respUsage(u *aiwire.Usage) *aiwire.RespUsage {
if u == nil {
return nil return nil
} }
return &aiwire.RespUsage{InputTokens: u.PromptTokens, OutputTokens: u.CompletionTokens, usage := &aiwire.Usage{PromptTokens: root.Usage.InputTokens,
TotalTokens: u.TotalTokens, CompletionTokens: root.Usage.OutputTokens, TotalTokens: root.Usage.TotalTokens}
InputTokensDetails: aiwire.RespInDetails{CachedTokens: u.CachedTokens()}} if cached := root.Usage.InputTokensDetails.CachedTokens; cached > 0 {
usage.PromptTokensDetails = &aiwire.PromptTokensDetails{CachedTokens: cached}
}
return usage
}
// RespStreamCompletedUsage 从一行 SSE data JSON 中提取 response.completed 事件的
// usage;非 completed 事件或解析失败返回 nil。流式直通逐行喂入,最后一次非 nil 生效。
func RespStreamCompletedUsage(data []byte) *aiwire.Usage {
var ev struct {
Type string `json:"type"`
Response json.RawMessage `json:"response"`
}
if json.Unmarshal(data, &ev) != nil || ev.Type != "response.completed" || len(ev.Response) == 0 {
return nil
}
return RespPassthroughUsage(ev.Response)
} }
-199
View File
@@ -1,199 +0,0 @@
package service
import (
"strings"
"oci-portal/internal/aiwire"
)
// RespEvent 是一条 Responses SSE 语义事件(event 名 + data 载荷)。
type RespEvent struct {
Event string
Data map[string]any
}
// RespStream 把 IR chunk 流聚合为 Responses 语义事件序列。
// 与 AnthStream 同构,但终态事件须携带完整 Response 快照,故全程缓冲文本与实参。
type RespStream struct {
id string
model string
created int64
seq int
started bool
items []aiwire.RespOutItem
msgOpen bool
text strings.Builder
tool aiwire.RespOutItem
toolIdx int
tOpen bool
args strings.Builder
finish string
usage *aiwire.Usage
}
// NewRespStream 构造状态机;id 形如 resp_*,created 为响应时间戳。
func NewRespStream(id, model string, created int64) *RespStream {
return &RespStream{id: id, model: model, created: created}
}
// Feed 消化一个上游 chunk,返回应立即下发的事件。
func (s *RespStream) Feed(chunk aiwire.ChatChunk) []RespEvent {
var evs []RespEvent
if !s.started {
s.started = true
evs = append(evs, s.respEvent("response.created", "in_progress"),
s.respEvent("response.in_progress", "in_progress"))
}
if chunk.Usage != nil {
s.usage = chunk.Usage
}
if len(chunk.Choices) == 0 {
return evs
}
choice := chunk.Choices[0]
if choice.Delta.Content != "" {
evs = append(evs, s.feedText(choice.Delta.Content)...)
}
for _, tc := range choice.Delta.ToolCalls {
evs = append(evs, s.feedTool(tc)...)
}
if choice.FinishReason != nil {
s.finish = *choice.FinishReason
}
return evs
}
// Finish 关闭未闭合的块并产出终态事件(带完整 Response 快照)。
func (s *RespStream) Finish() []RespEvent {
var evs []RespEvent
if !s.started {
s.started = true
evs = append(evs, s.respEvent("response.created", "in_progress"))
}
evs = append(evs, s.closeMsg()...)
evs = append(evs, s.closeTool()...)
status := "completed"
if s.finish == "length" {
status = "incomplete"
}
return append(evs, s.respEvent("response."+status, status))
}
// Usage 返回聚合到的用量(可能为 nil),供调用日志。
func (s *RespStream) Usage() *aiwire.Usage { return s.usage }
// ev 构造带自增 sequence_number 的事件。
func (s *RespStream) ev(typ string, kv map[string]any) RespEvent {
s.seq++
data := map[string]any{"type": typ, "sequence_number": s.seq}
for k, v := range kv {
data[k] = v
}
return RespEvent{Event: typ, Data: data}
}
// respEvent 构造携带 Response 快照的生命周期事件。
func (s *RespStream) respEvent(typ, status string) RespEvent {
return s.ev(typ, map[string]any{"response": s.snapshot(status)})
}
// snapshot 组装当前累计状态的 Response 对象。
func (s *RespStream) snapshot(status string) aiwire.Response {
resp := aiwire.Response{ID: s.id, Object: "response", CreatedAt: s.created,
Status: status, Model: s.model, Store: false,
Output: append([]aiwire.RespOutItem{}, s.items...)}
resp.Usage = respUsage(s.usage)
if status == "incomplete" {
resp.IncompleteDetails = &aiwire.RespIncomplete{Reason: "max_output_tokens"}
}
return resp
}
// feedText 处理文本增量:必要时开 message item 与 content part。
func (s *RespStream) feedText(delta string) []RespEvent {
evs := s.closeTool()
if !s.msgOpen {
s.msgOpen = true
s.text.Reset()
item := aiwire.RespOutItem{Type: "message", ID: s.itemID("msg"), Status: "in_progress",
Role: "assistant", Content: []aiwire.RespOutPart{}}
evs = append(evs, s.ev("response.output_item.added", map[string]any{
"output_index": len(s.items), "item": item}))
evs = append(evs, s.ev("response.content_part.added", map[string]any{
"item_id": item.ID, "output_index": len(s.items), "content_index": 0,
"part": aiwire.RespOutPart{Type: "output_text", Text: "", Annotations: []any{}}}))
}
s.text.WriteString(delta)
return append(evs, s.ev("response.output_text.delta", map[string]any{
"item_id": s.itemID("msg"), "output_index": len(s.items), "content_index": 0, "delta": delta}))
}
// closeMsg 闭合当前 message item(text done → part done → item done)。
func (s *RespStream) closeMsg() []RespEvent {
if !s.msgOpen {
return nil
}
s.msgOpen = false
id, idx, full := s.itemID("msg"), len(s.items), s.text.String()
part := aiwire.RespOutPart{Type: "output_text", Text: full, Annotations: []any{}}
item := aiwire.RespOutItem{Type: "message", ID: id, Status: "completed",
Role: "assistant", Content: []aiwire.RespOutPart{part}}
evs := []RespEvent{
s.ev("response.output_text.done", map[string]any{"item_id": id, "output_index": idx, "content_index": 0, "text": full}),
s.ev("response.content_part.done", map[string]any{"item_id": id, "output_index": idx, "content_index": 0, "part": part}),
s.ev("response.output_item.done", map[string]any{"output_index": idx, "item": item}),
}
s.items = append(s.items, item)
return evs
}
// feedTool 处理工具调用增量:新 Index 先闭合旧调用再开新 item。
func (s *RespStream) feedTool(tc aiwire.ToolCallDelta) []RespEvent {
evs := s.closeMsg()
if s.tOpen && tc.Index != s.toolIdx {
evs = append(evs, s.closeTool()...)
}
if !s.tOpen {
s.tOpen, s.toolIdx = true, tc.Index
s.args.Reset()
s.tool = aiwire.RespOutItem{Type: "function_call", ID: s.itemID("fc"), Status: "in_progress",
CallID: tc.ID, Name: tc.Function.Name}
evs = append(evs, s.ev("response.output_item.added", map[string]any{
"output_index": len(s.items), "item": s.tool}))
}
if tc.ID != "" && s.tool.CallID == "" {
s.tool.CallID = tc.ID
}
if tc.Function.Name != "" && s.tool.Name == "" {
s.tool.Name = tc.Function.Name
}
if tc.Function.Arguments == "" {
return evs
}
s.args.WriteString(tc.Function.Arguments)
return append(evs, s.ev("response.function_call_arguments.delta", map[string]any{
"item_id": s.tool.ID, "output_index": len(s.items), "delta": tc.Function.Arguments}))
}
// closeTool 闭合当前 function_call item(arguments done → item done)。
func (s *RespStream) closeTool() []RespEvent {
if !s.tOpen {
return nil
}
s.tOpen = false
s.tool.Arguments = respArgs(s.args.String())
s.tool.Status = "completed"
idx := len(s.items)
evs := []RespEvent{
s.ev("response.function_call_arguments.done", map[string]any{
"item_id": s.tool.ID, "output_index": idx, "arguments": s.tool.Arguments}),
s.ev("response.output_item.done", map[string]any{"output_index": idx, "item": s.tool}),
}
s.items = append(s.items, s.tool)
return evs
}
// itemID 按当前 output 序号派生确定性 item ID。
func (s *RespStream) itemID(kind string) string {
return respItemID(s.id, kind, len(s.items))
}
+102 -131
View File
@@ -17,47 +17,6 @@ func respReq(t *testing.T, raw string) aiwire.RespRequest {
return req return req
} }
func TestResponsesToIR(t *testing.T) {
req := respReq(t, `{
"model": "meta.llama-3.3-70b-instruct",
"instructions": "你是助手",
"max_output_tokens": 128,
"input": [
{"role": "user", "content": "查天气"},
{"type": "function_call", "call_id": "call_1", "name": "get_weather", "arguments": "{\"city\":\"上海\"}"},
{"type": "function_call_output", "call_id": "call_1", "output": "{\"temp\":31}"},
{"type": "message", "role": "assistant", "content": [{"type": "output_text", "text": "31 度"}]}
],
"tools": [{"type": "function", "name": "get_weather", "description": "查天气", "parameters": {"type": "object"}, "strict": true}],
"tool_choice": {"type": "function", "name": "get_weather"},
"text": {"format": {"type": "json_schema", "name": "out", "schema": {"type": "object"}}}
}`)
ir, err := ResponsesToIR(req)
if err != nil {
t.Fatalf("ResponsesToIR: %v", err)
}
roles := make([]string, 0, len(ir.Messages))
for _, m := range ir.Messages {
roles = append(roles, m.Role)
}
want := []string{"system", "user", "assistant", "tool", "assistant"}
if strings.Join(roles, ",") != strings.Join(want, ",") {
t.Errorf("roles = %v, want %v", roles, want)
}
if ir.Messages[2].ToolCalls[0].ID != "call_1" || ir.Messages[3].ToolCallID != "call_1" {
t.Errorf("tool call 链接错误: %+v", ir.Messages)
}
if deref(ir.MaxTokens) != 128 || len(ir.Tools) != 1 || ir.Tools[0].Function.Name != "get_weather" {
t.Errorf("参数映射错误: max=%v tools=%+v", ir.MaxTokens, ir.Tools)
}
if !strings.Contains(string(ir.ToolChoice), `"function"`) || !strings.Contains(string(ir.ToolChoice), "get_weather") {
t.Errorf("tool_choice = %s", ir.ToolChoice)
}
if ir.ResponseFormat == nil || ir.ResponseFormat.Type != "json_schema" {
t.Errorf("response_format = %+v", ir.ResponseFormat)
}
}
func deref(p *int) int { func deref(p *int) int {
if p == nil { if p == nil {
return 0 return 0
@@ -65,112 +24,124 @@ func deref(p *int) int {
return *p return *p
} }
func TestResponsesToIRRejects(t *testing.T) { func TestRespServerTools(t *testing.T) {
cases := []struct{ name, raw string }{ tests := []struct {
{"previous_response_id", `{"model":"m","input":"hi","previous_response_id":"resp_x"}`}, name string
{"conversation", `{"model":"m","input":"hi","conversation":{"id":"conv_1"}}`}, tools []aiwire.RespTool
{"background", `{"model":"m","input":"hi","background":true}`}, want bool
{"builtin tool", `{"model":"m","input":"hi","tools":[{"type":"web_search"}]}`}, }{
{"file_id image", `{"model":"m","input":[{"role":"user","content":[{"type":"input_image","file_id":"file_1"}]}]}`}, {"web_search", []aiwire.RespTool{{Type: "web_search"}}, true},
{"unknown item", `{"model":"m","input":[{"type":"item_reference","id":"x"}]}`}, {"x_search混用", []aiwire.RespTool{{Type: "function", Name: "f"}, {Type: "x_search"}}, true},
{"仅function", []aiwire.RespTool{{Type: "function", Name: "f"}}, false},
{"空", nil, false},
{"其他类型", []aiwire.RespTool{{Type: "code_interpreter"}}, false},
} }
for _, c := range cases { for _, test := range tests {
if _, err := ResponsesToIR(respReq(t, c.raw)); err == nil { t.Run(test.name, func(t *testing.T) {
t.Errorf("%s 应被拒绝", c.name) if got := RespServerTools(test.tools); got != test.want {
t.Fatalf("got %v, want %v", got, test.want)
}
})
} }
} }
// input 字符串形态 + store/reasoning 忽略项不报错
req := respReq(t, `{"model":"m","input":"你好","store":false,"reasoning":{"effort":"low"},"parallel_tool_calls":true}`) func TestRespPassthroughValidate(t *testing.T) {
ir, err := ResponsesToIR(req) prev := "resp_1"
if err != nil || len(ir.Messages) != 1 || ir.Messages[0].Role != "user" || ir.ReasoningEffort != "low" { bg := true
t.Errorf("字符串 input = %+v, %v", ir.Messages, err) tests := []struct {
name string
req aiwire.RespRequest
wantErr bool
}{
{"合法", aiwire.RespRequest{Model: "m", Tools: []aiwire.RespTool{{Type: "web_search"}}}, false},
{"混用function", aiwire.RespRequest{Model: "m", Tools: []aiwire.RespTool{{Type: "web_search"}, {Type: "function", Name: "f"}}}, false},
{"缺model", aiwire.RespRequest{Tools: []aiwire.RespTool{{Type: "web_search"}}}, true},
{"工具加流式拒绝", aiwire.RespRequest{Model: "m", Stream: true, Tools: []aiwire.RespTool{{Type: "web_search"}}}, true},
{"无工具流式放行", aiwire.RespRequest{Model: "m", Stream: true}, false},
{"function工具流式放行", aiwire.RespRequest{Model: "m", Stream: true, Tools: []aiwire.RespTool{{Type: "function", Name: "f"}}}, false},
{"有状态拒绝", aiwire.RespRequest{Model: "m", PreviousResponseID: prev}, true},
{"background拒绝", aiwire.RespRequest{Model: "m", Background: &bg}, true},
{"未知工具拒绝", aiwire.RespRequest{Model: "m", Tools: []aiwire.RespTool{{Type: "web_search"}, {Type: "mcp"}}}, true},
} }
// input_image(url 形态)放行并保留图文顺序 for _, test := range tests {
req2 := respReq(t, `{"model":"m","input":[{"role":"user","content":[{"type":"input_text","text":"看图"},{"type":"input_image","image_url":"https://x/1.png","detail":"low"}]}]}`) t.Run(test.name, func(t *testing.T) {
ir2, err := ResponsesToIR(req2) err := RespPassthroughValidate(test.req)
if (err != nil) != test.wantErr {
t.Fatalf("err = %v, wantErr %v", err, test.wantErr)
}
})
}
}
func TestRespPassthroughBody(t *testing.T) {
raw := []byte(`{"model":"m","input":"hi","stream":true,"store":true,"max_output_tokens":128,"custom_field":{"a":1.5}}`)
out, err := RespPassthroughBody(raw)
if err != nil { if err != nil {
t.Fatalf("input_image 应放行: %v", err) t.Fatalf("RespPassthroughBody: %v", err)
} }
parts := ir2.Messages[0].Content.Parts var body map[string]any
if len(parts) != 2 || parts[1].Type != "image_url" || parts[1].ImageURL == nil || parts[1].ImageURL.URL != "https://x/1.png" { if err := json.Unmarshal(out, &body); err != nil {
t.Errorf("parts = %+v", parts) t.Fatalf("unmarshal out: %v", err)
}
if body["store"] != false {
t.Errorf("store 应强制 false, got %v", body["store"])
}
if body["stream"] != true {
t.Errorf("stream 应原样保留, got %v", body["stream"])
}
if string(out) == "" || !strings.Contains(string(out), `"max_output_tokens":128`) {
t.Errorf("数值字段应保真: %s", out)
}
if !strings.Contains(string(out), `"custom_field"`) {
t.Errorf("未知字段应保留: %s", out)
}
if _, err := RespPassthroughBody([]byte("not-json")); err == nil {
t.Error("非法 JSON 应报错")
} }
} }
func TestIRRespToResponses(t *testing.T) { func TestRespPassthroughUsage(t *testing.T) {
resp := &aiwire.ChatResponse{Model: "m", Choices: []aiwire.Choice{{ tests := []struct {
Message: aiwire.ChatMessage{Role: "assistant", Content: aiwire.NewTextContent("你好"), name string
ToolCalls: []aiwire.ToolCall{{ID: "call_9", Type: "function", payload string
Function: aiwire.FunctionCall{Name: "f", Arguments: ""}}}}, wantNil bool
FinishReason: "length", prompt, cached int
}}, Usage: &aiwire.Usage{PromptTokens: 10, CompletionTokens: 5, TotalTokens: 15}} }{
out := IRRespToResponses(resp, "resp_abc", 1751966400) {"完整", `{"usage":{"input_tokens":9,"output_tokens":3,"total_tokens":12,"input_tokens_details":{"cached_tokens":5}}}`, false, 9, 5},
if out.Status != "incomplete" || out.IncompleteDetails == nil || out.IncompleteDetails.Reason != "max_output_tokens" { {"无细分", `{"usage":{"input_tokens":9,"output_tokens":3,"total_tokens":12}}`, false, 9, 0},
t.Errorf("length 应映射 incomplete: %+v", out) {"无usage", `{"id":"resp_1"}`, true, 0, 0},
{"非法JSON", `xx`, true, 0, 0},
} }
if len(out.Output) != 2 || out.Output[0].Type != "message" || out.Output[1].Type != "function_call" { for _, test := range tests {
t.Fatalf("output = %+v", out.Output) t.Run(test.name, func(t *testing.T) {
usage := RespPassthroughUsage([]byte(test.payload))
if (usage == nil) != test.wantNil {
t.Fatalf("usage = %v, wantNil %v", usage, test.wantNil)
} }
if out.Output[0].Content[0].Text != "你好" || out.Output[1].CallID != "call_9" || out.Output[1].Arguments != "{}" { if usage == nil {
t.Errorf("item 装配错误: %+v", out.Output) return
} }
if out.Usage.InputTokens != 10 || out.Usage.OutputTokens != 5 || out.Store { if usage.PromptTokens != test.prompt || usage.CachedTokens() != test.cached {
t.Errorf("usage/store 错误: %+v", out) t.Fatalf("prompt=%d cached=%d, want %d/%d", usage.PromptTokens, usage.CachedTokens(), test.prompt, test.cached)
}
})
} }
} }
func chunkText(text string) aiwire.ChatChunk { // TestRespStreamCompletedUsage 断言流式 usage 只从 completed 事件提取。
return aiwire.ChatChunk{Choices: []aiwire.ChunkChoice{{Delta: aiwire.Delta{Content: text}}}} func TestRespStreamCompletedUsage(t *testing.T) {
completed := []byte(`{"type":"response.completed","response":{"usage":{"input_tokens":10,"output_tokens":5,"total_tokens":15,"input_tokens_details":{"cached_tokens":4}}}}`)
if u := RespStreamCompletedUsage(completed); u == nil || u.PromptTokens != 10 || u.CompletionTokens != 5 ||
u.PromptTokensDetails == nil || u.PromptTokensDetails.CachedTokens != 4 {
t.Fatalf("completed 事件 usage 解析失败: %+v", u)
} }
for _, data := range []string{
func TestRespStreamTextAndTool(t *testing.T) { `{"type":"response.output_text.delta","delta":"hi"}`,
st := NewRespStream("resp_x", "m", 1751966400) `{"type":"response.completed"}`,
var types []string `not-json`,
collect := func(evs []RespEvent) { } {
for _, ev := range evs { if u := RespStreamCompletedUsage([]byte(data)); u != nil {
types = append(types, ev.Event) t.Fatalf("非 completed 或缺 response 应返回 nil: %s", data)
} }
} }
collect(st.Feed(chunkText("你")))
collect(st.Feed(chunkText("好")))
collect(st.Feed(aiwire.ChatChunk{Choices: []aiwire.ChunkChoice{{Delta: aiwire.Delta{
ToolCalls: []aiwire.ToolCallDelta{{Index: 0, ID: "call_1", Function: aiwire.FunctionCallDelta{Name: "f", Arguments: "{\"a\""}}}}}}}))
collect(st.Feed(aiwire.ChatChunk{Choices: []aiwire.ChunkChoice{{Delta: aiwire.Delta{
ToolCalls: []aiwire.ToolCallDelta{{Index: 0, Function: aiwire.FunctionCallDelta{Arguments: ":1}"}}}}}},
Usage: &aiwire.Usage{PromptTokens: 3, CompletionTokens: 7, TotalTokens: 10}}))
finish := st.Finish()
collect(finish)
want := []string{
"response.created", "response.in_progress",
"response.output_item.added", "response.content_part.added", "response.output_text.delta",
"response.output_text.delta",
"response.output_text.done", "response.content_part.done", "response.output_item.done",
"response.output_item.added", "response.function_call_arguments.delta",
"response.function_call_arguments.delta",
"response.function_call_arguments.done", "response.output_item.done",
"response.completed",
}
if strings.Join(types, "\n") != strings.Join(want, "\n") {
t.Errorf("事件序列 =\n%s\nwant\n%s", strings.Join(types, "\n"), strings.Join(want, "\n"))
}
last := finish[len(finish)-1].Data
resp := last["response"].(aiwire.Response)
if len(resp.Output) != 2 || resp.Output[0].Content[0].Text != "你好" || resp.Output[1].Arguments != "{\"a\":1}" {
t.Errorf("终态快照 = %+v", resp.Output)
}
if resp.Usage == nil || resp.Usage.TotalTokens != 10 {
t.Errorf("终态 usage = %+v", resp.Usage)
}
// sequence_number 严格递增
if seq, ok := last["sequence_number"].(int); !ok || seq != len(want) {
t.Errorf("最终 sequence_number = %v, want %d", last["sequence_number"], len(want))
}
}
func TestRespStreamEmpty(t *testing.T) {
st := NewRespStream("resp_e", "m", 1)
evs := st.Finish()
if len(evs) != 2 || evs[0].Event != "response.created" || evs[1].Event != "response.completed" {
t.Errorf("空流事件 = %+v", evs)
}
} }
+411
View File
@@ -0,0 +1,411 @@
package service
import (
"encoding/json"
"fmt"
"strings"
"oci-portal/internal/aiwire"
)
// Anthropic Messages ↔ OCI OpenAI 兼容面(/actions/v1/responses)直通转换。
// typed chat 面剔除后 Messages 入口的唯一上游通路。语义损失(README 已披露):
// stop_sequences / top_k / metadata / thinking 无对应字段,忽略;上游 reasoning
// 输出项与增量事件丢弃(Anthropic thinking 块含签名语义,不伪造)。
// ErrAiUnsupportedBlock 表示请求含网关无法承接的内容块(仅支持文本/图片/工具块)。
var ErrAiUnsupportedBlock = fmt.Errorf("暂不支持文本与图片以外的内容块")
// AnthropicToResponsesBody 把 Messages 请求转为直通 body(强制 store:false)。
func AnthropicToResponsesBody(req aiwire.MessagesRequest) ([]byte, error) {
input, err := anthInputItems(req.Messages)
if err != nil {
return nil, err
}
body := map[string]any{"model": req.Model, "max_output_tokens": req.MaxTokens,
"input": input, "store": false}
if sys := req.SystemText(); sys != "" {
body["instructions"] = sys
}
if req.Temperature != nil {
body["temperature"] = *req.Temperature
}
if req.TopP != nil {
body["top_p"] = *req.TopP
}
if tools := anthRespTools(req.Tools); tools != nil {
body["tools"] = tools
}
if tc := anthRespToolChoice(req.ToolChoice); tc != nil {
body["tool_choice"] = tc
}
if req.OutputConfig != nil && req.OutputConfig.Effort != "" {
body["reasoning"] = map[string]string{"effort": strings.ToLower(req.OutputConfig.Effort)}
}
if req.Stream {
body["stream"] = true
}
return json.Marshal(body)
}
// anthInputItems 把消息序列展开为 Responses input 项:tool_use / tool_result 为
// 独立 function_call / function_call_output 项,其余聚合为 message 项;遇独立项时
// 先冲刷已聚合部件,保持块间相对顺序。
func anthInputItems(messages []aiwire.AnthMessage) ([]any, error) {
var items []any
for _, m := range messages {
var parts []map[string]any
flush := func() {
if len(parts) > 0 {
items = append(items, map[string]any{"role": m.Role, "content": parts})
parts = nil
}
}
for _, b := range m.Content.AllBlocks() {
item, part, err := anthBlockItem(m.Role, b)
if err != nil {
return nil, err
}
if part != nil {
parts = append(parts, part)
}
if item != nil {
flush()
items = append(items, item)
}
}
flush()
}
return items, nil
}
// anthBlockItem 把单个内容块转为独立项或消息部件(thinking 忽略,未知块拒绝)。
func anthBlockItem(role string, b aiwire.AnthBlock) (item, part map[string]any, err error) {
switch b.Type {
case "text":
return nil, anthTextPart(role, b.Text), nil
case "image":
part, err = anthImageInput(b.Source)
return nil, part, err
case "tool_use":
return map[string]any{"type": "function_call", "call_id": b.ID,
"name": b.Name, "arguments": string(b.Input)}, nil, nil
case "tool_result":
return map[string]any{"type": "function_call_output",
"call_id": b.ToolUseID, "output": b.ResultText()}, nil, nil
case "thinking", "redacted_thinking":
return nil, nil, nil
default:
return nil, nil, ErrAiUnsupportedBlock
}
}
// anthTextPart 按角色选择 Responses 文本部件类型(assistant 历史为 output_text)。
func anthTextPart(role, text string) map[string]any {
if role == "assistant" {
return map[string]any{"type": "output_text", "text": text}
}
return map[string]any{"type": "input_text", "text": text}
}
// anthImageInput 把 Anthropic image source 转为 Responses input_image 部件。
func anthImageInput(source json.RawMessage) (map[string]any, error) {
var src struct {
Type string `json:"type"`
MediaType string `json:"media_type"`
Data string `json:"data"`
URL string `json:"url"`
}
if err := json.Unmarshal(source, &src); err != nil {
return nil, fmt.Errorf("image source 解析失败: %w", err)
}
switch src.Type {
case "base64":
if src.MediaType == "" || src.Data == "" {
return nil, fmt.Errorf("image source 缺少 media_type 或 data")
}
return map[string]any{"type": "input_image",
"image_url": "data:" + src.MediaType + ";base64," + src.Data}, nil
case "url":
if src.URL == "" {
return nil, fmt.Errorf("image source 缺少 url")
}
return map[string]any{"type": "input_image", "image_url": src.URL}, nil
}
return nil, fmt.Errorf("不支持的 image source 类型 %q", src.Type)
}
func anthRespTools(tools []aiwire.AnthTool) []map[string]any {
if len(tools) == 0 {
return nil
}
out := make([]map[string]any, 0, len(tools))
for _, t := range tools {
out = append(out, map[string]any{"type": "function", "name": t.Name,
"description": t.Description, "parameters": t.InputSchema})
}
return out
}
// anthRespToolChoice 映射 {type:auto|any|tool|none,name} → Responses 形态。
func anthRespToolChoice(raw json.RawMessage) any {
if len(raw) == 0 {
return nil
}
var tc struct {
Type string `json:"type"`
Name string `json:"name"`
}
if json.Unmarshal(raw, &tc) != nil {
return nil
}
switch tc.Type {
case "auto":
return "auto"
case "any":
return "required"
case "none":
return "none"
case "tool":
return map[string]string{"type": "function", "name": tc.Name}
}
return nil
}
// respPayload 是直通响应中本转换关心的子集(未知字段忽略)。
type respPayload struct {
Model string `json:"model"`
Status string `json:"status"`
IncompleteDetails *respIncomplete `json:"incomplete_details"`
Output []respOutputItem `json:"output"`
Usage *aiwire.RespUsage `json:"usage"`
Error *map[string]string `json:"error"`
}
type respIncomplete struct {
Reason string `json:"reason"`
}
type respOutputItem struct {
Type string `json:"type"`
CallID string `json:"call_id"`
Name string `json:"name"`
Arguments string `json:"arguments"`
Content []struct {
Type string `json:"type"`
Text string `json:"text"`
} `json:"content"`
}
// ResponsesToAnthropic 把直通非流式响应转为 Anthropic Messages 响应。
func ResponsesToAnthropic(payload []byte, msgID string) (*aiwire.MessagesResponse, error) {
var resp respPayload
if err := json.Unmarshal(payload, &resp); err != nil {
return nil, fmt.Errorf("解析上游响应: %w", err)
}
out := &aiwire.MessagesResponse{ID: msgID, Type: "message", Role: "assistant",
Model: resp.Model, Content: []aiwire.AnthBlock{}, StopReason: "end_turn"}
for _, item := range resp.Output {
switch item.Type {
case "message":
for _, part := range item.Content {
if part.Type == "output_text" && part.Text != "" {
out.Content = append(out.Content, aiwire.AnthBlock{Type: "text", Text: part.Text})
}
}
case "function_call":
out.Content = append(out.Content, aiwire.AnthBlock{Type: "tool_use", ID: item.CallID,
Name: item.Name, Input: argsToJSON(item.Arguments)})
out.StopReason = "tool_use"
}
}
if resp.Status == "incomplete" && resp.IncompleteDetails != nil &&
resp.IncompleteDetails.Reason == "max_output_tokens" {
out.StopReason = "max_tokens"
}
out.Usage = anthUsageFromResp(resp.Usage)
return out, nil
}
func anthUsageFromResp(u *aiwire.RespUsage) aiwire.AnthUsage {
if u == nil {
return aiwire.AnthUsage{}
}
return aiwire.AnthUsage{InputTokens: u.InputTokens, OutputTokens: u.OutputTokens,
CacheReadInputTokens: u.InputTokensDetails.CachedTokens}
}
// argsToJSON 保证 tool_use.input 是合法 JSON 对象(模型可能产出非法片段)。
func argsToJSON(args string) json.RawMessage {
trimmed := strings.TrimSpace(args)
if trimmed == "" {
return json.RawMessage(`{}`)
}
if json.Valid([]byte(trimmed)) {
return json.RawMessage(trimmed)
}
b, _ := json.Marshal(map[string]string{"_raw": args})
return b
}
// ---- Anthropic 流式桥:Responses SSE 事件 → Anthropic 事件序列 ----
// AnthEvent 是一条待写出的 Anthropic SSE 事件。
type AnthEvent struct {
Event string
Data any
}
// AnthRespBridge 把直通 SSE 事件流桥接为 Anthropic 事件序列:
// message_start → content_block_start/delta/stop(text 与 tool_use 分块)→ message_delta → message_stop。
// reasoning 系列事件丢弃。
type AnthRespBridge struct {
id, model string
started bool
blockOpen bool
blockIsTool bool
blockIndex int
stopReason string
usage aiwire.AnthUsage
}
// NewAnthRespBridge 构造桥;id 为响应消息 ID。
func NewAnthRespBridge(id, model string) *AnthRespBridge {
return &AnthRespBridge{id: id, model: model, blockIndex: -1, stopReason: "end_turn"}
}
// respStreamEvent 是直通 SSE data JSON 中桥关心的子集。
type respStreamEvent struct {
Type string `json:"type"`
Delta string `json:"delta"`
Item *respOutputItem `json:"item"`
Response *respPayload `json:"response"`
}
// Feed 消费一行 SSE data JSON,返回应立即写出的事件。
func (st *AnthRespBridge) Feed(data []byte) []AnthEvent {
var ev respStreamEvent
if json.Unmarshal(data, &ev) != nil {
return nil
}
var events []AnthEvent
if !st.started {
st.started = true
events = append(events, st.startEvent())
}
switch ev.Type {
case "response.output_item.added":
if ev.Item != nil && ev.Item.Type == "function_call" {
events = append(events, st.openBlock(true, ev.Item.CallID, ev.Item.Name)...)
st.stopReason = "tool_use"
}
case "response.output_text.delta":
events = append(events, st.textDelta(ev.Delta)...)
case "response.function_call_arguments.delta":
events = append(events, st.argsDelta(ev.Delta)...)
case "response.completed", "response.incomplete", "response.failed":
st.finishFrom(ev.Response)
}
return events
}
func (st *AnthRespBridge) textDelta(delta string) []AnthEvent {
var events []AnthEvent
if !st.blockOpen || st.blockIsTool {
events = append(events, st.openBlock(false, "", "")...)
}
events = append(events, AnthEvent{Event: "content_block_delta", Data: map[string]any{
"type": "content_block_delta", "index": st.blockIndex,
"delta": map[string]string{"type": "text_delta", "text": delta},
}})
return events
}
func (st *AnthRespBridge) argsDelta(delta string) []AnthEvent {
if !st.blockOpen || !st.blockIsTool {
return nil
}
return []AnthEvent{{Event: "content_block_delta", Data: map[string]any{
"type": "content_block_delta", "index": st.blockIndex,
"delta": map[string]string{"type": "input_json_delta", "partial_json": delta},
}}}
}
// finishFrom 记录终态:usage 与 stop_reason(max_output_tokens 截断 → max_tokens)。
func (st *AnthRespBridge) finishFrom(resp *respPayload) {
if resp == nil {
return
}
st.usage = anthUsageFromResp(resp.Usage)
if resp.Status == "incomplete" && resp.IncompleteDetails != nil &&
resp.IncompleteDetails.Reason == "max_output_tokens" {
st.stopReason = "max_tokens"
}
}
func (st *AnthRespBridge) startEvent() AnthEvent {
return AnthEvent{Event: "message_start", Data: map[string]any{
"type": "message_start",
"message": map[string]any{
"id": st.id, "type": "message", "role": "assistant", "model": st.model,
"content": []any{}, "stop_reason": nil,
"usage": map[string]int{"input_tokens": 0, "output_tokens": 0},
},
}}
}
// openBlock 关闭当前块并打开新块(text 或 tool_use)。
func (st *AnthRespBridge) openBlock(isTool bool, toolID, toolName string) []AnthEvent {
var events []AnthEvent
if st.blockOpen {
events = append(events, st.closeBlockEvent())
}
st.blockOpen, st.blockIsTool = true, isTool
st.blockIndex++
block := map[string]any{"type": "text", "text": ""}
if isTool {
block = map[string]any{"type": "tool_use", "id": toolID, "name": toolName, "input": map[string]any{}}
}
events = append(events, AnthEvent{Event: "content_block_start", Data: map[string]any{
"type": "content_block_start", "index": st.blockIndex, "content_block": block,
}})
return events
}
func (st *AnthRespBridge) closeBlockEvent() AnthEvent {
return AnthEvent{Event: "content_block_stop", Data: map[string]any{
"type": "content_block_stop", "index": st.blockIndex,
}}
}
// Finish 在上游流结束后收尾:关块 → message_delta(stop_reason+usage)→ message_stop。
func (st *AnthRespBridge) Finish() []AnthEvent {
var events []AnthEvent
if !st.started {
st.started = true
events = append(events, st.startEvent())
}
if st.blockOpen {
events = append(events, st.closeBlockEvent())
st.blockOpen = false
}
usage := map[string]int{"output_tokens": st.usage.OutputTokens}
if st.usage.InputTokens > 0 {
usage["input_tokens"] = st.usage.InputTokens
}
if st.usage.CacheReadInputTokens > 0 {
usage["cache_read_input_tokens"] = st.usage.CacheReadInputTokens
}
events = append(events,
AnthEvent{Event: "message_delta", Data: map[string]any{
"type": "message_delta",
"delta": map[string]any{"stop_reason": st.stopReason, "stop_sequence": nil},
"usage": usage,
}},
AnthEvent{Event: "message_stop", Data: map[string]any{"type": "message_stop"}},
)
return events
}
// Usage 返回聚合到的用量(供调用日志)。
func (st *AnthRespBridge) Usage() aiwire.AnthUsage { return st.usage }
+173
View File
@@ -0,0 +1,173 @@
package service
import (
"encoding/json"
"strings"
"testing"
"oci-portal/internal/aiwire"
)
func mustAnthReq(t *testing.T, raw string) aiwire.MessagesRequest {
t.Helper()
var req aiwire.MessagesRequest
if err := json.Unmarshal([]byte(raw), &req); err != nil {
t.Fatalf("解析请求: %v", err)
}
return req
}
// TestAnthropicToResponsesBody 断言 system/消息/工具/effort 的直通装配与 store 强制。
func TestAnthropicToResponsesBody(t *testing.T) {
req := mustAnthReq(t, `{
"model": "xai.grok-4.3", "max_tokens": 128, "system": "你是助手", "stream": true,
"temperature": 0.5,
"messages": [
{"role": "user", "content": "东京天气?"},
{"role": "assistant", "content": [
{"type": "text", "text": "查询中"},
{"type": "tool_use", "id": "t1", "name": "get_weather", "input": {"city": "东京"}}
]},
{"role": "user", "content": [
{"type": "tool_result", "tool_use_id": "t1", "content": "晴 25 度"},
{"type": "text", "text": "继续"}
]}
],
"tools": [{"name": "get_weather", "description": "查天气", "input_schema": {"type": "object"}}],
"tool_choice": {"type": "any"},
"output_config": {"effort": "HIGH"}
}`)
payload, err := AnthropicToResponsesBody(req)
if err != nil {
t.Fatalf("AnthropicToResponsesBody: %v", err)
}
var body map[string]any
if err := json.Unmarshal(payload, &body); err != nil {
t.Fatalf("unmarshal: %v", err)
}
if body["model"] != "xai.grok-4.3" || body["max_output_tokens"] != float64(128) ||
body["instructions"] != "你是助手" || body["store"] != false || body["stream"] != true {
t.Fatalf("顶层字段装配错误: %v", body)
}
if body["tool_choice"] != "required" {
t.Fatalf("tool_choice = %v, want required", body["tool_choice"])
}
reasoning, _ := body["reasoning"].(map[string]any)
if reasoning["effort"] != "high" {
t.Fatalf("effort = %v, want high(小写透传)", reasoning)
}
input, _ := body["input"].([]any)
// user 消息、assistant 文本消息、function_call、function_call_output、末条 user 文本
if len(input) != 5 {
t.Fatalf("input 项数 = %d, want 5: %s", len(input), payload)
}
kinds := make([]string, 0, len(input))
for _, it := range input {
m := it.(map[string]any)
if ty, ok := m["type"].(string); ok {
kinds = append(kinds, ty)
} else {
kinds = append(kinds, "message/"+m["role"].(string))
}
}
want := "message/user,message/assistant,function_call,function_call_output,message/user"
if got := strings.Join(kinds, ","); got != want {
t.Fatalf("input 顺序 = %s, want %s", got, want)
}
if !strings.Contains(string(payload), `"output_text"`) {
t.Fatalf("assistant 历史文本应为 output_text 部件: %s", payload)
}
}
// TestAnthropicToResponsesBodyRejects 断言不支持的内容块拒绝与图片装配。
func TestAnthropicToResponsesBodyRejects(t *testing.T) {
bad := mustAnthReq(t, `{"model":"m","max_tokens":8,"messages":[
{"role":"user","content":[{"type":"document","source":{}}]}]}`)
if _, err := AnthropicToResponsesBody(bad); err == nil {
t.Fatal("document 块应拒绝")
}
img := mustAnthReq(t, `{"model":"m","max_tokens":8,"messages":[
{"role":"user","content":[{"type":"image","source":{"type":"base64","media_type":"image/png","data":"QUJD"}}]}]}`)
payload, err := AnthropicToResponsesBody(img)
if err != nil || !strings.Contains(string(payload), "data:image/png;base64,QUJD") {
t.Fatalf("图片应转 data URI: %v %s", err, payload)
}
}
// TestResponsesToAnthropic 断言输出块、stop_reason 与 usage 的回转。
func TestResponsesToAnthropic(t *testing.T) {
payload := []byte(`{"model":"xai.grok-4.3","status":"completed","output":[
{"type":"reasoning","summary":[]},
{"type":"message","content":[{"type":"output_text","text":"你好"}]},
{"type":"function_call","call_id":"c1","name":"get_weather","arguments":"{\"city\":\"东京\"}"}],
"usage":{"input_tokens":10,"output_tokens":5,"total_tokens":15,"input_tokens_details":{"cached_tokens":4}}}`)
out, err := ResponsesToAnthropic(payload, "msg_1")
if err != nil {
t.Fatalf("ResponsesToAnthropic: %v", err)
}
if len(out.Content) != 2 || out.Content[0].Type != "text" || out.Content[0].Text != "你好" {
t.Fatalf("content 装配错误: %+v", out.Content)
}
if out.Content[1].Type != "tool_use" || out.Content[1].ID != "c1" || string(out.Content[1].Input) != `{"city":"东京"}` {
t.Fatalf("tool_use 装配错误: %+v", out.Content[1])
}
if out.StopReason != "tool_use" {
t.Fatalf("stop_reason = %s, want tool_use", out.StopReason)
}
if out.Usage.InputTokens != 10 || out.Usage.OutputTokens != 5 || out.Usage.CacheReadInputTokens != 4 {
t.Fatalf("usage = %+v", out.Usage)
}
trunc := []byte(`{"model":"m","status":"incomplete","incomplete_details":{"reason":"max_output_tokens"},
"output":[{"type":"message","content":[{"type":"output_text","text":"半"}]}]}`)
out2, err := ResponsesToAnthropic(trunc, "msg_2")
if err != nil || out2.StopReason != "max_tokens" {
t.Fatalf("截断 stop_reason = %v, %v", out2, err)
}
}
func bridgeEventTypes(events []AnthEvent) string {
kinds := make([]string, 0, len(events))
for _, ev := range events {
kinds = append(kinds, ev.Event)
}
return strings.Join(kinds, ",")
}
// TestAnthRespBridge 断言流桥:文本增量、工具调用切块、reasoning 丢弃与收尾事件。
func TestAnthRespBridge(t *testing.T) {
st := NewAnthRespBridge("msg_1", "m1")
var events []AnthEvent
feed := func(lines ...string) {
for _, l := range lines {
events = append(events, st.Feed([]byte(l))...)
}
}
feed(`{"type":"response.created","response":{"model":"m1"}}`,
`{"type":"response.reasoning_text.delta","delta":"思考中"}`,
`{"type":"response.output_text.delta","delta":"你"}`,
`{"type":"response.output_text.delta","delta":"好"}`,
`{"type":"response.output_item.added","item":{"type":"function_call","call_id":"c1","name":"f"}}`,
`{"type":"response.function_call_arguments.delta","delta":"{\"a\":1}"}`,
`{"type":"response.completed","response":{"status":"completed","usage":{"input_tokens":8,"output_tokens":4}}}`)
events = append(events, st.Finish()...)
got := bridgeEventTypes(events)
want := "message_start,content_block_start,content_block_delta,content_block_delta," +
"content_block_stop,content_block_start,content_block_delta,content_block_stop," +
"message_delta,message_stop"
if got != want {
t.Fatalf("事件序列:\n got %s\nwant %s", got, want)
}
if st.Usage().InputTokens != 8 || st.Usage().OutputTokens != 4 {
t.Fatalf("usage = %+v", st.Usage())
}
}
// TestAnthRespBridgeEmpty 断言空流也产出完整事件骨架。
func TestAnthRespBridgeEmpty(t *testing.T) {
st := NewAnthRespBridge("msg_1", "m1")
if got := bridgeEventTypes(st.Finish()); got != "message_start,message_delta,message_stop" {
t.Errorf("空流事件序列 = %s", got)
}
}