AI 网关切换 OpenAI 兼容面并移除 chat 端点,新增模型黑白名单
This commit is contained in:
+12
-3
@@ -2,19 +2,28 @@
|
||||
|
||||
格式遵循 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/),版本号遵循语义化版本。
|
||||
|
||||
## [0.2.0] - 2026-07-10
|
||||
## [0.2.0] - 2026-07-12
|
||||
|
||||
### Added
|
||||
|
||||
- AI 网关模型池自愈:同步与每日探测后自动逐个试调验证模型可用性(maxTokens=1),不可按需调用的模型(微调基座 / 区域未开放按需供给)自动标记并从模型列表、路由与探测候选中剔除;已标记模型每 20 小时复检,OCI 恢复供给后自动解除标记。`ai_model_caches` 新增 `unusable` / `unusable_reason` / `checked_at`(启动自动迁移),渠道详情接口保留标记行与原因便于排查
|
||||
- AI 网关模型黑名单:面板集中拉黑模型,全渠道剔除(模型列表 / 路由 / 探测候选不再出现,同步不入库),移除条目并重新同步后恢复;新表 `ai_model_blacklists`(启动自动迁移),管理接口 `/api/v1/ai-blacklist`
|
||||
- AI 密钥模型白名单:密钥可限定放行的模型列表(空 = 不限),白名单外调用 404,模型列表只返回交集
|
||||
- 推理力度(effort)直通:Responses `reasoning.effort` 与 Anthropic Messages `output_config.effort` 原样透传上游、不做档位校验;grok-4.3 五档(`none` 完全关闭思考)、gpt-oss `minimal`-`high`、multi-agent `xhigh`(控制并行 agent 数)实测生效
|
||||
- xAI Grok 服务端工具:`/ai/v1/responses` 的 `tools` 支持 `{"type":"web_search"}` / `{"type":"x_search"}`(官方 Agent Tools 格式,可与 `function` 混用,仅非流式),响应保留 `web_search_call` 输出项与引用
|
||||
|
||||
### Changed
|
||||
|
||||
- **AI 网关对话链路整体切换到 OCI OpenAI 兼容面**(`/actions/v1/responses`),typed chat 面全链路移除:`/ai/v1/responses` 请求体原样直通(仅强制 `store:false`),流式为上游原生事件流,gpt-oss 推理增量直达客户端;`/ai/v1/messages` 转换为 Responses 请求直通,流式经事件桥转回标准 Anthropic 事件序列,`stop_sequences` / `top_k` 等无兼容面对应物的字段改为静默忽略,思考输出不转 `thinking` 块;渠道探测改用兼容面最小请求;流式建立后绑定渠道,中断不重试不计熔断
|
||||
- 后台任务「立即执行」改异步触发:接口立即返回 202,执行结果经任务日志轮询呈现;同一任务在途时重复触发返回 409「任务正在执行中」,与 cron 重叠触发静默跳过(此前同步阻塞数十秒且同一任务可并发重复执行)
|
||||
- 渠道探测候选跨厂商分散(上限 8,voice 等不可对话形态排除):单一厂商在区域内全为微调基座时不再拖垮整个渠道的探测结论
|
||||
- 网关调用(对话 / 流式 / 向量化)遇「模型不可按需调用」类错误(微调基座 400 / 实体不存在 404)自动标记该模型并换渠道重试,且不计入渠道熔断;仅鉴权类错误才判定租户无配额
|
||||
- 网关调用遇「模型不可按需调用」类错误(微调基座 400 / 实体不存在 404)自动换渠道重试且不计入渠道熔断,错误信息带模型名便于加入黑名单;仅鉴权类错误才判定租户无配额
|
||||
- 同名模型多条目去重优先保留非微调基座条目,降低缓存到不可调用 OCID 的概率
|
||||
|
||||
### Removed
|
||||
|
||||
- **`/ai/v1/chat/completions` 端点移除**(404):OpenAI 官方新能力已集中到 Responses 且本面板真实流量为零,请迁移 `/ai/v1/responses` 或 `/ai/v1/messages`
|
||||
- google / cohere 对话模型不再供给(OCI OpenAI 兼容面不承接,上游 400;cohere embed 向量模型不受影响),模型目录按 `xai.` / `meta.` / `openai.` 厂商前缀过滤
|
||||
|
||||
### Fixed
|
||||
|
||||
- 渠道「同步模型成功但探测报错不可用」:法兰克福等区域的微调基座模型占满探测候选导致的误报(探测状态与真实可用性不符的根因)
|
||||
|
||||
+1
-1
@@ -1 +1 @@
|
||||
v0.1.0
|
||||
v0.2.0
|
||||
|
||||
@@ -34,7 +34,7 @@
|
||||
- **资源管理**:实例(含创建/电源操作/更换公网 IP/IPv6/VNIC/引导卷)、VCN/子网/安全列表、块存储、限额与成本查询,多区域/多区间支持
|
||||
- **抢机任务**:cron 周期尝试创建实例直到成功,支持熔断与通知
|
||||
- **网页控制台**:实例串行控制台(xterm)与 VNC(noVNC),两跳 SSH 隧道
|
||||
- **AI 网关**:OpenAI / Anthropic 兼容端点转发 OCI GenAI(对话/Responses/Embeddings),号池渠道加权负载均衡、熔断探测、密钥管理与用量日志
|
||||
- **AI 网关**:OpenAI Responses / Anthropic Messages / Embeddings 兼容端点转发 OCI GenAI,号池渠道加权负载均衡、熔断探测、密钥管理与用量日志;对话统一走 OCI OpenAI 兼容面(xai / meta / openai 厂商模型,流式事件与推理增量原样保真),支持 xAI Grok 服务端工具 `web_search` / `x_search`(格式同 xAI 官方 Agent Tools,仅非流式)——兼容面属实测可用但未见 Oracle 文档承诺的能力,行为可能随上游调整
|
||||
- **日志回传**:OCI Audit 事件经 Connector Hub → Notifications HTTPS 订阅回传入库,一键创建链路;自定义告警规则(事件类型/来源 IP 白名单/资源/频率阈值)命中即推送
|
||||
- **租户治理**:IAM 用户/MFA/API Key 管理、密码策略、身份提供商(SAML)、通知收件人;多 Identity Domain 租户可按域切换管理
|
||||
- **安全**:JWT + bcrypt(凭据变更旧令牌立即失效,可一键撤销全部会话)、TOTP 两步验证、OIDC/GitHub 外部登录、登录锁定、IP 限速、真实 IP 头可配、请求体/超时防护、系统操作审计
|
||||
@@ -60,7 +60,7 @@ mkdir -p ./data && sudo chown 65532:65532 ./data
|
||||
DATA_KEY=$(openssl rand -hex 32) JWT_SECRET=$(openssl rand -hex 32) ADMIN_PASSWORD=<初始密码> docker compose up -d
|
||||
```
|
||||
|
||||
默认只监听 `127.0.0.1:18888`;公网访问须置于 TLS 反向代理之后,见下文「反向代理」。
|
||||
默认只监听 `127.0.0.1:18888`;公网访问须置于 TLS 反向代理之后,见下文「反向代理」。
|
||||
|
||||
### 源码构建(单文件,含前端)
|
||||
|
||||
@@ -148,6 +148,87 @@ services:
|
||||
- 反代默认追加的 `X-Forwarded-For` 用于还原真实客户端 IP,系统日志留痕、登录锁定与 IP 限速都依赖它
|
||||
- 请求体上限建议与后端一致:`/api/*` 1MB、`/ai/*` 10MB(Caddy 用 `request_body` 按前缀分层)
|
||||
|
||||
## AI 网关接口
|
||||
|
||||
对外提供 OpenAI / Anthropic 兼容端点,转发 OCI GenAI On-Demand 推理。鉴权用面板创建的 AI 密钥,`Authorization: Bearer sk-...` 与 `x-api-key: sk-...` 双头均可;密钥可绑定渠道分组实现路由隔离,也可配置模型白名单(白名单外调用 404,模型列表只返回交集)。请求体上限 10MB。
|
||||
|
||||
| 端点 | 协议 | 流式 |
|
||||
| --- | --- | --- |
|
||||
| `POST /ai/v1/responses` | OpenAI Responses(无状态子集) | SSE 支持(服务端工具除外) |
|
||||
| `POST /ai/v1/messages` | Anthropic Messages | SSE 支持 |
|
||||
| `POST /ai/v1/embeddings` | OpenAI Embeddings | — |
|
||||
| `GET /ai/v1/models` | OpenAI 模型列表(来自渠道模型目录) | — |
|
||||
|
||||
对话端点统一转发 OCI OpenAI 兼容面(`/actions/v1/responses`),因此仅提供该面支持的 `xai.` / `meta.` / `openai.` 前缀模型;google / cohere 对话模型不在兼容面供给(上游 400),不再提供(cohere embed 模型不受影响)。兼容面属实测可用但无 Oracle 文档承诺的能力,行为可能随上游调整。**Chat Completions(`/ai/v1/chat/completions`)端点已移除**,请迁移到 Responses 或 Messages。
|
||||
|
||||
以下各表对照网关行为:✅ 转发上游;⚠️ 接受但忽略(静默丢弃,不影响请求);❌ 拒绝(400,不发上游)。
|
||||
|
||||
### Responses 字段
|
||||
|
||||
请求体除下列例外**原样直通上游**(未列字段一并转发,效果由上游决定):
|
||||
|
||||
| 字段 | 支持 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `model`、`input` | ✅ | `input` 接受 string 或 item 数组 |
|
||||
| `stream` | ✅ | SSE,上游原生事件流(含 gpt-oss 推理增量);与服务端工具互斥 |
|
||||
| `reasoning.effort` | ✅ | 直达上游不限档位,见下「推理力度」 |
|
||||
| `store` | ⚠️ | 网关无状态,强制改写为 `false` |
|
||||
| `previous_response_id`、`conversation`、`background` | ❌ | 网关不保存历史,请求需自带完整上下文 |
|
||||
| `web_search` / `x_search` / `function` 之外的工具类型 | ❌ | 如 `file_search`、`code_interpreter`、`mcp` |
|
||||
|
||||
**服务端工具**:`tools` 支持 `{"type":"web_search"}` 与 `{"type":"x_search"}`(xAI Grok 官方 Agent Tools 格式,可与 `function` 混用),响应保留 `web_search_call` 输出项与引用;**仅非流式**。响应 `usage.input_tokens_details.cached_tokens` 透传缓存命中量。
|
||||
|
||||
### Anthropic Messages 字段
|
||||
|
||||
网关把 Messages 请求转换为 Responses 请求送上游,响应(含流式事件序列)转回 Anthropic 形态:
|
||||
|
||||
| 字段 | 支持 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `model`、`max_tokens`、`messages` | ✅ | `max_tokens` 必填;content 块支持 `text` / `image` / `tool_use` / `tool_result`,其余块类型 400 |
|
||||
| `system` | ✅ | string 或块数组,映射为 `instructions` |
|
||||
| `temperature`、`top_p` | ✅ | |
|
||||
| `stream` | ✅ | SSE,标准 Anthropic 事件序列(`message_start` → `content_block_*` → `message_delta` → `message_stop`) |
|
||||
| `tools`、`tool_choice` | ✅ | 映射为 Responses `function` 工具;`tool_choice` 支持 `auto` / `any` / `none` / `tool` |
|
||||
| `output_config.effort` | ✅ | 直达上游不限档位,见下「推理力度」 |
|
||||
| `stop_sequences`、`top_k`、`metadata`、`thinking`、`service_tier` | ⚠️ | Responses 面无对应物,静默忽略 |
|
||||
|
||||
模型的思考输出(reasoning)不转为 `thinking` 块,静默丢弃;`usage.cache_read_input_tokens` 透传缓存命中量。
|
||||
|
||||
### Embeddings 字段
|
||||
|
||||
| 字段 | 支持 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `model`、`input` | ✅ | `input` 接受 string 或数组 |
|
||||
| `dimensions` | ✅ | |
|
||||
| `encoding_format`、`user` | ⚠️ | 输出恒为 float 数组 |
|
||||
|
||||
### 推理力度(effort)
|
||||
|
||||
两个对话协议各以原生字段控制推理深度,取值直达上游不做档位校验:
|
||||
|
||||
| 协议 | 字段 |
|
||||
| --- | --- |
|
||||
| Responses | `reasoning.effort` |
|
||||
| Anthropic Messages | `output_config.effort`(转小写后透传) |
|
||||
|
||||
是否支持、可用档位与实际效果由模型决定(2026-07 兼容面实测):
|
||||
|
||||
| 模型 | 支持档位 | 实测行为 |
|
||||
| --- | --- | --- |
|
||||
| `xai.grok-4.3` | `none` / `minimal` / `low` / `medium` / `high` | 全部生效:`none` 完全关闭推理,默认 `low`,`high` 推理量最大 |
|
||||
| `openai.gpt-oss-*` | `minimal` / `low` / `medium` / `high` | `none` 被上游 400(Harmony 格式不支持);思考计入 completion tokens |
|
||||
| `xai.grok-3-mini(-fast)` | 名义 `low` / `high` | 各档均被接受,但 `none` 并不关闭推理(仍产生思考) |
|
||||
| `meta.llama-*` | 接受任意档 | 非推理模型,参数无实际效果 |
|
||||
| `xai.grok-4.20-multi-agent*` | `low` / `medium` / `high` / `xhigh` | effort 控制并行 agent 数(4 / 16)而非推理深度 |
|
||||
| 其余 grok(`grok-3`、`grok-4`、`grok-4-fast-*`、`grok-4-1-fast-*`、`grok-code-fast-1`、`grok-4.20-*` 单/双型号全系) | 不支持 | 携带即被上游 400(`does not support parameter reasoningEffort`) |
|
||||
|
||||
不携带该字段时网关不下发,行为由模型默认档位决定。
|
||||
|
||||
### 通用行为
|
||||
|
||||
- 未知模型 404;无可用渠道 503;单次请求最多尝试 3 个渠道(429/5xx/网络错误自动换渠道并计入熔断,模型级 400/404 换渠道不计熔断,其余 4xx 原样透传);流式建立后绑定渠道,中断不重试
|
||||
- 调用计量(渠道、token 用量、缓存命中、时延、重试数)写入面板「调用日志」;密钥可按需开启限时内容抓取
|
||||
|
||||
## 环境变量
|
||||
|
||||
| 变量 | 必填 | 默认值 | 说明 |
|
||||
|
||||
+82
-30
@@ -15,34 +15,6 @@ const docTemplate = `{
|
||||
"host": "{{.Host}}",
|
||||
"basePath": "{{.BasePath}}",
|
||||
"paths": {
|
||||
"/ai/v1/chat/completions": {
|
||||
"post": {
|
||||
"tags": [
|
||||
"AI 网关"
|
||||
],
|
||||
"summary": "OpenAI 兼容对话补全",
|
||||
"parameters": [
|
||||
{
|
||||
"description": "OpenAI chat/completions 请求体(支持 stream)",
|
||||
"name": "body",
|
||||
"in": "body",
|
||||
"required": true,
|
||||
"schema": {
|
||||
"type": "object"
|
||||
}
|
||||
}
|
||||
],
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "OpenAI 兼容响应(流式为 SSE)",
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"additionalProperties": true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/ai/v1/embeddings": {
|
||||
"post": {
|
||||
"tags": [
|
||||
@@ -79,7 +51,7 @@ const docTemplate = `{
|
||||
"summary": "Anthropic Messages 兼容端点",
|
||||
"parameters": [
|
||||
{
|
||||
"description": "Anthropic messages 请求体(支持 stream)",
|
||||
"description": "Anthropic messages 请求体(支持 stream;经 OCI OpenAI 兼容面直通)",
|
||||
"name": "body",
|
||||
"in": "body",
|
||||
"required": true,
|
||||
@@ -124,7 +96,7 @@ const docTemplate = `{
|
||||
"summary": "OpenAI Responses 兼容端点",
|
||||
"parameters": [
|
||||
{
|
||||
"description": "OpenAI responses 请求体(支持 stream)",
|
||||
"description": "OpenAI responses 请求体(支持 stream;web_search/x_search 服务端工具仅非流式)",
|
||||
"name": "body",
|
||||
"in": "body",
|
||||
"required": true,
|
||||
@@ -166,6 +138,86 @@ const docTemplate = `{
|
||||
}
|
||||
}
|
||||
},
|
||||
"/api/v1/ai-blacklist": {
|
||||
"get": {
|
||||
"security": [
|
||||
{
|
||||
"BearerAuth": []
|
||||
}
|
||||
],
|
||||
"tags": [
|
||||
"AI 管理"
|
||||
],
|
||||
"summary": "模型黑名单列表",
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "OK",
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"additionalProperties": true
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"post": {
|
||||
"security": [
|
||||
{
|
||||
"BearerAuth": []
|
||||
}
|
||||
],
|
||||
"tags": [
|
||||
"AI 管理"
|
||||
],
|
||||
"summary": "添加模型黑名单",
|
||||
"parameters": [
|
||||
{
|
||||
"description": "请求体:{name: 模型名}",
|
||||
"name": "body",
|
||||
"in": "body",
|
||||
"required": true,
|
||||
"schema": {
|
||||
"type": "object"
|
||||
}
|
||||
}
|
||||
],
|
||||
"responses": {
|
||||
"201": {
|
||||
"description": "Created",
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"additionalProperties": true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/api/v1/ai-blacklist/{id}": {
|
||||
"delete": {
|
||||
"security": [
|
||||
{
|
||||
"BearerAuth": []
|
||||
}
|
||||
],
|
||||
"tags": [
|
||||
"AI 管理"
|
||||
],
|
||||
"summary": "移除模型黑名单(重新同步后模型恢复入池)",
|
||||
"parameters": [
|
||||
{
|
||||
"type": "integer",
|
||||
"description": "黑名单条目 ID",
|
||||
"name": "id",
|
||||
"in": "path",
|
||||
"required": true
|
||||
}
|
||||
],
|
||||
"responses": {
|
||||
"204": {
|
||||
"description": "无内容"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/api/v1/ai-channels": {
|
||||
"get": {
|
||||
"security": [
|
||||
|
||||
+82
-30
@@ -8,34 +8,6 @@
|
||||
},
|
||||
"basePath": "/",
|
||||
"paths": {
|
||||
"/ai/v1/chat/completions": {
|
||||
"post": {
|
||||
"tags": [
|
||||
"AI 网关"
|
||||
],
|
||||
"summary": "OpenAI 兼容对话补全",
|
||||
"parameters": [
|
||||
{
|
||||
"description": "OpenAI chat/completions 请求体(支持 stream)",
|
||||
"name": "body",
|
||||
"in": "body",
|
||||
"required": true,
|
||||
"schema": {
|
||||
"type": "object"
|
||||
}
|
||||
}
|
||||
],
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "OpenAI 兼容响应(流式为 SSE)",
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"additionalProperties": true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/ai/v1/embeddings": {
|
||||
"post": {
|
||||
"tags": [
|
||||
@@ -72,7 +44,7 @@
|
||||
"summary": "Anthropic Messages 兼容端点",
|
||||
"parameters": [
|
||||
{
|
||||
"description": "Anthropic messages 请求体(支持 stream)",
|
||||
"description": "Anthropic messages 请求体(支持 stream;经 OCI OpenAI 兼容面直通)",
|
||||
"name": "body",
|
||||
"in": "body",
|
||||
"required": true,
|
||||
@@ -117,7 +89,7 @@
|
||||
"summary": "OpenAI Responses 兼容端点",
|
||||
"parameters": [
|
||||
{
|
||||
"description": "OpenAI responses 请求体(支持 stream)",
|
||||
"description": "OpenAI responses 请求体(支持 stream;web_search/x_search 服务端工具仅非流式)",
|
||||
"name": "body",
|
||||
"in": "body",
|
||||
"required": true,
|
||||
@@ -159,6 +131,86 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"/api/v1/ai-blacklist": {
|
||||
"get": {
|
||||
"security": [
|
||||
{
|
||||
"BearerAuth": []
|
||||
}
|
||||
],
|
||||
"tags": [
|
||||
"AI 管理"
|
||||
],
|
||||
"summary": "模型黑名单列表",
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "OK",
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"additionalProperties": true
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"post": {
|
||||
"security": [
|
||||
{
|
||||
"BearerAuth": []
|
||||
}
|
||||
],
|
||||
"tags": [
|
||||
"AI 管理"
|
||||
],
|
||||
"summary": "添加模型黑名单",
|
||||
"parameters": [
|
||||
{
|
||||
"description": "请求体:{name: 模型名}",
|
||||
"name": "body",
|
||||
"in": "body",
|
||||
"required": true,
|
||||
"schema": {
|
||||
"type": "object"
|
||||
}
|
||||
}
|
||||
],
|
||||
"responses": {
|
||||
"201": {
|
||||
"description": "Created",
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"additionalProperties": true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/api/v1/ai-blacklist/{id}": {
|
||||
"delete": {
|
||||
"security": [
|
||||
{
|
||||
"BearerAuth": []
|
||||
}
|
||||
],
|
||||
"tags": [
|
||||
"AI 管理"
|
||||
],
|
||||
"summary": "移除模型黑名单(重新同步后模型恢复入池)",
|
||||
"parameters": [
|
||||
{
|
||||
"type": "integer",
|
||||
"description": "黑名单条目 ID",
|
||||
"name": "id",
|
||||
"in": "path",
|
||||
"required": true
|
||||
}
|
||||
],
|
||||
"responses": {
|
||||
"204": {
|
||||
"description": "无内容"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/api/v1/ai-channels": {
|
||||
"get": {
|
||||
"security": [
|
||||
|
||||
+50
-20
@@ -599,24 +599,6 @@ info:
|
||||
title: OCI Portal API
|
||||
version: 0.0.1
|
||||
paths:
|
||||
/ai/v1/chat/completions:
|
||||
post:
|
||||
parameters:
|
||||
- description: OpenAI chat/completions 请求体(支持 stream)
|
||||
in: body
|
||||
name: body
|
||||
required: true
|
||||
schema:
|
||||
type: object
|
||||
responses:
|
||||
"200":
|
||||
description: OpenAI 兼容响应(流式为 SSE)
|
||||
schema:
|
||||
additionalProperties: true
|
||||
type: object
|
||||
summary: OpenAI 兼容对话补全
|
||||
tags:
|
||||
- AI 网关
|
||||
/ai/v1/embeddings:
|
||||
post:
|
||||
parameters:
|
||||
@@ -638,7 +620,7 @@ paths:
|
||||
/ai/v1/messages:
|
||||
post:
|
||||
parameters:
|
||||
- description: Anthropic messages 请求体(支持 stream)
|
||||
- description: Anthropic messages 请求体(支持 stream;经 OCI OpenAI 兼容面直通)
|
||||
in: body
|
||||
name: body
|
||||
required: true
|
||||
@@ -667,7 +649,7 @@ paths:
|
||||
/ai/v1/responses:
|
||||
post:
|
||||
parameters:
|
||||
- description: OpenAI responses 请求体(支持 stream)
|
||||
- description: OpenAI responses 请求体(支持 stream;web_search/x_search 服务端工具仅非流式)
|
||||
in: body
|
||||
name: body
|
||||
required: true
|
||||
@@ -695,6 +677,54 @@ paths:
|
||||
summary: 返回构建与运行环境信息,「设置 · 关于」页展示
|
||||
tags:
|
||||
- 设置
|
||||
/api/v1/ai-blacklist:
|
||||
get:
|
||||
responses:
|
||||
"200":
|
||||
description: OK
|
||||
schema:
|
||||
additionalProperties: true
|
||||
type: object
|
||||
security:
|
||||
- BearerAuth: []
|
||||
summary: 模型黑名单列表
|
||||
tags:
|
||||
- AI 管理
|
||||
post:
|
||||
parameters:
|
||||
- description: '请求体:{name: 模型名}'
|
||||
in: body
|
||||
name: body
|
||||
required: true
|
||||
schema:
|
||||
type: object
|
||||
responses:
|
||||
"201":
|
||||
description: Created
|
||||
schema:
|
||||
additionalProperties: true
|
||||
type: object
|
||||
security:
|
||||
- BearerAuth: []
|
||||
summary: 添加模型黑名单
|
||||
tags:
|
||||
- AI 管理
|
||||
/api/v1/ai-blacklist/{id}:
|
||||
delete:
|
||||
parameters:
|
||||
- description: 黑名单条目 ID
|
||||
in: path
|
||||
name: id
|
||||
required: true
|
||||
type: integer
|
||||
responses:
|
||||
"204":
|
||||
description: 无内容
|
||||
security:
|
||||
- BearerAuth: []
|
||||
summary: 移除模型黑名单(重新同步后模型恢复入池)
|
||||
tags:
|
||||
- AI 管理
|
||||
/api/v1/ai-channels:
|
||||
get:
|
||||
responses:
|
||||
|
||||
@@ -20,6 +20,13 @@ type MessagesRequest struct {
|
||||
ToolChoice json.RawMessage `json:"tool_choice,omitempty"`
|
||||
Metadata json.RawMessage `json:"metadata,omitempty"`
|
||||
Thinking json.RawMessage `json:"thinking,omitempty"`
|
||||
// OutputConfig 是 Anthropic 官方输出控制(effort 等);网关映射 effort 到 OCI reasoningEffort。
|
||||
OutputConfig *AnthOutputConfig `json:"output_config,omitempty"`
|
||||
}
|
||||
|
||||
// AnthOutputConfig 是 Anthropic Messages 顶层 output_config 对象。
|
||||
type AnthOutputConfig struct {
|
||||
Effort string `json:"effort,omitempty"`
|
||||
}
|
||||
|
||||
// SystemText 把顶层 system(string 或块数组)拍平为文本。
|
||||
|
||||
+2
-209
@@ -1,35 +1,11 @@
|
||||
// Package aiwire 定义 AI 网关的线格式类型:OpenAI Chat Completions(兼作网关内部
|
||||
// 中间表示 IR,因 OCI GENERIC 格式即其镜像)与 Anthropic Messages。零内部依赖,
|
||||
// oci 层与 service 层共同引用。
|
||||
// Package aiwire 定义 AI 网关的线格式类型:OpenAI Responses / Anthropic Messages /
|
||||
// Embeddings 与通用记账、错误、模型列表结构。上游统一为 OCI OpenAI 兼容面。
|
||||
package aiwire
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// ChatRequest 是 OpenAI /v1/chat/completions 请求体;TopK 为 OCI/Anthropic 扩展字段。
|
||||
type ChatRequest struct {
|
||||
Model string `json:"model"`
|
||||
Messages []ChatMessage `json:"messages"`
|
||||
MaxTokens *int `json:"max_tokens,omitempty"`
|
||||
MaxCompletionTokens *int `json:"max_completion_tokens,omitempty"`
|
||||
Temperature *float64 `json:"temperature,omitempty"`
|
||||
TopP *float64 `json:"top_p,omitempty"`
|
||||
TopK *int `json:"top_k,omitempty"`
|
||||
FrequencyPenalty *float64 `json:"frequency_penalty,omitempty"`
|
||||
PresencePenalty *float64 `json:"presence_penalty,omitempty"`
|
||||
Stop StringList `json:"stop,omitempty"`
|
||||
Seed *int `json:"seed,omitempty"`
|
||||
N *int `json:"n,omitempty"`
|
||||
Stream bool `json:"stream,omitempty"`
|
||||
StreamOptions *StreamOptions `json:"stream_options,omitempty"`
|
||||
Tools []Tool `json:"tools,omitempty"`
|
||||
ToolChoice json.RawMessage `json:"tool_choice,omitempty"`
|
||||
ResponseFormat *ResponseFormat `json:"response_format,omitempty"`
|
||||
ReasoningEffort string `json:"reasoning_effort,omitempty"`
|
||||
}
|
||||
|
||||
// StringList 兼容 OpenAI stop 的 string 与 []string 两种线格式。
|
||||
type StringList []string
|
||||
|
||||
@@ -50,151 +26,6 @@ func (s *StringList) UnmarshalJSON(b []byte) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// StreamOptions 控制流式末尾是否附带 usage 事件。
|
||||
type StreamOptions struct {
|
||||
IncludeUsage bool `json:"include_usage"`
|
||||
}
|
||||
|
||||
// ChatMessage 是请求与非流式响应共用的消息结构。
|
||||
type ChatMessage struct {
|
||||
Role string `json:"role"`
|
||||
Content Content `json:"content"`
|
||||
Name string `json:"name,omitempty"`
|
||||
ToolCalls []ToolCall `json:"tool_calls,omitempty"`
|
||||
ToolCallID string `json:"tool_call_id,omitempty"`
|
||||
}
|
||||
|
||||
// Content 兼容 string 与多模态块数组两种线格式;一期仅透传文本。
|
||||
type Content struct {
|
||||
// Text 在原始值为字符串时填充
|
||||
Text string
|
||||
// Parts 在原始值为数组时填充
|
||||
Parts []ContentPart
|
||||
// isArray 记录原始形态,序列化时保形
|
||||
isArray bool
|
||||
}
|
||||
|
||||
// ContentPart 是块数组中的一段;text 与 image_url 被网关处理,其余类型拒绝。
|
||||
type ContentPart struct {
|
||||
Type string `json:"type"`
|
||||
Text string `json:"text,omitempty"`
|
||||
ImageURL *ImageURL `json:"image_url,omitempty"`
|
||||
}
|
||||
|
||||
// ImageURL 是 image_url 块负载;url 为 data URI(base64)或公网地址,与 OCI ImageUrl 同构。
|
||||
type ImageURL struct {
|
||||
URL string `json:"url"`
|
||||
Detail string `json:"detail,omitempty"`
|
||||
}
|
||||
|
||||
func (c *Content) UnmarshalJSON(b []byte) error {
|
||||
trimmed := strings.TrimSpace(string(b))
|
||||
if trimmed == "null" {
|
||||
return nil
|
||||
}
|
||||
if len(trimmed) > 0 && trimmed[0] == '[' {
|
||||
c.isArray = true
|
||||
return json.Unmarshal(b, &c.Parts)
|
||||
}
|
||||
return json.Unmarshal(b, &c.Text)
|
||||
}
|
||||
|
||||
func (c Content) MarshalJSON() ([]byte, error) {
|
||||
if c.isArray {
|
||||
return json.Marshal(c.Parts)
|
||||
}
|
||||
return json.Marshal(c.Text)
|
||||
}
|
||||
|
||||
// NewTextContent 构造字符串形态的内容。
|
||||
func NewTextContent(text string) Content {
|
||||
return Content{Text: text}
|
||||
}
|
||||
|
||||
// NewPartsContent 构造块数组形态的内容(多模态消息用)。
|
||||
func NewPartsContent(parts []ContentPart) Content {
|
||||
return Content{Parts: parts, isArray: true}
|
||||
}
|
||||
|
||||
// JoinText 返回全部文本内容(数组形态拼接 text 块)。
|
||||
func (c Content) JoinText() string {
|
||||
if !c.isArray {
|
||||
return c.Text
|
||||
}
|
||||
var sb strings.Builder
|
||||
for _, p := range c.Parts {
|
||||
if p.Type == "" || p.Type == "text" {
|
||||
sb.WriteString(p.Text)
|
||||
}
|
||||
}
|
||||
return sb.String()
|
||||
}
|
||||
|
||||
// HasUnsupported 报告是否含网关无法承接的内容块(text / image_url 之外的类型)。
|
||||
func (c Content) HasUnsupported() bool {
|
||||
for _, p := range c.Parts {
|
||||
switch p.Type {
|
||||
case "", "text":
|
||||
case "image_url":
|
||||
if p.ImageURL == nil || p.ImageURL.URL == "" {
|
||||
return true
|
||||
}
|
||||
default:
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// Tool 是 function 工具定义。
|
||||
type Tool struct {
|
||||
Type string `json:"type"`
|
||||
Function FunctionDef `json:"function"`
|
||||
}
|
||||
|
||||
// FunctionDef 描述工具的名称与 JSON Schema 参数。
|
||||
type FunctionDef struct {
|
||||
Name string `json:"name"`
|
||||
Description string `json:"description,omitempty"`
|
||||
Parameters json.RawMessage `json:"parameters,omitempty"`
|
||||
}
|
||||
|
||||
// ToolCall 是助手消息中的工具调用。
|
||||
type ToolCall struct {
|
||||
ID string `json:"id"`
|
||||
Type string `json:"type"`
|
||||
Function FunctionCall `json:"function"`
|
||||
}
|
||||
|
||||
// FunctionCall 携带调用名与 JSON 编码的实参。
|
||||
type FunctionCall struct {
|
||||
Name string `json:"name"`
|
||||
Arguments string `json:"arguments"`
|
||||
}
|
||||
|
||||
// ResponseFormat 对应 response_format(json_object / json_schema)。
|
||||
type ResponseFormat struct {
|
||||
Type string `json:"type"`
|
||||
JSONSchema json.RawMessage `json:"json_schema,omitempty"`
|
||||
}
|
||||
|
||||
// ChatResponse 是非流式响应体。
|
||||
type ChatResponse struct {
|
||||
ID string `json:"id"`
|
||||
Object string `json:"object"`
|
||||
Created int64 `json:"created"`
|
||||
Model string `json:"model"`
|
||||
Choices []Choice `json:"choices"`
|
||||
Usage *Usage `json:"usage,omitempty"`
|
||||
}
|
||||
|
||||
// Choice 是一条候选回复。
|
||||
type Choice struct {
|
||||
Index int `json:"index"`
|
||||
Message ChatMessage `json:"message"`
|
||||
FinishReason string `json:"finish_reason"`
|
||||
}
|
||||
|
||||
// Usage 是 token 用量。
|
||||
type Usage struct {
|
||||
PromptTokens int `json:"prompt_tokens"`
|
||||
@@ -217,44 +48,6 @@ func (u *Usage) CachedTokens() int {
|
||||
return u.PromptTokensDetails.CachedTokens
|
||||
}
|
||||
|
||||
// ChatChunk 是流式增量事件体(chat.completion.chunk)。
|
||||
type ChatChunk struct {
|
||||
ID string `json:"id"`
|
||||
Object string `json:"object"`
|
||||
Created int64 `json:"created"`
|
||||
Model string `json:"model"`
|
||||
Choices []ChunkChoice `json:"choices"`
|
||||
Usage *Usage `json:"usage,omitempty"`
|
||||
}
|
||||
|
||||
// ChunkChoice 是流式事件中的增量候选。
|
||||
type ChunkChoice struct {
|
||||
Index int `json:"index"`
|
||||
Delta Delta `json:"delta"`
|
||||
FinishReason *string `json:"finish_reason"`
|
||||
}
|
||||
|
||||
// Delta 是流式增量内容;Content 为纯文本片段。
|
||||
type Delta struct {
|
||||
Role string `json:"role,omitempty"`
|
||||
Content string `json:"content,omitempty"`
|
||||
ToolCalls []ToolCallDelta `json:"tool_calls,omitempty"`
|
||||
}
|
||||
|
||||
// ToolCallDelta 是工具调用的流式片段;Index 标识聚合目标。
|
||||
type ToolCallDelta struct {
|
||||
Index int `json:"index"`
|
||||
ID string `json:"id,omitempty"`
|
||||
Type string `json:"type,omitempty"`
|
||||
Function FunctionCallDelta `json:"function"`
|
||||
}
|
||||
|
||||
// FunctionCallDelta 是函数名 / 实参的增量片段。
|
||||
type FunctionCallDelta struct {
|
||||
Name string `json:"name,omitempty"`
|
||||
Arguments string `json:"arguments,omitempty"`
|
||||
}
|
||||
|
||||
// Model 是 /v1/models 列表项。
|
||||
type Model struct {
|
||||
ID string `json:"id"`
|
||||
|
||||
@@ -156,39 +156,6 @@ type RespReasoning struct {
|
||||
Effort string `json:"effort,omitempty"`
|
||||
}
|
||||
|
||||
// Response 是 /responses 响应对象。
|
||||
type Response struct {
|
||||
ID string `json:"id"`
|
||||
Object string `json:"object"`
|
||||
CreatedAt int64 `json:"created_at"`
|
||||
Status string `json:"status"`
|
||||
Model string `json:"model"`
|
||||
Output []RespOutItem `json:"output"`
|
||||
Usage *RespUsage `json:"usage,omitempty"`
|
||||
Error *RespError `json:"error"`
|
||||
IncompleteDetails *RespIncomplete `json:"incomplete_details"`
|
||||
Store bool `json:"store"`
|
||||
}
|
||||
|
||||
// RespOutItem 是 output 数组项:message 或 function_call。
|
||||
type RespOutItem struct {
|
||||
Type string `json:"type"`
|
||||
ID string `json:"id"`
|
||||
Status string `json:"status,omitempty"`
|
||||
Role string `json:"role,omitempty"`
|
||||
Content []RespOutPart `json:"content,omitempty"`
|
||||
CallID string `json:"call_id,omitempty"`
|
||||
Name string `json:"name,omitempty"`
|
||||
Arguments string `json:"arguments,omitempty"`
|
||||
}
|
||||
|
||||
// RespOutPart 是 message item 的内容部件(output_text)。
|
||||
type RespOutPart struct {
|
||||
Type string `json:"type"`
|
||||
Text string `json:"text"`
|
||||
Annotations []any `json:"annotations"`
|
||||
}
|
||||
|
||||
// RespUsage 是 Responses 口径的用量(input/output 命名)。
|
||||
type RespUsage struct {
|
||||
InputTokens int `json:"input_tokens"`
|
||||
|
||||
+62
-2
@@ -43,12 +43,13 @@ func (h *aiAdminHandler) createKey(c *gin.Context) {
|
||||
Name string `json:"name" binding:"required"`
|
||||
Value string `json:"value"`
|
||||
Group string `json:"group"`
|
||||
Models []string `json:"models"`
|
||||
}
|
||||
if err := c.ShouldBindJSON(&req); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
plaintext, key, err := h.gw.CreateKey(c.Request.Context(), req.Name, req.Value, req.Group)
|
||||
plaintext, key, err := h.gw.CreateKey(c.Request.Context(), req.Name, req.Value, req.Group, req.Models)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
return
|
||||
@@ -72,12 +73,13 @@ func (h *aiAdminHandler) updateKey(c *gin.Context) {
|
||||
Name string `json:"name"`
|
||||
Enabled *bool `json:"enabled"`
|
||||
Group *string `json:"group"`
|
||||
Models *[]string `json:"models"`
|
||||
}
|
||||
if err := c.ShouldBindJSON(&req); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
if err := h.gw.UpdateKey(c.Request.Context(), id, req.Name, req.Enabled, req.Group); err != nil {
|
||||
if err := h.gw.UpdateKey(c.Request.Context(), id, req.Name, req.Enabled, req.Group, req.Models); err != nil {
|
||||
respondError(c, err)
|
||||
return
|
||||
}
|
||||
@@ -285,6 +287,64 @@ func (h *aiAdminHandler) gatewayModels(c *gin.Context) {
|
||||
c.JSON(http.StatusOK, gin.H{"items": list.Data})
|
||||
}
|
||||
|
||||
// ---- 模型黑名单 ----
|
||||
|
||||
// @Summary 模型黑名单列表
|
||||
// @Tags AI 管理
|
||||
// @Success 200 {object} map[string]any
|
||||
// @Security BearerAuth
|
||||
// @Router /api/v1/ai-blacklist [get]
|
||||
func (h *aiAdminHandler) listBlacklist(c *gin.Context) {
|
||||
items, err := h.gw.Blacklist(c.Request.Context())
|
||||
if err != nil {
|
||||
respondError(c, err)
|
||||
return
|
||||
}
|
||||
c.JSON(http.StatusOK, gin.H{"items": items})
|
||||
}
|
||||
|
||||
// addBlacklist 拉黑模型:删除全部渠道缓存中的同名条目,后续同步 / 探测均过滤。
|
||||
//
|
||||
// @Summary 添加模型黑名单
|
||||
// @Tags AI 管理
|
||||
// @Param body body object true "请求体:{name: 模型名}"
|
||||
// @Success 201 {object} map[string]any
|
||||
// @Security BearerAuth
|
||||
// @Router /api/v1/ai-blacklist [post]
|
||||
func (h *aiAdminHandler) addBlacklist(c *gin.Context) {
|
||||
var req struct {
|
||||
Name string `json:"name" binding:"required"`
|
||||
}
|
||||
if err := c.ShouldBindJSON(&req); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
item, err := h.gw.AddBlacklist(c.Request.Context(), req.Name)
|
||||
if err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
c.JSON(http.StatusCreated, gin.H{"item": item})
|
||||
}
|
||||
|
||||
// @Summary 移除模型黑名单(重新同步后模型恢复入池)
|
||||
// @Tags AI 管理
|
||||
// @Param id path int true "黑名单条目 ID"
|
||||
// @Success 204 "无内容"
|
||||
// @Security BearerAuth
|
||||
// @Router /api/v1/ai-blacklist/{id} [delete]
|
||||
func (h *aiAdminHandler) removeBlacklist(c *gin.Context) {
|
||||
id, ok := aiPathID(c)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
if err := h.gw.RemoveBlacklist(c.Request.Context(), id); err != nil {
|
||||
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
|
||||
return
|
||||
}
|
||||
c.Status(http.StatusNoContent)
|
||||
}
|
||||
|
||||
// @Summary AI 调用日志列表
|
||||
// @Tags AI 管理
|
||||
// @Success 200 {object} map[string]any
|
||||
|
||||
+150
-178
@@ -1,13 +1,15 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"slices"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
@@ -86,6 +88,25 @@ func keyGroup(c *gin.Context) string {
|
||||
return ""
|
||||
}
|
||||
|
||||
// keyModels 取当前请求密钥的模型白名单(空 = 不限模型)。
|
||||
func keyModels(c *gin.Context) []string {
|
||||
if key, ok := c.Get(aiKeyCtx); ok {
|
||||
return key.(*model.AiKey).Models
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// checkKeyModel 校验请求模型是否在密钥白名单内;拒绝时按端点协议返回
|
||||
// 404 model_not_found(与未知模型同口径,不泄露密钥配置细节)并返回 false。
|
||||
func checkKeyModel(c *gin.Context, modelName string) bool {
|
||||
allowed := keyModels(c)
|
||||
if len(allowed) == 0 || slices.Contains(allowed, modelName) {
|
||||
return true
|
||||
}
|
||||
aiError(c, http.StatusNotFound, "model_not_found", "模型不存在或无权访问: "+modelName)
|
||||
return false
|
||||
}
|
||||
|
||||
// logEntry 组装调用日志骨架;usage / status 由调用方补齐。
|
||||
func (h *aiGatewayHandler) logEntry(c *gin.Context, endpoint, modelName string, stream bool, meta service.ChatMeta, start time.Time) model.AiCallLog {
|
||||
entry := model.AiCallLog{
|
||||
@@ -102,21 +123,6 @@ func (h *aiGatewayHandler) logEntry(c *gin.Context, endpoint, modelName string,
|
||||
}
|
||||
|
||||
// validateIR 做协议无关的入参检查(模型名、消息、不支持的内容块)。
|
||||
func validateIR(ir aiwire.ChatRequest) error {
|
||||
if strings.TrimSpace(ir.Model) == "" {
|
||||
return fmt.Errorf("model 不能为空")
|
||||
}
|
||||
if len(ir.Messages) == 0 {
|
||||
return fmt.Errorf("messages 不能为空")
|
||||
}
|
||||
for _, m := range ir.Messages {
|
||||
if m.Content.HasUnsupported() {
|
||||
return service.ErrAiUnsupportedBlock
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// maybeLogContent 在调用密钥显式开启内容日志且未过期时写正文(红线例外);
|
||||
// respBody 为 nil 时只记请求(流式响应与向量结果不记录);callLogID 关联同次调用日志。
|
||||
func (h *aiGatewayHandler) maybeLogContent(c *gin.Context, callLogID uint, endpoint, modelName string, stream bool, reqBody, respBody any) {
|
||||
@@ -147,47 +153,6 @@ func (h *aiGatewayHandler) logFailure(c *gin.Context, entry model.AiCallLog, req
|
||||
h.maybeLogContent(c, callID, entry.Endpoint, entry.Model, entry.Stream, reqBody, nil)
|
||||
}
|
||||
|
||||
// chatCompletions 是 OpenAI /ai/v1/chat/completions 端点。
|
||||
//
|
||||
// @Summary OpenAI 兼容对话补全
|
||||
// @Tags AI 网关
|
||||
// @Param body body object true "OpenAI chat/completions 请求体(支持 stream)"
|
||||
// @Success 200 {object} map[string]any "OpenAI 兼容响应(流式为 SSE)"
|
||||
// @Router /ai/v1/chat/completions [post]
|
||||
func (h *aiGatewayHandler) chatCompletions(c *gin.Context) {
|
||||
var ir aiwire.ChatRequest
|
||||
if err := c.ShouldBindJSON(&ir); err != nil {
|
||||
aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error())
|
||||
return
|
||||
}
|
||||
if err := validateIR(ir); err != nil {
|
||||
aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error())
|
||||
return
|
||||
}
|
||||
if ir.Stream {
|
||||
h.streamOpenAI(c, ir)
|
||||
return
|
||||
}
|
||||
start := time.Now()
|
||||
resp, meta, err := h.gw.Chat(c.Request.Context(), ir, keyGroup(c))
|
||||
entry := h.logEntry(c, "openai", ir.Model, false, meta, start)
|
||||
if err != nil {
|
||||
upstreamError(c, err)
|
||||
entry.ErrMsg = err.Error()
|
||||
h.logFailure(c, entry, ir)
|
||||
return
|
||||
}
|
||||
resp.ID = aiRandID("chatcmpl-")
|
||||
if resp.Created == 0 {
|
||||
resp.Created = time.Now().Unix()
|
||||
}
|
||||
entry.Status = http.StatusOK
|
||||
fillUsage(&entry, resp.Usage)
|
||||
callID := h.gw.LogCall(entry)
|
||||
h.maybeLogContent(c, callID, "openai", ir.Model, false, ir, resp)
|
||||
c.JSON(http.StatusOK, resp)
|
||||
}
|
||||
|
||||
func fillUsage(entry *model.AiCallLog, u *aiwire.Usage) {
|
||||
if u == nil {
|
||||
return
|
||||
@@ -217,6 +182,9 @@ func (h *aiGatewayHandler) embeddings(c *gin.Context) {
|
||||
aiError(c, http.StatusBadRequest, "invalid_request_error", "仅支持 encoding_format=float")
|
||||
return
|
||||
}
|
||||
if !checkKeyModel(c, req.Model) {
|
||||
return
|
||||
}
|
||||
start := time.Now()
|
||||
resp, meta, err := h.gw.Embeddings(c.Request.Context(), req, keyGroup(c))
|
||||
entry := h.logEntry(c, "embeddings", req.Model, false, meta, start)
|
||||
@@ -235,44 +203,6 @@ func (h *aiGatewayHandler) embeddings(c *gin.Context) {
|
||||
c.JSON(http.StatusOK, resp)
|
||||
}
|
||||
|
||||
// streamOpenAI 以 OpenAI SSE 直通流式响应,末尾发 [DONE]。
|
||||
func (h *aiGatewayHandler) streamOpenAI(c *gin.Context, ir aiwire.ChatRequest) {
|
||||
start := time.Now()
|
||||
stream, meta, err := h.gw.OpenStream(c.Request.Context(), ir, keyGroup(c))
|
||||
entry := h.logEntry(c, "openai", ir.Model, true, meta, start)
|
||||
if err != nil {
|
||||
upstreamError(c, err)
|
||||
entry.ErrMsg = err.Error()
|
||||
h.logFailure(c, entry, ir)
|
||||
return
|
||||
}
|
||||
defer stream.Close()
|
||||
sseHeaders(c)
|
||||
id, created := aiRandID("chatcmpl-"), time.Now().Unix()
|
||||
var usage *aiwire.Usage
|
||||
for {
|
||||
chunk, err := stream.Next()
|
||||
if err != nil {
|
||||
if !errors.Is(err, io.EOF) {
|
||||
entry.ErrMsg = err.Error()
|
||||
}
|
||||
break
|
||||
}
|
||||
chunk.ID, chunk.Created = id, created
|
||||
if chunk.Usage != nil {
|
||||
usage = chunk.Usage
|
||||
}
|
||||
writeSSEData(c, chunk)
|
||||
}
|
||||
c.Writer.WriteString("data: [DONE]\n\n")
|
||||
c.Writer.Flush()
|
||||
entry.Status = http.StatusOK
|
||||
entry.LatencyMs = time.Since(start).Milliseconds()
|
||||
fillUsage(&entry, usage)
|
||||
callID := h.gw.LogCall(entry)
|
||||
h.maybeLogContent(c, callID, "openai", ir.Model, true, ir, nil)
|
||||
}
|
||||
|
||||
func sseHeaders(c *gin.Context) {
|
||||
c.Header("Content-Type", "text/event-stream")
|
||||
c.Header("Cache-Control", "no-cache")
|
||||
@@ -280,22 +210,11 @@ func sseHeaders(c *gin.Context) {
|
||||
c.Writer.Flush()
|
||||
}
|
||||
|
||||
func writeSSEData(c *gin.Context, v any) {
|
||||
b, err := json.Marshal(v)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
c.Writer.WriteString("data: ")
|
||||
c.Writer.Write(b)
|
||||
c.Writer.WriteString("\n\n")
|
||||
c.Writer.Flush()
|
||||
}
|
||||
|
||||
// messages 是 Anthropic /ai/v1/messages 端点。
|
||||
//
|
||||
// @Summary Anthropic Messages 兼容端点
|
||||
// @Tags AI 网关
|
||||
// @Param body body object true "Anthropic messages 请求体(支持 stream)"
|
||||
// @Param body body object true "Anthropic messages 请求体(支持 stream;经 OCI OpenAI 兼容面直通)"
|
||||
// @Success 200 {object} map[string]any "Anthropic 兼容响应(流式为 SSE)"
|
||||
// @Router /ai/v1/messages [post]
|
||||
func (h *aiGatewayHandler) messages(c *gin.Context) {
|
||||
@@ -308,69 +227,92 @@ func (h *aiGatewayHandler) messages(c *gin.Context) {
|
||||
aiError(c, http.StatusBadRequest, "invalid_request_error", "max_tokens 必填且需大于 0")
|
||||
return
|
||||
}
|
||||
ir, err := service.AnthropicToIR(req)
|
||||
if len(req.Messages) == 0 {
|
||||
aiError(c, http.StatusBadRequest, "invalid_request_error", "messages 不能为空")
|
||||
return
|
||||
}
|
||||
if !checkKeyModel(c, req.Model) {
|
||||
return
|
||||
}
|
||||
body, err := service.AnthropicToResponsesBody(req)
|
||||
if err != nil {
|
||||
aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error())
|
||||
return
|
||||
}
|
||||
if err := validateIR(ir); err != nil {
|
||||
aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error())
|
||||
return
|
||||
}
|
||||
if req.Stream {
|
||||
h.streamAnthropic(c, ir)
|
||||
h.streamAnthropic(c, body, req)
|
||||
return
|
||||
}
|
||||
start := time.Now()
|
||||
resp, meta, err := h.gw.Chat(c.Request.Context(), ir, keyGroup(c))
|
||||
entry := h.logEntry(c, "anthropic", ir.Model, false, meta, start)
|
||||
payload, meta, err := h.gw.RespPassthrough(c.Request.Context(), body, req.Model, keyGroup(c))
|
||||
entry := h.logEntry(c, "anthropic", req.Model, false, meta, start)
|
||||
if err != nil {
|
||||
upstreamError(c, err)
|
||||
entry.ErrMsg = err.Error()
|
||||
h.logFailure(c, entry, req)
|
||||
return
|
||||
}
|
||||
out, err := service.ResponsesToAnthropic(payload, aiRandID("msg_"))
|
||||
if err != nil {
|
||||
aiError(c, http.StatusBadGateway, "api_error", err.Error())
|
||||
entry.ErrMsg = err.Error()
|
||||
h.logFailure(c, entry, req)
|
||||
return
|
||||
}
|
||||
entry.Status = http.StatusOK
|
||||
fillUsage(&entry, resp.Usage)
|
||||
fillUsage(&entry, service.RespPassthroughUsage(payload))
|
||||
callID := h.gw.LogCall(entry)
|
||||
out := service.IRRespToAnthropic(resp, aiRandID("msg_"))
|
||||
h.maybeLogContent(c, callID, "anthropic", ir.Model, false, req, out)
|
||||
h.maybeLogContent(c, callID, "anthropic", req.Model, false, req, out)
|
||||
c.JSON(http.StatusOK, out)
|
||||
}
|
||||
|
||||
// streamAnthropic 经状态机把 IR chunk 流聚合为 Anthropic SSE 事件流。
|
||||
func (h *aiGatewayHandler) streamAnthropic(c *gin.Context, ir aiwire.ChatRequest) {
|
||||
// streamAnthropic 流式直通并桥接:上游 Responses SSE 事件经 AnthRespBridge
|
||||
// 转为 Anthropic 事件序列逐块写出;usage 从 completed 事件记账。
|
||||
func (h *aiGatewayHandler) streamAnthropic(c *gin.Context, body []byte, req aiwire.MessagesRequest) {
|
||||
start := time.Now()
|
||||
stream, meta, err := h.gw.OpenStream(c.Request.Context(), ir, keyGroup(c))
|
||||
entry := h.logEntry(c, "anthropic", ir.Model, true, meta, start)
|
||||
upstream, meta, err := h.gw.RespPassthroughStream(c.Request.Context(), body, req.Model, keyGroup(c))
|
||||
entry := h.logEntry(c, "anthropic", req.Model, true, meta, start)
|
||||
if err != nil {
|
||||
upstreamError(c, err)
|
||||
entry.ErrMsg = err.Error()
|
||||
h.logFailure(c, entry, ir)
|
||||
h.logFailure(c, entry, req)
|
||||
return
|
||||
}
|
||||
defer stream.Close()
|
||||
defer upstream.Close()
|
||||
sseHeaders(c)
|
||||
st := service.NewAnthStream(aiRandID("msg_"), ir.Model)
|
||||
for {
|
||||
chunk, err := stream.Next()
|
||||
if err != nil {
|
||||
if !errors.Is(err, io.EOF) {
|
||||
bridge := service.NewAnthRespBridge(aiRandID("msg_"), req.Model)
|
||||
if err := forwardAnthSSE(c, upstream, bridge); err != nil {
|
||||
entry.ErrMsg = err.Error()
|
||||
}
|
||||
break
|
||||
}
|
||||
writeAnthEvents(c, st.Feed(chunk))
|
||||
}
|
||||
writeAnthEvents(c, st.Finish())
|
||||
writeAnthEvents(c, bridge.Finish())
|
||||
entry.Status = http.StatusOK
|
||||
entry.LatencyMs = time.Since(start).Milliseconds()
|
||||
usage := st.Usage()
|
||||
usage := bridge.Usage()
|
||||
entry.PromptTokens, entry.CompletionTokens = usage.InputTokens, usage.OutputTokens
|
||||
entry.TotalTokens = usage.InputTokens + usage.OutputTokens
|
||||
entry.CachedTokens = usage.CacheReadInputTokens
|
||||
callID := h.gw.LogCall(entry)
|
||||
h.maybeLogContent(c, callID, "anthropic", ir.Model, true, ir, nil)
|
||||
h.maybeLogContent(c, callID, "anthropic", req.Model, true, req, nil)
|
||||
}
|
||||
|
||||
// forwardAnthSSE 逐行读上游 SSE,把 data 行喂给桥并即时写出转换后的事件。
|
||||
func forwardAnthSSE(c *gin.Context, upstream io.Reader, bridge *service.AnthRespBridge) error {
|
||||
reader := bufio.NewReader(upstream)
|
||||
for {
|
||||
line, err := reader.ReadBytes('\n')
|
||||
if len(line) > 0 {
|
||||
trimmed := bytes.TrimSpace(line)
|
||||
if data, ok := bytes.CutPrefix(trimmed, []byte("data: ")); ok {
|
||||
writeAnthEvents(c, bridge.Feed(data))
|
||||
}
|
||||
}
|
||||
if err != nil {
|
||||
if errors.Is(err, io.EOF) {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func writeAnthEvents(c *gin.Context, events []service.AnthEvent) {
|
||||
@@ -398,6 +340,15 @@ func (h *aiGatewayHandler) listModels(c *gin.Context) {
|
||||
aiError(c, http.StatusInternalServerError, "api_error", "查询模型列表失败")
|
||||
return
|
||||
}
|
||||
if allowed := keyModels(c); len(allowed) > 0 {
|
||||
kept := make([]aiwire.Model, 0, len(list.Data))
|
||||
for _, m := range list.Data {
|
||||
if slices.Contains(allowed, m.ID) {
|
||||
kept = append(kept, m)
|
||||
}
|
||||
}
|
||||
list.Data = kept
|
||||
}
|
||||
c.JSON(http.StatusOK, list)
|
||||
}
|
||||
|
||||
@@ -405,31 +356,45 @@ func (h *aiGatewayHandler) listModels(c *gin.Context) {
|
||||
//
|
||||
// @Summary OpenAI Responses 兼容端点
|
||||
// @Tags AI 网关
|
||||
// @Param body body object true "OpenAI responses 请求体(支持 stream)"
|
||||
// @Param body body object true "OpenAI responses 请求体(支持 stream;web_search/x_search 服务端工具仅非流式)"
|
||||
// @Success 200 {object} map[string]any "OpenAI 兼容响应(流式为 SSE)"
|
||||
// @Router /ai/v1/responses [post]
|
||||
func (h *aiGatewayHandler) responses(c *gin.Context) {
|
||||
var req aiwire.RespRequest
|
||||
if err := c.ShouldBindJSON(&req); err != nil {
|
||||
aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error())
|
||||
return
|
||||
}
|
||||
ir, err := service.ResponsesToIR(req)
|
||||
raw, err := c.GetRawData()
|
||||
if err != nil {
|
||||
aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error())
|
||||
return
|
||||
}
|
||||
if err := validateIR(ir); err != nil {
|
||||
var req aiwire.RespRequest
|
||||
if err := json.Unmarshal(raw, &req); err != nil {
|
||||
aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error())
|
||||
return
|
||||
}
|
||||
if !checkKeyModel(c, req.Model) {
|
||||
return
|
||||
}
|
||||
h.responsesPassthrough(c, raw, req)
|
||||
}
|
||||
|
||||
// responsesPassthrough 直通链路(唯一上游):原始 body 改写后直达
|
||||
// OCI /actions/v1/responses,响应原样透传。
|
||||
func (h *aiGatewayHandler) responsesPassthrough(c *gin.Context, raw []byte, req aiwire.RespRequest) {
|
||||
if err := service.RespPassthroughValidate(req); err != nil {
|
||||
aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error())
|
||||
return
|
||||
}
|
||||
body, err := service.RespPassthroughBody(raw)
|
||||
if err != nil {
|
||||
aiError(c, http.StatusBadRequest, "invalid_request_error", err.Error())
|
||||
return
|
||||
}
|
||||
if req.Stream {
|
||||
h.streamResponses(c, ir)
|
||||
h.responsesPassthroughStream(c, body, req)
|
||||
return
|
||||
}
|
||||
start := time.Now()
|
||||
resp, meta, err := h.gw.Chat(c.Request.Context(), ir, keyGroup(c))
|
||||
entry := h.logEntry(c, "responses", ir.Model, false, meta, start)
|
||||
payload, meta, err := h.gw.RespPassthrough(c.Request.Context(), body, req.Model, keyGroup(c))
|
||||
entry := h.logEntry(c, "responses", req.Model, false, meta, start)
|
||||
if err != nil {
|
||||
upstreamError(c, err)
|
||||
entry.ErrMsg = err.Error()
|
||||
@@ -437,54 +402,61 @@ func (h *aiGatewayHandler) responses(c *gin.Context) {
|
||||
return
|
||||
}
|
||||
entry.Status = http.StatusOK
|
||||
fillUsage(&entry, resp.Usage)
|
||||
fillUsage(&entry, service.RespPassthroughUsage(payload))
|
||||
callID := h.gw.LogCall(entry)
|
||||
out := service.IRRespToResponses(resp, aiRandID("resp_"), time.Now().Unix())
|
||||
h.maybeLogContent(c, callID, "responses", ir.Model, false, req, out)
|
||||
c.JSON(http.StatusOK, out)
|
||||
h.maybeLogContent(c, callID, "responses", req.Model, false, req, json.RawMessage(payload))
|
||||
c.Data(http.StatusOK, "application/json; charset=utf-8", payload)
|
||||
}
|
||||
|
||||
// streamResponses 经状态机把 IR chunk 流聚合为 Responses 语义事件流。
|
||||
func (h *aiGatewayHandler) streamResponses(c *gin.Context, ir aiwire.ChatRequest) {
|
||||
// responsesPassthroughStream 流式直通:SSE 事件原样转发(推理增量等直达客户端),
|
||||
// 逐行扫描 completed 事件提取 usage 记账。
|
||||
func (h *aiGatewayHandler) responsesPassthroughStream(c *gin.Context, body []byte, req aiwire.RespRequest) {
|
||||
start := time.Now()
|
||||
stream, meta, err := h.gw.OpenStream(c.Request.Context(), ir, keyGroup(c))
|
||||
entry := h.logEntry(c, "responses", ir.Model, true, meta, start)
|
||||
upstream, meta, err := h.gw.RespPassthroughStream(c.Request.Context(), body, req.Model, keyGroup(c))
|
||||
entry := h.logEntry(c, "responses", req.Model, true, meta, start)
|
||||
if err != nil {
|
||||
upstreamError(c, err)
|
||||
entry.ErrMsg = err.Error()
|
||||
h.logFailure(c, entry, ir)
|
||||
h.logFailure(c, entry, req)
|
||||
return
|
||||
}
|
||||
defer stream.Close()
|
||||
defer upstream.Close()
|
||||
sseHeaders(c)
|
||||
st := service.NewRespStream(aiRandID("resp_"), ir.Model, time.Now().Unix())
|
||||
for {
|
||||
chunk, err := stream.Next()
|
||||
usage, err := forwardSSE(c, upstream)
|
||||
if err != nil {
|
||||
if !errors.Is(err, io.EOF) {
|
||||
entry.ErrMsg = err.Error()
|
||||
}
|
||||
break
|
||||
}
|
||||
writeRespEvents(c, st.Feed(chunk))
|
||||
}
|
||||
writeRespEvents(c, st.Finish())
|
||||
entry.Status = http.StatusOK
|
||||
entry.LatencyMs = time.Since(start).Milliseconds()
|
||||
fillUsage(&entry, st.Usage())
|
||||
fillUsage(&entry, usage)
|
||||
callID := h.gw.LogCall(entry)
|
||||
h.maybeLogContent(c, callID, "responses", ir.Model, true, ir, nil)
|
||||
h.maybeLogContent(c, callID, "responses", req.Model, true, req, nil)
|
||||
}
|
||||
|
||||
func writeRespEvents(c *gin.Context, events []service.RespEvent) {
|
||||
for _, ev := range events {
|
||||
b, err := json.Marshal(ev.Data)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
c.Writer.WriteString("event: " + ev.Event + "\ndata: ")
|
||||
c.Writer.Write(b)
|
||||
c.Writer.WriteString("\n\n")
|
||||
}
|
||||
// forwardSSE 把上游 SSE 逐行转发给客户端,空行(事件边界)即 flush;
|
||||
// 顺带从 data 行提取 response.completed 的 usage。
|
||||
func forwardSSE(c *gin.Context, upstream io.Reader) (*aiwire.Usage, error) {
|
||||
reader := bufio.NewReader(upstream)
|
||||
var usage *aiwire.Usage
|
||||
for {
|
||||
line, err := reader.ReadBytes('\n')
|
||||
if len(line) > 0 {
|
||||
c.Writer.Write(line)
|
||||
trimmed := bytes.TrimSpace(line)
|
||||
if len(trimmed) == 0 {
|
||||
c.Writer.Flush()
|
||||
} else if data, ok := bytes.CutPrefix(trimmed, []byte("data: ")); ok {
|
||||
if u := service.RespStreamCompletedUsage(data); u != nil {
|
||||
usage = u
|
||||
}
|
||||
}
|
||||
}
|
||||
if err != nil {
|
||||
c.Writer.Flush()
|
||||
if errors.Is(err, io.EOF) {
|
||||
return usage, nil
|
||||
}
|
||||
return usage, err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,206 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gorm.io/gorm"
|
||||
|
||||
"oci-portal/internal/model"
|
||||
)
|
||||
|
||||
// seedGatewayModel 直插启用渠道与模型缓存,绕过云同步。
|
||||
func seedGatewayModel(t *testing.T, db *gorm.DB, name string) {
|
||||
t.Helper()
|
||||
ch := &model.AiChannel{Name: "t-" + name, OciConfigID: 1, Region: "r-" + name, Enabled: true, Priority: 1, Weight: 1}
|
||||
if err := db.Create(ch).Error; err != nil {
|
||||
t.Fatalf("seed channel: %v", err)
|
||||
}
|
||||
cache := &model.AiModelCache{ChannelID: ch.ID, ModelOcid: "ocid1.." + name, Name: name, Vendor: "v", SyncedAt: time.Now()}
|
||||
if err := db.Create(cache).Error; err != nil {
|
||||
t.Fatalf("seed model cache: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAiGatewayKeyModelRestrict(t *testing.T) {
|
||||
r, auth, _, db := newTestRouterDB(t)
|
||||
token, _, err := auth.Login(context.Background(), "admin", "pass123", "127.0.0.1", "")
|
||||
if err != nil {
|
||||
t.Fatalf("login: %v", err)
|
||||
}
|
||||
|
||||
// 受限密钥:白名单含脏输入,入库应规范化为 2 项(cohere + ghost)
|
||||
w := doRequest(t, r, http.MethodPost, "/api/v1/ai-keys", token,
|
||||
`{"name":"limited","value":"limited-key-1234","models":[" cohere.command-a-03-2025 ","cohere.command-a-03-2025","","ghost-model"]}`)
|
||||
if w.Code != http.StatusCreated {
|
||||
t.Fatalf("创建受限密钥 status = %d, body %s", w.Code, w.Body.String())
|
||||
}
|
||||
var created struct {
|
||||
Item model.AiKey `json:"item"`
|
||||
}
|
||||
if err := json.Unmarshal(w.Body.Bytes(), &created); err != nil {
|
||||
t.Fatalf("decode: %v", err)
|
||||
}
|
||||
if len(created.Item.Models) != 2 || created.Item.Models[0] != "cohere.command-a-03-2025" {
|
||||
t.Fatalf("创建后 models 未规范化: %v", created.Item.Models)
|
||||
}
|
||||
// 不限密钥对照
|
||||
w = doRequest(t, r, http.MethodPost, "/api/v1/ai-keys", token, `{"name":"open","value":"open-key-12345"}`)
|
||||
if w.Code != http.StatusCreated {
|
||||
t.Fatalf("创建不限密钥 status = %d, body %s", w.Code, w.Body.String())
|
||||
}
|
||||
|
||||
seedGatewayModel(t, db, "cohere.command-a-03-2025")
|
||||
seedGatewayModel(t, db, "meta.llama-3.3-70b-instruct")
|
||||
|
||||
deny := []string{"model_not_found", "无权访问"}
|
||||
pass := []string{"未知模型"} // 穿过白名单闸门,由编排层以「未知模型」拒绝(池内无 ghost-model)
|
||||
tests := []struct {
|
||||
name string
|
||||
key string
|
||||
path string
|
||||
body string
|
||||
wantCode int
|
||||
wantSub []string
|
||||
}{
|
||||
{"responses 流式同样拦截", "limited-key-1234", "/ai/v1/responses",
|
||||
`{"model":"meta.llama-3.3-70b-instruct","stream":true,"input":"hi"}`, 404, deny},
|
||||
{"responses 白名单内穿透", "limited-key-1234", "/ai/v1/responses",
|
||||
`{"model":"ghost-model","input":"hi"}`, 404, pass},
|
||||
{"embeddings 白名单外拦截", "limited-key-1234", "/ai/v1/embeddings",
|
||||
`{"model":"meta.llama-3.3-70b-instruct","input":["hi"]}`, 404, deny},
|
||||
{"messages 白名单外拦截为 Anthropic 体", "limited-key-1234", "/ai/v1/messages",
|
||||
`{"model":"meta.llama-3.3-70b-instruct","max_tokens":16,"messages":[{"role":"user","content":"hi"}]}`, 404,
|
||||
[]string{`"type":"error"`, "model_not_found", "无权访问"}},
|
||||
{"responses 白名单外拦截", "limited-key-1234", "/ai/v1/responses",
|
||||
`{"model":"meta.llama-3.3-70b-instruct","input":"hi"}`, 404, deny},
|
||||
{"不限密钥穿透闸门", "open-key-12345", "/ai/v1/responses",
|
||||
`{"model":"ghost-model","input":"hi"}`, 404, pass},
|
||||
{"chat completions 端点已删除", "open-key-12345", "/ai/v1/chat/completions",
|
||||
`{"model":"ghost-model","messages":[{"role":"user","content":"hi"}]}`, 404, []string{"not found"}},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
w := doRequest(t, r, http.MethodPost, tt.path, tt.key, tt.body)
|
||||
if w.Code != tt.wantCode {
|
||||
t.Fatalf("status = %d, want %d, body %s", w.Code, tt.wantCode, w.Body.String())
|
||||
}
|
||||
for _, sub := range tt.wantSub {
|
||||
if !strings.Contains(w.Body.String(), sub) {
|
||||
t.Errorf("body 缺少 %q: %s", sub, w.Body.String())
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAiGatewayModelsFilteredByKey(t *testing.T) {
|
||||
r, auth, _, db := newTestRouterDB(t)
|
||||
token, _, err := auth.Login(context.Background(), "admin", "pass123", "127.0.0.1", "")
|
||||
if err != nil {
|
||||
t.Fatalf("login: %v", err)
|
||||
}
|
||||
seedGatewayModel(t, db, "cohere.command-a-03-2025")
|
||||
seedGatewayModel(t, db, "meta.llama-3.3-70b-instruct")
|
||||
w := doRequest(t, r, http.MethodPost, "/api/v1/ai-keys", token,
|
||||
`{"name":"limited","value":"limited-key-1234","models":["cohere.command-a-03-2025"]}`)
|
||||
if w.Code != http.StatusCreated {
|
||||
t.Fatalf("创建密钥 status = %d", w.Code)
|
||||
}
|
||||
w = doRequest(t, r, http.MethodPost, "/api/v1/ai-keys", token, `{"name":"open","value":"open-key-12345"}`)
|
||||
if w.Code != http.StatusCreated {
|
||||
t.Fatalf("创建密钥 status = %d", w.Code)
|
||||
}
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
key string
|
||||
want []string
|
||||
notWant []string
|
||||
}{
|
||||
{"受限密钥仅见交集", "limited-key-1234", []string{"cohere.command-a-03-2025"}, []string{"meta.llama-3.3-70b-instruct"}},
|
||||
{"不限密钥全量可见", "open-key-12345", []string{"cohere.command-a-03-2025", "meta.llama-3.3-70b-instruct"}, nil},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
w := doRequest(t, r, http.MethodGet, "/ai/v1/models", tt.key, "")
|
||||
if w.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, body %s", w.Code, w.Body.String())
|
||||
}
|
||||
for _, sub := range tt.want {
|
||||
if !strings.Contains(w.Body.String(), sub) {
|
||||
t.Errorf("应包含 %q: %s", sub, w.Body.String())
|
||||
}
|
||||
}
|
||||
for _, sub := range tt.notWant {
|
||||
if strings.Contains(w.Body.String(), sub) {
|
||||
t.Errorf("不应包含 %q: %s", sub, w.Body.String())
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAiKeyModelsUpdateRoundTrip(t *testing.T) {
|
||||
r, auth, _, _ := newTestRouterDB(t)
|
||||
token, _, err := auth.Login(context.Background(), "admin", "pass123", "127.0.0.1", "")
|
||||
if err != nil {
|
||||
t.Fatalf("login: %v", err)
|
||||
}
|
||||
w := doRequest(t, r, http.MethodPost, "/api/v1/ai-keys", token, `{"name":"k","value":"round-key-1234"}`)
|
||||
if w.Code != http.StatusCreated {
|
||||
t.Fatalf("创建 status = %d", w.Code)
|
||||
}
|
||||
var created struct {
|
||||
Item model.AiKey `json:"item"`
|
||||
}
|
||||
_ = json.Unmarshal(w.Body.Bytes(), &created)
|
||||
id := created.Item.ID
|
||||
|
||||
listModels := func() []string {
|
||||
w := doRequest(t, r, http.MethodGet, "/api/v1/ai-keys", token, "")
|
||||
var resp struct {
|
||||
Items []model.AiKey `json:"items"`
|
||||
}
|
||||
_ = json.Unmarshal(w.Body.Bytes(), &resp)
|
||||
for _, k := range resp.Items {
|
||||
if k.ID == id {
|
||||
return k.Models
|
||||
}
|
||||
}
|
||||
t.Fatalf("密钥 %d 不在列表中", id)
|
||||
return nil
|
||||
}
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
body string
|
||||
want []string
|
||||
}{
|
||||
{"设置白名单", `{"models":[" a-model ","a-model","b-model"]}`, []string{"a-model", "b-model"}},
|
||||
{"不传 models 保持不变", `{"name":"k2"}`, []string{"a-model", "b-model"}},
|
||||
{"空数组清空恢复不限", `{"models":[]}`, nil},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
w := doRequest(t, r, http.MethodPut, "/api/v1/ai-keys/"+strconv.Itoa(int(id)), token, tt.body)
|
||||
if w.Code != http.StatusNoContent {
|
||||
t.Fatalf("update status = %d, body %s", w.Code, w.Body.String())
|
||||
}
|
||||
got := listModels()
|
||||
if len(got) != len(tt.want) {
|
||||
t.Fatalf("models = %v, want %v", got, tt.want)
|
||||
}
|
||||
for i := range tt.want {
|
||||
if got[i] != tt.want[i] {
|
||||
t.Fatalf("models = %v, want %v", got, tt.want)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -57,6 +57,13 @@ func (nullClient) GetSubscription(context.Context, oci.Credentials, string) (oci
|
||||
}
|
||||
|
||||
func newTestRouter(t *testing.T) (*gin.Engine, *service.AuthService, *service.SystemLogService) {
|
||||
t.Helper()
|
||||
r, auth, systemLogs, _ := newTestRouterDB(t)
|
||||
return r, auth, systemLogs
|
||||
}
|
||||
|
||||
// newTestRouterDB 额外回传 db 句柄,供需要直插数据(如网关模型缓存)的测试使用。
|
||||
func newTestRouterDB(t *testing.T) (*gin.Engine, *service.AuthService, *service.SystemLogService, *gorm.DB) {
|
||||
t.Helper()
|
||||
gin.SetMode(gin.TestMode)
|
||||
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{
|
||||
@@ -90,7 +97,7 @@ func newTestRouter(t *testing.T) (*gin.Engine, *service.AuthService, *service.Sy
|
||||
logEvents := service.NewLogEventService(db)
|
||||
oauth := service.NewOAuthService(db, settings, auth)
|
||||
r := NewRouter(auth, oauth, ociConfigs, tasks, service.NewConsoleService(ociConfigs), settings, notifier, systemLogs, logEvents, service.NewProxyService(db, cipher), service.NewAiGatewayService(db, ociConfigs, nullClient{}))
|
||||
return r, auth, systemLogs
|
||||
return r, auth, systemLogs, db
|
||||
}
|
||||
|
||||
func doRequest(t *testing.T, r *gin.Engine, method, path, token, body string) *httptest.ResponseRecorder {
|
||||
|
||||
@@ -11,7 +11,6 @@ import (
|
||||
func registerAiGateway(r *gin.Engine, aiGateway *service.AiGatewayService) {
|
||||
aih := &aiGatewayHandler{gw: aiGateway}
|
||||
ai := r.Group("/ai/v1", bodyLimit(10<<20), aih.auth)
|
||||
ai.POST("/chat/completions", aih.chatCompletions)
|
||||
ai.POST("/responses", aih.responses)
|
||||
ai.POST("/messages", aih.messages)
|
||||
ai.POST("/embeddings", aih.embeddings)
|
||||
@@ -33,6 +32,9 @@ func registerAiAdmin(secured *gin.RouterGroup, aiGateway *service.AiGatewayServi
|
||||
secured.POST("/ai-channels/:id/probe", aiadmin.probeChannel)
|
||||
secured.POST("/ai-channels/:id/sync-models", aiadmin.syncChannelModels)
|
||||
secured.GET("/ai-models", aiadmin.gatewayModels)
|
||||
secured.GET("/ai-blacklist", aiadmin.listBlacklist)
|
||||
secured.POST("/ai-blacklist", aiadmin.addBlacklist)
|
||||
secured.DELETE("/ai-blacklist/:id", aiadmin.removeBlacklist)
|
||||
secured.GET("/ai-logs", aiadmin.listLogs)
|
||||
secured.GET("/ai-content-logs", aiadmin.listContentLogs)
|
||||
}
|
||||
|
||||
@@ -68,7 +68,7 @@ func autoMigrate(db *gorm.DB) error {
|
||||
&model.CheckSnapshot{}, &model.CostSnapshot{},
|
||||
&model.RegionCache{}, &model.CompartmentCache{}, &model.Setting{},
|
||||
&model.SystemLog{}, &model.LogEvent{}, &model.AlertRule{}, &model.AlertRuleHit{}, &model.Proxy{},
|
||||
&model.AiKey{}, &model.AiChannel{}, &model.AiModelCache{}, &model.AiCallLog{},
|
||||
&model.AiKey{}, &model.AiChannel{}, &model.AiModelCache{}, &model.AiModelBlacklist{}, &model.AiCallLog{},
|
||||
&model.AiContentLog{},
|
||||
)
|
||||
}
|
||||
|
||||
@@ -266,6 +266,8 @@ type AiKey struct {
|
||||
KeyHash string `gorm:"uniqueIndex;size:64" json:"-"`
|
||||
Tail string `gorm:"size:4" json:"tail"`
|
||||
Group string `gorm:"column:key_group;size:32" json:"group"`
|
||||
// Models 是模型白名单(精确匹配模型名);空 = 不限,非空时网关仅放行列表内模型
|
||||
Models []string `gorm:"serializer:json;type:text" json:"models"`
|
||||
Enabled bool `json:"enabled"`
|
||||
LastUsedAt *time.Time `json:"lastUsedAt"`
|
||||
// ContentLogUntil 是内容日志开启截止时间;nil = 永关(默认,红线),
|
||||
@@ -309,13 +311,14 @@ type AiModelCache struct {
|
||||
// 弃用后仍可调用,直到 RetiredAt(按需推理退役,同步层已剔除过期项)。
|
||||
DeprecatedAt *time.Time `json:"deprecatedAt"`
|
||||
RetiredAt *time.Time `json:"retiredAt"`
|
||||
// Unusable 标记该模型在此渠道不可按需调用(微调基座/区域未开放按需),
|
||||
// 由探测、真实调用或同步后的后台验证习得:不参与网关列表/路由/探测候选;
|
||||
// 任何同步都按 OCID 结转标记,已标记模型定期复检、恢复供给自动解除。
|
||||
Unusable bool `json:"unusable"`
|
||||
UnusableReason string `gorm:"size:200" json:"unusableReason,omitempty"`
|
||||
// CheckedAt 是最近一次可用性验证时间;NULL 表示从未验证(同步后的后台验证以此为队列)。
|
||||
CheckedAt *time.Time `json:"checkedAt"`
|
||||
}
|
||||
|
||||
// AiModelBlacklist 是全局模型黑名单(按模型名):加入即从全部渠道缓存删除该模型,
|
||||
// 同步与探测过滤不再入库;适用于微调基座等实际不可按需调用的模型,由用户手动维护。
|
||||
type AiModelBlacklist struct {
|
||||
ID uint `gorm:"primaryKey" json:"id"`
|
||||
Name string `gorm:"size:96;uniqueIndex" json:"name"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
}
|
||||
|
||||
// AiCallLog 是网关调用日志:仅元数据与 token 用量,绝不记录 prompt / 响应正文。
|
||||
|
||||
@@ -6,6 +6,7 @@ import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
@@ -99,10 +100,12 @@ type Client interface {
|
||||
AddVnicIpv6(ctx context.Context, cred Credentials, region, vnicID, address string) (string, error)
|
||||
// GenAI 网关:区域模型列表、聊天(非流式/流式)、配额探测、文本向量化。
|
||||
ListGenAiModels(ctx context.Context, cred Credentials, region string) ([]GenAiModel, error)
|
||||
GenAiChat(ctx context.Context, cred Credentials, region, modelOcid string, ir aiwire.ChatRequest) (*aiwire.ChatResponse, error)
|
||||
GenAiChatStream(ctx context.Context, cred Credentials, region, modelOcid string, ir aiwire.ChatRequest) (GenAiStream, error)
|
||||
GenAiProbeChat(ctx context.Context, cred Credentials, region, modelOcid, modelName string) (int, error)
|
||||
GenAiEmbed(ctx context.Context, cred Credentials, region, modelOcid string, inputs []string, dimensions *int) ([][]float32, *aiwire.Usage, error)
|
||||
// GenAiCompatResponses 直通 OpenAI Responses 请求体到 /actions/v1/responses(xAI 服务端工具通路)。
|
||||
GenAiCompatResponses(ctx context.Context, cred Credentials, region string, body []byte) ([]byte, error)
|
||||
// GenAiCompatResponsesStream 流式直通 /actions/v1/responses,建立成功返回 SSE body。
|
||||
GenAiCompatResponsesStream(ctx context.Context, cred Credentials, region string, body []byte) (io.ReadCloser, error)
|
||||
// 控制台连接:创建(VNC/串口连接串)、列出、删除。
|
||||
CreateConsoleConnection(ctx context.Context, cred Credentials, region, instanceID, sshPublicKey string) (ConsoleConnection, error)
|
||||
ListConsoleConnections(ctx context.Context, cred Credentials, region, instanceID string) ([]ConsoleConnection, error)
|
||||
|
||||
+28
-121
@@ -2,13 +2,11 @@ package oci
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/oracle/oci-go-sdk/v65/common"
|
||||
"github.com/oracle/oci-go-sdk/v65/generativeai"
|
||||
"github.com/oracle/oci-go-sdk/v65/generativeaiinference"
|
||||
|
||||
@@ -30,12 +28,6 @@ type GenAiModel struct {
|
||||
Retired *time.Time `json:"retired"`
|
||||
}
|
||||
|
||||
// GenAiStream 逐事件读取流式聊天;Next 在流结束时返回 io.EOF。
|
||||
type GenAiStream interface {
|
||||
Next() (aiwire.ChatChunk, error)
|
||||
Close() error
|
||||
}
|
||||
|
||||
func (c *RealClient) genAiClient(cred Credentials, region string) (generativeai.GenerativeAiClient, error) {
|
||||
gc, err := generativeai.NewGenerativeAiClientWithConfigurationProvider(provider(cred))
|
||||
if err != nil {
|
||||
@@ -152,121 +144,15 @@ func capStrings(caps []generativeai.ModelCapabilityEnum) []string {
|
||||
return out
|
||||
}
|
||||
|
||||
// GenAiChat 实现 Client:非流式聊天;modelOcid 走 on-demand serving,
|
||||
// cohere.* 模型走 COHERE 请求格式,其余走 GENERIC。
|
||||
func (c *RealClient) GenAiChat(ctx context.Context, cred Credentials, region, modelOcid string, ir aiwire.ChatRequest) (*aiwire.ChatResponse, error) {
|
||||
ic, err := c.genAiInferenceClient(cred, region)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req, err := buildChatRequest(ir, false)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
resp, err := ic.Chat(ctx, chatRequest(cred, modelOcid, req))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("genai chat: %w", err)
|
||||
}
|
||||
return chatResponseToIR(resp.ChatResult.ChatResponse, ir.Model)
|
||||
}
|
||||
|
||||
// buildChatRequest 按模型 vendor 组装底层请求体。
|
||||
func buildChatRequest(ir aiwire.ChatRequest, stream bool) (generativeaiinference.BaseChatRequest, error) {
|
||||
if isCohereModel(ir.Model) {
|
||||
return irToCohereSDK(ir, stream)
|
||||
}
|
||||
return irToSDK(ir, stream), nil
|
||||
}
|
||||
|
||||
// chatResponseToIR 按响应实际形态(GENERIC / COHERE)转回 IR。
|
||||
func chatResponseToIR(resp generativeaiinference.BaseChatResponse, model string) (*aiwire.ChatResponse, error) {
|
||||
switch r := resp.(type) {
|
||||
case generativeaiinference.GenericChatResponse:
|
||||
return sdkToIR(r, model), nil
|
||||
case generativeaiinference.CohereChatResponse:
|
||||
return cohereSDKToIR(r, model), nil
|
||||
default:
|
||||
return nil, fmt.Errorf("genai chat: unexpected response format %T", resp)
|
||||
}
|
||||
}
|
||||
|
||||
func chatRequest(cred Credentials, modelOcid string, req generativeaiinference.BaseChatRequest) generativeaiinference.ChatRequest {
|
||||
return generativeaiinference.ChatRequest{
|
||||
ChatDetails: generativeaiinference.ChatDetails{
|
||||
CompartmentId: &cred.TenancyOCID,
|
||||
ServingMode: generativeaiinference.OnDemandServingMode{ModelId: &modelOcid},
|
||||
ChatRequest: req,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// GenAiChatStream 实现 Client:流式聊天,返回逐事件读取器。
|
||||
// SDK 对 text/event-stream 跳过 unmarshal,原始流经 RawResponse 交由 SSEReader 消费。
|
||||
func (c *RealClient) GenAiChatStream(ctx context.Context, cred Credentials, region, modelOcid string, ir aiwire.ChatRequest) (GenAiStream, error) {
|
||||
ic, err := c.genAiInferenceClient(cred, region)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req, err := buildChatRequest(ir, true)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
resp, err := ic.Chat(ctx, chatRequest(cred, modelOcid, req))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("genai chat stream: %w", err)
|
||||
}
|
||||
reader, err := common.NewSSEReader(resp.RawResponse)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("genai chat stream: sse reader: %w", err)
|
||||
}
|
||||
return &genAiSSEStream{reader: reader, body: resp.RawResponse.Body, model: ir.Model}, nil
|
||||
}
|
||||
|
||||
// genAiSSEStream 把 OCI SSE 事件流适配为 IR chunk 流。
|
||||
type genAiSSEStream struct {
|
||||
reader *common.SseReader
|
||||
body io.ReadCloser
|
||||
model string
|
||||
}
|
||||
|
||||
// Next 读取下一条有效事件;空事件与 [DONE] 哨兵跳过,流尽返回 io.EOF。
|
||||
func (s *genAiSSEStream) Next() (aiwire.ChatChunk, error) {
|
||||
for {
|
||||
data, err := s.reader.ReadNextEvent()
|
||||
if err != nil {
|
||||
return aiwire.ChatChunk{}, err
|
||||
}
|
||||
text := strings.TrimSpace(string(data))
|
||||
if text == "" {
|
||||
continue
|
||||
}
|
||||
if text == "[DONE]" {
|
||||
return aiwire.ChatChunk{}, io.EOF
|
||||
}
|
||||
if chunk, ok := parseGenAiEvent([]byte(text), s.model); ok {
|
||||
return chunk, nil
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (s *genAiSSEStream) Close() error {
|
||||
if s.body != nil {
|
||||
return s.body.Close()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// GenAiProbeChat 实现 Client:配额探测专用的最小聊天(maxTokens=1),返回 HTTP 状态码;
|
||||
// GenAiProbeChat 实现 Client:经 OpenAI 兼容面(直通同链路)发一次极小请求探测渠道;配额探测专用的最小聊天(maxTokens=1),返回 HTTP 状态码;
|
||||
// modelName 决定请求格式(cohere.* 走 COHERE)。
|
||||
func (c *RealClient) GenAiProbeChat(ctx context.Context, cred Credentials, region, modelOcid, modelName string) (int, error) {
|
||||
one := 1
|
||||
ir := aiwire.ChatRequest{
|
||||
Model: modelName,
|
||||
Messages: []aiwire.ChatMessage{{Role: "user", Content: aiwire.NewTextContent("hi")}},
|
||||
MaxTokens: &one,
|
||||
body, err := json.Marshal(map[string]any{"model": modelName, "input": "hi",
|
||||
"max_output_tokens": 1, "store": false})
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
_, err := c.GenAiChat(ctx, cred, region, modelOcid, ir)
|
||||
if err == nil {
|
||||
if _, err = c.GenAiCompatResponses(ctx, cred, region, body); err == nil {
|
||||
return http.StatusOK, nil
|
||||
}
|
||||
if status, ok := ServiceStatus(err); ok {
|
||||
@@ -275,6 +161,27 @@ func (c *RealClient) GenAiProbeChat(ctx context.Context, cred Credentials, regio
|
||||
return 0, err
|
||||
}
|
||||
|
||||
// sdkUsageToIR 把 SDK 用量转为内部记账结构(缓存命中挂 details,仅命中时出现)。
|
||||
func sdkUsageToIR(u *generativeaiinference.Usage) *aiwire.Usage {
|
||||
if u == nil {
|
||||
return nil
|
||||
}
|
||||
out := &aiwire.Usage{}
|
||||
if u.PromptTokens != nil {
|
||||
out.PromptTokens = *u.PromptTokens
|
||||
}
|
||||
if u.CompletionTokens != nil {
|
||||
out.CompletionTokens = *u.CompletionTokens
|
||||
}
|
||||
if u.TotalTokens != nil {
|
||||
out.TotalTokens = *u.TotalTokens
|
||||
}
|
||||
if u.PromptTokensDetails != nil && u.PromptTokensDetails.CachedTokens != nil && *u.PromptTokensDetails.CachedTokens > 0 {
|
||||
out.PromptTokensDetails = &aiwire.PromptTokensDetails{CachedTokens: *u.PromptTokensDetails.CachedTokens}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// GenAiEmbed 实现 Client:文本向量化(on-demand serving);dimensions 透传 OutputDimensions。
|
||||
func (c *RealClient) GenAiEmbed(ctx context.Context, cred Credentials, region, modelOcid string, inputs []string, dimensions *int) ([][]float32, *aiwire.Usage, error) {
|
||||
ic, err := c.genAiInferenceClient(cred, region)
|
||||
|
||||
@@ -1,266 +0,0 @@
|
||||
package oci
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/oracle/oci-go-sdk/v65/generativeaiinference"
|
||||
|
||||
"oci-portal/internal/aiwire"
|
||||
)
|
||||
|
||||
// isCohereModel 依据模型名前缀判定 COHERE 系(走专属请求格式)。
|
||||
func isCohereModel(name string) bool {
|
||||
return strings.HasPrefix(strings.ToLower(name), "cohere.")
|
||||
}
|
||||
|
||||
// irToCohereSDK 把 IR 转为 COHERE 聊天请求;工具与结构化输出做有损降级
|
||||
// (参数仅取 JSON Schema 顶层 properties,tool_choice / json_schema 的 name、strict 无对应)。
|
||||
func irToCohereSDK(ir aiwire.ChatRequest, stream bool) (generativeaiinference.CohereChatRequest, error) {
|
||||
var req generativeaiinference.CohereChatRequest
|
||||
if err := cohereRejectUnsupported(ir); err != nil {
|
||||
return req, err
|
||||
}
|
||||
p, err := cohereSplitMessages(ir.Messages)
|
||||
if err != nil {
|
||||
return req, err
|
||||
}
|
||||
req = generativeaiinference.CohereChatRequest{
|
||||
Message: &p.message,
|
||||
ChatHistory: p.history,
|
||||
ToolResults: p.toolResults,
|
||||
Tools: cohereTools(ir.Tools),
|
||||
ResponseFormat: cohereResponseFormat(ir.ResponseFormat),
|
||||
MaxTokens: ir.MaxTokens,
|
||||
Temperature: ir.Temperature,
|
||||
TopP: ir.TopP,
|
||||
TopK: ir.TopK,
|
||||
FrequencyPenalty: ir.FrequencyPenalty,
|
||||
PresencePenalty: ir.PresencePenalty,
|
||||
Seed: ir.Seed,
|
||||
}
|
||||
if p.preamble != "" {
|
||||
req.PreambleOverride = &p.preamble
|
||||
}
|
||||
if len(ir.Stop) > 0 {
|
||||
req.StopSequences = ir.Stop
|
||||
}
|
||||
if stream {
|
||||
req.IsStream = &stream
|
||||
includeUsage := ir.StreamOptions == nil || ir.StreamOptions.IncludeUsage
|
||||
req.StreamOptions = &generativeaiinference.StreamOptions{IsIncludeUsage: &includeUsage}
|
||||
}
|
||||
return req, nil
|
||||
}
|
||||
|
||||
// cohereRejectUnsupported 拒绝 COHERE 无法承接的能力(多模态图片)。
|
||||
func cohereRejectUnsupported(ir aiwire.ChatRequest) error {
|
||||
for _, m := range ir.Messages {
|
||||
for _, p := range m.Content.Parts {
|
||||
if p.Type == "image_url" {
|
||||
return fmt.Errorf("cohere 系模型暂不支持图片输入,请改用 meta/google 等多模态模型")
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// cohereParts 是 IR 消息拆装为 COHERE 请求的中间结果。
|
||||
type cohereParts struct {
|
||||
message string
|
||||
history []generativeaiinference.CohereMessage
|
||||
preamble string
|
||||
toolResults []generativeaiinference.CohereToolResult
|
||||
}
|
||||
|
||||
// cohereSplitMessages 拆装消息:system 拼 preamble,末尾 user 抽为 message,
|
||||
// assistant(含 toolCalls)与其余 user 进 history,tool 结果转顶层 toolResults
|
||||
// (工具结果在末尾时 message 留空,COHERE 以 toolResults 续跑)。
|
||||
func cohereSplitMessages(msgs []aiwire.ChatMessage) (cohereParts, error) {
|
||||
lastUser, lastTool := -1, -1
|
||||
for i, m := range msgs {
|
||||
switch m.Role {
|
||||
case "user":
|
||||
lastUser = i
|
||||
case "tool":
|
||||
lastTool = i
|
||||
}
|
||||
}
|
||||
if lastUser == -1 && lastTool == -1 {
|
||||
return cohereParts{}, fmt.Errorf("cohere 系模型至少需要一条 user 消息")
|
||||
}
|
||||
var p cohereParts
|
||||
var preamble []string
|
||||
calls := map[string]generativeaiinference.CohereToolCall{}
|
||||
for i, m := range msgs {
|
||||
text := m.Content.JoinText()
|
||||
switch m.Role {
|
||||
case "system", "developer":
|
||||
preamble = append(preamble, text)
|
||||
case "assistant":
|
||||
p.history = append(p.history, cohereBotMessage(text, m.ToolCalls, calls))
|
||||
case "tool":
|
||||
p.toolResults = append(p.toolResults, cohereToolResult(m, calls))
|
||||
default: // user
|
||||
if i == lastUser && lastUser > lastTool {
|
||||
p.message = text
|
||||
continue
|
||||
}
|
||||
p.history = append(p.history, generativeaiinference.CohereUserMessage{Message: &text})
|
||||
}
|
||||
}
|
||||
p.preamble = strings.Join(preamble, "\n")
|
||||
return p, nil
|
||||
}
|
||||
|
||||
// cohereBotMessage 转 assistant 消息;toolCalls 同步登记进 id→call 映射供 tool 结果回查。
|
||||
func cohereBotMessage(text string, tcs []aiwire.ToolCall, calls map[string]generativeaiinference.CohereToolCall) generativeaiinference.CohereChatBotMessage {
|
||||
msg := generativeaiinference.CohereChatBotMessage{}
|
||||
if text != "" {
|
||||
msg.Message = &text
|
||||
}
|
||||
for _, tc := range tcs {
|
||||
name := tc.Function.Name
|
||||
var params interface{}
|
||||
if json.Unmarshal([]byte(tc.Function.Arguments), ¶ms) != nil || params == nil {
|
||||
params = map[string]any{}
|
||||
}
|
||||
call := generativeaiinference.CohereToolCall{Name: &name, Parameters: ¶ms}
|
||||
calls[tc.ID] = call
|
||||
msg.ToolCalls = append(msg.ToolCalls, call)
|
||||
}
|
||||
return msg
|
||||
}
|
||||
|
||||
// cohereToolResult 把 IR tool 消息转顶层工具结果;COHERE 工具调用无 id,
|
||||
// 靠 assistant 历史登记的映射回查,缺失时以 tool_call_id 名义调用兜底。
|
||||
func cohereToolResult(m aiwire.ChatMessage, calls map[string]generativeaiinference.CohereToolCall) generativeaiinference.CohereToolResult {
|
||||
call, ok := calls[m.ToolCallID]
|
||||
if !ok {
|
||||
name := m.ToolCallID
|
||||
var params interface{} = map[string]any{}
|
||||
call = generativeaiinference.CohereToolCall{Name: &name, Parameters: ¶ms}
|
||||
}
|
||||
text := m.Content.JoinText()
|
||||
var output interface{}
|
||||
if json.Unmarshal([]byte(text), &output) != nil || output == nil {
|
||||
output = map[string]any{"output": text}
|
||||
}
|
||||
if arr, isArr := output.([]interface{}); isArr {
|
||||
return generativeaiinference.CohereToolResult{Call: &call, Outputs: arr}
|
||||
}
|
||||
if _, isMap := output.(map[string]interface{}); !isMap {
|
||||
output = map[string]any{"output": output}
|
||||
}
|
||||
return generativeaiinference.CohereToolResult{Call: &call, Outputs: []interface{}{output}}
|
||||
}
|
||||
|
||||
// cohereTools 把 JSON Schema 工具定义降级为 COHERE 扁平参数表(嵌套结构有损:仅取顶层)。
|
||||
func cohereTools(tools []aiwire.Tool) []generativeaiinference.CohereTool {
|
||||
if len(tools) == 0 {
|
||||
return nil
|
||||
}
|
||||
out := make([]generativeaiinference.CohereTool, 0, len(tools))
|
||||
for _, t := range tools {
|
||||
name, desc := t.Function.Name, t.Function.Description
|
||||
if desc == "" {
|
||||
desc = name // Description 为 COHERE 必填
|
||||
}
|
||||
out = append(out, generativeaiinference.CohereTool{
|
||||
Name: &name, Description: &desc,
|
||||
ParameterDefinitions: cohereParams(t.Function.Parameters),
|
||||
})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// cohereParams 取 JSON Schema 顶层 properties 转扁平参数定义;嵌套 schema 只保留类型名。
|
||||
func cohereParams(schema json.RawMessage) map[string]generativeaiinference.CohereParameterDefinition {
|
||||
var s struct {
|
||||
Properties map[string]struct {
|
||||
Type string `json:"type"`
|
||||
Description string `json:"description"`
|
||||
} `json:"properties"`
|
||||
Required []string `json:"required"`
|
||||
}
|
||||
if json.Unmarshal(schema, &s) != nil || len(s.Properties) == 0 {
|
||||
return nil
|
||||
}
|
||||
req := map[string]bool{}
|
||||
for _, r := range s.Required {
|
||||
req[r] = true
|
||||
}
|
||||
out := make(map[string]generativeaiinference.CohereParameterDefinition, len(s.Properties))
|
||||
for k, v := range s.Properties {
|
||||
typ, desc := v.Type, v.Description
|
||||
if typ == "" {
|
||||
typ = "string"
|
||||
}
|
||||
pd := generativeaiinference.CohereParameterDefinition{Type: &typ}
|
||||
if desc != "" {
|
||||
pd.Description = &desc
|
||||
}
|
||||
if req[k] {
|
||||
t := true
|
||||
pd.IsRequired = &t
|
||||
}
|
||||
out[k] = pd
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// cohereResponseFormat 映射 json_object / json_schema(name、strict 无对应,有损降级)。
|
||||
func cohereResponseFormat(rf *aiwire.ResponseFormat) generativeaiinference.CohereResponseFormat {
|
||||
if rf == nil {
|
||||
return nil
|
||||
}
|
||||
switch rf.Type {
|
||||
case "json_object", "json_schema":
|
||||
out := generativeaiinference.CohereResponseJsonFormat{}
|
||||
var in struct {
|
||||
Schema json.RawMessage `json:"schema"`
|
||||
}
|
||||
if json.Unmarshal(rf.JSONSchema, &in) == nil && len(in.Schema) > 0 {
|
||||
var schema interface{}
|
||||
if json.Unmarshal(in.Schema, &schema) == nil {
|
||||
out.Schema = &schema
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// cohereSDKToIR 把 COHERE 非流式响应转回 IR;工具调用补派生 ID,存在时结束原因归 tool_calls。
|
||||
func cohereSDKToIR(resp generativeaiinference.CohereChatResponse, model string) *aiwire.ChatResponse {
|
||||
msg := aiwire.ChatMessage{Role: "assistant", Content: aiwire.NewTextContent(deref(resp.Text))}
|
||||
for i, tc := range resp.ToolCalls {
|
||||
msg.ToolCalls = append(msg.ToolCalls, cohereCallToIR(deref(tc.Name), tc.Parameters, i))
|
||||
}
|
||||
finish := mapFinishReason(string(resp.FinishReason))
|
||||
if len(msg.ToolCalls) > 0 {
|
||||
finish = "tool_calls"
|
||||
}
|
||||
return &aiwire.ChatResponse{
|
||||
Object: "chat.completion",
|
||||
Model: model,
|
||||
Choices: []aiwire.Choice{{Message: msg, FinishReason: finish}},
|
||||
Usage: sdkUsageToIR(resp.Usage),
|
||||
}
|
||||
}
|
||||
|
||||
// cohereCallToIR 生成 IR 工具调用;COHERE 无调用 id,按名称+序号派生(回传时按此回查)。
|
||||
func cohereCallToIR(name string, params *interface{}, idx int) aiwire.ToolCall {
|
||||
args := "{}"
|
||||
if params != nil && *params != nil {
|
||||
if b, err := json.Marshal(*params); err == nil {
|
||||
args = string(b)
|
||||
}
|
||||
}
|
||||
return aiwire.ToolCall{
|
||||
ID: fmt.Sprintf("call_%s_%d", name, idx),
|
||||
Type: "function",
|
||||
Function: aiwire.FunctionCall{Name: name, Arguments: args},
|
||||
}
|
||||
}
|
||||
@@ -1,367 +0,0 @@
|
||||
package oci
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
|
||||
"github.com/oracle/oci-go-sdk/v65/generativeaiinference"
|
||||
|
||||
"oci-portal/internal/aiwire"
|
||||
)
|
||||
|
||||
// irToSDK 把 IR(OpenAI 线格式)转为 OCI GENERIC 聊天请求;字段近一一镜像。
|
||||
func irToSDK(ir aiwire.ChatRequest, stream bool) generativeaiinference.GenericChatRequest {
|
||||
req := generativeaiinference.GenericChatRequest{
|
||||
Messages: irMessages(ir.Messages),
|
||||
MaxTokens: ir.MaxTokens,
|
||||
MaxCompletionTokens: ir.MaxCompletionTokens,
|
||||
Temperature: ir.Temperature,
|
||||
TopP: ir.TopP,
|
||||
TopK: ir.TopK,
|
||||
FrequencyPenalty: ir.FrequencyPenalty,
|
||||
PresencePenalty: ir.PresencePenalty,
|
||||
Seed: ir.Seed,
|
||||
NumGenerations: ir.N,
|
||||
Tools: irTools(ir.Tools),
|
||||
ToolChoice: irToolChoice(ir.ToolChoice),
|
||||
ResponseFormat: irResponseFormat(ir.ResponseFormat),
|
||||
}
|
||||
if len(ir.Stop) > 0 {
|
||||
req.Stop = ir.Stop
|
||||
}
|
||||
if ir.ReasoningEffort != "" {
|
||||
req.ReasoningEffort = generativeaiinference.GenericChatRequestReasoningEffortEnum(strings.ToUpper(ir.ReasoningEffort))
|
||||
}
|
||||
if stream {
|
||||
req.IsStream = &stream
|
||||
includeUsage := ir.StreamOptions == nil || ir.StreamOptions.IncludeUsage
|
||||
req.StreamOptions = &generativeaiinference.StreamOptions{IsIncludeUsage: &includeUsage}
|
||||
}
|
||||
return req
|
||||
}
|
||||
|
||||
// irMessages 按 role 拆装消息;user 消息保留图文块,其余角色只取文本。
|
||||
func irMessages(msgs []aiwire.ChatMessage) []generativeaiinference.Message {
|
||||
out := make([]generativeaiinference.Message, 0, len(msgs))
|
||||
for _, m := range msgs {
|
||||
switch m.Role {
|
||||
case "system", "developer":
|
||||
out = append(out, generativeaiinference.SystemMessage{Content: textContents(m.Content.JoinText())})
|
||||
case "assistant":
|
||||
out = append(out, generativeaiinference.AssistantMessage{
|
||||
Content: textContents(m.Content.JoinText()),
|
||||
ToolCalls: irToolCalls(m.ToolCalls),
|
||||
})
|
||||
case "tool":
|
||||
tm := generativeaiinference.ToolMessage{Content: textContents(m.Content.JoinText())}
|
||||
if m.ToolCallID != "" {
|
||||
id := m.ToolCallID
|
||||
tm.ToolCallId = &id
|
||||
}
|
||||
out = append(out, tm)
|
||||
default: // user
|
||||
out = append(out, generativeaiinference.UserMessage{Content: chatContents(m.Content)})
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// chatContents 把 IR 内容转 SDK 内容块;文本直通,image_url 转 IMAGE 块(data URI / 公网地址)。
|
||||
func chatContents(c aiwire.Content) []generativeaiinference.ChatContent {
|
||||
if len(c.Parts) == 0 {
|
||||
return textContents(c.Text)
|
||||
}
|
||||
var out []generativeaiinference.ChatContent
|
||||
for _, p := range c.Parts {
|
||||
switch {
|
||||
case p.Type == "image_url" && p.ImageURL != nil:
|
||||
img := generativeaiinference.ImageUrl{Url: &p.ImageURL.URL}
|
||||
if d, ok := generativeaiinference.GetMappingImageUrlDetailEnum(p.ImageURL.Detail); ok {
|
||||
img.Detail = d
|
||||
}
|
||||
out = append(out, generativeaiinference.ImageContent{ImageUrl: &img})
|
||||
case p.Text != "":
|
||||
t := p.Text
|
||||
out = append(out, generativeaiinference.TextContent{Text: &t})
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// textContents 把纯文本包装为单元素 TextContent 数组;空文本返回 nil。
|
||||
func textContents(text string) []generativeaiinference.ChatContent {
|
||||
if text == "" {
|
||||
return nil
|
||||
}
|
||||
return []generativeaiinference.ChatContent{generativeaiinference.TextContent{Text: &text}}
|
||||
}
|
||||
|
||||
func irToolCalls(calls []aiwire.ToolCall) []generativeaiinference.ToolCall {
|
||||
if len(calls) == 0 {
|
||||
return nil
|
||||
}
|
||||
out := make([]generativeaiinference.ToolCall, 0, len(calls))
|
||||
for _, c := range calls {
|
||||
id, name, args := c.ID, c.Function.Name, c.Function.Arguments
|
||||
out = append(out, generativeaiinference.FunctionCall{Id: &id, Name: &name, Arguments: &args})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func irTools(tools []aiwire.Tool) []generativeaiinference.ToolDefinition {
|
||||
if len(tools) == 0 {
|
||||
return nil
|
||||
}
|
||||
out := make([]generativeaiinference.ToolDefinition, 0, len(tools))
|
||||
for _, t := range tools {
|
||||
name, desc := t.Function.Name, t.Function.Description
|
||||
fd := generativeaiinference.FunctionDefinition{Name: &name}
|
||||
if desc != "" {
|
||||
fd.Description = &desc
|
||||
}
|
||||
if len(t.Function.Parameters) > 0 {
|
||||
var params interface{}
|
||||
if json.Unmarshal(t.Function.Parameters, ¶ms) == nil {
|
||||
fd.Parameters = ¶ms
|
||||
}
|
||||
}
|
||||
out = append(out, fd)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// irToolChoice 解析 "auto"/"none"/"required" 或 {type:function,function:{name}}。
|
||||
func irToolChoice(raw json.RawMessage) generativeaiinference.ToolChoice {
|
||||
if len(raw) == 0 {
|
||||
return nil
|
||||
}
|
||||
var s string
|
||||
if json.Unmarshal(raw, &s) == nil {
|
||||
switch s {
|
||||
case "none":
|
||||
return generativeaiinference.ToolChoiceNone{}
|
||||
case "required":
|
||||
return generativeaiinference.ToolChoiceRequired{}
|
||||
case "auto":
|
||||
return generativeaiinference.ToolChoiceAuto{}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
var obj struct {
|
||||
Function struct {
|
||||
Name string `json:"name"`
|
||||
} `json:"function"`
|
||||
}
|
||||
if json.Unmarshal(raw, &obj) == nil && obj.Function.Name != "" {
|
||||
return generativeaiinference.ToolChoiceFunction{Name: &obj.Function.Name}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func irResponseFormat(rf *aiwire.ResponseFormat) generativeaiinference.ResponseFormat {
|
||||
if rf == nil {
|
||||
return nil
|
||||
}
|
||||
switch rf.Type {
|
||||
case "json_object":
|
||||
return generativeaiinference.JsonObjectResponseFormat{}
|
||||
case "json_schema":
|
||||
return irJSONSchemaFormat(rf.JSONSchema)
|
||||
default:
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// irJSONSchemaFormat 映射 OpenAI json_schema{name,description,schema,strict}。
|
||||
func irJSONSchemaFormat(raw json.RawMessage) generativeaiinference.ResponseFormat {
|
||||
var in struct {
|
||||
Name string `json:"name"`
|
||||
Description string `json:"description"`
|
||||
Schema json.RawMessage `json:"schema"`
|
||||
Strict *bool `json:"strict"`
|
||||
}
|
||||
if json.Unmarshal(raw, &in) != nil || in.Name == "" {
|
||||
return generativeaiinference.JsonObjectResponseFormat{}
|
||||
}
|
||||
js := generativeaiinference.ResponseJsonSchema{Name: &in.Name, IsStrict: in.Strict}
|
||||
if in.Description != "" {
|
||||
js.Description = &in.Description
|
||||
}
|
||||
if len(in.Schema) > 0 {
|
||||
var schema interface{}
|
||||
if json.Unmarshal(in.Schema, &schema) == nil {
|
||||
js.Schema = &schema
|
||||
}
|
||||
}
|
||||
return generativeaiinference.JsonSchemaResponseFormat{JsonSchema: &js}
|
||||
}
|
||||
|
||||
// sdkToIR 把非流式 GenericChatResponse 转回 IR 响应;id/created 由调用方补。
|
||||
func sdkToIR(resp generativeaiinference.GenericChatResponse, model string) *aiwire.ChatResponse {
|
||||
out := &aiwire.ChatResponse{Object: "chat.completion", Model: model}
|
||||
if resp.TimeCreated != nil {
|
||||
out.Created = resp.TimeCreated.Unix()
|
||||
}
|
||||
for i, ch := range resp.Choices {
|
||||
idx := i
|
||||
if ch.Index != nil {
|
||||
idx = *ch.Index
|
||||
}
|
||||
out.Choices = append(out.Choices, aiwire.Choice{
|
||||
Index: idx,
|
||||
Message: sdkMessageToIR(ch.Message),
|
||||
FinishReason: mapFinishReason(deref(ch.FinishReason)),
|
||||
})
|
||||
}
|
||||
out.Usage = sdkUsageToIR(resp.Usage)
|
||||
return out
|
||||
}
|
||||
|
||||
func sdkUsageToIR(u *generativeaiinference.Usage) *aiwire.Usage {
|
||||
if u == nil {
|
||||
return nil
|
||||
}
|
||||
out := &aiwire.Usage{}
|
||||
if u.PromptTokens != nil {
|
||||
out.PromptTokens = *u.PromptTokens
|
||||
}
|
||||
if u.CompletionTokens != nil {
|
||||
out.CompletionTokens = *u.CompletionTokens
|
||||
}
|
||||
if u.TotalTokens != nil {
|
||||
out.TotalTokens = *u.TotalTokens
|
||||
}
|
||||
if d := u.PromptTokensDetails; d != nil && d.CachedTokens != nil && *d.CachedTokens > 0 {
|
||||
out.PromptTokensDetails = &aiwire.PromptTokensDetails{CachedTokens: *d.CachedTokens}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// sdkMessageToIR 提取助手消息文本与工具调用。
|
||||
func sdkMessageToIR(m generativeaiinference.Message) aiwire.ChatMessage {
|
||||
out := aiwire.ChatMessage{Role: "assistant"}
|
||||
am, ok := m.(generativeaiinference.AssistantMessage)
|
||||
if !ok {
|
||||
return out
|
||||
}
|
||||
var sb strings.Builder
|
||||
for _, c := range am.Content {
|
||||
if tc, ok := c.(generativeaiinference.TextContent); ok && tc.Text != nil {
|
||||
sb.WriteString(*tc.Text)
|
||||
}
|
||||
}
|
||||
out.Content = aiwire.NewTextContent(sb.String())
|
||||
for _, call := range am.ToolCalls {
|
||||
if fc, ok := call.(generativeaiinference.FunctionCall); ok {
|
||||
out.ToolCalls = append(out.ToolCalls, aiwire.ToolCall{
|
||||
ID: deref(fc.Id),
|
||||
Type: "function",
|
||||
Function: aiwire.FunctionCall{
|
||||
Name: deref(fc.Name),
|
||||
Arguments: deref(fc.Arguments),
|
||||
},
|
||||
})
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// mapFinishReason 归一化结束原因;OCI 与 OpenAI 取值同名,做小写与别名兜底。
|
||||
func mapFinishReason(reason string) string {
|
||||
switch strings.ToLower(reason) {
|
||||
case "", "null":
|
||||
return ""
|
||||
case "stop", "completed", "end_turn":
|
||||
return "stop"
|
||||
case "length", "max_tokens":
|
||||
return "length"
|
||||
case "tool_calls", "tool_call", "tool_use":
|
||||
return "tool_calls"
|
||||
case "complete":
|
||||
return "stop"
|
||||
case "error_toxic":
|
||||
return "content_filter"
|
||||
default:
|
||||
return strings.ToLower(reason)
|
||||
}
|
||||
}
|
||||
|
||||
// genAiStreamEvent 是 GENERIC 流式事件的宽容解析结构(字段按增量 choice 形状)。
|
||||
type genAiStreamEvent struct {
|
||||
Index *int `json:"index"`
|
||||
Message struct {
|
||||
Role string `json:"role"`
|
||||
Content []struct {
|
||||
Type string `json:"type"`
|
||||
Text string `json:"text"`
|
||||
} `json:"content"`
|
||||
ToolCalls []struct {
|
||||
ID string `json:"id"`
|
||||
Name string `json:"name"`
|
||||
Arguments string `json:"arguments"`
|
||||
} `json:"toolCalls"`
|
||||
} `json:"message"`
|
||||
FinishReason *string `json:"finishReason"`
|
||||
Usage *generativeaiinference.Usage `json:"usage"`
|
||||
Choices []json.RawMessage `json:"choices"` // 兼容整包 choices 形态
|
||||
Text string `json:"text"` // COHERE 流事件的顶层增量文本
|
||||
// CohereCalls 是 COHERE 流事件顶层的工具调用(与 GENERIC 的 message.toolCalls 不同层级)
|
||||
CohereCalls []struct {
|
||||
Name string `json:"name"`
|
||||
Parameters interface{} `json:"parameters"`
|
||||
} `json:"toolCalls"`
|
||||
}
|
||||
|
||||
// parseGenAiEvent 把一条 SSE 事件 JSON 解析为 IR chunk;无有效负载时 ok=false。
|
||||
func parseGenAiEvent(data []byte, model string) (aiwire.ChatChunk, bool) {
|
||||
var ev genAiStreamEvent
|
||||
if err := json.Unmarshal(data, &ev); err != nil {
|
||||
return aiwire.ChatChunk{}, false
|
||||
}
|
||||
if len(ev.Choices) > 0 { // choices 包裹形态:取首个展开重解析
|
||||
var inner genAiStreamEvent
|
||||
if json.Unmarshal(ev.Choices[0], &inner) == nil {
|
||||
inner.Usage = ev.Usage
|
||||
ev = inner
|
||||
}
|
||||
}
|
||||
chunk := aiwire.ChatChunk{Object: "chat.completion.chunk", Model: model}
|
||||
choice := aiwire.ChunkChoice{}
|
||||
if ev.Index != nil {
|
||||
choice.Index = *ev.Index
|
||||
}
|
||||
choice.Delta.Role = strings.ToLower(ev.Message.Role)
|
||||
var sb strings.Builder
|
||||
for _, c := range ev.Message.Content {
|
||||
sb.WriteString(c.Text)
|
||||
}
|
||||
if sb.Len() == 0 && ev.Text != "" { // COHERE 事件形态
|
||||
sb.WriteString(ev.Text)
|
||||
}
|
||||
choice.Delta.Content = sb.String()
|
||||
for i, tc := range ev.Message.ToolCalls {
|
||||
d := aiwire.ToolCallDelta{Index: i, ID: tc.ID}
|
||||
if tc.ID != "" {
|
||||
d.Type = "function"
|
||||
}
|
||||
d.Function.Name = tc.Name
|
||||
d.Function.Arguments = tc.Arguments
|
||||
choice.Delta.ToolCalls = append(choice.Delta.ToolCalls, d)
|
||||
}
|
||||
for i, tc := range ev.CohereCalls { // COHERE 顶层工具调用:整包出现,补派生 ID
|
||||
call := cohereCallToIR(tc.Name, &tc.Parameters, i)
|
||||
d := aiwire.ToolCallDelta{Index: len(choice.Delta.ToolCalls) + i, ID: call.ID, Type: "function"}
|
||||
d.Function.Name = call.Function.Name
|
||||
d.Function.Arguments = call.Function.Arguments
|
||||
choice.Delta.ToolCalls = append(choice.Delta.ToolCalls, d)
|
||||
}
|
||||
if r := mapFinishReason(deref(ev.FinishReason)); r != "" {
|
||||
choice.FinishReason = &r
|
||||
}
|
||||
hasPayload := choice.Delta.Content != "" || len(choice.Delta.ToolCalls) > 0 || choice.FinishReason != nil
|
||||
if hasPayload {
|
||||
chunk.Choices = []aiwire.ChunkChoice{choice}
|
||||
}
|
||||
chunk.Usage = sdkUsageToIR(ev.Usage)
|
||||
return chunk, hasPayload || chunk.Usage != nil
|
||||
}
|
||||
@@ -0,0 +1,496 @@
|
||||
package oci
|
||||
|
||||
// 真实 OCI GenAI effort 探针(重建版)。
|
||||
//
|
||||
// 前身 genai_cache_integration_test.go 承载缓存/亲和/服务端工具等历轮调研模式,
|
||||
// 调研结论均已归档(.trellis/tasks/archive/2026-07/*/research/),该文件已删除;
|
||||
// 本文件重建共享基础设施,保留 effort 支持矩阵探测,并新增 responses 端点探测
|
||||
// (multi-agent 模型仅允许走 Responses 面)。
|
||||
//
|
||||
// 运行(默认跳过,不触网):
|
||||
//
|
||||
// OCI_GENAI_CACHE_PROBE=effort-matrix OCI_CACHE_DB=<db 绝对路径> DATA_KEY=<主密钥> \
|
||||
// OCI_CACHE_RUN_ID=<唯一标识> go test ./internal/oci/ -run TestRealGenAiEffortMatrix -v
|
||||
//
|
||||
// 可选:OCI_CACHE_CONFIG_ID(默认 6)、OCI_CACHE_REGION(默认 us-chicago-1)、
|
||||
// OCI_EFFORT_CASES 自定义用例("model=effort" 逗号分隔;effort 后缀 "@responses"
|
||||
// 表示走 /actions/v1/responses 端点,如 "xai.grok-4.20-multi-agent=low@responses")。
|
||||
//
|
||||
// 纪律:数据库只读打开;OCI_GO_SDK_DEBUG 必须为空;日志不落原始请求体与凭据,
|
||||
// 错误消息经 OCID 遮掩后截断。
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"os"
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/glebarez/sqlite"
|
||||
"github.com/oracle/oci-go-sdk/v65/common"
|
||||
"github.com/oracle/oci-go-sdk/v65/generativeaiinference"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/logger"
|
||||
|
||||
projectcrypto "oci-portal/internal/crypto"
|
||||
"oci-portal/internal/model"
|
||||
)
|
||||
|
||||
const (
|
||||
effortProbeRegion = "us-chicago-1"
|
||||
effortProbeLimit = int64(4 << 20)
|
||||
)
|
||||
|
||||
var effortRunIDPattern = regexp.MustCompile(`^[A-Za-z0-9._-]{1,64}$`)
|
||||
|
||||
type effortProbeCase struct {
|
||||
model string
|
||||
effort string
|
||||
responses bool // true 走 /actions/v1/responses,否则 typed /actions/chat
|
||||
stream bool // 仅 responses 面:请求 SSE 流,观测事件类型序列
|
||||
}
|
||||
|
||||
type effortProbeResources struct {
|
||||
cred Credentials
|
||||
region string
|
||||
modelOCID map[string]string
|
||||
inference generativeaiinference.GenerativeAiInferenceClient
|
||||
}
|
||||
|
||||
type effortProbeResult struct {
|
||||
status int
|
||||
requestRef string
|
||||
duration time.Duration
|
||||
errorCode string
|
||||
errorMessage string
|
||||
completionTokens int
|
||||
reasoningTokens int
|
||||
answerLen int
|
||||
outputTypes []string
|
||||
}
|
||||
|
||||
// effortMatrixCases 默认批为 grok-4.3 五档基线;全模型矩阵结论见任务归档,
|
||||
// 复测用 OCI_EFFORT_CASES 自定义。
|
||||
func effortMatrixCases() []effortProbeCase {
|
||||
if raw := os.Getenv("OCI_EFFORT_CASES"); raw != "" {
|
||||
return parseEffortCases(raw)
|
||||
}
|
||||
return []effortProbeCase{
|
||||
{model: "xai.grok-4.3", effort: "none", responses: true},
|
||||
{model: "xai.grok-4.3", effort: "low", responses: true},
|
||||
{model: "xai.grok-4.3", effort: "high", responses: true},
|
||||
}
|
||||
}
|
||||
|
||||
// parseEffortCases 解析 "model=effort[@responses|@responses-stream]" 逗号分隔用例。
|
||||
func parseEffortCases(raw string) []effortProbeCase {
|
||||
var out []effortProbeCase
|
||||
for _, item := range strings.Split(raw, ",") {
|
||||
parts := strings.SplitN(strings.TrimSpace(item), "=", 2)
|
||||
if len(parts) != 2 || parts[0] == "" || parts[1] == "" {
|
||||
continue
|
||||
}
|
||||
effort, streaming := strings.CutSuffix(parts[1], "@responses-stream")
|
||||
if streaming {
|
||||
out = append(out, effortProbeCase{model: parts[0], effort: effort, responses: true, stream: true})
|
||||
continue
|
||||
}
|
||||
effort, _ = strings.CutSuffix(effort, "@responses") // 后缀兼容保留;所有用例均走 responses 面
|
||||
out = append(out, effortProbeCase{model: parts[0], effort: effort, responses: true})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func TestRealGenAiEffortMatrix(t *testing.T) {
|
||||
requireEffortProbeMode(t, "effort-matrix")
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Minute)
|
||||
defer cancel()
|
||||
resources := newEffortProbeResources(t, ctx)
|
||||
for _, tc := range effortMatrixCases() {
|
||||
runEffortCase(t, ctx, resources, tc)
|
||||
}
|
||||
}
|
||||
|
||||
func requireEffortProbeMode(t *testing.T, wanted string) {
|
||||
t.Helper()
|
||||
if os.Getenv("OCI_GENAI_CACHE_PROBE") != wanted {
|
||||
t.Skipf("set OCI_GENAI_CACHE_PROBE=%s to run", wanted)
|
||||
}
|
||||
if os.Getenv("OCI_GO_SDK_DEBUG") != "" {
|
||||
t.Fatal("OCI_GO_SDK_DEBUG must be unset to protect signed requests")
|
||||
}
|
||||
if !effortRunIDPattern.MatchString(os.Getenv("OCI_CACHE_RUN_ID")) {
|
||||
t.Fatal("set a unique OCI_CACHE_RUN_ID (1-64 safe characters)")
|
||||
}
|
||||
}
|
||||
|
||||
func runEffortCase(t *testing.T, ctx context.Context, resources effortProbeResources, tc effortProbeCase) {
|
||||
t.Helper()
|
||||
ocid, ok := resources.modelOCID[tc.model]
|
||||
if !ok {
|
||||
t.Logf("model=%s effort=%s resolve-failed: model not in on-demand catalog", tc.model, tc.effort)
|
||||
return
|
||||
}
|
||||
_ = ocid
|
||||
body, path, err := effortProbeBody(tc)
|
||||
if err != nil {
|
||||
t.Fatalf("build effort body: %v", err)
|
||||
}
|
||||
result := executeEffortProbe(ctx, resources, path, body)
|
||||
logEffortResult(t, tc, result)
|
||||
}
|
||||
|
||||
// effortProbeBody 构造 OpenAI Responses 直通请求体(typed chat 面已随网关剔除,
|
||||
// 探针仅保留 responses 端点;历史 typed 结论见任务归档)。
|
||||
func effortProbeBody(tc effortProbeCase) ([]byte, string, error) {
|
||||
question := "How many positive divisors does 360 have? Answer with the number only."
|
||||
body := map[string]interface{}{"model": tc.model, "input": question,
|
||||
"max_output_tokens": 2048, "store": false}
|
||||
if tc.effort != "" {
|
||||
body["reasoning"] = map[string]string{"effort": tc.effort}
|
||||
}
|
||||
if tc.stream {
|
||||
body["stream"] = true
|
||||
}
|
||||
payload, err := json.Marshal(body)
|
||||
return payload, "/actions/v1/responses", err
|
||||
}
|
||||
|
||||
// executeEffortProbe 以 IAM 签名裸 POST 指定路径;responses 面按生产实现补 compartment 头。
|
||||
func executeEffortProbe(ctx context.Context, resources effortProbeResources, path string, body []byte) effortProbeResult {
|
||||
client := resources.inference.BaseClient
|
||||
client.Configuration.CircuitBreaker = nil
|
||||
common.UpdateEndpointTemplateForOptions(&client)
|
||||
common.SetMissingTemplateParams(&client)
|
||||
request, err := http.NewRequestWithContext(ctx, http.MethodPost, path, strings.NewReader(string(body)))
|
||||
if err != nil {
|
||||
return effortProbeResult{errorCode: "BuildRequest", errorMessage: err.Error()}
|
||||
}
|
||||
request.Header.Set("Content-Type", "application/json")
|
||||
if path != "/actions/chat" {
|
||||
request.Header.Set("CompartmentId", resources.cred.TenancyOCID)
|
||||
request.Header.Set("opc-compartment-id", resources.cred.TenancyOCID)
|
||||
}
|
||||
return effortCallProbe(ctx, client, request)
|
||||
}
|
||||
|
||||
func effortCallProbe(ctx context.Context, client common.BaseClient, request *http.Request) effortProbeResult {
|
||||
started := time.Now()
|
||||
response, err := client.Call(ctx, request)
|
||||
result := effortProbeResult{duration: time.Since(started)}
|
||||
if response != nil {
|
||||
result.status = response.StatusCode
|
||||
result.requestRef = shortEffortRef(response.Header.Get("opc-request-id"))
|
||||
defer response.Body.Close()
|
||||
}
|
||||
if err != nil {
|
||||
result.errorCode, result.errorMessage = effortProbeError(err)
|
||||
return result
|
||||
}
|
||||
payload, readErr := io.ReadAll(io.LimitReader(response.Body, effortProbeLimit))
|
||||
if readErr != nil {
|
||||
result.errorCode, result.errorMessage = "ReadResponse", readErr.Error()
|
||||
return result
|
||||
}
|
||||
parseEffortPayload(&result, payload)
|
||||
return result
|
||||
}
|
||||
|
||||
func effortProbeError(err error) (string, string) {
|
||||
if serviceErr, ok := common.IsServiceError(err); ok {
|
||||
return fmt.Sprintf("%d", serviceErr.GetHTTPStatusCode()), sanitizeEffortText(serviceErr.GetMessage())
|
||||
}
|
||||
return "CallError", sanitizeEffortText(err.Error())
|
||||
}
|
||||
|
||||
// parseEffortPayload 兼容三种 usage 形态:typed 驼峰(chatResponse.usage.completionTokens)、
|
||||
// chat 兼容面下划线(usage.completion_tokens)、responses 面(usage.output_tokens)。
|
||||
func parseEffortPayload(result *effortProbeResult, payload []byte) {
|
||||
if text := strings.TrimSpace(string(payload)); strings.HasPrefix(text, "event:") || strings.HasPrefix(text, "data:") {
|
||||
parseEffortSSE(result, text)
|
||||
return
|
||||
}
|
||||
var root map[string]interface{}
|
||||
if json.Unmarshal(payload, &root) != nil {
|
||||
return
|
||||
}
|
||||
result.answerLen = len(parseEffortAnswer(root))
|
||||
result.outputTypes = parseEffortOutputTypes(root)
|
||||
if typed, ok := root["chatResponse"].(map[string]interface{}); ok {
|
||||
root = typed
|
||||
}
|
||||
usage, _ := root["usage"].(map[string]interface{})
|
||||
for _, key := range []string{"completionTokens", "completion_tokens", "output_tokens"} {
|
||||
if v, ok := effortInt(usage, key); ok {
|
||||
result.completionTokens = v
|
||||
break
|
||||
}
|
||||
}
|
||||
for _, detailsKey := range []string{"completionTokensDetails", "completion_tokens_details", "output_tokens_details"} {
|
||||
details, _ := usage[detailsKey].(map[string]interface{})
|
||||
for _, key := range []string{"reasoningTokens", "reasoning_tokens"} {
|
||||
if v, ok := effortInt(details, key); ok {
|
||||
result.reasoningTokens = v
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// parseEffortAnswer 提取正文文本:typed 取 chatResponse.choices[].message.content[].text,
|
||||
// responses 取 output[] 里 message 项的 content[].text。仅在内存中量长度,不落日志。
|
||||
func parseEffortAnswer(root map[string]interface{}) string {
|
||||
var sb strings.Builder
|
||||
if typed, ok := root["chatResponse"].(map[string]interface{}); ok {
|
||||
choices, _ := typed["choices"].([]interface{})
|
||||
for _, c := range choices {
|
||||
cm, _ := c.(map[string]interface{})
|
||||
msg, _ := cm["message"].(map[string]interface{})
|
||||
collectEffortText(&sb, msg["content"])
|
||||
}
|
||||
return sb.String()
|
||||
}
|
||||
output, _ := root["output"].([]interface{})
|
||||
for _, item := range output {
|
||||
im, _ := item.(map[string]interface{})
|
||||
if im["type"] == "message" {
|
||||
collectEffortText(&sb, im["content"])
|
||||
}
|
||||
}
|
||||
return sb.String()
|
||||
}
|
||||
|
||||
func collectEffortText(sb *strings.Builder, content interface{}) {
|
||||
parts, _ := content.([]interface{})
|
||||
for _, p := range parts {
|
||||
pm, _ := p.(map[string]interface{})
|
||||
if text, ok := pm["text"].(string); ok {
|
||||
sb.WriteString(text)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// parseEffortSSE 汇总 SSE 流:事件类型去重序列进 outputTypes,正文增量长度进 answerLen,
|
||||
// 末尾 completed 事件里的 usage 进 tokens 字段。
|
||||
func parseEffortSSE(result *effortProbeResult, text string) {
|
||||
seen := map[string]bool{}
|
||||
for _, line := range strings.Split(text, "\n") {
|
||||
if name, ok := strings.CutPrefix(line, "event: "); ok {
|
||||
if name = strings.TrimSpace(name); !seen[name] {
|
||||
seen[name] = true
|
||||
result.outputTypes = append(result.outputTypes, name)
|
||||
}
|
||||
continue
|
||||
}
|
||||
data, ok := strings.CutPrefix(line, "data: ")
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
var ev map[string]interface{}
|
||||
if json.Unmarshal([]byte(data), &ev) != nil {
|
||||
continue
|
||||
}
|
||||
if name, ok := ev["type"].(string); ok && !seen[name] { // 纯 data: 行流(无 event: 行)从事件体取类型
|
||||
seen[name] = true
|
||||
result.outputTypes = append(result.outputTypes, name)
|
||||
}
|
||||
if delta, ok := ev["delta"].(string); ok && strings.Contains(fmt.Sprint(ev["type"]), "output_text") {
|
||||
result.answerLen += len(delta)
|
||||
}
|
||||
if resp, ok := ev["response"].(map[string]interface{}); ok {
|
||||
usage, _ := resp["usage"].(map[string]interface{})
|
||||
if v, ok := effortInt(usage, "output_tokens"); ok {
|
||||
result.completionTokens = v
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// parseEffortOutputTypes 收集 responses 面 output 项类型序列(multi-agent 行为观测)。
|
||||
func parseEffortOutputTypes(root map[string]interface{}) []string {
|
||||
output, _ := root["output"].([]interface{})
|
||||
var types []string
|
||||
for _, item := range output {
|
||||
im, _ := item.(map[string]interface{})
|
||||
if s, ok := im["type"].(string); ok {
|
||||
types = append(types, s)
|
||||
}
|
||||
}
|
||||
return types
|
||||
}
|
||||
|
||||
func effortInt(root map[string]interface{}, key string) (int, bool) {
|
||||
if root == nil {
|
||||
return 0, false
|
||||
}
|
||||
if value, ok := root[key].(float64); ok {
|
||||
return int(value), true
|
||||
}
|
||||
return 0, false
|
||||
}
|
||||
|
||||
func logEffortResult(t *testing.T, tc effortProbeCase, result effortProbeResult) {
|
||||
t.Helper()
|
||||
endpoint := "typed-chat"
|
||||
if tc.responses {
|
||||
endpoint = "compat-responses"
|
||||
}
|
||||
t.Logf("endpoint=%s model=%s effort=%s status=%d completion=%d reasoning=%d answer_len=%d output_types=%v request=%s duration_ms=%d error=%s message=%q",
|
||||
endpoint, tc.model, tc.effort, result.status, result.completionTokens, result.reasoningTokens,
|
||||
result.answerLen, result.outputTypes, result.requestRef, result.duration.Milliseconds(),
|
||||
result.errorCode, result.errorMessage)
|
||||
}
|
||||
|
||||
// sanitizeEffortText 遮掩 OCID 并截断,避免错误消息携带租户可定位信息。
|
||||
func sanitizeEffortText(text string) string {
|
||||
text = regexp.MustCompile(`ocid1\.[a-z0-9._-]+`).ReplaceAllString(text, "ocid1.***")
|
||||
if len(text) > 300 {
|
||||
text = text[:300] + "..."
|
||||
}
|
||||
return text
|
||||
}
|
||||
|
||||
func shortEffortRef(ref string) string {
|
||||
if len(ref) > 12 {
|
||||
return ref[:12]
|
||||
}
|
||||
return ref
|
||||
}
|
||||
|
||||
func effortEnvOr(key, fallback string) string {
|
||||
if value := os.Getenv(key); value != "" {
|
||||
return value
|
||||
}
|
||||
return fallback
|
||||
}
|
||||
|
||||
// newEffortProbeResources 装配凭据、区域、inference 客户端与按需模型目录。
|
||||
func newEffortProbeResources(t *testing.T, ctx context.Context) effortProbeResources {
|
||||
t.Helper()
|
||||
cred := effortProbeCredentials(t)
|
||||
region := effortEnvOr("OCI_CACHE_REGION", effortProbeRegion)
|
||||
cred.Region = region
|
||||
real := &RealClient{}
|
||||
ic, err := real.genAiInferenceClient(cred, region)
|
||||
if err != nil {
|
||||
t.Fatalf("new inference client: %v", err)
|
||||
}
|
||||
models, err := real.ListGenAiModels(ctx, cred, region)
|
||||
if err != nil {
|
||||
t.Fatalf("list models: %s", sanitizeEffortText(err.Error()))
|
||||
}
|
||||
index := make(map[string]string, len(models))
|
||||
for _, m := range models {
|
||||
index[m.Name] = m.Ocid
|
||||
}
|
||||
return effortProbeResources{cred: cred, region: region, modelOCID: index, inference: ic}
|
||||
}
|
||||
|
||||
// effortProbeCredentials 优先从面板数据库(只读)取渠道凭据,否则退回本地 ini 测试凭据。
|
||||
func effortProbeCredentials(t *testing.T) Credentials {
|
||||
t.Helper()
|
||||
if dbPath := os.Getenv("OCI_CACHE_DB"); dbPath != "" {
|
||||
return effortDBCredentials(t, dbPath)
|
||||
}
|
||||
return loadTestCredentials(t, effortEnvOr("OCI_TEST_KEY", "试用期"))
|
||||
}
|
||||
|
||||
func effortDBCredentials(t *testing.T, dbPath string) Credentials {
|
||||
t.Helper()
|
||||
dsn := fmt.Sprintf("file:%s?mode=ro", dbPath)
|
||||
db, err := gorm.Open(sqlite.Open(dsn), &gorm.Config{Logger: logger.Default.LogMode(logger.Silent)})
|
||||
if err != nil {
|
||||
t.Fatalf("open read-only probe database: %v", err)
|
||||
}
|
||||
cipher, err := projectcrypto.NewCipher(os.Getenv("DATA_KEY"))
|
||||
if err != nil {
|
||||
t.Fatalf("create data cipher: %v", err)
|
||||
}
|
||||
var config model.OciConfig
|
||||
if err := db.First(&config, effortEnvOr("OCI_CACHE_CONFIG_ID", "6")).Error; err != nil {
|
||||
t.Fatalf("load OCI config: %v", err)
|
||||
}
|
||||
return decryptEffortCredentials(t, db, cipher, config)
|
||||
}
|
||||
|
||||
func decryptEffortCredentials(t *testing.T, db *gorm.DB, cipher *projectcrypto.Cipher, config model.OciConfig) Credentials {
|
||||
t.Helper()
|
||||
privateKey, err := cipher.DecryptString(config.PrivateKeyEnc)
|
||||
if err != nil {
|
||||
t.Fatalf("decrypt OCI config %d private key: %v", config.ID, err)
|
||||
}
|
||||
passphrase := ""
|
||||
if config.PassphraseEnc != "" {
|
||||
if passphrase, err = cipher.DecryptString(config.PassphraseEnc); err != nil {
|
||||
t.Fatalf("decrypt OCI config %d passphrase: %v", config.ID, err)
|
||||
}
|
||||
}
|
||||
return Credentials{TenancyOCID: config.TenancyOCID, UserOCID: config.UserOCID,
|
||||
Fingerprint: config.Fingerprint, Region: config.Region, PrivateKey: privateKey,
|
||||
Passphrase: passphrase, Proxy: loadEffortProxy(t, db, cipher, config.ProxyID)}
|
||||
}
|
||||
|
||||
func loadEffortProxy(t *testing.T, db *gorm.DB, cipher *projectcrypto.Cipher, proxyID *uint) *ProxySpec {
|
||||
t.Helper()
|
||||
if proxyID == nil {
|
||||
return nil
|
||||
}
|
||||
var proxy model.Proxy
|
||||
if err := db.First(&proxy, *proxyID).Error; err != nil {
|
||||
t.Fatalf("load proxy %d: %v", *proxyID, err)
|
||||
}
|
||||
password := ""
|
||||
if proxy.PasswordEnc != "" {
|
||||
decrypted, err := cipher.DecryptString(proxy.PasswordEnc)
|
||||
if err != nil {
|
||||
t.Fatalf("decrypt proxy %d password: %v", *proxyID, err)
|
||||
}
|
||||
password = decrypted
|
||||
}
|
||||
return &ProxySpec{Type: proxy.Type, Host: proxy.Host, Port: proxy.Port,
|
||||
Username: proxy.Username, Password: password}
|
||||
}
|
||||
|
||||
// TestParseEffortCases 断言自定义用例解析:端点后缀、空项与畸形项跳过。
|
||||
func TestParseEffortCases(t *testing.T) {
|
||||
got := parseEffortCases("a=low, b=high@responses ,bad,=x,c=")
|
||||
want := []effortProbeCase{{model: "a", effort: "low", responses: true}, {model: "b", effort: "high", responses: true}}
|
||||
if len(got) != len(want) {
|
||||
t.Fatalf("parseEffortCases = %+v, want %+v", got, want)
|
||||
}
|
||||
for i := range want {
|
||||
if got[i] != want[i] {
|
||||
t.Fatalf("case %d = %+v, want %+v", i, got[i], want[i])
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestParseEffortPayload 断言三种 usage 形态与 output 类型序列的解析。
|
||||
func TestParseEffortPayload(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
payload string
|
||||
completion int
|
||||
reasoning int
|
||||
types int
|
||||
}{
|
||||
{"typed 驼峰", `{"chatResponse":{"choices":[{"message":{"content":[{"type":"TEXT","text":"24"}]}}],"usage":{"completionTokens":5,"completionTokensDetails":{"reasoningTokens":9}}}}`, 5, 9, 0},
|
||||
{"responses 面", `{"output":[{"type":"reasoning"},{"type":"message","content":[{"type":"output_text","text":"24"}]}],"usage":{"output_tokens":7,"output_tokens_details":{"reasoning_tokens":3}}}`, 7, 3, 2},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
var result effortProbeResult
|
||||
parseEffortPayload(&result, []byte(tc.payload))
|
||||
if result.completionTokens != tc.completion || result.reasoningTokens != tc.reasoning || len(result.outputTypes) != tc.types {
|
||||
t.Fatalf("parse = %+v, want completion=%d reasoning=%d types=%d", result, tc.completion, tc.reasoning, tc.types)
|
||||
}
|
||||
if result.answerLen != 2 {
|
||||
t.Fatalf("answerLen = %d, want 2", result.answerLen)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,73 @@
|
||||
package oci
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
|
||||
"github.com/oracle/oci-go-sdk/v65/common"
|
||||
)
|
||||
|
||||
// compatResponsesLimit 限制直通响应体大小;web_search 输出含多段引用,给足余量。
|
||||
const compatResponsesLimit = int64(8 << 20)
|
||||
|
||||
// GenAiCompatResponses 实现 Client:把 OpenAI Responses 请求体直通到 OCI
|
||||
// `/20231130/actions/v1/responses`(IAM 签名)。该端点实测可执行 xAI 服务端工具
|
||||
// (web_search / x_search),但不在 Oracle 文档化工具白名单内,行为可能随服务
|
||||
// 版本、模型或区域变化;调用方须自行校验并改写请求体(store/stream)。
|
||||
func (c *RealClient) GenAiCompatResponses(ctx context.Context, cred Credentials, region string, body []byte) ([]byte, error) {
|
||||
ic, err := c.genAiInferenceClient(cred, region)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
client := ic.BaseClient
|
||||
common.UpdateEndpointTemplateForOptions(&client)
|
||||
common.SetMissingTemplateParams(&client)
|
||||
request, err := http.NewRequestWithContext(ctx, http.MethodPost, "/actions/v1/responses", bytes.NewReader(body))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("build compat responses request: %w", err)
|
||||
}
|
||||
request.Header.Set("Content-Type", "application/json")
|
||||
request.Header.Set("CompartmentId", cred.TenancyOCID)
|
||||
request.Header.Set("opc-compartment-id", cred.TenancyOCID)
|
||||
response, err := client.Call(ctx, request)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer response.Body.Close()
|
||||
payload, err := io.ReadAll(io.LimitReader(response.Body, compatResponsesLimit))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read compat responses body: %w", err)
|
||||
}
|
||||
return payload, nil
|
||||
}
|
||||
|
||||
// GenAiCompatResponsesStream 实现 Client:以流式直通 OCI `/actions/v1/responses`,
|
||||
// 建立成功(2xx)返回 SSE body(调用方负责 Close);建立失败返回 SDK ServiceError,
|
||||
// 与既有渠道切换/熔断错误分类兼容。请求体须由调用方置 stream:true。
|
||||
func (c *RealClient) GenAiCompatResponsesStream(ctx context.Context, cred Credentials, region string, body []byte) (io.ReadCloser, error) {
|
||||
ic, err := c.genAiInferenceClient(cred, region)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
client := ic.BaseClient
|
||||
common.UpdateEndpointTemplateForOptions(&client)
|
||||
common.SetMissingTemplateParams(&client)
|
||||
request, err := http.NewRequestWithContext(ctx, http.MethodPost, "/actions/v1/responses", bytes.NewReader(body))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("build compat responses stream request: %w", err)
|
||||
}
|
||||
request.Header.Set("Content-Type", "application/json")
|
||||
request.Header.Set("CompartmentId", cred.TenancyOCID)
|
||||
request.Header.Set("opc-compartment-id", cred.TenancyOCID)
|
||||
response, err := client.Call(ctx, request)
|
||||
if err != nil {
|
||||
if response != nil && response.Body != nil {
|
||||
response.Body.Close()
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
return response.Body, nil
|
||||
}
|
||||
+10
-176
@@ -1,193 +1,27 @@
|
||||
package oci
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"testing"
|
||||
|
||||
"github.com/oracle/oci-go-sdk/v65/generativeaiinference"
|
||||
|
||||
"oci-portal/internal/aiwire"
|
||||
)
|
||||
|
||||
// TestIrToSDK 断言 IR→GENERIC 的角色拆装、采样参数与工具映射。
|
||||
func TestIrToSDK(t *testing.T) {
|
||||
temp, mt := 0.7, 100
|
||||
ir := aiwire.ChatRequest{
|
||||
Model: "meta.llama-3.3-70b-instruct",
|
||||
Messages: []aiwire.ChatMessage{
|
||||
{Role: "system", Content: aiwire.NewTextContent("你是助手")},
|
||||
{Role: "user", Content: aiwire.NewTextContent("你好")},
|
||||
{Role: "assistant", ToolCalls: []aiwire.ToolCall{{ID: "c1", Type: "function", Function: aiwire.FunctionCall{Name: "get_weather", Arguments: `{"city":"东京"}`}}}},
|
||||
{Role: "tool", ToolCallID: "c1", Content: aiwire.NewTextContent("晴")},
|
||||
},
|
||||
Temperature: &temp,
|
||||
MaxTokens: &mt,
|
||||
Stop: aiwire.StringList{"END"},
|
||||
Tools: []aiwire.Tool{{Type: "function", Function: aiwire.FunctionDef{Name: "get_weather", Parameters: json.RawMessage(`{"type":"object"}`)}}},
|
||||
ToolChoice: json.RawMessage(`"auto"`),
|
||||
}
|
||||
req := irToSDK(ir, true)
|
||||
if len(req.Messages) != 4 {
|
||||
t.Fatalf("messages = %d, want 4", len(req.Messages))
|
||||
}
|
||||
if _, ok := req.Messages[0].(generativeaiinference.SystemMessage); !ok {
|
||||
t.Fatalf("messages[0] = %T, want SystemMessage", req.Messages[0])
|
||||
}
|
||||
am, ok := req.Messages[2].(generativeaiinference.AssistantMessage)
|
||||
if !ok || len(am.ToolCalls) != 1 {
|
||||
t.Fatalf("assistant tool calls 未映射: %T %+v", req.Messages[2], am)
|
||||
}
|
||||
tm, ok := req.Messages[3].(generativeaiinference.ToolMessage)
|
||||
if !ok || deref(tm.ToolCallId) != "c1" {
|
||||
t.Fatalf("tool message 未映射 toolCallId: %+v", tm)
|
||||
}
|
||||
if req.Temperature == nil || *req.Temperature != 0.7 {
|
||||
t.Fatalf("temperature 未直通")
|
||||
}
|
||||
if req.MaxTokens == nil || *req.MaxTokens != 100 || len(req.Stop) != 1 {
|
||||
t.Fatalf("maxTokens/stop 未直通")
|
||||
}
|
||||
if _, ok := req.ToolChoice.(generativeaiinference.ToolChoiceAuto); !ok {
|
||||
t.Fatalf("toolChoice = %T, want auto", req.ToolChoice)
|
||||
}
|
||||
if len(req.Tools) != 1 || req.IsStream == nil || !*req.IsStream || req.StreamOptions == nil {
|
||||
t.Fatalf("tools/stream 选项未映射")
|
||||
}
|
||||
}
|
||||
|
||||
// TestSdkToIR 断言响应文本、工具调用与用量的反向映射。
|
||||
func TestSdkToIR(t *testing.T) {
|
||||
text, fr := "东京晴", "tool_calls"
|
||||
idx, pt, ct, tt := 0, 10, 5, 15
|
||||
id, name, args := "c1", "get_weather", `{"city":"东京"}`
|
||||
resp := generativeaiinference.GenericChatResponse{
|
||||
Choices: []generativeaiinference.ChatChoice{{
|
||||
Index: &idx,
|
||||
Message: generativeaiinference.AssistantMessage{
|
||||
Content: []generativeaiinference.ChatContent{generativeaiinference.TextContent{Text: &text}},
|
||||
ToolCalls: []generativeaiinference.ToolCall{generativeaiinference.FunctionCall{Id: &id, Name: &name, Arguments: &args}},
|
||||
},
|
||||
FinishReason: &fr,
|
||||
}},
|
||||
Usage: &generativeaiinference.Usage{PromptTokens: &pt, CompletionTokens: &ct, TotalTokens: &tt},
|
||||
}
|
||||
out := sdkToIR(resp, "m1")
|
||||
if len(out.Choices) != 1 || out.Choices[0].Message.Content.JoinText() != "东京晴" {
|
||||
t.Fatalf("文本未映射: %+v", out)
|
||||
}
|
||||
if out.Choices[0].FinishReason != "tool_calls" || len(out.Choices[0].Message.ToolCalls) != 1 {
|
||||
t.Fatalf("finishReason/toolCalls 未映射: %+v", out.Choices[0])
|
||||
}
|
||||
if out.Usage == nil || out.Usage.TotalTokens != 15 {
|
||||
t.Fatalf("usage 未映射: %+v", out.Usage)
|
||||
}
|
||||
}
|
||||
|
||||
// TestParseGenAiEvent 断言流事件宽容解析:文本增量、结束原因与 usage 事件。
|
||||
func TestParseGenAiEvent(t *testing.T) {
|
||||
chunk, ok := parseGenAiEvent([]byte(`{"index":0,"message":{"role":"ASSISTANT","content":[{"type":"TEXT","text":"你"}]}}`), "m1")
|
||||
if !ok || len(chunk.Choices) != 1 || chunk.Choices[0].Delta.Content != "你" {
|
||||
t.Fatalf("文本增量解析失败: %+v", chunk)
|
||||
}
|
||||
chunk, ok = parseGenAiEvent([]byte(`{"finishReason":"stop","usage":{"promptTokens":3,"completionTokens":2,"totalTokens":5}}`), "m1")
|
||||
if !ok || chunk.Choices[0].FinishReason == nil || *chunk.Choices[0].FinishReason != "stop" {
|
||||
t.Fatalf("finishReason 解析失败: %+v", chunk)
|
||||
}
|
||||
if chunk.Usage == nil || chunk.Usage.TotalTokens != 5 {
|
||||
t.Fatalf("usage 解析失败: %+v", chunk.Usage)
|
||||
}
|
||||
if _, ok := parseGenAiEvent([]byte(`{}`), "m1"); ok {
|
||||
t.Fatal("空事件应返回 ok=false")
|
||||
}
|
||||
}
|
||||
|
||||
// TestSdkUsageCachedTokens 断言 embed 用量映射:缓存命中挂 details,仅命中 >0 时出现。
|
||||
func TestSdkUsageCachedTokens(t *testing.T) {
|
||||
p, c, tot, cached := 10, 5, 15, 8
|
||||
u := sdkUsageToIR(&generativeaiinference.Usage{PromptTokens: &p, CompletionTokens: &c, TotalTokens: &tot,
|
||||
PromptTokensDetails: &generativeaiinference.PromptTokensDetails{CachedTokens: &cached}})
|
||||
if u.CachedTokens() != 8 {
|
||||
t.Errorf("CachedTokens = %d, want 8", u.CachedTokens())
|
||||
if u.PromptTokens != 10 || u.CompletionTokens != 5 || u.TotalTokens != 15 {
|
||||
t.Fatalf("usage 映射错误: %+v", u)
|
||||
}
|
||||
if sdkUsageToIR(&generativeaiinference.Usage{PromptTokens: &p}).CachedTokens() != 0 {
|
||||
t.Error("无细分时 CachedTokens 应为 0")
|
||||
if u.PromptTokensDetails == nil || u.PromptTokensDetails.CachedTokens != 8 {
|
||||
t.Fatalf("cachedTokens 未映射: %+v", u.PromptTokensDetails)
|
||||
}
|
||||
}
|
||||
|
||||
func TestIrToCohereSDK(t *testing.T) {
|
||||
ir := aiwire.ChatRequest{
|
||||
Model: "cohere.command-r-plus",
|
||||
Messages: []aiwire.ChatMessage{
|
||||
{Role: "system", Content: aiwire.NewTextContent("你是助手")},
|
||||
{Role: "user", Content: aiwire.NewTextContent("第一问")},
|
||||
{Role: "assistant", Content: aiwire.NewTextContent("第一答")},
|
||||
{Role: "user", Content: aiwire.NewTextContent("第二问")},
|
||||
},
|
||||
zero := 0
|
||||
if got := sdkUsageToIR(&generativeaiinference.Usage{PromptTokens: &p, PromptTokensDetails: &generativeaiinference.PromptTokensDetails{CachedTokens: &zero}}); got.PromptTokensDetails != nil {
|
||||
t.Fatalf("零命中不应带 details: %+v", got.PromptTokensDetails)
|
||||
}
|
||||
req, err := irToCohereSDK(ir, true)
|
||||
if err != nil {
|
||||
t.Fatalf("irToCohereSDK: %v", err)
|
||||
}
|
||||
if *req.Message != "第二问" || *req.PreambleOverride != "你是助手" || len(req.ChatHistory) != 2 {
|
||||
t.Errorf("拆装错误: msg=%q preamble=%v history=%d", *req.Message, req.PreambleOverride, len(req.ChatHistory))
|
||||
}
|
||||
if _, ok := req.ChatHistory[0].(generativeaiinference.CohereUserMessage); !ok {
|
||||
t.Errorf("history[0] 应为 user: %T", req.ChatHistory[0])
|
||||
}
|
||||
if _, ok := req.ChatHistory[1].(generativeaiinference.CohereChatBotMessage); !ok {
|
||||
t.Errorf("history[1] 应为 chatbot: %T", req.ChatHistory[1])
|
||||
}
|
||||
if req.IsStream == nil || !*req.IsStream {
|
||||
t.Error("IsStream 未设置")
|
||||
}
|
||||
if req.StreamOptions == nil || req.StreamOptions.IsIncludeUsage == nil || !*req.StreamOptions.IsIncludeUsage {
|
||||
t.Error("流式应默认开启 usage 回传(StreamOptions.IsIncludeUsage)")
|
||||
}
|
||||
// 工具定义降级为扁平参数表(仅取 JSON Schema 顶层 properties)
|
||||
ir.Tools = []aiwire.Tool{{Type: "function", Function: aiwire.FunctionDef{
|
||||
Name: "get_weather", Parameters: json.RawMessage(`{"type":"object","properties":{"city":{"type":"string","description":"城市"}},"required":["city"]}`)}}}
|
||||
req2, err := irToCohereSDK(ir, false)
|
||||
if err != nil {
|
||||
t.Fatalf("工具请求应支持: %v", err)
|
||||
}
|
||||
tool := req2.Tools[0]
|
||||
if *tool.Name != "get_weather" || *tool.Description != "get_weather" {
|
||||
t.Errorf("tool = %+v", tool)
|
||||
}
|
||||
if pd, ok := tool.ParameterDefinitions["city"]; !ok || *pd.Type != "string" || pd.IsRequired == nil || !*pd.IsRequired {
|
||||
t.Errorf("param city = %+v", pd)
|
||||
}
|
||||
// 图片输入仍拒绝
|
||||
ir.Tools = nil
|
||||
ir.Messages = append(ir.Messages, aiwire.ChatMessage{Role: "user", Content: aiwire.NewPartsContent([]aiwire.ContentPart{
|
||||
{Type: "image_url", ImageURL: &aiwire.ImageURL{URL: "data:image/png;base64,xx"}}})})
|
||||
if _, err := irToCohereSDK(ir, false); err == nil {
|
||||
t.Error("图片输入应被拒绝")
|
||||
}
|
||||
// 无 user 消息被拒
|
||||
if _, err := irToCohereSDK(aiwire.ChatRequest{Model: "cohere.x", Messages: []aiwire.ChatMessage{{Role: "system", Content: aiwire.NewTextContent("s")}}}, false); err == nil {
|
||||
t.Error("无 user 消息应被拒绝")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCohereSDKToIR(t *testing.T) {
|
||||
text := "回答"
|
||||
resp := generativeaiinference.CohereChatResponse{
|
||||
Text: &text,
|
||||
FinishReason: generativeaiinference.CohereChatResponseFinishReasonComplete,
|
||||
}
|
||||
out := cohereSDKToIR(resp, "cohere.command-r-plus")
|
||||
if out.Choices[0].Message.Content.JoinText() != "回答" || out.Choices[0].FinishReason != "stop" {
|
||||
t.Errorf("cohereSDKToIR = %+v", out.Choices[0])
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseGenAiEventCohere(t *testing.T) {
|
||||
chunk, ok := parseGenAiEvent([]byte(`{"apiFormat":"COHERE","text":"你好"}`), "cohere.command-r")
|
||||
if !ok || chunk.Choices[0].Delta.Content != "你好" {
|
||||
t.Errorf("cohere text 事件解析 = %+v, %v", chunk, ok)
|
||||
}
|
||||
chunk, ok = parseGenAiEvent([]byte(`{"apiFormat":"COHERE","finishReason":"COMPLETE","usage":{"promptTokens":3,"completionTokens":5,"totalTokens":8}}`), "cohere.command-r")
|
||||
if !ok || *chunk.Choices[0].FinishReason != "stop" || chunk.Usage.TotalTokens != 8 {
|
||||
t.Errorf("cohere 终帧解析 = %+v, %v", chunk, ok)
|
||||
if sdkUsageToIR(nil) != nil {
|
||||
t.Fatal("nil usage 应返回 nil")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,332 +0,0 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"oci-portal/internal/aiwire"
|
||||
)
|
||||
|
||||
// ErrAiUnsupportedBlock 表示请求含网关无法承接的内容块(仅支持文本与图片)。
|
||||
var ErrAiUnsupportedBlock = fmt.Errorf("暂不支持文本与图片以外的内容块")
|
||||
|
||||
// AnthropicToIR 把 Anthropic Messages 请求转为 IR(OpenAI 线格式)。
|
||||
// tool_result 块拆为独立 tool 角色消息(一拆多),顶层 system 变首条 system 消息。
|
||||
func AnthropicToIR(req aiwire.MessagesRequest) (aiwire.ChatRequest, error) {
|
||||
ir := aiwire.ChatRequest{
|
||||
Model: req.Model,
|
||||
Temperature: req.Temperature,
|
||||
TopP: req.TopP,
|
||||
TopK: req.TopK,
|
||||
Stop: req.StopSequences,
|
||||
Stream: req.Stream,
|
||||
Tools: anthTools(req.Tools),
|
||||
ToolChoice: anthToolChoice(req.ToolChoice),
|
||||
}
|
||||
if req.MaxTokens > 0 {
|
||||
mt := req.MaxTokens
|
||||
ir.MaxTokens = &mt
|
||||
}
|
||||
if sys := req.SystemText(); sys != "" {
|
||||
ir.Messages = append(ir.Messages, aiwire.ChatMessage{Role: "system", Content: aiwire.NewTextContent(sys)})
|
||||
}
|
||||
for _, m := range req.Messages {
|
||||
msgs, err := anthMessageToIR(m)
|
||||
if err != nil {
|
||||
return ir, err
|
||||
}
|
||||
ir.Messages = append(ir.Messages, msgs...)
|
||||
}
|
||||
return ir, nil
|
||||
}
|
||||
|
||||
// anthMessageToIR 拆解单条 Anthropic 消息;user 消息里的 tool_result 前置为独立 tool 消息,
|
||||
// image 块转为 IR image_url 部件。
|
||||
func anthMessageToIR(m aiwire.AnthMessage) ([]aiwire.ChatMessage, error) {
|
||||
var out []aiwire.ChatMessage
|
||||
var parts []aiwire.ContentPart
|
||||
var toolCalls []aiwire.ToolCall
|
||||
hasImage := false
|
||||
for _, b := range m.Content.AllBlocks() {
|
||||
switch b.Type {
|
||||
case "text":
|
||||
parts = append(parts, aiwire.ContentPart{Type: "text", Text: b.Text})
|
||||
case "image":
|
||||
p, err := anthImagePart(b.Source)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
parts, hasImage = append(parts, p), true
|
||||
case "tool_use":
|
||||
toolCalls = append(toolCalls, aiwire.ToolCall{ID: b.ID, Type: "function",
|
||||
Function: aiwire.FunctionCall{Name: b.Name, Arguments: string(b.Input)}})
|
||||
case "tool_result":
|
||||
out = append(out, aiwire.ChatMessage{Role: "tool", ToolCallID: b.ToolUseID,
|
||||
Content: aiwire.NewTextContent(b.ResultText())})
|
||||
case "thinking", "redacted_thinking":
|
||||
// 一期忽略 thinking 块
|
||||
default:
|
||||
return nil, ErrAiUnsupportedBlock
|
||||
}
|
||||
}
|
||||
content := partsContent(parts, hasImage)
|
||||
if hasImage || content.JoinText() != "" || len(toolCalls) > 0 {
|
||||
out = append(out, aiwire.ChatMessage{Role: m.Role, Content: content, ToolCalls: toolCalls})
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// anthImagePart 把 Anthropic image 块转为 IR image_url 部件(base64 → data URI)。
|
||||
func anthImagePart(source json.RawMessage) (aiwire.ContentPart, error) {
|
||||
var src struct {
|
||||
Type string `json:"type"`
|
||||
MediaType string `json:"media_type"`
|
||||
Data string `json:"data"`
|
||||
URL string `json:"url"`
|
||||
}
|
||||
if err := json.Unmarshal(source, &src); err != nil {
|
||||
return aiwire.ContentPart{}, fmt.Errorf("image source 解析失败: %w", err)
|
||||
}
|
||||
switch src.Type {
|
||||
case "base64":
|
||||
if src.MediaType == "" || src.Data == "" {
|
||||
return aiwire.ContentPart{}, fmt.Errorf("image source 缺少 media_type 或 data")
|
||||
}
|
||||
url := "data:" + src.MediaType + ";base64," + src.Data
|
||||
return aiwire.ContentPart{Type: "image_url", ImageURL: &aiwire.ImageURL{URL: url}}, nil
|
||||
case "url":
|
||||
if src.URL == "" {
|
||||
return aiwire.ContentPart{}, fmt.Errorf("image source 缺少 url")
|
||||
}
|
||||
return aiwire.ContentPart{Type: "image_url", ImageURL: &aiwire.ImageURL{URL: src.URL}}, nil
|
||||
}
|
||||
return aiwire.ContentPart{}, fmt.Errorf("不支持的 image source 类型 %q", src.Type)
|
||||
}
|
||||
|
||||
// partsContent 无图片时退回字符串形态(与上游线格式习惯一致),含图片时保留块数组。
|
||||
func partsContent(parts []aiwire.ContentPart, hasImage bool) aiwire.Content {
|
||||
if !hasImage {
|
||||
var sb strings.Builder
|
||||
for _, p := range parts {
|
||||
sb.WriteString(p.Text)
|
||||
}
|
||||
return aiwire.NewTextContent(sb.String())
|
||||
}
|
||||
return aiwire.NewPartsContent(parts)
|
||||
}
|
||||
|
||||
func anthTools(tools []aiwire.AnthTool) []aiwire.Tool {
|
||||
if len(tools) == 0 {
|
||||
return nil
|
||||
}
|
||||
out := make([]aiwire.Tool, 0, len(tools))
|
||||
for _, t := range tools {
|
||||
out = append(out, aiwire.Tool{Type: "function", Function: aiwire.FunctionDef{
|
||||
Name: t.Name, Description: t.Description, Parameters: t.InputSchema}})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// anthToolChoice 映射 {type:auto|any|tool,name} → OpenAI 形态。
|
||||
func anthToolChoice(raw json.RawMessage) json.RawMessage {
|
||||
if len(raw) == 0 {
|
||||
return nil
|
||||
}
|
||||
var tc struct {
|
||||
Type string `json:"type"`
|
||||
Name string `json:"name"`
|
||||
}
|
||||
if json.Unmarshal(raw, &tc) != nil {
|
||||
return nil
|
||||
}
|
||||
switch tc.Type {
|
||||
case "auto":
|
||||
return json.RawMessage(`"auto"`)
|
||||
case "any":
|
||||
return json.RawMessage(`"required"`)
|
||||
case "tool":
|
||||
b, _ := json.Marshal(map[string]any{"type": "function", "function": map[string]string{"name": tc.Name}})
|
||||
return b
|
||||
case "none":
|
||||
return json.RawMessage(`"none"`)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// IRRespToAnthropic 把 IR 非流式响应转为 Anthropic Messages 响应。
|
||||
func IRRespToAnthropic(resp *aiwire.ChatResponse, id string) aiwire.MessagesResponse {
|
||||
out := aiwire.MessagesResponse{ID: id, Type: "message", Role: "assistant", Model: resp.Model, Content: []aiwire.AnthBlock{}}
|
||||
if len(resp.Choices) > 0 {
|
||||
choice := resp.Choices[0]
|
||||
if text := choice.Message.Content.JoinText(); text != "" {
|
||||
out.Content = append(out.Content, aiwire.AnthBlock{Type: "text", Text: text})
|
||||
}
|
||||
for _, tc := range choice.Message.ToolCalls {
|
||||
out.Content = append(out.Content, aiwire.AnthBlock{Type: "tool_use", ID: tc.ID,
|
||||
Name: tc.Function.Name, Input: argsToJSON(tc.Function.Arguments)})
|
||||
}
|
||||
out.StopReason = anthStopReason(choice.FinishReason)
|
||||
}
|
||||
if resp.Usage != nil {
|
||||
out.Usage = aiwire.AnthUsage{InputTokens: resp.Usage.PromptTokens, OutputTokens: resp.Usage.CompletionTokens,
|
||||
CacheReadInputTokens: resp.Usage.CachedTokens()}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// argsToJSON 保证 tool_use.input 是合法 JSON 对象(模型可能产出非法片段)。
|
||||
func argsToJSON(args string) json.RawMessage {
|
||||
trimmed := strings.TrimSpace(args)
|
||||
if trimmed == "" {
|
||||
return json.RawMessage(`{}`)
|
||||
}
|
||||
if json.Valid([]byte(trimmed)) {
|
||||
return json.RawMessage(trimmed)
|
||||
}
|
||||
b, _ := json.Marshal(map[string]string{"_raw": args})
|
||||
return b
|
||||
}
|
||||
|
||||
func anthStopReason(finish string) string {
|
||||
switch finish {
|
||||
case "length":
|
||||
return "max_tokens"
|
||||
case "tool_calls":
|
||||
return "tool_use"
|
||||
default:
|
||||
return "end_turn"
|
||||
}
|
||||
}
|
||||
|
||||
// ---- Anthropic 流式状态机 ----
|
||||
|
||||
// AnthEvent 是一条待写出的 Anthropic SSE 事件。
|
||||
type AnthEvent struct {
|
||||
Event string
|
||||
Data any
|
||||
}
|
||||
|
||||
// AnthStream 把 IR chunk 流聚合为 Anthropic 事件序列:
|
||||
// message_start → content_block_start/delta/stop(text 与 tool_use 分块)→ message_delta → message_stop。
|
||||
type AnthStream struct {
|
||||
id, model string
|
||||
started bool
|
||||
blockOpen bool
|
||||
blockIsTool bool
|
||||
toolID string
|
||||
blockIndex int
|
||||
stopReason string
|
||||
usage aiwire.AnthUsage
|
||||
}
|
||||
|
||||
// NewAnthStream 构造状态机;id 为响应消息 ID。
|
||||
func NewAnthStream(id, model string) *AnthStream {
|
||||
return &AnthStream{id: id, model: model, blockIndex: -1, stopReason: "end_turn"}
|
||||
}
|
||||
|
||||
// Feed 消费一个 IR chunk,返回应立即写出的事件。
|
||||
func (st *AnthStream) Feed(chunk aiwire.ChatChunk) []AnthEvent {
|
||||
var events []AnthEvent
|
||||
if !st.started {
|
||||
st.started = true
|
||||
events = append(events, st.startEvent())
|
||||
}
|
||||
if chunk.Usage != nil {
|
||||
st.usage = aiwire.AnthUsage{InputTokens: chunk.Usage.PromptTokens, OutputTokens: chunk.Usage.CompletionTokens,
|
||||
CacheReadInputTokens: chunk.Usage.CachedTokens()}
|
||||
}
|
||||
for _, choice := range chunk.Choices {
|
||||
events = append(events, st.feedDelta(choice.Delta)...)
|
||||
if choice.FinishReason != nil && *choice.FinishReason != "" {
|
||||
st.stopReason = anthStopReason(*choice.FinishReason)
|
||||
}
|
||||
}
|
||||
return events
|
||||
}
|
||||
|
||||
func (st *AnthStream) startEvent() AnthEvent {
|
||||
return AnthEvent{Event: "message_start", Data: map[string]any{
|
||||
"type": "message_start",
|
||||
"message": map[string]any{
|
||||
"id": st.id, "type": "message", "role": "assistant", "model": st.model,
|
||||
"content": []any{}, "stop_reason": nil,
|
||||
"usage": map[string]int{"input_tokens": 0, "output_tokens": 0},
|
||||
},
|
||||
}}
|
||||
}
|
||||
|
||||
// feedDelta 处理文本与工具调用增量,必要时切块。
|
||||
func (st *AnthStream) feedDelta(d aiwire.Delta) []AnthEvent {
|
||||
var events []AnthEvent
|
||||
if d.Content != "" {
|
||||
if !st.blockOpen || st.blockIsTool {
|
||||
events = append(events, st.openBlock(false, "", "")...)
|
||||
}
|
||||
events = append(events, AnthEvent{Event: "content_block_delta", Data: map[string]any{
|
||||
"type": "content_block_delta", "index": st.blockIndex,
|
||||
"delta": map[string]string{"type": "text_delta", "text": d.Content},
|
||||
}})
|
||||
}
|
||||
for _, tc := range d.ToolCalls {
|
||||
if tc.ID != "" && (!st.blockOpen || !st.blockIsTool || st.toolID != tc.ID) {
|
||||
events = append(events, st.openBlock(true, tc.ID, tc.Function.Name)...)
|
||||
}
|
||||
if tc.Function.Arguments != "" && st.blockOpen && st.blockIsTool {
|
||||
events = append(events, AnthEvent{Event: "content_block_delta", Data: map[string]any{
|
||||
"type": "content_block_delta", "index": st.blockIndex,
|
||||
"delta": map[string]string{"type": "input_json_delta", "partial_json": tc.Function.Arguments},
|
||||
}})
|
||||
}
|
||||
}
|
||||
return events
|
||||
}
|
||||
|
||||
// openBlock 关闭当前块并打开新块(text 或 tool_use)。
|
||||
func (st *AnthStream) openBlock(isTool bool, toolID, toolName string) []AnthEvent {
|
||||
var events []AnthEvent
|
||||
if st.blockOpen {
|
||||
events = append(events, st.closeBlockEvent())
|
||||
}
|
||||
st.blockOpen, st.blockIsTool, st.toolID = true, isTool, toolID
|
||||
st.blockIndex++
|
||||
block := map[string]any{"type": "text", "text": ""}
|
||||
if isTool {
|
||||
block = map[string]any{"type": "tool_use", "id": toolID, "name": toolName, "input": map[string]any{}}
|
||||
}
|
||||
events = append(events, AnthEvent{Event: "content_block_start", Data: map[string]any{
|
||||
"type": "content_block_start", "index": st.blockIndex, "content_block": block,
|
||||
}})
|
||||
return events
|
||||
}
|
||||
|
||||
func (st *AnthStream) closeBlockEvent() AnthEvent {
|
||||
return AnthEvent{Event: "content_block_stop", Data: map[string]any{
|
||||
"type": "content_block_stop", "index": st.blockIndex,
|
||||
}}
|
||||
}
|
||||
|
||||
// Finish 在上游流结束后收尾:关块 → message_delta(stop_reason+usage)→ message_stop。
|
||||
func (st *AnthStream) Finish() []AnthEvent {
|
||||
var events []AnthEvent
|
||||
if !st.started {
|
||||
events = append(events, st.startEvent())
|
||||
st.started = true
|
||||
}
|
||||
if st.blockOpen {
|
||||
events = append(events, st.closeBlockEvent())
|
||||
st.blockOpen = false
|
||||
}
|
||||
events = append(events,
|
||||
AnthEvent{Event: "message_delta", Data: map[string]any{
|
||||
"type": "message_delta",
|
||||
"delta": map[string]any{"stop_reason": st.stopReason, "stop_sequence": nil},
|
||||
"usage": map[string]int{"output_tokens": st.usage.OutputTokens},
|
||||
}},
|
||||
AnthEvent{Event: "message_stop", Data: map[string]any{"type": "message_stop"}},
|
||||
)
|
||||
return events
|
||||
}
|
||||
|
||||
// Usage 返回聚合到的用量(供调用日志)。
|
||||
func (st *AnthStream) Usage() aiwire.AnthUsage { return st.usage }
|
||||
@@ -1,136 +0,0 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"oci-portal/internal/aiwire"
|
||||
)
|
||||
|
||||
func mustAnthReq(t *testing.T, body string) aiwire.MessagesRequest {
|
||||
t.Helper()
|
||||
var req aiwire.MessagesRequest
|
||||
if err := json.Unmarshal([]byte(body), &req); err != nil {
|
||||
t.Fatalf("unmarshal anthropic request: %v", err)
|
||||
}
|
||||
return req
|
||||
}
|
||||
|
||||
func TestAnthropicToIR(t *testing.T) {
|
||||
req := mustAnthReq(t, `{
|
||||
"model": "meta.llama-3.3-70b-instruct", "max_tokens": 128, "system": "你是助手",
|
||||
"messages": [
|
||||
{"role": "user", "content": "东京天气?"},
|
||||
{"role": "assistant", "content": [
|
||||
{"type": "text", "text": "查询中"},
|
||||
{"type": "tool_use", "id": "t1", "name": "get_weather", "input": {"city": "东京"}}
|
||||
]},
|
||||
{"role": "user", "content": [
|
||||
{"type": "tool_result", "tool_use_id": "t1", "content": "晴 25 度"},
|
||||
{"type": "text", "text": "继续"}
|
||||
]}
|
||||
],
|
||||
"tools": [{"name": "get_weather", "input_schema": {"type": "object"}}],
|
||||
"tool_choice": {"type": "any"}
|
||||
}`)
|
||||
ir, err := AnthropicToIR(req)
|
||||
if err != nil {
|
||||
t.Fatalf("AnthropicToIR: %v", err)
|
||||
}
|
||||
roles := make([]string, 0, len(ir.Messages))
|
||||
for _, m := range ir.Messages {
|
||||
roles = append(roles, m.Role)
|
||||
}
|
||||
want := []string{"system", "user", "assistant", "tool", "user"}
|
||||
if strings.Join(roles, ",") != strings.Join(want, ",") {
|
||||
t.Fatalf("roles = %v, want %v", roles, want)
|
||||
}
|
||||
if ir.Messages[2].ToolCalls[0].Function.Name != "get_weather" {
|
||||
t.Errorf("tool_use 未映射: %+v", ir.Messages[2])
|
||||
}
|
||||
if ir.Messages[3].ToolCallID != "t1" || ir.Messages[3].Content.JoinText() != "晴 25 度" {
|
||||
t.Errorf("tool_result 未拆为 tool 消息: %+v", ir.Messages[3])
|
||||
}
|
||||
if ir.MaxTokens == nil || *ir.MaxTokens != 128 || len(ir.Tools) != 1 {
|
||||
t.Errorf("max_tokens/tools 未直通")
|
||||
}
|
||||
if string(ir.ToolChoice) != `"required"` {
|
||||
t.Errorf("tool_choice any → %s, want required", ir.ToolChoice)
|
||||
}
|
||||
// image 块拒绝
|
||||
bad := mustAnthReq(t, `{"model":"m","max_tokens":1,"messages":[{"role":"user","content":[{"type":"image","source":{}}]}]}`)
|
||||
if _, err := AnthropicToIR(bad); err == nil {
|
||||
t.Error("image 块应被拒绝")
|
||||
}
|
||||
}
|
||||
|
||||
func TestIRRespToAnthropic(t *testing.T) {
|
||||
resp := &aiwire.ChatResponse{
|
||||
Model: "m1",
|
||||
Choices: []aiwire.Choice{{
|
||||
Message: aiwire.ChatMessage{
|
||||
Role: "assistant",
|
||||
Content: aiwire.NewTextContent("查到了"),
|
||||
ToolCalls: []aiwire.ToolCall{{ID: "t1", Type: "function",
|
||||
Function: aiwire.FunctionCall{Name: "get_weather", Arguments: `{"city":"东京"}`}}},
|
||||
},
|
||||
FinishReason: "tool_calls",
|
||||
}},
|
||||
Usage: &aiwire.Usage{PromptTokens: 10, CompletionTokens: 5},
|
||||
}
|
||||
out := IRRespToAnthropic(resp, "msg_1")
|
||||
if len(out.Content) != 2 || out.Content[0].Type != "text" || out.Content[1].Type != "tool_use" {
|
||||
t.Fatalf("content 块 = %+v", out.Content)
|
||||
}
|
||||
if out.StopReason != "tool_use" || out.Usage.InputTokens != 10 || out.Usage.OutputTokens != 5 {
|
||||
t.Errorf("stop_reason/usage 未映射: %+v", out)
|
||||
}
|
||||
if string(out.Content[1].Input) != `{"city":"东京"}` {
|
||||
t.Errorf("tool input = %s", out.Content[1].Input)
|
||||
}
|
||||
}
|
||||
|
||||
// eventTypes 提取事件类型序列便于断言。
|
||||
func eventTypes(events []AnthEvent) string {
|
||||
types := make([]string, 0, len(events))
|
||||
for _, e := range events {
|
||||
types = append(types, e.Event)
|
||||
}
|
||||
return strings.Join(types, ",")
|
||||
}
|
||||
|
||||
func TestAnthStreamTextAndTool(t *testing.T) {
|
||||
st := NewAnthStream("msg_1", "m1")
|
||||
fr := "tool_calls"
|
||||
var events []AnthEvent
|
||||
events = append(events, st.Feed(aiwire.ChatChunk{Choices: []aiwire.ChunkChoice{{Delta: aiwire.Delta{Content: "你"}}}})...)
|
||||
events = append(events, st.Feed(aiwire.ChatChunk{Choices: []aiwire.ChunkChoice{{Delta: aiwire.Delta{Content: "好"}}}})...)
|
||||
events = append(events, st.Feed(aiwire.ChatChunk{Choices: []aiwire.ChunkChoice{{Delta: aiwire.Delta{
|
||||
ToolCalls: []aiwire.ToolCallDelta{{ID: "t1", Function: aiwire.FunctionCallDelta{Name: "get_weather", Arguments: `{"ci`}}},
|
||||
}}}})...)
|
||||
events = append(events, st.Feed(aiwire.ChatChunk{
|
||||
Choices: []aiwire.ChunkChoice{{Delta: aiwire.Delta{ToolCalls: []aiwire.ToolCallDelta{{ID: "t1", Function: aiwire.FunctionCallDelta{Arguments: `ty":"东京"}`}}}}, FinishReason: &fr}},
|
||||
Usage: &aiwire.Usage{PromptTokens: 8, CompletionTokens: 4},
|
||||
})...)
|
||||
events = append(events, st.Finish()...)
|
||||
|
||||
got := eventTypes(events)
|
||||
want := "message_start,content_block_start,content_block_delta,content_block_delta," +
|
||||
"content_block_stop,content_block_start,content_block_delta,content_block_delta," +
|
||||
"content_block_stop,message_delta,message_stop"
|
||||
if got != want {
|
||||
t.Fatalf("事件序列:\n got %s\nwant %s", got, want)
|
||||
}
|
||||
if st.Usage().OutputTokens != 4 {
|
||||
t.Errorf("usage 未聚合: %+v", st.Usage())
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnthStreamEmpty(t *testing.T) {
|
||||
st := NewAnthStream("msg_1", "m1")
|
||||
events := st.Finish()
|
||||
if got := eventTypes(events); got != "message_start,message_delta,message_stop" {
|
||||
t.Errorf("空流事件序列 = %s", got)
|
||||
}
|
||||
}
|
||||
+147
-177
@@ -5,6 +5,7 @@ import (
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log"
|
||||
@@ -30,10 +31,6 @@ const (
|
||||
aiBackoffCap = 30 * time.Minute
|
||||
// aiKeyTouchGap 是 LastUsedAt 的最小写库间隔,避免高频调用刷库
|
||||
aiKeyTouchGap = time.Minute
|
||||
// modelRecheckGap 是已标记不可用模型的复检间隔(恢复供给自动解除标记);
|
||||
// validateBatchCap 限制单渠道单轮验证的试调次数
|
||||
modelRecheckGap = 20 * time.Hour
|
||||
validateBatchCap = 32
|
||||
// 内容日志(红线例外)约束:开启必须限时(上限 7 天),正文截断,短保留
|
||||
aiContentLogMaxHours = 168
|
||||
aiContentLogRetention = 7 * 24 * time.Hour
|
||||
@@ -59,17 +56,13 @@ type AiGatewayService struct {
|
||||
// touchMu 保护各密钥的最近触达时间(内存节流,不追求跨实例精确)
|
||||
touchMu sync.Mutex
|
||||
lastTouch map[uint]time.Time
|
||||
// validateMu 保护 validating:同一渠道的模型验证不并发
|
||||
validateMu sync.Mutex
|
||||
validating map[uint]bool
|
||||
// onChannelsChanged 在渠道增删后触发,由 main 装配为探测任务同步钩子
|
||||
onChannelsChanged func(context.Context)
|
||||
}
|
||||
|
||||
// NewAiGatewayService 组装依赖;调用 StartCleanup 后开始调用日志周期清理。
|
||||
func NewAiGatewayService(db *gorm.DB, configs *OciConfigService, client oci.Client) *AiGatewayService {
|
||||
return &AiGatewayService{db: db, configs: configs, client: client,
|
||||
lastTouch: map[uint]time.Time{}, validating: map[uint]bool{}}
|
||||
return &AiGatewayService{db: db, configs: configs, client: client, lastTouch: map[uint]time.Time{}}
|
||||
}
|
||||
|
||||
// SetOnChannelsChanged 注册渠道数量变化钩子(渠道创建/删除成功后调用)。
|
||||
@@ -87,7 +80,7 @@ func (s *AiGatewayService) fireChannelsChanged(ctx context.Context) {
|
||||
|
||||
// CreateKey 生成网关密钥并返回明文(仅此一次);customValue 非空时使用给定值,
|
||||
// group 非空时该密钥只在同分组渠道内路由。
|
||||
func (s *AiGatewayService) CreateKey(ctx context.Context, name, customValue, group string) (string, *model.AiKey, error) {
|
||||
func (s *AiGatewayService) CreateKey(ctx context.Context, name, customValue, group string, models []string) (string, *model.AiKey, error) {
|
||||
name = strings.TrimSpace(name)
|
||||
if name == "" {
|
||||
return "", nil, fmt.Errorf("密钥名称不能为空")
|
||||
@@ -103,13 +96,28 @@ func (s *AiGatewayService) CreateKey(ctx context.Context, name, customValue, gro
|
||||
if len(raw) < 8 {
|
||||
return "", nil, fmt.Errorf("自定义密钥至少 8 个字符")
|
||||
}
|
||||
key := &model.AiKey{Name: name, KeyHash: hashKey(raw), Tail: raw[len(raw)-4:], Group: strings.TrimSpace(group), Enabled: true}
|
||||
key := &model.AiKey{Name: name, KeyHash: hashKey(raw), Tail: raw[len(raw)-4:], Group: strings.TrimSpace(group), Models: normalizeKeyModels(models), Enabled: true}
|
||||
if err := s.db.WithContext(ctx).Create(key).Error; err != nil {
|
||||
return "", nil, fmt.Errorf("密钥名称或取值与现有密钥重复")
|
||||
}
|
||||
return raw, key, nil
|
||||
}
|
||||
|
||||
// normalizeKeyModels 规范化模型白名单:trim、剔空串、保序去重;结果为空返回 nil(= 不限)。
|
||||
func normalizeKeyModels(models []string) []string {
|
||||
var out []string
|
||||
seen := map[string]bool{}
|
||||
for _, m := range models {
|
||||
m = strings.TrimSpace(m)
|
||||
if m == "" || seen[m] {
|
||||
continue
|
||||
}
|
||||
seen[m] = true
|
||||
out = append(out, m)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func hashKey(raw string) string {
|
||||
sum := sha256.Sum256([]byte(raw))
|
||||
return hex.EncodeToString(sum[:])
|
||||
@@ -122,8 +130,8 @@ func (s *AiGatewayService) Keys(ctx context.Context) ([]model.AiKey, error) {
|
||||
return keys, err
|
||||
}
|
||||
|
||||
// UpdateKey 修改密钥名称 / 启用状态 / 分组(group 指针非空即覆盖,可置空)。
|
||||
func (s *AiGatewayService) UpdateKey(ctx context.Context, id uint, name string, enabled *bool, group *string) error {
|
||||
// UpdateKey 修改密钥名称 / 启用状态 / 分组 / 模型白名单(指针非空即覆盖,可置空)。
|
||||
func (s *AiGatewayService) UpdateKey(ctx context.Context, id uint, name string, enabled *bool, group *string, models *[]string) error {
|
||||
updates := map[string]any{}
|
||||
if name = strings.TrimSpace(name); name != "" {
|
||||
updates["name"] = name
|
||||
@@ -134,6 +142,14 @@ func (s *AiGatewayService) UpdateKey(ctx context.Context, id uint, name string,
|
||||
if group != nil {
|
||||
updates["key_group"] = strings.TrimSpace(*group)
|
||||
}
|
||||
if models != nil {
|
||||
// 手动序列化走 map 更新,不依赖 GORM map 路径对 serializer 的支持
|
||||
b, err := json.Marshal(normalizeKeyModels(*models))
|
||||
if err != nil {
|
||||
return fmt.Errorf("serialize models: %w", err)
|
||||
}
|
||||
updates["models"] = string(b)
|
||||
}
|
||||
if len(updates) == 0 {
|
||||
return nil
|
||||
}
|
||||
@@ -298,12 +314,17 @@ func (s *AiGatewayService) ProbeChannel(ctx context.Context, id uint) (*model.Ai
|
||||
return &fresh, nil
|
||||
}
|
||||
|
||||
// probe 执行探测并返回 (状态, 错误摘要);同时完成模型缓存同步(保留不可用标记)。
|
||||
// probe 执行探测并返回 (状态, 错误摘要);同时完成模型缓存同步(黑名单模型不入库)。
|
||||
func (s *AiGatewayService) probe(ctx context.Context, cred oci.Credentials, ch *model.AiChannel) (string, string) {
|
||||
models, err := s.client.ListGenAiModels(ctx, cred, ch.Region)
|
||||
if err != nil {
|
||||
return classifyProbeErr(err), truncateErr(oci.CompactError(err))
|
||||
}
|
||||
models = supportedGatewayModels(models)
|
||||
models, err = s.withoutBlacklisted(ctx, models)
|
||||
if err != nil {
|
||||
return "error", truncateErr(err.Error())
|
||||
}
|
||||
if len(models) == 0 {
|
||||
_ = s.replaceModels(ctx, ch.ID, nil)
|
||||
return "no_service", "区域无可用模型(GenAI 服务不可用或未开放)"
|
||||
@@ -311,31 +332,12 @@ func (s *AiGatewayService) probe(ctx context.Context, cred oci.Credentials, ch *
|
||||
if err := s.replaceModels(ctx, ch.ID, models); err != nil {
|
||||
return "error", truncateErr(err.Error())
|
||||
}
|
||||
usable, err := s.usableModels(ctx, ch.ID, models)
|
||||
if err != nil {
|
||||
return "error", truncateErr(err.Error())
|
||||
}
|
||||
return s.probeChat(ctx, cred, ch, usable)
|
||||
return s.probeChat(ctx, cred, ch, models)
|
||||
}
|
||||
|
||||
// usableModels 过滤掉缓存中已标记不可按需调用的模型,探测候选不再反复踩坑。
|
||||
func (s *AiGatewayService) usableModels(ctx context.Context, channelID uint, models []oci.GenAiModel) ([]oci.GenAiModel, error) {
|
||||
marks, err := s.loadModelMarks(ctx, channelID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out := make([]oci.GenAiModel, 0, len(models))
|
||||
for _, m := range models {
|
||||
if !marks[m.Ocid].Unusable {
|
||||
out = append(out, m)
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// probeChat 按候选顺序试调(上限 8):遇「模型不可按需调用」标记剔除并换下一个;
|
||||
// 401/403 与鉴权类 404 属租户级直接定论 no_quota;其余错误(部分模型元数据标 CHAT
|
||||
// 但实际不可对话,如 voice agent)累计 3 次止损。
|
||||
// probeChat 按候选顺序试调(上限 8):遇「模型不可按需调用」(微调基座 400 / 实体
|
||||
// 不存在 404)换下一个候选,错误信息带模型名供用户加入黑名单;401/403 与鉴权类 404
|
||||
// 属租户级直接定论 no_quota;其余错误(元数据标 CHAT 但实际不可对话等)累计 3 次止损。
|
||||
func (s *AiGatewayService) probeChat(ctx context.Context, cred oci.Credentials, ch *model.AiChannel, models []oci.GenAiModel) (string, string) {
|
||||
status, detail := "error", "无可试调对话模型"
|
||||
errBudget := 3
|
||||
@@ -345,8 +347,7 @@ func (s *AiGatewayService) probeChat(ctx context.Context, cred oci.Credentials,
|
||||
case code == 200 || code == 429:
|
||||
return "ok", ""
|
||||
case oci.IsModelUnavailable(err):
|
||||
s.markModelUnusable(ctx, ch.ID, m.Ocid, oci.CompactError(err))
|
||||
status, detail = "error", truncateErr(fmt.Sprintf("%s: 不可按需调用,已从模型池剔除", m.Name))
|
||||
status, detail = "error", truncateErr(fmt.Sprintf("%s: 不可按需调用,建议加入模型黑名单", m.Name))
|
||||
case code == 401 || code == 403 || code == 404:
|
||||
return "no_quota", truncateErr(oci.CompactError(err))
|
||||
default:
|
||||
@@ -359,106 +360,8 @@ func (s *AiGatewayService) probeChat(ctx context.Context, cred oci.Credentials,
|
||||
return status, detail
|
||||
}
|
||||
|
||||
// markModelUnusable 把 (渠道, 模型OCID) 标记为不可按需调用;失败仅记日志不阻断主流程。
|
||||
func (s *AiGatewayService) markModelUnusable(ctx context.Context, channelID uint, ocid, reason string) {
|
||||
err := s.db.WithContext(ctx).Model(&model.AiModelCache{}).
|
||||
Where("channel_id = ? AND model_ocid = ?", channelID, ocid).
|
||||
Updates(map[string]any{"unusable": true, "unusable_reason": shortReason(reason), "checked_at": time.Now()}).Error
|
||||
if err != nil {
|
||||
log.Printf("mark model unusable: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func shortReason(reason string) string {
|
||||
if len(reason) > 200 {
|
||||
return reason[:200]
|
||||
}
|
||||
return reason
|
||||
}
|
||||
|
||||
// beginValidate 抢占渠道的验证执行权,同渠道同时只跑一轮。
|
||||
func (s *AiGatewayService) beginValidate(id uint) bool {
|
||||
s.validateMu.Lock()
|
||||
defer s.validateMu.Unlock()
|
||||
if s.validating[id] {
|
||||
return false
|
||||
}
|
||||
s.validating[id] = true
|
||||
return true
|
||||
}
|
||||
|
||||
func (s *AiGatewayService) endValidate(id uint) {
|
||||
s.validateMu.Lock()
|
||||
delete(s.validating, id)
|
||||
s.validateMu.Unlock()
|
||||
}
|
||||
|
||||
// validateTargets 取待验证行:从未验证的新模型,以及标记超过复检间隔的模型(仅对话能力,
|
||||
// 兼容存量空串);单轮上限 validateBatchCap 控制调用量。
|
||||
func (s *AiGatewayService) validateTargets(ctx context.Context, channelID uint) ([]model.AiModelCache, error) {
|
||||
stale := time.Now().Add(-modelRecheckGap)
|
||||
var rows []model.AiModelCache
|
||||
err := s.db.WithContext(ctx).
|
||||
Where("channel_id = ? AND capability IN ?", channelID, []string{"CHAT", ""}).
|
||||
Where("checked_at IS NULL OR (unusable = ? AND checked_at < ?)", true, stale).
|
||||
Order("id ASC").Limit(validateBatchCap).Find(&rows).Error
|
||||
return rows, err
|
||||
}
|
||||
|
||||
// validateChannelModels 逐个试调渠道内待验证模型并落结论,把不可按需调用的模型
|
||||
// 从池中剔除、把恢复供给的解除标记;ListModels 无字段可事先判别,只能试调习得。
|
||||
func (s *AiGatewayService) validateChannelModels(ctx context.Context, channelID uint) {
|
||||
if !s.beginValidate(channelID) {
|
||||
return
|
||||
}
|
||||
defer s.endValidate(channelID)
|
||||
var ch model.AiChannel
|
||||
if err := s.db.WithContext(ctx).First(&ch, channelID).Error; err != nil {
|
||||
return
|
||||
}
|
||||
cred, err := s.configs.credentialsByID(ctx, ch.OciConfigID)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
rows, err := s.validateTargets(ctx, channelID)
|
||||
if err != nil {
|
||||
log.Printf("validate channel %d models: %v", channelID, err)
|
||||
return
|
||||
}
|
||||
for _, r := range rows {
|
||||
s.validateOne(ctx, cred, &ch, r)
|
||||
}
|
||||
}
|
||||
|
||||
// validateOne 试调单个模型并落库:可用(200/429)清除标记;模型级不可用标记剔除;
|
||||
// 其余 4xx 记录已检不改状态;5xx/网络类瞬态不落 checked_at,下轮再验。
|
||||
func (s *AiGatewayService) validateOne(ctx context.Context, cred oci.Credentials, ch *model.AiChannel, r model.AiModelCache) {
|
||||
code, err := s.client.GenAiProbeChat(ctx, cred, ch.Region, r.ModelOcid, r.Name)
|
||||
updates := map[string]any{"checked_at": time.Now()}
|
||||
switch {
|
||||
case code == 200 || code == 429:
|
||||
updates["unusable"], updates["unusable_reason"] = false, ""
|
||||
case oci.IsModelUnavailable(err):
|
||||
updates["unusable"], updates["unusable_reason"] = true, shortReason(oci.CompactError(err))
|
||||
case code == 0 || code >= 500:
|
||||
return
|
||||
}
|
||||
s.db.WithContext(ctx).Model(&model.AiModelCache{}).Where("id = ?", r.ID).Updates(updates)
|
||||
}
|
||||
|
||||
// validateModelsAsync 后台验证渠道模型池(手动同步后触发,不阻塞接口响应)。
|
||||
func (s *AiGatewayService) validateModelsAsync(ctx context.Context, channelID uint) {
|
||||
s.wg.Add(1)
|
||||
go func() {
|
||||
defer s.wg.Done()
|
||||
vctx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 3*time.Minute)
|
||||
defer cancel()
|
||||
s.validateChannelModels(vctx, channelID)
|
||||
}()
|
||||
}
|
||||
|
||||
// probeCandidateCap 是单次探测的候选上限:模型级不可用会被标记剔除不再重试,
|
||||
// 放宽到 8 让一次探测有机会越过整批坏模型找到可用者;其他错误另有 3 次止损预算。
|
||||
// probeCandidateCap 是单次探测的候选上限:放宽到 8 让一次探测有机会越过整批
|
||||
// 不可按需调用的坏模型找到可用者;其他错误另有 3 次止损预算。
|
||||
const probeCandidateCap = 8
|
||||
|
||||
// probeCandidates 只取对话模型,按可靠度排序后跨厂商取候选:主流文本模型优先;
|
||||
@@ -551,8 +454,7 @@ func truncateErr(msg string) string {
|
||||
return msg
|
||||
}
|
||||
|
||||
// SyncModels 重新拉取渠道区域的模型列表并覆盖缓存(标记按 OCID 结转),
|
||||
// 随后触发后台验证:新模型逐个试调,不可按需调用的数十秒内从池中剔除。
|
||||
// SyncModels 重新拉取渠道区域的模型列表并覆盖缓存,黑名单中的模型不入库。
|
||||
func (s *AiGatewayService) SyncModels(ctx context.Context, id uint) ([]model.AiModelCache, error) {
|
||||
var ch model.AiChannel
|
||||
if err := s.db.WithContext(ctx).First(&ch, id).Error; err != nil {
|
||||
@@ -566,29 +468,23 @@ func (s *AiGatewayService) SyncModels(ctx context.Context, id uint) ([]model.AiM
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("同步模型失败:%s", oci.CompactError(err))
|
||||
}
|
||||
models = supportedGatewayModels(models)
|
||||
if models, err = s.withoutBlacklisted(ctx, models); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := s.replaceModels(ctx, id, models); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
s.validateModelsAsync(ctx, id)
|
||||
return s.channelModels(ctx, id)
|
||||
}
|
||||
|
||||
// replaceModels 以事务整组覆盖渠道模型缓存,按 OCID 结转不可用标记与验证时间
|
||||
// (OCID 变化视为新条目,自然回到待验证状态)。
|
||||
// replaceModels 以事务整组覆盖渠道模型缓存。
|
||||
func (s *AiGatewayService) replaceModels(ctx context.Context, channelID uint, models []oci.GenAiModel) error {
|
||||
marks, err := s.loadModelMarks(ctx, channelID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
rows := make([]model.AiModelCache, 0, len(models))
|
||||
now := time.Now()
|
||||
for _, m := range models {
|
||||
row := model.AiModelCache{ChannelID: channelID, ModelOcid: m.Ocid, Name: m.Name, Vendor: m.Vendor,
|
||||
Capability: m.Capability, SyncedAt: now, DeprecatedAt: m.Deprecated, RetiredAt: m.Retired}
|
||||
if prev, ok := marks[m.Ocid]; ok {
|
||||
row.Unusable, row.UnusableReason, row.CheckedAt = prev.Unusable, prev.UnusableReason, prev.CheckedAt
|
||||
}
|
||||
rows = append(rows, row)
|
||||
rows = append(rows, model.AiModelCache{ChannelID: channelID, ModelOcid: m.Ocid, Name: m.Name, Vendor: m.Vendor,
|
||||
Capability: m.Capability, SyncedAt: now, DeprecatedAt: m.Deprecated, RetiredAt: m.Retired})
|
||||
}
|
||||
return s.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
|
||||
if err := tx.Where("channel_id = ?", channelID).Delete(&model.AiModelCache{}).Error; err != nil {
|
||||
@@ -601,33 +497,17 @@ func (s *AiGatewayService) replaceModels(ctx context.Context, channelID uint, mo
|
||||
})
|
||||
}
|
||||
|
||||
// loadModelMarks 取渠道内带标记或已验证过的行(OCID → 行),供同步结转与候选过滤。
|
||||
func (s *AiGatewayService) loadModelMarks(ctx context.Context, channelID uint) (map[string]model.AiModelCache, error) {
|
||||
var rows []model.AiModelCache
|
||||
err := s.db.WithContext(ctx).Select("model_ocid", "unusable", "unusable_reason", "checked_at").
|
||||
Where("channel_id = ? AND (unusable = ? OR checked_at IS NOT NULL)", channelID, true).Find(&rows).Error
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("load model marks: %w", err)
|
||||
}
|
||||
marks := make(map[string]model.AiModelCache, len(rows))
|
||||
for _, r := range rows {
|
||||
marks[r.ModelOcid] = r
|
||||
}
|
||||
return marks, nil
|
||||
}
|
||||
|
||||
func (s *AiGatewayService) channelModels(ctx context.Context, channelID uint) ([]model.AiModelCache, error) {
|
||||
var rows []model.AiModelCache
|
||||
err := s.db.WithContext(ctx).Where("channel_id = ?", channelID).Order("name ASC").Find(&rows).Error
|
||||
return rows, err
|
||||
}
|
||||
|
||||
// GatewayModels 聚合启用渠道的可用模型(按名称去重,剔除不可按需调用标记),
|
||||
// 供 /ai/v1/models;group 非空时仅聚合该分组渠道(与密钥分组路由口径一致)。
|
||||
// GatewayModels 聚合启用渠道的可用模型(按名称去重),供 /ai/v1/models;
|
||||
// group 非空时仅聚合该分组渠道(与密钥分组路由口径一致)。
|
||||
func (s *AiGatewayService) GatewayModels(ctx context.Context, group string) (aiwire.ModelList, error) {
|
||||
q := s.db.WithContext(ctx).
|
||||
Joins("JOIN ai_channels ON ai_channels.id = ai_model_caches.channel_id AND ai_channels.enabled = ?", true).
|
||||
Where("(ai_model_caches.unusable = ? OR ai_model_caches.unusable IS NULL)", false)
|
||||
Joins("JOIN ai_channels ON ai_channels.id = ai_model_caches.channel_id AND ai_channels.enabled = ?", true)
|
||||
if group != "" {
|
||||
q = q.Where("ai_channels.channel_group = ?", group)
|
||||
}
|
||||
@@ -658,7 +538,6 @@ func (s *AiGatewayService) DeprecatingModels(ctx context.Context, within time.Du
|
||||
err := s.db.WithContext(ctx).
|
||||
Where("(retired_at IS NOT NULL AND retired_at > ? AND retired_at <= ?) OR (deprecated_at IS NOT NULL AND deprecated_at >= ? AND deprecated_at <= ?)",
|
||||
now, deadline, now, deadline).
|
||||
Where("(unusable = ? OR unusable IS NULL)", false).
|
||||
Order("name ASC").Find(&rows).Error
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -679,7 +558,7 @@ func (s *AiGatewayService) DeprecatingModels(ctx context.Context, within time.Du
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// ProbeAll 逐个探测全部渠道并顺带验证模型池,返回状态汇总;供 AI 探测后台任务调用。
|
||||
// ProbeAll 逐个探测全部渠道,返回状态汇总;供 AI 探测后台任务调用。
|
||||
func (s *AiGatewayService) ProbeAll(ctx context.Context) (string, error) {
|
||||
chs, err := s.Channels(ctx)
|
||||
if err != nil {
|
||||
@@ -695,8 +574,6 @@ func (s *AiGatewayService) ProbeAll(ctx context.Context) (string, error) {
|
||||
continue
|
||||
}
|
||||
counts[fresh.ProbeStatus]++
|
||||
// 后台任务里同步执行:新模型验证入池、坏模型定期复检,零人工收敛
|
||||
s.validateChannelModels(ctx, ch.ID)
|
||||
}
|
||||
msg := fmt.Sprintf("probed %d: %d ok, %d no_service, %d no_quota, %d error",
|
||||
len(chs), counts["ok"], counts["no_service"], counts["no_quota"], counts["error"])
|
||||
@@ -706,6 +583,99 @@ func (s *AiGatewayService) ProbeAll(ctx context.Context) (string, error) {
|
||||
return msg, nil
|
||||
}
|
||||
|
||||
// ---- 模型黑名单 ----
|
||||
|
||||
// Blacklist 列出全部黑名单模型(按名称排序)。
|
||||
func (s *AiGatewayService) Blacklist(ctx context.Context) ([]model.AiModelBlacklist, error) {
|
||||
var rows []model.AiModelBlacklist
|
||||
err := s.db.WithContext(ctx).Order("name ASC").Find(&rows).Error
|
||||
return rows, err
|
||||
}
|
||||
|
||||
// AddBlacklist 把模型名加入黑名单并删除全部渠道缓存中的同名条目;
|
||||
// 该模型此后同步 / 探测均被过滤,直到移出黑名单后重新同步。
|
||||
func (s *AiGatewayService) AddBlacklist(ctx context.Context, name string) (*model.AiModelBlacklist, error) {
|
||||
name = strings.TrimSpace(name)
|
||||
if name == "" {
|
||||
return nil, fmt.Errorf("模型名不能为空")
|
||||
}
|
||||
row := model.AiModelBlacklist{Name: name}
|
||||
err := s.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
|
||||
var n int64
|
||||
if err := tx.Model(&model.AiModelBlacklist{}).Where("name = ?", name).Count(&n).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
if n > 0 {
|
||||
return fmt.Errorf("模型已在黑名单中")
|
||||
}
|
||||
if err := tx.Create(&row).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Where("name = ?", name).Delete(&model.AiModelCache{}).Error
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &row, nil
|
||||
}
|
||||
|
||||
// RemoveBlacklist 把模型移出黑名单;缓存不回填,下次同步 / 探测自然恢复入池。
|
||||
func (s *AiGatewayService) RemoveBlacklist(ctx context.Context, id uint) error {
|
||||
res := s.db.WithContext(ctx).Delete(&model.AiModelBlacklist{}, id)
|
||||
if res.Error != nil {
|
||||
return res.Error
|
||||
}
|
||||
if res.RowsAffected == 0 {
|
||||
return fmt.Errorf("黑名单条目不存在")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// withoutBlacklisted 过滤掉黑名单中的模型,同步与探测入库前统一经此收口。
|
||||
// supportedGatewayModels 过滤模型目录:对话模型仅保留实测支持 OpenAI 兼容面的
|
||||
// 厂商(xai / meta / openai)——typed chat 面已剔除,google / cohere 对话模型无
|
||||
// 上游通路,不入目录(不出现在列表、路由与探测候选);EMBEDDING 模型不受影响。
|
||||
func supportedGatewayModels(models []oci.GenAiModel) []oci.GenAiModel {
|
||||
out := make([]oci.GenAiModel, 0, len(models))
|
||||
for _, m := range models {
|
||||
if m.Capability == "CHAT" && !compatChatVendor(m.Name) {
|
||||
continue
|
||||
}
|
||||
out = append(out, m)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func compatChatVendor(model string) bool {
|
||||
for _, prefix := range []string{"xai.", "meta.", "openai."} {
|
||||
if strings.HasPrefix(model, prefix) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (s *AiGatewayService) withoutBlacklisted(ctx context.Context, models []oci.GenAiModel) ([]oci.GenAiModel, error) {
|
||||
var names []string
|
||||
if err := s.db.WithContext(ctx).Model(&model.AiModelBlacklist{}).Pluck("name", &names).Error; err != nil {
|
||||
return nil, fmt.Errorf("读取模型黑名单: %w", err)
|
||||
}
|
||||
if len(names) == 0 {
|
||||
return models, nil
|
||||
}
|
||||
banned := make(map[string]bool, len(names))
|
||||
for _, n := range names {
|
||||
banned[n] = true
|
||||
}
|
||||
out := make([]oci.GenAiModel, 0, len(models))
|
||||
for _, m := range models {
|
||||
if !banned[m.Name] {
|
||||
out = append(out, m)
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// ---- 调用日志 ----
|
||||
|
||||
// LogCall 落一条调用日志(仅元数据与用量,永不含请求 / 响应正文),返回落库 ID 供内容日志关联(失败为 0)。
|
||||
|
||||
@@ -3,6 +3,7 @@ package service
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"math/rand"
|
||||
"time"
|
||||
|
||||
@@ -26,24 +27,25 @@ type aiCandidate struct {
|
||||
modelOcid string
|
||||
}
|
||||
|
||||
// Chat 编排非流式调用:选渠道 → 调用 → 可重试错误换渠道(整请求上限 3 次)。
|
||||
// group 非空时只在同分组渠道内路由(取自调用密钥)。
|
||||
func (s *AiGatewayService) Chat(ctx context.Context, ir aiwire.ChatRequest, group string) (*aiwire.ChatResponse, ChatMeta, error) {
|
||||
// RespPassthrough 编排一次非流式直通调用:选渠道(priority→加权随机)→ 调用 →
|
||||
// 可重试错误换渠道(整请求上限 3 次)并维护熔断;group 非空时只在同分组渠道内路由。
|
||||
// 上游为 OpenAI-compatible /actions/v1/responses(实测可用,无 Oracle 文档合同)。
|
||||
func (s *AiGatewayService) RespPassthrough(ctx context.Context, raw []byte, modelName, group string) ([]byte, ChatMeta, error) {
|
||||
meta := ChatMeta{}
|
||||
excluded := map[uint]bool{}
|
||||
var lastErr error
|
||||
for attempt := 0; attempt < 3; attempt++ {
|
||||
cand, err := s.pick(ctx, ir.Model, group, "CHAT", excluded)
|
||||
cand, err := s.pick(ctx, modelName, group, "CHAT", excluded)
|
||||
if err != nil {
|
||||
return nil, meta, firstErr(lastErr, err)
|
||||
}
|
||||
meta.ChannelID, meta.ChannelName = cand.ch.ID, cand.ch.Name
|
||||
resp, err := s.callOnce(ctx, cand, ir)
|
||||
payload, err := s.passthroughOnce(ctx, cand, raw)
|
||||
if err == nil {
|
||||
s.markSuccess(ctx, cand.ch.ID)
|
||||
return resp, meta, nil
|
||||
return payload, meta, nil
|
||||
}
|
||||
retry, penalize := s.noteCallErr(ctx, cand, err)
|
||||
retry, penalize := switchable(err)
|
||||
if !retry {
|
||||
return nil, meta, err
|
||||
}
|
||||
@@ -57,22 +59,22 @@ func (s *AiGatewayService) Chat(ctx context.Context, ir aiwire.ChatRequest, grou
|
||||
return nil, meta, lastErr
|
||||
}
|
||||
|
||||
func (s *AiGatewayService) callOnce(ctx context.Context, cand *aiCandidate, ir aiwire.ChatRequest) (*aiwire.ChatResponse, error) {
|
||||
func (s *AiGatewayService) passthroughOnce(ctx context.Context, cand *aiCandidate, raw []byte) ([]byte, error) {
|
||||
cred, err := s.configs.credentialsByID(ctx, cand.ch.OciConfigID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return s.client.GenAiChat(ctx, cred, cand.ch.Region, cand.modelOcid, ir)
|
||||
return s.client.GenAiCompatResponses(ctx, cred, cand.ch.Region, raw)
|
||||
}
|
||||
|
||||
// OpenStream 编排流式调用:流建立成功后即绑定渠道,建立失败可换渠道重试。
|
||||
// group 语义与 Chat 相同。
|
||||
func (s *AiGatewayService) OpenStream(ctx context.Context, ir aiwire.ChatRequest, group string) (oci.GenAiStream, ChatMeta, error) {
|
||||
// RespPassthroughStream 编排流式直通:流建立成功即绑定渠道,建立失败按 switchable
|
||||
// 换渠道重试;建立后的中断不重试、不计熔断(与 OpenStream 语义一致)。
|
||||
func (s *AiGatewayService) RespPassthroughStream(ctx context.Context, raw []byte, modelName, group string) (io.ReadCloser, ChatMeta, error) {
|
||||
meta := ChatMeta{}
|
||||
excluded := map[uint]bool{}
|
||||
var lastErr error
|
||||
for attempt := 0; attempt < 3; attempt++ {
|
||||
cand, err := s.pick(ctx, ir.Model, group, "CHAT", excluded)
|
||||
cand, err := s.pick(ctx, modelName, group, "CHAT", excluded)
|
||||
if err != nil {
|
||||
return nil, meta, firstErr(lastErr, err)
|
||||
}
|
||||
@@ -81,12 +83,12 @@ func (s *AiGatewayService) OpenStream(ctx context.Context, ir aiwire.ChatRequest
|
||||
if err != nil {
|
||||
return nil, meta, err
|
||||
}
|
||||
stream, err := s.client.GenAiChatStream(ctx, cred, cand.ch.Region, cand.modelOcid, ir)
|
||||
stream, err := s.client.GenAiCompatResponsesStream(ctx, cred, cand.ch.Region, raw)
|
||||
if err == nil {
|
||||
s.markSuccess(ctx, cand.ch.ID)
|
||||
return stream, meta, nil
|
||||
}
|
||||
retry, penalize := s.noteCallErr(ctx, cand, err)
|
||||
retry, penalize := switchable(err)
|
||||
if !retry {
|
||||
return nil, meta, err
|
||||
}
|
||||
@@ -108,12 +110,11 @@ func firstErr(lastErr, pickErr error) error {
|
||||
return pickErr
|
||||
}
|
||||
|
||||
// noteCallErr 汇总一次调用失败:模型级不可用(微调基座 400 / 实体不存在 404)先标记
|
||||
// 剔除该 (渠道, 模型),换渠道重试且不计熔断——这是模型×区域供给问题而非渠道健康问题;
|
||||
// 429 / 5xx / 网络错误换渠道并计熔断;其余 4xx 直接透传。
|
||||
func (s *AiGatewayService) noteCallErr(ctx context.Context, cand *aiCandidate, err error) (retry, penalize bool) {
|
||||
// switchable 判断调用失败是否换渠道重试、是否计入熔断:模型级不可用(微调基座
|
||||
// 400 / 实体不存在 404)换渠道但不计熔断——这是模型×区域供给问题而非渠道健康
|
||||
// 问题,持久解决靠加入模型黑名单;429 / 5xx / 网络错误换渠道并计熔断;其余 4xx 直接透传。
|
||||
func switchable(err error) (retry, penalize bool) {
|
||||
if oci.IsModelUnavailable(err) {
|
||||
s.markModelUnusable(ctx, cand.ch.ID, cand.modelOcid, oci.CompactError(err))
|
||||
return true, false
|
||||
}
|
||||
if status, ok := oci.ServiceStatus(err); ok {
|
||||
@@ -154,11 +155,10 @@ func (s *AiGatewayService) pick(ctx context.Context, modelName, group, capabilit
|
||||
return &aiCandidate{ch: chosen, modelOcid: ocids[chosen.ID]}, nil
|
||||
}
|
||||
|
||||
// modelChannels 查出提供该模型的渠道 ID 及各自的模型 OCID(剔除不可按需调用标记);
|
||||
// capability=CHAT 时兼容存量空串(加列前只同步对话模型);unusable 需兼容 NULL
|
||||
// (AutoMigrate 加列后、首次重同步前的存量行)。
|
||||
// modelChannels 查出提供该模型的渠道 ID 及各自的模型 OCID;
|
||||
// capability=CHAT 时兼容存量空串(加列前只同步对话模型)。
|
||||
func (s *AiGatewayService) modelChannels(ctx context.Context, modelName, capability string) (map[uint]string, []uint, error) {
|
||||
q := s.db.WithContext(ctx).Where("name = ? AND (unusable = ? OR unusable IS NULL)", modelName, false)
|
||||
q := s.db.WithContext(ctx).Where("name = ?", modelName)
|
||||
if capability == "CHAT" {
|
||||
q = q.Where("capability IN ?", []string{"CHAT", ""})
|
||||
} else {
|
||||
@@ -234,7 +234,7 @@ func (s *AiGatewayService) Embeddings(ctx context.Context, req aiwire.Embeddings
|
||||
s.markSuccess(ctx, cand.ch.ID)
|
||||
return resp, meta, nil
|
||||
}
|
||||
retry, penalize := s.noteCallErr(ctx, cand, err)
|
||||
retry, penalize := switchable(err)
|
||||
if !retry {
|
||||
return nil, meta, err
|
||||
}
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"oci-portal/internal/oci"
|
||||
)
|
||||
|
||||
// TestSupportedGatewayModels 断言 CHAT 目录仅保留兼容面厂商,EMBEDDING 不受影响。
|
||||
func TestSupportedGatewayModels(t *testing.T) {
|
||||
in := []oci.GenAiModel{
|
||||
{Name: "xai.grok-4.3", Capability: "CHAT"},
|
||||
{Name: "meta.llama-4-maverick-17b-128e", Capability: "CHAT"},
|
||||
{Name: "openai.gpt-oss-120b", Capability: "CHAT"},
|
||||
{Name: "google.gemini-2.5-flash", Capability: "CHAT"},
|
||||
{Name: "cohere.command-a-03-2025", Capability: "CHAT"},
|
||||
{Name: "cohere.embed-v4.0", Capability: "EMBEDDING"},
|
||||
}
|
||||
out := supportedGatewayModels(in)
|
||||
names := make([]string, 0, len(out))
|
||||
for _, m := range out {
|
||||
names = append(names, m.Name)
|
||||
}
|
||||
want := "xai.grok-4.3,meta.llama-4-maverick-17b-128e,openai.gpt-oss-120b,cohere.embed-v4.0"
|
||||
if got := strings.Join(names, ","); got != want {
|
||||
t.Fatalf("过滤结果 = %s, want %s", got, want)
|
||||
}
|
||||
}
|
||||
+262
-224
@@ -1,9 +1,11 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
@@ -46,14 +48,45 @@ type gatewayStubClient struct {
|
||||
probeErr error
|
||||
// probeSeq 非空时逐次弹出,模拟按候选依次试调;弹尽后回落 probeCode/probeErr
|
||||
probeSeq []probeResult
|
||||
chatResp *aiwire.ChatResponse
|
||||
chatErrs []error
|
||||
// probedNames 记录试调过的模型名,供断言候选过滤
|
||||
probedNames []string
|
||||
chatCalls int
|
||||
regions []string
|
||||
|
||||
embedVecs [][]float32
|
||||
embedUsage *aiwire.Usage
|
||||
embedErr error
|
||||
|
||||
passPayload []byte
|
||||
passErrs []error
|
||||
passCalls int
|
||||
passRegions []string
|
||||
}
|
||||
|
||||
func (f *gatewayStubClient) GenAiCompatResponses(ctx context.Context, cred oci.Credentials, region string, body []byte) ([]byte, error) {
|
||||
f.passCalls++
|
||||
f.passRegions = append(f.passRegions, region)
|
||||
if len(f.passErrs) > 0 {
|
||||
err := f.passErrs[0]
|
||||
f.passErrs = f.passErrs[1:]
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
return f.passPayload, nil
|
||||
}
|
||||
|
||||
func (f *gatewayStubClient) GenAiCompatResponsesStream(ctx context.Context, cred oci.Credentials, region string, body []byte) (io.ReadCloser, error) {
|
||||
f.passCalls++
|
||||
f.passRegions = append(f.passRegions, region)
|
||||
if len(f.passErrs) > 0 {
|
||||
err := f.passErrs[0]
|
||||
f.passErrs = f.passErrs[1:]
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
return io.NopCloser(bytes.NewReader(f.passPayload)), nil
|
||||
}
|
||||
|
||||
func (f *gatewayStubClient) GenAiEmbed(ctx context.Context, cred oci.Credentials, region, modelOcid string, inputs []string, dimensions *int) ([][]float32, *aiwire.Usage, error) {
|
||||
@@ -71,6 +104,7 @@ type probeResult struct {
|
||||
}
|
||||
|
||||
func (f *gatewayStubClient) GenAiProbeChat(ctx context.Context, cred oci.Credentials, region, modelOcid, modelName string) (int, error) {
|
||||
f.probedNames = append(f.probedNames, modelName)
|
||||
if len(f.probeSeq) > 0 {
|
||||
r := f.probeSeq[0]
|
||||
f.probeSeq = f.probeSeq[1:]
|
||||
@@ -79,23 +113,10 @@ func (f *gatewayStubClient) GenAiProbeChat(ctx context.Context, cred oci.Credent
|
||||
return f.probeCode, f.probeErr
|
||||
}
|
||||
|
||||
func (f *gatewayStubClient) GenAiChat(ctx context.Context, cred oci.Credentials, region, modelOcid string, ir aiwire.ChatRequest) (*aiwire.ChatResponse, error) {
|
||||
f.chatCalls++
|
||||
f.regions = append(f.regions, region)
|
||||
if len(f.chatErrs) > 0 {
|
||||
err := f.chatErrs[0]
|
||||
f.chatErrs = f.chatErrs[1:]
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
return f.chatResp, nil
|
||||
}
|
||||
|
||||
func newTestGateway(t *testing.T, client oci.Client) (*AiGatewayService, *OciConfigService) {
|
||||
t.Helper()
|
||||
svc := newTestService(t, client)
|
||||
if err := svc.db.AutoMigrate(&model.AiKey{}, &model.AiChannel{}, &model.AiModelCache{}, &model.AiCallLog{}); err != nil {
|
||||
if err := svc.db.AutoMigrate(&model.AiKey{}, &model.AiChannel{}, &model.AiModelCache{}, &model.AiModelBlacklist{}, &model.AiCallLog{}); err != nil {
|
||||
t.Fatalf("auto migrate ai tables: %v", err)
|
||||
}
|
||||
return NewAiGatewayService(svc.db, svc, client), svc
|
||||
@@ -105,14 +126,14 @@ func TestAiKeyLifecycle(t *testing.T) {
|
||||
gw, _ := newTestGateway(t, &gatewayStubClient{fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}})
|
||||
ctx := context.Background()
|
||||
|
||||
raw, key, err := gw.CreateKey(ctx, "免费-ai-api-key", "test-api-key", "")
|
||||
raw, key, err := gw.CreateKey(ctx, "免费-ai-api-key", "test-api-key", "", nil)
|
||||
if err != nil || raw != "test-api-key" || key.Tail != "-key" {
|
||||
t.Fatalf("CreateKey 自定义值 = %q %+v, %v", raw, key, err)
|
||||
}
|
||||
if _, _, err := gw.CreateKey(ctx, "short", "abc", ""); err == nil {
|
||||
if _, _, err := gw.CreateKey(ctx, "short", "abc", "", nil); err == nil {
|
||||
t.Error("过短自定义密钥应被拒绝")
|
||||
}
|
||||
auto, _, err := gw.CreateKey(ctx, "auto", "", "")
|
||||
auto, _, err := gw.CreateKey(ctx, "auto", "", "", nil)
|
||||
if err != nil || len(auto) < 40 || auto[:3] != "sk-" {
|
||||
t.Fatalf("CreateKey 随机值 = %q, %v", auto, err)
|
||||
}
|
||||
@@ -124,7 +145,7 @@ func TestAiKeyLifecycle(t *testing.T) {
|
||||
t.Errorf("错误密钥 err = %v", err)
|
||||
}
|
||||
off := false
|
||||
_ = gw.UpdateKey(ctx, key.ID, "", &off, nil)
|
||||
_ = gw.UpdateKey(ctx, key.ID, "", &off, nil, nil)
|
||||
if _, err := gw.VerifyKey(ctx, "test-api-key"); !errors.Is(err, ErrAiKeyInvalid) {
|
||||
t.Errorf("禁用后 VerifyKey err = %v", err)
|
||||
}
|
||||
@@ -187,51 +208,6 @@ func seedChannel(t *testing.T, gw *AiGatewayService, cfgID uint, region string,
|
||||
return ch
|
||||
}
|
||||
|
||||
func TestAiChatRetrySwitchesChannel(t *testing.T) {
|
||||
client := &gatewayStubClient{
|
||||
fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}},
|
||||
chatResp: &aiwire.ChatResponse{Model: "meta.llama-3.3-70b-instruct", Choices: []aiwire.Choice{{Message: aiwire.ChatMessage{Role: "assistant", Content: aiwire.NewTextContent("hi")}, FinishReason: "stop"}}},
|
||||
chatErrs: []error{stubServiceError{status: 429}},
|
||||
}
|
||||
gw, svc := newTestGateway(t, client)
|
||||
cfg := importAliveConfig(t, svc)
|
||||
// 两个同优先级渠道
|
||||
seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1)
|
||||
seedChannel(t, gw, cfg.ID, "us-chicago-1", 1, 1)
|
||||
ctx := context.Background()
|
||||
|
||||
resp, meta, err := gw.Chat(ctx, aiwire.ChatRequest{Model: "meta.llama-3.3-70b-instruct", Messages: []aiwire.ChatMessage{{Role: "user", Content: aiwire.NewTextContent("你好")}}}, "")
|
||||
if err != nil || resp == nil {
|
||||
t.Fatalf("Chat = %v, %v", resp, err)
|
||||
}
|
||||
if meta.Retries != 1 || client.chatCalls != 2 {
|
||||
t.Errorf("应换渠道重试一次: retries=%d calls=%d", meta.Retries, client.chatCalls)
|
||||
}
|
||||
if len(client.regions) != 2 && client.regions[0] == client.regions[1] {
|
||||
t.Errorf("重试未换渠道: %v", client.regions)
|
||||
}
|
||||
// 未知模型
|
||||
if _, _, err := gw.Chat(ctx, aiwire.ChatRequest{Model: "no-such-model"}, ""); !errors.Is(err, ErrAiUnknownModel) {
|
||||
t.Errorf("未知模型 err = %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAiChatNonRetryablePassThrough(t *testing.T) {
|
||||
client := &gatewayStubClient{
|
||||
fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}},
|
||||
chatErrs: []error{stubServiceError{status: 400}},
|
||||
}
|
||||
gw, svc := newTestGateway(t, client)
|
||||
cfg := importAliveConfig(t, svc)
|
||||
seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1)
|
||||
seedChannel(t, gw, cfg.ID, "us-chicago-1", 1, 1)
|
||||
|
||||
_, meta, err := gw.Chat(context.Background(), aiwire.ChatRequest{Model: "meta.llama-3.3-70b-instruct"}, "")
|
||||
if err == nil || meta.Retries != 0 || client.chatCalls != 1 {
|
||||
t.Errorf("400 应直接透传不重试: err=%v retries=%d calls=%d", err, meta.Retries, client.chatCalls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPickPriorityAndBreaker(t *testing.T) {
|
||||
gw, svc := newTestGateway(t, &gatewayStubClient{fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}})
|
||||
cfg := importAliveConfig(t, svc)
|
||||
@@ -283,7 +259,7 @@ func TestMarkFailureBackoff(t *testing.T) {
|
||||
func TestAiGroupRouting(t *testing.T) {
|
||||
client := &gatewayStubClient{
|
||||
fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}},
|
||||
chatResp: &aiwire.ChatResponse{Model: "meta.llama-3.3-70b-instruct", Choices: []aiwire.Choice{{Message: aiwire.ChatMessage{Role: "assistant", Content: aiwire.NewTextContent("hi")}, FinishReason: "stop"}}},
|
||||
passPayload: []byte(`{"id":"resp_1","usage":{"input_tokens":3,"output_tokens":1}}`),
|
||||
}
|
||||
gw, svc := newTestGateway(t, client)
|
||||
cfg := importAliveConfig(t, svc)
|
||||
@@ -292,17 +268,17 @@ func TestAiGroupRouting(t *testing.T) {
|
||||
gw.db.Model(vip).Update("channel_group", "vip")
|
||||
gw.db.Create(&model.AiModelCache{ChannelID: other.ID, ModelOcid: "ocid1..cr", Name: "cohere.command-r", Vendor: "cohere", SyncedAt: time.Now()})
|
||||
ctx := context.Background()
|
||||
req := aiwire.ChatRequest{Model: "meta.llama-3.3-70b-instruct", Messages: []aiwire.ChatMessage{{Role: "user", Content: aiwire.NewTextContent("你好")}}}
|
||||
body := []byte(`{"model":"meta.llama-3.3-70b-instruct","input":"你好"}`)
|
||||
|
||||
// 分组密钥只落同分组渠道
|
||||
for i := 0; i < 5; i++ {
|
||||
_, meta, err := gw.Chat(ctx, req, "vip")
|
||||
_, meta, err := gw.RespPassthrough(ctx, body, "meta.llama-3.3-70b-instruct", "vip")
|
||||
if err != nil || meta.ChannelID != vip.ID {
|
||||
t.Fatalf("vip 组第 %d 次落点 = %d, err %v, want %d", i, meta.ChannelID, err, vip.ID)
|
||||
}
|
||||
}
|
||||
// 分组内无渠道 → 无可用渠道
|
||||
if _, _, err := gw.Chat(ctx, req, "nope"); !errors.Is(err, ErrAiNoChannel) {
|
||||
if _, _, err := gw.RespPassthrough(ctx, body, "meta.llama-3.3-70b-instruct", "nope"); !errors.Is(err, ErrAiNoChannel) {
|
||||
t.Errorf("空分组 err = %v, want ErrAiNoChannel", err)
|
||||
}
|
||||
// 模型列表按分组过滤
|
||||
@@ -315,12 +291,12 @@ func TestAiGroupRouting(t *testing.T) {
|
||||
t.Errorf("不限组模型数 = %d, want 2", len(all.Data))
|
||||
}
|
||||
// 密钥分组落库
|
||||
_, key, err := gw.CreateKey(ctx, "vip-key", "vip-secret-1234", "vip")
|
||||
_, key, err := gw.CreateKey(ctx, "vip-key", "vip-secret-1234", "vip", nil)
|
||||
if err != nil || key.Group != "vip" {
|
||||
t.Fatalf("CreateKey group = %+v, %v", key, err)
|
||||
}
|
||||
empty := ""
|
||||
_ = gw.UpdateKey(ctx, key.ID, "", nil, &empty)
|
||||
_ = gw.UpdateKey(ctx, key.ID, "", nil, &empty, nil)
|
||||
var fresh model.AiKey
|
||||
gw.db.First(&fresh, key.ID)
|
||||
if fresh.Group != "" {
|
||||
@@ -465,7 +441,8 @@ func entityNotFoundErr() stubServiceError {
|
||||
}
|
||||
|
||||
func TestProbeSkipsUnavailableModels(t *testing.T) {
|
||||
// 首选 llama 不可按需调用(基座 400 / 实体 404)→ 标记剔除 → 换候选成功 → 渠道判可用
|
||||
// 首选 llama 不可按需调用(基座 400 / 实体 404)→ 换候选成功 → 渠道判可用;
|
||||
// 模型全量入库不再自动标记,持久剔除交由用户手动拉黑
|
||||
tests := []struct {
|
||||
name string
|
||||
bad probeResult
|
||||
@@ -496,112 +473,16 @@ func TestProbeSkipsUnavailableModels(t *testing.T) {
|
||||
if err != nil || probed.ProbeStatus != "ok" {
|
||||
t.Fatalf("坏候选后应换候选并判可用: %+v, %v", probed, err)
|
||||
}
|
||||
var row model.AiModelCache
|
||||
if err := gw.db.Where("channel_id = ? AND model_ocid = ?", ch.ID, "m1").First(&row).Error; err != nil || !row.Unusable {
|
||||
t.Fatalf("首个坏候选应被标记不可用: %+v, %v", row, err)
|
||||
}
|
||||
// 再次探测:同步保留标记,m1 不再进候选(probeSeq 只需一次 200)
|
||||
client.probeSeq = []probeResult{{200, nil}}
|
||||
probed, err = gw.ProbeChannel(ctx, ch.ID)
|
||||
if err != nil || probed.ProbeStatus != "ok" {
|
||||
t.Fatalf("复测应跳过已标记模型: %+v, %v", probed, err)
|
||||
}
|
||||
var again model.AiModelCache
|
||||
gw.db.Where("channel_id = ? AND model_ocid = ?", ch.ID, "m1").First(&again)
|
||||
if !again.Unusable {
|
||||
t.Error("探测触发的同步应保留不可用标记")
|
||||
}
|
||||
// 手动同步同样保留标记(坏模型不随重新同步复活);近期已检的标记不被后台验证翻转
|
||||
client.probeCode = 200
|
||||
if _, err := gw.SyncModels(ctx, ch.ID); err != nil {
|
||||
t.Fatalf("SyncModels: %v", err)
|
||||
}
|
||||
gw.Wait()
|
||||
var kept model.AiModelCache
|
||||
gw.db.Where("channel_id = ? AND model_ocid = ?", ch.ID, "m1").First(&kept)
|
||||
if !kept.Unusable {
|
||||
t.Error("手动同步不应清除不可用标记")
|
||||
rows, _ := gw.channelModels(ctx, ch.ID)
|
||||
if len(rows) != 3 {
|
||||
t.Errorf("模型应全量入库, got %d", len(rows))
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateModelsAfterSync(t *testing.T) {
|
||||
// 同步后后台验证:坏模型标记剔除、好模型记录已检、其他 4xx 不改状态、非对话模型不试调
|
||||
client := &gatewayStubClient{
|
||||
fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}},
|
||||
models: []oci.GenAiModel{
|
||||
{Ocid: "v1", Name: "meta.llama-4-maverick", Vendor: "meta", Capability: "CHAT"},
|
||||
{Ocid: "v2", Name: "xai.grok-4", Vendor: "xai", Capability: "CHAT"},
|
||||
{Ocid: "v3", Name: "xai.grok-voice-agent", Vendor: "xai", Capability: "CHAT"},
|
||||
{Ocid: "v4", Name: "cohere.embed-v4.0", Vendor: "cohere", Capability: "EMBEDDING"},
|
||||
},
|
||||
probeSeq: []probeResult{{404, entityNotFoundErr()}, {200, nil}, {400, stubServiceError{status: 400}}},
|
||||
}
|
||||
gw, svc := newTestGateway(t, client)
|
||||
cfg := importAliveConfig(t, svc)
|
||||
ctx := context.Background()
|
||||
ch, err := gw.CreateChannel(ctx, ChannelInput{OciConfigID: cfg.ID, Region: "us-ashburn-1"})
|
||||
if err != nil {
|
||||
t.Fatalf("CreateChannel: %v", err)
|
||||
}
|
||||
if _, err := gw.SyncModels(ctx, ch.ID); err != nil {
|
||||
t.Fatalf("SyncModels: %v", err)
|
||||
}
|
||||
gw.Wait()
|
||||
|
||||
want := map[string]struct {
|
||||
unusable bool
|
||||
checked bool
|
||||
}{
|
||||
"v1": {true, true}, // 实体 404 → 标记剔除
|
||||
"v2": {false, true}, // 200 → 可用已检
|
||||
"v3": {false, true}, // 普通 400 → 已检不标记
|
||||
"v4": {false, false}, // EMBEDDING 不试调
|
||||
}
|
||||
rows, _ := gw.channelModels(ctx, ch.ID)
|
||||
for _, r := range rows {
|
||||
w := want[r.ModelOcid]
|
||||
if r.Unusable != w.unusable || (r.CheckedAt != nil) != w.checked {
|
||||
t.Errorf("%s: unusable=%v checked=%v, want %+v", r.ModelOcid, r.Unusable, r.CheckedAt != nil, w)
|
||||
}
|
||||
}
|
||||
list, err := gw.GatewayModels(ctx, "")
|
||||
if err != nil || len(list.Data) != 3 {
|
||||
t.Errorf("网关列表应只剔除被标记的坏模型(余 v2/v3/v4), got %+v, %v", list.Data, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateRecheckUnmarksRecovered(t *testing.T) {
|
||||
// 已标记模型超过复检间隔后重验:恢复供给(200)自动解除标记
|
||||
client := &gatewayStubClient{fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}, probeCode: 200}
|
||||
gw, svc := newTestGateway(t, client)
|
||||
cfg := importAliveConfig(t, svc)
|
||||
ch := seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1)
|
||||
ctx := context.Background()
|
||||
old := time.Now().Add(-25 * time.Hour)
|
||||
gw.db.Model(&model.AiModelCache{}).Where("channel_id = ?", ch.ID).
|
||||
Updates(map[string]any{"unusable": true, "unusable_reason": "x", "checked_at": old})
|
||||
|
||||
gw.validateChannelModels(ctx, ch.ID)
|
||||
var row model.AiModelCache
|
||||
gw.db.Where("channel_id = ?", ch.ID).First(&row)
|
||||
if row.Unusable || row.UnusableReason != "" || row.CheckedAt == nil || !row.CheckedAt.After(old) {
|
||||
t.Errorf("超期复检应解除标记并刷新已检时间: %+v", row)
|
||||
}
|
||||
// 未超期的标记不复检(probeSeq 为空、fallback 200 也不会被消费)
|
||||
fresh := time.Now()
|
||||
gw.db.Model(&model.AiModelCache{}).Where("channel_id = ?", ch.ID).
|
||||
Updates(map[string]any{"unusable": true, "checked_at": fresh})
|
||||
gw.validateChannelModels(ctx, ch.ID)
|
||||
gw.db.Where("channel_id = ?", ch.ID).First(&row)
|
||||
if !row.Unusable {
|
||||
t.Error("未超期的标记不应被复检翻转")
|
||||
}
|
||||
}
|
||||
|
||||
func TestProbeAuth404StillNoQuota(t *testing.T) {
|
||||
// 鉴权类 404(NotAuthorizedOrNotFound)仍属租户级,直接定论 no_quota 且不标记模型
|
||||
// 鉴权类 404(NotAuthorizedOrNotFound)仍属租户级,直接定论 no_quota
|
||||
client := &gatewayStubClient{
|
||||
fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}},
|
||||
models: []oci.GenAiModel{{Ocid: "m1", Name: "meta.llama-3.3-70b-instruct", Vendor: "meta"}},
|
||||
@@ -617,31 +498,26 @@ func TestProbeAuth404StillNoQuota(t *testing.T) {
|
||||
if probed.ProbeStatus != "no_quota" {
|
||||
t.Errorf("鉴权 404 status = %q, want no_quota", probed.ProbeStatus)
|
||||
}
|
||||
var marked int64
|
||||
gw.db.Model(&model.AiModelCache{}).Where("unusable = ?", true).Count(&marked)
|
||||
if marked != 0 {
|
||||
t.Errorf("鉴权 404 不应标记模型, marked=%d", marked)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAiChatFinetuneSwitchesChannelWithoutPenalty(t *testing.T) {
|
||||
// 微调基座 400 换渠道重试成功,且不计入熔断失败
|
||||
client := &gatewayStubClient{
|
||||
fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}},
|
||||
chatResp: &aiwire.ChatResponse{Model: "meta.llama-3.3-70b-instruct", Choices: []aiwire.Choice{{Message: aiwire.ChatMessage{Role: "assistant", Content: aiwire.NewTextContent("hi")}, FinishReason: "stop"}}},
|
||||
chatErrs: []error{finetuneBaseErr()},
|
||||
passPayload: []byte(`{"id":"resp_1","usage":{"input_tokens":3,"output_tokens":1}}`),
|
||||
passErrs: []error{finetuneBaseErr()},
|
||||
}
|
||||
gw, svc := newTestGateway(t, client)
|
||||
cfg := importAliveConfig(t, svc)
|
||||
seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1)
|
||||
seedChannel(t, gw, cfg.ID, "us-chicago-1", 1, 1)
|
||||
|
||||
resp, meta, err := gw.Chat(context.Background(), aiwire.ChatRequest{Model: "meta.llama-3.3-70b-instruct", Messages: []aiwire.ChatMessage{{Role: "user", Content: aiwire.NewTextContent("你好")}}}, "")
|
||||
resp, meta, err := gw.RespPassthrough(context.Background(), []byte(`{}`), "meta.llama-3.3-70b-instruct", "")
|
||||
if err != nil || resp == nil {
|
||||
t.Fatalf("Chat = %v, %v", resp, err)
|
||||
t.Fatalf("RespPassthrough = %v, %v", resp, err)
|
||||
}
|
||||
if meta.Retries != 1 || client.chatCalls != 2 {
|
||||
t.Errorf("应换渠道重试一次: retries=%d calls=%d", meta.Retries, client.chatCalls)
|
||||
if meta.Retries != 1 || client.passCalls != 2 {
|
||||
t.Errorf("应换渠道重试一次: retries=%d calls=%d", meta.Retries, client.passCalls)
|
||||
}
|
||||
var chs []model.AiChannel
|
||||
gw.db.Find(&chs)
|
||||
@@ -650,38 +526,93 @@ func TestAiChatFinetuneSwitchesChannelWithoutPenalty(t *testing.T) {
|
||||
t.Errorf("微调基座 400 不应计入熔断: 渠道 %s failCount=%d", ch.Name, ch.FailCount)
|
||||
}
|
||||
}
|
||||
// 失败渠道的该模型被标记,不再参与路由;成功渠道不受影响
|
||||
region := client.regions[0]
|
||||
var row model.AiModelCache
|
||||
gw.db.Where("model_ocid = ?", "ocid1..m-"+region).First(&row)
|
||||
if !row.Unusable {
|
||||
t.Errorf("失败渠道的模型应被标记不可用: %+v", row)
|
||||
}
|
||||
|
||||
func TestBlacklistLifecycle(t *testing.T) {
|
||||
// 拉黑:全渠道同名缓存删除、列表与路由立即不可见;重复/空名被拒;
|
||||
// 移出黑名单后重新同步恢复入池
|
||||
client := &gatewayStubClient{
|
||||
fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}},
|
||||
models: []oci.GenAiModel{{Ocid: "ocid1..m-eu-frankfurt-1", Name: "meta.llama-3.3-70b-instruct", Vendor: "meta"}},
|
||||
}
|
||||
var usable int64
|
||||
gw.db.Model(&model.AiModelCache{}).Where("unusable = ?", false).Count(&usable)
|
||||
if usable != 1 {
|
||||
t.Errorf("成功渠道模型应保持可用, usable=%d", usable)
|
||||
gw, svc := newTestGateway(t, client)
|
||||
cfg := importAliveConfig(t, svc)
|
||||
ch := seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1)
|
||||
seedChannel(t, gw, cfg.ID, "us-chicago-1", 1, 1)
|
||||
ctx := context.Background()
|
||||
|
||||
item, err := gw.AddBlacklist(ctx, " meta.llama-3.3-70b-instruct ")
|
||||
if err != nil || item.Name != "meta.llama-3.3-70b-instruct" {
|
||||
t.Fatalf("AddBlacklist = %+v, %v", item, err)
|
||||
}
|
||||
var left int64
|
||||
gw.db.Model(&model.AiModelCache{}).Count(&left)
|
||||
if left != 0 {
|
||||
t.Errorf("拉黑应删除全部渠道的同名缓存, 剩 %d", left)
|
||||
}
|
||||
if _, _, err := gw.RespPassthrough(ctx, []byte(`{}`), "meta.llama-3.3-70b-instruct", ""); !errors.Is(err, ErrAiUnknownModel) {
|
||||
t.Errorf("拉黑后路由应按未知模型拒绝: %v", err)
|
||||
}
|
||||
if _, err := gw.AddBlacklist(ctx, "meta.llama-3.3-70b-instruct"); err == nil {
|
||||
t.Error("重复拉黑应被拒绝")
|
||||
}
|
||||
if _, err := gw.AddBlacklist(ctx, " "); err == nil {
|
||||
t.Error("空模型名应被拒绝")
|
||||
}
|
||||
rows, err := gw.Blacklist(ctx)
|
||||
if err != nil || len(rows) != 1 {
|
||||
t.Fatalf("Blacklist = %+v, %v", rows, err)
|
||||
}
|
||||
if err := gw.RemoveBlacklist(ctx, rows[0].ID); err != nil {
|
||||
t.Fatalf("RemoveBlacklist: %v", err)
|
||||
}
|
||||
if err := gw.RemoveBlacklist(ctx, rows[0].ID); err == nil {
|
||||
t.Error("移除不存在的条目应报错")
|
||||
}
|
||||
// 移出后重新同步:模型恢复入池
|
||||
if _, err := gw.SyncModels(ctx, ch.ID); err != nil {
|
||||
t.Fatalf("SyncModels: %v", err)
|
||||
}
|
||||
list, _ := gw.GatewayModels(ctx, "")
|
||||
if len(list.Data) != 1 {
|
||||
t.Errorf("移出黑名单并同步后应恢复入池: %+v", list.Data)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUnusableModelExcludedFromPoolAndRouting(t *testing.T) {
|
||||
// 唯一渠道的模型被标记后:网关列表不再展示,路由按未知模型拒绝,渠道详情仍可见标记
|
||||
gw, svc := newTestGateway(t, &gatewayStubClient{fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}})
|
||||
func TestSyncAndProbeFilterBlacklisted(t *testing.T) {
|
||||
// 同步与探测都过滤黑名单模型:不入库、不进试调候选
|
||||
client := &gatewayStubClient{
|
||||
fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}},
|
||||
models: []oci.GenAiModel{
|
||||
{Ocid: "m1", Name: "meta.llama-3-70b-instruct", Vendor: "meta"},
|
||||
{Ocid: "m2", Name: "cohere.command-a-03-2025", Vendor: "cohere"},
|
||||
},
|
||||
probeCode: 200,
|
||||
}
|
||||
gw, svc := newTestGateway(t, client)
|
||||
cfg := importAliveConfig(t, svc)
|
||||
ch := seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1)
|
||||
ctx := context.Background()
|
||||
|
||||
gw.markModelUnusable(ctx, ch.ID, "ocid1..m-eu-frankfurt-1", "Entity with key … not found")
|
||||
list, err := gw.GatewayModels(ctx, "")
|
||||
if err != nil || len(list.Data) != 0 {
|
||||
t.Errorf("已标记模型不应出现在网关列表: %+v, %v", list.Data, err)
|
||||
if _, err := gw.AddBlacklist(ctx, "meta.llama-3-70b-instruct"); err != nil {
|
||||
t.Fatalf("AddBlacklist: %v", err)
|
||||
}
|
||||
if _, _, err := gw.Chat(ctx, aiwire.ChatRequest{Model: "meta.llama-3.3-70b-instruct"}, ""); !errors.Is(err, ErrAiUnknownModel) {
|
||||
t.Errorf("已标记模型路由应拒绝: %v", err)
|
||||
ch, err := gw.CreateChannel(ctx, ChannelInput{OciConfigID: cfg.ID, Region: "eu-frankfurt-1"})
|
||||
if err != nil {
|
||||
t.Fatalf("CreateChannel: %v", err)
|
||||
}
|
||||
rows, err := gw.channelModels(ctx, ch.ID)
|
||||
if err != nil || len(rows) != 1 || !rows[0].Unusable || rows[0].UnusableReason == "" {
|
||||
t.Errorf("渠道详情应保留标记行便于排查: %+v, %v", rows, err)
|
||||
probed, err := gw.ProbeChannel(ctx, ch.ID)
|
||||
if err != nil || probed.ProbeStatus != "ok" {
|
||||
t.Fatalf("ProbeChannel = %+v, %v", probed, err)
|
||||
}
|
||||
rows, _ := gw.channelModels(ctx, ch.ID)
|
||||
if len(rows) != 1 || rows[0].Name != "cohere.command-a-03-2025" {
|
||||
t.Errorf("探测同步应过滤黑名单模型: %+v", rows)
|
||||
}
|
||||
if len(client.probedNames) != 1 || client.probedNames[0] != "cohere.command-a-03-2025" {
|
||||
t.Errorf("试调候选不应包含黑名单模型: %v", client.probedNames)
|
||||
}
|
||||
models, err := gw.SyncModels(ctx, ch.ID)
|
||||
if err != nil || len(models) != 1 || models[0].Name != "cohere.command-a-03-2025" {
|
||||
t.Errorf("SyncModels 应过滤黑名单模型: %+v, %v", models, err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -745,7 +676,7 @@ func TestAiEmbeddings(t *testing.T) {
|
||||
t.Errorf("chat 模型走 embeddings err = %v, want ErrAiUnknownModel", err)
|
||||
}
|
||||
// embedding 模型名打 chat:同样未知模型
|
||||
if _, _, err := gw.Chat(ctx, aiwire.ChatRequest{Model: "cohere.embed-v4.0", Messages: []aiwire.ChatMessage{{Role: "user", Content: aiwire.NewTextContent("x")}}}, ""); !errors.Is(err, ErrAiUnknownModel) {
|
||||
if _, _, err := gw.RespPassthrough(ctx, []byte(`{}`), "cohere.embed-v4.0", ""); !errors.Is(err, ErrAiUnknownModel) {
|
||||
t.Errorf("embedding 模型走 chat err = %v, want ErrAiUnknownModel", err)
|
||||
}
|
||||
}
|
||||
@@ -756,7 +687,7 @@ func TestAiContentLogSwitch(t *testing.T) {
|
||||
t.Fatalf("migrate content log: %v", err)
|
||||
}
|
||||
ctx := context.Background()
|
||||
_, key, err := gw.CreateKey(ctx, "k1", "content-key-1234", "")
|
||||
_, key, err := gw.CreateKey(ctx, "k1", "content-key-1234", "", nil)
|
||||
if err != nil {
|
||||
t.Fatalf("CreateKey: %v", err)
|
||||
}
|
||||
@@ -793,23 +724,130 @@ func TestAiContentLogSwitch(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestLegacyNullUnusableRowsStayVisible(t *testing.T) {
|
||||
// 升级路径回归:AutoMigrate 加列后、首次重同步前,存量行 unusable 为 NULL,
|
||||
// 网关列表与路由必须照常包含这些行,不能因 unusable = false 过滤而整池消失
|
||||
gw, svc := newTestGateway(t, &gatewayStubClient{fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}})
|
||||
func TestRespPassthroughSwitchesChannel(t *testing.T) {
|
||||
client := &gatewayStubClient{
|
||||
fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}},
|
||||
passPayload: []byte(`{"id":"resp_1","usage":{"input_tokens":9,"output_tokens":3}}`),
|
||||
passErrs: []error{stubServiceError{status: 429}},
|
||||
}
|
||||
gw, svc := newTestGateway(t, client)
|
||||
cfg := importAliveConfig(t, svc)
|
||||
ch := seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1)
|
||||
ctx := context.Background()
|
||||
if err := gw.db.Exec("UPDATE ai_model_caches SET unusable = NULL WHERE channel_id = ?", ch.ID).Error; err != nil {
|
||||
t.Fatalf("set legacy null: %v", err)
|
||||
}
|
||||
seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1)
|
||||
seedChannel(t, gw, cfg.ID, "us-chicago-1", 1, 1)
|
||||
|
||||
list, err := gw.GatewayModels(ctx, "")
|
||||
if err != nil || len(list.Data) != 1 {
|
||||
t.Errorf("存量 NULL 行应仍在网关列表: %+v, %v", list.Data, err)
|
||||
payload, meta, err := gw.RespPassthrough(context.Background(), []byte(`{}`), "meta.llama-3.3-70b-instruct", "")
|
||||
if err != nil || len(payload) == 0 {
|
||||
t.Fatalf("RespPassthrough = %v, %v", payload, err)
|
||||
}
|
||||
_, ids, err := gw.modelChannels(ctx, "meta.llama-3.3-70b-instruct", "CHAT")
|
||||
if err != nil || len(ids) != 1 {
|
||||
t.Errorf("存量 NULL 行应仍参与路由: %v, %v", ids, err)
|
||||
if meta.Retries != 1 || client.passCalls != 2 {
|
||||
t.Errorf("应换渠道重试一次: retries=%d calls=%d", meta.Retries, client.passCalls)
|
||||
}
|
||||
if len(client.passRegions) == 2 && client.passRegions[0] == client.passRegions[1] {
|
||||
t.Errorf("重试未换渠道: %v", client.passRegions)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRespPassthroughNonRetryable(t *testing.T) {
|
||||
client := &gatewayStubClient{
|
||||
fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}},
|
||||
passErrs: []error{stubServiceError{status: 400}},
|
||||
}
|
||||
gw, svc := newTestGateway(t, client)
|
||||
cfg := importAliveConfig(t, svc)
|
||||
seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1)
|
||||
seedChannel(t, gw, cfg.ID, "us-chicago-1", 1, 1)
|
||||
|
||||
_, meta, err := gw.RespPassthrough(context.Background(), []byte(`{}`), "meta.llama-3.3-70b-instruct", "")
|
||||
if err == nil || meta.Retries != 0 || client.passCalls != 1 {
|
||||
t.Errorf("400 应直接透传不重试: err=%v retries=%d calls=%d", err, meta.Retries, client.passCalls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNormalizeKeyModels(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
in []string
|
||||
want []string
|
||||
}{
|
||||
{"nil 输入", nil, nil},
|
||||
{"全空串", []string{"", " "}, nil},
|
||||
{"trim 与保序去重", []string{" a ", "b", "a", "", "b "}, []string{"a", "b"}},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got := normalizeKeyModels(tt.in)
|
||||
if len(got) != len(tt.want) {
|
||||
t.Fatalf("normalizeKeyModels(%v) = %v, want %v", tt.in, got, tt.want)
|
||||
}
|
||||
for i := range tt.want {
|
||||
if got[i] != tt.want[i] {
|
||||
t.Fatalf("normalizeKeyModels(%v) = %v, want %v", tt.in, got, tt.want)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestAiKeyModelsPersistence 钉住 serializer:json 字段经 Create 与 Updates(map) 两条路径的往返。
|
||||
func TestAiKeyModelsPersistence(t *testing.T) {
|
||||
gw, _ := newTestGateway(t, &gatewayStubClient{fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}}})
|
||||
ctx := context.Background()
|
||||
|
||||
_, key, err := gw.CreateKey(ctx, "m-key", "model-key-1234", "", []string{" x-model ", "x-model", "y-model", ""})
|
||||
if err != nil {
|
||||
t.Fatalf("CreateKey: %v", err)
|
||||
}
|
||||
fresh, err := gw.VerifyKey(ctx, "model-key-1234")
|
||||
if err != nil || len(fresh.Models) != 2 || fresh.Models[0] != "x-model" || fresh.Models[1] != "y-model" {
|
||||
t.Fatalf("创建后读回 models = %v, %v", fresh.Models, err)
|
||||
}
|
||||
set := []string{"z-model"}
|
||||
if err := gw.UpdateKey(ctx, key.ID, "", nil, nil, &set); err != nil {
|
||||
t.Fatalf("UpdateKey 覆盖: %v", err)
|
||||
}
|
||||
fresh, _ = gw.VerifyKey(ctx, "model-key-1234")
|
||||
if len(fresh.Models) != 1 || fresh.Models[0] != "z-model" {
|
||||
t.Fatalf("覆盖后 models = %v", fresh.Models)
|
||||
}
|
||||
if err := gw.UpdateKey(ctx, key.ID, "renamed", nil, nil, nil); err != nil {
|
||||
t.Fatalf("UpdateKey 不传 models: %v", err)
|
||||
}
|
||||
fresh, _ = gw.VerifyKey(ctx, "model-key-1234")
|
||||
if len(fresh.Models) != 1 {
|
||||
t.Fatalf("未传 models 却被改动: %v", fresh.Models)
|
||||
}
|
||||
empty := []string{}
|
||||
if err := gw.UpdateKey(ctx, key.ID, "", nil, nil, &empty); err != nil {
|
||||
t.Fatalf("UpdateKey 清空: %v", err)
|
||||
}
|
||||
fresh, _ = gw.VerifyKey(ctx, "model-key-1234")
|
||||
if len(fresh.Models) != 0 {
|
||||
t.Fatalf("清空后 models = %v, want 空", fresh.Models)
|
||||
}
|
||||
}
|
||||
|
||||
// TestRespPassthroughStreamSwitchesChannel 断言流式直通建立失败按 switchable 换渠道。
|
||||
func TestRespPassthroughStreamSwitchesChannel(t *testing.T) {
|
||||
client := &gatewayStubClient{
|
||||
fakeClient: &fakeClient{tenancy: oci.TenancyInfo{Name: "t"}},
|
||||
passPayload: []byte("data: {\"type\":\"response.completed\"}\n\n"),
|
||||
passErrs: []error{stubServiceError{status: 503}},
|
||||
}
|
||||
gw, svc := newTestGateway(t, client)
|
||||
cfg := importAliveConfig(t, svc)
|
||||
seedChannel(t, gw, cfg.ID, "eu-frankfurt-1", 1, 1)
|
||||
seedChannel(t, gw, cfg.ID, "us-chicago-1", 1, 1)
|
||||
|
||||
stream, meta, err := gw.RespPassthroughStream(context.Background(), []byte(`{}`), "meta.llama-3.3-70b-instruct", "")
|
||||
if err != nil || stream == nil {
|
||||
t.Fatalf("RespPassthroughStream = %v", err)
|
||||
}
|
||||
defer stream.Close()
|
||||
if meta.Retries != 1 || client.passCalls != 2 {
|
||||
t.Errorf("应换渠道重试一次: retries=%d calls=%d", meta.Retries, client.passCalls)
|
||||
}
|
||||
payload, _ := io.ReadAll(stream)
|
||||
if !bytes.Contains(payload, []byte("response.completed")) {
|
||||
t.Errorf("流内容未透传: %s", payload)
|
||||
}
|
||||
}
|
||||
|
||||
+60
-205
@@ -8,38 +8,8 @@ import (
|
||||
"oci-portal/internal/aiwire"
|
||||
)
|
||||
|
||||
// ResponsesToIR 把 Responses API 请求转换为网关 IR(OpenAI Chat 形态)。
|
||||
// 有状态特性与内置工具不支持,直接报错(API 层 400)。
|
||||
func ResponsesToIR(req aiwire.RespRequest) (aiwire.ChatRequest, error) {
|
||||
if err := respRejectUnsupported(req); err != nil {
|
||||
return aiwire.ChatRequest{}, err
|
||||
}
|
||||
ir := aiwire.ChatRequest{
|
||||
Model: req.Model,
|
||||
MaxTokens: req.MaxOutputTokens,
|
||||
Temperature: req.Temperature,
|
||||
TopP: req.TopP,
|
||||
Stream: req.Stream,
|
||||
ToolChoice: respToolChoice(req.ToolChoice),
|
||||
}
|
||||
if req.Reasoning != nil {
|
||||
ir.ReasoningEffort = req.Reasoning.Effort
|
||||
}
|
||||
if req.Instructions != "" {
|
||||
ir.Messages = append(ir.Messages, aiwire.ChatMessage{Role: "system", Content: aiwire.NewTextContent(req.Instructions)})
|
||||
}
|
||||
msgs, err := respInputToMessages(req.Input)
|
||||
if err != nil {
|
||||
return aiwire.ChatRequest{}, err
|
||||
}
|
||||
ir.Messages = append(ir.Messages, msgs...)
|
||||
ir.Tools = respTools(req.Tools)
|
||||
ir.ResponseFormat = respFormat(req.Text)
|
||||
return ir, nil
|
||||
}
|
||||
|
||||
// respRejectUnsupported 拒绝无状态网关无法承接的请求特性。
|
||||
func respRejectUnsupported(req aiwire.RespRequest) error {
|
||||
// respRejectStateful 拒绝有状态特性(网关无状态)。
|
||||
func respRejectStateful(req aiwire.RespRequest) error {
|
||||
if req.PreviousResponseID != "" {
|
||||
return fmt.Errorf("previous_response_id 不支持:网关不保存历史响应,请在 input 中自带完整上下文(store:false 模式)")
|
||||
}
|
||||
@@ -49,195 +19,80 @@ func respRejectUnsupported(req aiwire.RespRequest) error {
|
||||
if req.Background != nil && *req.Background {
|
||||
return fmt.Errorf("background 模式不支持")
|
||||
}
|
||||
for _, t := range req.Tools {
|
||||
if t.Type != "function" {
|
||||
return fmt.Errorf("不支持的工具类型 %q:仅支持 function 工具", t.Type)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// respInputToMessages 把 input(string 或 item 数组)展开为 IR 消息序列。
|
||||
func respInputToMessages(in aiwire.RespInput) ([]aiwire.ChatMessage, error) {
|
||||
if !in.IsArray {
|
||||
if strings.TrimSpace(in.Text) == "" {
|
||||
return nil, fmt.Errorf("input 不能为空")
|
||||
}
|
||||
return []aiwire.ChatMessage{{Role: "user", Content: aiwire.NewTextContent(in.Text)}}, nil
|
||||
}
|
||||
var msgs []aiwire.ChatMessage
|
||||
for _, it := range in.Items {
|
||||
out, err := respItemToMessages(it, msgs)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
msgs = out
|
||||
}
|
||||
return msgs, nil
|
||||
}
|
||||
|
||||
// respItemToMessages 追加一个 input item;连续 function_call 合并进同一 assistant 消息。
|
||||
func respItemToMessages(it aiwire.RespItem, msgs []aiwire.ChatMessage) ([]aiwire.ChatMessage, error) {
|
||||
switch it.Type {
|
||||
case "", "message":
|
||||
if it.Content.HasUnsupported() {
|
||||
return nil, ErrAiUnsupportedBlock
|
||||
}
|
||||
role := respRole(it.Role)
|
||||
return append(msgs, aiwire.ChatMessage{Role: role, Content: respContentToIR(it.Content)}), nil
|
||||
case "function_call":
|
||||
call := aiwire.ToolCall{ID: it.CallID, Type: "function",
|
||||
Function: aiwire.FunctionCall{Name: it.Name, Arguments: it.Arguments}}
|
||||
if n := len(msgs); n > 0 && msgs[n-1].Role == "assistant" && len(msgs[n-1].ToolCalls) > 0 {
|
||||
msgs[n-1].ToolCalls = append(msgs[n-1].ToolCalls, call)
|
||||
return msgs, nil
|
||||
}
|
||||
return append(msgs, aiwire.ChatMessage{Role: "assistant", ToolCalls: []aiwire.ToolCall{call}}), nil
|
||||
case "function_call_output":
|
||||
return append(msgs, aiwire.ChatMessage{Role: "tool", ToolCallID: it.CallID,
|
||||
Content: aiwire.NewTextContent(it.OutputText())}), nil
|
||||
case "reasoning":
|
||||
return msgs, nil // 推理块不回灌上游
|
||||
default:
|
||||
return nil, fmt.Errorf("不支持的 input item 类型 %q", it.Type)
|
||||
}
|
||||
}
|
||||
|
||||
// respRole 归一 item 角色;developer 视为 system,缺省 user。
|
||||
func respRole(role string) string {
|
||||
switch role {
|
||||
case "system", "developer":
|
||||
return "system"
|
||||
case "assistant":
|
||||
return "assistant"
|
||||
default:
|
||||
return "user"
|
||||
}
|
||||
}
|
||||
|
||||
// respContentToIR 把 Responses 内容转 IR;纯文本拍平,含图片时保留部件顺序。
|
||||
func respContentToIR(c aiwire.RespContent) aiwire.Content {
|
||||
var parts []aiwire.ContentPart
|
||||
hasImage := false
|
||||
for _, p := range c.Parts {
|
||||
switch p.Type {
|
||||
case "input_image":
|
||||
parts = append(parts, aiwire.ContentPart{Type: "image_url",
|
||||
ImageURL: &aiwire.ImageURL{URL: p.ImageURL, Detail: p.Detail}})
|
||||
hasImage = true
|
||||
default:
|
||||
if p.Text != "" {
|
||||
parts = append(parts, aiwire.ContentPart{Type: "text", Text: p.Text})
|
||||
}
|
||||
}
|
||||
}
|
||||
if !c.IsArray || !hasImage {
|
||||
return aiwire.NewTextContent(c.JoinText())
|
||||
}
|
||||
return aiwire.NewPartsContent(parts)
|
||||
}
|
||||
|
||||
// respTools 把扁平工具定义还原为嵌套 Chat 形态;strict 无 OCI 对应,忽略。
|
||||
func respTools(tools []aiwire.RespTool) []aiwire.Tool {
|
||||
if len(tools) == 0 {
|
||||
return nil
|
||||
}
|
||||
out := make([]aiwire.Tool, 0, len(tools))
|
||||
// RespServerTools 报告工具列表是否含 xAI 服务端工具(web_search / x_search)。
|
||||
func RespServerTools(tools []aiwire.RespTool) bool {
|
||||
for _, t := range tools {
|
||||
out = append(out, aiwire.Tool{Type: "function",
|
||||
Function: aiwire.FunctionDef{Name: t.Name, Description: t.Description, Parameters: t.Parameters}})
|
||||
if t.Type == "web_search" || t.Type == "x_search" {
|
||||
return true
|
||||
}
|
||||
return out
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// respToolChoice 转换 tool_choice:字符串直通;{type:function,name} 转嵌套;
|
||||
// allowed_tools 等对象形态降级 auto。
|
||||
func respToolChoice(raw json.RawMessage) json.RawMessage {
|
||||
if len(raw) == 0 {
|
||||
return nil
|
||||
// RespPassthroughValidate 校验直通请求:模型必填,有状态特性不支持,工具类型
|
||||
// 只放行 function 与已实测的 web_search / x_search;流式仅在含服务端工具时拒绝
|
||||
// (工具流式事件形态未实测,不放开)。
|
||||
func RespPassthroughValidate(req aiwire.RespRequest) error {
|
||||
if strings.TrimSpace(req.Model) == "" {
|
||||
return fmt.Errorf("model 不能为空")
|
||||
}
|
||||
t := strings.TrimSpace(string(raw))
|
||||
if strings.HasPrefix(t, "\"") {
|
||||
return raw
|
||||
if req.Stream && RespServerTools(req.Tools) {
|
||||
return fmt.Errorf("服务端工具暂不支持流式:请去掉 stream 或改用 function 工具")
|
||||
}
|
||||
var obj struct {
|
||||
Type string `json:"type"`
|
||||
Name string `json:"name"`
|
||||
if err := respRejectStateful(req); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := json.Unmarshal(raw, &obj); err != nil || obj.Type != "function" || obj.Name == "" {
|
||||
return json.RawMessage(`"auto"`)
|
||||
}
|
||||
out, _ := json.Marshal(map[string]any{"type": "function", "function": map[string]string{"name": obj.Name}})
|
||||
return out
|
||||
}
|
||||
|
||||
// respFormat 把 text.format 转成 response_format;text 形态无需显式指定。
|
||||
func respFormat(text *aiwire.RespText) *aiwire.ResponseFormat {
|
||||
if text == nil || text.Format == nil {
|
||||
return nil
|
||||
}
|
||||
switch text.Format.Type {
|
||||
case "json_object":
|
||||
return &aiwire.ResponseFormat{Type: "json_object"}
|
||||
case "json_schema":
|
||||
spec, _ := json.Marshal(map[string]any{
|
||||
"name": text.Format.Name, "schema": json.RawMessage(text.Format.Schema), "strict": text.Format.Strict,
|
||||
})
|
||||
return &aiwire.ResponseFormat{Type: "json_schema", JSONSchema: spec}
|
||||
for _, t := range req.Tools {
|
||||
switch t.Type {
|
||||
case "function", "web_search", "x_search":
|
||||
default:
|
||||
return fmt.Errorf("不支持的工具类型 %q:服务端工具仅支持 web_search / x_search", t.Type)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// RespPassthroughBody 以原始请求体为基构造上游 body:强制 store:false(禁上游
|
||||
// 存态),stream 原样保留(流式直通);用 json.Number 保真未知字段与数值。
|
||||
func RespPassthroughBody(raw []byte) ([]byte, error) {
|
||||
dec := json.NewDecoder(strings.NewReader(string(raw)))
|
||||
dec.UseNumber()
|
||||
var body map[string]any
|
||||
if err := dec.Decode(&body); err != nil {
|
||||
return nil, fmt.Errorf("解析请求体: %w", err)
|
||||
}
|
||||
body["store"] = false
|
||||
return json.Marshal(body)
|
||||
}
|
||||
|
||||
// RespPassthroughUsage 从直通响应提取用量;缺失时返回 nil(日志记零)。
|
||||
func RespPassthroughUsage(payload []byte) *aiwire.Usage {
|
||||
var root struct {
|
||||
Usage *aiwire.RespUsage `json:"usage"`
|
||||
}
|
||||
if json.Unmarshal(payload, &root) != nil || root.Usage == nil {
|
||||
return nil
|
||||
}
|
||||
usage := &aiwire.Usage{PromptTokens: root.Usage.InputTokens,
|
||||
CompletionTokens: root.Usage.OutputTokens, TotalTokens: root.Usage.TotalTokens}
|
||||
if cached := root.Usage.InputTokensDetails.CachedTokens; cached > 0 {
|
||||
usage.PromptTokensDetails = &aiwire.PromptTokensDetails{CachedTokens: cached}
|
||||
}
|
||||
return usage
|
||||
}
|
||||
|
||||
// IRRespToResponses 把 IR 非流式响应装配为 Response 对象。
|
||||
func IRRespToResponses(resp *aiwire.ChatResponse, id string, created int64) aiwire.Response {
|
||||
out := aiwire.Response{ID: id, Object: "response", CreatedAt: created,
|
||||
Status: "completed", Model: resp.Model, Output: []aiwire.RespOutItem{}, Store: false}
|
||||
if len(resp.Choices) == 0 {
|
||||
return out
|
||||
// RespStreamCompletedUsage 从一行 SSE data JSON 中提取 response.completed 事件的
|
||||
// usage;非 completed 事件或解析失败返回 nil。流式直通逐行喂入,最后一次非 nil 生效。
|
||||
func RespStreamCompletedUsage(data []byte) *aiwire.Usage {
|
||||
var ev struct {
|
||||
Type string `json:"type"`
|
||||
Response json.RawMessage `json:"response"`
|
||||
}
|
||||
choice := resp.Choices[0]
|
||||
if text := choice.Message.Content.JoinText(); text != "" {
|
||||
out.Output = append(out.Output, respMessageItem(id, 0, text))
|
||||
}
|
||||
for i, tc := range choice.Message.ToolCalls {
|
||||
out.Output = append(out.Output, aiwire.RespOutItem{Type: "function_call",
|
||||
ID: respItemID(id, "fc", len(out.Output)+i), Status: "completed",
|
||||
CallID: tc.ID, Name: tc.Function.Name, Arguments: respArgs(tc.Function.Arguments)})
|
||||
}
|
||||
if choice.FinishReason == "length" {
|
||||
out.Status = "incomplete"
|
||||
out.IncompleteDetails = &aiwire.RespIncomplete{Reason: "max_output_tokens"}
|
||||
}
|
||||
out.Usage = respUsage(resp.Usage)
|
||||
return out
|
||||
}
|
||||
|
||||
func respMessageItem(respID string, idx int, text string) aiwire.RespOutItem {
|
||||
return aiwire.RespOutItem{Type: "message", ID: respItemID(respID, "msg", idx),
|
||||
Status: "completed", Role: "assistant",
|
||||
Content: []aiwire.RespOutPart{{Type: "output_text", Text: text, Annotations: []any{}}}}
|
||||
}
|
||||
|
||||
// respItemID 从响应 ID 派生确定性 item ID。
|
||||
func respItemID(respID, kind string, idx int) string {
|
||||
return fmt.Sprintf("%s_%s_%d", kind, strings.TrimPrefix(respID, "resp_"), idx)
|
||||
}
|
||||
|
||||
// respArgs 保证 arguments 是合法 JSON 字符串(空实参回退 {})。
|
||||
func respArgs(args string) string {
|
||||
if strings.TrimSpace(args) == "" {
|
||||
return "{}"
|
||||
}
|
||||
return args
|
||||
}
|
||||
|
||||
// respUsage 把 IR 用量改写为 Responses 命名口径。
|
||||
func respUsage(u *aiwire.Usage) *aiwire.RespUsage {
|
||||
if u == nil {
|
||||
if json.Unmarshal(data, &ev) != nil || ev.Type != "response.completed" || len(ev.Response) == 0 {
|
||||
return nil
|
||||
}
|
||||
return &aiwire.RespUsage{InputTokens: u.PromptTokens, OutputTokens: u.CompletionTokens,
|
||||
TotalTokens: u.TotalTokens,
|
||||
InputTokensDetails: aiwire.RespInDetails{CachedTokens: u.CachedTokens()}}
|
||||
return RespPassthroughUsage(ev.Response)
|
||||
}
|
||||
|
||||
@@ -1,199 +0,0 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"strings"
|
||||
|
||||
"oci-portal/internal/aiwire"
|
||||
)
|
||||
|
||||
// RespEvent 是一条 Responses SSE 语义事件(event 名 + data 载荷)。
|
||||
type RespEvent struct {
|
||||
Event string
|
||||
Data map[string]any
|
||||
}
|
||||
|
||||
// RespStream 把 IR chunk 流聚合为 Responses 语义事件序列。
|
||||
// 与 AnthStream 同构,但终态事件须携带完整 Response 快照,故全程缓冲文本与实参。
|
||||
type RespStream struct {
|
||||
id string
|
||||
model string
|
||||
created int64
|
||||
seq int
|
||||
started bool
|
||||
items []aiwire.RespOutItem
|
||||
msgOpen bool
|
||||
text strings.Builder
|
||||
tool aiwire.RespOutItem
|
||||
toolIdx int
|
||||
tOpen bool
|
||||
args strings.Builder
|
||||
finish string
|
||||
usage *aiwire.Usage
|
||||
}
|
||||
|
||||
// NewRespStream 构造状态机;id 形如 resp_*,created 为响应时间戳。
|
||||
func NewRespStream(id, model string, created int64) *RespStream {
|
||||
return &RespStream{id: id, model: model, created: created}
|
||||
}
|
||||
|
||||
// Feed 消化一个上游 chunk,返回应立即下发的事件。
|
||||
func (s *RespStream) Feed(chunk aiwire.ChatChunk) []RespEvent {
|
||||
var evs []RespEvent
|
||||
if !s.started {
|
||||
s.started = true
|
||||
evs = append(evs, s.respEvent("response.created", "in_progress"),
|
||||
s.respEvent("response.in_progress", "in_progress"))
|
||||
}
|
||||
if chunk.Usage != nil {
|
||||
s.usage = chunk.Usage
|
||||
}
|
||||
if len(chunk.Choices) == 0 {
|
||||
return evs
|
||||
}
|
||||
choice := chunk.Choices[0]
|
||||
if choice.Delta.Content != "" {
|
||||
evs = append(evs, s.feedText(choice.Delta.Content)...)
|
||||
}
|
||||
for _, tc := range choice.Delta.ToolCalls {
|
||||
evs = append(evs, s.feedTool(tc)...)
|
||||
}
|
||||
if choice.FinishReason != nil {
|
||||
s.finish = *choice.FinishReason
|
||||
}
|
||||
return evs
|
||||
}
|
||||
|
||||
// Finish 关闭未闭合的块并产出终态事件(带完整 Response 快照)。
|
||||
func (s *RespStream) Finish() []RespEvent {
|
||||
var evs []RespEvent
|
||||
if !s.started {
|
||||
s.started = true
|
||||
evs = append(evs, s.respEvent("response.created", "in_progress"))
|
||||
}
|
||||
evs = append(evs, s.closeMsg()...)
|
||||
evs = append(evs, s.closeTool()...)
|
||||
status := "completed"
|
||||
if s.finish == "length" {
|
||||
status = "incomplete"
|
||||
}
|
||||
return append(evs, s.respEvent("response."+status, status))
|
||||
}
|
||||
|
||||
// Usage 返回聚合到的用量(可能为 nil),供调用日志。
|
||||
func (s *RespStream) Usage() *aiwire.Usage { return s.usage }
|
||||
|
||||
// ev 构造带自增 sequence_number 的事件。
|
||||
func (s *RespStream) ev(typ string, kv map[string]any) RespEvent {
|
||||
s.seq++
|
||||
data := map[string]any{"type": typ, "sequence_number": s.seq}
|
||||
for k, v := range kv {
|
||||
data[k] = v
|
||||
}
|
||||
return RespEvent{Event: typ, Data: data}
|
||||
}
|
||||
|
||||
// respEvent 构造携带 Response 快照的生命周期事件。
|
||||
func (s *RespStream) respEvent(typ, status string) RespEvent {
|
||||
return s.ev(typ, map[string]any{"response": s.snapshot(status)})
|
||||
}
|
||||
|
||||
// snapshot 组装当前累计状态的 Response 对象。
|
||||
func (s *RespStream) snapshot(status string) aiwire.Response {
|
||||
resp := aiwire.Response{ID: s.id, Object: "response", CreatedAt: s.created,
|
||||
Status: status, Model: s.model, Store: false,
|
||||
Output: append([]aiwire.RespOutItem{}, s.items...)}
|
||||
resp.Usage = respUsage(s.usage)
|
||||
if status == "incomplete" {
|
||||
resp.IncompleteDetails = &aiwire.RespIncomplete{Reason: "max_output_tokens"}
|
||||
}
|
||||
return resp
|
||||
}
|
||||
|
||||
// feedText 处理文本增量:必要时开 message item 与 content part。
|
||||
func (s *RespStream) feedText(delta string) []RespEvent {
|
||||
evs := s.closeTool()
|
||||
if !s.msgOpen {
|
||||
s.msgOpen = true
|
||||
s.text.Reset()
|
||||
item := aiwire.RespOutItem{Type: "message", ID: s.itemID("msg"), Status: "in_progress",
|
||||
Role: "assistant", Content: []aiwire.RespOutPart{}}
|
||||
evs = append(evs, s.ev("response.output_item.added", map[string]any{
|
||||
"output_index": len(s.items), "item": item}))
|
||||
evs = append(evs, s.ev("response.content_part.added", map[string]any{
|
||||
"item_id": item.ID, "output_index": len(s.items), "content_index": 0,
|
||||
"part": aiwire.RespOutPart{Type: "output_text", Text: "", Annotations: []any{}}}))
|
||||
}
|
||||
s.text.WriteString(delta)
|
||||
return append(evs, s.ev("response.output_text.delta", map[string]any{
|
||||
"item_id": s.itemID("msg"), "output_index": len(s.items), "content_index": 0, "delta": delta}))
|
||||
}
|
||||
|
||||
// closeMsg 闭合当前 message item(text done → part done → item done)。
|
||||
func (s *RespStream) closeMsg() []RespEvent {
|
||||
if !s.msgOpen {
|
||||
return nil
|
||||
}
|
||||
s.msgOpen = false
|
||||
id, idx, full := s.itemID("msg"), len(s.items), s.text.String()
|
||||
part := aiwire.RespOutPart{Type: "output_text", Text: full, Annotations: []any{}}
|
||||
item := aiwire.RespOutItem{Type: "message", ID: id, Status: "completed",
|
||||
Role: "assistant", Content: []aiwire.RespOutPart{part}}
|
||||
evs := []RespEvent{
|
||||
s.ev("response.output_text.done", map[string]any{"item_id": id, "output_index": idx, "content_index": 0, "text": full}),
|
||||
s.ev("response.content_part.done", map[string]any{"item_id": id, "output_index": idx, "content_index": 0, "part": part}),
|
||||
s.ev("response.output_item.done", map[string]any{"output_index": idx, "item": item}),
|
||||
}
|
||||
s.items = append(s.items, item)
|
||||
return evs
|
||||
}
|
||||
|
||||
// feedTool 处理工具调用增量:新 Index 先闭合旧调用再开新 item。
|
||||
func (s *RespStream) feedTool(tc aiwire.ToolCallDelta) []RespEvent {
|
||||
evs := s.closeMsg()
|
||||
if s.tOpen && tc.Index != s.toolIdx {
|
||||
evs = append(evs, s.closeTool()...)
|
||||
}
|
||||
if !s.tOpen {
|
||||
s.tOpen, s.toolIdx = true, tc.Index
|
||||
s.args.Reset()
|
||||
s.tool = aiwire.RespOutItem{Type: "function_call", ID: s.itemID("fc"), Status: "in_progress",
|
||||
CallID: tc.ID, Name: tc.Function.Name}
|
||||
evs = append(evs, s.ev("response.output_item.added", map[string]any{
|
||||
"output_index": len(s.items), "item": s.tool}))
|
||||
}
|
||||
if tc.ID != "" && s.tool.CallID == "" {
|
||||
s.tool.CallID = tc.ID
|
||||
}
|
||||
if tc.Function.Name != "" && s.tool.Name == "" {
|
||||
s.tool.Name = tc.Function.Name
|
||||
}
|
||||
if tc.Function.Arguments == "" {
|
||||
return evs
|
||||
}
|
||||
s.args.WriteString(tc.Function.Arguments)
|
||||
return append(evs, s.ev("response.function_call_arguments.delta", map[string]any{
|
||||
"item_id": s.tool.ID, "output_index": len(s.items), "delta": tc.Function.Arguments}))
|
||||
}
|
||||
|
||||
// closeTool 闭合当前 function_call item(arguments done → item done)。
|
||||
func (s *RespStream) closeTool() []RespEvent {
|
||||
if !s.tOpen {
|
||||
return nil
|
||||
}
|
||||
s.tOpen = false
|
||||
s.tool.Arguments = respArgs(s.args.String())
|
||||
s.tool.Status = "completed"
|
||||
idx := len(s.items)
|
||||
evs := []RespEvent{
|
||||
s.ev("response.function_call_arguments.done", map[string]any{
|
||||
"item_id": s.tool.ID, "output_index": idx, "arguments": s.tool.Arguments}),
|
||||
s.ev("response.output_item.done", map[string]any{"output_index": idx, "item": s.tool}),
|
||||
}
|
||||
s.items = append(s.items, s.tool)
|
||||
return evs
|
||||
}
|
||||
|
||||
// itemID 按当前 output 序号派生确定性 item ID。
|
||||
func (s *RespStream) itemID(kind string) string {
|
||||
return respItemID(s.id, kind, len(s.items))
|
||||
}
|
||||
@@ -17,47 +17,6 @@ func respReq(t *testing.T, raw string) aiwire.RespRequest {
|
||||
return req
|
||||
}
|
||||
|
||||
func TestResponsesToIR(t *testing.T) {
|
||||
req := respReq(t, `{
|
||||
"model": "meta.llama-3.3-70b-instruct",
|
||||
"instructions": "你是助手",
|
||||
"max_output_tokens": 128,
|
||||
"input": [
|
||||
{"role": "user", "content": "查天气"},
|
||||
{"type": "function_call", "call_id": "call_1", "name": "get_weather", "arguments": "{\"city\":\"上海\"}"},
|
||||
{"type": "function_call_output", "call_id": "call_1", "output": "{\"temp\":31}"},
|
||||
{"type": "message", "role": "assistant", "content": [{"type": "output_text", "text": "31 度"}]}
|
||||
],
|
||||
"tools": [{"type": "function", "name": "get_weather", "description": "查天气", "parameters": {"type": "object"}, "strict": true}],
|
||||
"tool_choice": {"type": "function", "name": "get_weather"},
|
||||
"text": {"format": {"type": "json_schema", "name": "out", "schema": {"type": "object"}}}
|
||||
}`)
|
||||
ir, err := ResponsesToIR(req)
|
||||
if err != nil {
|
||||
t.Fatalf("ResponsesToIR: %v", err)
|
||||
}
|
||||
roles := make([]string, 0, len(ir.Messages))
|
||||
for _, m := range ir.Messages {
|
||||
roles = append(roles, m.Role)
|
||||
}
|
||||
want := []string{"system", "user", "assistant", "tool", "assistant"}
|
||||
if strings.Join(roles, ",") != strings.Join(want, ",") {
|
||||
t.Errorf("roles = %v, want %v", roles, want)
|
||||
}
|
||||
if ir.Messages[2].ToolCalls[0].ID != "call_1" || ir.Messages[3].ToolCallID != "call_1" {
|
||||
t.Errorf("tool call 链接错误: %+v", ir.Messages)
|
||||
}
|
||||
if deref(ir.MaxTokens) != 128 || len(ir.Tools) != 1 || ir.Tools[0].Function.Name != "get_weather" {
|
||||
t.Errorf("参数映射错误: max=%v tools=%+v", ir.MaxTokens, ir.Tools)
|
||||
}
|
||||
if !strings.Contains(string(ir.ToolChoice), `"function"`) || !strings.Contains(string(ir.ToolChoice), "get_weather") {
|
||||
t.Errorf("tool_choice = %s", ir.ToolChoice)
|
||||
}
|
||||
if ir.ResponseFormat == nil || ir.ResponseFormat.Type != "json_schema" {
|
||||
t.Errorf("response_format = %+v", ir.ResponseFormat)
|
||||
}
|
||||
}
|
||||
|
||||
func deref(p *int) int {
|
||||
if p == nil {
|
||||
return 0
|
||||
@@ -65,112 +24,124 @@ func deref(p *int) int {
|
||||
return *p
|
||||
}
|
||||
|
||||
func TestResponsesToIRRejects(t *testing.T) {
|
||||
cases := []struct{ name, raw string }{
|
||||
{"previous_response_id", `{"model":"m","input":"hi","previous_response_id":"resp_x"}`},
|
||||
{"conversation", `{"model":"m","input":"hi","conversation":{"id":"conv_1"}}`},
|
||||
{"background", `{"model":"m","input":"hi","background":true}`},
|
||||
{"builtin tool", `{"model":"m","input":"hi","tools":[{"type":"web_search"}]}`},
|
||||
{"file_id image", `{"model":"m","input":[{"role":"user","content":[{"type":"input_image","file_id":"file_1"}]}]}`},
|
||||
{"unknown item", `{"model":"m","input":[{"type":"item_reference","id":"x"}]}`},
|
||||
func TestRespServerTools(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
tools []aiwire.RespTool
|
||||
want bool
|
||||
}{
|
||||
{"web_search", []aiwire.RespTool{{Type: "web_search"}}, true},
|
||||
{"x_search混用", []aiwire.RespTool{{Type: "function", Name: "f"}, {Type: "x_search"}}, true},
|
||||
{"仅function", []aiwire.RespTool{{Type: "function", Name: "f"}}, false},
|
||||
{"空", nil, false},
|
||||
{"其他类型", []aiwire.RespTool{{Type: "code_interpreter"}}, false},
|
||||
}
|
||||
for _, c := range cases {
|
||||
if _, err := ResponsesToIR(respReq(t, c.raw)); err == nil {
|
||||
t.Errorf("%s 应被拒绝", c.name)
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
if got := RespServerTools(test.tools); got != test.want {
|
||||
t.Fatalf("got %v, want %v", got, test.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
// input 字符串形态 + store/reasoning 忽略项不报错
|
||||
req := respReq(t, `{"model":"m","input":"你好","store":false,"reasoning":{"effort":"low"},"parallel_tool_calls":true}`)
|
||||
ir, err := ResponsesToIR(req)
|
||||
if err != nil || len(ir.Messages) != 1 || ir.Messages[0].Role != "user" || ir.ReasoningEffort != "low" {
|
||||
t.Errorf("字符串 input = %+v, %v", ir.Messages, err)
|
||||
}
|
||||
|
||||
func TestRespPassthroughValidate(t *testing.T) {
|
||||
prev := "resp_1"
|
||||
bg := true
|
||||
tests := []struct {
|
||||
name string
|
||||
req aiwire.RespRequest
|
||||
wantErr bool
|
||||
}{
|
||||
{"合法", aiwire.RespRequest{Model: "m", Tools: []aiwire.RespTool{{Type: "web_search"}}}, false},
|
||||
{"混用function", aiwire.RespRequest{Model: "m", Tools: []aiwire.RespTool{{Type: "web_search"}, {Type: "function", Name: "f"}}}, false},
|
||||
{"缺model", aiwire.RespRequest{Tools: []aiwire.RespTool{{Type: "web_search"}}}, true},
|
||||
{"工具加流式拒绝", aiwire.RespRequest{Model: "m", Stream: true, Tools: []aiwire.RespTool{{Type: "web_search"}}}, true},
|
||||
{"无工具流式放行", aiwire.RespRequest{Model: "m", Stream: true}, false},
|
||||
{"function工具流式放行", aiwire.RespRequest{Model: "m", Stream: true, Tools: []aiwire.RespTool{{Type: "function", Name: "f"}}}, false},
|
||||
{"有状态拒绝", aiwire.RespRequest{Model: "m", PreviousResponseID: prev}, true},
|
||||
{"background拒绝", aiwire.RespRequest{Model: "m", Background: &bg}, true},
|
||||
{"未知工具拒绝", aiwire.RespRequest{Model: "m", Tools: []aiwire.RespTool{{Type: "web_search"}, {Type: "mcp"}}}, true},
|
||||
}
|
||||
// input_image(url 形态)放行并保留图文顺序
|
||||
req2 := respReq(t, `{"model":"m","input":[{"role":"user","content":[{"type":"input_text","text":"看图"},{"type":"input_image","image_url":"https://x/1.png","detail":"low"}]}]}`)
|
||||
ir2, err := ResponsesToIR(req2)
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
err := RespPassthroughValidate(test.req)
|
||||
if (err != nil) != test.wantErr {
|
||||
t.Fatalf("err = %v, wantErr %v", err, test.wantErr)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestRespPassthroughBody(t *testing.T) {
|
||||
raw := []byte(`{"model":"m","input":"hi","stream":true,"store":true,"max_output_tokens":128,"custom_field":{"a":1.5}}`)
|
||||
out, err := RespPassthroughBody(raw)
|
||||
if err != nil {
|
||||
t.Fatalf("input_image 应放行: %v", err)
|
||||
t.Fatalf("RespPassthroughBody: %v", err)
|
||||
}
|
||||
parts := ir2.Messages[0].Content.Parts
|
||||
if len(parts) != 2 || parts[1].Type != "image_url" || parts[1].ImageURL == nil || parts[1].ImageURL.URL != "https://x/1.png" {
|
||||
t.Errorf("parts = %+v", parts)
|
||||
var body map[string]any
|
||||
if err := json.Unmarshal(out, &body); err != nil {
|
||||
t.Fatalf("unmarshal out: %v", err)
|
||||
}
|
||||
if body["store"] != false {
|
||||
t.Errorf("store 应强制 false, got %v", body["store"])
|
||||
}
|
||||
if body["stream"] != true {
|
||||
t.Errorf("stream 应原样保留, got %v", body["stream"])
|
||||
}
|
||||
if string(out) == "" || !strings.Contains(string(out), `"max_output_tokens":128`) {
|
||||
t.Errorf("数值字段应保真: %s", out)
|
||||
}
|
||||
if !strings.Contains(string(out), `"custom_field"`) {
|
||||
t.Errorf("未知字段应保留: %s", out)
|
||||
}
|
||||
if _, err := RespPassthroughBody([]byte("not-json")); err == nil {
|
||||
t.Error("非法 JSON 应报错")
|
||||
}
|
||||
}
|
||||
|
||||
func TestIRRespToResponses(t *testing.T) {
|
||||
resp := &aiwire.ChatResponse{Model: "m", Choices: []aiwire.Choice{{
|
||||
Message: aiwire.ChatMessage{Role: "assistant", Content: aiwire.NewTextContent("你好"),
|
||||
ToolCalls: []aiwire.ToolCall{{ID: "call_9", Type: "function",
|
||||
Function: aiwire.FunctionCall{Name: "f", Arguments: ""}}}},
|
||||
FinishReason: "length",
|
||||
}}, Usage: &aiwire.Usage{PromptTokens: 10, CompletionTokens: 5, TotalTokens: 15}}
|
||||
out := IRRespToResponses(resp, "resp_abc", 1751966400)
|
||||
if out.Status != "incomplete" || out.IncompleteDetails == nil || out.IncompleteDetails.Reason != "max_output_tokens" {
|
||||
t.Errorf("length 应映射 incomplete: %+v", out)
|
||||
func TestRespPassthroughUsage(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
payload string
|
||||
wantNil bool
|
||||
prompt, cached int
|
||||
}{
|
||||
{"完整", `{"usage":{"input_tokens":9,"output_tokens":3,"total_tokens":12,"input_tokens_details":{"cached_tokens":5}}}`, false, 9, 5},
|
||||
{"无细分", `{"usage":{"input_tokens":9,"output_tokens":3,"total_tokens":12}}`, false, 9, 0},
|
||||
{"无usage", `{"id":"resp_1"}`, true, 0, 0},
|
||||
{"非法JSON", `xx`, true, 0, 0},
|
||||
}
|
||||
if len(out.Output) != 2 || out.Output[0].Type != "message" || out.Output[1].Type != "function_call" {
|
||||
t.Fatalf("output = %+v", out.Output)
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
usage := RespPassthroughUsage([]byte(test.payload))
|
||||
if (usage == nil) != test.wantNil {
|
||||
t.Fatalf("usage = %v, wantNil %v", usage, test.wantNil)
|
||||
}
|
||||
if out.Output[0].Content[0].Text != "你好" || out.Output[1].CallID != "call_9" || out.Output[1].Arguments != "{}" {
|
||||
t.Errorf("item 装配错误: %+v", out.Output)
|
||||
if usage == nil {
|
||||
return
|
||||
}
|
||||
if out.Usage.InputTokens != 10 || out.Usage.OutputTokens != 5 || out.Store {
|
||||
t.Errorf("usage/store 错误: %+v", out)
|
||||
if usage.PromptTokens != test.prompt || usage.CachedTokens() != test.cached {
|
||||
t.Fatalf("prompt=%d cached=%d, want %d/%d", usage.PromptTokens, usage.CachedTokens(), test.prompt, test.cached)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func chunkText(text string) aiwire.ChatChunk {
|
||||
return aiwire.ChatChunk{Choices: []aiwire.ChunkChoice{{Delta: aiwire.Delta{Content: text}}}}
|
||||
}
|
||||
|
||||
func TestRespStreamTextAndTool(t *testing.T) {
|
||||
st := NewRespStream("resp_x", "m", 1751966400)
|
||||
var types []string
|
||||
collect := func(evs []RespEvent) {
|
||||
for _, ev := range evs {
|
||||
types = append(types, ev.Event)
|
||||
// TestRespStreamCompletedUsage 断言流式 usage 只从 completed 事件提取。
|
||||
func TestRespStreamCompletedUsage(t *testing.T) {
|
||||
completed := []byte(`{"type":"response.completed","response":{"usage":{"input_tokens":10,"output_tokens":5,"total_tokens":15,"input_tokens_details":{"cached_tokens":4}}}}`)
|
||||
if u := RespStreamCompletedUsage(completed); u == nil || u.PromptTokens != 10 || u.CompletionTokens != 5 ||
|
||||
u.PromptTokensDetails == nil || u.PromptTokensDetails.CachedTokens != 4 {
|
||||
t.Fatalf("completed 事件 usage 解析失败: %+v", u)
|
||||
}
|
||||
for _, data := range []string{
|
||||
`{"type":"response.output_text.delta","delta":"hi"}`,
|
||||
`{"type":"response.completed"}`,
|
||||
`not-json`,
|
||||
} {
|
||||
if u := RespStreamCompletedUsage([]byte(data)); u != nil {
|
||||
t.Fatalf("非 completed 或缺 response 应返回 nil: %s", data)
|
||||
}
|
||||
collect(st.Feed(chunkText("你")))
|
||||
collect(st.Feed(chunkText("好")))
|
||||
collect(st.Feed(aiwire.ChatChunk{Choices: []aiwire.ChunkChoice{{Delta: aiwire.Delta{
|
||||
ToolCalls: []aiwire.ToolCallDelta{{Index: 0, ID: "call_1", Function: aiwire.FunctionCallDelta{Name: "f", Arguments: "{\"a\""}}}}}}}))
|
||||
collect(st.Feed(aiwire.ChatChunk{Choices: []aiwire.ChunkChoice{{Delta: aiwire.Delta{
|
||||
ToolCalls: []aiwire.ToolCallDelta{{Index: 0, Function: aiwire.FunctionCallDelta{Arguments: ":1}"}}}}}},
|
||||
Usage: &aiwire.Usage{PromptTokens: 3, CompletionTokens: 7, TotalTokens: 10}}))
|
||||
finish := st.Finish()
|
||||
collect(finish)
|
||||
want := []string{
|
||||
"response.created", "response.in_progress",
|
||||
"response.output_item.added", "response.content_part.added", "response.output_text.delta",
|
||||
"response.output_text.delta",
|
||||
"response.output_text.done", "response.content_part.done", "response.output_item.done",
|
||||
"response.output_item.added", "response.function_call_arguments.delta",
|
||||
"response.function_call_arguments.delta",
|
||||
"response.function_call_arguments.done", "response.output_item.done",
|
||||
"response.completed",
|
||||
}
|
||||
if strings.Join(types, "\n") != strings.Join(want, "\n") {
|
||||
t.Errorf("事件序列 =\n%s\nwant\n%s", strings.Join(types, "\n"), strings.Join(want, "\n"))
|
||||
}
|
||||
last := finish[len(finish)-1].Data
|
||||
resp := last["response"].(aiwire.Response)
|
||||
if len(resp.Output) != 2 || resp.Output[0].Content[0].Text != "你好" || resp.Output[1].Arguments != "{\"a\":1}" {
|
||||
t.Errorf("终态快照 = %+v", resp.Output)
|
||||
}
|
||||
if resp.Usage == nil || resp.Usage.TotalTokens != 10 {
|
||||
t.Errorf("终态 usage = %+v", resp.Usage)
|
||||
}
|
||||
// sequence_number 严格递增
|
||||
if seq, ok := last["sequence_number"].(int); !ok || seq != len(want) {
|
||||
t.Errorf("最终 sequence_number = %v, want %d", last["sequence_number"], len(want))
|
||||
}
|
||||
}
|
||||
|
||||
func TestRespStreamEmpty(t *testing.T) {
|
||||
st := NewRespStream("resp_e", "m", 1)
|
||||
evs := st.Finish()
|
||||
if len(evs) != 2 || evs[0].Event != "response.created" || evs[1].Event != "response.completed" {
|
||||
t.Errorf("空流事件 = %+v", evs)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,411 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"oci-portal/internal/aiwire"
|
||||
)
|
||||
|
||||
// Anthropic Messages ↔ OCI OpenAI 兼容面(/actions/v1/responses)直通转换。
|
||||
// typed chat 面剔除后 Messages 入口的唯一上游通路。语义损失(README 已披露):
|
||||
// stop_sequences / top_k / metadata / thinking 无对应字段,忽略;上游 reasoning
|
||||
// 输出项与增量事件丢弃(Anthropic thinking 块含签名语义,不伪造)。
|
||||
|
||||
// ErrAiUnsupportedBlock 表示请求含网关无法承接的内容块(仅支持文本/图片/工具块)。
|
||||
var ErrAiUnsupportedBlock = fmt.Errorf("暂不支持文本与图片以外的内容块")
|
||||
|
||||
// AnthropicToResponsesBody 把 Messages 请求转为直通 body(强制 store:false)。
|
||||
func AnthropicToResponsesBody(req aiwire.MessagesRequest) ([]byte, error) {
|
||||
input, err := anthInputItems(req.Messages)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
body := map[string]any{"model": req.Model, "max_output_tokens": req.MaxTokens,
|
||||
"input": input, "store": false}
|
||||
if sys := req.SystemText(); sys != "" {
|
||||
body["instructions"] = sys
|
||||
}
|
||||
if req.Temperature != nil {
|
||||
body["temperature"] = *req.Temperature
|
||||
}
|
||||
if req.TopP != nil {
|
||||
body["top_p"] = *req.TopP
|
||||
}
|
||||
if tools := anthRespTools(req.Tools); tools != nil {
|
||||
body["tools"] = tools
|
||||
}
|
||||
if tc := anthRespToolChoice(req.ToolChoice); tc != nil {
|
||||
body["tool_choice"] = tc
|
||||
}
|
||||
if req.OutputConfig != nil && req.OutputConfig.Effort != "" {
|
||||
body["reasoning"] = map[string]string{"effort": strings.ToLower(req.OutputConfig.Effort)}
|
||||
}
|
||||
if req.Stream {
|
||||
body["stream"] = true
|
||||
}
|
||||
return json.Marshal(body)
|
||||
}
|
||||
|
||||
// anthInputItems 把消息序列展开为 Responses input 项:tool_use / tool_result 为
|
||||
// 独立 function_call / function_call_output 项,其余聚合为 message 项;遇独立项时
|
||||
// 先冲刷已聚合部件,保持块间相对顺序。
|
||||
func anthInputItems(messages []aiwire.AnthMessage) ([]any, error) {
|
||||
var items []any
|
||||
for _, m := range messages {
|
||||
var parts []map[string]any
|
||||
flush := func() {
|
||||
if len(parts) > 0 {
|
||||
items = append(items, map[string]any{"role": m.Role, "content": parts})
|
||||
parts = nil
|
||||
}
|
||||
}
|
||||
for _, b := range m.Content.AllBlocks() {
|
||||
item, part, err := anthBlockItem(m.Role, b)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if part != nil {
|
||||
parts = append(parts, part)
|
||||
}
|
||||
if item != nil {
|
||||
flush()
|
||||
items = append(items, item)
|
||||
}
|
||||
}
|
||||
flush()
|
||||
}
|
||||
return items, nil
|
||||
}
|
||||
|
||||
// anthBlockItem 把单个内容块转为独立项或消息部件(thinking 忽略,未知块拒绝)。
|
||||
func anthBlockItem(role string, b aiwire.AnthBlock) (item, part map[string]any, err error) {
|
||||
switch b.Type {
|
||||
case "text":
|
||||
return nil, anthTextPart(role, b.Text), nil
|
||||
case "image":
|
||||
part, err = anthImageInput(b.Source)
|
||||
return nil, part, err
|
||||
case "tool_use":
|
||||
return map[string]any{"type": "function_call", "call_id": b.ID,
|
||||
"name": b.Name, "arguments": string(b.Input)}, nil, nil
|
||||
case "tool_result":
|
||||
return map[string]any{"type": "function_call_output",
|
||||
"call_id": b.ToolUseID, "output": b.ResultText()}, nil, nil
|
||||
case "thinking", "redacted_thinking":
|
||||
return nil, nil, nil
|
||||
default:
|
||||
return nil, nil, ErrAiUnsupportedBlock
|
||||
}
|
||||
}
|
||||
|
||||
// anthTextPart 按角色选择 Responses 文本部件类型(assistant 历史为 output_text)。
|
||||
func anthTextPart(role, text string) map[string]any {
|
||||
if role == "assistant" {
|
||||
return map[string]any{"type": "output_text", "text": text}
|
||||
}
|
||||
return map[string]any{"type": "input_text", "text": text}
|
||||
}
|
||||
|
||||
// anthImageInput 把 Anthropic image source 转为 Responses input_image 部件。
|
||||
func anthImageInput(source json.RawMessage) (map[string]any, error) {
|
||||
var src struct {
|
||||
Type string `json:"type"`
|
||||
MediaType string `json:"media_type"`
|
||||
Data string `json:"data"`
|
||||
URL string `json:"url"`
|
||||
}
|
||||
if err := json.Unmarshal(source, &src); err != nil {
|
||||
return nil, fmt.Errorf("image source 解析失败: %w", err)
|
||||
}
|
||||
switch src.Type {
|
||||
case "base64":
|
||||
if src.MediaType == "" || src.Data == "" {
|
||||
return nil, fmt.Errorf("image source 缺少 media_type 或 data")
|
||||
}
|
||||
return map[string]any{"type": "input_image",
|
||||
"image_url": "data:" + src.MediaType + ";base64," + src.Data}, nil
|
||||
case "url":
|
||||
if src.URL == "" {
|
||||
return nil, fmt.Errorf("image source 缺少 url")
|
||||
}
|
||||
return map[string]any{"type": "input_image", "image_url": src.URL}, nil
|
||||
}
|
||||
return nil, fmt.Errorf("不支持的 image source 类型 %q", src.Type)
|
||||
}
|
||||
|
||||
func anthRespTools(tools []aiwire.AnthTool) []map[string]any {
|
||||
if len(tools) == 0 {
|
||||
return nil
|
||||
}
|
||||
out := make([]map[string]any, 0, len(tools))
|
||||
for _, t := range tools {
|
||||
out = append(out, map[string]any{"type": "function", "name": t.Name,
|
||||
"description": t.Description, "parameters": t.InputSchema})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// anthRespToolChoice 映射 {type:auto|any|tool|none,name} → Responses 形态。
|
||||
func anthRespToolChoice(raw json.RawMessage) any {
|
||||
if len(raw) == 0 {
|
||||
return nil
|
||||
}
|
||||
var tc struct {
|
||||
Type string `json:"type"`
|
||||
Name string `json:"name"`
|
||||
}
|
||||
if json.Unmarshal(raw, &tc) != nil {
|
||||
return nil
|
||||
}
|
||||
switch tc.Type {
|
||||
case "auto":
|
||||
return "auto"
|
||||
case "any":
|
||||
return "required"
|
||||
case "none":
|
||||
return "none"
|
||||
case "tool":
|
||||
return map[string]string{"type": "function", "name": tc.Name}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// respPayload 是直通响应中本转换关心的子集(未知字段忽略)。
|
||||
type respPayload struct {
|
||||
Model string `json:"model"`
|
||||
Status string `json:"status"`
|
||||
IncompleteDetails *respIncomplete `json:"incomplete_details"`
|
||||
Output []respOutputItem `json:"output"`
|
||||
Usage *aiwire.RespUsage `json:"usage"`
|
||||
Error *map[string]string `json:"error"`
|
||||
}
|
||||
|
||||
type respIncomplete struct {
|
||||
Reason string `json:"reason"`
|
||||
}
|
||||
|
||||
type respOutputItem struct {
|
||||
Type string `json:"type"`
|
||||
CallID string `json:"call_id"`
|
||||
Name string `json:"name"`
|
||||
Arguments string `json:"arguments"`
|
||||
Content []struct {
|
||||
Type string `json:"type"`
|
||||
Text string `json:"text"`
|
||||
} `json:"content"`
|
||||
}
|
||||
|
||||
// ResponsesToAnthropic 把直通非流式响应转为 Anthropic Messages 响应。
|
||||
func ResponsesToAnthropic(payload []byte, msgID string) (*aiwire.MessagesResponse, error) {
|
||||
var resp respPayload
|
||||
if err := json.Unmarshal(payload, &resp); err != nil {
|
||||
return nil, fmt.Errorf("解析上游响应: %w", err)
|
||||
}
|
||||
out := &aiwire.MessagesResponse{ID: msgID, Type: "message", Role: "assistant",
|
||||
Model: resp.Model, Content: []aiwire.AnthBlock{}, StopReason: "end_turn"}
|
||||
for _, item := range resp.Output {
|
||||
switch item.Type {
|
||||
case "message":
|
||||
for _, part := range item.Content {
|
||||
if part.Type == "output_text" && part.Text != "" {
|
||||
out.Content = append(out.Content, aiwire.AnthBlock{Type: "text", Text: part.Text})
|
||||
}
|
||||
}
|
||||
case "function_call":
|
||||
out.Content = append(out.Content, aiwire.AnthBlock{Type: "tool_use", ID: item.CallID,
|
||||
Name: item.Name, Input: argsToJSON(item.Arguments)})
|
||||
out.StopReason = "tool_use"
|
||||
}
|
||||
}
|
||||
if resp.Status == "incomplete" && resp.IncompleteDetails != nil &&
|
||||
resp.IncompleteDetails.Reason == "max_output_tokens" {
|
||||
out.StopReason = "max_tokens"
|
||||
}
|
||||
out.Usage = anthUsageFromResp(resp.Usage)
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func anthUsageFromResp(u *aiwire.RespUsage) aiwire.AnthUsage {
|
||||
if u == nil {
|
||||
return aiwire.AnthUsage{}
|
||||
}
|
||||
return aiwire.AnthUsage{InputTokens: u.InputTokens, OutputTokens: u.OutputTokens,
|
||||
CacheReadInputTokens: u.InputTokensDetails.CachedTokens}
|
||||
}
|
||||
|
||||
// argsToJSON 保证 tool_use.input 是合法 JSON 对象(模型可能产出非法片段)。
|
||||
func argsToJSON(args string) json.RawMessage {
|
||||
trimmed := strings.TrimSpace(args)
|
||||
if trimmed == "" {
|
||||
return json.RawMessage(`{}`)
|
||||
}
|
||||
if json.Valid([]byte(trimmed)) {
|
||||
return json.RawMessage(trimmed)
|
||||
}
|
||||
b, _ := json.Marshal(map[string]string{"_raw": args})
|
||||
return b
|
||||
}
|
||||
|
||||
// ---- Anthropic 流式桥:Responses SSE 事件 → Anthropic 事件序列 ----
|
||||
|
||||
// AnthEvent 是一条待写出的 Anthropic SSE 事件。
|
||||
type AnthEvent struct {
|
||||
Event string
|
||||
Data any
|
||||
}
|
||||
|
||||
// AnthRespBridge 把直通 SSE 事件流桥接为 Anthropic 事件序列:
|
||||
// message_start → content_block_start/delta/stop(text 与 tool_use 分块)→ message_delta → message_stop。
|
||||
// reasoning 系列事件丢弃。
|
||||
type AnthRespBridge struct {
|
||||
id, model string
|
||||
started bool
|
||||
blockOpen bool
|
||||
blockIsTool bool
|
||||
blockIndex int
|
||||
stopReason string
|
||||
usage aiwire.AnthUsage
|
||||
}
|
||||
|
||||
// NewAnthRespBridge 构造桥;id 为响应消息 ID。
|
||||
func NewAnthRespBridge(id, model string) *AnthRespBridge {
|
||||
return &AnthRespBridge{id: id, model: model, blockIndex: -1, stopReason: "end_turn"}
|
||||
}
|
||||
|
||||
// respStreamEvent 是直通 SSE data JSON 中桥关心的子集。
|
||||
type respStreamEvent struct {
|
||||
Type string `json:"type"`
|
||||
Delta string `json:"delta"`
|
||||
Item *respOutputItem `json:"item"`
|
||||
Response *respPayload `json:"response"`
|
||||
}
|
||||
|
||||
// Feed 消费一行 SSE data JSON,返回应立即写出的事件。
|
||||
func (st *AnthRespBridge) Feed(data []byte) []AnthEvent {
|
||||
var ev respStreamEvent
|
||||
if json.Unmarshal(data, &ev) != nil {
|
||||
return nil
|
||||
}
|
||||
var events []AnthEvent
|
||||
if !st.started {
|
||||
st.started = true
|
||||
events = append(events, st.startEvent())
|
||||
}
|
||||
switch ev.Type {
|
||||
case "response.output_item.added":
|
||||
if ev.Item != nil && ev.Item.Type == "function_call" {
|
||||
events = append(events, st.openBlock(true, ev.Item.CallID, ev.Item.Name)...)
|
||||
st.stopReason = "tool_use"
|
||||
}
|
||||
case "response.output_text.delta":
|
||||
events = append(events, st.textDelta(ev.Delta)...)
|
||||
case "response.function_call_arguments.delta":
|
||||
events = append(events, st.argsDelta(ev.Delta)...)
|
||||
case "response.completed", "response.incomplete", "response.failed":
|
||||
st.finishFrom(ev.Response)
|
||||
}
|
||||
return events
|
||||
}
|
||||
|
||||
func (st *AnthRespBridge) textDelta(delta string) []AnthEvent {
|
||||
var events []AnthEvent
|
||||
if !st.blockOpen || st.blockIsTool {
|
||||
events = append(events, st.openBlock(false, "", "")...)
|
||||
}
|
||||
events = append(events, AnthEvent{Event: "content_block_delta", Data: map[string]any{
|
||||
"type": "content_block_delta", "index": st.blockIndex,
|
||||
"delta": map[string]string{"type": "text_delta", "text": delta},
|
||||
}})
|
||||
return events
|
||||
}
|
||||
|
||||
func (st *AnthRespBridge) argsDelta(delta string) []AnthEvent {
|
||||
if !st.blockOpen || !st.blockIsTool {
|
||||
return nil
|
||||
}
|
||||
return []AnthEvent{{Event: "content_block_delta", Data: map[string]any{
|
||||
"type": "content_block_delta", "index": st.blockIndex,
|
||||
"delta": map[string]string{"type": "input_json_delta", "partial_json": delta},
|
||||
}}}
|
||||
}
|
||||
|
||||
// finishFrom 记录终态:usage 与 stop_reason(max_output_tokens 截断 → max_tokens)。
|
||||
func (st *AnthRespBridge) finishFrom(resp *respPayload) {
|
||||
if resp == nil {
|
||||
return
|
||||
}
|
||||
st.usage = anthUsageFromResp(resp.Usage)
|
||||
if resp.Status == "incomplete" && resp.IncompleteDetails != nil &&
|
||||
resp.IncompleteDetails.Reason == "max_output_tokens" {
|
||||
st.stopReason = "max_tokens"
|
||||
}
|
||||
}
|
||||
|
||||
func (st *AnthRespBridge) startEvent() AnthEvent {
|
||||
return AnthEvent{Event: "message_start", Data: map[string]any{
|
||||
"type": "message_start",
|
||||
"message": map[string]any{
|
||||
"id": st.id, "type": "message", "role": "assistant", "model": st.model,
|
||||
"content": []any{}, "stop_reason": nil,
|
||||
"usage": map[string]int{"input_tokens": 0, "output_tokens": 0},
|
||||
},
|
||||
}}
|
||||
}
|
||||
|
||||
// openBlock 关闭当前块并打开新块(text 或 tool_use)。
|
||||
func (st *AnthRespBridge) openBlock(isTool bool, toolID, toolName string) []AnthEvent {
|
||||
var events []AnthEvent
|
||||
if st.blockOpen {
|
||||
events = append(events, st.closeBlockEvent())
|
||||
}
|
||||
st.blockOpen, st.blockIsTool = true, isTool
|
||||
st.blockIndex++
|
||||
block := map[string]any{"type": "text", "text": ""}
|
||||
if isTool {
|
||||
block = map[string]any{"type": "tool_use", "id": toolID, "name": toolName, "input": map[string]any{}}
|
||||
}
|
||||
events = append(events, AnthEvent{Event: "content_block_start", Data: map[string]any{
|
||||
"type": "content_block_start", "index": st.blockIndex, "content_block": block,
|
||||
}})
|
||||
return events
|
||||
}
|
||||
|
||||
func (st *AnthRespBridge) closeBlockEvent() AnthEvent {
|
||||
return AnthEvent{Event: "content_block_stop", Data: map[string]any{
|
||||
"type": "content_block_stop", "index": st.blockIndex,
|
||||
}}
|
||||
}
|
||||
|
||||
// Finish 在上游流结束后收尾:关块 → message_delta(stop_reason+usage)→ message_stop。
|
||||
func (st *AnthRespBridge) Finish() []AnthEvent {
|
||||
var events []AnthEvent
|
||||
if !st.started {
|
||||
st.started = true
|
||||
events = append(events, st.startEvent())
|
||||
}
|
||||
if st.blockOpen {
|
||||
events = append(events, st.closeBlockEvent())
|
||||
st.blockOpen = false
|
||||
}
|
||||
usage := map[string]int{"output_tokens": st.usage.OutputTokens}
|
||||
if st.usage.InputTokens > 0 {
|
||||
usage["input_tokens"] = st.usage.InputTokens
|
||||
}
|
||||
if st.usage.CacheReadInputTokens > 0 {
|
||||
usage["cache_read_input_tokens"] = st.usage.CacheReadInputTokens
|
||||
}
|
||||
events = append(events,
|
||||
AnthEvent{Event: "message_delta", Data: map[string]any{
|
||||
"type": "message_delta",
|
||||
"delta": map[string]any{"stop_reason": st.stopReason, "stop_sequence": nil},
|
||||
"usage": usage,
|
||||
}},
|
||||
AnthEvent{Event: "message_stop", Data: map[string]any{"type": "message_stop"}},
|
||||
)
|
||||
return events
|
||||
}
|
||||
|
||||
// Usage 返回聚合到的用量(供调用日志)。
|
||||
func (st *AnthRespBridge) Usage() aiwire.AnthUsage { return st.usage }
|
||||
@@ -0,0 +1,173 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"oci-portal/internal/aiwire"
|
||||
)
|
||||
|
||||
func mustAnthReq(t *testing.T, raw string) aiwire.MessagesRequest {
|
||||
t.Helper()
|
||||
var req aiwire.MessagesRequest
|
||||
if err := json.Unmarshal([]byte(raw), &req); err != nil {
|
||||
t.Fatalf("解析请求: %v", err)
|
||||
}
|
||||
return req
|
||||
}
|
||||
|
||||
// TestAnthropicToResponsesBody 断言 system/消息/工具/effort 的直通装配与 store 强制。
|
||||
func TestAnthropicToResponsesBody(t *testing.T) {
|
||||
req := mustAnthReq(t, `{
|
||||
"model": "xai.grok-4.3", "max_tokens": 128, "system": "你是助手", "stream": true,
|
||||
"temperature": 0.5,
|
||||
"messages": [
|
||||
{"role": "user", "content": "东京天气?"},
|
||||
{"role": "assistant", "content": [
|
||||
{"type": "text", "text": "查询中"},
|
||||
{"type": "tool_use", "id": "t1", "name": "get_weather", "input": {"city": "东京"}}
|
||||
]},
|
||||
{"role": "user", "content": [
|
||||
{"type": "tool_result", "tool_use_id": "t1", "content": "晴 25 度"},
|
||||
{"type": "text", "text": "继续"}
|
||||
]}
|
||||
],
|
||||
"tools": [{"name": "get_weather", "description": "查天气", "input_schema": {"type": "object"}}],
|
||||
"tool_choice": {"type": "any"},
|
||||
"output_config": {"effort": "HIGH"}
|
||||
}`)
|
||||
payload, err := AnthropicToResponsesBody(req)
|
||||
if err != nil {
|
||||
t.Fatalf("AnthropicToResponsesBody: %v", err)
|
||||
}
|
||||
var body map[string]any
|
||||
if err := json.Unmarshal(payload, &body); err != nil {
|
||||
t.Fatalf("unmarshal: %v", err)
|
||||
}
|
||||
if body["model"] != "xai.grok-4.3" || body["max_output_tokens"] != float64(128) ||
|
||||
body["instructions"] != "你是助手" || body["store"] != false || body["stream"] != true {
|
||||
t.Fatalf("顶层字段装配错误: %v", body)
|
||||
}
|
||||
if body["tool_choice"] != "required" {
|
||||
t.Fatalf("tool_choice = %v, want required", body["tool_choice"])
|
||||
}
|
||||
reasoning, _ := body["reasoning"].(map[string]any)
|
||||
if reasoning["effort"] != "high" {
|
||||
t.Fatalf("effort = %v, want high(小写透传)", reasoning)
|
||||
}
|
||||
input, _ := body["input"].([]any)
|
||||
// user 消息、assistant 文本消息、function_call、function_call_output、末条 user 文本
|
||||
if len(input) != 5 {
|
||||
t.Fatalf("input 项数 = %d, want 5: %s", len(input), payload)
|
||||
}
|
||||
kinds := make([]string, 0, len(input))
|
||||
for _, it := range input {
|
||||
m := it.(map[string]any)
|
||||
if ty, ok := m["type"].(string); ok {
|
||||
kinds = append(kinds, ty)
|
||||
} else {
|
||||
kinds = append(kinds, "message/"+m["role"].(string))
|
||||
}
|
||||
}
|
||||
want := "message/user,message/assistant,function_call,function_call_output,message/user"
|
||||
if got := strings.Join(kinds, ","); got != want {
|
||||
t.Fatalf("input 顺序 = %s, want %s", got, want)
|
||||
}
|
||||
if !strings.Contains(string(payload), `"output_text"`) {
|
||||
t.Fatalf("assistant 历史文本应为 output_text 部件: %s", payload)
|
||||
}
|
||||
}
|
||||
|
||||
// TestAnthropicToResponsesBodyRejects 断言不支持的内容块拒绝与图片装配。
|
||||
func TestAnthropicToResponsesBodyRejects(t *testing.T) {
|
||||
bad := mustAnthReq(t, `{"model":"m","max_tokens":8,"messages":[
|
||||
{"role":"user","content":[{"type":"document","source":{}}]}]}`)
|
||||
if _, err := AnthropicToResponsesBody(bad); err == nil {
|
||||
t.Fatal("document 块应拒绝")
|
||||
}
|
||||
img := mustAnthReq(t, `{"model":"m","max_tokens":8,"messages":[
|
||||
{"role":"user","content":[{"type":"image","source":{"type":"base64","media_type":"image/png","data":"QUJD"}}]}]}`)
|
||||
payload, err := AnthropicToResponsesBody(img)
|
||||
if err != nil || !strings.Contains(string(payload), "data:image/png;base64,QUJD") {
|
||||
t.Fatalf("图片应转 data URI: %v %s", err, payload)
|
||||
}
|
||||
}
|
||||
|
||||
// TestResponsesToAnthropic 断言输出块、stop_reason 与 usage 的回转。
|
||||
func TestResponsesToAnthropic(t *testing.T) {
|
||||
payload := []byte(`{"model":"xai.grok-4.3","status":"completed","output":[
|
||||
{"type":"reasoning","summary":[]},
|
||||
{"type":"message","content":[{"type":"output_text","text":"你好"}]},
|
||||
{"type":"function_call","call_id":"c1","name":"get_weather","arguments":"{\"city\":\"东京\"}"}],
|
||||
"usage":{"input_tokens":10,"output_tokens":5,"total_tokens":15,"input_tokens_details":{"cached_tokens":4}}}`)
|
||||
out, err := ResponsesToAnthropic(payload, "msg_1")
|
||||
if err != nil {
|
||||
t.Fatalf("ResponsesToAnthropic: %v", err)
|
||||
}
|
||||
if len(out.Content) != 2 || out.Content[0].Type != "text" || out.Content[0].Text != "你好" {
|
||||
t.Fatalf("content 装配错误: %+v", out.Content)
|
||||
}
|
||||
if out.Content[1].Type != "tool_use" || out.Content[1].ID != "c1" || string(out.Content[1].Input) != `{"city":"东京"}` {
|
||||
t.Fatalf("tool_use 装配错误: %+v", out.Content[1])
|
||||
}
|
||||
if out.StopReason != "tool_use" {
|
||||
t.Fatalf("stop_reason = %s, want tool_use", out.StopReason)
|
||||
}
|
||||
if out.Usage.InputTokens != 10 || out.Usage.OutputTokens != 5 || out.Usage.CacheReadInputTokens != 4 {
|
||||
t.Fatalf("usage = %+v", out.Usage)
|
||||
}
|
||||
|
||||
trunc := []byte(`{"model":"m","status":"incomplete","incomplete_details":{"reason":"max_output_tokens"},
|
||||
"output":[{"type":"message","content":[{"type":"output_text","text":"半"}]}]}`)
|
||||
out2, err := ResponsesToAnthropic(trunc, "msg_2")
|
||||
if err != nil || out2.StopReason != "max_tokens" {
|
||||
t.Fatalf("截断 stop_reason = %v, %v", out2, err)
|
||||
}
|
||||
}
|
||||
|
||||
func bridgeEventTypes(events []AnthEvent) string {
|
||||
kinds := make([]string, 0, len(events))
|
||||
for _, ev := range events {
|
||||
kinds = append(kinds, ev.Event)
|
||||
}
|
||||
return strings.Join(kinds, ",")
|
||||
}
|
||||
|
||||
// TestAnthRespBridge 断言流桥:文本增量、工具调用切块、reasoning 丢弃与收尾事件。
|
||||
func TestAnthRespBridge(t *testing.T) {
|
||||
st := NewAnthRespBridge("msg_1", "m1")
|
||||
var events []AnthEvent
|
||||
feed := func(lines ...string) {
|
||||
for _, l := range lines {
|
||||
events = append(events, st.Feed([]byte(l))...)
|
||||
}
|
||||
}
|
||||
feed(`{"type":"response.created","response":{"model":"m1"}}`,
|
||||
`{"type":"response.reasoning_text.delta","delta":"思考中"}`,
|
||||
`{"type":"response.output_text.delta","delta":"你"}`,
|
||||
`{"type":"response.output_text.delta","delta":"好"}`,
|
||||
`{"type":"response.output_item.added","item":{"type":"function_call","call_id":"c1","name":"f"}}`,
|
||||
`{"type":"response.function_call_arguments.delta","delta":"{\"a\":1}"}`,
|
||||
`{"type":"response.completed","response":{"status":"completed","usage":{"input_tokens":8,"output_tokens":4}}}`)
|
||||
events = append(events, st.Finish()...)
|
||||
|
||||
got := bridgeEventTypes(events)
|
||||
want := "message_start,content_block_start,content_block_delta,content_block_delta," +
|
||||
"content_block_stop,content_block_start,content_block_delta,content_block_stop," +
|
||||
"message_delta,message_stop"
|
||||
if got != want {
|
||||
t.Fatalf("事件序列:\n got %s\nwant %s", got, want)
|
||||
}
|
||||
if st.Usage().InputTokens != 8 || st.Usage().OutputTokens != 4 {
|
||||
t.Fatalf("usage = %+v", st.Usage())
|
||||
}
|
||||
}
|
||||
|
||||
// TestAnthRespBridgeEmpty 断言空流也产出完整事件骨架。
|
||||
func TestAnthRespBridgeEmpty(t *testing.T) {
|
||||
st := NewAnthRespBridge("msg_1", "m1")
|
||||
if got := bridgeEventTypes(st.Finish()); got != "message_start,message_delta,message_stop" {
|
||||
t.Errorf("空流事件序列 = %s", got)
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user