修复 AI 网关流式断流:错误透传、自动降级,max_tokens 可缺省
CI / test (push) Successful in 33s
Release / release (push) Successful in 56s

This commit is contained in:
2026-07-13 15:06:01 +08:00
parent 81d4650f3d
commit 9309ad1ffc
11 changed files with 420 additions and 28 deletions
+14
View File
@@ -2,6 +2,20 @@
格式遵循 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/),版本号遵循语义化版本。 格式遵循 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/),版本号遵循语义化版本。
## [0.3.1] - 2026-07-13
### Added
- AI 网关流式断流自动降级:Messages 与 Chat Completions 流式请求在客户端尚未收到任何输出时遭遇上游断流,自动降级为非流式重做,结果按标准事件 / chunk 序列一次推送,调用日志记 `retries=1` 与降级标记。实测 OCI 兼容面对 `instructions``tools` 合计超约 64.5KB 的流式请求会静默断连(无错误事件,非流式正常,消息正文不计入),Claude Code 等大体量系统提示客户端极易触发;Responses 直通因初始事件已转发无法透明降级,日志记「上游流提前终止」;README 增补「已知上游限制:大 system 区流式断流」小节
### Changed
- Messages 的 `max_tokens` 改为可缺省:缺省或 ≤0 时按默认值 8192 放行(此前返回 400;部分客户端将该字段视为选填)
### Fixed
- AI 网关流式假成功:上游 error / `response.failed` 事件此前被吞掉,流提前 EOF 也被伪装成正常结束,调用日志呈现 200 · 0/0 且无错误信息;现 Messages 将上游失败转为 Anthropic `error` 事件,三个流式端点日志均记录上游错误消息或「上游流提前终止,未返回终态事件」
## [0.3.0] - 2026-07-13 ## [0.3.0] - 2026-07-13
### Added ### Added
+14 -4
View File
@@ -216,6 +216,16 @@ AI 网关使用面板创建的独立密钥鉴权,支持 `Authorization: Bearer
- Chat Completions 只承担协议转换与兼容修复;新能力优先在 Responses 和 Messages 提供 - Chat Completions 只承担协议转换与兼容修复;新能力优先在 Responses 和 Messages 提供
- 单次请求最多尝试三个渠道;可重试错误会切换渠道,流式响应建立后不会换渠道重试 - 单次请求最多尝试三个渠道;可重试错误会切换渠道,流式响应建立后不会换渠道重试
### 已知上游限制:大 system 区流式断流
实测(2026-07-13OCI 兼容面对 `instructions` 与 `tools` 合计超约 64.5KB 的**流式**请求会在发出少量事件后静默断开连接(无任何错误事件;同请求非流式正常),与模型、字符集、消息正文大小均无关——消息正文(`input`)不计入该限制。Chat Completions 与 Messages 的 system/developer 提示会转换为 `instructions`,因此 Claude Code 等自带大体量系统提示与工具定义的客户端极易触发。
网关侧应对:
- Messages 与 Chat Completions 的流式请求在客户端尚未收到任何输出时遭遇上游断流,会自动降级为非流式重做,并按标准事件/chunk 序列一次推送;调用日志记 `retries=1` 与降级标记
- Responses 直通因初始事件已转发、协议上无法透明降级,调用日志记「上游流提前终止」,客户端需自行回退非流式
- 应急规避:将超长 system 内容移入首条 user 消息正文可绕过该限制(正文不计入),但语义有别,根治有待上游修复
这里提供的是兼容接口而非 OpenAI / Anthropic 协议的完整实现。OCI OpenAI 兼容面的部分行为来自实测,未见 Oracle 文档承诺,可能随上游调整。路由与鉴权定义以 [Swagger YAML](docs/swagger.yaml) 或运行时 Swagger UI 为准;无法由 OpenAPI 完整表达的兼容边界列于上方。 这里提供的是兼容接口而非 OpenAI / Anthropic 协议的完整实现。OCI OpenAI 兼容面的部分行为来自实测,未见 Oracle 文档承诺,可能随上游调整。路由与鉴权定义以 [Swagger YAML](docs/swagger.yaml) 或运行时 Swagger UI 为准;无法由 OpenAPI 完整表达的兼容边界列于上方。
### 字段兼容矩阵 ### 字段兼容矩阵
@@ -289,7 +299,7 @@ Chat Completions 会先转换为 Responses 请求,再把 OCI Responses 响应
| `max_completion_tokens` | 🔄 | 转为 `max_output_tokens`,优先于 `max_tokens` | | `max_completion_tokens` | 🔄 | 转为 `max_output_tokens`,优先于 `max_tokens` |
| `max_tokens` | 🔄 | 未提供 `max_completion_tokens` 时转为 `max_output_tokens` | | `max_tokens` | 🔄 | 未提供 `max_completion_tokens` 时转为 `max_output_tokens` |
| `temperature`、`top_p`、`parallel_tool_calls` | ◐ | 原值写入 Responses 请求,但不校验范围或模型能力 | | `temperature`、`top_p`、`parallel_tool_calls` | ◐ | 原值写入 Responses 请求,但不校验范围或模型能力 |
| `stream` | 🔄 | OCI Responses SSE 桥接为 `chat.completion.chunk`,末尾补 `data: [DONE]` | | `stream` | 🔄 | OCI Responses SSE 桥接为 `chat.completion.chunk`,末尾补 `data: [DONE]`;上游断流且尚无输出时自动降级非流式重做,结果按 chunk 序列一次推送 |
| `stream_options.include_usage` | 🔄 | 控制网关在终块后追加 `choices: []` 的 usage 块 | | `stream_options.include_usage` | 🔄 | 控制网关在终块后追加 `choices: []` 的 usage 块 |
| `stream_options.include_obfuscation` | ⚠️ | 未建模,静默忽略 | | `stream_options.include_obfuscation` | ⚠️ | 未建模,静默忽略 |
| `tools[].type=function` | ◐ | `name`、`description`、`parameters` 支持;`function.strict` 被忽略 | | `tools[].type=function` | ◐ | `name`、`description`、`parameters` 支持;`function.strict` 被忽略 |
@@ -346,13 +356,13 @@ Chat Completions 会先转换为 Responses 请求,再把 OCI Responses 响应
| 标准字段 | 状态 | 网关行为 | | 标准字段 | 状态 | 网关行为 |
| --- | :---: | --- | | --- | :---: | --- |
| `model` | ◐ | 用于模型与渠道选择,但空字符串不会在 handler 中按参数错误拒绝,通常最终返回模型不存在 | | `model` | ◐ | 用于模型与渠道选择,但空字符串不会在 handler 中按参数错误拒绝,通常最终返回模型不存在 |
| `max_tokens` | 🔄 | 必填且必须大于 0,转换为 `max_output_tokens` | | `max_tokens` | 🔄 | 可缺省(缺省或 ≤0 时按默认值 8192),转换为 `max_output_tokens` |
| `messages` | ◐ | 必须非空;角色、交替顺序和空内容不做完整校验 | | `messages` | ◐ | 必须非空;角色、交替顺序和空内容不做完整校验 |
| `system` | ◐ | 支持字符串或 text 块数组;多个文本块直接拼接,`cache_control` 等附加字段被忽略 | | `system` | ◐ | 支持字符串或 text 块数组;多个文本块直接拼接,`cache_control` 等附加字段被忽略 |
| `temperature`、`top_p` | ◐ | 写入 Responses 请求,不做取值范围或模型能力校验 | | `temperature`、`top_p` | ◐ | 写入 Responses 请求,不做取值范围或模型能力校验 |
| `top_k` | ⚠️ | 能解析但不会传给上游 | | `top_k` | ⚠️ | 能解析但不会传给上游 |
| `stop_sequences` | ⚠️ | 能解析但不会传给上游;响应 `stop_sequence` 恒为 `null` | | `stop_sequences` | ⚠️ | 能解析但不会传给上游;响应 `stop_sequence` 恒为 `null` |
| `stream` | 🔄 | Responses SSE 桥接为 Anthropic 事件序列 | | `stream` | 🔄 | Responses SSE 桥接为 Anthropic 事件序列;上游断流且尚无输出时自动降级非流式重做,结果按事件序列一次推送 |
| `tools` | ◐ | 每个工具都转换成 Responses `function`;自定义客户端工具可用,Anthropic 服务端工具类型不保留原语义 | | `tools` | ◐ | 每个工具都转换成 Responses `function`;自定义客户端工具可用,Anthropic 服务端工具类型不保留原语义 |
| `tool_choice` | ◐ | 支持 `auto`、`any`、`none`、具名 `tool``disable_parallel_tool_use` 等附加字段被忽略 | | `tool_choice` | ◐ | 支持 `auto`、`any`、`none`、具名 `tool``disable_parallel_tool_use` 等附加字段被忽略 |
| `metadata` | ⚠️ | 能解析但不会传给上游 | | `metadata` | ⚠️ | 能解析但不会传给上游 |
@@ -380,7 +390,7 @@ Chat Completions 会先转换为 Responses 请求,再把 OCI Responses 响应
- `stop_reason` 只生成 `end_turn`、`tool_use`、`max_tokens``stop_sequence` 恒为 `null` - `stop_reason` 只生成 `end_turn`、`tool_use`、`max_tokens``stop_sequence` 恒为 `null`
- reasoning 不会生成 Anthropic `thinking` / `redacted_thinking` 块,也没有 signature - reasoning 不会生成 Anthropic `thinking` / `redacted_thinking` 块,也没有 signature
- usage 只保留 `input_tokens`、`output_tokens` 和 `cache_read_input_tokens`,不提供 `cache_creation_input_tokens` - usage 只保留 `input_tokens`、`output_tokens` 和 `cache_read_input_tokens`,不提供 `cache_creation_input_tokens`
- 流式输出标准事件骨架,但不生成 `thinking_delta``signature_delta` 或上游失败对应的 Anthropic `error` 事件 - 流式输出标准事件骨架,但不生成 `thinking_delta``signature_delta`;上游错误事件与无终态断流会转成 Anthropic `error` 事件
实现依据:[`anthresponses.go`](internal/service/anthresponses.go) · [`anthropic.go`](internal/aiwire/anthropic.go) · [`aigateway.go`](internal/api/aigateway.go) 实现依据:[`anthresponses.go`](internal/service/anthresponses.go) · [`anthropic.go`](internal/aiwire/anthropic.go) · [`aigateway.go`](internal/api/aigateway.go)
+1 -1
View File
@@ -79,7 +79,7 @@ const docTemplate = `{
"summary": "Anthropic Messages 兼容端点", "summary": "Anthropic Messages 兼容端点",
"parameters": [ "parameters": [
{ {
"description": "Anthropic messages 请求体(支持 stream;经 OCI OpenAI 兼容面直通)", "description": "Anthropic messages 请求体(支持 stream;经 OCI OpenAI 兼容面直通;max_tokens 可缺省,默认 8192)",
"name": "body", "name": "body",
"in": "body", "in": "body",
"required": true, "required": true,
+1 -1
View File
@@ -72,7 +72,7 @@
"summary": "Anthropic Messages 兼容端点", "summary": "Anthropic Messages 兼容端点",
"parameters": [ "parameters": [
{ {
"description": "Anthropic messages 请求体(支持 stream;经 OCI OpenAI 兼容面直通)", "description": "Anthropic messages 请求体(支持 stream;经 OCI OpenAI 兼容面直通;max_tokens 可缺省,默认 8192)",
"name": "body", "name": "body",
"in": "body", "in": "body",
"required": true, "required": true,
+2 -1
View File
@@ -617,7 +617,8 @@ paths:
/ai/v1/messages: /ai/v1/messages:
post: post:
parameters: parameters:
- description: Anthropic messages 请求体(支持 stream;经 OCI OpenAI 兼容面直通) - description: Anthropic messages 请求体(支持 stream;经 OCI OpenAI 兼容面直通;max_tokens
可缺省,默认 8192)
in: body in: body
name: body name: body
required: true required: true
+111 -10
View File
@@ -210,11 +210,15 @@ func sseHeaders(c *gin.Context) {
c.Writer.Flush() c.Writer.Flush()
} }
// anthDefaultMaxTokens 是 max_tokens 缺省时的默认输出上限:Anthropic 协议
// 该字段必填,但部分客户端当选填不传,按默认值放行而非 400 拒绝。
const anthDefaultMaxTokens = 8192
// messages 是 Anthropic /ai/v1/messages 端点。 // messages 是 Anthropic /ai/v1/messages 端点。
// //
// @Summary Anthropic Messages 兼容端点 // @Summary Anthropic Messages 兼容端点
// @Tags AI 网关 // @Tags AI 网关
// @Param body body object true "Anthropic messages 请求体(支持 stream;经 OCI OpenAI 兼容面直通)" // @Param body body object true "Anthropic messages 请求体(支持 stream;经 OCI OpenAI 兼容面直通;max_tokens 可缺省,默认 8192)"
// @Success 200 {object} map[string]any "Anthropic 兼容响应(流式为 SSE)" // @Success 200 {object} map[string]any "Anthropic 兼容响应(流式为 SSE)"
// @Router /ai/v1/messages [post] // @Router /ai/v1/messages [post]
func (h *aiGatewayHandler) messages(c *gin.Context) { func (h *aiGatewayHandler) messages(c *gin.Context) {
@@ -224,8 +228,7 @@ func (h *aiGatewayHandler) messages(c *gin.Context) {
return return
} }
if req.MaxTokens <= 0 { if req.MaxTokens <= 0 {
aiError(c, http.StatusBadRequest, "invalid_request_error", "max_tokens 必填且需大于 0") req.MaxTokens = anthDefaultMaxTokens
return
} }
if len(req.Messages) == 0 { if len(req.Messages) == 0 {
aiError(c, http.StatusBadRequest, "invalid_request_error", "messages 不能为空") aiError(c, http.StatusBadRequest, "invalid_request_error", "messages 不能为空")
@@ -281,10 +284,30 @@ func (h *aiGatewayHandler) streamAnthropic(c *gin.Context, body []byte, req aiwi
defer upstream.Close() defer upstream.Close()
sseHeaders(c) sseHeaders(c)
bridge := service.NewAnthRespBridge(aiRandID("msg_"), req.Model) bridge := service.NewAnthRespBridge(aiRandID("msg_"), req.Model)
if err := forwardSSEData(upstream, func(data []byte) { writeAnthEvents(c, bridge.Feed(data)) }); err != nil { emitted := 0
if err := forwardSSEData(upstream, func(data []byte) {
evs := bridge.Feed(data)
emitted += len(evs)
writeAnthEvents(c, evs)
}); err != nil {
entry.ErrMsg = err.Error() entry.ErrMsg = err.Error()
} }
// 上游断流且客户端尚未收到任何事件(OCI 兼容面对大请求 + 推理模型的
// 流式通道会在 reasoning 阶段掐断):降级非流式重做,结果按事件序列推送
if emitted == 0 && !bridge.SawTerminal() && c.Request.Context().Err() == nil {
if h.anthFallback(c, &entry, req) {
entry.Status = http.StatusOK
entry.LatencyMs = time.Since(start).Milliseconds()
callID := h.gw.LogCall(entry)
h.maybeLogContent(c, callID, "anthropic", req.Model, true, req, nil)
return
}
}
writeAnthEvents(c, bridge.Finish()) writeAnthEvents(c, bridge.Finish())
// 上游错误事件 / 提前终止:SSE 头已发出维持 200,错误落日志可查
if msg := bridge.Err(); msg != "" && entry.ErrMsg == "" {
entry.ErrMsg = msg
}
entry.Status = http.StatusOK entry.Status = http.StatusOK
entry.LatencyMs = time.Since(start).Milliseconds() entry.LatencyMs = time.Since(start).Milliseconds()
usage := bridge.Usage() usage := bridge.Usage()
@@ -295,6 +318,30 @@ func (h *aiGatewayHandler) streamAnthropic(c *gin.Context, body []byte, req aiwi
h.maybeLogContent(c, callID, "anthropic", req.Model, true, req, nil) h.maybeLogContent(c, callID, "anthropic", req.Model, true, req, nil)
} }
// anthFallback 用非流式重做同一请求并把完整结果按事件序列推送;
// 成功返回 true 并把渠道 / 用量 / 降级标注写入日志条目。
func (h *aiGatewayHandler) anthFallback(c *gin.Context, entry *model.AiCallLog, req aiwire.MessagesRequest) bool {
req.Stream = false
body, err := service.AnthropicToResponsesBody(req)
if err != nil {
return false
}
payload, meta, err := h.gw.RespPassthrough(c.Request.Context(), body, req.Model, keyGroup(c))
if err != nil {
return false
}
out, err := service.ResponsesToAnthropic(payload, aiRandID("msg_"))
if err != nil {
return false
}
writeAnthEvents(c, service.AnthMessageEvents(out))
entry.ChannelID, entry.ChannelName = meta.ChannelID, meta.ChannelName
entry.Retries++
entry.ErrMsg = "流式上游断流,已降级非流式完成"
fillUsage(entry, service.RespPassthroughUsage(payload))
return true
}
// forwardSSEData 逐行读上游 SSE,把 data 行交给 emit 即时转换写出; // forwardSSEData 逐行读上游 SSE,把 data 行交给 emit 即时转换写出;
// Anthropic 与 Chat Completions 两条流式桥共用。 // Anthropic 与 Chat Completions 两条流式桥共用。
func forwardSSEData(upstream io.Reader, emit func([]byte)) error { func forwardSSEData(upstream io.Reader, emit func([]byte)) error {
@@ -403,12 +450,31 @@ func (h *aiGatewayHandler) streamChat(c *gin.Context, body []byte, req aiwire.Ch
sseHeaders(c) sseHeaders(c)
includeUsage := req.StreamOptions != nil && req.StreamOptions.IncludeUsage includeUsage := req.StreamOptions != nil && req.StreamOptions.IncludeUsage
bridge := service.NewChatRespBridge(aiRandID("chatcmpl-"), req.Model, time.Now().Unix(), includeUsage) bridge := service.NewChatRespBridge(aiRandID("chatcmpl-"), req.Model, time.Now().Unix(), includeUsage)
if err := forwardSSEData(upstream, func(data []byte) { writeChatChunks(c, bridge.Feed(data)) }); err != nil { emitted := 0
if err := forwardSSEData(upstream, func(data []byte) {
chunks := bridge.Feed(data)
emitted += len(chunks)
writeChatChunks(c, chunks)
}); err != nil {
entry.ErrMsg = err.Error() entry.ErrMsg = err.Error()
} }
// 上游断流且客户端尚未收到任何块:降级非流式重做(成因同 messages 端点)
if emitted == 0 && bridge.Usage() == nil && c.Request.Context().Err() == nil {
if h.chatFallback(c, &entry, req, includeUsage) {
entry.Status = http.StatusOK
entry.LatencyMs = time.Since(start).Milliseconds()
callID := h.gw.LogCall(entry)
h.maybeLogContent(c, callID, "openai", req.Model, true, req, nil)
return
}
}
writeChatChunks(c, bridge.Finish()) writeChatChunks(c, bridge.Finish())
c.Writer.WriteString("data: [DONE]\n\n") c.Writer.WriteString("data: [DONE]\n\n")
c.Writer.Flush() c.Writer.Flush()
// 未见 completed 即结束:usage 缺失说明上游流异常提前终止,落日志可查
if bridge.Usage() == nil && entry.ErrMsg == "" {
entry.ErrMsg = "上游流提前终止,未返回终态事件"
}
entry.Status = http.StatusOK entry.Status = http.StatusOK
entry.LatencyMs = time.Since(start).Milliseconds() entry.LatencyMs = time.Since(start).Milliseconds()
fillUsage(&entry, bridge.Usage()) fillUsage(&entry, bridge.Usage())
@@ -416,6 +482,32 @@ func (h *aiGatewayHandler) streamChat(c *gin.Context, body []byte, req aiwire.Ch
h.maybeLogContent(c, callID, "openai", req.Model, true, req, nil) h.maybeLogContent(c, callID, "openai", req.Model, true, req, nil)
} }
// chatFallback 用非流式重做同一请求并把完整结果按 chunk 序列推送;
// 成功返回 true 并把渠道 / 用量 / 降级标注写入日志条目。
func (h *aiGatewayHandler) chatFallback(c *gin.Context, entry *model.AiCallLog, req aiwire.ChatRequest, includeUsage bool) bool {
req.Stream = false
body, err := service.ChatToResponsesBody(req)
if err != nil {
return false
}
payload, meta, err := h.gw.RespPassthrough(c.Request.Context(), body, req.Model, keyGroup(c))
if err != nil {
return false
}
out, err := service.ResponsesToChat(payload, aiRandID("chatcmpl-"), time.Now().Unix())
if err != nil {
return false
}
writeChatChunks(c, service.ChatResponseChunks(out, includeUsage))
c.Writer.WriteString("data: [DONE]\n\n")
c.Writer.Flush()
entry.ChannelID, entry.ChannelName = meta.ChannelID, meta.ChannelName
entry.Retries++
entry.ErrMsg = "流式上游断流,已降级非流式完成"
fillUsage(entry, service.RespPassthroughUsage(payload))
return true
}
// writeChatChunks 逐块写出 chunk 的 SSE data 行并 flush。 // writeChatChunks 逐块写出 chunk 的 SSE data 行并 flush。
func writeChatChunks(c *gin.Context, chunks []aiwire.ChatChunk) { func writeChatChunks(c *gin.Context, chunks []aiwire.ChatChunk) {
for _, ch := range chunks { for _, ch := range chunks {
@@ -526,9 +618,14 @@ func (h *aiGatewayHandler) responsesPassthroughStream(c *gin.Context, body []byt
} }
defer upstream.Close() defer upstream.Close()
sseHeaders(c) sseHeaders(c)
usage, err := forwardSSE(c, upstream) usage, upErr, err := forwardSSE(c, upstream)
if err != nil { if err != nil {
entry.ErrMsg = err.Error() entry.ErrMsg = err.Error()
} else if upErr != "" {
// 上游错误事件已原样转发给客户端,这里落日志可查
entry.ErrMsg = upErr
} else if usage == nil {
entry.ErrMsg = "上游流提前终止,未返回终态事件"
} }
entry.Status = http.StatusOK entry.Status = http.StatusOK
entry.LatencyMs = time.Since(start).Milliseconds() entry.LatencyMs = time.Since(start).Milliseconds()
@@ -538,10 +635,11 @@ func (h *aiGatewayHandler) responsesPassthroughStream(c *gin.Context, body []byt
} }
// forwardSSE 把上游 SSE 逐行转发给客户端,空行(事件边界)即 flush; // forwardSSE 把上游 SSE 逐行转发给客户端,空行(事件边界)即 flush;
// 顺带从 data 行提取 response.completed 的 usage。 // 顺带从 data 行提取 response.completed 的 usage 与错误事件消息
func forwardSSE(c *gin.Context, upstream io.Reader) (*aiwire.Usage, error) { func forwardSSE(c *gin.Context, upstream io.Reader) (*aiwire.Usage, string, error) {
reader := bufio.NewReader(upstream) reader := bufio.NewReader(upstream)
var usage *aiwire.Usage var usage *aiwire.Usage
var upErr string
for { for {
line, err := reader.ReadBytes('\n') line, err := reader.ReadBytes('\n')
if len(line) > 0 { if len(line) > 0 {
@@ -553,14 +651,17 @@ func forwardSSE(c *gin.Context, upstream io.Reader) (*aiwire.Usage, error) {
if u := service.RespStreamCompletedUsage(data); u != nil { if u := service.RespStreamCompletedUsage(data); u != nil {
usage = u usage = u
} }
if m := service.RespStreamErrorMsg(data); m != "" && upErr == "" {
upErr = m
}
} }
} }
if err != nil { if err != nil {
c.Writer.Flush() c.Writer.Flush()
if errors.Is(err, io.EOF) { if errors.Is(err, io.EOF) {
return usage, nil return usage, upErr, nil
} }
return usage, err return usage, upErr, err
} }
} }
} }
+2
View File
@@ -85,6 +85,8 @@ func TestAiGatewayKeyModelRestrict(t *testing.T) {
`{"model":"meta.llama-3.3-70b-instruct","messages":[{"role":"user","content":"hi"}]}`, 404, deny}, `{"model":"meta.llama-3.3-70b-instruct","messages":[{"role":"user","content":"hi"}]}`, 404, deny},
{"chat completions 白名单内穿透", "limited-key-1234", "/ai/v1/chat/completions", {"chat completions 白名单内穿透", "limited-key-1234", "/ai/v1/chat/completions",
`{"model":"ghost-model","messages":[{"role":"user","content":"hi"}]}`, 404, pass}, `{"model":"ghost-model","messages":[{"role":"user","content":"hi"}]}`, 404, pass},
{"messages 缺 max_tokens 按默认值放行", "open-key-12345", "/ai/v1/messages",
`{"model":"ghost-model","messages":[{"role":"user","content":"hi"}]}`, 404, pass},
{"chat completions 缺 messages 拒绝", "open-key-12345", "/ai/v1/chat/completions", {"chat completions 缺 messages 拒绝", "open-key-12345", "/ai/v1/chat/completions",
`{"model":"ghost-model"}`, 400, []string{"invalid_request_error"}}, `{"model":"ghost-model"}`, 400, []string{"invalid_request_error"}},
{"chat completions 非 function 工具拒绝", "open-key-12345", "/ai/v1/chat/completions", {"chat completions 非 function 工具拒绝", "open-key-12345", "/ai/v1/chat/completions",
+22
View File
@@ -104,3 +104,25 @@ func RespStreamCompletedUsage(data []byte) *aiwire.Usage {
} }
return RespPassthroughUsage(ev.Response) return RespPassthroughUsage(ev.Response)
} }
// RespStreamErrorMsg 从一行 SSE data JSON 中提取 error / response.failed 事件的
// 错误消息;非错误事件返回空串。流式直通据此把上游错误写入调用日志。
func RespStreamErrorMsg(data []byte) string {
var ev respStreamEvent
if json.Unmarshal(data, &ev) != nil {
return ""
}
switch ev.Type {
case "error":
if ev.Message != "" {
return ev.Message
}
return "上游返回错误事件 error"
case "response.failed":
if m := respErrorMsg(ev.Response); m != "" {
return m
}
return "上游返回错误事件 response.failed"
}
return ""
}
+124 -6
View File
@@ -258,7 +258,7 @@ type AnthEvent struct {
// AnthRespBridge 把直通 SSE 事件流桥接为 Anthropic 事件序列: // AnthRespBridge 把直通 SSE 事件流桥接为 Anthropic 事件序列:
// message_start → content_block_start/delta/stop(text 与 tool_use 分块)→ message_delta → message_stop。 // message_start → content_block_start/delta/stop(text 与 tool_use 分块)→ message_delta → message_stop。
// reasoning 系列事件丢弃。 // reasoning 系列事件丢弃;上游 error / response.failed 转 Anthropic error 事件透传
type AnthRespBridge struct { type AnthRespBridge struct {
id, model string id, model string
started bool started bool
@@ -267,6 +267,11 @@ type AnthRespBridge struct {
blockIndex int blockIndex int
stopReason string stopReason string
usage aiwire.AnthUsage usage aiwire.AnthUsage
// sawTerminal 标记收到过终态事件(completed/incomplete/failed);
// 上游流提前 EOF 时据此发 error 事件而非伪装正常结束
sawTerminal bool
// errMsg 记录上游错误消息,非空即本流已失败(供调用日志)
errMsg string
} }
// NewAnthRespBridge 构造桥;id 为响应消息 ID。 // NewAnthRespBridge 构造桥;id 为响应消息 ID。
@@ -278,6 +283,7 @@ func NewAnthRespBridge(id, model string) *AnthRespBridge {
type respStreamEvent struct { type respStreamEvent struct {
Type string `json:"type"` Type string `json:"type"`
Delta string `json:"delta"` Delta string `json:"delta"`
Message string `json:"message"`
Item *respOutputItem `json:"item"` Item *respOutputItem `json:"item"`
Response *respPayload `json:"response"` Response *respPayload `json:"response"`
} }
@@ -288,27 +294,130 @@ func (st *AnthRespBridge) Feed(data []byte) []AnthEvent {
if json.Unmarshal(data, &ev) != nil { if json.Unmarshal(data, &ev) != nil {
return nil return nil
} }
var events []AnthEvent if ev.Type == "error" || ev.Type == "response.failed" {
if !st.started { return st.failWith(ev)
st.started = true
events = append(events, st.startEvent())
} }
// message_start 延迟到首个可见输出事件:created / reasoning 阶段不向客户端
// 写任何字节,上游此段断流时 handler 才有降级非流式重做的无感窗口
var events []AnthEvent
switch ev.Type { switch ev.Type {
case "response.output_item.added": case "response.output_item.added":
if ev.Item != nil && ev.Item.Type == "function_call" { if ev.Item != nil && ev.Item.Type == "function_call" {
events = append(events, st.ensureStarted()...)
events = append(events, st.openBlock(true, ev.Item.CallID, ev.Item.Name)...) events = append(events, st.openBlock(true, ev.Item.CallID, ev.Item.Name)...)
st.stopReason = "tool_use" st.stopReason = "tool_use"
} }
case "response.output_text.delta": case "response.output_text.delta":
events = append(events, st.ensureStarted()...)
events = append(events, st.textDelta(ev.Delta)...) events = append(events, st.textDelta(ev.Delta)...)
case "response.function_call_arguments.delta": case "response.function_call_arguments.delta":
events = append(events, st.argsDelta(ev.Delta)...) events = append(events, st.argsDelta(ev.Delta)...)
case "response.completed", "response.incomplete", "response.failed": case "response.completed", "response.incomplete":
st.sawTerminal = true
st.finishFrom(ev.Response) st.finishFrom(ev.Response)
} }
return events return events
} }
// ensureStarted 在首个可见输出事件前补发 message_start(仅一次)。
func (st *AnthRespBridge) ensureStarted() []AnthEvent {
if st.started {
return nil
}
st.started = true
return []AnthEvent{st.startEvent()}
}
// failWith 记录上游错误并产出 Anthropic error 事件(每流至多一次)。
func (st *AnthRespBridge) failWith(ev respStreamEvent) []AnthEvent {
if st.errMsg != "" {
return nil
}
st.sawTerminal = true
msg := ev.Message
if ev.Type == "response.failed" {
st.finishFrom(ev.Response)
if m := respErrorMsg(ev.Response); m != "" {
msg = m
}
}
if msg == "" {
msg = "上游返回错误事件 " + ev.Type
}
st.errMsg = msg
return []AnthEvent{st.errorEvent()}
}
// respErrorMsg 提取 response.failed 载荷中的错误消息。
func respErrorMsg(resp *respPayload) string {
if resp == nil || resp.Error == nil {
return ""
}
return (*resp.Error)["message"]
}
// errorEvent 按 Anthropic 流式协议产出 error 事件。
func (st *AnthRespBridge) errorEvent() AnthEvent {
return AnthEvent{Event: "error", Data: map[string]any{
"type": "error",
"error": map[string]string{"type": "api_error", "message": st.errMsg},
}}
}
// Err 返回上游错误消息;空串表示流正常(供调用日志)。
func (st *AnthRespBridge) Err() string { return st.errMsg }
// SawTerminal 报告是否收到过终态事件;false 即上游流提前终止。
func (st *AnthRespBridge) SawTerminal() bool { return st.sawTerminal }
// AnthMessageEvents 把完整 Messages 响应展开为标准事件序列,
// 供流式上游断流后的非流式降级结果推送(客户端协议不变)。
func AnthMessageEvents(m *aiwire.MessagesResponse) []AnthEvent {
events := []AnthEvent{{Event: "message_start", Data: map[string]any{
"type": "message_start",
"message": map[string]any{
"id": m.ID, "type": "message", "role": m.Role, "model": m.Model,
"content": []any{}, "stop_reason": nil,
"usage": map[string]int{"input_tokens": 0, "output_tokens": 0},
},
}}}
for i, b := range m.Content {
events = append(events, anthBlockEvents(i, b)...)
}
usage := map[string]int{"input_tokens": m.Usage.InputTokens, "output_tokens": m.Usage.OutputTokens}
if m.Usage.CacheReadInputTokens > 0 {
usage["cache_read_input_tokens"] = m.Usage.CacheReadInputTokens
}
events = append(events,
AnthEvent{Event: "message_delta", Data: map[string]any{
"type": "message_delta",
"delta": map[string]any{"stop_reason": m.StopReason, "stop_sequence": nil},
"usage": usage,
}},
AnthEvent{Event: "message_stop", Data: map[string]any{"type": "message_stop"}})
return events
}
// anthBlockEvents 把一个内容块展开为 start / delta / stop 三事件。
func anthBlockEvents(index int, b aiwire.AnthBlock) []AnthEvent {
var start, delta map[string]any
if b.Type == "tool_use" {
start = map[string]any{"type": "tool_use", "id": b.ID, "name": b.Name, "input": map[string]any{}}
delta = map[string]any{"type": "input_json_delta", "partial_json": string(b.Input)}
} else {
start = map[string]any{"type": "text", "text": ""}
delta = map[string]any{"type": "text_delta", "text": b.Text}
}
return []AnthEvent{
{Event: "content_block_start", Data: map[string]any{
"type": "content_block_start", "index": index, "content_block": start}},
{Event: "content_block_delta", Data: map[string]any{
"type": "content_block_delta", "index": index, "delta": delta}},
{Event: "content_block_stop", Data: map[string]any{
"type": "content_block_stop", "index": index}},
}
}
func (st *AnthRespBridge) textDelta(delta string) []AnthEvent { func (st *AnthRespBridge) textDelta(delta string) []AnthEvent {
var events []AnthEvent var events []AnthEvent
if !st.blockOpen || st.blockIsTool { if !st.blockOpen || st.blockIsTool {
@@ -378,7 +487,16 @@ func (st *AnthRespBridge) closeBlockEvent() AnthEvent {
} }
// Finish 在上游流结束后收尾:关块 → message_delta(stop_reason+usage)→ message_stop。 // Finish 在上游流结束后收尾:关块 → message_delta(stop_reason+usage)→ message_stop。
// 已发过 error 事件的流不再补终态;未见终态事件即 EOF 视为上游提前终止,
// 发 error 事件而非伪装正常结束(否则客户端拿到"成功的空消息")。
func (st *AnthRespBridge) Finish() []AnthEvent { func (st *AnthRespBridge) Finish() []AnthEvent {
if st.errMsg != "" {
return nil
}
if !st.sawTerminal {
st.errMsg = "上游流提前终止,未返回终态事件"
return []AnthEvent{st.errorEvent()}
}
var events []AnthEvent var events []AnthEvent
if !st.started { if !st.started {
st.started = true st.started = true
+99 -4
View File
@@ -164,10 +164,105 @@ func TestAnthRespBridge(t *testing.T) {
} }
} }
// TestAnthRespBridgeEmpty 断言空流也产出完整事件骨架。 // TestAnthRespBridgeFailure 断言异常流的错误透传:上游 error / response.failed
func TestAnthRespBridgeEmpty(t *testing.T) { // 事件转 Anthropic error 事件,提前 EOF(未见终态)不再伪装正常结束。
func TestAnthRespBridgeFailure(t *testing.T) {
cases := []struct {
name string
lines []string
wantKinds string
wantErr string
}{
{
name: "上游 error 事件透传",
lines: []string{`{"type":"error","message":"model overloaded"}`},
wantKinds: "error",
wantErr: "model overloaded",
},
{
name: "response.failed 提取错误消息",
lines: []string{
`{"type":"response.output_text.delta","delta":"你"}`,
`{"type":"response.failed","response":{"status":"failed","error":{"message":"content filtered"}}}`,
},
wantKinds: "message_start,content_block_start,content_block_delta,error",
wantErr: "content filtered",
},
{
name: "空流提前终止",
lines: nil,
wantKinds: "error",
wantErr: "上游流提前终止,未返回终态事件",
},
{
name: "输出中途 EOF 无终态",
lines: []string{
`{"type":"response.output_text.delta","delta":"你"}`,
},
wantKinds: "message_start,content_block_start,content_block_delta,error",
wantErr: "上游流提前终止,未返回终态事件",
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
st := NewAnthRespBridge("msg_1", "m1") st := NewAnthRespBridge("msg_1", "m1")
if got := bridgeEventTypes(st.Finish()); got != "message_start,message_delta,message_stop" { var events []AnthEvent
t.Errorf("空流事件序列 = %s", got) for _, l := range tc.lines {
events = append(events, st.Feed([]byte(l))...)
}
events = append(events, st.Finish()...)
if got := bridgeEventTypes(events); got != tc.wantKinds {
t.Fatalf("事件序列 = %s, want %s", got, tc.wantKinds)
}
if st.Err() != tc.wantErr {
t.Fatalf("Err() = %q, want %q", st.Err(), tc.wantErr)
}
})
}
}
// TestRespStreamErrorMsg 断言直通流错误事件消息提取。
func TestRespStreamErrorMsg(t *testing.T) {
cases := []struct {
name string
data string
want string
}{
{name: "error 事件", data: `{"type":"error","message":"boom"}`, want: "boom"},
{name: "error 无消息用占位", data: `{"type":"error"}`, want: "上游返回错误事件 error"},
{name: "failed 事件", data: `{"type":"response.failed","response":{"error":{"message":"bad"}}}`, want: "bad"},
{name: "正常事件返回空", data: `{"type":"response.completed","response":{}}`, want: ""},
{name: "非 JSON 返回空", data: `<html>`, want: ""},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
if got := RespStreamErrorMsg([]byte(tc.data)); got != tc.want {
t.Fatalf("RespStreamErrorMsg = %q, want %q", got, tc.want)
}
})
}
}
// TestAnthMessageEvents 断言非流式降级结果展开的事件序列与 usage。
func TestAnthMessageEvents(t *testing.T) {
msg := &aiwire.MessagesResponse{
ID: "msg_1", Type: "message", Role: "assistant", Model: "m1",
Content: []aiwire.AnthBlock{
{Type: "text", Text: "好"},
{Type: "tool_use", ID: "c1", Name: "f", Input: json.RawMessage(`{"a":1}`)},
},
StopReason: "tool_use",
Usage: aiwire.AnthUsage{InputTokens: 9, OutputTokens: 3, CacheReadInputTokens: 5},
}
events := AnthMessageEvents(msg)
want := "message_start,content_block_start,content_block_delta,content_block_stop," +
"content_block_start,content_block_delta,content_block_stop,message_delta,message_stop"
if got := bridgeEventTypes(events); got != want {
t.Fatalf("事件序列:\n got %s\nwant %s", got, want)
}
delta := events[len(events)-2].Data.(map[string]any)
usage := delta["usage"].(map[string]int)
if usage["input_tokens"] != 9 || usage["output_tokens"] != 3 || usage["cache_read_input_tokens"] != 5 {
t.Fatalf("usage = %+v", usage)
} }
} }
+29
View File
@@ -358,3 +358,32 @@ func (b *ChatRespBridge) Finish() []aiwire.ChatChunk {
// Usage 返回聚合到的用量(供调用日志),上游未报告时为 nil。 // Usage 返回聚合到的用量(供调用日志),上游未报告时为 nil。
func (b *ChatRespBridge) Usage() *aiwire.Usage { return b.usage } func (b *ChatRespBridge) Usage() *aiwire.Usage { return b.usage }
// ChatResponseChunks 把完整 Chat 响应展开为 chunk 序列(内容与工具调用 →
// 终块 → 可选 usage 块),供流式上游断流后的非流式降级结果推送。
func ChatResponseChunks(resp *aiwire.ChatResponse, includeUsage bool) []aiwire.ChatChunk {
if len(resp.Choices) == 0 {
return nil
}
choice := resp.Choices[0]
mk := func(delta aiwire.Delta, finish *string) aiwire.ChatChunk {
return aiwire.ChatChunk{ID: resp.ID, Object: "chat.completion.chunk", Created: resp.Created,
Model: resp.Model, Choices: []aiwire.ChunkChoice{{Index: 0, Delta: delta, FinishReason: finish}}}
}
delta := aiwire.Delta{Role: "assistant", Content: choice.Message.Content.Text}
for i, tc := range choice.Message.ToolCalls {
delta.ToolCalls = append(delta.ToolCalls, aiwire.ToolCallDelta{Index: i, ID: tc.ID,
Type: "function", Function: aiwire.FunctionCallDelta{Name: tc.Function.Name, Arguments: tc.Function.Arguments}})
}
finish := choice.FinishReason
chunks := []aiwire.ChatChunk{mk(delta, nil), mk(aiwire.Delta{}, &finish)}
if includeUsage {
usage := resp.Usage
if usage == nil {
usage = &aiwire.Usage{}
}
chunks = append(chunks, aiwire.ChatChunk{ID: resp.ID, Object: "chat.completion.chunk",
Created: resp.Created, Model: resp.Model, Choices: []aiwire.ChunkChoice{}, Usage: usage})
}
return chunks
}