歪门邪道：凭借 HttpClientHandler 阻拦恳求，体会 Semantic Kernel 插件
前天测验经过 one-api + dashscope(阿里云灵积) + qwen(通义千问)运转 Semantic Kernel 插件（Plugin） ，成果测验失利，详见前天的博文。
今日换一种方法测验，挑选了一个歪门邪道走走看，看能不能在不运用大模型的情况下让 Semantic Kernel 插件运转起来，这个歪门邪道便是从 Stephen Toub 那偷学到的一招 —— 凭借 DelegatingHandler(new HttpClientHandler()) 阻拦 HttpClient 恳求，直接以模仿数据进行呼应。
先创立一个 .NET 控制台项目
dotnet new console
dotnet add package Microsoft.SemanticKernel
dotnet add package Microsoft.Extensions.Http
参照 Semantic Kernel 源码中的示例代码创立一个十分简略的插件 LightPlugin
public class LightPlugin
{
public bool IsOn { get; set; } = false;
[KernelFunction]
[Description("帮看一下灯是开是关")]
public string GetState() => IsOn ? "on" : "off";
[KernelFunction]
[Description("开灯或许关灯")]
public string ChangeState(bool newState)
{
IsOn = newState;
var state = GetState();
Console.WriteLine(state == "on" ? $"[开灯啦]" : "[关灯咯]");
return state;
}
}
接着创立歪门邪道 BackdoorHandler，先完结一个最简略的功用，打印 HttpClient 恳求内容
public class BypassHandler() : DelegatingHandler(new HttpClientHandler())
{
protected override async Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request, CancellationToken cancellationToken)
{
Console.WriteLine(await request.Content!.ReadAsStringAsync());
// return await base.SendAsync(request, cancellationToken);
return new HttpResponseMessage(HttpStatusCode.OK);
}
}
然后携 LightPlugin 与 BypassHandler 创立 Semantic Kernel 的 Kernel
var builder = Kernel.CreateBuilder();
builder.Services.AddOpenAIChatCompletion("qwen-max", "sk-xxxxxx");
builder.Services.ConfigureHttpClientDefaults(b =>
b.ConfigurePrimaryHttpMessageHandler(() => new BypassHandler()));
builder.Plugins.AddFromType<LightPlugin>();
Kernel kernel = builder.Build();
再然后，发送带着 prompt 的恳求并获取呼应内容
var history = new ChatHistory();
history.AddUserMessage("请开灯");
Console.WriteLine("User > " + history[0].Content);
var chatCompletionService = kernel.GetRequiredService<IChatCompletionService>();
// Enable auto function calling
OpenAIPromptExecutionSettings openAIPromptExecutionSettings = new()
{
ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions
};
var result = await chatCompletionService.GetChatMessageContentAsync(
history,
executionSettings: openAIPromptExecutionSettings,
kernel: kernel);
Console.WriteLine("Assistant > " + result);
运转控制台程序，BypassHandler 就会在控制台输出恳求的 json 内容（为了阅览便利对json进行了格式化）：
{
"messages": [
{
"content": "Assistant is a large language model.",
"role": "system"
},
{
"content": "\u8BF7\u5F00\u706F",
"role": "user"
}
],
"temperature": 1,
"top_p": 1,
"n": 1,
"presence_penalty": 0,
"frequency_penalty": 0,
"model": "qwen-max",
"tools": [
{
"function": {
"name": "LightPlugin-GetState",
"description": "\u5E2E\u770B\u4E00\u4E0B\u706F\u662F\u5F00\u662F\u5173",
"parameters": {
"type": "object",
"required": [],
"properties": {}
}
},
"type": "function"
},
{
"function": {
"name": "LightPlugin-ChangeState",
"description": "\u5F00\u706F\u6216\u8005\u5173\u706F",
"parameters": {
"type": "object",
"required": [
"newState"
],
"properties": {
"newState": {
"type": "boolean"
}
}
}
},
"type": "function"
}
],
"tool_choice": "auto"
}
为了能反序列化这个 json ，咱们需求界说一个类型 ChatCompletionRequest，Sermantic Kernel 中没有现成能够运用的，完结代码如下：
public class ChatCompletionRequest
{
[JsonPropertyName("messages")]
public IReadOnlyList<RequestMessage>? Messages { get; set; }
[JsonPropertyName("temperature")]
public double Temperature { get; set; } = 1;
[JsonPropertyName("top_p")]
public double TopP { get; set; } = 1;
[JsonPropertyName("n")]
public int? N { get; set; } = 1;
[JsonPropertyName("presence_penalty")]
public double PresencePenalty { get; set; } = 0;
[JsonPropertyName("frequency_penalty")]
public double FrequencyPenalty { get; set; } = 0;
[JsonPropertyName("model")]
public required string Model { get; set; }
[JsonPropertyName("tools")]
public IReadOnlyList<Tool>? Tools { get; set; }
[JsonPropertyName("tool_choice")]
public string? ToolChoice { get; set; }
}
public class RequestMessage
{
[JsonPropertyName("role")]
public string? Role { get; set; }
[JsonPropertyName("name")]
public string? Name { get; set; }
[JsonPropertyName("content")]
public string? Content { get; set; }
}
public class Tool
{
[JsonPropertyName("function")]
public FunctionDefinition? Function { get; set; }
[JsonPropertyName("type")]
public string? Type { get; set; }
}
public class FunctionDefinition
{
[JsonPropertyName("name")]
public string? Name { get; set; }
[JsonPropertyName("description")]
public string? Description { get; set; }
[JsonPropertyName("parameters")]
public ParameterDefinition Parameters { get; set; }
public struct ParameterDefinition
{
[JsonPropertyName("type")]
public required string Type { get; set; }
[JsonPropertyName("description")]
public string? Description { get; set; }
[JsonPropertyName("required")]
public string[]? Required { get; set; }
[JsonPropertyName("properties")]
public Dictionary<string, PropertyDefinition>? Properties { get; set; }
public struct PropertyDefinition
{
[JsonPropertyName("type")]
public required PropertyType Type { get; set; }
}
[JsonConverter(typeof(JsonStringEnumConverter))]
public enum PropertyType
{
Number,
String,
Boolean
}
}
}
有了这个类，咱们就能够从恳求中获取对应 Plugin 的 function 信息，比方下面的代码：
var function = chatCompletionRequest?.Tools.FirstOrDefault(x => x.Function.Description.Contains("开灯"))?.Function;
var functionName = function.Name;
var parameterName = function.Parameters.Properties.FirstOrDefault(x => x.Value.Type == PropertyType.Boolean).Key;
接下来便是歪门邪道的要害，直接在 BypassHandler 中呼应 Semantic Kernel 经过 OpenAI.ClientCore 宣布的 http 恳求。
首要创立用于 json 序列化的类 ChatCompletionResponse：
public class ChatCompletionResponse
{
[JsonPropertyName("id")]
public string? Id { get; set; }
[JsonPropertyName("object")]
public string? Object { get; set; }
[JsonPropertyName("created")]
public long Created { get; set; }
[JsonPropertyName("model")]
public string? Model { get; set; }
[JsonPropertyName("usage"), JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
public Usage? Usage { get; set; }
[JsonPropertyName("choices")]
public List<Choice>? Choices { get; set; }
}
public class Choice
{
[JsonPropertyName("message")]
public ResponseMessage? Message { get; set; }
/// <summary>
/// The message in this response (when streaming a response).
/// </summary>
[JsonPropertyName("delta"), JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
public ResponseMessage? Delta { get; set; }
[JsonPropertyName("finish_reason")]
public string? FinishReason { get; set; }
/// <summary>
/// The index of this response in the array of choices.
/// </summary>
[JsonPropertyName("index")]
public int Index { get; set; }
}
public class ResponseMessage
{
[JsonPropertyName("role")]
public string? Role { get; set; }
[JsonPropertyName("name"), JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
public string? Name { get; set; }
[JsonPropertyName("content"), JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
public string? Content { get; set; }
[JsonPropertyName("tool_calls")]
public IReadOnlyList<ToolCall>? ToolCalls { get; set; }
}
public class ToolCall
{
[JsonPropertyName("id")]
public string? Id { get; set; }
[JsonPropertyName("function")]
public FunctionCall? Function { get; set; }
[JsonPropertyName("type")]
public string? Type { get; set; }
}
public class Usage
{
[JsonPropertyName("prompt_tokens")]
public int PromptTokens { get; set; }
[JsonPropertyName("completion_tokens")]
public int CompletionTokens { get; set; }
[JsonPropertyName("total_tokens")]
public int TotalTokens { get; set; }
}
public class FunctionCall
{
[JsonPropertyName("name")]
public string Name { get; set; } = string.Empty;
[JsonPropertyName("arguments")]
public string Arguments { get; set; } = string.Empty;
}
先试试不履行 function calling ，直接以 assistant 人物回复一句话
public class BypassHandler() : DelegatingHandler(new HttpClientHandler())
{
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
var chatCompletion = new ChatCompletionResponse
{
Id = Guid.NewGuid().ToString(),
Model = "fake-mode",
Object = "chat.completion",
Created = DateTimeOffset.Now.ToUnixTimeSeconds(),
Choices =
[
new()
{
Message = new ResponseMessage
{
Content = "自己着手，锦衣玉食",
Role = "assistant"
},
FinishReason = "stop"
}
]
};
var json = JsonSerializer.Serialize(chatCompletion, GetJsonSerializerOptions());
return new HttpResponseMessage
{
Content = new StringContent(json, Encoding.UTF8, "application/json")
};
}
}
运转控制台程序，输出如下：
User > 请开灯
Assistant > 自己着手，锦衣玉食
成功呼应，到此，歪门邪道成功了一半。
接下来在之前创立的 chatCompletion 基础上增加针对 function calling 的 ToolCall 部分。
先准备好 ChangeState(bool newState) 的参数值
Dictionary<string, bool> arguments = new()
{
{ parameterName, true }
};
并将回复内容由 "自己着手，锦衣玉食" 改为 "客官，灯已开"
Message = new ResponseMessage
{
Content = "客官，灯已开",
Role = "assistant"
}
然后为 chatCompletion 创立 ToolCalls 实例用于呼应 function calling
var messages = chatCompletionRequest.Messages;
if (messages.First(x => x.Role == "user").Content.Contains("开灯") == true)
{
chatCompletion.Choices[0].Message.ToolCalls = new List<ToolCall>()
{
new ToolCall
{
Id = Guid.NewGuid().ToString(),
Type = "function",
Function = new FunctionCall
{
Name = function.Name,
Arguments = JsonSerializer.Serialize(arguments, GetJsonSerializerOptions())
}
}
};
}
运转控制台程序看看作用
User > 请开灯
[开灯啦]
[开灯啦]
[开灯啦]
[开灯啦]
[开灯啦]
Assistant > 客官，灯已开
耶！成功开灯！可是，居然开了5次，差点把灯给开爆了。
在 BypassHandler 中打印一下恳求内容看看哪里出了问题
var json = await request.Content!.ReadAsStringAsync();
Console.WriteLine(json);
本来别离恳求/呼应了5次，第2次恳求开端，json 中 messages 部分多了 tool_calls 与 tool_call_id 内容
{
"messages": [
{
"content": "\u5BA2\u5B98\uFF0C\u706F\u5DF2\u5F00",
"tool_calls": [
{
"function": {
"name": "LightPlugin-ChangeState",
"arguments": "{\u0022newState\u0022:true}"
},
"type": "function",
"id": "76f8dead-b5ad-4e6d-b343-7f78d68fac8e"
}
],
"role": "assistant"
},
{
"content": "on",
"tool_call_id": "76f8dead-b5ad-4e6d-b343-7f78d68fac8e",
"role": "tool"
}
]
}
这时茅塞顿开，之前 AI assistant 对 function calling 的呼应仅仅让 Plugin 履行对应的 function，assistant 还需求根据履行的成果决议下一下做什么，第2次恳求中的 tool_calls 与 tool_call_id 便是为了告知 assistant 履行的成果，所以，还需求针对这个恳求进行专门的呼应。
到了歪门邪道最终100米冲刺的时间！
给 RequestMessage 增加 ToolCallId 特点
public class RequestMessage
{
[JsonPropertyName("role")]
public string? Role { get; set; }
[JsonPropertyName("name")]
public string? Name { get; set; }
[JsonPropertyName("content")]
public string? Content { get; set; }
[JsonPropertyName("tool_call_id")]
public string? ToolCallId { get; set; }
}
在 BypassHandler 中呼应时判别一下 ToolCallId，假如是针对 Plugin 的 function 履行成果的恳求，只回来 Message.Content，不进行 function calling 呼应
var messages = chatCompletionRequest.Messages;
var toolCallId = "76f8dead- b5ad-4e6d-b343-7f78d68fac8e";
var toolCallIdMessage = messages.FirstOrDefault(x => x.Role == "tool" && x.ToolCallId == toolCallId);
if (toolCallIdMessage != null && toolCallIdMessage.Content == "on")
{
chatCompletion.Choices[0].Message.Content = "客官，灯已开";
}
else if (messages.First(x => x.Role == "user").Content.Contains("开灯") == true)
{
chatCompletion.Choices[0].Message.Content = "";
//..
}
改善代码完结，到了最终10米冲刺的时间，再次运转控制台程序
User > 请开灯
[开灯啦]
Assistant > 客官，灯已开
只要一次开灯，冲刺成功，歪门邪道走通，用这种方法体会一下 Semantic Kernel Plugin，也别有一番风味。
完好示例代码已上传到 github https://github.com/cnblogs-dudu/sk-plugin-sample-101
弥补：假如不运用 BypassHandler，直接接入 OpenAI，运转成果如下
User > 请开灯
[开灯啦]
Assistant > 灯现已打开了。
