Json object modeled class and inheritance when using Structured Outputs

[HunterProduction]

Hello!

Thanks for the splendid work on this repo. I was trying to upgrade my code to implement Structured Outputs from chat completions.
I created one class called BaseStructuredResponse which I want to be used as a base for other possible wanted response formats in future

public abstract class BaseStructuredResponse
{
    /// <summary>
    /// The response retrieved by the chat completion without any processing performed.
    /// </summary>
    [JsonProperty("raw")]
    public string Raw { get; protected set; }

    /// <summary>
    /// Part of the response that represents the natural human-like answer to the speaker request.
    /// </summary>
    [JsonProperty("answer")]
    public string Answer { get; protected set; }

    /// <summary>
    /// Part of the response that stores results from the tool calls, if any was detected.
    /// </summary>
    [JsonProperty("toolcall_results")]
    public IReadOnlyList<string> ToolCallResults { get; protected set; }

    public override string ToString() => JsonConvert.SerializeObject(this);
}

Then I wrote for example a child class CodeResponse which adds another field code_block:

public class CodeResponse : BaseStructuredResponse
{
    /// <summary>
    /// Part of the response that contains any code provided by the chat response.
    /// </summary>
    [JsonProperty("code_block")]
    public string CodeBlock {  get; private set; }

    public override string ToString() => JsonConvert.SerializeObject(this);
}

In my components, I wrote a generic method to perform chat completion:

protected virtual async Task<T> PerformChatCompletion<T>(ChatRequest chatRequest) where T : BaseStructuredResponse
{
    if (!IsReady)
    {
        Log($"[{GetType().Name}] LanguageProcessor not ready");
        return default;
    }

    Log($"[{GetType().Name}] Responding...");

    var start = DateTime.Now;
    //var response = await Api.ChatEndpoint.GetCompletionAsync(chatRequest);
    var (structuredResponse, response) = await Api.ChatEndpoint.GetCompletionAsync<T>(chatRequest);
    var latency = DateTime.Now.Subtract(start).TotalMilliseconds;

    var choice = response.FirstChoice.Message;

    Log($"[{GetType().Name}] Request latency: {latency} | Finish reason: {response.FirstChoice.FinishReason}\nStructured Response:\n\n{structuredResponse}");

    // Add new response to the history
    MessageHistory.Add(choice);

    return structuredResponse;
}

The problem is that the chat response doesn't seem to be aware of the Json properties declared in the base class, but populates only the CodeBlock field. I'm quite sure of this because I tried to write the Answer field directly in the CodeResponse class, and the chat completion splitted the two sections correctly.

Is this inheritance architecture actually supported? Or do I have to give up on this road?

Thank you!

[HunterProduction]

An update with the work I proceeded today. The solution I found for now is to make the properties in BaseStructuredResponse abstract too. The child class simply implement them with an override without changing anything, just to be sure the properties are explicitely declared inside the class used for the Structured Output:

public class CodeResponse : BaseStructuredResponse
{
    public override string RawResponse { get; protected set; }

    public override string Answer { get; protected set; }

    public override IReadOnlyList<string> ToolCallResults { get; protected set; }

    /// <summary>
    /// Part of the response that contains any code provided by the chat response.
    /// </summary>
    [JsonProperty("code_block")]
    [Description("Part of the response that contains any code provided by the chat response.")]
    public string CodeBlock {  get; private set; }
}

It's not the most elegant or scalable approach but for now it works. Do you have any answer on why the previous solution wasn't working properly?

[StephenHodgson]

Howdy @HunterProduction! Sorry I didn't get back right away.

I think this seems okay. Just be sure to follow the rules for Newtonsoft serialization for inherited members.

It might also be worth while to maybe simplify your schema or ask chatGPT for a schema it thinks would fit your use case and generate Json object models from that.

Let me know what ends up working for you!