Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suggested path for JsonSchema.Net integration #1191

Closed
wants to merge 10 commits into from
Closed

Suggested path for JsonSchema.Net integration #1191

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
f25b0e0
added draft4 keyword implementations for jsonschema.net; openapidocum…
gregsdennis Mar 24, 2023
f9eb5d0
pass-through support for draft6+ exclusive*
gregsdennis Mar 24, 2023
8a6192a
initial experimentation with using jsonschema.net types more directly…
gregsdennis Mar 25, 2023
09568a2
added more draft 4 support (need to update jsonschema.net); added dra…
gregsdennis Mar 26, 2023
d66ea86
converted some of the tests to use JsonSchemaBuilder
gregsdennis Mar 26, 2023
2be6fe7
updated header, media type, and parameter to use jsonschema
gregsdennis Mar 26, 2023
a1db8b6
update deserializers that'll probably be removed eventually anyway; u…
gregsdennis Mar 27, 2023
832efcd
just serialization things
gregsdennis Mar 27, 2023
6177047
use jsonschema.net v4.0.1
gregsdennis Mar 27, 2023
c14ca66
cleaned up pkg references
gregsdennis Mar 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions src/Microsoft.OpenApi.Readers/Microsoft.OpenApi.Readers.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,6 @@
</PropertyGroup>

<ItemGroup>
<PackageReference Include="JsonSchema.Net" Version="4.0.0" />
<PackageReference Include="JsonSchema.Net.OpenApi" Version="1.1.0-beta3" />
<PackageReference Include="SharpYaml" Version="2.1.0" />
</ItemGroup>

Expand Down
8 changes: 7 additions & 1 deletion src/Microsoft.OpenApi.Readers/OpenApiTextReaderReader.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@

using System.IO;
using System.Linq;
using System.Text.Json;
using System.Threading.Tasks;
using Microsoft.OpenApi.Interfaces;
using Microsoft.OpenApi.Models;
Expand Down Expand Up @@ -50,7 +51,12 @@ public OpenApiDocument Read(TextReader input, out OpenApiDiagnostic diagnostic)
return new OpenApiDocument();
}

return new OpenApiYamlDocumentReader(this._settings).Read(yamlDocument, out diagnostic);
var asJsonNode = yamlDocument.ToJsonNode();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we are going to make the switch to do Yaml => JsonNodes => OpenAPIDocument. This will facilitate us to use the JsonSchema.Net Examples and hopefully get rid of OpenApiAny and all of the OpenApiPrimitive classes.


diagnostic = null; // TBD
return asJsonNode.Deserialize<OpenApiDocument>();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I honestly don't see how using Deserialize is going to work for us. Using an automatic deserialization mechanism strongly couples the serialized format to the in-memory model. We specifically wrote the deserializer classes so we can deserialize any version of OpenAPI into the "latest" in-memory model.

It is not clear to me from this PR how you are proposing we continue allowing our users to import OpenAPI without caring what version of OAS the input format is. And to clarify, this is one of our most important features.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strictly speaking, this change isn't necessary because this code will be deleted.

Using an automatic deserialization mechanism strongly couples the serialized format to the in-memory model.

How so? I'm doing the actual parsing with YamlSharp; both YAML and JSON are supported through this. It parses into the YamlDocument model, which I then translate to JsonNode. Then I just use the built-in JsonSerializer to deserialize it into OpenApiDocument.

Going the other way, if you want JSON, you just use JsonSerializer.Serialize(openApiDoc). If you want YAML, you use this method which we'll need to expose as an extension.

We specifically wrote the deserializer classes so we can deserialize any version of OpenAPI into the "latest" in-memory model.

This supports the same thing. I can drop a 2.0 or a 3.1 schema into my DevTest test and it still deserializes just fine. Users don't need to care which OpenAPI version they have.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does the JsonDeserializer.Deserialize convert body parameters into a requestBody object? How does the JsonSerializer.Deserialize know how to convert produces/consumes into media type objects? We have many of these kinds of translations that System.Text.Json knows nothing about.

Copy link
Author

@gregsdennis gregsdennis Apr 2, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't follow. The paths use OpenApiParameter which isn't associated with OpenApiRequestBody (only used in components) that I can see.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In our V2 Deserializer, we convert parameters that are of type body or form into a request body. See code here:

https://github.com/microsoft/OpenAPI.NET/blob/vnext/src/Microsoft.OpenApi.Readers/V2/OpenApiOperationDeserializer.cs#L126-L138


//return new OpenApiYamlDocumentReader(this._settings).Read(yamlDocument, out diagnostic);
}

/// <summary>
Expand Down
6 changes: 3 additions & 3 deletions src/Microsoft.OpenApi.Readers/V2/OpenApiDocumentDeserializer.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,9 +63,9 @@ internal static partial class OpenApiV2Deserializer
o.Components = new OpenApiComponents();
}

o.Components.Schemas = n.CreateMapWithReference(
ReferenceType.Schema,
LoadSchema);
//o.Components.Schemas = n.CreateMapWithReference(
// ReferenceType.Schema,
// LoadSchema);
}
},
{
Expand Down
48 changes: 24 additions & 24 deletions src/Microsoft.OpenApi.Readers/V2/OpenApiHeaderDeserializer.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT license.

using System;
Expand Down Expand Up @@ -136,33 +136,33 @@ internal static partial class OpenApiV2Deserializer
private static readonly AnyFieldMap<OpenApiHeader> _headerAnyFields =
new AnyFieldMap<OpenApiHeader>
{
{
OpenApiConstants.Default,
new AnyFieldMapParameter<OpenApiHeader>(
p => p.Schema?.Default,
(p, v) =>
{
if(p.Schema == null) return;
p.Schema.Default = v;
},
p => p.Schema)
}
//{
// OpenApiConstants.Default,
// new AnyFieldMapParameter<OpenApiHeader>(
// p => p.Schema?.Default,
// (p, v) =>
// {
// if(p.Schema == null) return;
// p.Schema.Default = v;
// },
// p => p.Schema)
//}
};

private static readonly AnyListFieldMap<OpenApiHeader> _headerAnyListFields =
new AnyListFieldMap<OpenApiHeader>
{
{
OpenApiConstants.Enum,
new AnyListFieldMapParameter<OpenApiHeader>(
p => p.Schema?.Enum,
(p, v) =>
{
if(p.Schema == null) return;
p.Schema.Enum = v;
},
p => p.Schema)
},
//{
// OpenApiConstants.Enum,
// new AnyListFieldMapParameter<OpenApiHeader>(
// p => p.Schema?.Enum,
// (p, v) =>
// {
// if(p.Schema == null) return;
// p.Schema.Enum = v;
// },
// p => p.Schema)
//},
};

public static OpenApiHeader LoadHeader(ParseNode node)
Expand All @@ -177,7 +177,7 @@ public static OpenApiHeader LoadHeader(ParseNode node)
var schema = node.Context.GetFromTempStorage<OpenApiSchema>("schema");
if (schema != null)
{
header.Schema = schema;
//header.Schema = schema;
node.Context.SetTempStorage("schema", null);
}

Expand Down
26 changes: 13 additions & 13 deletions src/Microsoft.OpenApi.Readers/V2/OpenApiOperationDeserializer.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,19 +163,19 @@ private static OpenApiRequestBody CreateFormBody(ParsingContext context, List<Op
{
var mediaType = new OpenApiMediaType
{
Schema = new OpenApiSchema
{
Properties = formParameters.ToDictionary(
k => k.Name,
v =>
{
var schema = v.Schema;
schema.Description = v.Description;
schema.Extensions = v.Extensions;
return schema;
}),
Required = new HashSet<string>(formParameters.Where(p => p.Required).Select(p => p.Name))
}
//Schema = new OpenApiSchema
//{
// Properties = formParameters.ToDictionary(
// k => k.Name,
// v =>
// {
// var schema = v.Schema;
// schema.Description = v.Description;
// schema.Extensions = v.Extensions;
// return schema;
// }),
// Required = new HashSet<string>(formParameters.Where(p => p.Required).Select(p => p.Name))
//}
};

var consumes = context.GetFromTempStorage<List<string>>(TempStorageKeys.OperationConsumes) ??
Expand Down
61 changes: 31 additions & 30 deletions src/Microsoft.OpenApi.Readers/V2/OpenApiParameterDeserializer.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using Json.Schema;
using Microsoft.OpenApi.Extensions;
using Microsoft.OpenApi.Models;
using Microsoft.OpenApi.Readers.ParseNodes;
Expand Down Expand Up @@ -130,7 +131,7 @@ internal static partial class OpenApiV2Deserializer
{
"schema", (o, n) =>
{
o.Schema = LoadSchema(n);
//o.Schema = LoadSchema(n);
}
},
};
Expand All @@ -144,35 +145,35 @@ internal static partial class OpenApiV2Deserializer
private static readonly AnyFieldMap<OpenApiParameter> _parameterAnyFields =
new AnyFieldMap<OpenApiParameter>
{
{
OpenApiConstants.Default,
new AnyFieldMapParameter<OpenApiParameter>(
p => p.Schema?.Default,
(p, v) => {
if (p.Schema != null || v != null)
{
GetOrCreateSchema(p).Default = v;
}
},
p => p.Schema)
}
//{
// OpenApiConstants.Default,
// new AnyFieldMapParameter<OpenApiParameter>(
// p => p.Schema?.Default,
// (p, v) => {
// if (p.Schema != null || v != null)
// {
// GetOrCreateSchema(p).Default = v;
// }
// },
// p => p.Schema)
//}
};

private static readonly AnyListFieldMap<OpenApiParameter> _parameterAnyListFields =
new AnyListFieldMap<OpenApiParameter>
{
{
OpenApiConstants.Enum,
new AnyListFieldMapParameter<OpenApiParameter>(
p => p.Schema?.Enum,
(p, v) => {
if (p.Schema != null || v != null && v.Count > 0)
{
GetOrCreateSchema(p).Enum = v;
}
},
p => p.Schema)
},
//{
// OpenApiConstants.Enum,
// new AnyListFieldMapParameter<OpenApiParameter>(
// p => p.Schema?.Enum,
// (p, v) => {
// if (p.Schema != null || v != null && v.Count > 0)
// {
// GetOrCreateSchema(p).Enum = v;
// }
// },
// p => p.Schema)
//},
};

private static void LoadStyle(OpenApiParameter p, string v)
Expand Down Expand Up @@ -208,20 +209,20 @@ private static OpenApiSchema GetOrCreateSchema(OpenApiParameter p)
{
if (p.Schema == null)
{
p.Schema = new OpenApiSchema();
p.Schema = JsonSchema.Empty;
}

return p.Schema;
return new OpenApiSchema();
}

private static OpenApiSchema GetOrCreateSchema(OpenApiHeader p)
{
if (p.Schema == null)
{
p.Schema = new OpenApiSchema();
p.Schema = JsonSchema.Empty;
}

return p.Schema;
return new OpenApiSchema();
}

private static void ProcessIn(OpenApiParameter o, ParseNode n)
Expand Down Expand Up @@ -284,7 +285,7 @@ public static OpenApiParameter LoadParameter(ParseNode node, bool loadRequestBod
var schema = node.Context.GetFromTempStorage<OpenApiSchema>("schema");
if (schema != null)
{
parameter.Schema = schema;
//parameter.Schema = schema;
node.Context.SetTempStorage("schema", null);
}

Expand Down
19 changes: 10 additions & 9 deletions src/Microsoft.OpenApi.Readers/V2/OpenApiResponseDeserializer.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
// Licensed under the MIT license.

using System.Collections.Generic;
using Json.Schema;
using Microsoft.OpenApi.Extensions;
using Microsoft.OpenApi.Models;
using Microsoft.OpenApi.Readers.ParseNodes;
Expand Down Expand Up @@ -51,13 +52,13 @@ internal static partial class OpenApiV2Deserializer
private static readonly AnyFieldMap<OpenApiMediaType> _mediaTypeAnyFields =
new AnyFieldMap<OpenApiMediaType>
{
{
OpenApiConstants.Example,
new AnyFieldMapParameter<OpenApiMediaType>(
m => m.Example,
(m, v) => m.Example = v,
m => m.Schema)
}
//{
// OpenApiConstants.Example,
// new AnyFieldMapParameter<OpenApiMediaType>(
// m => m.Example,
// (m, v) => m.Example = v,
// m => m.Schema)
//}
};

private static void ProcessProduces(MapNode mapNode, OpenApiResponse response, ParsingContext context)
Expand All @@ -78,7 +79,7 @@ private static void ProcessProduces(MapNode mapNode, OpenApiResponse response, P
{
foreach (var produce in produces)
{
var schema = context.GetFromTempStorage<OpenApiSchema>(TempStorageKeys.ResponseSchema, response);
var schema = context.GetFromTempStorage<JsonSchema>(TempStorageKeys.ResponseSchema, response);

if (response.Content.ContainsKey(produce) && response.Content[produce] != null)
{
Expand Down Expand Up @@ -131,7 +132,7 @@ private static void LoadExample(OpenApiResponse response, string mediaType, Pars
{
mediaTypeObject = new OpenApiMediaType
{
Schema = node.Context.GetFromTempStorage<OpenApiSchema>(TempStorageKeys.ResponseSchema, response)
Schema = node.Context.GetFromTempStorage<JsonSchema>(TempStorageKeys.ResponseSchema, response)
};
response.Content.Add(mediaType, mediaTypeObject);
}
Expand Down
2 changes: 1 addition & 1 deletion src/Microsoft.OpenApi.Readers/V3/OpenApiComponentsDeserializer.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ internal static partial class OpenApiV3Deserializer
{
private static FixedFieldMap<OpenApiComponents> _componentsFixedFields = new FixedFieldMap<OpenApiComponents>
{
{"schemas", (o, n) => o.Schemas = n.CreateMapWithReference(ReferenceType.Schema, LoadSchema)},
//{"schemas", (o, n) => o.Schemas = n.CreateMapWithReference(ReferenceType.Schema, LoadSchema)},
{"responses", (o, n) => o.Responses = n.CreateMapWithReference(ReferenceType.Response, LoadResponse)},
{"parameters", (o, n) => o.Parameters = n.CreateMapWithReference(ReferenceType.Parameter, LoadParameter)},
{"examples", (o, n) => o.Examples = n.CreateMapWithReference(ReferenceType.Example, LoadExample)},
Expand Down
2 changes: 1 addition & 1 deletion src/Microsoft.OpenApi.Readers/V3/OpenApiHeaderDeserializer.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ internal static partial class OpenApiV3Deserializer
{
"schema", (o, n) =>
{
o.Schema = LoadSchema(n);
//o.Schema = LoadSchema(n);
}
},
{
Expand Down
32 changes: 16 additions & 16 deletions src/Microsoft.OpenApi.Readers/V3/OpenApiMediaTypeDeserializer.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ internal static partial class OpenApiV3Deserializer
{
OpenApiConstants.Schema, (o, n) =>
{
o.Schema = LoadSchema(n);
//o.Schema = LoadSchema(n);
}
},
{
Expand Down Expand Up @@ -54,27 +54,27 @@ internal static partial class OpenApiV3Deserializer

private static readonly AnyFieldMap<OpenApiMediaType> _mediaTypeAnyFields = new AnyFieldMap<OpenApiMediaType>
{
{
OpenApiConstants.Example,
new AnyFieldMapParameter<OpenApiMediaType>(
s => s.Example,
(s, v) => s.Example = v,
s => s.Schema)
}
//{
// OpenApiConstants.Example,
// new AnyFieldMapParameter<OpenApiMediaType>(
// s => s.Example,
// (s, v) => s.Example = v,
// s => s.Schema)
//}
};


private static readonly AnyMapFieldMap<OpenApiMediaType, OpenApiExample> _mediaTypeAnyMapOpenApiExampleFields =
new AnyMapFieldMap<OpenApiMediaType, OpenApiExample>
{
{
OpenApiConstants.Examples,
new AnyMapFieldMapParameter<OpenApiMediaType, OpenApiExample>(
m => m.Examples,
e => e.Value,
(e, v) => e.Value = v,
m => m.Schema)
}
//{
// OpenApiConstants.Examples,
// new AnyMapFieldMapParameter<OpenApiMediaType, OpenApiExample>(
// m => m.Examples,
// e => e.Value,
// (e, v) => e.Value = v,
// m => m.Schema)
//}
};

public static OpenApiMediaType LoadMediaType(ParseNode node)
Expand Down
Loading