[Shopify] Add Metafields to Products and Variants (#26185)

This is a PR without a dedicated issue as it's part of delivery for
development agreed directly with @AndreiPanko

Here's the overview of the changes:

Obsolete fields:
• Obsoleted Value Type as it‘s no longer used in Shopify API -> Replaced
with Type field
• Obsoleted Owner Resource field. I‘m not sure why this was a free text
that was then always populated through a case statement. It‘s replaced
with an Owner Type field that is an Enum.

New enums and interfaces:
• „Shpfy Metafield Type“ and „Shpfy IMetafield Type“ – Used for handling
the metafield type. The interface defines if the type has a special
AssistEdit, logic to execute the assist edit, and validations when free
input is allowed for a value. I have a couple of questions here, see
below. Rating, Rich Text, and List Types are not included, Rating and
Rich Text options are added but commented out so it‘s clear that it was
intentional. It also has String and Integer types which are there to
support backward compatibility, if old values are still in Shopify.
• „Shpfy Metafield Owner Type“ and „Shpfy IMetafield Owner Type“ – Used
for building the full owner resource ID
(gid://shopify/<ownerType>/<ownerId>) and for retrieving metafield IDs
of a single resource (used during export, to identify which metafields
should be sent for update based on UpdatedAt)
• Metafield Dimenion/Volume/Weight Type – three enums to support a
selection of Units when assist editing these metafield types

Pages:
• „Shpfy Metafields“ – used for viewing and editing metafields for a
resource. It‘s only editable if Shop is syncing Products to Shopify and
Can update Shopify products. On Insert of a new record, it‘s immediately
sent to Shopify and is only inserted in the DB when we get back the
Shopify ID for the metafield. Types that have AssistEdit functionality
defined cannot be edited directly on the line. OnValidate of Value a
check is executed to ensure value is correct for a type.
• „Shpfy Metafield Assist Edit“ – used for assist edit of complex
metafield types. It has several groups defined, and only the appropriate
one is displayed to the user.
• Action for display metafields for a resource has been added to the
Products and Variants pages.

GraphQL query changes:
• ProductById and VariantById have been modified so that they retrieve
the first 50 metafields (previously the first 10). Previously there was
a filter to only retrieve Metafields with the
„Microsoft.Dynamics365.BusinessCentral“ namespace. I‘ve removed that to
get all the metafields of a resource.
• ProductMetafieldIds and VariantMetafieldIds were added. These are used
during the export of metafields to determine which BC metafields should
be sent for an update
• MetafieldsSet – This is used to create/update metafields for a
resource

Changes to API codeunits:
• Product API – on UpdateShopifyProductFields a metafields part of JSON
is now passed to Metafields API where it creates or updates Metafields
in BC based on information retrieved from Shopify
• Variant API – on UpdateShopifyVariantFields the metafields part of the
JSON is passed to Metafields API, same as above.
• „Shpfy Product Export“ – on UpdateProductData execute Metafields API
to update/create metafields for the product and all variants that belong
to the product
• „Metafield API“ – Implements logic to parse the „metafields“JSON that
is retrieved with products/variants during import and creates/updates
metafield records in BC.
Adds logic to export metafields for a specific resource. Only sends an
update for metafields that have a lower UpdatedAt timestamp than BC
record‘s last updated at timestamp. It batches the Metafields to 25 at a
time, as that‘s the max MetafieldsSet mutation can handle as per the
documentation.

Added logic to delete metafields on delete of a product/variant


A couple of questions or concerns:
• New metafields that get created in Shopify do not have a Metafield
Definition attached to them. Should we care about this right now? If
yes, should we also store metafield definitions in BC, or just create
new definitions if any are missing in Shopify?
• I‘ve obsoleted the two fields and one enum, should I be wrapping the
obsoletions in some preprocessing symbols? Which one?
• I‘m doing some regex validations when checking if the value of
metafield is correct. I‘m not sure if the one for mixed reference and
file reference is correct. I‘d appreciate it if someone could take a
look at that.
• If value validation fails, it throws a pretty „simple“ error message
right now, should I make this more expressive?
• Currently, the „update metafields“ has no error handling in place if
the update query fails. Technically I‘m trying to catch everything on
data input, but there can always be unexpected situations. How should I
handle this?
• The code for sending the metafield to Shopify immediately after it‘s
inserted through UI is currently on the Page‘s OnInsertRecord trigger.
I‘m not sure about it, should I move it to the table trigger? The
hesitation is because there already is some logic on the table from the
partial implementation of metafields on Customers.
• I‘ve placed interface implementations in a separate folder under
Codeunits folder, that‘s not the usual practice on MS apps, so I wanted
to double-check it.
• Money metafield type needs to have the same currency as the shop,
which leads to this completely separate code path after validating the
Value. I‘m not sure if there‘s a better approach to handle this.
• I saw that most procedures in the app have documentation triggers that
are not really beneficial. I‘ve only added them to the public or
internal procedures. I‘ve also skipped on most object documentation
triggers.

Fixes #26819
Fixes
[AB#443908](https://dynamicssmb2.visualstudio.com/1fcb79e7-ab07-432a-a3c6-6cf5a88ba4a5/_workitems/edit/443908)

---------

Co-authored-by: Tine Staric <[email protected]>
Co-authored-by: Jesper Schulz-Wedde <[email protected]>
Co-authored-by: Onat Buyukakkus <[email protected]>

Apps/W1/Shopify/app/app.json

@@ -21,7 +21,7 @@
     "idRanges":  [
                      {
                          "from":  30100,
-                         "to":  30350
+                         "to":  30360
                      }
                  ],
     "internalsVisibleTo":  [

Apps/W1/Shopify/app/src/Customers/Codeunits/ShpfyCustomerExport.Codeunit.al

@@ -64,7 +64,7 @@ codeunit 30116 "Shpfy Customer Export"
             Metafield.Namespace := 'Microsoft.Dynamics365.BusinessCentral';
             Metafield.Validate("Parent Table No.", Database::"Shpfy Customer");
             Metafield."Owner Id" := ShopifyCustomer.Id;
-            Metafield."Value Type" := Metafield."Value Type"::String;
+            Metafield.Type := Metafield.Type::string;
             Metafield.Value := Format(MetadataFieldRef.Value);
         end;
     end;

Apps/W1/Shopify/app/src/Customers/Tables/ShpfyCustomer.Table.al

@@ -148,6 +148,7 @@ table 30105 "Shpfy Customer"
         if not Tag.IsEmpty() then
             Tag.DeleteAll();
 
+        Metafield.SetRange("Parent Table No.", Database::"Shpfy Customer");
         Metafield.SetRange("Owner Id", Id);
         if not Metafield.IsEmpty then
             Metafield.DeleteAll();

Apps/W1/Shopify/app/src/GraphQL/Codeunits/ShpfyGQLMetafieldsSet.Codeunit.al

@@ -0,0 +1,27 @@
+namespace Microsoft.Integration.Shopify;
+
+/// <summary>
+/// Codeunit Shpfy GQL MetafieldSet (ID 30168) implements Interface Shpfy IGraphQL.
+/// </summary>
+codeunit 30350 "Shpfy GQL MetafieldsSet" implements "Shpfy IGraphQL"
+{
+    Access = Internal;
+
+    /// <summary>
+    /// GetGraphQL.
+    /// </summary>
+    /// <returns>Return value of type Text.</returns>
+    internal procedure GetGraphQL(): Text
+    begin
+        exit('{"query": "mutation { metafieldsSet(metafields: [{{Metafields}}]) { metafields {legacyResourceId namespace key} userErrors {field, message}}}"}');
+    end;
+
+    /// <summary>
+    /// GetExpectedCost.
+    /// </summary>
+    /// <returns>Return value of type Integer.</returns>
+    internal procedure GetExpectedCost(): Integer
+    begin
+        exit(10);
+    end;
+}

Apps/W1/Shopify/app/src/GraphQL/Codeunits/ShpfyGQLProductById.Codeunit.al

@@ -13,7 +13,7 @@ codeunit 30146 "Shpfy GQL ProductById" implements "Shpfy IGraphQL"
     /// <returns>Return value of type Text.</returns>
     internal procedure GetGraphQL(): Text
     begin
-        exit('{"query":"{product(id: \"gid://shopify/Product/{{ProductId}}\") {createdAt updatedAt hasOnlyDefaultVariant description(truncateAt: {{MaxLengthDescription}}) descriptionHtml onlineStorePreviewUrl onlineStoreUrl productType status tags title vendor seo{description, title} images(first: 1) {edges{node{id}}} metafields(namespace: \"Microsoft.Dynamics365.BusinessCentral\", first: 10) {edges {node {id namespace type legacyResourceId key value}}}}}"}');
+        exit('{"query":"{product(id: \"gid://shopify/Product/{{ProductId}}\") {createdAt updatedAt hasOnlyDefaultVariant description(truncateAt: {{MaxLengthDescription}}) descriptionHtml onlineStorePreviewUrl onlineStoreUrl productType status tags title vendor seo{description, title} images(first: 1) {edges{node{id}}} metafields(first: 50) {edges {node {id namespace type legacyResourceId key value}}}}}"}');
     end;
 
     /// <summary>

Apps/W1/Shopify/app/src/GraphQL/Codeunits/ShpfyGQLProductMetafieldIds.Codeunit.al

@@ -0,0 +1,28 @@
+namespace Microsoft.Integration.Shopify;
+
+/// <summary>
+/// Codeunit Shpfy GQL ProductMetafieldIds (ID 30332) implements Interface Shpfy IGraphQL.
+/// </summary>
+codeunit 30332 "Shpfy GQL ProductMetafieldIds" implements "Shpfy IGraphQL"
+{
+    Access = Internal;
+
+    /// <summary>
+    /// GetGraphQL.
+    /// </summary>
+    /// <returns>Return value of type Text.</returns>
+    internal procedure GetGraphQL(): Text
+    begin
+        exit('{"query":"{product(id: \"gid://shopify/Product/{{ProductId}}\") { metafields(first: 50) {edges{node{legacyResourceId updatedAt}}}}}"}');
+    end;
+
+    /// <summary>
+    /// GetExpectedCost.
+    /// </summary>
+    /// <returns>Return value of type Integer.</returns>
+    internal procedure GetExpectedCost(): Integer
+    begin
+        exit(50);
+    end;
+
+}

Apps/W1/Shopify/app/src/GraphQL/Codeunits/ShpfyGQLVariantById.Codeunit.al

@@ -13,7 +13,7 @@ codeunit 30150 "Shpfy GQL VariantById" implements "Shpfy IGraphQL"
     /// <returns>Return value of type Text.</returns>
     internal procedure GetGraphQL(): Text
     begin
-        exit('{"query":"{productVariant(id: \"gid://shopify/ProductVariant/{{VariantId}}\") {createdAt updatedAt availableForSale barcode compareAtPrice displayName inventoryPolicy position price sku taxCode taxable title weight product{id}selectedOptions{name value} inventoryItem{countryCodeOfOrigin createdAt id inventoryHistoryUrl legacyResourceId provinceCodeOfOrigin requiresShipping sku tracked updatedAt unitCost { amount currencyCode }} metafields(namespace: \"Microsoft.Dynamics365.BusinessCentral\", first: 10) {edges {node {id namespace ownerType legacyResourceId key value}}}}}"}');
+        exit('{"query":"{productVariant(id: \"gid://shopify/ProductVariant/{{VariantId}}\") {createdAt updatedAt availableForSale barcode compareAtPrice displayName inventoryPolicy position price sku taxCode taxable title weight product{id}selectedOptions{name value} inventoryItem{countryCodeOfOrigin createdAt id inventoryHistoryUrl legacyResourceId provinceCodeOfOrigin requiresShipping sku tracked updatedAt unitCost { amount currencyCode }} metafields(first: 50) {edges {node {id namespace ownerType legacyResourceId key value}}}}}"}');
     end;
 
     /// <summary>

Apps/W1/Shopify/app/src/GraphQL/Codeunits/ShpfyGQLVariantMetafieldIds.Codeunit.al

@@ -0,0 +1,28 @@
+namespace Microsoft.Integration.Shopify;
+
+/// <summary>
+/// Codeunit Shpfy GQL VariantMetafieldIds (ID 30336) implements Interface Shpfy IGraphQL.
+/// </summary>
+codeunit 30336 "Shpfy GQL VariantMetafieldIds" implements "Shpfy IGraphQL"
+{
+    Access = Internal;
+
+    /// <summary>
+    /// GetGraphQL.
+    /// </summary>
+    /// <returns>Return value of type Text.</returns>
+    internal procedure GetGraphQL(): Text
+    begin
+        exit('{"query":"{productVariant(id: \"gid://shopify/ProductVariant/{{VariantId}}\") { metafields(first: 50) {edges{ node{legacyResourceId updatedAt}}}}}"}');
+    end;
+
+    /// <summary>
+    /// GetExpectedCost.
+    /// </summary>
+    /// <returns>Return value of type Integer.</returns>
+    internal procedure GetExpectedCost(): Integer
+    begin
+        exit(50);
+    end;
+
+}

Apps/W1/Shopify/app/src/GraphQL/Enums/ShpfyGraphQLType.Enum.al

@@ -405,6 +405,21 @@ enum 30111 "Shpfy GraphQL Type" implements "Shpfy IGraphQL"
         Caption = 'Get Order Transactions';
         Implementation = "Shpfy IGraphQL" = "Shpfy GQL OrderTransactions";
     }
+    value(79; MetafieldSet)
+    {
+        Caption = 'MetfieldSet';
+        Implementation = "Shpfy IGraphQL" = "Shpfy GQL MetafieldsSet";
+    }
+    value(80; ProductMetafieldIds)
+    {
+        Caption = 'Product Metafield Ids';
+        Implementation = "Shpfy IGraphQL" = "Shpfy GQL ProductMetafieldIds";
+    }
+    value(81; VariantMetafieldIds)
+    {
+        Caption = 'Variant Metafield Ids';
+        Implementation = "Shpfy IGraphQL" = "Shpfy GQL VariantMetafieldIds";
+    }
     value(85; ProductVariantDelete)
     {
         Caption = 'Product Variant Delete';

Apps/W1/Shopify/app/src/Metafields/Codeunits/IMetafieldType/ShpfyMtfldTypeBoolean.Codeunit.al

@@ -0,0 +1,27 @@
+namespace Microsoft.Integration.Shopify;
+
+codeunit 30338 "Shpfy Mtfld Type Boolean" implements "Shpfy IMetafield Type"
+{
+    procedure HasAssistEdit(): Boolean
+    begin
+        exit(false);
+    end;
+
+    procedure IsValidValue(Value: Text): Boolean
+    var
+        DummyBoolean: Boolean;
+    begin
+        exit(Evaluate(DummyBoolean, Value, 9));
+    end;
+
+    procedure AssistEdit(var Value: Text[2048]): Boolean
+    begin
+        Value := Value;
+        exit(false);
+    end;
+
+    procedure GetExampleValue(): Text
+    begin
+        exit('true');
+    end;
+}

Apps/W1/Shopify/app/src/Metafields/Codeunits/IMetafieldType/ShpfyMtfldTypeCollectRef.Codeunit.al

@@ -0,0 +1,29 @@
+namespace Microsoft.Integration.Shopify;
+
+using System.Utilities;
+
+codeunit 30321 "Shpfy Mtfld Type Collect. Ref" implements "Shpfy IMetafield Type"
+{
+    procedure HasAssistEdit(): Boolean
+    begin
+        exit(false);
+    end;
+
+    procedure IsValidValue(Value: Text): Boolean
+    var
+        Regex: Codeunit Regex;
+    begin
+        exit(Regex.IsMatch(Value, '^gid:\/\/shopify\/Collection\/\d+$'));
+    end;
+
+    procedure AssistEdit(var Value: Text[2048]): Boolean
+    begin
+        Value := Value;
+        exit(false);
+    end;
+
+    procedure GetExampleValue(): Text
+    begin
+        exit('gid://shopify/Collection/1234567890');
+    end;
+}

Apps/W1/Shopify/app/src/Metafields/Codeunits/IMetafieldType/ShpfyMtfldTypeColor.Codeunit.al

@@ -0,0 +1,29 @@
+namespace Microsoft.Integration.Shopify;
+
+using System.Utilities;
+
+codeunit 30319 "Shpfy Mtfld Type Color" implements "Shpfy IMetafield Type"
+{
+    procedure HasAssistEdit(): Boolean
+    begin
+        exit(false);
+    end;
+
+    procedure IsValidValue(Value: Text): Boolean
+    var
+        Regex: Codeunit Regex;
+    begin
+        exit(Regex.IsMatch(Value, '^#[0-9A-Fa-f]{6}$'));
+    end;
+
+    procedure AssistEdit(var Value: Text[2048]): Boolean
+    begin
+        Value := Value;
+        exit(false);
+    end;
+
+    procedure GetExampleValue(): Text
+    begin
+        exit('#fff123');
+    end;
+}

Apps/W1/Shopify/app/src/Metafields/Codeunits/IMetafieldType/ShpfyMtfldTypeDate.Codeunit.al

@@ -0,0 +1,27 @@
+namespace Microsoft.Integration.Shopify;
+
+codeunit 30318 "Shpfy Mtfld Type Date" implements "Shpfy IMetafield Type"
+{
+    procedure HasAssistEdit(): Boolean
+    begin
+        exit(false);
+    end;
+
+    procedure IsValidValue(Value: Text): Boolean
+    var
+        DummyDate: Date;
+    begin
+        exit(Evaluate(DummyDate, Value, 9));
+    end;
+
+    procedure AssistEdit(var Value: Text[2048]): Boolean
+    begin
+        Value := Value;
+        exit(false);
+    end;
+
+    procedure GetExampleValue(): Text
+    begin
+        exit('2022-02-02');
+    end;
+}