Skip to content

Commit

Permalink
[Shopify] Log Skipped Records on export processes (#27539)
Browse files Browse the repository at this point in the history
This pull request does not have a related issue as it's part of delivery
for development agreed directly with @AndreiPanko
Fixes #26819 

### Created new structure to log records skipped on export

Solution has new structure possible to open by typing "Shopify Skipped
Records". On page user can see all the skipped records with reasons and
date and time when the record was skipped. User can also open the
related record using "Show record" action or clicking the "Description"
field value.

### Log cases:

Customer:
- Customer has empty email
- Customer with same email or phone exists

Posted Sales Invoice:
- Customer does not exist in Shopify
- Payment terms do not exist in Shopify
- Customer No. is Default Customer No. for Shopify Shop
- Customer No. is used in Shopify Customer Template
- No lines existing in sales invoice
- Invalid Quantity
- Empty No. value

Product:
-  Item is blocked/sales blocked (Item Variant)
-  Item is blocked

Shipments:
 - Related Shopify Order does not exist
 - No lines in Posted Sales Shipment applicable for fulfillment 
 - No corresponding fulfillment found in Shopify.

If the condition met log record is being created.

### Retention policy
Retention policy is inserted on extension install. It can be turned on
"Retention Policies" page.

Fixes
[AB#473306](https://dynamicssmb2.visualstudio.com/1fcb79e7-ab07-432a-a3c6-6cf5a88ba4a5/_workitems/edit/473306)

---------

Co-authored-by: Piotr Michalak <[email protected]>
Co-authored-by: Gediminas Gaubys <[email protected]>
  • Loading branch information
3 people authored Oct 30, 2024
1 parent 7ca090c commit c8b01d7
Show file tree
Hide file tree
Showing 17 changed files with 1,436 additions and 104 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ codeunit 30273 "Shpfy Installer"
var
LogEntry: Record "Shpfy Log Entry";
DataCapture: Record "Shpfy Data Capture";
SkippedRecord: Record "Shpfy Skipped Record";
RetentionPolicySetup: Codeunit "Retention Policy Setup";
RetenPolAllowedTables: Codeunit "Reten. Pol. Allowed Tables";
UpgradeTag: Codeunit "Upgrade Tag";
Expand All @@ -38,12 +39,14 @@ codeunit 30273 "Shpfy Installer"

RetenPolAllowedTables.AddAllowedTable(Database::"Shpfy Log Entry", LogEntry.FieldNo(SystemCreatedAt));
RetenPolAllowedTables.AddAllowedTable(Database::"Shpfy Data Capture", DataCapture.FieldNo(SystemModifiedAt));
RetenPolAllowedTables.AddAllowedTable(Database::"Shpfy Skipped Record", SkippedRecord.FieldNo(SystemCreatedAt));

if not IsInitialSetup then
exit;

CreateRetentionPolicySetup(Database::"Shpfy Log Entry", RetentionPolicySetup.FindOrCreateRetentionPeriod("Retention Period Enum"::"1 Month"));
CreateRetentionPolicySetup(Database::"Shpfy Data Capture", RetentionPolicySetup.FindOrCreateRetentionPeriod("Retention Period Enum"::"1 Month"));
CreateRetentionPolicySetup(Database::"Shpfy Skipped Record", RetentionPolicySetup.FindOrCreateRetentionPeriod("Retention Period Enum"::"1 Month"));
UpgradeTag.SetUpgradeTag(GetShopifyLogEntryAddedToAllowedListUpgradeTag());
end;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ codeunit 30284 "Shpfy Company Export"
Shop: Record "Shpfy Shop";
CompanyAPI: Codeunit "Shpfy Company API";
CatalogAPI: Codeunit "Shpfy Catalog API";
SkippedRecord: Codeunit "Shpfy Skipped Record";

Check failure on line 39 in Apps/W1/Shopify/app/src/Companies/Codeunits/ShpfyCompanyExport.Codeunit.al

View workflow job for this annotation

GitHub Actions / Build 1st Party Apps (W1) (Translated) / 1st Party Apps (W1) (Translated)

AL0275 'Shpfy Skipped Record' is an ambiguous reference between 'Shpfy Skipped Record' defined by the extension 'Shopify Connector by Microsoft (26.0.1143.0)' and 'Shpfy Skipped Record' defined by the extension 'Shopify Connector by Microsoft (26.0.1143.0)'.
CreateCustomers: Boolean;
CountyCodeTooLongLbl: Label 'Can not export customer %1 %2. The length of the string is %3, but it must be less than or equal to %4 characters. Value: %5, field: %6', Comment = '%1 - Customer No., %2 - Customer Name, %3 - Length, %4 - Max Length, %5 - Value, %6 - Field Name';

Expand All @@ -44,9 +45,12 @@ codeunit 30284 "Shpfy Company Export"
ShopifyCompany: Record "Shpfy Company";
ShopifyCustomer: Record "Shpfy Customer";
CompanyLocation: Record "Shpfy Company Location";
EmptyEmailAddressLbl: Label 'Customer (Company) has no e-mail address.';
begin
if Customer."E-Mail" = '' then
if Customer."E-Mail" = '' then begin
SkippedRecord.LogSkippedRecord(Customer.RecordId, EmptyEmailAddressLbl, Shop);
exit;
end;

if CreateCompanyMainContact(Customer, ShopifyCustomer) then
if FillInShopifyCompany(Customer, ShopifyCompany, CompanyLocation) then
Expand Down Expand Up @@ -180,10 +184,13 @@ codeunit 30284 "Shpfy Company Export"
var
ShopifyCompany: Record "Shpfy Company";
CompanyLocation: Record "Shpfy Company Location";
CompanyWithPhoneNoOrEmailExistsLbl: Label 'Company already exists with the same e-mail or phone.';
begin
ShopifyCompany.Get(CompanyId);
if ShopifyCompany."Customer SystemId" <> Customer.SystemId then
if ShopifyCompany."Customer SystemId" <> Customer.SystemId then begin
SkippedRecord.LogSkippedRecord(ShopifyCompany.Id, Customer.RecordId, CompanyWithPhoneNoOrEmailExistsLbl, Shop);
exit;
end;

CompanyLocation.SetRange("Company SystemId", ShopifyCompany.SystemId);
CompanyLocation.FindFirst();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ codeunit 30116 "Shpfy Customer Export"
var
Shop: Record "Shpfy Shop";
CustomerApi: Codeunit "Shpfy Customer API";
SkippedRecord: Codeunit "Shpfy Skipped Record";

Check failure on line 42 in Apps/W1/Shopify/app/src/Customers/Codeunits/ShpfyCustomerExport.Codeunit.al

View workflow job for this annotation

GitHub Actions / Build 1st Party Apps (W1) (Translated) / 1st Party Apps (W1) (Translated)

AL0275 'Shpfy Skipped Record' is an ambiguous reference between 'Shpfy Skipped Record' defined by the extension 'Shopify Connector by Microsoft (26.0.1143.0)' and 'Shpfy Skipped Record' defined by the extension 'Shopify Connector by Microsoft (26.0.1143.0)'.
CreateCustomers: Boolean;
CountyCodeTooLongLbl: Label 'Can not export customer %1 %2. The length of the string is %3, but it must be less than or equal to %4 characters. Value: %5, field: %6', Comment = '%1 - Customer No., %2 - Customer Name, %3 - Length, %4 - Max Length, %5 - Value, %6 - Field Name';

Expand Down Expand Up @@ -87,9 +88,12 @@ codeunit 30116 "Shpfy Customer Export"
var
ShopifyCustomer: Record "Shpfy Customer";
CustomerAddress: Record "Shpfy Customer Address";
EmptyEmailAddressLbl: Label 'Customer has no e-mail address.';
begin
if Customer."E-Mail" = '' then
if Customer."E-Mail" = '' then begin
SkippedRecord.LogSkippedRecord(Customer.RecordId, EmptyEmailAddressLbl, Shop);
exit;
end;

Clear(ShopifyCustomer);
Clear(CustomerAddress);
Expand Down Expand Up @@ -296,10 +300,13 @@ codeunit 30116 "Shpfy Customer Export"
var
ShopifyCustomer: Record "Shpfy Customer";
CustomerAddress: Record "Shpfy Customer Address";
CustomerWithPhoneNoOrEmailExistsLbl: Label 'Customer already exists with the same e-mail or phone.';
begin
ShopifyCustomer.Get(CustomerID);
if ShopifyCustomer."Customer SystemId" <> Customer.SystemId then
if ShopifyCustomer."Customer SystemId" <> Customer.SystemId then begin
SkippedRecord.LogSkippedRecord(ShopifyCustomer.Id, Customer.RecordId, CustomerWithPhoneNoOrEmailExistsLbl, Shop);
exit; // An other customer with the same e-mail or phone is the source of it.
end;

CustomerAddress.SetRange("Customer Id", CustomerId);
CustomerAddress.SetRange(Default, true);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ codeunit 30362 "Shpfy Posted Invoice Export"
DraftOrdersAPI: Codeunit "Shpfy Draft Orders API";
FulfillmentAPI: Codeunit "Shpfy Fulfillment API";
JsonHelper: Codeunit "Shpfy Json Helper";
SkippedRecord: Codeunit "Shpfy Skipped Record";

Check failure on line 20 in Apps/W1/Shopify/app/src/Invoicing/Codeunits/ShpfyPostedInvoiceExport.Codeunit.al

View workflow job for this annotation

GitHub Actions / Build 1st Party Apps (W1) (Translated) / 1st Party Apps (W1) (Translated)

AL0275 'Shpfy Skipped Record' is an ambiguous reference between 'Shpfy Skipped Record' defined by the extension 'Shopify Connector by Microsoft (26.0.1143.0)' and 'Shpfy Skipped Record' defined by the extension 'Shopify Connector by Microsoft (26.0.1143.0)'.

trigger OnRun()
begin
Expand Down Expand Up @@ -92,22 +93,34 @@ codeunit 30362 "Shpfy Posted Invoice Export"
var
ShopifyCompany: Record "Shpfy Company";
ShopifyCustomer: Record "Shpfy Customer";
CustomerNotExistInShopifyLbl: Label 'Customer does not exists as Shopify company or customer.';
PaymentTermsNotExistLbl: Label 'Payment terms %1 do not exist in Shopify.', Comment = '%1 = Payment Terms Code.';
CustomerNoIsDefaultCustomerNoLbl: Label 'Bill-to customer no. is the default customer no. for Shopify shop.';
CustomerTemplateExistsLbl: Label 'Shopify customer template exists for customer no. %1 shop %2.', Comment = '%1 = Customer No., %2 = Shop Code';
begin
ShopifyCompany.SetRange("Customer No.", SalesInvoiceHeader."Bill-to Customer No.");
if ShopifyCompany.IsEmpty() then begin
ShopifyCustomer.SetRange("Customer No.", SalesInvoiceHeader."Bill-to Customer No.");
if ShopifyCustomer.IsEmpty() then
if ShopifyCustomer.IsEmpty() then begin
SkippedRecord.LogSkippedRecord(SalesInvoiceHeader.RecordId, CustomerNotExistInShopifyLbl, Shop);
exit(false);
end;
end;

if not ShopifyPaymentTermsExists(SalesInvoiceHeader."Payment Terms Code") then
if not ShopifyPaymentTermsExists(SalesInvoiceHeader."Payment Terms Code") then begin
SkippedRecord.LogSkippedRecord(SalesInvoiceHeader.RecordId, StrSubstNo(PaymentTermsNotExistLbl, SalesInvoiceHeader."Payment Terms Code"), Shop);
exit(false);
end;

if Shop."Default Customer No." = SalesInvoiceHeader."Bill-to Customer No." then
if Shop."Default Customer No." = SalesInvoiceHeader."Bill-to Customer No." then begin
SkippedRecord.LogSkippedRecord(SalesInvoiceHeader.RecordId, CustomerNoIsDefaultCustomerNoLbl, Shop);
exit(false);
end;

if CheckCustomerTemplates(SalesInvoiceHeader."Bill-to Customer No.") then
if CheckCustomerTemplates(SalesInvoiceHeader."Bill-to Customer No.") then begin
SkippedRecord.LogSkippedRecord(SalesInvoiceHeader.RecordId, StrSubstNo(CustomerTemplateExistsLbl, SalesInvoiceHeader."Bill-to Customer No.", Shop.Code), Shop);
exit(false);
end;

if not CheckSalesInvoiceHeaderLines(SalesInvoiceHeader) then
exit(false);
Expand Down Expand Up @@ -145,21 +158,31 @@ codeunit 30362 "Shpfy Posted Invoice Export"
local procedure CheckSalesInvoiceHeaderLines(SalesInvoiceHeader: Record "Sales Invoice Header"): Boolean
var
SalesInvoiceLine: Record "Sales Invoice Line";
NoLinesInSalesInvoiceLbl: Label 'No relevant sales invoice lines exist.';
InvalidQuantityLbl: Label 'Invalid quantity in sales invoice line.';
EmptyNoInLineLbl: Label 'No. field is empty in Sales Invoice Line.';
begin
SalesInvoiceLine.SetRange("Document No.", SalesInvoiceHeader."No.");
SalesInvoiceLine.SetFilter(Type, '<>%1', SalesInvoiceLine.Type::" ");
if SalesInvoiceLine.IsEmpty() then
if SalesInvoiceLine.IsEmpty() then begin
SkippedRecord.LogSkippedRecord(SalesInvoiceHeader.RecordId, NoLinesInSalesInvoiceLbl, Shop);
exit(false);
end;

SalesInvoiceLine.Reset();

SalesInvoiceLine.SetRange("Document No.", SalesInvoiceHeader."No.");
if SalesInvoiceLine.FindSet() then
repeat
if (SalesInvoiceLine.Quantity <> 0) and (SalesInvoiceLine.Quantity <> Round(SalesInvoiceLine.Quantity, 1)) then
if (SalesInvoiceLine.Quantity <> 0) and (SalesInvoiceLine.Quantity <> Round(SalesInvoiceLine.Quantity, 1)) then begin
SkippedRecord.LogSkippedRecord(SalesInvoiceLine.RecordId, InvalidQuantityLbl, Shop);
exit(false);
end;

if (SalesInvoiceLine.Type <> SalesInvoiceLine.Type::" ") and (SalesInvoiceLine."No." = '') then
if (SalesInvoiceLine.Type <> SalesInvoiceLine.Type::" ") and (SalesInvoiceLine."No." = '') then begin
SkippedRecord.LogSkippedRecord(SalesInvoiceLine.RecordId, EmptyNoInLineLbl, Shop);
exit(false);
end;
until SalesInvoiceLine.Next() = 0;

exit(true);
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
namespace Microsoft.Integration.Shopify;

/// <summary>
/// Codeunit Shpfy Skip Record (ID 30313).
/// </summary>
codeunit 30313 "Shpfy Skipped Record"

Check failure on line 6 in Apps/W1/Shopify/app/src/Logs/Codeunits/ShpfySkipRecordMgt.Codeunit.al

View workflow job for this annotation

GitHub Actions / Build 1st Party Apps (W1) (Translated) / 1st Party Apps (W1) (Translated)

AL0197 An application object of type 'Codeunit' with name 'Shpfy Skipped Record' is already declared by the extension 'Shopify Connector by Microsoft (26.0.1143.0)'

Check failure on line 6 in Apps/W1/Shopify/app/src/Logs/Codeunits/ShpfySkipRecordMgt.Codeunit.al

View workflow job for this annotation

GitHub Actions / Build 1st Party Apps (W1) (Translated) / 1st Party Apps (W1) (Translated)

AL0264 An application object of type 'Codeunit' with ID '30313' is already declared by the extension 'Shopify Connector by Microsoft (26.0.1143.0)'
{
Access = Internal;
Permissions = tabledata "Shpfy Skipped Record" = rimd;

/// <summary>
/// Creates log entry for skipped record.
/// </summary>
/// <param name="ShopifyId">Related Shopify Id of the record.</param>
/// <param name="TableId">Table Id of the record.</param>
/// <param name="RecordId">Record Id of the record.</param>
/// <param name="SkippedReason">Reason for skipping the record.</param>
/// <param name="Shop">Shop record.</param>
internal procedure LogSkippedRecord(ShopifyId: BigInteger; RecordId: RecordID; SkippedReason: Text[250]; Shop: Record "Shpfy Shop")
var
SkippedRecord: Record "Shpfy Skipped Record";
begin
if Shop."Logging Mode" = Enum::"Shpfy Logging Mode"::Disabled then
exit;
SkippedRecord.Init();
SkippedRecord.Validate("Shopify Id", ShopifyId);
SkippedRecord.Validate("Table ID", RecordId.TableNo());
SkippedRecord.Validate("Record ID", RecordId);
SkippedRecord.Validate("Skipped Reason", SkippedReason);
SkippedRecord.Insert(true);
end;

/// <summary>
/// Creates log entry for skipped recordwith empty Shopify Id.
/// </summary>
/// <param name="RecordId">Record Id of the record.</param>
/// <param name="SkippedReason">Reason for skipping the record.</param>
/// <param name="Shop">Shop record.</param>
internal procedure LogSkippedRecord(RecordId: RecordID; SkippedReason: Text[250]; Shop: Record "Shpfy Shop")
begin
LogSkippedRecord(0, RecordId, SkippedReason, Shop);
end;

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
namespace Microsoft.Integration.Shopify;

/// <summary>
/// Codeunit Shpfy Skip Record (ID 30313).
/// </summary>
codeunit 30313 "Shpfy Skipped Record"

Check failure on line 6 in Apps/W1/Shopify/app/src/Logs/Codeunits/ShpfySkippedRecord.Codeunit.al

View workflow job for this annotation

GitHub Actions / Build 1st Party Apps (W1) (Translated) / 1st Party Apps (W1) (Translated)

AL0197 An application object of type 'Codeunit' with name 'Shpfy Skipped Record' is already declared by the extension 'Shopify Connector by Microsoft (26.0.1143.0)'

Check failure on line 6 in Apps/W1/Shopify/app/src/Logs/Codeunits/ShpfySkippedRecord.Codeunit.al

View workflow job for this annotation

GitHub Actions / Build 1st Party Apps (W1) (Translated) / 1st Party Apps (W1) (Translated)

AL0264 An application object of type 'Codeunit' with ID '30313' is already declared by the extension 'Shopify Connector by Microsoft (26.0.1143.0)'
{
Access = Internal;
Permissions = tabledata "Shpfy Skipped Record" = rimd;

/// <summary>
/// Creates log entry for skipped record.
/// </summary>
/// <param name="ShopifyId">Related Shopify Id of the record.</param>
/// <param name="TableId">Table Id of the record.</param>
/// <param name="RecordId">Record Id of the record.</param>
/// <param name="SkippedReason">Reason for skipping the record.</param>
/// <param name="Shop">Shop record.</param>
internal procedure LogSkippedRecord(ShopifyId: BigInteger; RecordId: RecordID; SkippedReason: Text[250]; Shop: Record "Shpfy Shop")
var
SkippedRecord: Record "Shpfy Skipped Record";
begin
if Shop."Logging Mode" = Enum::"Shpfy Logging Mode"::Disabled then
exit;
SkippedRecord.Init();
SkippedRecord.Validate("Shopify Id", ShopifyId);
SkippedRecord.Validate("Table ID", RecordId.TableNo());
SkippedRecord.Validate("Record ID", RecordId);
SkippedRecord.Validate("Skipped Reason", SkippedReason);
SkippedRecord.Insert(true);
end;

/// <summary>
/// Creates log entry for skipped recordwith empty Shopify Id.
/// </summary>
/// <param name="RecordId">Record Id of the record.</param>
/// <param name="SkippedReason">Reason for skipping the record.</param>
/// <param name="Shop">Shop record.</param>
internal procedure LogSkippedRecord(RecordId: RecordID; SkippedReason: Text[250]; Shop: Record "Shpfy Shop")
begin
LogSkippedRecord(0, RecordId, SkippedReason, Shop);
end;

}
95 changes: 95 additions & 0 deletions Apps/W1/Shopify/app/src/Logs/Pages/ShpfySkippedRecords.Page.al
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
namespace Microsoft.Integration.Shopify;

/// <summary>
/// Page Shpfy Skipped Records (ID 30166).
/// </summary>
page 30166 "Shpfy Skipped Records"
{
ApplicationArea = All;
Caption = 'Shopify Skipped Records';
PageType = List;
SourceTable = "Shpfy Skipped Record";
UsageCategory = Lists;
Editable = false;
InsertAllowed = false;
SourceTableView = sorting("Entry No.") order(descending);

layout
{
area(Content)
{
repeater(General)
{
field("Shopify Id"; Rec."Shopify Id") { }
field("Table ID"; Rec."Table ID") { }
field("Table Name"; Rec."Table Name") { }
field(Description; Rec.Description)
{
trigger OnDrillDown()
begin
Rec.ShowPage();
end;
}
field("Skipped Reason"; Rec."Skipped Reason") { }
}
}
}

actions
{
area(Promoted)
{
group(Category_Process)
{
actionref(Show_Promoted; Show) { }
}

group(Category_Category4)
{
Caption = 'Log Entries';

actionref(Delete7days_Promoted; Delete7days) { }
actionref(Delete0days_Promoted; Delete0days) { }
}
}
area(Processing)
{
action(Show)
{
ApplicationArea = All;
Caption = 'Show record';
Image = View;
ToolTip = 'Show the details of the selected record.';

trigger OnAction()
begin
Rec.ShowPage();
end;
}
action(Delete7days)
{
ApplicationArea = All;
Caption = 'Delete Entries Older Than 7 Days';
Image = ClearLog;
ToolTip = 'Clear the list of skipped records that are older than 7 days.';

trigger OnAction();
begin
Rec.DeleteEntries(7);
end;
}
action(Delete0days)
{
ApplicationArea = All;
Caption = 'Delete All Entries';
Image = Delete;
ToolTip = 'Clear the list of all skipped records.';

trigger OnAction();
begin
Rec.DeleteEntries(0);
end;
}
}
}
}
Loading

0 comments on commit c8b01d7

Please sign in to comment.