Skip to content

Commit 7c8e804

Browse files
authored
Wrong dependency resolution plus performance optimizations when building (#3757)
Two new parameter in Sort-AppFoldersByDependencies - onlyTheseAppFoldersPlusDepending and SkipApps. If specified, then only the appFolders specified in the first array plus the apps depending on these recursively will be sorted. The remaining folder not included in the sorted appfolders list will be returned in skipApps. This is used for incrementals builds in AL-Go for GitHub. Remove some excessive logging If Import-TestToolkitToBcContainer was running with useCompilerFolder, then it would require a container to be present at that time. Make Run-AlPipeline do just-in-time creation of CompilerFolder and Containers - meaning that if you are using useCompilerFolder and do not have doNotPublishApps set - it will create a compilerFolder and compile all apps using that, and afterwards - it will create the container, publish the apps and run the tests. Also... Fixes microsoft/AL-Go#1338 --------- Co-authored-by: freddydk <[email protected]>
1 parent f4791a3 commit 7c8e804

8 files changed

+619
-416
lines changed

.github/workflows/CI.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -119,7 +119,7 @@ jobs:
119119
}
120120
121121
Linux:
122-
runs-on: [ ubuntu-latest ]
122+
runs-on: [ ubuntu-24.04 ]
123123
defaults:
124124
run:
125125
shell: pwsh

.github/workflows/RunTests.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -131,7 +131,7 @@ jobs:
131131
}
132132
133133
Linux:
134-
runs-on: [ ubuntu-latest ]
134+
runs-on: [ ubuntu-24.04 ]
135135
needs: [ AnalyzeTests ]
136136
if: needs.AnalyzeTests.outputs.linuxtests != '[]'
137137
strategy:

AppHandling/Run-AlPipeline.ps1

+546-394
Large diffs are not rendered by default.

AppHandling/Sort-AppFilesByDependencies.ps1

+2-2
Original file line numberDiff line numberDiff line change
@@ -101,8 +101,8 @@ function Sort-AppFilesByDependencies {
101101
}
102102
else {
103103
if (-not ($script:unresolvedDependencies | Where-Object { $_ } | Where-Object { "$(if ($_.PSObject.Properties.name -eq 'AppId') { $_.AppId } else { $_.Id })" -eq $dependencyAppId })) {
104-
$appFileName = "$($dependency.publisher)_$($dependency.name)_$($dependency.version)).app".Split([System.IO.Path]::GetInvalidFileNameChars()) -join ''
105-
if ($dependencyAppid -ne '63ca2fa4-4f03-4f2b-a480-172fef340d3f' -and $dependencyAppId -ne '437dbf0e-84ff-417a-965d-ed2bb9650972') {
104+
$appFileName = "$($dependency.publisher)_$($dependency.name)_$($dependency.version).app".Split([System.IO.Path]::GetInvalidFileNameChars()) -join ''
105+
if ($dependencyAppid -ne '63ca2fa4-4f03-4f2b-a480-172fef340d3f' -and $dependencyAppId -ne '437dbf0e-84ff-417a-965d-ed2bb9650972' -and $dependencyAppId -ne 'f3552374-a1f2-4356-848e-196002525837') {
106106
Write-Warning "Dependency $($dependencyAppId):$appFileName not found"
107107
}
108108
$script:unresolvedDependencies += @($dependency)

AppHandling/Sort-AppFoldersByDependencies.ps1

+45-9
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,10 @@
1111
If specified, this reference parameter will contain unresolved dependencies after sorting
1212
.Parameter knownApps
1313
If specified, this reference parameter will contain all known appids
14+
.Parameter skippedApps
15+
If specified, this reference parameter will contain all skipped appids
16+
.Parameter selectSubordinates
17+
If specified, this is the list of appFolders to include (together with subordinates - i.e. appFolders depending on these appFolders)
1418
.Example
1519
$folders = Sort-AppFoldersByDependencies -appFolders @($folder1, $folder2)
1620
#>
@@ -23,7 +27,10 @@ function Sort-AppFoldersByDependencies {
2327
[Parameter(Mandatory=$false)]
2428
[ref] $unknownDependencies,
2529
[Parameter(Mandatory=$false)]
26-
[ref] $knownApps
30+
[ref] $knownApps,
31+
[Parameter(Mandatory=$false)]
32+
[ref] $skippedApps,
33+
[string[]] $selectSubordinates = @()
2734
)
2835

2936
$telemetryScope = InitTelemetryScope -name $MyInvocation.InvocationName -parameterValues $PSBoundParameters -includeParameters @()
@@ -40,6 +47,7 @@ try {
4047
# Read all app.json objects, populate $apps
4148
$apps = $()
4249
$folders = @{}
50+
$script:includeAppIds = @()
4351
$appFolders | ForEach-Object {
4452
$appFolder = "$baseFolder$_"
4553
$appJsonFile = Join-Path $appFolder "app.json"
@@ -68,14 +76,20 @@ try {
6876
$appJson | Add-Member -Name "dependencies" -Type NoteProperty -Value @()
6977
}
7078
if ($appJson.psobject.Members | Where-Object name -eq "application") {
71-
if ($appJson.Id -ne "63ca2fa4-4f03-4f2b-a480-172fef340d3f") {
79+
if ($appJson.Id -ne "63ca2fa4-4f03-4f2b-a480-172fef340d3f" -and $appJson.Id -ne "f3552374-a1f2-4356-848e-196002525837" -and $appJson.Id -ne "437dbf0e-84ff-417a-965d-ed2bb9650972") {
7280
$appJson.dependencies += @( New-Object psobject -Property ([ordered]@{ "appId" = "437dbf0e-84ff-417a-965d-ed2bb9650972"; "publisher" = "Microsoft"; "name" = "Base Application"; "version" = $appJson.application }) )
7381
$appJson.dependencies += @( New-Object psobject -Property ([ordered]@{ "appId" = "63ca2fa4-4f03-4f2b-a480-172fef340d3f"; "publisher" = "Microsoft"; "name" = "System Application"; "version" = $appJson.application }) )
82+
if ([System.Version]$appJson.application -ge [System.Version]"24.0.0.0") {
83+
$appJson.dependencies += @( New-Object psobject -Property ([ordered]@{ "appId" = "f3552374-a1f2-4356-848e-196002525837"; "publisher" = "Microsoft"; "name" = "Business Foundation"; "version" = $appJson.application }) )
84+
}
7485
}
7586
}
7687

7788
$folders += @{ "$($appJson.Id):$($appJson.Version)" = $appFolder }
7889
$apps += @($appJson)
90+
if ($selectSubordinates -contains $_) {
91+
$script:includeAppIds += @($appJson.Id)
92+
}
7993
}
8094
}
8195

@@ -84,50 +98,72 @@ try {
8498
$script:unresolvedDependencies = $()
8599

86100
function AddAnApp { Param($anApp)
101+
$includeThis = $false
87102
$alreadyAdded = $script:sortedApps | Where-Object { $_.Id -eq $anApp.Id }
88103
if (-not ($alreadyAdded)) {
89-
AddDependencies -anApp $anApp
104+
if (AddDependencies -anApp $anApp) {
105+
if ($script:includeAppIds -notcontains $anApp.Id) {
106+
$script:includeAppIds += @($anApp.Id)
107+
}
108+
$includeThis = $true
109+
}
90110
$script:sortedApps += $anApp
91111
}
112+
return $includeThis
92113
}
93114

94115
function AddDependency { Param($dependency)
95116
$dependencyAppId = "$(if ($dependency.PSObject.Properties.name -eq 'AppId') { $dependency.AppId } else { $dependency.Id })"
117+
$includeThis = $script:includeAppIds -contains $dependencyAppId
96118
$dependentApp = $apps | Where-Object { $_.Id -eq $dependencyAppId } | Sort-Object -Property @{ "Expression" = "[System.Version]Version" }
97119
if ($dependentApp) {
98120
if ($dependentApp -is [Array]) {
99121
Write-Host -ForegroundColor Yellow "AppFiles contains multiple versions of the app with AppId $dependencyAppId"
100122
$dependentApp = $dependentApp | Select-Object -Last 1
101123
}
102-
AddAnApp -AnApp $dependentApp
124+
if (AddAnApp -AnApp $dependentApp) {
125+
$includeThis = $true
126+
}
103127
}
104128
else {
105129
if (-not ($script:unresolvedDependencies | Where-Object { $_ } | Where-Object { "$(if ($_.PSObject.Properties.name -eq 'AppId') { $_.AppId } else { $_.Id })" -eq $dependencyAppId })) {
106-
$appFileName = "$($dependency.publisher)_$($dependency.name)_$($dependency.version)).app".Split([System.IO.Path]::GetInvalidFileNameChars()) -join ''
107-
if ($dependencyAppid -ne '63ca2fa4-4f03-4f2b-a480-172fef340d3f' -and $dependencyAppId -ne '437dbf0e-84ff-417a-965d-ed2bb9650972') {
130+
$appFileName = "$($dependency.publisher)_$($dependency.name)_$($dependency.version).app".Split([System.IO.Path]::GetInvalidFileNameChars()) -join ''
131+
if ($dependencyAppid -ne '63ca2fa4-4f03-4f2b-a480-172fef340d3f' -and $dependencyAppId -ne '437dbf0e-84ff-417a-965d-ed2bb9650972' -and $dependencyAppId -ne 'f3552374-a1f2-4356-848e-196002525837') {
108132
Write-Warning "Dependency $($dependencyAppId):$appFileName not found"
109133
}
110134
$script:unresolvedDependencies += @($dependency)
111135
}
112136
}
137+
return $includeThis
113138
}
114139

115140
function AddDependencies { Param($anApp)
141+
$includeThis = $false
116142
if ($anApp) {
117143
if ($anApp.psobject.Members | Where-Object name -eq "dependencies") {
118144
if ($anApp.Dependencies) {
119-
$anApp.Dependencies | ForEach-Object { AddDependency -Dependency $_ }
145+
$anApp.Dependencies | ForEach-Object {
146+
if (AddDependency -Dependency $_) {
147+
$includeThis = $true
148+
}
149+
}
120150
}
121151
}
122152
}
153+
return $includeThis
123154
}
124155

125-
$apps | Where-Object { $_.Name -eq "Application" } | ForEach-Object { AddAnApp -anApp $_ }
126-
$apps | ForEach-Object { AddAnApp -AnApp $_ }
156+
$apps | Where-Object { $_.Name -eq "Application" } | ForEach-Object { AddAnApp -anApp $_ | Out-Null }
157+
$apps | ForEach-Object { AddAnApp -AnApp $_ | Out-Null }
127158

128159
$script:sortedApps | ForEach-Object {
129160
($folders["$($_.id):$($_.version)"]).SubString($baseFolder.Length)
130161
}
162+
if ($skippedApps -and $selectSubordinates) {
163+
$skippedApps.value = $script:sortedApps | Where-Object { $script:includeAppIds -notcontains $_.id } | ForEach-Object {
164+
($folders["$($_.id):$($_.version)"]).SubString($baseFolder.Length)
165+
}
166+
}
131167
if ($knownApps) {
132168
$knownApps.value += @($script:sortedApps | ForEach-Object {
133169
$_.Id

HelperFunctions.ps1

+9-7
Original file line numberDiff line numberDiff line change
@@ -1055,7 +1055,7 @@ function GetAppInfo {
10551055
}
10561056
Set-Location (Split-Path $cacheAppInfoPath -parent)
10571057
}
1058-
Write-GroupStart -Message "Getting .app info $cacheAppInfoPath"
1058+
Write-Host "Getting .app info $cacheAppInfoPath"
10591059
$binPath = Join-Path $compilerFolder 'compiler/extension/bin'
10601060
$alToolDll = ''
10611061
if ($isLinux) {
@@ -1107,11 +1107,9 @@ function GetAppInfo {
11071107
$package = $null
11081108
try {
11091109
foreach($path in $appFiles) {
1110-
Write-Host -NoNewline "- $([System.IO.Path]::GetFileName($path))"
11111110
$relativePath = Resolve-Path -Path $path -Relative
11121111
if ($appInfoCache -and $appInfoCache.PSObject.Properties.Name -eq $relativePath) {
11131112
$appInfo = $appInfoCache."$relativePath"
1114-
Write-Host " (cached)"
11151113
}
11161114
else {
11171115
if ($alToolExists) {
@@ -1130,7 +1128,6 @@ function GetAppInfo {
11301128
"propagateDependencies" = ($manifest.PSObject.Properties.Name -eq 'PropagateDependencies') -and $manifest.PropagateDependencies
11311129
"dependencies" = @(if($manifest.PSObject.Properties.Name -eq 'dependencies'){$manifest.dependencies | ForEach-Object { if ($_.PSObject.Properties.Name -eq 'id') { $id = $_.id } else { $id = $_.AppId }; @{ "id" = $id; "name" = $_.name; "publisher" = $_.publisher; "version" = $_.version }}})
11321130
}
1133-
Write-Host " (succeeded using altool)"
11341131
}
11351132
else {
11361133
if (!$assembliesAdded) {
@@ -1158,7 +1155,6 @@ function GetAppInfo {
11581155
"propagateDependencies" = $manifest.PropagateDependencies
11591156
}
11601157
$packageStream.Close()
1161-
Write-Host " (succeeded using codeanalysis)"
11621158
}
11631159
if ($cacheAppInfoPath) {
11641160
$appInfoCache | Add-Member -MemberType NoteProperty -Name $relativePath -Value $appInfo
@@ -1183,7 +1179,6 @@ function GetAppInfo {
11831179
}
11841180
}
11851181
catch [System.Reflection.ReflectionTypeLoadException] {
1186-
Write-Host " (failed)"
11871182
if ($_.Exception.LoaderExceptions) {
11881183
$_.Exception.LoaderExceptions | Select-Object -Property Message | Select-Object -Unique | ForEach-Object {
11891184
Write-Host "LoaderException: $($_.Message)"
@@ -1200,7 +1195,6 @@ function GetAppInfo {
12001195
}
12011196
Pop-Location
12021197
}
1203-
Write-GroupEnd
12041198
}
12051199

12061200
function GetLatestAlLanguageExtensionVersionAndUrl {
@@ -1635,4 +1629,12 @@ function GetHostOs {
16351629
}
16361630
}
16371631
return $hostOs
1632+
}
1633+
1634+
function Write-PSCallStack {
1635+
Param(
1636+
[string] $message
1637+
)
1638+
Write-Host "PS CallStack $message :"
1639+
Get-PSCallStack | ForEach-Object { Write-Host "- $($_.FunctionName) ($([System.IO.Path]::GetFileName($_.ScriptName)) Line $($_.ScriptLineNumber))" }
16381640
}

ObjectHandling/Import-TestToolkitToNavContainer.ps1

+10-2
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,8 @@
3939
Only Test Runner and Test Framework are/will be available in the online Business Central environment
4040
.Parameter environment
4141
Environment in which you want to import test toolkit.
42+
.Parameter appSymbolsFolder
43+
Folder where the app symbols should be stored when using compilerfolder
4244
.Example
4345
Import-TestToolkitToBcContainer -containerName test2
4446
.Example
@@ -68,8 +70,8 @@ function Import-TestToolkitToBcContainer {
6870
[switch] $useDevEndpoint,
6971
[hashtable] $replaceDependencies = $null,
7072
[Hashtable] $bcAuthContext,
71-
[string] $environment
72-
73+
[string] $environment,
74+
[string] $appSymbolsFolder
7375
)
7476

7577
$telemetryScope = InitTelemetryScope `
@@ -121,6 +123,12 @@ try {
121123
}
122124
Write-Host -ForegroundColor Green "TestToolkit successfully published"
123125
}
126+
elseif ($compilerFolder) {
127+
$appFiles = GetTestToolkitApps -compilerFolder $compilerFolder -includeTestRunnerOnly:$includeTestRunnerOnly -includeTestFrameworkOnly:$includeTestFrameworkOnly -includeTestLibrariesOnly:$includeTestLibrariesOnly -includePerformanceToolkit:$includePerformanceToolkit
128+
$appFiles | ForEach-Object {
129+
Copy-Item -Path $_ -Destination $appSymbolsFolder -Force
130+
}
131+
}
124132
else {
125133
$inspect = docker inspect $containerName | ConvertFrom-Json
126134
if ($inspect.Config.Labels.psobject.Properties.Match('maintainer').Count -eq 0 -or $inspect.Config.Labels.maintainer -ne "Dynamics SMB") {

ReleaseNotes.txt

+5
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,11 @@ Issue 1303 from AL-Go repository - renew federated token when access token needs
55
Issue 3590 EarliestMatching select option added to Download-BcNuGetPackageToFolder
66
Regression when from PR 3760 - Push-BcNuGetPackage doesn't work on PowerSHell 5
77
Issue 3772 Get-BcNuGetPackageId name length limit
8+
Two new parameter in Sort-AppFoldersByDependencies - selectSubordinates and SkippedApps. If specified, then only the appFolders specified in the first array plus the apps depending on these recursively will be sorted. The remaining folder not included in the sorted appfolders list will be returned in skippedApps.
9+
Remove some excessive logging
10+
If Import-TestToolkitToBcContainer was running with useCompilerFolder, then it would require a container to be present at that time.
11+
Make Run-AlPipeline do just-in-time creation of CompilerFolder and Containers
12+
Issue 1338 from AL-Go repository: Multi-Project Repository Test App Dependencies Not Resolved
813

914
6.0.29
1015
Issue 3591 When using Publish-NAVApp to publish an app, which fails compilation in the service, the command might hang forever - the fix for this is a temporary hack put in place for the versions which doesn't work.

0 commit comments

Comments
 (0)