Custom mirror #1
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
iQQBot
force-pushed
the
custom-mirror
branch
2 times, most recently
from
9c035c9
to
7f38bfa
Compare
Mirrors in `RegistryConfig.Mirrors` can be specified using a full URL (schema, trailing slashes) but registries in the input map are keyed by their hostname. Previous code used the mirror URL as key which resulted in an empty `RegistryConfig` being passed to the `fillInsecureOpts` function and didn't set the insecure options. Use Host part of the parsed registry as a key instead. Signed-off-by: Paweł Gronowski <[email protected]> (cherry picked from commit ec65557) Signed-off-by: Paweł Gronowski <[email protected]>
[0.12 backport] util/resolver: Fix insecure mirrors
If both Insecure and PlainHTTP is requested for the host, use a transport that falls back to HTTP in case of an HTTP request to a HTTPS client error. This also changes the order - before that an HTTP connection was attempted first. Now an HTTPS connection with insecure TLS check will be attempted first and will only fallback to HTTP if the former fails. This fixes push to an insecure HTTPS-only registry. Signed-off-by: Paweł Gronowski <[email protected]> (cherry picked from commit 1096d71) Signed-off-by: Paweł Gronowski <[email protected]>
It's no longer needed to return multiple hosts. Signed-off-by: Paweł Gronowski <[email protected]> (cherry picked from commit baf7c09) Signed-off-by: Paweł Gronowski <[email protected]>
If build contains multiple subbuilds all of their sources are tracked in provenance attestations. When some subbuilds are coming from same source file (eg. same Dockerfile but different targets) currently the same file would appear in multiple times. This detects such duplicates and makes sure definitions from multiple subbuilds can map to same file. Signed-off-by: Tonis Tiigi <[email protected]> (cherry picked from commit 1bbf73e) Signed-off-by: Justin Chadwell <[email protected]>
Signed-off-by: Tonis Tiigi <[email protected]> (cherry picked from commit 641c552) Signed-off-by: Justin Chadwell <[email protected]>
Signed-off-by: Chris Goller <[email protected]> (cherry picked from commit e0ccc47) Signed-off-by: Justin Chadwell <[email protected]>
Signed-off-by: Chris Goller <[email protected]> (cherry picked from commit 4d4fc4d) Signed-off-by: Justin Chadwell <[email protected]>
Signed-off-by: Chris Goller <[email protected]> (cherry picked from commit 40741df) Signed-off-by: Justin Chadwell <[email protected]>
This reverts commit 1aef766. Signed-off-by: Justin Chadwell <[email protected]> (cherry picked from commit d51edce) Signed-off-by: Justin Chadwell <[email protected]>
When calling client.Wait, we want to avoid the default backoff behavior, because we want to achieve a quick response back once the server becomes active. To do this, without modifying the entire client's exponential backoff configuration, we can use conn.ResetConnectBackoff, while attempting to reconnect every second. Here are some common scenarios: - Server is listening: the call to Info succeeds quickly, and we return. - Server is listening, but is behind several proxies and so latency is high: the call to Info succeeds slowly (up to minConnectTimeout=20s), and we return. - Server is not listening and gets "connection refused": the call to Info fails quickly, and we wait a second before retrying. - Server is not listening and does not respond (e.g. firewall dropping packets): the call to Info fails slowly (by default after minConnectTimeout=20s). After the call fails, we wait a second before retrying. Signed-off-by: Justin Chadwell <[email protected]> (cherry picked from commit f1d7f2e) Signed-off-by: Justin Chadwell <[email protected]>
ResolveImageConfig can be called concurrently - for example, by
dockerfile2llb during conversion, we loop through each stage and resolve
the base image for that stage.
In the case that two calls to ResolveImageConfig finish at roughly the
same time, we can hit an edge case where we attempt to modify the
bridge's image records at the same time.
To fix this, we just need to use the bridge's mutex to prevent
concurrent access here.
This should fix the following stack trace found in CI:
sandbox.go:144: goroutine 1079 [running]:
sandbox.go:144: github.com/moby/buildkit/solver/llbsolver.(*provenanceBridge).ResolveImageConfig(0xc000431e00, {0x1c2b040?, 0xc0008e5b30?}, {0xc00094ba00?, 0xc0003728f0?}, {0x0, 0xc0006cb580, {0x19ba868, 0x7}, {0xc0008f7500, ...}, ...})
sandbox.go:144: /src/solver/llbsolver/provenance.go:139 +0x1fb
sandbox.go:144: github.com/moby/buildkit/frontend/dockerfile/dockerfile2llb.toDispatchState.func3.1()
sandbox.go:144: /src/frontend/dockerfile/dockerfile2llb/convert.go:405 +0x5fe
sandbox.go:144: golang.org/x/sync/errgroup.(*Group).Go.func1()
sandbox.go:144: /src/vendor/golang.org/x/sync/errgroup/errgroup.go:75 +0x64
sandbox.go:144: created by golang.org/x/sync/errgroup.(*Group).Go
sandbox.go:144: /src/vendor/golang.org/x/sync/errgroup/errgroup.go:72 +0xa5
--- FAIL: TestIntegration/TestNoCache/worker=oci-rootless/frontend=builtin (4.45s)
No other explanation for this failure makes sense - `b` cannot be `nil`
at this point, since a call to `b.llbBridge.ResolveImageConfig` has just
succeeded (also because that would be very strange).
Signed-off-by: Justin Chadwell <[email protected]>
(cherry picked from commit c08f767)
Signed-off-by: Justin Chadwell <[email protected]>
Fixes moby#4108 Signed-off-by: Mark Yen <[email protected]> (cherry picked from commit d48bf06) Signed-off-by: Justin Chadwell <[email protected]>
This responds to review feedback from moby#4308 (review) Signed-off-by: Mark Yen <[email protected]> (cherry picked from commit f9ccb09) Signed-off-by: Justin Chadwell <[email protected]>
Before this change, all platforms that loosely match the provided platform will be fetched even though we only care about 1 of them. As an example when linux/amd64 is requested it will also fetch linux/386 because it is a compatible architecture. This means extra round trips to the registry, potentially even for content that doesn't exist in the remote. This is especially a problem when resolve mode is prefer-local because we'll have the index locally but most likely only one manifest. In this case we'll end up reaching out to the registry to fetch the other manifests unncessarily. With this change instead of fetching all matching platforms it chooses only the best matching platform. Signed-off-by: Brian Goff <[email protected]> (cherry picked from commit 575cb10) Signed-off-by: Justin Chadwell <[email protected]>
[v0.12] Cherry-picks for v0.12.3
Signed-off-by: CrazyMax <[email protected]> (cherry picked from commit 39086be)
[0.12] cherry-picks 0.12.3 take 2
Signed-off-by: CrazyMax <[email protected]> (cherry picked from commit cc7b649)
Signed-off-by: CrazyMax <[email protected]> (cherry picked from commit 5955ccf)
[0.12 backport] fix riscv64 build
…-0.12 [0.12 backport] util/resolver: Perform Insecure HTTPS + HTTP fallback in one `docker.RegistryHost`
fixes data corruption with zstd output in "best" - 1.17.2 diff: klauspost/compress@v1.17.1...v1.17.2 - full diff: klauspost/compress@v1.16.3...v1.17.2 Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit c4c7d7b) Signed-off-by: Sebastiaan van Stijn <[email protected]>
[0.12 backport] vendor: github.com/klauspost/compress v1.17.2
Signed-off-by: Tonis Tiigi <[email protected]> (cherry picked from commit e99bfa9)
Signed-off-by: Tonis Tiigi <[email protected]> (cherry picked from commit 7f8eedc)
The events for currently active builds were sent through pubsub channel instead of directly to the current request, like it was done for completed builds for example. This meant that if there are more active builds running than the pubsub channel buffer (32) the sends will block. Because the history API mutex is held in this process it will eventually block the requests for builds that try to update their history records. Signed-off-by: Tonis Tiigi <[email protected]> (cherry picked from commit 1ed9931)
[v0.12] vendor: github.com/cyphar/filepath-securejoin v0.2.4
Signed-off-by: CrazyMax <[email protected]> (cherry picked from commit bb18da8) Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit 295c899) Signed-off-by: Sebastiaan van Stijn <[email protected]>
[0.12 backport] hack/test: allow ALPINE_VERSION to be set from env
Signed-off-by: CrazyMax <[email protected]> (cherry picked from commit 8d0b15f)
Signed-off-by: CrazyMax <[email protected]> (cherry picked from commit 0e70765)
…ce-context [0.12 backport] use local context and disable image push if not upstream repo
Signed-off-by: Tonis Tiigi <[email protected]> (cherry picked from commit ef536af15b2d351b8f0459022decc2a4955b1cb2) (cherry picked from commit a8a6bc5180696624b18b5dc4ed4f9cf1a278ef27)
Signed-off-by: Tonis Tiigi <[email protected]> (cherry picked from commit d293ec3208f87fefab7a1caadffa3f3f50604796) (cherry picked from commit 42b95935d606b262a33374eeeb452bb7c299c729)
Signed-off-by: Tonis Tiigi <[email protected]> (cherry picked from commit 432ece72ae124ce8a29ced6854a08206f09f3a73) (cherry picked from commit e4bd60baf77b4ec92aba60f568831fb3076fc158)
Signed-off-by: Tonis Tiigi <[email protected]> (cherry picked from commit 4e2569e796aae398648082689d70ca1d4f4f74a8) (cherry picked from commit caea271063973c6903be08c1ebbc7c103f67805f)
Signed-off-by: Tonis Tiigi <[email protected]> (cherry picked from commit 00060c60c26b07015133edacfa32f569ceefea2e) (cherry picked from commit c890068b0da9d746cfa0f2627e0ee5cc60f869d3)
Running interactive container APIs was done by giving the gateway implementation access to worker controller directly, but it should be passed with a build job instead. Signed-off-by: Tonis Tiigi <[email protected]> (cherry picked from commit 0971dffaab93d91e51af984b44c745b35b3c5b4d) (cherry picked from commit 564f884e7bb6db9c63e03c3b081ea71e15aa7980)
Ensure interactive calls validate same conditions that the build requests do. Refactor of the build side is to ensure we use the same validation function for both cases. There was no validation issue with the LLB validation. Signed-off-by: Tonis Tiigi <[email protected]> (cherry picked from commit d1970522d7145be5f4a1f1a028b1910bb527126c) (cherry picked from commit e1e30278d0a491dfd34bd80fa66b54106614cffa)
Signed-off-by: Tonis Tiigi <[email protected]> (cherry picked from commit 96ccaec09c51176a6d954fd7c4ce57d519bae1b2) (cherry picked from commit a9523c6476f39bb44dd02bcab19e8cb25c5bc37b)
Signed-off-by: Tonis Tiigi <[email protected]> (cherry picked from commit 42d866e) (cherry picked from commit e81066f8a8623dc876f3d64fae8f693c17ecdc1a)
While submount paths were already validated there are some cases where the parent mount may not be immutable while the submount is created. Signed-off-by: Tonis Tiigi <[email protected]> (cherry picked from commit 2529ec4121bcd8c35bcd96218083da175c2e5b77) (cherry picked from commit cbc233b3b695918d92fd5b1407b829296c53db70)
Signed-off-by: Tonis Tiigi <[email protected]>
|
Updated master via sync fork. |
|
Will rebase these changes off of https://github.com/moby/buildkit/releases/tag/v0.12.5 |
kylos101
force-pushed
the
custom-mirror
branch
from
5b62060
to
af78548
Compare
|
Instructions for updating. After syncing our fork's default branch: First, set some params: # the release tag we want to patch from
export LATEST_RELEASE=v0.12.5
# the SHA of the immediate commit before our changes, to help create patch files
export COMMIT_REF=567a99433ca23402d5e9b9f9124005d2e59b8861
export FEATURE_BRANCH=custom-mirrorThen, do the update: git checkout $FEATURE_BRANCH
git format-patch -i $COMMIT_REF
git branch -m old
git fetch upstream --tags
git checkout $LATEST_RELEASE # this tag must exist locally
git switch -c $FEATURE_BRANCH # use $LATEST_RELEASE as the base branch (or the version you want to publish)
git am *.patch --3way
git push --set-upstream origin $FEATURE_BRANCH --force |
15 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.