modules/nixos/buildbot-repo-allowlist: add microvm.nix #1598

zowoq · 2024-12-07T00:22:19Z

Didn't start automatically so I triggered an eval manually:

warning: ignoring the client-specified setting 'trusted-public-keys', because it is a restricted setting and you are not a trusted user
warning: ignoring untrusted substituter 'https://microvm.cachix.org', you are not a trusted user.
Run `man nix.conf` for more information on the `substituters` configuration option.
warning: ignoring the client-specified setting 'trusted-public-keys', because it is a restricted setting and you are not a trusted user
error: flake 'git+file:///var/lib/buildbot-worker/worker-081/astro_microvm_nix_nix-eval/build' does not provide attribute 'checks'
warning: ignoring untrusted substituter 'https://microvm.cachix.org', you are not a trusted user.
Run `man nix.conf` for more information on the `substituters` configuration option.
warning: ignoring the client-specified setting 'trusted-public-keys', because it is a restricted setting and you are not a trusted user
error: flake 'git+file:///var/lib/buildbot-worker/worker-081/astro_microvm_nix_nix-eval/build' does not provide attribute 'checks'
warning: ignoring untrusted substituter 'https://microvm.cachix.org', you are not a trusted user.
Run `man nix.conf` for more information on the `substituters` configuration option.
warning: ignoring the client-specified setting 'trusted-public-keys', because it is a restricted setting and you are not a trusted user
error: flake 'git+file:///var/lib/buildbot-worker/worker-081/astro_microvm_nix_nix-eval/build' does not provide attribute 'checks'
warning: ignoring untrusted substituter 'https://microvm.cachix.org', you are not a trusted user.
Run `man nix.conf` for more information on the `substituters` configuration option.
warning: ignoring the client-specified setting 'trusted-public-keys', because it is a restricted setting and you are not a trusted user
error: flake 'git+file:///var/lib/buildbot-worker/worker-081/astro_microvm_nix_nix-eval/build' does not provide attribute 'checks'
error: flake 'git+file:///var/lib/buildbot-worker/worker-081/astro_microvm_nix_nix-eval/build' does not provide attribute 'checks'
warning: ignoring untrusted substituter 'https://microvm.cachix.org', you are not a trusted user.
Run `man nix.conf` for more information on the `substituters` configuration option.
warning: ignoring the client-specified setting 'trusted-public-keys', because it is a restricted setting and you are not a trusted user
warning: ignoring untrusted substituter 'https://microvm.cachix.org', you are not a trusted user.
Run `man nix.conf` for more information on the `substituters` configuration option.
warning: ignoring the client-specified setting 'trusted-public-keys', because it is a restricted setting and you are not a trusted user
error: flake 'git+file:///var/lib/buildbot-worker/worker-081/astro_microvm_nix_nix-eval/build' does not provide attribute 'checks'
warning: ignoring untrusted substituter 'https://microvm.cachix.org', you are not a trusted user.
Run `man nix.conf` for more information on the `substituters` configuration option.
warning: ignoring the client-specified setting 'trusted-public-keys', because it is a restricted setting and you are not a trusted user
error: flake 'git+file:///var/lib/buildbot-worker/worker-081/astro_microvm_nix_nix-eval/build' does not provide attribute 'checks'
warning: ignoring untrusted substituter 'https://microvm.cachix.org', you are not a trusted user.
Run `man nix.conf` for more information on the `substituters` configuration option.
warning: ignoring the client-specified setting 'trusted-public-keys', because it is a restricted setting and you are not a trusted user
error: flake 'git+file:///var/lib/buildbot-worker/worker-081/astro_microvm_nix_nix-eval/build' does not provide attribute 'checks'
warning: ignoring untrusted substituter 'https://microvm.cachix.org', you are not a trusted user.
Run `man nix.conf` for more information on the `substituters` configuration option.
warning: ignoring the client-specified setting 'trusted-public-keys', because it is a restricted setting and you are not a trusted user
error: flake 'git+file:///var/lib/buildbot-worker/worker-081/astro_microvm_nix_nix-eval/build' does not provide attribute 'checks'
error: flake 'git+file:///var/lib/buildbot-worker/worker-081/astro_microvm_nix_nix-eval/build' does not provide attribute 'checks'
error: worker error: error: flake 'git+file:///var/lib/buildbot-worker/worker-081/astro_microvm_nix_nix-eval/build' does not provide attribute 'checks'

$ sudo cat /var/lib/buildbot-worker/worker-081/astro_microvm_nix_nix-eval/build/buildbot-nix.toml
attribute = "hydraJobs"

qowoz/microvm.nix@4843601

~~seems to ignore buildbot-nix.toml, works after the flake is changed to checks~~ seems to have been fixed in buildbot-nix
eval uses a massive amount of memory

zowoq · 2024-12-19T02:51:37Z

modules/nixos/buildbot.nix

+      "astro/microvm.nix" = {
+        url = "https://github.com/qowoz/microvm.nix.git";
+        defaultBranch = "main";
+      };


Suggested change

"astro/microvm.nix" = {

url = "https://github.com/qowoz/microvm.nix.git";

defaultBranch = "main";

};

"astro/microvm.nix" = {

url = "https://github.com/qowoz/microvm.nix.git";

defaultBranch = "main";

evalWorkerCount = 4;

};

@Mic92 Could we make evalWorkerCount configurable per repo? This repo uses a lot of memory and I really don't want to have to reduce it just for one repo.

@zowoq sounds doable and useful. Please open an issue. We have most of the code needed for this to happen already there.

I found it was easy to add it as a config file option so for now I'll use that: nix-community/buildbot-nix@2d039c3

Unfortunately found other problems. Even with just 8 workers it still uses a lot of memory, too much considering that we other jobs running. Also very slow to eval, after 20 minutes it hasn't even finished aarch64-linux, it'll block jobs other repos. Not sure if we'd want change the current eval lock to something like a per repo eval lock and limit the buildbot instance to 2 concurrent evals.

How many virtual machines does this have?
@astro are you sharing nixpkgs instances in your microvm machines tests. Do you disable documentation? Are you applying many overlays?
Maybe we also need to make configurable what it evaluates i.e. only a single architecture.

I can easily get the test matrix above 128 tests. It's all individual flake check entries but the nixpkgs are pinned through flake.lock. microvm.nix overlay only.

hyperfine -r 5 'nix-eval-jobs --workers 1 --force-recurse --flake github:astro/microvm.nix#hydraJobs.x86_64-linux.crosvm-startup-shutdown'

Benchmark 1: nix-eval-jobs --workers 1 --force-recurse --flake github:astro/microvm.nix#hydraJobs.x86_64-linux.crosvm-startup-shutdown Time (mean ± σ): 91.982 s ± 0.193 s [User: 54.372 s, System: 25.974 s] Range (min … max): 91.669 s … 92.138 s 5 runs

hyperfine -r 5 'nix-eval-jobs --workers 1 --force-recurse --flake github:astro/microvm.nix#hydraJobs.aarch64-linux.qemu-9pstore-overlay-shutdown-command'

Benchmark 1: nix-eval-jobs --workers 1 --force-recurse --flake github:astro/microvm.nix#hydraJobs.aarch64-linux.qemu-9pstore-overlay-shutdown-command Time (mean ± σ): 92.361 s ± 0.502 s [User: 54.808 s, System: 26.118 s] Range (min … max): 91.557 s … 92.925 s 5 runs

Seems I only pasted the benchmarks and forgot to include a comment.

I found that lots of the hydraJobs attributes took around 90 seconds each to eval.

The combination of the number of tests, the test attributes are slow to eval and that they also use a significant amount memory is a problem. I need to limit the number of parallel evals to reduce memory usage, which then makes the overall eval time to slow to be practical.

Good news is that soon we'll be able to offer buildbot CI app to external repos so if we can address eval issues you could have PR tests without needing to move the repo into nix-community.

This reverts commit da442d4.

Mic92 · 2025-02-15T09:13:23Z

modules/nixos/buildbot-repo-allowlist.nix

@@ -0,0 +1,23 @@
+[
+  "qowoz/microvm.nix"
+  "nix-community/authentik-nix"


Isn't the github topic less work overall than having this list synchronously managed?

github-actions bot added the terraform label Dec 7, 2024

zowoq force-pushed the microvm branch 2 times, most recently from 6ca7533 to d58f537 Compare December 10, 2024 03:47

zowoq commented Dec 19, 2024

View reviewed changes

zowoq added 4 commits February 15, 2025 12:40

modules/nixos/buildbot: disable topic, switch to repoAllowlist

254bcda

Revert "terraform/hydra-projects: add microvm.nix"

f897fab

This reverts commit da442d4.

modules/nixos/buildbot-repo-allowlist: add microvm.nix

1c47af3

test

f9d1e0b

zowoq force-pushed the microvm branch from d58f537 to f9d1e0b Compare February 15, 2025 02:43

github-actions bot added the docs label Feb 15, 2025

Mic92 reviewed Feb 15, 2025

View reviewed changes

zowoq mentioned this pull request Feb 16, 2025

modules/nixos/buildbot: disable topic, switch to repoAllowlist #1707

Draft

zowoq changed the title ~~modules/nixos/buildbot: add pullBased repo microvm.nix~~ modules/nixos/buildbot-repo-allowlist: add microvm.nix Feb 16, 2025

astro mentioned this pull request Feb 22, 2025

feat: Add microvm.credentialFiles for passing credentials to guests astro/microvm.nix#337

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

modules/nixos/buildbot-repo-allowlist: add microvm.nix #1598

modules/nixos/buildbot-repo-allowlist: add microvm.nix #1598

zowoq commented Dec 7, 2024 •

edited

Loading

zowoq Dec 19, 2024

Mic92 Dec 19, 2024

zowoq Jan 5, 2025

Mic92 Jan 9, 2025 •

edited

Loading

astro Jan 9, 2025

zowoq Jan 15, 2025

zowoq Feb 16, 2025

Mic92 Feb 15, 2025

modules/nixos/buildbot-repo-allowlist: add microvm.nix #1598

Are you sure you want to change the base?

modules/nixos/buildbot-repo-allowlist: add microvm.nix #1598

Conversation

zowoq commented Dec 7, 2024 • edited Loading

zowoq Dec 19, 2024

Choose a reason for hiding this comment

Mic92 Dec 19, 2024

Choose a reason for hiding this comment

zowoq Jan 5, 2025

Choose a reason for hiding this comment

Mic92 Jan 9, 2025 • edited Loading

Choose a reason for hiding this comment

astro Jan 9, 2025

Choose a reason for hiding this comment

zowoq Jan 15, 2025

Choose a reason for hiding this comment

zowoq Feb 16, 2025

Choose a reason for hiding this comment

Mic92 Feb 15, 2025

Choose a reason for hiding this comment

zowoq commented Dec 7, 2024 •

edited

Loading

Mic92 Jan 9, 2025 •

edited

Loading