feat(arx): add experimental macos support via fakedir #107

cassandracomar · 2024-08-08T18:58:44Z

fixes #17

see fakedir for limitations and caveats.

I've tested the nix bundle interface and am able to produce a working executable arx. I need to find a machine that doesn't have the /nix/store paths already loaded to do a final validation. I tested using ruby and I'm able to run basic ruby scripts. I'm going to continue testing with a more complicated application that makes use of native extensions next.

basic flake for testing:

{
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-24.05";
    flake-utils.url = "github:numtide/flake-utils";
    nix-bundle.url = "github:cassandracomar/nix-bundle";
    bundlers = {
      url = "github:nixos/bundlers";
      inputs.nix-bundle.follows = "nix-bundle";
    };
  };

  outputs = {
    nixpkgs,
    flake-utils,
    nix-bundle,
    bundlers,
    ...
  }:
    flake-utils.lib.eachDefaultSystem (
      system: let
        pkgs = import nixpkgs {
          inherit system;
        };
      in {
        defaultBundler = bundlers.defaultBundler.${system};
      }
    );
}

use this flake as your bundler:

nix bundle --bundler path:/path/to/flake nixpkgs#app

once this PR merges and we're able to update the nix-bundle reference in nixos/bundlers, nix bundle should work out of the box on macos, with the caveat that fakedir has to be able to intercept the syscalls in use by your application.

fixes nix-community#17 see: [fakedir](https://github.com/nixie-dev/fakedir/tree/b0f70b805aa86fe881eeca642f12069c7983a8e7) for limitations and caveats.

thread argv0 through the arx call stack so we can select the binary in the target's bin path based on it. this also fixes an issue with fakedir where running `Dir.pwd` within ruby would cause a segfault due to a null ptr dereference.

cassandracomar · 2024-08-11T01:39:37Z

I added a patch to fix a null pointer dereference when ruby calls getcwd() (it passes a null destination buffer to getcwd(), which I'm not sure is totally sensible). I'll upstream the patch when I figure out what to do about the fact that this patch technically leaks the buffer.

I've also made it possible to run multiple executables within the same target package (in the same /bin directory). it uses argv0 to decide which binary to execute so symlink the bottle to the other binaries you want to reach and use symlinkJoin to produce a composite package with all the tools you need in one place.

lastly, because arx changes PWD and because fakedir can't set up a chroot with paths bind mounted into the right places, I added a way to restore PWD context, so bottled tools work just like a natively installed tool -- there's now a TMPX_RESTORE_PWD env var set by the tmpx script before calling the startup script that you can use to reset the PWD context after saving the TMPX unpack dir.

the null buffer issue has been resolved upstreamed with a PR. also, fix an issue with symlinks/shebangs not being rewritten for the initial program executed by the startup script -- we need to avoid that call to execvp happening in the process environment that existed prior to the injection of the fakedir libraries. to do that, run a new shell where the exported env vars are set and avoid that shell just execing by first testing if the fakedir library is present and loaded, then running the specified executable.

running into errors with a deficient sh that lacks `exec -a` when running in GitHub actions. switching to bash as a) it's available everywhere except windows and b) ensures we can still pass the bin name through as $0.

HannesGitH · 2025-03-26T13:33:48Z

unfortunately this does not work rn

➜  test git:(main) ✗ nix bundle --bundler . 'nixpkgs#hello' -L --show-trace
warning: not writing modified lock file of flake 'git+file:///Users/blingmember/Repos/bling_cli?dir=test':
• Added input 'bundlers':
    'github:nixos/bundlers/e3ea69055b77cc2b17c6ce4067b7e3bb01871245?narHash=sha256-eQyHKjL/u8w/9WiLfLtFPzqX0swn4VPqZMJ5/kKd3wc%3D' (2024-11-04)
• Added input 'bundlers/nix-bundle':
    follows 'nix-bundle'
• Added input 'bundlers/nix-utils':
    'github:juliosueiras-nix/nix-utils/b44e1ffd726aa03056db9df469efb497d8b9871b?narHash=sha256-9G8zo%2B0nfYAALV5umCyQR/2hVUFNH10JropBkyxZGGw%3D' (2021-09-30)
• Added input 'bundlers/nix-utils/flake-utils':
    'github:numtide/flake-utils/f7e004a55b120c02ecb6219596820fcd32ca8772?narHash=sha256-A8BU7bjS5GirpAUv4QA%2BQnJ4CceLHkcXdRp4xITDB0s%3D' (2021-06-16)
• Added input 'bundlers/nix-utils/nixpkgs':
    'github:nixos/nixpkgs/3788c68def67ca7949e0864c27638d484389363d?narHash=sha256-Aj20gmGBs8TG7pyaQqgbsqAQ6cB%2BTVuL18Pk3DPBxcQ%3D' (2021-08-18)
• Added input 'bundlers/nixpkgs':
    'github:NixOS/nixpkgs/5633bcff0c6162b9e4b5f1264264611e950c8ec7?narHash=sha256-9UTxR8eukdg%2BXZeHgxW5hQA9fIKHsKCdOIUycTryeVw%3D' (2024-10-09)
• Added input 'flake-utils':
    'github:numtide/flake-utils/11707dc2f618dd54ca8739b309ec4fc024de578b?narHash=sha256-l0KFg5HjrsfsO/JpG%2Br7fRrqm12kzFHyUHqHCVpMMbI%3D' (2024-11-13)
• Added input 'flake-utils/systems':
    'github:nix-systems/default/da67096a3b9bf56a91d16901293e51ba5b49a27e?narHash=sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768%3D' (2023-04-09)
• Added input 'nix-bundle':
    'github:cassandracomar/nix-bundle/30c2ad34a5999251e9f08c157e0bd3c4d372a889?narHash=sha256-7BH4TvfSLy4NHMjS4xUbXFDtqq7lmcKNgL6MdaspGFg%3D' (2024-09-17)
• Added input 'nix-bundle/fakedir-pkgs':
    'github:nixie-dev/fakedir/a2bd8d0eb974bcbfbc4963e7c61a7162968cee6b?narHash=sha256-krWOwpqjg83Y5e9ly8IJ81icqH4YUNbkIIW1jqiaYhY%3D' (2024-08-12)
• Added input 'nix-bundle/fakedir-pkgs/nixpkgs':
    'github:NixOS/nixpkgs/d412d7956754408c347291516d1d3e3ea469642b?narHash=sha256-WlJapOhp/D/r8SjajToSBsrX9r0%2BjeKmZ6C8OZ3BJ94%3D' (2023-02-21)
• Added input 'nix-bundle/fakedir-pkgs/utils':
    'github:numtide/flake-utils/3db36a8b464d0c4532ba1c7dda728f4576d6d073?narHash=sha256-XX2f9c3iySLCw54rJ/CZs%2BZK6IQy7GXNY4nSOyu2QG4%3D' (2023-02-13)
• Added input 'nix-bundle/nixpkgs':
    'github:NixOS/nixpkgs/7cec143201c32c4937f2c153af4a9f28a3d9bec1?narHash=sha256-bFOTvmkJ2c1ku%2BE0gvqmNEF2D1PSmujDFLofKAMF/pM%3D' (2024-08-07)
• Added input 'nixpkgs':
    'github:nixos/nixpkgs/b134951a4c9f3c995fd7be05f3243f8ecd65d798?narHash=sha256-OnSAY7XDSx7CtDoqNh8jwVwh4xNL/2HaJxGjryLWzX8%3D' (2024-12-30)
error:
       … while evaluating the attribute 'defaultBundler'
         at /nix/store/5f8nydan34q0r5y59dyrx0cxz1p0d1p1-source/test/flake.nix:25:9:
           24|       in {
           25|         defaultBundler = bundlers.defaultBundler.${system};
             |         ^
           26|       }

       error: attribute 'defaultBundler' missing
       at /nix/store/5f8nydan34q0r5y59dyrx0cxz1p0d1p1-source/test/flake.nix:25:26:
           24|       in {
           25|         defaultBundler = bundlers.defaultBundler.${system};
             |                          ^
           26|       }

swapping defaultBundler = bundlers.defaultBundler.${system}; for defaultBundler = bundlers.bundlers.${system}.default; doesnt help either and results in

error:
      … while evaluating the attribute 'defaultBundler'
        at /nix/store/vh9xpn2ald80s4asxwpwj2kcq0d7hq97-source/test/flake.nix:25:9:
          24|       in {
          25|         defaultBundler = bundlers.bundlers.${system}.default;
            |         ^
          26|       }

      … while evaluating the attribute 'bundlers.aarch64-darwin.default'
        at /nix/store/1gyz5qxc7gqzabf2ds6rz2h86s3qzb8p-source/flake.nix:47:7:
          46|
          47|       default = toArx;
            |       ^
          48|       toArx = nix-bundle.bundlers.${system}.nix-bundle;

      … while evaluating the attribute 'bundlers.aarch64-darwin.nix-bundle'
        at /nix/store/6mjljsdwws9cwwkznvi0q8wzfl5pxgr0-source/flake.nix:8:5:
           7|   outputs = { self, nixpkgs, fakedir-pkgs }: {
           8|     bundlers = {
            |     ^
           9|       nix-bundle = { program, system }: let

      error: attribute 'aarch64-darwin' missing
      at /nix/store/1gyz5qxc7gqzabf2ds6rz2h86s3qzb8p-source/flake.nix:48:15:
          47|       default = toArx;
          48|       toArx = nix-bundle.bundlers.${system}.nix-bundle;
            |               ^
          49|

HannesGitH · 2025-03-26T13:42:08Z

@cassandracomar if you could help me fix this seemingly small problem, i can check if the bundled executable runs on macs without nix installed

cassandracomar · 2025-03-27T21:50:50Z

can you post your flake? want to compare with the testing I've done.

HannesGitH · 2025-03-28T08:51:05Z

Thanks for the response!

for that error reproduction I used the exact flake you provided :)
but it probably got locked to newer nixpkgs, bundlers thats why i provided that output as well

cassandracomar · 2025-03-29T01:09:46Z

hmm, I'll see this weekend if I can figure out what's changed upstream.

feat(arx): add macos support via fakedir

9116f7c

fixes nix-community#17 see: [fakedir](https://github.com/nixie-dev/fakedir/tree/b0f70b805aa86fe881eeca642f12069c7983a8e7) for limitations and caveats.

cassandracomar changed the title ~~feat(arx): add macos support via fakedir~~ feat(arx): add experimental macos support via fakedir Aug 8, 2024

feat: execute binary specified by $0

8a9cac1

thread argv0 through the arx call stack so we can select the binary in the target's bin path based on it. this also fixes an issue with fakedir where running `Dir.pwd` within ruby would cause a segfault due to a null ptr dereference.

cassandracomar force-pushed the master branch from a8b10a1 to f15e455 Compare August 13, 2024 17:02

Cassandra Comar added 4 commits August 13, 2024 15:32

tested working on clean macos VM

16c20b3

fix: prefer bash over sh

10bcbdf

running into errors with a deficient sh that lacks `exec -a` when running in GitHub actions. switching to bash as a) it's available everywhere except windows and b) ensures we can still pass the bin name through as $0.

fix: remove unneeded echos

59af1bb

fix: make the linux chroot also have CWD set properly

6100bf4

cassandracomar force-pushed the master branch from 078473c to 6100bf4 Compare September 7, 2024 21:45

fix: multi-argument commands

71db641

cassandracomar force-pushed the master branch from eeecf8f to 71db641 Compare September 16, 2024 23:33

fix: ensure stdin still gets passed through

30c2ad3

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(arx): add experimental macos support via fakedir #107

feat(arx): add experimental macos support via fakedir #107

cassandracomar commented Aug 8, 2024 •

edited

Loading

cassandracomar commented Aug 11, 2024 •

edited

Loading

HannesGitH commented Mar 26, 2025

HannesGitH commented Mar 26, 2025

cassandracomar commented Mar 27, 2025

HannesGitH commented Mar 28, 2025

cassandracomar commented Mar 29, 2025

feat(arx): add experimental macos support via fakedir #107

Are you sure you want to change the base?

feat(arx): add experimental macos support via fakedir #107

Conversation

cassandracomar commented Aug 8, 2024 • edited Loading

cassandracomar commented Aug 11, 2024 • edited Loading

HannesGitH commented Mar 26, 2025

HannesGitH commented Mar 26, 2025

cassandracomar commented Mar 27, 2025

HannesGitH commented Mar 28, 2025

cassandracomar commented Mar 29, 2025

cassandracomar commented Aug 8, 2024 •

edited

Loading

cassandracomar commented Aug 11, 2024 •

edited

Loading