Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker adaptor can retrieve an image from the local docker service #246

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

mih
Copy link
Member

@mih mih commented Oct 10, 2023

Closes: #199

@bpinsard would be great if you could give this a spin!

Demo

❯ datalad create busydemo
❯ cd busydemo
❯ datalad containers-add -u dhub://busybox:latest busy
❯ datalad drop .datalad/environments --reckless availability
❯ git annex info | grep 'local annex'
local annex keys: 0
local annex size: 0 bytes
❯ cat .datalad/config
[datalad "dataset"]
        id = b7adee52-a65a-43fc-a85b-c0d5e2d5b67c
[datalad "containers.busy"]
        image = .datalad/environments/busy/image
        cmdexec = {python} -m datalad_container.adapters.docker run {img} {cmd}
❯ datalad containers-run -n busy uname
[INFO   ] Saved busybox:latest to /tmp/busydemo/.datalad/environments/busy/image/tmpzrdd7mj2
[INFO   ] Making sure inputs are available (this may take some time)
[INFO   ] == Command start (output follows) =====
Linux
[INFO   ] == Command exit (modification check follows) =====
run(ok): /tmp/busydemo (dataset) [/home/mih/env/datalad-dev/bin/python -m ...]

Closes: datalad#199

Demo

```
❯ datalad create busydemo
❯ cd busydemo
❯ datalad containers-add -u dhub://busybox:latest busy
❯ datalad drop .datalad/environments --reckless availability
❯ git annex info | grep 'local annex'
local annex keys: 0
local annex size: 0 bytes
❯ cat .datalad/config
[datalad "dataset"]
        id = b7adee52-a65a-43fc-a85b-c0d5e2d5b67c
[datalad "containers.busy"]
        image = .datalad/environments/busy/image
        cmdexec = {python} -m datalad_container.adapters.docker run {img} {cmd}
❯ datalad containers-run -n busy uname
[INFO   ] Saved busybox:latest to /tmp/busydemo/.datalad/environments/busy/image/tmpzrdd7mj2
[INFO   ] Making sure inputs are available (this may take some time)
[INFO   ] == Command start (output follows) =====
Linux
[INFO   ] == Command exit (modification check follows) =====
run(ok): /tmp/busydemo (dataset) [/home/mih/env/datalad-dev/bin/python -m ...]
```
@codeclimate
Copy link

codeclimate bot commented Oct 10, 2023

Code Climate has analyzed commit 24a11ce and detected 11 issues on this pull request.

Here's the issue category breakdown:

Category Count
Security 11

View more on Code Climate.

@codecov
Copy link

codecov bot commented Oct 10, 2023

Codecov Report

Attention: 18 lines in your changes are missing coverage. Please review.

Comparison is base (c505e52) 94.69% compared to head (24a11ce) 93.23%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #246      +/-   ##
==========================================
- Coverage   94.69%   93.23%   -1.46%     
==========================================
  Files          25       25              
  Lines        1093     1124      +31     
==========================================
+ Hits         1035     1048      +13     
- Misses         58       76      +18     
Files Coverage Δ
datalad_container/containers_run.py 84.00% <77.77%> (-0.85%) ⬇️
datalad_container/adapters/docker.py 82.08% <27.27%> (-10.77%) ⬇️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@bpinsard
Copy link

Works for me, it seems to retrieve the layers correctly.

I currently no longer have a specific usecase for the docker adapter, but that is a great performance enhancement.

@yarikoptic
Copy link
Member

sorry, not grasping atm the whole machinery... if code works for you -- go ahead with the merge.

@yarikoptic yarikoptic added the patch Increment the patch version when merged label Oct 12, 2023
@mih
Copy link
Member Author

mih commented Oct 12, 2023

Thanks for the feedback! I am not sure whether I will have the chance to try the practicality of this implementation sufficiently. The next week will tell me that. For now, I will convert this to a draft PR.

If I get this chance to try this out sufficiently I will finish it out and repropose.

If this message is read after two weeks from now, and the PR is still in draft. Please close without waiting for me. Thanks!

@mih mih marked this pull request as draft October 12, 2023 06:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
patch Increment the patch version when merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

docker: do not get layers if present in local docker service
3 participants