-
Notifications
You must be signed in to change notification settings - Fork 31
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: improve handling of URLs in file functions. #369
Conversation
* allow `glob` to operate off of `file://` URLs. * allow `size` to operate off of `file://` URLs. * allow `basename` to operate off of any URLs. This commit also removes custom diagnostics from the stdlib functions so that all of the standard library functions now use `function_called_failed` diagnostics for errors. It gives a more consistent diagnostic for function calls.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Two questions about using fewer hardcoded strings in the error messages. I'm not sure how feasible either request is, so maybe they can be ignored. Just trying to think of easing future maintenance.
Otherwise, LGTM
CI is failing due to docker hub rate limiting. We need to fix the pull image functionality in Crankshaft to gate to a single request if there are a number of concurrent tasks all requesting to pull the same image; I think that may be contributing to the problem. |
As a workaround, I think we could do what bollard does in CI and use a local docker hub container that we've pushed any images we need for the tests, which will limit the pull requests to Docker Hub to only the number of images needed to run the tests. |
all that sounds good, but a perhaps simpler workaround is to pull from quay.io instead of docker hub to avoid rate limits entirely 🤷 |
CI is failing on windows for the |
As far as I know, there's no way to tell the local docker daemon to use an alternative default namespace for the image names, so we'd have to run the workflow/task evaluation tests with an engine config that overrides the default container and also update any explicit container names in the test WDLs (some of which are 1:1 with the example WDL in the spec). Not terribly difficult to do, though. Alternatively, I think if we pulled |
glob
to operate off offile://
URLs.size
to operate off offile://
URLs.basename
to operate off of any URLs.This PR also removes custom diagnostics from the stdlib functions so that all of the standard library functions now use
function_called_failed
diagnostics for errors. It gives a more consistent diagnostic for function calls.Before submitting this PR, please make sure:
changes (when appropriate).
CHANGELOG.md
(see"keep a changelog" for more information).