Add support for configurable container engines (docker) #963
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
[no ci]
✅ Deploy Preview for ohsu-comp-bio-funnel ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
- Test command: ```sh funnel task create examples/md5sum.json ``` [no ci]
- Can be used in HPC environment with unique docker setups (e.g. exadocker) as well as future command templating efforts. [no ci]
- github.com/docker/docker versions above v24.0.9 resulted in `panic: Can't connect docker client`:
```
➜ go test tests/auth/auth_test.go
--- FAIL: TestBasicAuthFail (0.01s)
panic: Can't connect docker client [recovered]
panic: Can't connect docker client
FAIL command-line-arguments 0.673s
FAIL
```
- This error appears to be originating from [util/dockerutil/docker.go](https://github.com/ohsu-comp-bio/funnel/blob/f5acbabe85d28f45f2f86db959c4d7bc929334a4/util/dockerutil/docker.go#L28).
- github.com/getkin/kin-openapi versions above v0.112.0 resulted in type errors:
v0.113.0:
```
➜ go test util/openapi2proto/main.go
util/openapi2proto/main.go:54:38: invalid operation: p.Value.AdditionalProperties != nil (mismatched types openapi3.AdditionalProperties and untyped nil)
util/openapi2proto/main.go:55:24: cannot use p.Value.AdditionalProperties (variable of type openapi3.AdditionalProperties) as *openapi3.SchemaRef value in argument to getType
FAIL command-line-arguments [build failed]
FAIL
```
v0.123.0 (latest):
```
➜ go test util/openapi2proto/main.go
util/openapi2proto/main.go:43:7: cannot convert "integer" (untyped string constant) to type *openapi3.Types
util/openapi2proto/main.go:45:7: cannot convert "boolean" (untyped string constant) to type *openapi3.Types
util/openapi2proto/main.go:47:7: cannot convert "number" (untyped string constant) to type *openapi3.Types
util/openapi2proto/main.go:49:7: cannot convert "object" (untyped string constant) to type *openapi3.Types
util/openapi2proto/main.go:54:38: invalid operation: p.Value.AdditionalProperties != nil (mismatched types openapi3.AdditionalProperties and untyped nil)
util/openapi2proto/main.go:55:24: cannot use p.Value.AdditionalProperties (variable of type openapi3.AdditionalProperties) as *openapi3.SchemaRef value in argument to getType
util/openapi2proto/main.go:60:7: cannot convert "array" (untyped string constant) to type *openapi3.Types
util/openapi2proto/main.go:72:17: cannot use p.Value.Type (variable of type *openapi3.Types) as string value in return statement
util/openapi2proto/main.go:84:32: invalid operation: param.Schema.Value.Type != "" (mismatched types *openapi3.Types and untyped string)
util/openapi2proto/main.go:175:17: resp.Get undefined (type *openapi3.Responses has no field or method Get)
util/openapi2proto/main.go:175:17: too many errors
FAIL command-line-arguments [build failed]
FAIL
```
- TODO: investigate why this is necessary...
- Wildcards in outputs will be uploaded to a directory, with no special handling for the filename.
```yaml
tesOutput:
properties:
url:
description: |-
URL at which the TES server makes the output accessible after the task is complete. When tesOutput.path contains wildcards, it must be a directory; see `tesOutput.path_prefix` for details on how output URLs are constructed in this case.
```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
Add support for users to be able to pass in their desired container engine to the Funnel workers.
Related Github Issues
Additional Resources