Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support docker-compose build with a git remote path as build.context #367

Open
pingu8007 opened this issue Jan 5, 2023 · 7 comments · May be fixed by #373
Open

Support docker-compose build with a git remote path as build.context #367

pingu8007 opened this issue Jan 5, 2023 · 7 comments · May be fixed by #373
Labels
feature-request New feature or request

Comments

@pingu8007
Copy link

Problem

Devcontainer-cli failed to build container with build context pointing to remote path (in this case, git).

sample docker-compose.yml:

version: '3.9'
services:
  workspace:
    build:
      context: https://my-private-repo/devcontainer.git
      dockerfile: jdk.dockerfile
      target: jdk15-mvn38-node14
    command: sleep infinity
    volumes:
      - ..:/workspace

This docker-compose.yml built correctly when using docker compose but failed when using devcontainer-cli build

Version

[email protected] /home/pingu/.nvm/versions/node/v14.21.2/lib/node_modules/@devcontainers/cli

Log file

pingu@XXXXXX:~/repo/vsnotes/.devcontainer$ devcontainer build --workspace-folder ..
[10 ms] @devcontainers/cli 0.27.1. Node.js v14.21.2. linux 5.10.102.1-microsoft-standard-WSL2 x64.
[273 ms] Start: Run: docker-compose -f /home/pingu/repo/vsnotes/.devcontainer/docker-compose.yml --profile * config
[359 ms] name: devcontainer
services:
  workspace:
    build:
      context: https://my-private-repo/devcontainer.git
      dockerfile: jdk.dockerfile
      target: jdk15-mvn38-node14
    command:
    - sleep
    - infinity
    networks:
      default: null
    volumes:
    - type: bind
      source: /home/pingu/repo/vsnotes
      target: /workspace
      bind:
        create_host_path: true
networks:
  default:
    name: devcontainer_default
Error: ENOENT: no such file or directory, open '/home/pingu/repo/vsnotes/.devcontainer/https:/my-private-repo/devcontainer.git/jdk.dockerfile'
{"outcome":"error","message":"ENOENT: no such file or directory, open '/home/pingu/repo/vsnotes/.devcontainer/https:/my-private-repo/devcontainer.git/jdk.dockerfile'","description":"An error occurred building the container."}
@samruddhikhandale
Copy link
Member

Looking at https://containers.dev/implementors/json_schema/. It seems like the Dev Containers Extension does not support a git remote path yet.

We'd use this issue to track this feature request. Thanks.

@samruddhikhandale samruddhikhandale added the feature-request New feature or request label Jan 5, 2023
@samruddhikhandale samruddhikhandale changed the title Failed with external build context Support docker-compose build with a git remote path as build.context Jan 5, 2023
@pingu8007
Copy link
Author

pingu8007 commented Jan 5, 2023 via email

@samruddhikhandale
Copy link
Member

samruddhikhandale commented Jan 5, 2023 •
edited
Loading

Error: ENOENT: no such file or directory, open '/home/pingu/repo/vsnotes/.devcontainer/https:/my-private-repo/devcontainer.git/jdk.dockerfile'
{"outcome":"error","message":"ENOENT: no such file or directory, open '/home/pingu/repo/vsnotes/.devcontainer/https:/my-private-repo/devcontainer.git/jdk.dockerfile'","description":"An error occurred building the container."}

Schema

"context": {
    "type": "string",
    "description": "The location of the context folder for building the Docker image. The path is relative to the folder containing the `devcontainer.json` file."
}

Looking at the error & the schema, devcontainers CLI is expecting a file path and not a git remote path in docker-compose.yml.

When I run devcontainer-cli build against devcontainer.json file which referencing same yml file, it fails.

It's expected as devcontainer-cli does not support it yet.

@pingu8007
Copy link
Author

pingu8007 commented Jan 5, 2023 via email

@ShauryaAg
Copy link
Contributor

@samruddhikhandale @joshspicer I would like to implement this feature for devcontainers.

While going through the docker's codebase to understand how they are doing it, I realised they are cloning the repository in a temp directory and setting that as the context.
It can be seen here:

https://github.com/docker/cli/blob/master/cli/command/image/build/context.go#L188-L193

Since, docker already clones the repository itself, I believe it could be implemented in one of the following ways:

  1. Use the directory where docker cloned the repository, and start dev-container in that container (using the container id)
  2. Clone the repository in a temp dir (or optionally, a specified directory) and update the docker-compose.yaml to set the context as the newly cloned repository's path.

Please let me know what you think about the two approaches.

@samruddhikhandale
Copy link
Member

@samruddhikhandale @joshspicer I would like to implement this feature for devcontainers.

Thanks @ShauryaAg, we appreciate it!

devcontainer build --workspace-folder ..

Currently, this command fails as it is trying to validate that the dockerfile mentioned by the docer-compose.yml file exists & then insect the image within the dockerfile.

Once that's done, it calls the docker build command to do the actual build.

I think the changes could be as simple as changing the inspect logic when .git is specified in the context. We could clone it in temp directory & validate/inspect it. Then leave the other things to docker build.

@chrmarti looking for you thoughts here. Also, feel free to correct my understanding of how the build works.

@chrmarti
Copy link
Contributor

Cloning the context to a temp folder when the dockfile is a relative path and the context is a Git repository location sounds good. We should then also update the context to that temporary folder (using the override docker-compose.yml we add for building), so Docker doesn't run another clone. The relevant code starts here:

cli/src/spec-node/dockerCompose.ts

Line 164 in d1c0dac

const resolvedDockerfilePath = cliHost.path.isAbsolute(dockerfilePath) ? dockerfilePath : path.resolve(context, dockerfilePath);

@ShauryaAg ShauryaAg linked a pull request Jan 13, 2023 that will close this issue
Open
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature-request New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add support for using git url in docker-compos build context ShauryaAg/devcontainers-cli
4 participants
@pingu8007 @chrmarti @samruddhikhandale @ShauryaAg