Ensure that the temporary directory that we create is never itself the
top-level directory of the content that we're downloading, in case it's
an archive which includes a "." with weird permissions.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Signed-off-by: David Shea <dshea@redhat.com>
Support splitting build logs for multi-arch builds ( `--platform` ) by
using `--logfile` and `--logsplit` which splits generated log file in the
convention `${file}_${platform_os}_${platform_arch}`.
Closes: https://github.com/containers/buildah/issues/3960
Signed-off-by: Aditya R <arajan@redhat.com>
As per `OCI` `image-spec` setting history is optional so buildah must
support these use-cases via `--omit-history=true` some of the use-cases
are discussed below.
* There are use-cases when image-spec contributes to the image size by
adding build history to image-spec in certain use-cases this image
history is not required by the end-users but there is no way to remove
it. See: https://github.com/containers/buildah/issues/3513
* Certain build tools don't honor setting build history in image spec
however buildah does not allow processing images without build history
since setting build history is still optional as per `OCI` image-spec
( https://github.com/opencontainers/image-spec/blob/main/config.md#properties)
so buildah must support such use-cases.
Closes: https://github.com/containers/buildah/issues/4025
Closes: https://github.com/containers/buildah/issues/3513
Signed-off-by: Aditya R <arajan@redhat.com>
As builds got more complicated, the ability to only access files from one location became quite limiting. With `multi-stage` builds where you can `copy` files from other parts of the Containerfile by adding the `--from` flag and pointing it to the name of another Containerfile stage or a remote image.
The new named build context feature is an extension of this pattern. You can now define additional build contexts when running the build command, give them a name, and then access them inside a Dockerfile the same way you previously did with build stages.
Additional build contexts can be defined with a new `--build-context [name]=[value]` flag. The key component defines the name for your build context and the value can be:
```console
Local directory – e.g. --build-context project2=../path/to/project2/src
HTTP URL to a tarball – e.g. --build-context src=https://example.org/releases/src.tar
Container image – Define with a docker-image:// prefix, e.g. --build-context alpine=docker-image://alpine:3.15, ( also supports docker://, container-image:// )
```
On the Containerfile side, you can reference the build context on all commands that accept the “from” parameter. Here’s how that might look:
```Dockerfile
FROM [name]
COPY --from=[name] ...
RUN --mount=from=[name] …
```
The value of [name] is matched with the following priority order:
* Named build context defined with `--build-context [name]=..`
* Stage defined with `AS [name]` inside Dockerfile
* Remote image `[name]` in a container registry
Added Features
* Pinning images for `FROM` and `COPY`
* Specifying multiple buildcontexts from different projects
and using them with `--from` in `ADD` and `COPY` directive
* Override a Remote Dependency with a Local One.
* Using additional context from external `Tar`
Signed-off-by: Aditya R <arajan@redhat.com>
Buildah now supports renaming devices while performing a build using
`--device <some-name>:<new-name>`. Implementation is similar to `podman`
where we prefer using `bind-mount` for devices instead of `mknod` in
`rootless` setups.
Usage
```console
buildah build -t test --device /dev/null:/test/dev/null .
```
Closes: https://github.com/containers/buildah/issues/4002
Signed-off-by: Aditya R <arajan@redhat.com>
Teach `buildah build` and `buildah config` about the OS version and
features fields. We don't tend to use them on Linux, but they're in the
OCI and Docker config structures, so we need to be able to preserve and
manipulate their values, much as we do for variant information.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add a `--env` option to `buildah build` that functions similarly to the
`buildah config --env` option, to complement `buildah build`'s
`--unsetenv` option.
Document that `buildah config`'s `--env` function fetches the current
value for a variable when the name is supplied, but no `=` or value
follows it.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Allows end-users to export final build content or rootfs to external formats.
By default, a local container image is created from the build result. The --output (or -o) flag allows you to override this behavior, and a specify a custom exporter. For example, custom exporters allow you to export the build artifacts as files on the local filesystem instead of a Container image, which can be useful for generating local binaries, code generation etc.
The value for --output is a CSV-formatted string defining the exporter type and options. Currently, local and tar exporters are supported. The local exporter writes the resulting build files to a directory on the client side. The tar exporter is similar but writes the files as a single tarball (.tar).
```console
buildah build --output type=local,dest=dir .
buildah build --output type=tar,dest=rootfs.tar .
buildah build -o dir .
```
Reference: https://docs.docker.com/engine/reference/commandline/build/#custom-build-outputs
Signed-off-by: Aditya R <arajan@redhat.com>
Allow end users to remove default identity labels if they want to.
Since there are instances where images can be reproduced across version
hence users must have option to suppress default labels.
Closes: https://github.com/containers/buildah/issues/3826
Signed-off-by: Aditya R <arajan@redhat.com>
Users want to turn off addition of /etc/hosts file while building
container images, this would allow them to customize the /etc/hosts
file within the image.
Fixes: https://github.com/containers/buildah/issues/3808
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
We need to use the default from containers.conf and not hardcode them in
buildah. This fixes an issue with the cni network backend since it would
try to access /etc/cni/net.d/ even as rootless user. This regression was
introduced in commit f9cff07b81.
Also hide the cni flags as we do not expect users to change this. The
recommended way is to change them in containers.conf.
[NO NEW TESTS NEEDED]
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Podman uses the new netavark network stack. Buildah should be able to do
the same. Both projects should use the same networking code which was
move to c/common/libnetwork. The new network interface can use either
CNI or netvavark. Using the same code for podman and buildah is
important to ensure that both use the same backend. Mixing CNI and
netavark is not supported.
This also fixes some outstanding CNI issues, e.g. buildah trying to
connect all cni networks.
[NO NEW TESTS NEEDED]
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
This idea of this PR is to allow Podman to add a Podman
prefix to containers. This would allow it to keep track
of containers created by Podman and make it easier to remove
them when it wants to remove all.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
When define.TempDirForURL() runs an external command, if it fails,
include the command's output in the error that's handed back to its
caller so that the end-user has some idea of what happened.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Run secrets can now be created from an environment variable. The
environment variable is read and is briefly stored as a file on /dev/shm
when it's being used, and the file is removed after the RUN command is
finished.
Fixes: #3524
Signed-off-by: Ashley Cui <acui@redhat.com>
Users have accidently had a .containerignore or .dockerignore
file in their context directly which blocked all content.
Currently we report that no globs matched, but do not
identify where the globs came from.
This change is an attempt to add this data to the error.
Example: https://github.com/containers/buildah/issues/3318
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Add a --all-platforms that instructs the builder to build for the
intersection of all platforms for which the build's base images are
available. Returns an error if any of them aren't references to
manifest lists. We've learned that we can't really trust architecture
and OS information stored in image config blobs, so we don't try to
salvage that case.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
The build with git URL recognizes git URLs by either a git:// scheme prefix, or
any other URL with other schemes such as https:// and a .git suffix.
The build with git URL and branch recognizes the branch name as the fragment
part of the URL, the part after the #.
The .git suffix check is performed by checking that the suffix of the full URL
is .git, however this causes URLs that begin with scheme https://, and have a
.git path suffix, to not be recognized as git URLs when a branch name is
appended.
The logic for checking if the URL path is suffixed with .git should parse the
URL first and consider only the path when looking for the suffix. This change
is implemented in this commit.
This commit also removes the guard within the cloneToDirectory function because
it is unnecessary since every location the function is called the check has
already been carried out.
Signed-off-by: Leigh McCulloch <351529+leighmcculloch@users.noreply.github.com>
Move multiple-platform build juggling logic from the CLI wrapper
directly into the imagebuildah package, to make using it easier for
packages that consume us as a library.
This requires reading Dockerfiles into byte slices so that we can
re-parse them for each per-platform build, rather than parsing them
directly, as we used to, since building modifies the parsed tree.
When building for multiple platforms, prefix progress log messages with
the platform description.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Allow ssh socket from host or certain ssh keys to be exposed to a
certain RUN instruction, but not any other instructions, as well as not
showing up in the final image.
This is done by spawining a new agent from buildah and mounting
the listening socket inside the run. SSH_AUTH_SOCK inside the container
will be set to the socket mountpoint. The defualt mountpoint is
/run/buildkit/ssh_agent.{i}
Signed-off-by: Ashley Cui <acui@redhat.com>
Add a pkg/parse.PlatformsFromOptions() which understands a "variant"
value as an optional third value in an OS/ARCH[/VARIANT] argument value,
which accepts a comma-separated list of them, and which returns a list
of platforms.
Teach "from" and "pull" about the --platform option and add integration
tests for them, warning if --platform was given multiple values.
Add a define.BuildOptions.JobSemaphore which an imagebuildah executor
will use in preference to one that it might allocate for itself.
In main(), allocate a JobSemaphore if the number of jobs is not 0 (which
we treat as "unlimited", and continue to allow executors to do).
In addManifest(), take a lock on the manifest list's image ID so that we
don't overwrite changes that another thread might be making while we're
attempting to make changes to it. In main(), create an empty list if
the list doesn't already exist before we start down this path, so that
we don't get two threads trying to create that manifest list at the same
time later on. Two processes could still try to create the same list
twice, but it's an incremental improvement.
Finally, if we've been given multiple platforms to build for, run their
builds concurrently and gather up their results.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Currently the rusage is reported to stdout and rather cumbersome to parse. The
new flag rusage-logfile can be used to specify a file to which the log will be
written instead.
Signed-off-by: Dan Čermák <dcermak@suse.com>
Add support for secrets. Secrets is a two-part flag that allows secret files to
be accessed for a certain RUN instruction, but not any other
instructions, as well as now showing up in the final image.
Signed-off-by: Ashley Cui <acui@redhat.com>
David Shea
Nalin Dahyabhai
tomsweeneyredhat
Aditya R
Daniel J Walsh
Paul Holzinger
Ashley Cui
Leigh McCulloch
Kirill Shirinkin
openshift-ci[bot]
Dan Čermák