Make the traditional handling of volumes (where they're "frozen" and can
only be modified by ADD or COPY, which requires that we cache their
contents and save/restore them before/after RUN instructions) an option
that is not enabled by default.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
This is causing the buildah build --help line to output a strange string
--pull string[="true"] Should be
--pull string[="missing"]
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
buildah from and buildah build will now default to --pull=missing
as opposed to --pull=always, which they did before. This better
matches to the defaults in docker and podman.
No longer document --pull=true|false
Fixes: https://github.com/containers/buildah/issues/5406
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Add support for using CDI to resolve `--device` devices for RUN
instructions during `buildah build`, `buildah from`, and `buildah run`,
as `podman run` does.
This generally requires that we stop resolving device specifications
(arguments passed to --device) earlier and deferring that until it's
time to run a process, because CDI wants to pick over those values,
modify a runtime spec to set up the ones that it knows about, and then
hand back the list of values that it doesn't know about.
We don't want to do a dry run of this during CLI processing because that
would create a window where the underlying hardware state could change,
and that could produce some hard-to-diagnose errors.
Being able to test this is going to require that we add the `--device`
flag to `buildah run` (`--security-opt` affects how we build the
container's layer, so it has to be done at `buildah from`).
The default configured devices list is pulled in by CLI flag processing
during `buildah from` and `buildah build`, so it doesn't also need to be
explicitly passed to `buildah run` or the internal Run() method.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
... in an attempt to try to get UID 0 in a user namespace to stop trying
to read files from root's home directory, where the permissions error is
treated as a hard failure.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add a --sbom flag to `buildah build` and `buildah commit` which will
scan the rootfs and specified context directories to build SPDX or
CycloneDX SBOMs and lists of package URLs.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Tweak the wording for the descriptions of --squash and --layers. Make
it clearer that the variant is optional in --platform, and mark the
syntax of the argument instead of a possible value for use in the help
output.
[NO NEW TESTS NEEDED]
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
All `[]string`s in containers.conf have now been migrated to attributed
string slices which require some adjustments in Buildah and Podman.
[NO NEW TESTS NEEDED]
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
This change is the first step of integrating appendable string arrays
into containers.conf and starts with enabling the `Env`, `Mounts`, and
`Volumes` fields in the `[Containers]` table.
Both, Buildah and Podman, read (and sometimes write) the fields of the
`Config` struct at various places, so I decided to migrate the fields
step-by-step. The ones in this change are most critical ones for
customers. Once all string slices/arrays are migrated, the docs of
containers.conf will be updated. The current changes are entirely
transparent to users.
[NO NEW TESTS NEEDED]
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Add a --cw option to `buildah build` and `buildah commit`, which takes a
comma-separated list of arguments and produces an image laid out for use
as a confidential workload:
type: sev or snp
attestation_url: location of a key broker server
cpus: expected number of virtual CPUs to run with
memory: expected megabytes of memory to run with
workload_id: a distinguishing identifier for the key broker server
ignore_attestation_errors: ignore errors registering the workload
passphrase: for encrypting the disk image
slop: extra space to allocate for the disk image
At least one of attestation_url and passphrase must be specified in
order for the encrypted disk image to be decryptable at run-time. Other
arguments can be omitted. ignore_attestation_errors is intentionally
undocumented, as it's mainly used to permit some amount of testing on
systems which don't have the required hardware.
Add an `mkcw` top-level command, for converting directly from an image
to a confidential workload.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
`--layer-label` allows users to set labels on intermediate labels agnostic
of the labels set on actual image. Since there are use-cases where users
want to perform operation on intermediate images only on the basis of
certain labels.
Closes: https://github.com/containers/buildah/issues/4933
Signed-off-by: Aditya R <arajan@redhat.com>
We want to share these functions with Podman, Podman currently
has a slightly different version which is correct, so use correct
version in Buildah and vendor it into Podman.
Fixing: https://github.com/containers/podman/issues/18196
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Allows codifying build arguments into a structured file, perhaps one
named arguments.Containerargs
The build arg file also accepts comments starting #, so automated
tooling or CI/CD workflows can monitor arguments like versions to
ensure they are up-to-date.
Signed-off-by: Jack Wearden <jack@jackwearden.co.uk>
Allow containers running under buildah to use
--group-add keep-groups, so that they can inherit
access to the users groups.
Also allow users to add supplimental groups to the container.
Fixes: https://github.com/containers/buildah/issues/4476
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Buildah must support multiple sources for remote cache when using
`--cache-to` and `--cache-from` so users can distribute and collect
cache from various sources.
We also need for compat with buildkit api which in past broke `podman`
compat build API see: https://github.com/containers/podman/pull/16380
More discussion here: https://github.com/containers/podman/pull/16380
Signed-off-by: Aditya R <arajan@redhat.com>
PR #4249 added --skip-unused-stages (cool) but also added
what seems to be an unnecessary command-completion incantation.
This incantation breaks podman unit tests when vendored in
the treadmill.
[NO NEW TESTS NEEDED]
Signed-off-by: Ed Santiago <santiago@redhat.com>
In multi-stage builds buildah will skip stages which are unused (i.e
stages which don't contribute anything to target stage directly or
indirectly) however in certain cases users need to process these unused
stages hence add support for `--skip-unused-stages` which allows users
to control this behaviour.
Ref: https://github.com/GoogleContainerTools/kaniko#flag---skip-unused-stages
Closes: https://github.com/containers/buildah/issues/4243
Signed-off-by: Aditya R <arajan@redhat.com>
Allows users to configure `--retry` attempts and `--retry-delay`
duration using two additional flags for commands
* buildah build
* buildah pull
* buildah push
* buildah from
* buildah add
* buildah copy
Closes: https://github.com/containers/buildah/issues/4018
Not sure how we can test retry attempts in CI, but added a test in
tests/bud.bats which verfies we parse and added flag in other options
for sanity parsing checking.
Signed-off-by: Aditya R <arajan@redhat.com>
When `buildah manifest create` is given the `--amend` flag and a list
with the name that was specified for the to-be-created list already
exists, just reuse the list.
Make the `--insecure` flag, if we see it, conflict with `--tls-verify`,
but have the reverse of the effect that `--tls-verify` does, and teach
the `buildah manifest` `add`, `create`, and `push` subcommands about
it.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
`build` or `bud` now supports a new flag `--cache-ttl` which accepts
duration and allows end users to ignore cache images which are not under
the specified duration.
Following flag is useful for setups/platforms which heavily relies on
`--layer` and buildah caching but want recompute certain `RUN` steps
after specified duration to make sure specific `RUN` steps are always
updated. Example `RUN dnf update` or `RUN dnf install`
Closes: https://github.com/containers/buildah/issues/4160
Somewhat similar to kaniko's: https://github.com/GoogleContainerTools/kaniko#--cache-ttl-duration
Signed-off-by: Aditya R <arajan@redhat.com>
Following commit
* Initiates `cacheKey` or `layerKey` for intermediate images generated
for layers.
* Allows end users to upload cached layers with `cacheKey` to remote
sources using `--cache-to`. `--cache-to` is a optional flag to be used
with `buildah build` which publishes cached layers to remote sources.
* Allows end users to use cached layers from `remote` sources with
`--cache-from`. `--cache-from` is a optional flag to be used with
`buildah build` and it pulls cached layers from remote sources in a step
by step manner only if is a valid cache hit.
Example
* Populate cache source or use cached layers if already present
```bash
buildah build -t test --layers --cache-to registry/myrepo/cache --cache-from registry/myrepo/cache .
```
Future:
* `cacheKey` or `layerKey` model is only being used when working with
remote sources however local cache lookup can be also optimized if its
is altered to use `cacheKey` model instead of iterating through all the
images in local storage. As discussed here
References:
* Feature is quite similar to `kaniko`'s `--cache-repo`: https://github.com/GoogleContainerTools/kaniko#--cache-repo
Closes: issues#620
Signed-off-by: Aditya R <arajan@redhat.com>
We now use the golang error wrapping format specifier `%w` instead of
the deprecated github.com/pkg/errors package.
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
Following PR adds support for OCI hooks attached to each of ephemeral
build contains which are created by `RUN` step invoked during `buildah
build` command.
Following PR also adds `--hooks-dir` flag to `buildah build` option
which allows end-users to specify one or more configuration directories
for OCI-hooks.
Note: Following PR migrates `pkg/hooks` from `podman` to `buildah` so
buildah could implement this feature and once merged `podman` can start
using `buildah/pkg/hooks`.
For usage see man-page and example test-case.
See: https://github.com/opencontainers/runtime-spec/blob/main/config.md#posix-platform-hooks
Closes: https://github.com/containers/buildah/issues/4068
Signed-off-by: Aditya R <arajan@redhat.com>
Support splitting build logs for multi-arch builds ( `--platform` ) by
using `--logfile` and `--logsplit` which splits generated log file in the
convention `${file}_${platform_os}_${platform_arch}`.
Closes: https://github.com/containers/buildah/issues/3960
Signed-off-by: Aditya R <arajan@redhat.com>
As per `OCI` `image-spec` setting history is optional so buildah must
support these use-cases via `--omit-history=true` some of the use-cases
are discussed below.
* There are use-cases when image-spec contributes to the image size by
adding build history to image-spec in certain use-cases this image
history is not required by the end-users but there is no way to remove
it. See: https://github.com/containers/buildah/issues/3513
* Certain build tools don't honor setting build history in image spec
however buildah does not allow processing images without build history
since setting build history is still optional as per `OCI` image-spec
( https://github.com/opencontainers/image-spec/blob/main/config.md#properties)
so buildah must support such use-cases.
Closes: https://github.com/containers/buildah/issues/4025
Closes: https://github.com/containers/buildah/issues/3513
Signed-off-by: Aditya R <arajan@redhat.com>
As builds got more complicated, the ability to only access files from one location became quite limiting. With `multi-stage` builds where you can `copy` files from other parts of the Containerfile by adding the `--from` flag and pointing it to the name of another Containerfile stage or a remote image.
The new named build context feature is an extension of this pattern. You can now define additional build contexts when running the build command, give them a name, and then access them inside a Dockerfile the same way you previously did with build stages.
Additional build contexts can be defined with a new `--build-context [name]=[value]` flag. The key component defines the name for your build context and the value can be:
```console
Local directory – e.g. --build-context project2=../path/to/project2/src
HTTP URL to a tarball – e.g. --build-context src=https://example.org/releases/src.tar
Container image – Define with a docker-image:// prefix, e.g. --build-context alpine=docker-image://alpine:3.15, ( also supports docker://, container-image:// )
```
On the Containerfile side, you can reference the build context on all commands that accept the “from” parameter. Here’s how that might look:
```Dockerfile
FROM [name]
COPY --from=[name] ...
RUN --mount=from=[name] …
```
The value of [name] is matched with the following priority order:
* Named build context defined with `--build-context [name]=..`
* Stage defined with `AS [name]` inside Dockerfile
* Remote image `[name]` in a container registry
Added Features
* Pinning images for `FROM` and `COPY`
* Specifying multiple buildcontexts from different projects
and using them with `--from` in `ADD` and `COPY` directive
* Override a Remote Dependency with a Local One.
* Using additional context from external `Tar`
Signed-off-by: Aditya R <arajan@redhat.com>
Make more of our usage information in --help output consistent with
itself, i.e., initial lower case.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Teach `buildah build` and `buildah config` about the OS version and
features fields. We don't tend to use them on Linux, but they're in the
OCI and Docker config structures, so we need to be able to preserve and
manipulate their values, much as we do for variant information.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add a `--env` option to `buildah build` that functions similarly to the
`buildah config --env` option, to complement `buildah build`'s
`--unsetenv` option.
Document that `buildah config`'s `--env` function fetches the current
value for a variable when the name is supplied, but no `=` or value
follows it.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Allows end-users to export final build content or rootfs to external formats.
By default, a local container image is created from the build result. The --output (or -o) flag allows you to override this behavior, and a specify a custom exporter. For example, custom exporters allow you to export the build artifacts as files on the local filesystem instead of a Container image, which can be useful for generating local binaries, code generation etc.
The value for --output is a CSV-formatted string defining the exporter type and options. Currently, local and tar exporters are supported. The local exporter writes the resulting build files to a directory on the client side. The tar exporter is similar but writes the files as a single tarball (.tar).
```console
buildah build --output type=local,dest=dir .
buildah build --output type=tar,dest=rootfs.tar .
buildah build -o dir .
```
Reference: https://docs.docker.com/engine/reference/commandline/build/#custom-build-outputs
Signed-off-by: Aditya R <arajan@redhat.com>
Allow end users to remove default identity labels if they want to.
Since there are instances where images can be reproduced across version
hence users must have option to suppress default labels.
Closes: https://github.com/containers/buildah/issues/3826
Signed-off-by: Aditya R <arajan@redhat.com>
Users want to turn off addition of /etc/hosts file while building
container images, this would allow them to customize the /etc/hosts
file within the image.
Fixes: https://github.com/containers/buildah/issues/3808
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Per @edsantiago 's suggestion, make the pull commands consistent, always
accepting a value. Currently we have:
--pull
--pull=true
--pull=false
--pull-never
--pull-always
With this changes, we will only have pull with a variety of options,
ala:
--pull
--pull=true
--pull=false
--pull=never
--pull=always
For backward compatibility, the --pull-never and --pull-always
options will remain operational, however they are not documented
and are conisdered deprecated.
Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
We need to use the default from containers.conf and not hardcode them in
buildah. This fixes an issue with the cni network backend since it would
try to access /etc/cni/net.d/ even as rootless user. This regression was
introduced in commit f9cff07b81.
Also hide the cni flags as we do not expect users to change this. The
recommended way is to change them in containers.conf.
[NO NEW TESTS NEEDED]
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Add a --all-platforms that instructs the builder to build for the
intersection of all platforms for which the build's base images are
available. Returns an error if any of them aren't references to
manifest lists. We've learned that we can't really trust architecture
and OS information stored in image config blobs, so we don't try to
salvage that case.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When we're told to add built images to a manifest list, manipulate the
list ourselves, so that if we're creating a list, we won't have a
partially-populated list if some of the builds fail.
This also lets us include all of the platform information (including
variant info, which we can't sniff out after the fact) that we were
given when we started building the images.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Allow ssh socket from host or certain ssh keys to be exposed to a
certain RUN instruction, but not any other instructions, as well as not
showing up in the final image.
This is done by spawining a new agent from buildah and mounting
the listening socket inside the run. SSH_AUTH_SOCK inside the container
will be set to the socket mountpoint. The defualt mountpoint is
/run/buildkit/ssh_agent.{i}
Signed-off-by: Ashley Cui <acui@redhat.com>
Nalin Dahyabhai
Daniel J Walsh
Valentin Rothberg
Aditya R
Chris Bandy
Jack Wearden
Ed Santiago
Sascha Grunert
tomsweeneyredhat
Kir Kolyshkin
Paul Holzinger
Kirill Shirinkin
openshift-ci[bot]
Ashley Cui