Buildah's `--cache-to` and `--cache-from` is different from
docker/buildkit so lets document that and mention how is it different.
Closes: https://github.com/containers/buildah/issues/4612
Signed-off-by: Aditya R <arajan@redhat.com>
Before executing `buildah unshare` for mounting a container from scratch
as a non-root user, we need to export the variable `newcontainer` so
that it is known inside the modified user namespace.
Signed-off-by: Steffen Straub <22006250+Steffen1989@users.noreply.github.com>
In the tutorial, switch to calling `DefaultStoreOptionsAutoDetectUID()`
instead of `DefaultStoreOptions()`, which should figure things out
better.
In the tutorial, add an example of using Run(), where for API backward
compatibility reasons, we can't tell the difference between "grant no
capabilties by default" and "grant the default set of capabilities by
default".
The default isolation can be set automatically, so start doing that at
run-time, but have the tutorial look it up anyway because the tutorial
on the web will be newer than our current release for at least a while.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Buildah uses the default list of capabilities as specified in
containers.conf. The man pages should reflect this.
Fixes: https://github.com/containers/buildah/issues/4576
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
When multiple files are specified buildah considers `FROM` instruction
from the last file specified and so does `buildkit` and `docker` so lets
specify that in docs.
[NO NEW TESTS NEEDED]
[NO TESTS NEEDED]
Closes: https://github.com/containers/buildah/issues/4544
Signed-off-by: Aditya R <arajan@redhat.com>
Touch up a couple of diversity issues in one of the tutorials.
Changes the word "master" to "primary".
[NO NEW TESTS NEEDED]
Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
We handle --label command line arguments by appending LABEL instructions
to the Dockerfile contents before we parse it. Previously, we were
appending a separate line for each label-value pair. Consolidate them
for the sake of tools that arbitrarily limit the length of histories
that they're willing to accept in images (boo!).
Add a similar implementation for --env command line arguments.
Previously, we'd set them in the initial configuration for each stage
and also set them at commit-time, and that potentially overrode any
values that were explicitly in the stage itself, and which would have
affected RUN instructions. Remove the set-at-commit-time logic so that
the history reflects what ends up in the image.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
* Just like buildkit buildah must allow cleaning the buildcache and cache
generated on host by `--mount=type=cache` just like buildkit's `prune`
command.
* Also expose `CleanCacheMount` API so other tools like `podman` can use
it.
See: https://github.com/moby/buildkit#cache
Closes: https://github.com/containers/buildah/issues/4486
Signed-off-by: Aditya R <arajan@redhat.com>
Allow containers running under buildah to use
--group-add keep-groups, so that they can inherit
access to the users groups.
Also allow users to add supplimental groups to the container.
Fixes: https://github.com/containers/buildah/issues/4476
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Buildah must support multiple sources for remote cache when using
`--cache-to` and `--cache-from` so users can distribute and collect
cache from various sources.
We also need for compat with buildkit api which in past broke `podman`
compat build API see: https://github.com/containers/podman/pull/16380
More discussion here: https://github.com/containers/podman/pull/16380
Signed-off-by: Aditya R <arajan@redhat.com>
In multi-stage builds buildah will skip stages which are unused (i.e
stages which don't contribute anything to target stage directly or
indirectly) however in certain cases users need to process these unused
stages hence add support for `--skip-unused-stages` which allows users
to control this behaviour.
Ref: https://github.com/GoogleContainerTools/kaniko#flag---skip-unused-stages
Closes: https://github.com/containers/buildah/issues/4243
Signed-off-by: Aditya R <arajan@redhat.com>
If user explicitly specified `--cache-ttl=0s
it would effectively mean that user is asking
to use no cache at all. In such use cases
buildah can skip looking for cache entierly
by setting `--no-cache=true` internally.
Closes: https://github.com/containers/buildah/issues/4244
Signed-off-by: Aditya R <arajan@redhat.com>
Allows users to configure `--retry` attempts and `--retry-delay`
duration using two additional flags for commands
* buildah build
* buildah pull
* buildah push
* buildah from
* buildah add
* buildah copy
Closes: https://github.com/containers/buildah/issues/4018
Not sure how we can test retry attempts in CI, but added a test in
tests/bud.bats which verfies we parse and added flag in other options
for sanity parsing checking.
Signed-off-by: Aditya R <arajan@redhat.com>
`--mount=type=cache` is buildah's internal construct and actual location
is not managed by user so enable `z` by default is `SELinux` is enabled
on the host machine, instead of asking users to do it.
Signed-off-by: Aditya R <arajan@redhat.com>
When `buildah manifest create` is given the `--amend` flag and a list
with the name that was specified for the to-be-created list already
exists, just reuse the list.
Make the `--insecure` flag, if we see it, conflict with `--tls-verify`,
but have the reverse of the effect that `--tls-verify` does, and teach
the `buildah manifest` `add`, `create`, and `push` subcommands about
it.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
`build` or `bud` now supports a new flag `--cache-ttl` which accepts
duration and allows end users to ignore cache images which are not under
the specified duration.
Following flag is useful for setups/platforms which heavily relies on
`--layer` and buildah caching but want recompute certain `RUN` steps
after specified duration to make sure specific `RUN` steps are always
updated. Example `RUN dnf update` or `RUN dnf install`
Closes: https://github.com/containers/buildah/issues/4160
Somewhat similar to kaniko's: https://github.com/GoogleContainerTools/kaniko#--cache-ttl-duration
Signed-off-by: Aditya R <arajan@redhat.com>
When using URL to a git repo as build context allow specifying `commit`
to make sure specific `commit` is used as the build context.
Examples
```console
buildah build -t test myrepo.git#abcef123
buildah build -t test myrepo.git#abcef123
buildah build -t test myrepo.git#abcef123:somesubdir
```
Closes: https://github.com/containers/buildah/issues/4148
Signed-off-by: Aditya R <arajan@redhat.com>
Following commit
* Initiates `cacheKey` or `layerKey` for intermediate images generated
for layers.
* Allows end users to upload cached layers with `cacheKey` to remote
sources using `--cache-to`. `--cache-to` is a optional flag to be used
with `buildah build` which publishes cached layers to remote sources.
* Allows end users to use cached layers from `remote` sources with
`--cache-from`. `--cache-from` is a optional flag to be used with
`buildah build` and it pulls cached layers from remote sources in a step
by step manner only if is a valid cache hit.
Example
* Populate cache source or use cached layers if already present
```bash
buildah build -t test --layers --cache-to registry/myrepo/cache --cache-from registry/myrepo/cache .
```
Future:
* `cacheKey` or `layerKey` model is only being used when working with
remote sources however local cache lookup can be also optimized if its
is altered to use `cacheKey` model instead of iterating through all the
images in local storage. As discussed here
References:
* Feature is quite similar to `kaniko`'s `--cache-repo`: https://github.com/GoogleContainerTools/kaniko#--cache-repo
Closes: issues#620
Signed-off-by: Aditya R <arajan@redhat.com>
Going through containers/podman/issues/14846 let me improve the
code documentation in containers/common/pull/1085 which is now
copied over to Buildah and will eventually into Podman.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Following PR adds support for OCI hooks attached to each of ephemeral
build contains which are created by `RUN` step invoked during `buildah
build` command.
Following PR also adds `--hooks-dir` flag to `buildah build` option
which allows end-users to specify one or more configuration directories
for OCI-hooks.
Note: Following PR migrates `pkg/hooks` from `podman` to `buildah` so
buildah could implement this feature and once merged `podman` can start
using `buildah/pkg/hooks`.
For usage see man-page and example test-case.
See: https://github.com/opencontainers/runtime-spec/blob/main/config.md#posix-platform-hooks
Closes: https://github.com/containers/buildah/issues/4068
Signed-off-by: Aditya R <arajan@redhat.com>
Buildah now supports `--userns=auto` which can automatically pick an empty range and create an user namespace for the container.
The `--userns=auto` flag, requires that the user name containers and a range of subordinate user ids that the build container is allowed to use be specified in the /etc/subuid and /etc/subgid files.
Example: `containers:2147483647:2147483648`.
Buildah allocates unique ranges of UIDs and GIDs from the containers subordinate user ids. The size of the ranges is based on the number of UIDs required in the image. The number of UIDs and GIDs can be overridden with the size option.
Valid auto options:
* gidmapping=CONTAINER_GID:HOST_GID:SIZE: to force a GID mapping to be present in the user namespace.
* size=SIZE: to specify an explicit size for the automatic user namespace. e.g. --userns=auto:size=8192. If size is not specified, auto will estimate a size for the user namespace.
* uidmapping=CONTAINER_UID:HOST_UID:SIZE: to force a UID mapping to be present in the user namespace.
Signed-off-by: Aditya R <arajan@redhat.com>
Support splitting build logs for multi-arch builds ( `--platform` ) by
using `--logfile` and `--logsplit` which splits generated log file in the
convention `${file}_${platform_os}_${platform_arch}`.
Closes: https://github.com/containers/buildah/issues/3960
Signed-off-by: Aditya R <arajan@redhat.com>
As per `OCI` `image-spec` setting history is optional so buildah must
support these use-cases via `--omit-history=true` some of the use-cases
are discussed below.
* There are use-cases when image-spec contributes to the image size by
adding build history to image-spec in certain use-cases this image
history is not required by the end-users but there is no way to remove
it. See: https://github.com/containers/buildah/issues/3513
* Certain build tools don't honor setting build history in image spec
however buildah does not allow processing images without build history
since setting build history is still optional as per `OCI` image-spec
( https://github.com/opencontainers/image-spec/blob/main/config.md#properties)
so buildah must support such use-cases.
Closes: https://github.com/containers/buildah/issues/4025
Closes: https://github.com/containers/buildah/issues/3513
Signed-off-by: Aditya R <arajan@redhat.com>
As builds got more complicated, the ability to only access files from one location became quite limiting. With `multi-stage` builds where you can `copy` files from other parts of the Containerfile by adding the `--from` flag and pointing it to the name of another Containerfile stage or a remote image.
The new named build context feature is an extension of this pattern. You can now define additional build contexts when running the build command, give them a name, and then access them inside a Dockerfile the same way you previously did with build stages.
Additional build contexts can be defined with a new `--build-context [name]=[value]` flag. The key component defines the name for your build context and the value can be:
```console
Local directory – e.g. --build-context project2=../path/to/project2/src
HTTP URL to a tarball – e.g. --build-context src=https://example.org/releases/src.tar
Container image – Define with a docker-image:// prefix, e.g. --build-context alpine=docker-image://alpine:3.15, ( also supports docker://, container-image:// )
```
On the Containerfile side, you can reference the build context on all commands that accept the “from” parameter. Here’s how that might look:
```Dockerfile
FROM [name]
COPY --from=[name] ...
RUN --mount=from=[name] …
```
The value of [name] is matched with the following priority order:
* Named build context defined with `--build-context [name]=..`
* Stage defined with `AS [name]` inside Dockerfile
* Remote image `[name]` in a container registry
Added Features
* Pinning images for `FROM` and `COPY`
* Specifying multiple buildcontexts from different projects
and using them with `--from` in `ADD` and `COPY` directive
* Override a Remote Dependency with a Local One.
* Using additional context from external `Tar`
Signed-off-by: Aditya R <arajan@redhat.com>
Render the progress bar on stdout by default. Add a --quiet/-q option
to re-silence if desired.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
When using `build -t <tag> <some-url-to-git-repo>` buildah does not
accepts syntax `url#branch:subdir` following commit allows end users to
switch context to any subdirectory in the git repo.
Usage
```console
buildah build -t test 'https://github.com/containers/podman.git#main:contrib/hello'
```
Signed-off-by: Aditya R <arajan@redhat.com>
Restore podman's CLI compatibility with `docker build` by making its
`--label` and `--annotate` options create empty labels and annotations
when they're given a name but no `=` or value.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Make more of our usage information in --help output consistent with
itself, i.e., initial lower case.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Teach `buildah build` and `buildah config` about the OS version and
features fields. We don't tend to use them on Linux, but they're in the
OCI and Docker config structures, so we need to be able to preserve and
manipulate their values, much as we do for variant information.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Have the handling of the `--annotation` and `--label` options for
`buildah build` explicitly remove values when they're not given an "="
before an empty value, instead of setting them to empty values. The
final result doesn't seem to care about the distinction.
Allow the value of the `io.buildah.version` label to be manually
specified on the `buildah build` command line if someone really wants to
override the value that we would otherwise set by default.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add a `--env` option to `buildah build` that functions similarly to the
`buildah config --env` option, to complement `buildah build`'s
`--unsetenv` option.
Document that `buildah config`'s `--env` function fetches the current
value for a variable when the name is supplied, but no `=` or value
follows it.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Allows end-users to export final build content or rootfs to external formats.
By default, a local container image is created from the build result. The --output (or -o) flag allows you to override this behavior, and a specify a custom exporter. For example, custom exporters allow you to export the build artifacts as files on the local filesystem instead of a Container image, which can be useful for generating local binaries, code generation etc.
The value for --output is a CSV-formatted string defining the exporter type and options. Currently, local and tar exporters are supported. The local exporter writes the resulting build files to a directory on the client side. The tar exporter is similar but writes the files as a single tarball (.tar).
```console
buildah build --output type=local,dest=dir .
buildah build --output type=tar,dest=rootfs.tar .
buildah build -o dir .
```
Reference: https://docs.docker.com/engine/reference/commandline/build/#custom-build-outputs
Signed-off-by: Aditya R <arajan@redhat.com>
The sed implementation has a strict interpretation of posix 'basic'
regular expressions. It would be better to re-implement this using
'extended' regular expressions but for now, just use GNU sed.
Signed-off-by: Doug Rabson <dfr@rabson.org>
Allow end users to remove default identity labels if they want to.
Since there are instances where images can be reproduced across version
hence users must have option to suppress default labels.
Closes: https://github.com/containers/buildah/issues/3826
Signed-off-by: Aditya R <arajan@redhat.com>
Enforce alphabetical ordering of command-line options in
man pages. Not as simple as with podman, because conventions
are different.
Reference: https://github.com/containers/podman/pull/13625
Signed-off-by: Ed Santiago <santiago@redhat.com>
Users want to turn off addition of /etc/hosts file while building
container images, this would allow them to customize the /etc/hosts
file within the image.
Fixes: https://github.com/containers/buildah/issues/3808
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Aditya R
Steffen Straub
Nalin Dahyabhai
Daniel J Walsh
tomsweeneyredhat
Daniel Celentano
Mattias Warnqvist
Valentin Rothberg
Lokesh Mandvekar
OpenShift Merge Robot
Doug Rabson
Ed Santiago