As per `OCI` `image-spec` setting history is optional so buildah must
support these use-cases via `--omit-history=true` some of the use-cases
are discussed below.
* There are use-cases when image-spec contributes to the image size by
adding build history to image-spec in certain use-cases this image
history is not required by the end-users but there is no way to remove
it. See: https://github.com/containers/buildah/issues/3513
* Certain build tools don't honor setting build history in image spec
however buildah does not allow processing images without build history
since setting build history is still optional as per `OCI` image-spec
( https://github.com/opencontainers/image-spec/blob/main/config.md#properties)
so buildah must support such use-cases.
Closes: https://github.com/containers/buildah/issues/4025
Closes: https://github.com/containers/buildah/issues/3513
Signed-off-by: Aditya R <arajan@redhat.com>
As builds got more complicated, the ability to only access files from one location became quite limiting. With `multi-stage` builds where you can `copy` files from other parts of the Containerfile by adding the `--from` flag and pointing it to the name of another Containerfile stage or a remote image.
The new named build context feature is an extension of this pattern. You can now define additional build contexts when running the build command, give them a name, and then access them inside a Dockerfile the same way you previously did with build stages.
Additional build contexts can be defined with a new `--build-context [name]=[value]` flag. The key component defines the name for your build context and the value can be:
```console
Local directory – e.g. --build-context project2=../path/to/project2/src
HTTP URL to a tarball – e.g. --build-context src=https://example.org/releases/src.tar
Container image – Define with a docker-image:// prefix, e.g. --build-context alpine=docker-image://alpine:3.15, ( also supports docker://, container-image:// )
```
On the Containerfile side, you can reference the build context on all commands that accept the “from” parameter. Here’s how that might look:
```Dockerfile
FROM [name]
COPY --from=[name] ...
RUN --mount=from=[name] …
```
The value of [name] is matched with the following priority order:
* Named build context defined with `--build-context [name]=..`
* Stage defined with `AS [name]` inside Dockerfile
* Remote image `[name]` in a container registry
Added Features
* Pinning images for `FROM` and `COPY`
* Specifying multiple buildcontexts from different projects
and using them with `--from` in `ADD` and `COPY` directive
* Override a Remote Dependency with a Local One.
* Using additional context from external `Tar`
Signed-off-by: Aditya R <arajan@redhat.com>
Allows end-users to export final build content or rootfs to external formats.
By default, a local container image is created from the build result. The --output (or -o) flag allows you to override this behavior, and a specify a custom exporter. For example, custom exporters allow you to export the build artifacts as files on the local filesystem instead of a Container image, which can be useful for generating local binaries, code generation etc.
The value for --output is a CSV-formatted string defining the exporter type and options. Currently, local and tar exporters are supported. The local exporter writes the resulting build files to a directory on the client side. The tar exporter is similar but writes the files as a single tarball (.tar).
```console
buildah build --output type=local,dest=dir .
buildah build --output type=tar,dest=rootfs.tar .
buildah build -o dir .
```
Reference: https://docs.docker.com/engine/reference/commandline/build/#custom-build-outputs
Signed-off-by: Aditya R <arajan@redhat.com>
Allow end users to remove default identity labels if they want to.
Since there are instances where images can be reproduced across version
hence users must have option to suppress default labels.
Closes: https://github.com/containers/buildah/issues/3826
Signed-off-by: Aditya R <arajan@redhat.com>
Implementation of `GetVolumes` seems to be changing with addition of new
features hence moving it from exposed parse package to internal parse
package and move needed helpers with it.
Signed-off-by: Aditya R <arajan@redhat.com>
A shared cache on host must support locking so other parallel/concurrent builds
will wait for current executing RUN statement to finish.
* Locks the cache store as soon as RUN is triggered.
* Locked target is added to cleanup list so it can be unlocked as soon
as RUN step is completed.
Signed-off-by: Aditya R <arajan@redhat.com>
Users want to turn off addition of /etc/hosts file while building
container images, this would allow them to customize the /etc/hosts
file within the image.
Fixes: https://github.com/containers/buildah/issues/3808
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
The backend logic already supports specifying custom network names. This
only adds the support for the frontend parsing.
Fixescontainers/podman#12282
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
In situations where you don't want/need Cobra climbing behavior nor
Cobra at all using FlagSet is the easier sell.
Signed-off-by: Andreas Bergmeier <abergmeier@gmx.net>
In situations where you don't want/need Cobra climbing behavior nor
Cobra at all using FlagSet is the easier sell.
Signed-off-by: Andreas Bergmeier <abergmeier@gmx.net>
In situations where you don't want/need Cobra climbing behavior nor
Cobra at all using FlagSet is the easier sell.
Signed-off-by: Andreas Bergmeier <abergmeier@gmx.net>
In situations where you don't want/need Cobra climbing behavior nor
Cobra at all using FlagSet is the easier sell.
Signed-off-by: Andreas Bergmeier <abergmeier@gmx.net>
Following commit adds buildkit like support for `from` field to `--mount=type=bind`
and `--mount=type=cache` so images and stage can be used as mount source.
Usage looks like
```dockerfile
RUN --mount=type=bind,source=.,from=<your-image>,target=/path ls /path
```
and
```dockerfile
RUN --mount=type=cache,from=<your-image>,target=/path ls /path
```
Signed-off-by: Aditya Rajan <arajan@redhat.com>
Buildah is mainly building with Host Namespaces, this changes most
namespaces to be private matching Podman and using containers.conf
Fixes: https://github.com/containers/buildah/issues/3634
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
NetworkPolicy should be set as NetworkEnabled when expliclty specified using
`--network` instead of silently marking it as NetworkDefault.
NetworkDefault is equivalent to NetworkEnabled but we can use
NetworkEnabled to verify if user explicitly used `--network` and prevent
override from builder configs.
Signed-off-by: Aditya Rajan <arajan@redhat.com>
Run secrets can now be created from an environment variable. The
environment variable is read and is briefly stored as a file on /dev/shm
when it's being used, and the file is removed after the RUN command is
finished.
Fixes: #3524
Signed-off-by: Ashley Cui <acui@redhat.com>
In linux, directory can contains colon.
Add support to mount path contains colon.
buildah run --volume /root/a\\🅱️/root/test:O
Signed-off-by: chenk008 <kongchen28@gmail.com>
Users have accidently had a .containerignore or .dockerignore
file in their context directly which blocked all content.
Currently we report that no globs matched, but do not
identify where the globs came from.
This change is an attempt to add this data to the error.
Example: https://github.com/containers/buildah/issues/3318
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Following PR inroduces a new mount type=cache in parity to buildkit
which allows users to share persistant cache between different builds.
Allowing users to cache content generated by bussiness logic or enhance
build performance by caching components across builds.
Signed-off-by: Aditya Rajan <arajan@redhat.com>
Following commit adds support for using buildkit like
`--mount=type=bind` with `RUN` statements. Mounts created by `--mount`
are transient in nature and only scoped to current RUN statements.
Signed-off-by: Aditya Rajan <arajan@redhat.com>
Add a --all-platforms that instructs the builder to build for the
intersection of all platforms for which the build's base images are
available. Returns an error if any of them aren't references to
manifest lists. We've learned that we can't really trust architecture
and OS information stored in image config blobs, so we don't try to
salvage that case.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
The golang.org/x/crypto/ssh/terminal package has been deprecated and
replaced upstream by golang.org/x/term, so switch to that. It's a
simple 1:1 replacement.
[NO NEW TESTS NEEDED]
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Allow ssh socket from host or certain ssh keys to be exposed to a
certain RUN instruction, but not any other instructions, as well as not
showing up in the final image.
This is done by spawining a new agent from buildah and mounting
the listening socket inside the run. SSH_AUTH_SOCK inside the container
will be set to the socket mountpoint. The defualt mountpoint is
/run/buildkit/ssh_agent.{i}
Signed-off-by: Ashley Cui <acui@redhat.com>
Add a pkg/parse.PlatformsFromOptions() which understands a "variant"
value as an optional third value in an OS/ARCH[/VARIANT] argument value,
which accepts a comma-separated list of them, and which returns a list
of platforms.
Teach "from" and "pull" about the --platform option and add integration
tests for them, warning if --platform was given multiple values.
Add a define.BuildOptions.JobSemaphore which an imagebuildah executor
will use in preference to one that it might allocate for itself.
In main(), allocate a JobSemaphore if the number of jobs is not 0 (which
we treat as "unlimited", and continue to allow executors to do).
In addManifest(), take a lock on the manifest list's image ID so that we
don't overwrite changes that another thread might be making while we're
attempting to make changes to it. In main(), create an empty list if
the list doesn't already exist before we start down this path, so that
we don't get two threads trying to create that manifest list at the same
time later on. Two processes could still try to create the same list
twice, but it's an incremental improvement.
Finally, if we've been given multiple platforms to build for, run their
builds concurrently and gather up their results.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add new `buildah source {create,add,push,pull}` commands. All commands
are marked as experimental. None of it is meant to be officially
supported at the time of writing. All code resides in `internal/source`
and is hence not visible to external consumers of Buildah; just to be
on the safe side.
A source container or source image is an OCI artifact, that is an OCI
image with custom config (media type). There is a longer history behind
source images which are intended to ship the source artifacts of an
ordinary "executable" container image. Until now, source images at
Red Hat are built with github.com/containers/BuildSourceImage. We had a
growing desire (and always the long-term plan) to eventually replace
BuildSurceImage with something else, in this case Buildah.
This commit adds the initial base functionality along with tests to make
sure we're not regressing. The new commands do the following:
* `create` - creates an empty and initialized source image
* `add` - tar up a local path and add it as a layer to the souce image
* `push/pull` - intentionally separate commands from `buildah push/pull`
to allow for an easier usage and prevent the
implementations from undesired (future) interference
Further note: also vendor in c/image@master which ships a required fix.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
At some point code from buildah/pkg/parse we copied into
containers/common/pkg/parse. Buildah should use the new code
now, so that we only have to make changes in one place.
[NO TESTS NEEDED] Since there is no new code added.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Add support for secrets. Secrets is a two-part flag that allows secret files to
be accessed for a certain RUN instruction, but not any other
instructions, as well as now showing up in the final image.
Signed-off-by: Ashley Cui <acui@redhat.com>
Move all code related handling container image over to the new
`libimage` package in containers/common. The new package is an
attempt to consolidate the code across the containers tools under the
github.com/containers umbrella.
The new `libimage` packages provides functionality to perform all kinds
of operations for managing images such as local lookups, pushing,
pulling, listing, removing, etc.
The following packages have been moved over the containers/common:
`manifests` -> `common/image/manifests`
`pkg/manifests` -> `common/pkg/manifests`
`pkg/supplemented` -> `common/pkg/supplemented`
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Golangs os.* functions return the name of the file/directory they
fail to use. We should not wrap these errors with the file/directory
to use names, causes stuttering when the user sees the errors, and looks
bad having huge error messages.
Since this is just code cleanup, existing tests should handle the
changes.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
When we parse isolation we expect users to input chroot, oci, rootless.
So when we translate the constants back to strings, we should use the
same values.
These human names need to be passed over the podman-remote build
bindings, so we need to make them match.
Also docker describes an isolation of "default", which we should also
handle for potential scripts.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
We want to shrink the size of the import when importing pkg from
buildah. This should help us shrink the size of the golang bindings
in podman.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1900822
Currently we are not passing the TMPDIR to the destination context
so when copying to containers storage we always fall back to /var/tmp.
This change sets the destination to match the source, so we can better
handle the storage.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Remove --override-os and --override-arch flags.
Also use --platform option if specified when generating the SystemContext.
Conflict --platform option with --os, --arch and --variant options.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Currently if you give a bogus --network flag,
buildah just ignores it and continues on.
This change will throw proper errors.
Add tests to make sure it works correctly.
Also remove some stuttering to clean up error messages.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
When we carried over the linting configuration from podman, we carried
over the list of checkers that were disabled for podman's sake, even
ones that don't complain about the code in this repository.
Make trivial changes to make gosimple happy.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Change calls to Split to SplitN when we are splitting on an "=" sign to make sure
we're not over chopping what the caller requested.
I started this search after seeing https://github.com/containers/podman/pull/7759.
I'll do similar for Podman, but initial results on my search is order of
magnitudes bigger there.
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
Podman namespaces flags handle the options slightly differently.
Specifically namespace flag `container` is very different.
Replace the `container` option with `private` to better match podman
but continue to support `container`
Similarly Podman expects --network ns:/PATH to allow the indication of a
path to a precreated network namespace. Buildah just supported
--network /PATH, buildah will now support both formats, but document
ns:/path.
Finally use Aliases for fix some duplicated code in buildah.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2443: Allow "readonly" as alias to "ro" in mount options r=rhatdan a=mariushoch
Apparently docker supports this as well, see [Use a read-only volume](https://docs.docker.com/storage/volumes/#use-a-read-only-volume) in the docker docs.
#### What type of PR is this?
/kind bug
#### What this PR does / why we need it:
To be fully compatible with docker in this regard.
#### How to verify it
With a mount option like `type=bind,source=/var/source,target=/src,readonly`.
```release-note
None
```
Co-authored-by: Marius Hoch <hoo@online.de>
In porting containers.conf to libpod, we found that buildah needed
to handle the containers.conf on the server side rather then from
the CLI.
Since the `podman-remote build` would probably not have the same content
as containers.conf on the server, the processesing of the defaults needs
to be handled in imagebuildah. The CapAdd and CapDrop values need to be
passed in.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
When building images we now allow the architecture and operating system
to be overwritten via the new `--os` and `--arch` flags. This allows the
use case of packing pre-built or cross-compiled binaries into container
images with specifying the correct environment. The `--platform` flag
now takes the `os/arch` format into account as well.
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
Closes: #2097
Approved by: rhatdan
This is a rework of Qi Wang's patches.
Import package pkg/config from containers/common to read containers.conf
This patch allows users to specify default values stored in containers.conf
that will modify the behaviour of buildah tool.
Signed-off-by: Qi Wang <qiwan@redhat.com>
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Closes: #2011
Approved by: TomSweeneyRedHat
If the volume option was specified like: `--volume "${HOME}/.cache/zae9ujei:/myvol:rw,Z"`,
the COBRA code that processed the input from the user was considering the value to be a
SliceVar with two volumes `${HOME}/.cache/zae9ujei:/myvol:rw` and `Z` due to the comma.
Converted the COBRA code to consider the input as an ArrayVar instead and the value is
handled appropriately. Also increased the testing to catch this going forward.
Addresses: #2000
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
Closes: #2039
Approved by: rhatdan
Or set it to /var/tmp if the user did not specify.
Currently certain large workloads can not be handled because users are running
out of space on pulls/ and pushes. Containers/image stores data temporarily in
the file system. This allows the user to overide the location of the temporary
storage.
Also update containers/image to v5.0.1
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Thanks to Dmitry Smirnov @onlyjob for suggesting this tool.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Closes: #1985
Approved by: TomSweeneyRedHat
This change is required to allow podman for windows to build.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Closes: #1855
Approved by: TomSweeneyRedHat
Some Dockerfiles (fuse-overlay) require additional devices to be in the
build environment.
This patch allows the user to specify additional devices.
Also I noticed that CapAdd and CapDrop was not working in buildah bud situations,
so this patch also fixes this.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Closes: #1820
Approved by: @TomSweeneyRedHat
Also, ensure ordering is correct on other entries. The current
code will increment *after* checking for validity in some cases,
which allows such options as '-v /tmp:/test1:ro,rw' to pass
validation when they obviously don't make much sense.
Signed-off-by: Matthew Heon <matthew.heon@pm.me>
Closes: #1794
Approved by: rhatdan
when running in rootless mode, use fuse-overlayfs for mounting the
overlay file system on the host. Then create a bind mount inside the
container.
Closes: https://github.com/containers/buildah/issues/1741
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Closes: #1743
Approved by: rhatdan
This commit enabled to golint linter in golangci-lint and applies all
necessary fixes.
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
Closes: #1740
Approved by: rhatdan
move ValidateVolumeOpts from libpod to buildah so libpod can vendor it in.
Signed-off-by: Qi Wang <qiwan@redhat.com>
Closes: #1698
Approved by: rhatdan
close#1464
fix the paring typo cpu-shared of --cpu-shares.
fix expect value in from.bats
Signed-off-by: Qi Wang <qiwan@redhat.com>
Closes: #1662
Approved by: TomSweeneyRedHat
Overlay mounts allow buildah bud and buildah from to
specify a directory on the disk that will be mounted
as an overlay into the container, where the overlay can be written to
but when the RUN or buildah run exits, the modified files will dissapear.
The basic idea is to be able to mount cache from the disk for things like yum/dnf/apt
to be able to be used and modified in the contianer on a run command, but to be
kept fresh for each RUN.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Closes: #1560
Approved by: giuseppe
Fix the bug that buildah reads /var/run/containers.. as authentication file.(in readme files it should be /run/containers/uid).
Add `buildah login`, `buildah logout` to close#1329, enable to use buildah login before running buildah bud for multi-stage bud and avoiding using --creds flag.
Signed-off-by: Qi Wang <qiwan@redhat.com>
Closes: #1501
Approved by: rhatdan
buildah bud is ignoring --volumes flag.
This patch parses the volumes and then passes them into the builder to be used.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Closes: #1559
Approved by: vrothberg
the podman remote-client for windows pulls in some buildah code for
things like commit and build. we need to perform some slight
refactoring of buildah code to accomodate that build.
Signed-off-by: baude <bbaude@redhat.com>
Closes: #1551
Approved by: rhatdan
- pass all proxy env vars
- --http-proxy option for bud and from
- bash_completion and docs
Signed-off-by: James Cassell <code@james.cassell.me>
Closes: #1525
Approved by: giuseppe
use --dns* flags to change /etc/resolv.conf in the container during the build.
Signed-off-by: Qi Wang <qiwan@redhat.com>
Closes: #1491
Approved by: rhatdan
Cobra does not allow a "," in a StringSlice, so we should
be using StringArrays. This fixes issues with setting
security-opts level flags.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Closes: #1467
Approved by: TomSweeneyRedHat
if --net is not specified, default to use the host network namespace.
It is still possible to use slirp4netns with --network container.
Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1690209
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Closes: #1421
Approved by: rhatdan
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
Fix the polarity of tls-verify, during the Cobra conversion
it was inadvertently flipped.
At least partially addresses: #1332
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
Closes: #1351
Approved by: rhatdan
Mostly this is a straightforward elimination of manual GetRegistries calls.
In getCopyOptions, we just remove setting the DockerInsecureSkipTLSVerify
values because the docker:// tranport now does that automatically. (This
actually changes behavior, because docker:// supports namespace prefixes
in addition to matching only by hostnames, but that's a superset of the
previous behavior.)
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Closes: #1214
Approved by: rhatdan
To verify the value of the isolation, verify that all strings are the same, not just the beginning of the string.
Avoid users mistakenly thinking that the wrong value entered is also a correct value.
```
➜ buildah git:(isolation-fix) ✗ buildah from --isolation chrootcc cc510acfcd70
fedora-working-container-7
```
After Change:
```
➜ buildah git:(isolation-fix) ✗ ./buildah from --isolation chrootcc cc510acfcd70
unrecognized isolation type "chrootcc"
ERRO[0000] exit status 1
```
Signed-off-by: Zhou Hao <zhouhao@cn.fujitsu.com>
Closes: #1067
Approved by: rhatdan
The image library's copy routine doesn't itself consult the registries
configuration in order to decide whether or not to disable TLS
verification when communicating with a registry, so it's on us to use
the name of a source or destination image to decide whether to set the
flag for that behavior.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1056
Approved by: rhatdan
Add a new Isolation value: IsolationOCIRootless, for which we add a
global --rootless=true flag and a local --no-new-keyring flag when
creating a runtime container, and make some changes to the mounts list,
default namespacing configurations, and supplemental groups list.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #873
Approved by: rhatdan
When using chroot isolation, if we're configured to raise any process
limits above their current values, do so in the grandparent process,
before it transfers execution to a child that it starts in a user
namespace, which won't have the privileges to do so.
The child can still lower resource limits and set limits to the values
that it inherited, so let it continue to do so.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #891
Approved by: rhatdan
We want to set the maximum number of processes and number of open files
to match the Docker defaults.
nofile=1048576:1048576, nproc=1048576:1048576
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Closes: #891
Approved by: rhatdan
Add an IsolationChroot that trades flexibility and isolation for being
able to do what it does in a host environment that's already isolated to
the point where we're not allowed to set up some of that isolation,
producing a result that leans more toward chroot(1) than runc(1) does.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #836
Approved by: rhatdan
Add the basics of handling the "--isolation" option, though at the
moment, the only recognized option is "oci", which is our default.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #822
Approved by: rhatdan
Aditya R
Navid Yaghoobi
Daniel J Walsh
OpenShift Merge Robot
Paul Holzinger
Andreas Bergmeier
Ashley Cui
chenk008
Nalin Dahyabhai
openshift-ci[bot]
Valentin Rothberg
Josh Soref
Eduardo Vega
TomSweeneyRedHat
bors[bot]
Marius Hoch
dependabot-preview[bot]
Qi Wang
Brandon Lum
Sascha Grunert
Miloslav Trmač
Matthew Heon
Giuseppe Scrivano
Jordan Webb
baude
James Cassell
Zhou Hao