Following commit adds buildkit like support for `from` field to `--mount=type=bind`
and `--mount=type=cache` so images and stage can be used as mount source.
Usage looks like
```dockerfile
RUN --mount=type=bind,source=.,from=<your-image>,target=/path ls /path
```
and
```dockerfile
RUN --mount=type=cache,from=<your-image>,target=/path ls /path
```
Signed-off-by: Aditya Rajan <arajan@redhat.com>
Buildah is mainly building with Host Namespaces, this changes most
namespaces to be private matching Podman and using containers.conf
Fixes: https://github.com/containers/buildah/issues/3634
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
NetworkPolicy should be set as NetworkEnabled when expliclty specified using
`--network` instead of silently marking it as NetworkDefault.
NetworkDefault is equivalent to NetworkEnabled but we can use
NetworkEnabled to verify if user explicitly used `--network` and prevent
override from builder configs.
Signed-off-by: Aditya Rajan <arajan@redhat.com>
Run secrets can now be created from an environment variable. The
environment variable is read and is briefly stored as a file on /dev/shm
when it's being used, and the file is removed after the RUN command is
finished.
Fixes: #3524
Signed-off-by: Ashley Cui <acui@redhat.com>
In linux, directory can contains colon.
Add support to mount path contains colon.
buildah run --volume /root/a\\🅱️/root/test:O
Signed-off-by: chenk008 <kongchen28@gmail.com>
Users have accidently had a .containerignore or .dockerignore
file in their context directly which blocked all content.
Currently we report that no globs matched, but do not
identify where the globs came from.
This change is an attempt to add this data to the error.
Example: https://github.com/containers/buildah/issues/3318
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Following PR inroduces a new mount type=cache in parity to buildkit
which allows users to share persistant cache between different builds.
Allowing users to cache content generated by bussiness logic or enhance
build performance by caching components across builds.
Signed-off-by: Aditya Rajan <arajan@redhat.com>
Following commit adds support for using buildkit like
`--mount=type=bind` with `RUN` statements. Mounts created by `--mount`
are transient in nature and only scoped to current RUN statements.
Signed-off-by: Aditya Rajan <arajan@redhat.com>
Add a --all-platforms that instructs the builder to build for the
intersection of all platforms for which the build's base images are
available. Returns an error if any of them aren't references to
manifest lists. We've learned that we can't really trust architecture
and OS information stored in image config blobs, so we don't try to
salvage that case.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
The golang.org/x/crypto/ssh/terminal package has been deprecated and
replaced upstream by golang.org/x/term, so switch to that. It's a
simple 1:1 replacement.
[NO NEW TESTS NEEDED]
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Allow ssh socket from host or certain ssh keys to be exposed to a
certain RUN instruction, but not any other instructions, as well as not
showing up in the final image.
This is done by spawining a new agent from buildah and mounting
the listening socket inside the run. SSH_AUTH_SOCK inside the container
will be set to the socket mountpoint. The defualt mountpoint is
/run/buildkit/ssh_agent.{i}
Signed-off-by: Ashley Cui <acui@redhat.com>
Add a pkg/parse.PlatformsFromOptions() which understands a "variant"
value as an optional third value in an OS/ARCH[/VARIANT] argument value,
which accepts a comma-separated list of them, and which returns a list
of platforms.
Teach "from" and "pull" about the --platform option and add integration
tests for them, warning if --platform was given multiple values.
Add a define.BuildOptions.JobSemaphore which an imagebuildah executor
will use in preference to one that it might allocate for itself.
In main(), allocate a JobSemaphore if the number of jobs is not 0 (which
we treat as "unlimited", and continue to allow executors to do).
In addManifest(), take a lock on the manifest list's image ID so that we
don't overwrite changes that another thread might be making while we're
attempting to make changes to it. In main(), create an empty list if
the list doesn't already exist before we start down this path, so that
we don't get two threads trying to create that manifest list at the same
time later on. Two processes could still try to create the same list
twice, but it's an incremental improvement.
Finally, if we've been given multiple platforms to build for, run their
builds concurrently and gather up their results.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add new `buildah source {create,add,push,pull}` commands. All commands
are marked as experimental. None of it is meant to be officially
supported at the time of writing. All code resides in `internal/source`
and is hence not visible to external consumers of Buildah; just to be
on the safe side.
A source container or source image is an OCI artifact, that is an OCI
image with custom config (media type). There is a longer history behind
source images which are intended to ship the source artifacts of an
ordinary "executable" container image. Until now, source images at
Red Hat are built with github.com/containers/BuildSourceImage. We had a
growing desire (and always the long-term plan) to eventually replace
BuildSurceImage with something else, in this case Buildah.
This commit adds the initial base functionality along with tests to make
sure we're not regressing. The new commands do the following:
* `create` - creates an empty and initialized source image
* `add` - tar up a local path and add it as a layer to the souce image
* `push/pull` - intentionally separate commands from `buildah push/pull`
to allow for an easier usage and prevent the
implementations from undesired (future) interference
Further note: also vendor in c/image@master which ships a required fix.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
At some point code from buildah/pkg/parse we copied into
containers/common/pkg/parse. Buildah should use the new code
now, so that we only have to make changes in one place.
[NO TESTS NEEDED] Since there is no new code added.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Add support for secrets. Secrets is a two-part flag that allows secret files to
be accessed for a certain RUN instruction, but not any other
instructions, as well as now showing up in the final image.
Signed-off-by: Ashley Cui <acui@redhat.com>
Move all code related handling container image over to the new
`libimage` package in containers/common. The new package is an
attempt to consolidate the code across the containers tools under the
github.com/containers umbrella.
The new `libimage` packages provides functionality to perform all kinds
of operations for managing images such as local lookups, pushing,
pulling, listing, removing, etc.
The following packages have been moved over the containers/common:
`manifests` -> `common/image/manifests`
`pkg/manifests` -> `common/pkg/manifests`
`pkg/supplemented` -> `common/pkg/supplemented`
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Golangs os.* functions return the name of the file/directory they
fail to use. We should not wrap these errors with the file/directory
to use names, causes stuttering when the user sees the errors, and looks
bad having huge error messages.
Since this is just code cleanup, existing tests should handle the
changes.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
When we parse isolation we expect users to input chroot, oci, rootless.
So when we translate the constants back to strings, we should use the
same values.
These human names need to be passed over the podman-remote build
bindings, so we need to make them match.
Also docker describes an isolation of "default", which we should also
handle for potential scripts.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
We want to shrink the size of the import when importing pkg from
buildah. This should help us shrink the size of the golang bindings
in podman.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1900822
Currently we are not passing the TMPDIR to the destination context
so when copying to containers storage we always fall back to /var/tmp.
This change sets the destination to match the source, so we can better
handle the storage.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Remove --override-os and --override-arch flags.
Also use --platform option if specified when generating the SystemContext.
Conflict --platform option with --os, --arch and --variant options.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Currently if you give a bogus --network flag,
buildah just ignores it and continues on.
This change will throw proper errors.
Add tests to make sure it works correctly.
Also remove some stuttering to clean up error messages.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
When we carried over the linting configuration from podman, we carried
over the list of checkers that were disabled for podman's sake, even
ones that don't complain about the code in this repository.
Make trivial changes to make gosimple happy.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Change calls to Split to SplitN when we are splitting on an "=" sign to make sure
we're not over chopping what the caller requested.
I started this search after seeing https://github.com/containers/podman/pull/7759.
I'll do similar for Podman, but initial results on my search is order of
magnitudes bigger there.
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
Podman namespaces flags handle the options slightly differently.
Specifically namespace flag `container` is very different.
Replace the `container` option with `private` to better match podman
but continue to support `container`
Similarly Podman expects --network ns:/PATH to allow the indication of a
path to a precreated network namespace. Buildah just supported
--network /PATH, buildah will now support both formats, but document
ns:/path.
Finally use Aliases for fix some duplicated code in buildah.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2443: Allow "readonly" as alias to "ro" in mount options r=rhatdan a=mariushoch
Apparently docker supports this as well, see [Use a read-only volume](https://docs.docker.com/storage/volumes/#use-a-read-only-volume) in the docker docs.
#### What type of PR is this?
/kind bug
#### What this PR does / why we need it:
To be fully compatible with docker in this regard.
#### How to verify it
With a mount option like `type=bind,source=/var/source,target=/src,readonly`.
```release-note
None
```
Co-authored-by: Marius Hoch <hoo@online.de>
In porting containers.conf to libpod, we found that buildah needed
to handle the containers.conf on the server side rather then from
the CLI.
Since the `podman-remote build` would probably not have the same content
as containers.conf on the server, the processesing of the defaults needs
to be handled in imagebuildah. The CapAdd and CapDrop values need to be
passed in.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
When building images we now allow the architecture and operating system
to be overwritten via the new `--os` and `--arch` flags. This allows the
use case of packing pre-built or cross-compiled binaries into container
images with specifying the correct environment. The `--platform` flag
now takes the `os/arch` format into account as well.
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
Closes: #2097
Approved by: rhatdan
This is a rework of Qi Wang's patches.
Import package pkg/config from containers/common to read containers.conf
This patch allows users to specify default values stored in containers.conf
that will modify the behaviour of buildah tool.
Signed-off-by: Qi Wang <qiwan@redhat.com>
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Closes: #2011
Approved by: TomSweeneyRedHat
If the volume option was specified like: `--volume "${HOME}/.cache/zae9ujei:/myvol:rw,Z"`,
the COBRA code that processed the input from the user was considering the value to be a
SliceVar with two volumes `${HOME}/.cache/zae9ujei:/myvol:rw` and `Z` due to the comma.
Converted the COBRA code to consider the input as an ArrayVar instead and the value is
handled appropriately. Also increased the testing to catch this going forward.
Addresses: #2000
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
Closes: #2039
Approved by: rhatdan
Or set it to /var/tmp if the user did not specify.
Currently certain large workloads can not be handled because users are running
out of space on pulls/ and pushes. Containers/image stores data temporarily in
the file system. This allows the user to overide the location of the temporary
storage.
Also update containers/image to v5.0.1
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Thanks to Dmitry Smirnov @onlyjob for suggesting this tool.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Closes: #1985
Approved by: TomSweeneyRedHat
Aditya Rajan
Daniel J Walsh
OpenShift Merge Robot
Ashley Cui
chenk008
Nalin Dahyabhai
openshift-ci[bot]
Valentin Rothberg
Josh Soref
Eduardo Vega
TomSweeneyRedHat
bors[bot]
Marius Hoch
dependabot-preview[bot]
Qi Wang
Brandon Lum
Sascha Grunert