When container is created with specific uid and gid also add container
gid to supplementary/additional group.
Signed-off-by: Aditya R <arajan@redhat.com>
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Following PR makes sure that buildah mirrors --authfile to a temporary
file in filesystem if arg is pointing to an FD instead of actual file
as FD can be only consumed once.
Fixes: https://github.com/containers/buildah/issues/3070
Signed-off-by: Aditya Rajan <arajan@redhat.com>
The build with git URL recognizes git URLs by either a git:// scheme prefix, or
any other URL with other schemes such as https:// and a .git suffix.
The build with git URL and branch recognizes the branch name as the fragment
part of the URL, the part after the #.
The .git suffix check is performed by checking that the suffix of the full URL
is .git, however this causes URLs that begin with scheme https://, and have a
.git path suffix, to not be recognized as git URLs when a branch name is
appended.
The logic for checking if the URL path is suffixed with .git should parse the
URL first and consider only the path when looking for the suffix. This change
is implemented in this commit.
This commit also removes the guard within the cloneToDirectory function because
it is unnecessary since every location the function is called the check has
already been carried out.
Signed-off-by: Leigh McCulloch <351529+leighmcculloch@users.noreply.github.com>
When processing a directory tree, only descend into a directory that is
marked for exclusion if its path is literally a prefix of an exception
pattern.
Subtly, but in a way that's compatible with docker, this means that if
we exclude directory "subdir", but we've been told to also include
"**/file" (with an exclusion pattern of "!**/file"), we won't descend
into "subdir" and find a file named "subdir/file", because "**/file"
doesn't start with "subdir/".
More generally, exclusion patterns that start with "!" which include any
wildcards before their final component technically won't be treated
correctly.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Currently buildah images does not have an easy way to get the epoch
creation time. This field is available in `podman images` as the
Created field. Adding to buildah images to make it consistent.
Fixes: https://github.com/containers/buildah/issues/3478
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
When we're told to add built images to a manifest list, manipulate the
list ourselves, so that if we're creating a list, we won't have a
partially-populated list if some of the builds fail.
This also lets us include all of the platform information (including
variant info, which we can't sniff out after the fact) that we were
given when we started building the images.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add a pkg/parse.PlatformsFromOptions() which understands a "variant"
value as an optional third value in an OS/ARCH[/VARIANT] argument value,
which accepts a comma-separated list of them, and which returns a list
of platforms.
Teach "from" and "pull" about the --platform option and add integration
tests for them, warning if --platform was given multiple values.
Add a define.BuildOptions.JobSemaphore which an imagebuildah executor
will use in preference to one that it might allocate for itself.
In main(), allocate a JobSemaphore if the number of jobs is not 0 (which
we treat as "unlimited", and continue to allow executors to do).
In addManifest(), take a lock on the manifest list's image ID so that we
don't overwrite changes that another thread might be making while we're
attempting to make changes to it. In main(), create an empty list if
the list doesn't already exist before we start down this path, so that
we don't get two threads trying to create that manifest list at the same
time later on. Two processes could still try to create the same list
twice, but it's an incremental improvement.
Finally, if we've been given multiple platforms to build for, run their
builds concurrently and gather up their results.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Since opencontainers/image-spec/pull/822/ the OCI spec supports two new
annotations to set the fully-qualified name and the digest of the base
image.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
This is much scarier than I had intended. The intention is to
start migrating from 'expect_output' and to use 'assert'
instead; the reason is that 'assert' allows much more
flexibility, particularly negative tests.
We've long wanted something like "assert that output does
not contain 'foo'". I've been too lazy to implement it,
but last week I noticed code in bud.bats that does:
! expect_output "sdfsdfsdf"
This is a really super bad idea: although it works fine when
things are good and tests pass, it's a disaster when tests
fail because the poor person debugging test output now sees
a test failure log, and starts tracking it down, and then
much later realizes that it was an intentional failure but
the '!' was negating it, and all that track-down work was
wasted.
To keep this PR reasonable, I'm keeping all positive uses
of expect_output untouched. We can migrate those (if desired)
over time. All negative uses are now assertions.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Between runc-1.0.0-70.rc92 and 1.0.1-3, debug messages changed
entirely. Old runc is a short and sweet list:
time="..." level=debug msg="nsexec:601 nsexec started"
time="..." level=debug msg="child process in init()"
time="..." level=debug msg="logging has already been configured"
New runc is pages and pages of gobbledygook which I'm not going to
paste here but which, basically, is completely different. Better,
because most messages now include "runc", but different.
These buildah tests need to pass in environments with old and
new runc. As best I can determine, the "child process in init"
message is the only string common to both old and new runc.
Use it as our gauge. (Note: I considered using a regex pattern
containing both "nsexec" and "runc". That's less maintainable.
If/when runc changes debug messages again, we may need to go
that route, but for now let's keep things clean).
Signed-off-by: Ed Santiago <santiago@redhat.com>
In prepare(), don't check if the image name that it's passed is a
pseudonym for the result of a stage in the Dockerfile. Its callers
already did that.
When execute() knows that the image it's told to use as a base is a
pseudonym for the result of another stage in the Dockerfile, force the
pull policy to "never" to prevent an error when --pull-always=true.
Make imagebuildah.Mount a type alias instead of its own type, since we
never needed it to be a distinct type.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
This test was observed failing in upstream CI on all platforms due to
the contents of debug messages changing after updating runc. Since the
system tests need to function on multiple platforms with inconsistent
runc versions (depending on testing context), match a more general
output message on success. Specifically, the test really only cares
that debugging output appears and that runc is always used.
Signed-off-by: Chris Evich <cevich@redhat.com>
When copying a single file to the workdir make sure to create the workdir
and not to overwrite it as file.
Fixescontainers/podman#10671
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Currently the rusage is reported to stdout and rather cumbersome to parse. The
new flag rusage-logfile can be used to specify a file to which the log will be
written instead.
Signed-off-by: Dan Čermák <dcermak@suse.com>
* Don't try to count COMMIT as a step; it's very confusing and doesn't
match the behavior of traditional docker build.
* Include the step count for the stage, which is easy if we're not
trying to predict COMMIT, which we don't always do, because we don't
always have to, in multi-stage builds.
* In multi-stage builds, prefix the stage number and stage count, which
is fun to see when we're building stages in parallel.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Currently the /usr/bin/cpp will blow up if a user adds a
comment to a containerfile that is not a preprocessor.
Since the Containerfile.in could include other Containerfile
which may have comments, begining with `#` this can cause
problems.
If we just warn on these errors, we can successfully process
all of the containerfiles.
Fixes: https://github.com/containers/buildah/issues/3229
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Fix a regression in `buildah bud`. In case of a cache hit or an
intermediate image, the pull policy should be set to "never" so that we
enforce a local-only lookup without even attempting to reach out to a
registry.
The regression was detected in Podman which defaults to --layers=true
which ultimately broke when setting --pull or --pull-always. In case of
a cache hit, the image reference refers to a local image which conflicts
with an "always" pull policy.
Also extend an existing bud test to make sure we don't regress on it in
the future.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
resolveSymlink() was parsing the combined stdout and stderr from its
child process, which would include warning messages the storage library
printed to stderr during the child process's init() functions.
Instead of fixing just that, drop resolveSymlink() and replace it with a
call to copier.Eval().
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Stop using one-argument cat and avoid using `expect_output --substring`
as much as possible.
Signed-off-by: Hironori Shiina <shiina.hironori@jp.fujitsu.com>
Add support for secrets. Secrets is a two-part flag that allows secret files to
be accessed for a certain RUN instruction, but not any other
instructions, as well as now showing up in the final image.
Signed-off-by: Ashley Cui <acui@redhat.com>
Move all code related handling container image over to the new
`libimage` package in containers/common. The new package is an
attempt to consolidate the code across the containers tools under the
github.com/containers umbrella.
The new `libimage` packages provides functionality to perform all kinds
of operations for managing images such as local lookups, pushing,
pulling, listing, removing, etc.
The following packages have been moved over the containers/common:
`manifests` -> `common/image/manifests`
`pkg/manifests` -> `common/pkg/manifests`
`pkg/supplemented` -> `common/pkg/supplemented`
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
In Docker if you are copying more then one object, and
one of them is successful, then the command is successful. Currently in
buildah each glob has to be successful. This PR matches Buildah to
Docker.
Fixes: https://github.com/containers/podman/issues/9594
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Buildah currently handles multiple tags when building, but does not
report it to the user by default. This reports the tags back to the
user.
Removed some unused code from commit.go, that would blow up if a user
specified AdditionalTags to the commit command, even though this is not
exposed to the user currently. In a previous try to fix this, the
removed code was causing breakage, and I did not see a real purpose in
the code.
Fixes: https://github.com/containers/buildah/issues/3084
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
If a Containerfile had an ARG specified with a value and it was
referenced later in a multifrom scenario, a Warning would be raised
unless there was a `--build-arg` option specified for the argument.
This change removes the warning if the ARG has a value specified
in the Containerfile. We still need the warning however, if it
is not specified via a `--build-arg` or within the Containerfile.
Fixes: #3020
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
Move the platform check into `resolveLocalImage` and inspect each of the
candidates in the given order. This preserves the local lookup order
and fixes a bug reported in containers/podman/issues/9915.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Check if a builtin volume is a symlink. If it is,
follow the symlink and ensure that the destination
exists.
Add tests for symlink and no symlink case.
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
Primarily desired for the podman CI setup in which we
run bud.bats tests using podman-build; but these are
all worthwhile changes useful for buildah itself.
bud.bats:
- from-scratch-iid test: use expect_output to confirm
leading 'sha256' (was: [[ ... ]])
- hardlinks test: add test-name arg to expect_output
(to help diagnose test failures, such as just happened
in podman tests)
- build-arg-cache test: add test-name string to expect_output,
to help someone narrow down which test failed.
- stdin tests: use 'run_buildah <' instead of '| buildah'
- remove trailing whitespace
helpers.bash: reindent, converting tabs to spaces. TL;DR this
is needed for running buildah-bud tests under podman(*). As
of this commit, there are no tab characters in the file.
* Long version: podman CI runs bud.bats using 'podman build'.
This requires a few changes to bud.bats and helpers.bash,
changes which need to be carried in podman itself, not in
the buildah repo (like grokking 'remote', a concept that
is meaningless in buildah-land). We use 'git diff', which
is funny about whitespace, which in turn causes podman
CI hooks to fail when we try to check in the .diff file.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Currently if the user specifies a --file path
and path is a directory, we only append on Dockerfile.
This PR searches for Containerfile and then Dockerfile.
Fixes: https://github.com/containers/buildah/issues/3078
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Currently if you pull one image and then pullifmissing with
a different --arch, Buildah does not pull the different arch,
even though the arch is missing.
This PR checks the existing image to see if it matches the arch, os
and variant of the specified image, before using the local image.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Aditya R
Aditya Rajan
Leigh McCulloch
Nalin Dahyabhai
Daniel J Walsh
Kirill Shirinkin
Valentin Rothberg
Ed Santiago
OpenShift Merge Robot
Hironori Shiina
Chris Evich
Paul Holzinger
Dan Čermák
Hironori Shiina
Ashley Cui
TomSweeneyRedHat
Urvashi Mohnani