update some URLs
The origin repository renamed its "master" branch to "main", so we need
to update our references to items in that branch.
[NO NEW TESTS NEEDED]
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When analyzing a layer blob's contents, don't break if the blob has more
zeroes padding it out even after the tar reader thinks it's hit the end
of the archive.
Add more detail to the diagnostic error we print when there's a digest
or length mismatch, too, in case it's triggered by something other than
zero padding.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When comparing layer payloads during conformance tests, mask off any
file type bits that the tar headers in the layers might have included.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Signed-off-by: Chris Evich <cevich@redhat.com>
The registry.centos.org service has been decommissioned. Update the
conformance test references to point into the static CI images under
the `quay.io/libpod` repositories.
Signed-off-by: Aditya R <arajan@redhat.com>
Signed-off-by: Ed Santiago <santiago@redhat.com>
Signed-off-by: Chris Evich <cevich@redhat.com>
When container is created with specific uid and gid also add container
gid to supplementary/additional group.
Signed-off-by: Aditya R <arajan@redhat.com>
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Check that the inheritable capabilities are set to 0, even when we
explicitly try to add capabilities.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When checking if something that we want to overwrite with a directory is
already a directory or not, use lstat instead of stat. If it's a
symbolic link, it's not a directory.
This is a subtle behavior change, but it's in line with docker build.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
* Change DEST_BRANCH in .cirrus.yml from main to release-1.23
* Set GITVALIDATE_EPOCH for validation to the release-1.23 branch point
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Following PR makes sure that buildah mirrors --authfile to a temporary
file in filesystem if arg is pointing to an FD instead of actual file
as FD can be only consumed once.
Fixes: https://github.com/containers/buildah/issues/3070
Signed-off-by: Aditya Rajan <arajan@redhat.com>
The build with git URL recognizes git URLs by either a git:// scheme prefix, or
any other URL with other schemes such as https:// and a .git suffix.
The build with git URL and branch recognizes the branch name as the fragment
part of the URL, the part after the #.
The .git suffix check is performed by checking that the suffix of the full URL
is .git, however this causes URLs that begin with scheme https://, and have a
.git path suffix, to not be recognized as git URLs when a branch name is
appended.
The logic for checking if the URL path is suffixed with .git should parse the
URL first and consider only the path when looking for the suffix. This change
is implemented in this commit.
This commit also removes the guard within the cloneToDirectory function because
it is unnecessary since every location the function is called the check has
already been carried out.
Signed-off-by: Leigh McCulloch <351529+leighmcculloch@users.noreply.github.com>
Following commit makes sure that `buildah manifest rm <list>` only removes
the manifest list not referenced images.
Signed-off-by: Aditya Rajan <arajan@redhat.com>
Add a --max-parallel-downloads flag to allow tuning down from the
default of six blobs at a time, and tune it down to one at a time when
we invoke the helper.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When processing a directory tree, only descend into a directory that is
marked for exclusion if its path is literally a prefix of an exception
pattern.
Subtly, but in a way that's compatible with docker, this means that if
we exclude directory "subdir", but we've been told to also include
"**/file" (with an exclusion pattern of "!**/file"), we won't descend
into "subdir" and find a file named "subdir/file", because "**/file"
doesn't start with "subdir/".
More generally, exclusion patterns that start with "!" which include any
wildcards before their final component technically won't be treated
correctly.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Following commit makes sure when `buildah tag` is invoked on a manifest
list, it tags the same manifest list instead of resolving image and
tagiing it.
Signed-off-by: Aditya Rajan <flouthoc.git@gmail.com>
Modify how we process exceptions in conformance tests so that we don't
unintentionally disable an attribute comparison (such as "mtime") on
items below a directory when we try to skip that attribute comparison on
the directory itself.
Fix some incorrect specifications for filesystem differences that we're
supposed to ignore, and use the -t flag in more places where we RUN the
`touch` command to create files that end up in the final image.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Currently buildah images does not have an easy way to get the epoch
creation time. This field is available in `podman images` as the
Created field. Adding to buildah images to make it consistent.
Fixes: https://github.com/containers/buildah/issues/3478
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
The golang.org/x/crypto/ssh/terminal package has been deprecated and
replaced upstream by golang.org/x/term, so switch to that. It's a
simple 1:1 replacement.
[NO NEW TESTS NEEDED]
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When we're told to add built images to a manifest list, manipulate the
list ourselves, so that if we're creating a list, we won't have a
partially-populated list if some of the builds fail.
This also lets us include all of the platform information (including
variant info, which we can't sniff out after the fact) that we were
given when we started building the images.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When firing up a minimal server to serve up content so that we can
retrieve it using HTTP, make it possible to let the kernel assign us
which port to use, so that the script that's calling us doesn't have to
make a(n occasionally bad) guess.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Allow ssh socket from host or certain ssh keys to be exposed to a
certain RUN instruction, but not any other instructions, as well as not
showing up in the final image.
This is done by spawining a new agent from buildah and mounting
the listening socket inside the run. SSH_AUTH_SOCK inside the container
will be set to the socket mountpoint. The defualt mountpoint is
/run/buildkit/ssh_agent.{i}
Signed-off-by: Ashley Cui <acui@redhat.com>
Make sure to use the correct resolv.conf in case of systemd-resolved.
Also filter out 127... nameserver when run in private netns.
Fixes#2780
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Add a pkg/parse.PlatformsFromOptions() which understands a "variant"
value as an optional third value in an OS/ARCH[/VARIANT] argument value,
which accepts a comma-separated list of them, and which returns a list
of platforms.
Teach "from" and "pull" about the --platform option and add integration
tests for them, warning if --platform was given multiple values.
Add a define.BuildOptions.JobSemaphore which an imagebuildah executor
will use in preference to one that it might allocate for itself.
In main(), allocate a JobSemaphore if the number of jobs is not 0 (which
we treat as "unlimited", and continue to allow executors to do).
In addManifest(), take a lock on the manifest list's image ID so that we
don't overwrite changes that another thread might be making while we're
attempting to make changes to it. In main(), create an empty list if
the list doesn't already exist before we start down this path, so that
we don't get two threads trying to create that manifest list at the same
time later on. Two processes could still try to create the same list
twice, but it's an incremental improvement.
Finally, if we've been given multiple platforms to build for, run their
builds concurrently and gather up their results.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Since opencontainers/image-spec/pull/822/ the OCI spec supports two new
annotations to set the fully-qualified name and the digest of the base
image.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
This is much scarier than I had intended. The intention is to
start migrating from 'expect_output' and to use 'assert'
instead; the reason is that 'assert' allows much more
flexibility, particularly negative tests.
We've long wanted something like "assert that output does
not contain 'foo'". I've been too lazy to implement it,
but last week I noticed code in bud.bats that does:
! expect_output "sdfsdfsdf"
This is a really super bad idea: although it works fine when
things are good and tests pass, it's a disaster when tests
fail because the poor person debugging test output now sees
a test failure log, and starts tracking it down, and then
much later realizes that it was an intentional failure but
the '!' was negating it, and all that track-down work was
wasted.
To keep this PR reasonable, I'm keeping all positive uses
of expect_output untouched. We can migrate those (if desired)
over time. All negative uses are now assertions.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Between runc-1.0.0-70.rc92 and 1.0.1-3, debug messages changed
entirely. Old runc is a short and sweet list:
time="..." level=debug msg="nsexec:601 nsexec started"
time="..." level=debug msg="child process in init()"
time="..." level=debug msg="logging has already been configured"
New runc is pages and pages of gobbledygook which I'm not going to
paste here but which, basically, is completely different. Better,
because most messages now include "runc", but different.
These buildah tests need to pass in environments with old and
new runc. As best I can determine, the "child process in init"
message is the only string common to both old and new runc.
Use it as our gauge. (Note: I considered using a regex pattern
containing both "nsexec" and "runc". That's less maintainable.
If/when runc changes debug messages again, we may need to go
that route, but for now let's keep things clean).
Signed-off-by: Ed Santiago <santiago@redhat.com>
In prepare(), don't check if the image name that it's passed is a
pseudonym for the result of a stage in the Dockerfile. Its callers
already did that.
When execute() knows that the image it's told to use as a base is a
pseudonym for the result of another stage in the Dockerfile, force the
pull policy to "never" to prevent an error when --pull-always=true.
Make imagebuildah.Mount a type alias instead of its own type, since we
never needed it to be a distinct type.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
By proxy by vendoring containers/common. Previously, a "dangling" image
was an untagged image; just a described in the Docker docs. The
definition of dangling has now been refined to an untagged image without
children to be compatible with Docker.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Add new `buildah source {create,add,push,pull}` commands. All commands
are marked as experimental. None of it is meant to be officially
supported at the time of writing. All code resides in `internal/source`
and is hence not visible to external consumers of Buildah; just to be
on the safe side.
A source container or source image is an OCI artifact, that is an OCI
image with custom config (media type). There is a longer history behind
source images which are intended to ship the source artifacts of an
ordinary "executable" container image. Until now, source images at
Red Hat are built with github.com/containers/BuildSourceImage. We had a
growing desire (and always the long-term plan) to eventually replace
BuildSurceImage with something else, in this case Buildah.
This commit adds the initial base functionality along with tests to make
sure we're not regressing. The new commands do the following:
* `create` - creates an empty and initialized source image
* `add` - tar up a local path and add it as a layer to the souce image
* `push/pull` - intentionally separate commands from `buildah push/pull`
to allow for an easier usage and prevent the
implementations from undesired (future) interference
Further note: also vendor in c/image@master which ships a required fix.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Sometime around `docker.io 20.10.2-0ubuntu1~20.04.2` several error
message strings were updated from `file not found` to `file does not
exist`. This breaks conformance testing. Fix this by adding in
the new error message.
Signed-off-by: Chris Evich <cevich@redhat.com>
This test was observed failing in upstream CI on all platforms due to
the contents of debug messages changing after updating runc. Since the
system tests need to function on multiple platforms with inconsistent
runc versions (depending on testing context), match a more general
output message on success. Specifically, the test really only cares
that debugging output appears and that runc is always used.
Signed-off-by: Chris Evich <cevich@redhat.com>
Set extended attributes on files _after_ setting their ownership, so
that security-sensitive attributes ("security.capability" among them)
won't get quietly cleared from under us if we set them before calling
chown().
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Nalin Dahyabhai
Aditya R
Aditya Rajan
Leigh McCulloch
OpenShift Merge Robot
Daniel J Walsh
Kirill Shirinkin
openshift-ci[bot]
Ashley Cui
Paul Holzinger
Valentin Rothberg
Ed Santiago
dependabot[bot]
Hironori Shiina
Chris Evich
Jacob Gillespie