run: after we've picked up the exit status of the "main" process that
we're running, reap anything that was reparented to us before returning.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When building or committing an image in OCI format, default to setting
the org.opencontainers.image.created annotation to the value used in the
image's config blob for the image's creation date. The behavior can be
controlled using the new --created-annotation flag.
Add --annotation and --unsetannotation flags to `buildah commit` which
mimic the same flags for `buildah build`.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When RUN requires us to create the target for a mountpoint, make note of
it and any parent directories that needed to be created, and clear them
all out if they look basically the same after the command finishes.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Pass BUILDAH_RUNTIME through to tests that we run inside of containers,
and discard the CI_DESIRED_RUNTIME variable, which sort of did the same
thing. We still set OCI to the name of the runtime because tests
consult it directly.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Use $SOURCE_DATE_EPOCH as the default for the --source-date-epoch flag
to the "build" CLI.
When a source-date-epoch is set, we'll use it when writing new history
entries, force timestamps in data written for --output to the specified
timestamp, and populate a "SOURCE_DATE_EPOCH" ARG that we treat as
always being set, and which we don't complain about being left unused.
By default, this will not affect timestamps in newly-added layers.
Add a --rewrite-timestamp flag, which "clamps" timestamps in newly-added
layers to not be later than the --source-date-epoch value if the
--source-date-epoch flag is set, but has no effect otherwise.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add a dummy "runtime" that just dumps its runtime config, either the
entirety of it, or a section of it corresponding to each command line
argument. Tests can use it to ensure that we set the right thing in the
configuration without also depending on the runtime to do as its asked,
which isn't always something we have control over.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add some helper functions for parsing the information from dir: and oci:
locations that we'd just have to do over and over with minor variations,
to try to save some time and a refactor down the road.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add `run_with_log` to mkcw tests.
Add `sleep 1` during cleanup between attempting `luksClose`
and unmounting the filesystem mounted on the device /dev/mapper/"$uuid".
Without this somehow we end up in a state where mount is still being
used by the kernel because when we do `lsof /dev/mapper/"$uuid"` it
shows nothing but `dmsetup info -c $uuid` shows the device is still
under use. Adding `sleep 1` in between somehow fixes this.
Also this problem with `cryptsetup` is pretty common for reference
one thread which I found https://lore.kernel.org/all/508950BA.1030309@dennis.birkholz.biz/T/
Signed-off-by: flouthoc <flouthoc.git@gmail.com>
The _prefetch helper, introduced in #2036, is not parallel-safe: two
or more parallel jobs fetching the same image can step on each other
and produce garbage images.
Although we still can't run buildah tests in parallel (see #5552),
we can at least set up the scaffolding for that to happen. This
commit reworks _prefetch() such that the image work is wrapped
inside flock. It has been working fine for months in #5552,
and is IMO safe for production. This can then make it much
easier to flip the parallelization switch once the final zstd
bug is squashed.
Signed-off-by: Ed Santiago <santiago@redhat.com>
- all images pushed to a local registry must have a unique name.
Bring in safename() helper from podman tests.
- all cache tests must use a private TMPDIR
- in force-compression test, use a custom-crafted image with
no possibility of sharing layers with any other image that
any other test might push to the registry.
- use a private crun tmpdir in seccomp test, because crun
does some unexpected caching.
And, forgive me, a little refactoring of unpleasant duplication
Signed-off-by: Ed Santiago <santiago@redhat.com>
Use a listener helper to bind to an available-according-to-the-kernel
listening port and run a command with its stdio more or less tied to the
connection instead of trying to launch a git daemon directly using a
port number that we can only guess is available.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add CertPath and InsecureSkipTLSVerify flags to AddAndCopyOptions, and
connect the CLI flag values passed to `buildah add` and `buildah build`
so that Builder.Add() gets those.
Add MaxRetries and RetryDelay fields to AddAndCopyOptions, and connect
them to the values passed on the command line to `buildah add` and
`buildah build`.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add optional third and fourth arguments to starthttpd() which can
specify a location to store a generated self-signed TLS certificate and
key which will be used to start the helper HTTP server with TLS, and an
optional fifth argument to which it will write its PID.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Make tests pass when using a local cache registry (as triggered
by $CI_USE_REGISTRY_CACHE being non-empty).
Mostly just change images: consolidate to a smaller set, like,
replace the few instances of debian with ubuntu, use $SAFEIMAGE
where we can.
The most significant change was with some s1 (schema 1?) images.
Those exist on quay, but cannot be mirrored locally: the local
registry forbids schema 1 images. I choose to skip these tests
when running against a local registry.
Second most significant change is running containerized tests
with --net=host. We need this to access the registry running
on a localhost port. (One alternative would be to bind the
registry on all interfaces, cringe, then perform sed magic
on the registries-cache.conf file changing 127.0.0.1 to
host.containers.internal, more cringe).
Also, some cleanup.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Per consent in 2024-03-20 Planning, remove support for cgroups v1.
And, per discovery on 2024-05-28, systemd 256 no longer even
boots on a cgroups v1 system.
This commit switches to VMs built in:
https://github.com/containers/automation_images/pull/338
...in which Debian is now cgroups v2 with crun.
Requires disabling two conformance tests which fail when run
against docker 26.1; see #5526 for context on those.
Also requires disabling two bats tests on debian because
something changed there in ulimits. I'll look into them
later but right now this gives us breathing space.
And, latest git on f40 refuses to serve non-root files when
run as root ("dubious ownership"); so, in start_git_daemon(),
chown the extracted files.
Signed-off-by: Ed Santiago <santiago@redhat.com>
When fetching images in the _prefetch() function, use the same registry
configuration (shortname configuration, registry aliases) that we're
already telling buildah to use when we call it, so that we prefetch from
the locations that buildah would subsequently use if we didn't prefetch.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
unshare on RHEL8 does not support --setuid. This is causing
gating tests to fail.
Solution: check for option, skip test if unavailable
Signed-off-by: Ed Santiago <santiago@redhat.com>
Check if `unshare` supports the `--map-users` option in
`skip_if_no_unshare`, since we're depending on that in the only
integration test that uses `skip_if_no_unshare`.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Pay better attention to dev/nodev/exec/noexec/suid/nosuid/ro/rw flags on
bind, overlay, and tmpfs mounts when any of them are specified. Stop
quietly adding "nodev" when it isn't asked for.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Allow containers running under buildah to use
--group-add keep-groups, so that they can inherit
access to the users groups.
Also allow users to add supplimental groups to the container.
Fixes: https://github.com/containers/buildah/issues/4476
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
The ubi8 image changed some architecture string yesterday and
broke all our CI on all branches, including podman because
it too runs bud.bats tests.
Solution: use a safe, reliable, trustworthy test image (the
one built and used for podman). Because that image does not
have the same labels, confirm pull using .Architecture
instead.
We should stop using ubi8 and registry.redhat (#4318). They
are unreliable. This PR doesn't fix that; it's just an
emergency patch for one specific CI break. We can use
this as a basis for future removals of ubi8.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Primary purpose: fix "preconfigured TARGETARCH/etc" test so
it will work under podman and on multiarch.
Root cause of it not working: I mistakenly advised @flouthoc,
in #4310, to write a containerfile in $TEST_SCRATCH_DIR. I
thought it was an empty directory. Big, big mistake. (Sorry,
Aditya). Document this near the variable definition, and
fix the test once again.
@nalind pointed out that the containerfile doesn't need to
be generated on-the-fly, so, use a static one. In the spirit
of DIE, read the TARGETxxx vars from it. Not that we're
expecting more variables, but, it's just cleaner.
Also, as long as I'm here: in run_buildah, when logging the
command being run, use #/$ prompt for root/rootless. I was
getting too confused looking at logs of root runs.
Signed-off-by: Ed Santiago <santiago@redhat.com>
When we encounter an error parsing an additional Dockerfile, complain
about the right file, instead of the one we parsed before it.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Set the default for user namespaces correctly for callers that don't use
our CLI, e.g. OpenShift or our own tutorials. When we don't do that,
commands invoked through RUN instructions can see weird things, in this
case an empty /sys/fs/cgroup directory.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
We now use the golang error wrapping format specifier `%w` instead of
the deprecated github.com/pkg/errors package.
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
Use the new etchosts package to generate the hosts file.
This will ensure that we use the same logic in podman and buildah.
New features are:
- no duplicated entries
- adds entries for the network/slirp4netns ips
- configure the host.containers.internal entry in containers.conf
- configure the base hosts file in containers.conf
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Right now `assert t:[1] != t:[1]` passes. This is obviously incorrect
and some test might not be working correctly because of this. We have to
special case this like the "==" case.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
When a test needs to talk to a registry server, launch one as part of
the test rather than depending on it having been started by someone
else.
Use run_buildah where we used to use 'run buildah' without checking the
return code, and in a few cases where we did check it.
In the "from with non buildah container" test, use "podman create" with
host networking, in an attempt to avoid messing with networking in cases
where we're running on a system with a version of podman that will
create a bridge with CNI that we'll also create with netavark. We're
not sharing storage between the two invocations, so the logic that tries
to detect this problem won't detect it.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Mount and umount can must be unshared for rootless environment.
Co-authored-by: Ed Santiago <santiago@redhat.com>
Signed-off-by: Aditya R <arajan@redhat.com>
Instrument test so that they can be executed in rootless environment.
Certain tests perform mount directly and they need a seperate
usernamespace, these tests are marked to be skipped for `rootless
environment`
Signed-off-by: Aditya R <arajan@redhat.com>
Fixes gating test failure:
```
/usr/share/buildah/test/system/./helpers.bash: line 474: !is_cgroupsv2: command not found
```
Co-authored-by: Yiqiao Pu <ypu@redhat.com>
I'm just the committer.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
When testing that we can handle contexts that are described using the
git:// protocol, launch a local git-daemon with a minimal repository to
point the binary at instead of github, which doesn't do that any more.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add a --max-parallel-downloads flag to allow tuning down from the
default of six blobs at a time, and tune it down to one at a time when
we invoke the helper.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When firing up a minimal server to serve up content so that we can
retrieve it using HTTP, make it possible to let the kernel assign us
which port to use, so that the script that's calling us doesn't have to
make a(n occasionally bad) guess.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Allow ssh socket from host or certain ssh keys to be exposed to a
certain RUN instruction, but not any other instructions, as well as not
showing up in the final image.
This is done by spawining a new agent from buildah and mounting
the listening socket inside the run. SSH_AUTH_SOCK inside the container
will be set to the socket mountpoint. The defualt mountpoint is
/run/buildkit/ssh_agent.{i}
Signed-off-by: Ashley Cui <acui@redhat.com>
Make sure to use the correct resolv.conf in case of systemd-resolved.
Also filter out 127... nameserver when run in private netns.
Fixes#2780
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
This is much scarier than I had intended. The intention is to
start migrating from 'expect_output' and to use 'assert'
instead; the reason is that 'assert' allows much more
flexibility, particularly negative tests.
We've long wanted something like "assert that output does
not contain 'foo'". I've been too lazy to implement it,
but last week I noticed code in bud.bats that does:
! expect_output "sdfsdfsdf"
This is a really super bad idea: although it works fine when
things are good and tests pass, it's a disaster when tests
fail because the poor person debugging test output now sees
a test failure log, and starts tracking it down, and then
much later realizes that it was an intentional failure but
the '!' was negating it, and all that track-down work was
wasted.
To keep this PR reasonable, I'm keeping all positive uses
of expect_output untouched. We can migrate those (if desired)
over time. All negative uses are now assertions.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Make it possible to override the location of the copy test helper by
setting its location in an environment variable, as we do for imgtype.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Nalin Dahyabhai
Lokesh Mandvekar
flouthoc
Ed Santiago
Daniel J Walsh
Sascha Grunert
Paul Holzinger
OpenShift Merge Robot
Giuseppe Scrivano
Aditya R
Lokesh Mandvekar
openshift-ci[bot]
Ashley Cui