They compare buildah-bud against podman-run, which breaks if
/usr/bin/podman changes its ulimits, as happened recently
in podman PR 24335.
Signed-off-by: Ed Santiago <santiago@redhat.com>
As I used this locally and the binaires already existed make did not
rebuild. While we could list all go files here nobody should be
modifying files under vendor/ directly so just checking go.mod/sum seems
easiest.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
When the target location of a RUN --mount is specified as a relative
path, we normally try to convert it to an absolute path by combining it
with the currently-configured working directory. If there is no such
value, though, the result is still not an absolute path. Work around
this by using "/" when the configured working directory is "".
Set this field in the `runMountInfo` struct on FreeBSD, as we already
did on Linux.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
- all images pushed to a local registry must have a unique name.
Bring in safename() helper from podman tests.
- all cache tests must use a private TMPDIR
- in force-compression test, use a custom-crafted image with
no possibility of sharing layers with any other image that
any other test might push to the registry.
- use a private crun tmpdir in seccomp test, because crun
does some unexpected caching.
And, forgive me, a little refactoring of unpleasant duplication
Signed-off-by: Ed Santiago <santiago@redhat.com>
Prep work for parallelizing bats tests: when pushing images to
local registry, use a pseudorandom unique name to avoid
possibility of collisions/conflicts with other tests. This
is good practice regardless of whether we run serial or parallel.
Also slight refactor of some duplicate code.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Write output files only to $TEST_SCRATCH_DIR, never to cwd
Reformat overly-long lines for readability (whitespace only changes)
And, in the last test, the output files are plaintext, not json.
Fix filenames accordingly.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Bugs found during testing of parallel bats tests:
- write randomfiles in $TEST_SCRATCH_DIR, not cwd!
- remove unused & confusing & broken expectedEnv code
And, I couldn't help myself, improve & refactor one test
Signed-off-by: Ed Santiago <santiago@redhat.com>
This wrapper doesn't need to load anything from helpers.bash, because
the various .bats files already do so on their own.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Use a listener helper to bind to an available-according-to-the-kernel
listening port and run a command with its stdio more or less tied to the
connection instead of trying to launch a git daemon directly using a
port number that we can only guess is available.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Describe the "env" and "type" options in the buildah-build(1) man page.
When parsing the "--secret=" flag for the CLI, instead of ignoring an
option that we don't recognize, return an error.
Even though the set of meaningful "id" values for secrets is passed in
via the command line, don't directly use it to construct a file path.
Change the default mode for SSH agent sockets that we create from 0o620
to 0o600.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
The `--mount type=cache` argument to the `RUN` instruction in
Dockerfiles was using `filepath.Join` on user input, allowing
crafted paths to be used to gain access to paths on the host,
when the command should normally be limited only to Buildah;s own
cache and context directories. Switch to `filepath.SecureJoin` to
resolve the issue.
Fixes CVE-2024-9675
Signed-off-by: Matt Heon <mheon@redhat.com>
In podman we also run the bud tests, there I noticed a issue with the
podman skip logic as it was unable to fine the build-add-https-retry-ca
test name as it always expects quotes[1]
In general names should be human readable so add quotes and use spaces
over a dash as word separator.
[1] https://github.com/containers/podman/pull/24135
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
CVE-2024-9407: validate that the value for the "bind-propagation" flag
when handling "bind" and "cache" mounts in `buildah run` or in RUN
instructions is one of the values that we would accept without the
"bind-propagation=" prefix.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
github.com/moby/sys/capability is a fork of the (no longer maintained)
github.com/syndtr/gocapability package.
For the list of changes since the fork took place, see
https://github.com/moby/sys/blob/main/capability/CHANGELOG.md
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Changed the `--all` option of `buildah manifest push` to be true by
default. This matches the behavior of the equivalent Podman option
(`podman manifest push --all`), making it easier to switch between
Podman and Buildah.
Updated buildah.manifest.push.1 docs to reflect this change.
Added test "manifest-push-all-default-true" to `tests/lists.bats`.
Closes: #5547
Signed-off-by: Paul Nettleton <k9@k9withabone.dev>
Rootless users cannot set the `security.ima` xattr on files
(presumably for security reasons, they get an EPERM on trying to
do so). We will normally try and preserve that xattr, so when
trying to add a file with an IMA xattr to a build on a Buildah
without this patch, you get an error. With this patch, the error
is downgraded to a warning, as it's better to successfully build
with a missing xattr than blocking all builds which want to
include the offending file.
The urgency on this has become somewhat higher as it seems like
F41/Rawhide are installing rpm-plugin-ima by default, which is
setting IMA xattrs on some files that Podman relies on - for
example, the catatonit binary we use for pid pause images.
Without this patch, building the pause image as rootless will
always fail on a system with rpm-plugin-ima installed.
Fixes: https://github.com/containers/podman/issues/18543
Signed-off-by: Matt Heon <mheon@redhat.com>
Clean up the distinctions between the volumes slice and the volumeCache
and volumeCacheInfo maps so that --compat-volumes will work correctly
when we're building with multiple layers.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
If we're not given an explicit platform or arch or os to target for a
build, but someone defined TARGETPLATFORM as a build argument, parse it.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Don't error out when `manifest add --artifact` is given multiple files,
and add a test which should have checked that.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
We always map 169.254.1.2 with pasta to the host now so ensure the
host.containers.internal entry is set correctly.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Handle glob patterns with "**" path components by expanding "**" to the
set of subdirectories and calling filepath.Glob() on the results.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
The default configuration that `docker build` applies to images built
using "scratch" has changed from classic builds to BuildKit. Add a
toggle for selecting which behavior to mimic.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
set `avoidLookingCache` to `true` if `--mount` is using a freshly built
stage and not for `additional-build-context`.
Signed-off-by: flouthoc <flouthoc.git@gmail.com>
Remove certain weirdly-named files being used by a conformance test out
of the repository and create them at test-time. Github refused to
generate archives of the repository when they were present in the commit
being requested.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
1. Use gofmt to format the code:
git ls-files \*.go | grep -Ev '/?vendor/' | xargs gofmt -w -s
2. Add gofmt to golangci-lint (it's not enabled by default).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This change is generated by `go1.23rc2 fix ./...`.
Had to use go1.23rc2, since all released go versions have a bug
preventing it from working with `go 1.22.0` in go.mod (opened
https://github.com/golang/go/issues/68825,
https://github.com/golang/go/issues/68824 for awareness).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
When collecting items we matched using the passed-in globs during Add(),
be sure that globbing performed by Get() will match the items exactly,
by escaping special matching characters.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Use busybox and alpine images from mirror.gcr.io, where possible, to
avoid tripping pull limits in CI.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Don't end a loop iteration with an "else" block if the "then" block
would cause the loop iteration to finish, due to a "break", "continue",
or known-to-not-return function call.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
They largely duplicate other integration tests. Add an integration test
to cover the "output from inspect is valid JSON" case.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add CertPath and InsecureSkipTLSVerify flags to AddAndCopyOptions, and
connect the CLI flag values passed to `buildah add` and `buildah build`
so that Builder.Add() gets those.
Add MaxRetries and RetryDelay fields to AddAndCopyOptions, and connect
them to the values passed on the command line to `buildah add` and
`buildah build`.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add optional third and fourth arguments to starthttpd() which can
specify a location to store a generated self-signed TLS certificate and
key which will be used to start the helper HTTP server with TLS, and an
optional fifth argument to which it will write its PID.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Don't expect to be able to resolve an ARG in the FROM instruction for
the second stage if the ARG was only declared in the first stage. Only
ARGs introduced in the header are available for FROM instructions.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Don't expect to be able to resolve an ARG in the FROM instruction for
the second stage if the ARG was only declared in the first stage. Only
ARGs introduced in the header are available for FROM instructions.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Make tests pass when using a local cache registry (as triggered
by $CI_USE_REGISTRY_CACHE being non-empty).
Mostly just change images: consolidate to a smaller set, like,
replace the few instances of debian with ubuntu, use $SAFEIMAGE
where we can.
The most significant change was with some s1 (schema 1?) images.
Those exist on quay, but cannot be mirrored locally: the local
registry forbids schema 1 images. I choose to skip these tests
when running against a local registry.
Second most significant change is running containerized tests
with --net=host. We need this to access the registry running
on a localhost port. (One alternative would be to bind the
registry on all interfaces, cringe, then perform sed magic
on the registries-cache.conf file changing 127.0.0.1 to
host.containers.internal, more cringe).
Also, some cleanup.
Signed-off-by: Ed Santiago <santiago@redhat.com>
As of https://github.com/containers/automation_images/pull/357
our CI VMs include a local registry preloaded with all(*)
images used in tests.
* where "all" means "most".
This commit defines a new registries-cached.conf, used in tests,
that redirects docker and quay to the new local registry. The
hope is that this will reduce CI flakes.
Since tests change over time, and new tests may require new
images, this commit also adds a mechanism for pulling in
remote images at test run time. Obviously this negates
the purpose of the cache, since it introduces a flake
pain point. The idea is: DO NOT DO THIS UNLESS ABSOLUTELY
NECESSARY, and then, if we have to do this, hurry up and
spin new CI VMs that include the new image(s).
Signed-off-by: Ed Santiago <santiago@redhat.com>
Add a conformance test that attempts to "chown" a volume declared in a
base image, which produces different results depending on whether we're
using the BuildKit-based builder or the V1 "classic" builder. For now,
don't try to change our behavior, and continue imitating the behavior of
the classic builder.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Make the traditional handling of volumes (where they're "frozen" and can
only be modified by ADD or COPY, which requires that we cache their
contents and save/restore them before/after RUN instructions) an option
that is not enabled by default.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
If FROM was used with a --platform flag, then the imagebuilder.Builder
will have its Platform field set, and we should include it when logging
the instruction.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When `buildah copy` is invoked with a `--from` flag, default to
preserving ownerships that were set in the source container or image.
Retain the "set it to 0:0 by default" behavior when `--from` is not
being used.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add flags for specifying not just that we use BuildKit, but specifically
which docker builder we request, and which value of CompatSetParent we
set at the same time.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
If we're prepending history entries before the one for "this" commit,
make sure the "FROM $baseimage" comment gets set on the first history
entry that we add, not just the one goes with this (maybe) layer diff.
In layers=false mode, the output was so, so confusing otherwise.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
buildah from and buildah build will now default to --pull=missing
as opposed to --pull=always, which they did before. This better
matches to the defaults in docker and podman.
No longer document --pull=true|false
Fixes: https://github.com/containers/buildah/issues/5406
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Renovate updated references "FROM centos:7" to "FROM centos:8", but we
changed various others references to it to quay.io/libpod/centos:7 some
time ago. Update these stragglers.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Check "WORKDIR" both with and without path separators at the end of the
instruction argument (e.g. "/foo" and "/foo/").
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Pass the test registry password to `docker login` using stdin instead of
passing it on the command line.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Starting in common 0.59.1, github.com/containers/common's libnetwork no
longer attempts to detect when it's running in a user namespace to
decide where a lock file that it uses will be. Since this test is doing
user namespace setup on its own as part of the test, we need to
explicitly tell it to not use the one that belongs to the node's root
user.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Use slices.Clone() and maps.Clone() instead of our own non-generic
functions. We have to be more careful in a couple of places where we
set items in maps which aren't unconditionally initialized.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Per consent in 2024-03-20 Planning, remove support for cgroups v1.
And, per discovery on 2024-05-28, systemd 256 no longer even
boots on a cgroups v1 system.
This commit switches to VMs built in:
https://github.com/containers/automation_images/pull/338
...in which Debian is now cgroups v2 with crun.
Requires disabling two conformance tests which fail when run
against docker 26.1; see #5526 for context on those.
Also requires disabling two bats tests on debian because
something changed there in ulimits. I'll look into them
later but right now this gives us breathing space.
And, latest git on f40 refuses to serve non-root files when
run as root ("dubious ownership"); so, in start_git_daemon(),
chown the extracted files.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Before #5083, when running with chroot isolation ro mounts like secrets
from env vars would explicitly have the unix.MS_NOEXEC, unix.MS_NOSUID
and unix.MS_NODEV flags set when they were remounted. Now when running
with chroot isolation ro mounts like secrets from env vars are not
getting those same flags set and so the remount operation fails.
Specifically it looks like we are missing the unix.MS_NOSUID and
unix.MS_NODEV flags.
This change adds special handling for read-only mounts when we need to do
a remount to try to get the desired flags to stick. If we've requested
a read-only mount (unix.ST_RDONLY is set in requestFlags), then we add any
possibleImportantFlags that are set in fs.Flags to remountFlags so the remount
operation doesn't fail because they are missing. I've also added a test to
bud.bats that covers this case.
Signed-off-by: Jonah Bull <jonah.bull@elastic.co>
Only add the default OS to a build target platform struct if the
architecture was specified without one, so that the pull logic doesn't
override our pull policy when it doesn't need to.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
1. Difference in error messages, podman does not emit "building
system context". I think it's OK to just check for the
"--platform may not be used with" portion.
2. Very weird: $TESTSDIR (the plural one) was eliminated in April 2022
but one instance snuck in, probably a PR that was not rebased
onto main. This is blowing up in podman tests because of course
there is no $TESTSDIR. But I have **NO IDEA** how this is passing
in buildah, nor how it passed in podman for two years.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Build a fake layers-but-no-history image that should work more or less
as well for the tests that we were previously using nixery.dev/shell for.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When handling RUN instructions that use heredoc syntax, don't bother
interpolating environment variables and argument values, and let the
command that's running handle it.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Use the directories configured in containers.conf, now that
containers/common can tell us what they are, and now that it provides a
place to configure defaults for container tools, always override the
library's default set, even if it's empty, which means we do nothing.
Switch to the default CDI registry instead of the recenty-deprecated
non-global one.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Replace some of the base images we've been using (particularly centos:8,
which will EOL soon) with other images hosted on quay.io.
We already use registries.conf at test-time to redirect some image
references there, so this will slightly reduce the number of registries
which we need to be able to reach while running these tests.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Merge the two tar filters, if we need two, that we use when committing
an image. Try to improve passing of error information from the writing
end of a pipe to the reader, so that it can be reported better.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
It's no longer an active release for the rhcontainerbot/podman-next COPR
at https://copr.fedorainfracloud.org/coprs/rhcontainerbot/podman-next/.
Add "adjustments to .packit.yaml" to the list of things we don't require
updated tests for.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Instead of assuming which labels are set in our base image, and using
the --unsetlabel flag to suppress them in a derived image, query the
base image and unset all of them, except for our own version label.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Don't bother removing specific images at the beginning or the end of
tests, since we don't carry over the storage used between tests anyway.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Attempt to make more thorough use of cached copies of images during
integration tests, except in tests which need to pull a new image while
they're running, either because they test pulling directly, or because
they expect to pull a given image for a non-default platform or for
multiple platforms.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When fetching images in the _prefetch() function, use the same registry
configuration (shortname configuration, registry aliases) that we're
already telling buildah to use when we call it, so that we prefetch from
the locations that buildah would subsequently use if we didn't prefetch.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
If we are copying heredoc files, we need to temporary place
them in the context dir and then move to container via copier
there are cases where .containerignore can have a patterns like
which can match our heredoc files so let's not set any excludes
or IgnoreFile for this copy.
Closes: https://github.com/containers/buildah/issues/5391
Signed-off-by: flouthoc <flouthoc.git@gmail.com>
A comment states that avoidLookingCache is set when a previous stage
that executed as part of the build is referenced by --mount, to avoid
reusing content from an older build of the stage:
// Only attempt to find cache if its needed, this part
// so that if a step is using RUN --mount and mounts
// previous stages then it uses the freshly built stage
// of re-using the older stage from the store.
However, stages consisting of COPY/ADD seem to be flagged with
didExecute even if they were fetched from cache instead. I believe
this is an oversight, and these stages should not prevent subsequent
caching.
Also, avoidLookingCache would prevent a cache push, but I think it
should only prevent cache lookups, since populating the cache is still
useful in these caess.
It's very possible I'm misunderstanding something, but I believe the
RUN step in test case I've added wrongly skips cache, and I'd appreciate
some pointers in the right direction if what I've proposed here isn't
the right solution.
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
Add support for using CDI to resolve `--device` devices for RUN
instructions during `buildah build`, `buildah from`, and `buildah run`,
as `podman run` does.
This generally requires that we stop resolving device specifications
(arguments passed to --device) earlier and deferring that until it's
time to run a process, because CDI wants to pick over those values,
modify a runtime spec to set up the ones that it knows about, and then
hand back the list of values that it doesn't know about.
We don't want to do a dry run of this during CLI processing because that
would create a window where the underlying hardware state could change,
and that could produce some hard-to-diagnose errors.
Being able to test this is going to require that we add the `--device`
flag to `buildah run` (`--security-opt` affects how we build the
container's layer, so it has to be done at `buildah from`).
The default configured devices list is pulled in by CLI flag processing
during `buildah from` and `buildah build`, so it doesn't also need to be
explicitly passed to `buildah run` or the internal Run() method.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Just before merging #5411, a comment was made
about changing a RUN command to a comment. This
completes that.
Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
Addresses CVE-2024-1753 which allowed a user to write files to the
`/` directory of the host machine if selinux was not enabled.
Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
Previously buildah may have created a incorrect hosts/resolv.conf file,
when netavark, slirp4netns or pasta are used we have to actually
consider their special setup and use the correct nameservers.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
...to allow bypassing the check. Just like on podman.
Also, bring up to code:
- grep -F, not fgrep
- fix regression test script (was using wrong branch envariable)
- add new test case
Signed-off-by: Ed Santiago <santiago@redhat.com>
This should have been done a long time ago and this would have made
clear that it did not work properly. However now that pasta is the
default and we fixed all the remaining problems we can easily enable
them.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
When analyzing a layer blob's contents, don't break if the blob has more
zeroes padding it out even after the tar reader thinks it's hit the end
of the archive.
Add more detail to the diagnostic error we print when there's a digest
or length mismatch, too, in case it's triggered by something other than
zero padding.
Don't ignore errors which might be encountered when we try to use skopeo
to copy an image to a directory.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add a conformance test for cases where an intermediate stage mounts the
contents of a previous stage in a read-write fashion and modifies it.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
unshare on RHEL8 does not support --setuid. This is causing
gating tests to fail.
Solution: check for option, skip test if unavailable
Signed-off-by: Ed Santiago <santiago@redhat.com>
buildah build --pull=false is documented to never pull the image, but it
is currently ignored.
Fixes: https://github.com/containers/podman/issues/21783
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
When figuring out which platforms the base images allow us to build
derived images for, screen out any images with non-empty artifactType
values. Also screen out any which use empty values or the word
"unknown" in the OS and Architecture platform fields, and any
Architecture values that the compiler hasn't heard of.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add functionality for creating artifact manifests and adding them to
image indexes. `buildah manifest add` gets a `--artifact` option for
telling it to create artifact manifests, and `--artifact-type`,
`--artifact-config`, `--artifact-config-type`, `--artifact-layer-type`,
`--artifact-exclude-titles`, and `--subject` options to fine-tune the
contents of the artifact manifests it creates.
Add a `--index` flag to `buildah manifest annotate` so that it can be
told to set annotations on the index itself instead of on one of the
entries in the image index.
Add a `--subject` flag to `buildah manifest annotate` for setting the
`subject` field of an image index.
Add a `--annotation` flag to `buildah manifest create` to allow for
adding annotations to the new image index.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
In tests/validate/pr-should-include-tests and
tests/validate/whitespace.sh, use grep -E instead of egrep, because
egrep keeps telling us to switch.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
... in an attempt to try to get UID 0 in a user namespace to stop trying
to read files from root's home directory, where the permissions error is
treated as a hard failure.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Buildah must show summary of heredoc content in build output so its easy
for developers to understand which heredoc got executed, this is similar
to what buildkit does for heredoc content.
See: https://github.com/moby/buildkit/blob/master/frontend/dockerfile/dockerfile2llb/convert.go#L1853
Sample output of buildah
```console
STEP 1/5: FROM docker.io/library/python:latest
STEP 2/5: RUN <<EOF (echo "Hello" >> /hello...)
STEP 3/5: RUN python3 <<EOF (with open("/hello", "w") as f:...)
STEP 4/5: RUN ls -a
```
Signed-off-by: flouthoc <flouthoc.git@gmail.com>
Just like `buildah push`, `buildah manifest push` should also support
`--retry` and `--retry-delay` options, see documentation in same commit
for more details.
Closes: https://github.com/containers/buildah/issues/5254
Signed-off-by: flouthoc <flouthoc.git@gmail.com>
Podman currently sets the ulimits of nofile and nproc
to max in rootless mode, if the user does not override.
Buildah on the other hand just passes in the current defaults.
Podman build should match podman run, and this will fix that problem.
Fixes: https://github.com/containers/buildah/issues/5273
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Some new heredoc test added "FROM blah blah python whatever",
an image that (presumably) exists on docker.io but does not
exist in our cache.
Plus, test was completely broken anyway. It would've found
the "this is the output" lines even without python, as
part of the verbose build.
Solution: don't use python. You don't need python to test a shebang.
You can use anything. 'cat' is traditional, but I choose 'rev'
because that makes it nearly impossible for the test to match
merely due to a build-step echo.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Add a --sbom flag to `buildah build` and `buildah commit` which will
scan the rootfs and specified context directories to build SPDX or
CycloneDX SBOMs and lists of package URLs.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
If the parent image has layers but no history, force our own omitHistory
setting on.
The alternative is to create a history that only explains the presence
of some of the layers in our output image, which looks broken to
everyone who might consume that image, including ourselves if we try to
use it as a base image later.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When we have extra files to add to the image, handle them by adding them
to the upper overlay layer before creating the plaintext filesystem
image.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
If there are any shebang in heredoc file then buildah must honor that.
Consider a case of
```Dockerfile
FROM python:3.11-slim-bullseye
RUN <<EOF
print('hello world')
EOF
```
Closes: https://github.com/containers/buildah/issues/5251
Signed-off-by: flouthoc <flouthoc.git@gmail.com>
When using buildah with `--layers` then buildah must correctly burst
layer cache if `heredoc` content is changed. Following is achieved via
properly adding `heredoc` content to the history of the built image.
Closes: https://github.com/containers/buildah/issues/5225
Signed-off-by: flouthoc <flouthoc.git@gmail.com>
Add a flag to `buildah commit` which allows adding arbitrary files to
the image while we're committing it. When not squashing, they'll take
the form of a second new layer.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When using the working container's rootfs to populate a plaintext disk
image with mkfs, instead of writing .krun_config.json to the rootfs and
then removing it afterward (since we don't want it to show up if the
same working container is later committed to non confidential-workload
image), mount an overlay filesystem using a temporary directory as the
upper and the rootfs as the lower, create the .krun_config.json file in
the overlay filesystem, and use the overlay filesystem as the source
directory for mkfs.
Add the necessary stubs to allow pkg/overlay to at least compile on
non-Linux systems. Change the naming scheme for a test so that the path
names it uses for temporary directories don't include "," or "=", which
can confuse the kernel.
Creating confidential workload images will now only be possible on Linux
systems, but we exec'd out to sevctl to read platform certificates, and
that requires kernel support with vendor firmware, so I don't know that
anyone will actually be impacted by the change.
Teach pkg/overlay.MountWithOptions() to accept `nil` as a pointer to a
struct parameter that is otherwise optional.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When creating a container using a prefetched image, use --quiet and
--pull=false for the pair of tests which didn't, bringing them in line
with the rest of the tests in the file.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Check if `unshare` supports the `--map-users` option in
`skip_if_no_unshare`, since we're depending on that in the only
integration test that uses `skip_if_no_unshare`.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Ignore whether or not the final build instruction should produce a layer
if we're squashing or producing a confidential workload, when we'd still
have to produce a layer containing the contents of the base image.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add an OverrideChanges and an OverrideConfig field to CommitOptions,
both of which can be used to make last-minute edits to the configuration
of an image that we're committing.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Following PR is a attempt to add `Heredoc` support to buildah.
Once this PR is merged buildah is supposed to honor heredoc syntax while
processing containerfiles
Expected syntax to work
```Dockerfile
FROM docker.io/library/python:latest
RUN <<EOF
echo "Hello" >> /hello
echo "World!" >> /hello
EOF
RUN python3 <<EOF
with open("/hello", "w") as f:
print("Hello", file=f)
print("Something", file=f)
EOF
RUN ls -a
RUN cat hello
```
Signed-off-by: Aditya R <arajan@redhat.com>
When uploading a context directory to dockerd, generate the archive
ourselves so that we can force the ownerships in it to 0:0, which
BuildKit seems to expect clients to do.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Co-authored-by: flouthoc <flouthoc@users.noreply.github.com>
It seems we try to reuse blobs only for the specified registry, however
we can have valid known compressed digests across registry as well
following pr attempts to use that by doing following steps.
* `CandidateLocations2` now processes all known blobs and appends them
to returned candidates at the lowest priority. As a result when
`TryReusingBlob` tries to process these candidates and if the blobs
filtered by the `Opaque` set by the `transport` fail to match then
attempt is made against all known blobs (ones which do not belong to the
current registry).
* Increase the sample set of potential blob reuse to all known
compressed digests , also involving the one which do not belong to
current registry.
* If a blob is found match it against the registry where we are
attempting to push. If blob is already there consider it a `CACHE
HIT!` and reply skipping blob, since its already there.
----
```console
$ skopeo copy docker://registry.fedoraproject.org/fedora-minimal docker://quay.io/fl/test:some-tag
$ buildah pull registry.fedoraproject.org/fedora-minimal
$ buildah tag registry.fedoraproject.org/fedora-minimal quay.io/fl/test
$ buildah push quay.io/fl/test
```
```console
Getting image source signatures
Copying blob a3497ca15bbf skipped: already exists
Copying config f7e02de757 done
Writing manifest to image destination
Storing signatures
```
Testing: https://github.com/containers/image/pull/1645
Signed-off-by: Aditya R <arajan@redhat.com>
When we don't have any instructions to process, but we do have an
unsetEnv or unsetLabel list, force a commit, like we already did if we
were passed an out-of-band labels or annotations list, or if we were
squashing, and now also do so if we're doing a confidential workload.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
This mostly just inherits the c/common/pkg/auth implementation,
except that AuthFilePath and DockerCompatAuthFilePath can not be set
simultaneously, so don't always set AuthFilePath. c/common already
defaults to the same locations internally.
Test handle only invalid commands; a true interoperability test
would require a running Docker on the CI systems, which is not currently
available. That interoperability was tested manually
(and is presumed to be integration-tested in the Podman repo).
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
go-dockerclient gained the ability to let us ask for a build kicked off
using its API to be done using BuildKit, so we don't have to work around
that by calling the Docker client package any more when doing
conformance testing.
The go-dockerclient method also reports errors in a way that's easier
for us to consume, which we didn't have fully debugged on the other code
path.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Make sure that we accept both upper- and lower-cased versions of the
names of TEE types as arguments for `buildah build`'s --cw flag and for
`buildah mkcw`'s --type flag. We previously only understood lower-case
versions, but the docs always used upper case.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Use require.NoErrorf() as a check instead of the more general
require.Nil(), which was both less specific and didn't expect a format
string.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Pay better attention to dev/nodev/exec/noexec/suid/nosuid/ro/rw flags on
bind, overlay, and tmpfs mounts when any of them are specified. Stop
quietly adding "nodev" when it isn't asked for.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When building a multi-stage image ( without `--layers` ) and a
stage contains only a base-image buildah removes the base-image
itself as part of cleanup process. This is a bug and following commit
fixes that.
Reproducer
```Dockerfile
FROM parent
FROM another-base
COPY --from=0 somefile .
```
`buildah build -t multi-stage .`
Closes: https://github.com/containers/podman/issues/20291
Signed-off-by: Aditya R <arajan@redhat.com>
Previously, when mounting multiple mounts, if any mount had a `type`
specified, it would override the default type for subsequent mounts.
This meant, for example, that having a RUN step like:
```
RUN --mount=type=cache,target=/cache --mount=source=.,target=/src
```
would incorrectly mount the second source as a cache, instead of a
bind-mount.
This fix ensures the default is reset between each iteration of the
loop, ensuring we get the right mount type.
Signed-off-by: Benjamin Schubert <contact@benschubert.me>
`Docker` and `Buildkit` creates no layers when only `FROM scratch` is
used as content for building image, buildah must do the same.
Signed-off-by: Aditya R <arajan@redhat.com>
Make sure that `add` and `copy` handle relative paths given as input
correctly, both with and without a context directory being specified.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add a --cw option to `buildah build` and `buildah commit`, which takes a
comma-separated list of arguments and produces an image laid out for use
as a confidential workload:
type: sev or snp
attestation_url: location of a key broker server
cpus: expected number of virtual CPUs to run with
memory: expected megabytes of memory to run with
workload_id: a distinguishing identifier for the key broker server
ignore_attestation_errors: ignore errors registering the workload
passphrase: for encrypting the disk image
slop: extra space to allocate for the disk image
At least one of attestation_url and passphrase must be specified in
order for the encrypted disk image to be decryptable at run-time. Other
arguments can be omitted. ignore_attestation_errors is intentionally
undocumented, as it's mainly used to permit some amount of testing on
systems which don't have the required hardware.
Add an `mkcw` top-level command, for converting directly from an image
to a confidential workload.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Don't try to look up an image by name when we're committing an image,
because we don't want to accidentally take advantage of any fuzzy
matching that libimage might start doing. Instead, just use the
normalization call.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When pulling from an OCI source, make sure to preseve the optional name.
For instance, a podman pull oci:/tmp/foo:quay.io/foo/bar:latest should
pull the image and name it quay.io/foo/bar:latest.
While at it, also fix a bug when pulling an OCI without the optional
name. Previously, we used the path to name the image which will error in
most cases due to invalid characters (e.g., capital ones). Hence, apply
the same trick as for the dir transport and generate a sha.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
With discussion from here:
https://github.com/containers/podman/pull/19640, it was decided that
`--force-compression` must be automatically `true` in case when
`--compression-format` is set.
Signed-off-by: Aditya R <arajan@redhat.com>
The Podman repo changed and removed a subdir that this tests was relying
on. Rather then relying on a different repo, it would be better to
stick to buildah repo for github tests.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
build-arg warnings must honor globally defined arguments and must not
produce warnings if an arguments is already defined globally.
Closes: https://github.com/containers/buildah/issues/4981
Signed-off-by: Aditya R <arajan@redhat.com>
Adds support for `--force-compression` which allows end-users to force
push blobs with the selected compresison in `--compression` option, in
order to make sure that `blobs` of other compression on registry are not
reused.
Is equivalent to: `force-compression` here: https://docs.docker.com/build/exporters/#compression
Closes: https://github.com/containers/buildah/issues/4613
Also Implements:
`--compression-format` and `--compression-level` for `manifest push` just like
`podman`'s `manifest push`
Signed-off-by: Aditya R <arajan@redhat.com>
`--layer-label` allows users to set labels on intermediate labels agnostic
of the labels set on actual image. Since there are use-cases where users
want to perform operation on intermediate images only on the basis of
certain labels.
Closes: https://github.com/containers/buildah/issues/4933
Signed-off-by: Aditya R <arajan@redhat.com>
Adds support for `--add-compression` which accepts multiple compression
formats and when used it will add all instances in a manifest list with
requested compression formats.
Signed-off-by: Aditya R <arajan@redhat.com>
Use the `default_rootless_network_cmd` containers.conf options to know
which rootless network program to use as default. This setting is
important so distros and user can actually set a different default if
they wish.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Like podman allow buildah and therefore podman build to use the network
mode pasta. The pasta integration is very simple and we do not even
need a teardown handler for that as pasta will exit on its own when the
netns path is removed.
However right now this is broken, pasta will fail to open
/proc/$pid/ns/net. I send a patch[1] to fix this upstream in pasta.
I assume this will land quickly so I like to get this in now just so we
have this included in podman v4.6. Thus the test is skipped for now.
[1] https://archives.passt.top/passt-dev/20230623082531.25947-2-pholzing@redhat.com/
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
This uses the same code as podman for slirp4netns, this means
- ipv6 is enabled by default
- slirp4netns options are read from contianers.conf
- slirp4netns options can now be set on the cli. This required some
small rework on where we parse the network string.
Lastly I updated the --network docs, to document the new slirp4netns
mode. That included fixing up buildah-from and buildah-run pages which
were incomplete in that regard. Now we show the same for all options.
Fixes#3968
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Some tools depend on the hostname being present in /etc/hosts. I would
argue they are broken but its not like we can do anything about that.
This adds the hostname with the local host ip when the host network is
used. For private networking we already add the hostname.
We also now correctly force host networking in chroot mode, it was
silently ignored before thus causing extra confusion here.
Fixes#4446
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Golangci-lint < v1.51.1 can't work on systems with golang 1.20.
Updating to 1.51.2 then has dependent modules that require a minimum of
golang 1.17, requiring a re-vendoring of tooling.
Signed-off-by: Chris Evich <cevich@redhat.com>
A non-nil but empty decryption configuration
seems to be valid enough to trigger decryption in some
configurations, per
https://github.com/containers/podman/issues/18196 .
Like in Skopeo and Podman, only decrypt when the user explicitly
instructs us to (e.g. not triggering decryption based on environment
variables).
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Commit beae5647c0 updated the conformance test references to a
deprecated repository. However, by pointing at the docker-hub it
inadvertantly introduce a significant possibility for flakes. This is
because anonymous docker hub access is rate-limited by IP. We cannot
predict the IP used for CI VMs & Containers, any of which could be at or
close to the limit. Fix this by pointing explicitly at a
`quay.io/libpod/centos` repo. which is excluesively for use by CI, with
static images.
Signed-off-by: Chris Evich <cevich@redhat.com>
Test is currently very hard to debug on failure. Fix that
by adding unique test descriptors and a little whitespace.
Also, fix a broken/NOP test (copypaste artifact)
Signed-off-by: Ed Santiago <santiago@redhat.com>
In https://github.com/containers/buildah/pull/4673 we made a change were
we were applying labels to end of each stage, which is different than
what we were doing before i.e applying label at the end of the each
step.
However buildkit does not adds label to any stage or steps it only adds
label at the end of final stage so lets do that.
Closes: https://github.com/containers/buildah/issues/4804
Signed-off-by: Aditya R <arajan@redhat.com>
Kir Kolyshkin
Ed Santiago
openshift-merge-bot[bot]
Paul Holzinger
Nalin Dahyabhai
Daniel J Walsh
Matt Heon
Paul Nettleton
danishprakash
Jan Rodák
flouthoc
Asutorufa
Jonah Bull
Giuseppe Scrivano
Aaron Lehmann
tomsweeneyredhat
Aditya R
renovate[bot]
Miloslav Trmač
Jean-Francois Roy
Benjamin Schubert
Valentin Rothberg
Chris Evich