Two tests `TestCommit` and `TestConformance` can be made parallel given
that we are already using multiple cores for conformance tests.
Signed-off-by: flouthoc <flouthoc.git@gmail.com>
Normally, we select and distinguish --mount=type=cache directories that
we create by either the "id" or "target" value used when mounting them,
but we should also be distinguishing them by the "uid" and "gid" flags,
or lack thereof.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
These experimental packages are now available in the Go standard
library since Go 1.21:
1. golang.org/x/exp/slices -> slices [1]
2. golang.org/x/exp/maps -> maps [2]
[1]: https://go.dev/doc/go1.21#slices
[2]: https://go.dev/doc/go1.21#maps
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Handle glob patterns with "**" path components by expanding "**" to the
set of subdirectories and calling filepath.Glob() on the results.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
The default configuration that `docker build` applies to images built
using "scratch" has changed from classic builds to BuildKit. Add a
toggle for selecting which behavior to mimic.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Remove certain weirdly-named files being used by a conformance test out
of the repository and create them at test-time. Github refused to
generate archives of the repository when they were present in the commit
being requested.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
1. Use gofmt to format the code:
git ls-files \*.go | grep -Ev '/?vendor/' | xargs gofmt -w -s
2. Add gofmt to golangci-lint (it's not enabled by default).
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
When collecting items we matched using the passed-in globs during Add(),
be sure that globbing performed by Get() will match the items exactly,
by escaping special matching characters.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Use busybox and alpine images from mirror.gcr.io, where possible, to
avoid tripping pull limits in CI.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add a conformance test that attempts to "chown" a volume declared in a
base image, which produces different results depending on whether we're
using the BuildKit-based builder or the V1 "classic" builder. For now,
don't try to change our behavior, and continue imitating the behavior of
the classic builder.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add flags for specifying not just that we use BuildKit, but specifically
which docker builder we request, and which value of CompatSetParent we
set at the same time.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Check "WORKDIR" both with and without path separators at the end of the
instruction argument (e.g. "/foo" and "/foo/").
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Use slices.Clone() and maps.Clone() instead of our own non-generic
functions. We have to be more careful in a couple of places where we
set items in maps which aren't unconditionally initialized.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Per consent in 2024-03-20 Planning, remove support for cgroups v1.
And, per discovery on 2024-05-28, systemd 256 no longer even
boots on a cgroups v1 system.
This commit switches to VMs built in:
https://github.com/containers/automation_images/pull/338
...in which Debian is now cgroups v2 with crun.
Requires disabling two conformance tests which fail when run
against docker 26.1; see #5526 for context on those.
Also requires disabling two bats tests on debian because
something changed there in ulimits. I'll look into them
later but right now this gives us breathing space.
And, latest git on f40 refuses to serve non-root files when
run as root ("dubious ownership"); so, in start_git_daemon(),
chown the extracted files.
Signed-off-by: Ed Santiago <santiago@redhat.com>
When handling RUN instructions that use heredoc syntax, don't bother
interpolating environment variables and argument values, and let the
command that's running handle it.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When analyzing a layer blob's contents, don't break if the blob has more
zeroes padding it out even after the tar reader thinks it's hit the end
of the archive.
Add more detail to the diagnostic error we print when there's a digest
or length mismatch, too, in case it's triggered by something other than
zero padding.
Don't ignore errors which might be encountered when we try to use skopeo
to copy an image to a directory.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add a conformance test for cases where an intermediate stage mounts the
contents of a previous stage in a read-write fashion and modifies it.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add an OverrideChanges and an OverrideConfig field to CommitOptions,
both of which can be used to make last-minute edits to the configuration
of an image that we're committing.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Following PR is a attempt to add `Heredoc` support to buildah.
Once this PR is merged buildah is supposed to honor heredoc syntax while
processing containerfiles
Expected syntax to work
```Dockerfile
FROM docker.io/library/python:latest
RUN <<EOF
echo "Hello" >> /hello
echo "World!" >> /hello
EOF
RUN python3 <<EOF
with open("/hello", "w") as f:
print("Hello", file=f)
print("Something", file=f)
EOF
RUN ls -a
RUN cat hello
```
Signed-off-by: Aditya R <arajan@redhat.com>
When uploading a context directory to dockerd, generate the archive
ourselves so that we can force the ownerships in it to 0:0, which
BuildKit seems to expect clients to do.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Co-authored-by: flouthoc <flouthoc@users.noreply.github.com>
go-dockerclient gained the ability to let us ask for a build kicked off
using its API to be done using BuildKit, so we don't have to work around
that by calling the Docker client package any more when doing
conformance testing.
The go-dockerclient method also reports errors in a way that's easier
for us to consume, which we didn't have fully debugged on the other code
path.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Use require.NoErrorf() as a check instead of the more general
require.Nil(), which was both less specific and didn't expect a format
string.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
We handle --label command line arguments by appending LABEL instructions
to the Dockerfile contents before we parse it. Previously, we were
appending a separate line for each label-value pair. Consolidate them
for the sake of tools that arbitrarily limit the length of histories
that they're willing to accept in images (boo!).
Add a similar implementation for --env command line arguments.
Previously, we'd set them in the initial configuration for each stage
and also set them at commit-time, and that potentially overrode any
values that were explicitly in the stage itself, and which would have
affected RUN instructions. Remove the set-at-commit-time logic so that
the history reflects what ends up in the image.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
In golang 1.19, `io/ioutil` is fully deprecated preventing Buildah from
compiling. Replace all calls with equivalent calls from the `os`
package.
Signed-off-by: Chris Evich <cevich@redhat.com>
When noting that a non-symlink has setuid/setgid/sticky bits, switch
from using "syscall" package bits and syscall.Chmod() to using "os"
package bits and os.Chmod(), and if the item's a directory, record the
updated mode information in the "directoryModes" map that we'll use to
reset its permissions later.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Podman adds an Error: to every error message. So starting an error
message with "error" ends up being reported to the user as
Error: error ...
This patch removes the stutter.
Also ioutil.ReadFile errors report the Path, so wrapping the err message
with the path causes a stutter.
Signed-off-by: Daniel J Walsh dwalsh@redhat.com
[NO NEW TESTS NEEDED]
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.
Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
defer func() {
if err := os.RemoveAll(dir); err != nil {
t.Fatal(err)
}
}
is also tedious, but `t.TempDir` handles this for us nicely.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
We now use the golang error wrapping format specifier `%w` instead of
the deprecated github.com/pkg/errors package.
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
The test passes currently, but testing the reverse of the
replace-symlink-with-directory case seems like a good idea.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When checking if something that we want to overwrite with a directory is
already a directory or not, use lstat instead of stat. If it's a
symbolic link, it's not a directory.
This is a subtle behavior change, but it's in line with docker build.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
The API flag that we'd need to set to in a build request to get dockerd
use BuildKit instead of the classic docker builder isn't available in
go-dockerclient, so add a second docker-based path that uses the API
types and client library, which the conformance tests were already
pulling in as indirect dependencies, but skip tests that set the flag if
we're not on the current client version.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When processing a directory tree, only descend into a directory that is
marked for exclusion if its path is literally a prefix of an exception
pattern.
Subtly, but in a way that's compatible with docker, this means that if
we exclude directory "subdir", but we've been told to also include
"**/file" (with an exclusion pattern of "!**/file"), we won't descend
into "subdir" and find a file named "subdir/file", because "**/file"
doesn't start with "subdir/".
More generally, exclusion patterns that start with "!" which include any
wildcards before their final component technically won't be treated
correctly.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Modify how we process exceptions in conformance tests so that we don't
unintentionally disable an attribute comparison (such as "mtime") on
items below a directory when we try to skip that attribute comparison on
the directory itself.
Fix some incorrect specifications for filesystem differences that we're
supposed to ignore, and use the -t flag in more places where we RUN the
`touch` command to create files that end up in the final image.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
flouthoc
Nalin Dahyabhai
Eng Zer Jun
Daniel J Walsh
Kir Kolyshkin
openshift-merge-bot[bot]
Ed Santiago
Aditya R
Chris Evich
Sascha Grunert
Giuseppe Scrivano