Commit Graph

137 Commits

Author SHA1 Message Date
Nalin Dahyabhai 7aedebdccd modernize: JSON doesn't do "omitempty" structs, so stop asking
Drop the "omitempty" tag from the "json" tag on struct members that are
themselves structs, since the JSON encoder doesn't actually do that, per
golang.org/x/tools/gopls/internal/analysis/modernize/cmd/modernize

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2025-09-10 13:19:38 -04:00
Nalin Dahyabhai 16680a4dfd modernize: use strings.CutPrefix/SplitSeq/FieldsSeq
Use the CutPrefix(), SplitSeq(), and FieldsSeq() functions from the
strings package when chopping up or iterating over parts of strings, per
golang.org/x/tools/gopls/internal/analysis/modernize/cmd/modernize

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2025-09-10 13:11:00 -04:00
Jan Kaluza a0a9ac6380 Switch common, storage and image to monorepo.
Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
2025-08-29 15:04:28 +02:00
Nalin Dahyabhai bf2cbe1645 Note that we have to build `true` first for the sake of its tests
Add a note that we need a test binary built for the sake of a few
conformance tests, for people who run the conformance tests directly
instead of using the top-level makefile target.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2025-08-11 15:49:28 -04:00
Rahil Bhimjiani 7a482acebc
Remove BUILDTAG btrfs_noversion as no longer effective
https://github.com/containers/storage/pull/2308

Signed-off-by: Rahil Bhimjiani <me@rahil.rocks>
2025-07-07 19:30:09 +05:30
Nalin Dahyabhai df36bb835a commit: exclude parents of mount targets, too
When RUN requires us to create the target for a mountpoint, make note of
it and any parent directories that needed to be created, and filter them
out when generating a layer diff or --output data.

The exceptions will be directories that the conformance tests confirm
that BuildKit also leaves behind, though for compatibility with the
classic builder, we have to make that conditional.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2025-06-20 11:19:27 -04:00
Nalin Dahyabhai c12808594b conformance: use mirrored frontend and base images
In conformance tests, use more mirrored frontend and base images.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2025-06-17 15:30:56 -04:00
Nalin Dahyabhai c0c4148fc5 Update to avoid deprecated types
Replace our use of the
github.com/docker/docker/api/types.BuildCachePruneOptions with the
github.com/docker/docker/api/types/build.CachePruneOptions type, which
the former is now an alias for.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2025-05-28 17:55:13 -04:00
Nalin Dahyabhai 26f4f3a025 Use Fedora 42 instead of 41 in that one conformance test
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2025-05-21 14:35:12 -04:00
Kir Kolyshkin 17777cf8ac Use for range over integers
Available since Go 1.22 (see https://tip.golang.org/ref/spec#For_range).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2025-04-08 10:19:51 -07:00
Kir Kolyshkin 1ede7ddce7 Use any instead of interface{}
Brought to you by

	gofmt -r 'interface{} -> any' -w .

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2025-04-08 10:19:51 -07:00
Kir Kolyshkin e8dba98314 ci: add nolintlint, fix found issues
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2025-04-07 13:02:39 -07:00
Kir Kolyshkin 8b3c045638 tests/conformance: fix unused warning
There is some code in tests/conformance which is only used by tests.
Move it to *_test.go files.

Found by golangci-lint run --tests=false, which shows this warning:

> tests/conformance/selinux.go:9:6: func `selinuxMountFlag` is unused (unused)
> func selinuxMountFlag() string {
>      ^

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2025-04-07 13:01:56 -07:00
Kir Kolyshkin b13670be84 tests/conformance: do not double import (fix ST1019)
This fixes the following warning:

> tests/conformance/conformance_test.go:37:2: ST1019: package "github.com/containers/image/v5/storage" is being imported more than once (staticcheck)
> 	is "github.com/containers/image/v5/storage"
> 	^
> tests/conformance/conformance_test.go:38:2: ST1019(related information): other import of "github.com/containers/image/v5/storage" (staticcheck)
> 	istorage "github.com/containers/image/v5/storage"
> 	^

Fixes: 3a61cc099 ("Add OverrideChanges and OverrideConfig to CommitOptions")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2025-04-07 13:01:56 -07:00
Kir Kolyshkin 6cf417995b tests/conformance: fix QF1012 warnings
These:

> tests/conformance/conformance_test.go:1361:16: QF1012: Use fmt.Fprintf(...) instead of Write([]byte(fmt.Sprintf(...))) (staticcheck)
> 		if _, err := tw.Write([]byte(fmt.Sprintf("Field\tDocker\t%s\n", notDocker))); err != nil {
> 		             ^
> tests/conformance/conformance_test.go:1393:16: QF1012: Use fmt.Fprintf(...) instead of Write([]byte(fmt.Sprintf(...))) (staticcheck)
> 		if _, err := tw.Write([]byte(fmt.Sprintf("File:attr\tDocker\t%s\n", notDocker))); err != nil {
> 		             ^

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2025-04-07 13:01:56 -07:00
Jan Rodák 38df169fcb
Use builtin arg defaults from imagebuilder
Signed-off-by: Jan Rodák <hony.com@seznam.cz>
2025-03-21 15:06:10 +01:00
Jan Rodák 7ca9f3464b
Add --parents option for COPY in Dockerfiles
It also includes an implementation of the --parents flag for the buildah copy command.

Fixes: https://issues.redhat.com/browse/RUN-2193
Fixes: https://github.com/containers/buildah/issues/5557

Signed-off-by: Jan Rodák <hony.com@seznam.cz>
2025-03-18 21:58:27 +01:00
Miloslav Trmač cc4cca08d4 Use UnparsedInstance.Manifest instead of ImageSource.GetManifest
... to validate that the manifests match expected digests, if any.

In some cases, using an UnparsedInstance can also avoid redundant I/O.

Do this everywhere, even where we read local storage which is
mostly trusted, because it is cheap enough and being consistent
makes it less likely for the code to be copied into other
contexts where the sources are not trusted.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2025-02-28 20:16:07 +01:00
Nalin Dahyabhai d5b30b000a tests/conformance/testdata/Dockerfile.add: update some URLs
The origin repository renamed its "master" branch to "main", so we need
to update our references to items in that branch.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2025-02-27 14:09:15 -05:00
Jan Rodák 177b845bd6
Vendor imagebuilder
New imagebuilder contains fix that sets empty TARGETVARIANT and BUILDVARIANT. Docker does not set these values for ARM64.
This fixes the TestConformance/multistage-builtin-args and TestConformance/builtins tests on ARM64.

Signed-off-by: Jan Rodák <hony.com@seznam.cz>
2025-02-25 17:01:21 +01:00
Jan Rodák 1f142eb581
Fix source of OS, architecture and variant
This fixes getting of cpu variants for ARM64
Fixes the TestConformance/setuid-file-in-archive test on ARM64.

Signed-off-by: Jan Rodák <hony.com@seznam.cz>
2025-02-25 17:01:21 +01:00
flouthoc fca3ca46bf
conformance: make TestCommit and TestConformance parallel
Two tests `TestCommit` and `TestConformance` can be made parallel given
that we are already using multiple cores for conformance tests.

Signed-off-by: flouthoc <flouthoc.git@gmail.com>
2025-02-19 08:36:40 -08:00
Nalin Dahyabhai 24826435f8 Distinguish --mount=type=cache locations by ownership, too
Normally, we select and distinguish --mount=type=cache directories that
we create by either the "id" or "target" value used when mounting them,
but we should also be distinguishing them by the "uid" and "gid" flags,
or lack thereof.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2025-02-07 13:38:56 -05:00
Eng Zer Jun 59024c41ad
refactor: replace golang.org/x/exp with stdlib
These experimental packages are now available in the Go standard
library since Go 1.21:

	1. golang.org/x/exp/slices -> slices [1]
	2. golang.org/x/exp/maps -> maps [2]

[1]: https://go.dev/doc/go1.21#slices
[2]: https://go.dev/doc/go1.21#maps

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2025-01-24 00:27:55 +08:00
Nalin Dahyabhai 25a3b385af internal/volume.GetBindMount(): discard writes in bind mounts
When handling RUN --mount=type=bind, where the mount is read-write,
instead of a simple bind mount, create an overlay mount with an upper
directory that will be discarded after the overlay mount is unmounted.
This brings us in line with the expected behavior, wherein writes to
bind mounts should be discarded.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2025-01-17 09:59:55 -05:00
Nalin Dahyabhai cfd98d782a copier: use .PAXRecords instead of .Xattrs
When processing tar headers, use the .PAXRecords field instead of the
deprecated .Xattrs field.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2024-12-12 14:54:30 -05:00
Daniel J Walsh aeb6ffeece
Add support for COPY --exclude and ADD --exclude options
Fixes: https://github.com/containers/buildah/issues/5678

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2024-10-16 16:30:16 -04:00
Nalin Dahyabhai 9e374f9fd4 copier: handle globbing with "**" path components
Handle glob patterns with "**" path components by expanding "**" to the
set of subdirectories and calling filepath.Glob() on the results.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2024-09-03 11:46:24 -04:00
Nalin Dahyabhai f88579d1fe imagebuildah: make scratch config handling toggleable
The default configuration that `docker build` applies to images built
using "scratch" has changed from classic builds to BuildKit.  Add a
toggle for selecting which behavior to mimic.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2024-08-28 10:21:16 -04:00
Nalin Dahyabhai 8ae99121c1 CI: enable the gofumpt linter
Turn on the gofumpt linter.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2024-08-15 13:17:44 -04:00
Nalin Dahyabhai d07c3dfb0e conformance: move weirdly-named files out of the repository
Remove certain weirdly-named files being used by a conformance test out
of the repository and create them at test-time.  Github refused to
generate archives of the repository when they were present in the commit
being requested.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2024-08-14 14:59:08 -04:00
Kir Kolyshkin fd3aba9917 *: use gofmt -s, add gofmt linter
1. Use gofmt to format the code:

	git ls-files \*.go | grep -Ev '/?vendor/' | xargs gofmt -w -s

2. Add gofmt to golangci-lint (it's not enabled by default).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2024-08-09 17:05:58 -07:00
Kir Kolyshkin 24da18800e *: fix build tags
This change is generated by `go1.23rc2 fix ./...`.

Had to use go1.23rc2, since all released go versions have a bug
preventing it from working with `go 1.22.0` in go.mod (opened
https://github.com/golang/go/issues/68825,
https://github.com/golang/go/issues/68824 for awareness).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2024-08-09 17:05:30 -07:00
openshift-merge-bot[bot] f2156795f4
Merge pull request #5676 from nalind/escape-globs
Add(): re-escape any globbed items that included escapes
2024-08-09 10:54:31 +00:00
Nalin Dahyabhai 3ea43566ac Add(): re-escape any globbed items that included escapes
When collecting items we matched using the passed-in globs during Add(),
be sure that globbing performed by Get() will match the items exactly,
by escaping special matching characters.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2024-08-08 16:46:47 -04:00
Nalin Dahyabhai 05da9a7450 conformance tests: use mirror.gcr.io for most images
Use busybox and alpine images from mirror.gcr.io, where possible, to
avoid tripping pull limits in CI.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2024-08-08 15:59:06 -04:00
Nalin Dahyabhai fdf1c75cd3 linters: unused arguments shouldn't have names
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2024-08-07 10:10:35 -04:00
Nalin Dahyabhai 3e38e5e138 imagebuildah: always have default values for $TARGET... args ready
Always have values for $TARGET... build args ready to be used.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2024-07-24 11:05:34 -04:00
Nalin Dahyabhai 9bb25a6daf conformance tests: enable testing CompatVolumes
Add flags for specifying which value of CompatVolumes we set, testing
both with and without it set.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2024-07-15 09:38:28 -04:00
Nalin Dahyabhai 1861c13207 conformance tests: add a test that tries to chown a volume
Add a conformance test that attempts to "chown" a volume declared in a
base image, which produces different results depending on whether we're
using the BuildKit-based builder or the V1 "classic" builder.  For now,
don't try to change our behavior, and continue imitating the behavior of
the classic builder.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2024-07-15 09:38:28 -04:00
Nalin Dahyabhai bfa84dfcbb conformance tests: enable testing CompatSetParent
Add flags for specifying not just that we use BuildKit, but specifically
which docker builder we request, and which value of CompatSetParent we
set at the same time.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2024-06-18 10:15:00 -04:00
Nalin Dahyabhai 5891afffea conformance: use quay.io/libpod/centos:7 instead of centos:8
Renovate updated references "FROM centos:7" to "FROM centos:8", but we
changed various others references to it to quay.io/libpod/centos:7 some
time ago.  Update these stragglers.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2024-06-07 11:32:12 -04:00
Nalin Dahyabhai 56fb08092e Stop setting "parent" in docker format
Newer docker build doesn't set it, so we need to stop.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2024-06-07 11:32:12 -04:00
Nalin Dahyabhai ba0947a3d5 conformance: check if workdir trims path separator suffixes
Check "WORKDIR" both with and without path separators at the end of the
instruction argument (e.g. "/foo" and "/foo/").

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2024-06-07 11:32:12 -04:00
Nalin Dahyabhai afe1a5dd55 Re-enable the "copy with chown" conformance test
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2024-06-07 11:32:12 -04:00
Nalin Dahyabhai a42019d614 Drop copyStringSlice() and copyStringStringMap()
Use slices.Clone() and maps.Clone() instead of our own non-generic
functions.  We have to be more careful in a couple of places where we
set items in maps which aren't unconditionally initialized.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2024-06-05 11:01:07 -04:00
Ed Santiago 7602e9e1d6 CI VMs: bump, to debian with cgroups v2
Per consent in 2024-03-20 Planning, remove support for cgroups v1.
And, per discovery on 2024-05-28, systemd 256 no longer even
boots on a cgroups v1 system.

This commit switches to VMs built in:

  https://github.com/containers/automation_images/pull/338

...in which Debian is now cgroups v2 with crun.

Requires disabling two conformance tests which fail when run
against docker 26.1; see #5526 for context on those.

Also requires disabling two bats tests on debian because
something changed there in ulimits. I'll look into them
later but right now this gives us breathing space.

And, latest git on f40 refuses to serve non-root files when
run as root ("dubious ownership"); so, in start_git_daemon(),
chown the extracted files.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2024-05-29 10:04:49 -06:00
Nalin Dahyabhai 97382f9288 Don't expand RUN heredocs ourselves, let the shell do it
When handling RUN instructions that use heredoc syntax, don't bother
interpolating environment variables and argument values, and let the
command that's running handle it.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2024-05-20 09:35:48 -04:00
Daniel J Walsh da4e2d7ad9 Setting --arch should set the TARGETARCH build arg
Also fix a long standing FIXME in the test framework.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2024-05-06 19:49:09 -04:00
Nalin Dahyabhai b2504c2636 conformance tests: don't break on trailing zeroes in layer blobs
When analyzing a layer blob's contents, don't break if the blob has more
zeroes padding it out even after the tar reader thinks it's hit the end
of the archive.

Add more detail to the diagnostic error we print when there's a digest
or length mismatch, too, in case it's triggered by something other than
zero padding.

Don't ignore errors which might be encountered when we try to use skopeo
to copy an image to a directory.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2024-03-06 17:37:49 -05:00