When analyzing a layer blob's contents, don't break if the blob has more
zeroes padding it out even after the tar reader thinks it's hit the end
of the archive.
Add more detail to the diagnostic error we print when there's a digest
or length mismatch, too, in case it's triggered by something other than
zero padding.
Don't ignore errors which might be encountered when we try to use skopeo
to copy an image to a directory.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
It seems we try to reuse blobs only for the specified registry, however
we can have valid known compressed digests across registry as well
following pr attempts to use that by doing following steps.
* `CandidateLocations2` now processes all known blobs and appends them
to returned candidates at the lowest priority. As a result when
`TryReusingBlob` tries to process these candidates and if the blobs
filtered by the `Opaque` set by the `transport` fail to match then
attempt is made against all known blobs (ones which do not belong to the
current registry).
* Increase the sample set of potential blob reuse to all known
compressed digests , also involving the one which do not belong to
current registry.
* If a blob is found match it against the registry where we are
attempting to push. If blob is already there consider it a `CACHE
HIT!` and reply skipping blob, since its already there.
----
```console
$ skopeo copy docker://registry.fedoraproject.org/fedora-minimal docker://quay.io/fl/test:some-tag
$ buildah pull registry.fedoraproject.org/fedora-minimal
$ buildah tag registry.fedoraproject.org/fedora-minimal quay.io/fl/test
$ buildah push quay.io/fl/test
```
```console
Getting image source signatures
Copying blob a3497ca15bbf skipped: already exists
Copying config f7e02de757 done
Writing manifest to image destination
Storing signatures
```
Testing: https://github.com/containers/image/pull/1645
Signed-off-by: Aditya R <arajan@redhat.com>
In kubernetes 1.25, the default source for the pause image is now
registry.k8s.io instead of k8s.gcr.io.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Split out into its own commit, because it's a big one
to review.
The same code was literally copy-pasted a bunch of times, with
the only difference being the use of $doomeddir vs $destdir in
a couple of spots. Refactor, and while we're at it refactor the
tests for expected matches and mismatches.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Closes: #2029
Approved by: rhatdan
PR #1935 removed the default --debug logging; so all run_buildah
calls now use the default (error). It is safe to remove unnecessary
instances of --log-level=error .
For ease of review, this commit was entirely machine-generated via:
$ perl -pi -e 's/ --log-level=error / /' *.bats
Signed-off-by: Ed Santiago <santiago@redhat.com>
Closes: #2029
Approved by: rhatdan
The previous log-level implementation does not seem to work, which is
now fixed and aligns to other projects like podman and CRI-O. Therefore
the `--debug` flag has been deprecated and is now hidden.
Added documentation as well as integration tests.
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
Closes: #1811
Approved by: rhatdan
Replace use of the kubernetes/pause image in tests with
k8s.gcr.io/pause, except for the case that expects that we'll get a
schema 1 image when we pull without specifying a tag, since the new
image is a schema 2 image.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1564
Approved by: vrothberg
This started off as bug fixes necessary to get BATS tests
actually working on RHEL8 (they weren't). It grew. I will
defend my actions in the first comment post.
Primary change: import some helpers from podman BATS tests,
most importantly 'run_buildah' and 'is'. The vast majority
of the changes you'll see here are of the form:
- run buildah ...
- [ $status = 0 ]
- [ check $output ]
+ run_buildah ... ! automatically checks status
+ is "$output" "..."
Also: chmod'ed some files -x. Necessary because rpmbuild tries
to be oh-so-clever about requirements, and when it sees an
executable file with a shebang line like '#!env bats' it
helpfully adds 'Requires: /usr/bin/bats' to the rpm, which
then fails to install because RHEL8 does not have bats.
Also: refactored duplicate code in a few places, by writing
and invoking module-specific helper functions.
Also: changed a handful of 'buildah's to run_buildah, in
order to get error checking and debug logging.
Also: added descriptive reasons to many "skip"s.
Also: selinux test: some tweakery to make it run on production
system (context is different if /usr/bin/buildah is chcon'ed
appropriately). I can't get this test to pass on Fedora from
a build dir, and I'm actually not convinced that this test
has ever passed, but let's see what CI shows.
Also: selinux test: skip broken test (#1465).
Also: version test: skip parts of it if running w/o sources.
Tests are now passing as root on RHEL8; rootless has numerous
failures which I don't believe are related to this PR.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Closes: #1472
Approved by: TomSweeneyRedHat
When committing an image to local storage, compressing the new layers
only wastes time because they need to be decompressed again when being
written to local storage.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1274
Approved by: rhatdan
Remove error-prone parts of the explicit push tests that push to a
directory without using the blob-cache. The proneness relates directly
to the fact that the layers of pulled images (i.e., the busybox base
image) are compressed with another gzip library the blobs in Buildah's
cache, which now use pgzip.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Check that when we use blob caching, the blobs that we commit and push
which came from the base image are indeed the same as the ones from the
base image.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1149
Approved by: rhatdan
Nalin Dahyabhai
Aditya R
Ed Santiago
Sascha Grunert
Valentin Rothberg