Commit Graph

17 Commits

Author SHA1 Message Date
Valentin Rothberg dcd2a92e56 use new containers/common/libimage package
Move all code related handling container image over to the new
`libimage` package in containers/common.  The new package is an
attempt to consolidate the code across the containers tools under the
github.com/containers umbrella.

The new `libimage` packages provides functionality to perform all kinds
of operations for managing images such as local lookups, pushing,
pulling, listing, removing, etc.

The following packages have been moved over the containers/common:

`manifests` -> `common/image/manifests`
`pkg/manifests` -> `common/pkg/manifests`
`pkg/supplemented` -> `common/pkg/supplemented`

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-04-22 17:39:00 +02:00
Daniel J Walsh 08201577e5
Remove some stuttering on returns errors
Golangs os.* functions return the name of the file/directory they
fail to use.  We should not wrap these errors with the file/directory
to use names, causes stuttering when the user sees the errors, and looks
bad having huge error messages.

Since this is just code cleanup, existing tests should handle the
changes.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-04-03 06:03:13 -04:00
Nalin Dahyabhai 12a17e5dfa blobcache: fix sequencing error
Wait for the goroutine that's saving a decompressed copy of the blob to
finish before checking the digest that it's computing, to avoid a race
condition which would cause us to read the digest value before we're
finished computing it.
Refuse to get involved when compression isn't gzip, pending a bigger
update to handle zstd correctly.
Set CompressionOperation and MediaType when suggesting substitutions.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2021-03-29 13:22:57 -04:00
Nalin Dahyabhai 289a250a2f blobcache: avoid an unnecessary NewImage()
Avoid calling NewImage() on the source reference when the ImageSource
that we already have returns nil from its LayerInfosForCopy() method.

For our container-as-image references, that causes a re-extraction of
uncached blobs, recomputing their digests, rebuilding the config blobs
and manifests.

The image library's copy.Image() function then asks the source reference
that we're wrapping for blobs that we listed in the rebuilt manifest,
and if any of those values differ, we fail to read them.

This would have only affected builds that specified a blob cache.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2020-07-31 10:05:32 -04:00
Nalin Dahyabhai 0817460525 Update for manifest-lists API changes in containers/image
Update the blobcache implementation to match API changes to the image
library.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>

Closes: #1902
Approved by: rhatdan
2019-10-28 15:15:34 +00:00
Nalin Dahyabhai f0cf07bb60 Move to containers/image v5.0.0
Bump to containers/image's 5.0 release.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>

Closes: #1902
Approved by: rhatdan
2019-10-28 15:15:34 +00:00
Miloslav Trmač 797e618cbe Update c/image to v4.0.1
This requires updating all import paths throughout.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1891
Approved by: vrothberg
2019-10-04 07:34:03 +00:00
Sascha Grunert 6b214d2921 Add golint linter and apply fixes
This commit enabled to golint linter in golangci-lint and applies all
necessary fixes.

Signed-off-by: Sascha Grunert <sgrunert@suse.com>

Closes: #1740
Approved by: rhatdan
2019-07-27 09:45:49 +00:00
Sascha Grunert 1d35489ca0
Add misspell linter and apply fixes
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
2019-07-18 10:45:52 +02:00
Valentin Rothberg acb46bfc7b blobcache: log error when draining the pipe
Reported by golangci-lint.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>

Closes: #1624
Approved by: TomSweeneyRedHat
2019-06-17 18:45:00 +00:00
Antonio Murdaca e7ca330f92 pkg/blobcache: add synchronization
Signed-off-by: Antonio Murdaca <runcom@linux.com>

Closes: #1265
Approved by: nalind
2019-01-08 00:27:29 +00:00
Valentin Rothberg 08e942383b vendor parallel-copy from containers/image
Parallel copying of layers is currently supported when pulling from
a registry to the storage.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2019-01-04 17:23:09 +01:00
Miloslav Trmač 32543998a2 Inline blobCache.putBlob into blobCacheDestination.PutBlob
It's now the only caller.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1218
Approved by: rhatdan
2018-12-07 15:11:36 +00:00
Miloslav Trmač 43c7bf7d10 Move saveStream and putBlob near blobCacheDestination.PutBlob
This does not change a single line of the two functions, committed
separately to make reviews of actual changes easier.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1218
Approved by: rhatdan
2018-12-07 15:11:36 +00:00
Miloslav Trmač baeea1febb Remove BlobCache.PutBlob
It has no users, and removing it removes an unused code branch in the putBlob
implementation.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1218
Approved by: rhatdan
2018-12-07 15:11:36 +00:00
Miloslav Trmač b77856eae3 Update for API changes
NOTE: This is a very minimal update; blobinfocache.go should be
substantially reworked to benefit from the cache instead of decompressing
layers to get the DiffID values.

Also removes some HasBlob checks in the blobinfocache test suite which don't have
a direct equivalent in the new API, and makes the test suite independent
of host's signature policy.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>

Closes: #1218
Approved by: rhatdan
2018-12-07 15:11:36 +00:00
Nalin Dahyabhai e213ed78c3 Add a blob cache implementation
Add a blob cache reference type which wraps other reference types.  When
used to create source and destination images, it returns wrappers for
source and destination images created by the wrapped references.

The cache reference must be created with a specified directory.

Blobs that are written to a cache image destination are stored as-is on
disk.  When reading a blob from a cache image source, if a blob with the
matching digest (and if specified, a matching size) can be found on
disk, read it directly instead of reading it from the real image source.

If compression or decompression is requested, the cache may return a
related blob instead of an exactly-matching blob.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>

Closes: #1149
Approved by: rhatdan
2018-12-04 18:53:15 +00:00