This allows callers to provide custom SourceLookupReferenceFunc and
DestinationLookupReferenceFunc for cache pull/push. These can be used to
implement custom blob caches, and to wrap the reference being
pushed/pulled to influence the copy behavior.
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
We now use the golang error wrapping format specifier `%w` instead of
the deprecated github.com/pkg/errors package.
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
After clean install, it is necessary to decide the network backend
before any image is pulled so that `netavark` is chosen correctly.
Without this change, if `buildah pull` is executed at first, the network
backend is not determined and an image is pulled. This results in
choosing `cni` at a next command because an image already exists while
`netavark` is chosen if `buildah from` or `buildah bud` is called at
first.
Signed-off-by: Hironori Shiina <shiina.hironori@jp.fujitsu.com>
Buildah's pkg/blobcache has been moved into containers/image [1] to
consolidate implementations of interfaces such as image destinations
and image sources.
Since there are no functional changes: [NO NEW TESTS NEEDED]
[1] https://github.com/containers/image/pull/1459
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Update Buildah to the latest libimage. Migrating Podman over to
libimage entailed a number of fixes and changes to libimage which
we need to account for in Buildah.
Most notably:
* `(*Runtime).LookupImage()` now returns `storage.ErrImageUnknown`
instead of `nil` in case no matching image is found.
* `(*Runtime).LookupImage()` now does quite a bit more work finding
a local image and will also look at the repotags (or digests) of
all local images if needed.
* The signature of `(*Runtime).RemoveImages()` was changed and now
returns a slice of reports and errors. The reports aggregate the
data of a removed image which allows the function to be used by
`podman image prune` which is also interested in the size of the
removed data. The slice of errors is also needed in Podman which
needs to have a closer look at _all_ rmi errors in order to determine
the appropriate exit code (Docker compat).
* `libimage/types` has been removed. Pull policies have been merged
into already existing logic in `pkg/config`.
Please refer to containers/podman/pull/10147 for a more detailed
changelog.
[NO NEW TESTS NEEDED]
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Restore the push and pull API that commit dcd2a92e56 removed.
These changes would break vendoring into openshift/builder due
to build errors.
For the same reason, restore `util.FindImage` and `util.AddImageNames`
but deprecate the `findRegistry` argument.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Move all code related handling container image over to the new
`libimage` package in containers/common. The new package is an
attempt to consolidate the code across the containers tools under the
github.com/containers umbrella.
The new `libimage` packages provides functionality to perform all kinds
of operations for managing images such as local lookups, pushing,
pulling, listing, removing, etc.
The following packages have been moved over the containers/common:
`manifests` -> `common/image/manifests`
`pkg/manifests` -> `common/pkg/manifests`
`pkg/supplemented` -> `common/pkg/supplemented`
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
We want to shrink the size of the import when importing pkg from
buildah. This should help us shrink the size of the golang bindings
in podman.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1900822
Currently we are not passing the TMPDIR to the destination context
so when copying to containers storage we always fall back to /var/tmp.
This change sets the destination to match the source, so we can better
handle the storage.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
If the registry is set to insecure allowd using BUILD_REGISTRY_SOURCES, hardcode to skip the tls verify to avoid the errors.
Returns error if set insecureRegistries but force to use tls-verify.
Signed-off-by: Qi Wang <qiwan@redhat.com>
Pass our own API values for retry delays to common's retry package when
we use it to handle retrying image pull/commit/push operations.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
If PullOptions/BuilderOptions/CommitOptions/PushOptions includes a
MaxRetries value other than 0, retry operations except for (currently)
connection-refused, authentication, and no-such-repository/no-such-tag
errors, at a default-but-configurable interval of 5 seconds.
Set the default for `buildah pull/from/commit/push` to 3 retries at 2
second intervals.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add the --sign-by option to `buildah build-using-dockerfile`,
`buildah commit`, `buildah push`, and `buildah manifest push`. Add the
`--remove-signatures` option to `buildah pull`, `buildah push`, and
`buildah manifest push`. We just pass them to the image library, which
does all of the heavy lifting.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #2085
Approved by: rhatdan
If $BUILD_REGISTRY_SOURCES is set, expect it to be a valid
github.com/openshift/api/config/v1.Image, and parse its
AllowedRegistries and BlockedRegistries lists when, pulling, committing,
or pushing images.
Override the local signature policy when committing or pushing an image
to ensure that local storage is always allowed.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1787
Approved by: rhatdan
This is a breaking change and modifies the resulting image name when
pull from an directory via `oci:...` or `dir:...`.
Without this patch, the image names pulled via a local directory got
prefixed with `docker.io/{library/}`, which is not correct.
We now use either the full path to the image, or the relative path as
image name, but prefixed with `localhost` to indicate that the image is
not being pulled from a remote location.
Fixes: https://github.com/containers/buildah/issues/1797
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
Closes: #1800
Approved by: nalind
This commit enabled the `unparam` linter and applies all reported issues.
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
Closes: #1719
Approved by: rhatdan
Avoid calling fmt.Printf() to print things in library logic, which can't
be controlled or suppressed by callers. Prefer returning values and
printing them in our CLI wrapper, as callers would.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1596
Approved by: rhatdan
should omit type ... from declaration of var ...; it will be
inferred from the right-hand side
Signed-off-by: Douglas Schilling Landgraf <dougsland@redhat.com>
Closes: #1426
Approved by: nalind
Miloslav had some good comments on a previous commit.
https://github.com/containers/buildah/pull/1411
These changes address his issues by removing them.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Closes: #1412
Approved by: mtrmac
Currently rootless podman attempts to write to /var/lib/containers/cache
and fails. This causes us to repeatedly push images that have already been
pushed. This cache directory should be relative to the location of containers/storage
and not always stored in the same directory.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Closes: #1411
Approved by: TomSweeneyRedHat
The code is already calling docker.GetRepositoryTags
immediately below, so the dependency already exists, and this
removes an unnecessary dependency on alltransports.
Should not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Closes: #1361
Approved by: rhatdan
Use a typed value, to hopefully decrease further temptation to process strings
manually, and to avoid the unnecessary alltransports.ParseImageName which
resolveImage has already called.
This may change the strings used in some error/debug messages, which
now use transports.ImageName instead of the original input; the strings
should by definition have the same semantics.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Closes: #1361
Approved by: rhatdan
Right now, we (conceptually unnecesarily) require an image with an existing
tag on the remote repository to list all other tags.
Given that, use the user-specified name:tag, if any, instead of discarding the
tag and requiring :latest to exist on the remote registry.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Closes: #1361
Approved by: rhatdan
It should always be redundant with the reference itself; so,
use srcRef.StringWithinTransport() in the cases where we do
need to understand and hard-code the string syntax, after all.
Also improve the oci: format parsing a bit, to be robust
against including an image name.
NOTE: This might change the semantics a bit because StringWithinTransport
does not guarantee preserving the original string (e.g. paths
tend to be normalized not to contain symlinks). Using local paths
as docker/distribution image names is conceptually so problematic
that this seems worth the code cleanup - but I might be wrong.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Closes: #1361
Approved by: rhatdan
Both callers now consistently pass the transport in the "transport"
parameter, so parsing imageName could only be incorrect.
This could possibly fix cases like pulling docker://dir:localpath,
and the debug log will no longer say
"error parsing image name %q, trying with transport %q: %v" on every pull attempt.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Closes: #1361
Approved by: rhatdan
Should not change behavior, except possibly failing early if the server
returns an invalid tag name.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Closes: #1361
Approved by: rhatdan
pullImage has a dedicated transport: parameter, don't pass the transport
in the image name as well. The semantics of the imageName parameter to
pullImage is now unambiguous.
Should not change behavior, pullImage was trying
alltransports.ParseImageName(imageName) first.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Closes: #1361
Approved by: rhatdan
The result of ParseImageName("docker://"...) is not a storageRef.
Should not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Closes: #1361
Approved by: rhatdan
For some reason, the CI does not report any of these; on macOS
I see many more reports (including complaints about the standard
library), this only cleans up the trivial cases.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Closes: #1365
Approved by: rhatdan
Change references to Transfer to transfer to make it internal only.
It should be determined from the image specification and only determined
in one place.
Make buildah.Pull use registries.conf
Currently buildah pull does not resolve images based on registries.conf
This does not match the behaviour of buildah from or buildah bud
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Closes: #1319
Approved by: rhatdan
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
Add Quiet to the PullOptions and PushOptions structures.
If set to true output will be in 'quiet' mode. This
will primarily be used by callers such as OpenShift.
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
Closes: #1302
Approved by: rhatdan
Add API hooks for designating locations to be used as blob caches when
pulling and pushing images. When we commit read-only copies of
container layers for use in images, if we're using blob caching, store a
copy of the layer in the blob cache directory so that it can be found.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1149
Approved by: rhatdan
Copying an image now returns the manifest of the written image. Return
the digest of that manifest, and a canonical reference, from our Commit
and Push APIs.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1097
Approved by: rhatdan
Use the value now returned by util.ResolveImage instead of trying to
recompute it.
Then drop the no longer used getRegistries. (It might be reasonable
to split that part of util.ResolveImage to make it shorter; but it should
not ideally have any independent second-guessing callers. So, just
keep the inlined one instead; that way we certainly don't break it.)
Also drop the no longer used hasRegistry.
CHANGES BEHAVIOR:
- Most notably, the "short name but no search registries" code
has been broken for some time; pullImage was called with
localhost/$shortname, which was a qualified name, so the
specialized error handling was never attempted.
- Temporarily, the error handling in the "short name but no
search registries" code trigers even if there were actually
valid values to try (in practice there is always localhost/$shortname,
and possibly also options.Registry/$shortname). The next commit
will improve it again.
- We now have more legitimate access to the original short name,
so include it in the error message (it was technically available
before, but using it was awkward).
NOTE: registriesConfPath is computed using the sysregistries
package, but actual access happens using the sysregistriesv2 package.
That should be cleaned up eventually.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Closes: #909
Approved by: rhatdan
Nalin Dahyabhai
Aaron Lehmann
Sascha Grunert
Hironori Shiina
Valentin Rothberg
Valentin Rothberg
Daniel J Walsh
OpenShift Merge Robot
Qi Wang
Brandon Lum
Miloslav Trmač
Sascha Grunert
Douglas Schilling Landgraf
TomSweeneyRedHat
Zhou Hao