When generating layer diffs or extracting container contents, scrub the
user and group name fields in tar headers before saving them.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
If errors for which os.IsExist() or os.IsNotExist() would have returned
true have been wrapped using fmt.Errorf()'s "%w" verb, os.IsExist() and
os.IsNotExist(), not having been retrofitted to use errors.Is(), will
return false.
Use errors.Is() to check if an error is an os.ErrExist or os.ErrNotExist
error instead of calling os.IsExist() or os.IsNotExist().
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
We now use the golang error wrapping format specifier `%w` instead of
the deprecated github.com/pkg/errors package.
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
As per `OCI` `image-spec` setting history is optional so buildah must
support these use-cases via `--omit-history=true` some of the use-cases
are discussed below.
* There are use-cases when image-spec contributes to the image size by
adding build history to image-spec in certain use-cases this image
history is not required by the end-users but there is no way to remove
it. See: https://github.com/containers/buildah/issues/3513
* Certain build tools don't honor setting build history in image spec
however buildah does not allow processing images without build history
since setting build history is still optional as per `OCI` image-spec
( https://github.com/opencontainers/image-spec/blob/main/config.md#properties)
so buildah must support such use-cases.
Closes: https://github.com/containers/buildah/issues/4025
Closes: https://github.com/containers/buildah/issues/3513
Signed-off-by: Aditya R <arajan@redhat.com>
Allows end-users to export final build content or rootfs to external formats.
By default, a local container image is created from the build result. The --output (or -o) flag allows you to override this behavior, and a specify a custom exporter. For example, custom exporters allow you to export the build artifacts as files on the local filesystem instead of a Container image, which can be useful for generating local binaries, code generation etc.
The value for --output is a CSV-formatted string defining the exporter type and options. Currently, local and tar exporters are supported. The local exporter writes the resulting build files to a directory on the client side. The tar exporter is similar but writes the files as a single tarball (.tar).
```console
buildah build --output type=local,dest=dir .
buildah build --output type=tar,dest=rootfs.tar .
buildah build -o dir .
```
Reference: https://docs.docker.com/engine/reference/commandline/build/#custom-build-outputs
Signed-off-by: Aditya R <arajan@redhat.com>
In image-spec 1.0.2, the MediaType field is now available in OCI
manifests, so set it.
Update the imgtype helper to output the MediaType field as-is from OCI
image manifests instead of just always supplying the expected value.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When creating a new image we can avoid hashing before and after compression
when we know compression will not be used. This significantly improves speed
for hashing large containers with --compress=false
There should be no functional differences so tests are not
changed/added.
[NO NEW TESTS NEEDED]
Signed-off-by: Kurt Kartaltepe <kkartaltepe@google.com>
Instead of extracting layer content to a temporary file when we're
committing to transports that aren't containers-storage, record the ID
of a layer and the uncompressed size it has recorded for its contents.
When later asked for a blob, if we cached a layer ID and size, generate
the layer diff on the fly, otherwise check for a file named after the
digest of the requested blob in our cache directory location (usually
used for new layers that we're adding) and the supplemental location
(which can be supplied by a caller).
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Remove the parent image's ID from the config blob when we're squashing
the image, since after squashing, we share no layers or history with
what was once our base image, and leaving it set triggers verification
errors in registries that expect consistency between parent IDs and
perhaps layers and history.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Golangs os.* functions return the name of the file/directory they
fail to use. We should not wrap these errors with the file/directory
to use names, causes stuttering when the user sees the errors, and looks
bad having huge error messages.
Since this is just code cleanup, existing tests should handle the
changes.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
every error from copier.Get was previously lost and not propagated
back. That caused the caller to believe the operation was successful
and lead to incomplete images on errors.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
This PR removes the pkg/auth which brings in docker/docker
since it really is not needed, and was only there to help users
discover the settings of where the authfile was, when the environment
variables were set. Would almost never be of any value.
Move imagebuildah.BuildOptions to define.BuildOptions
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Add the name of the base image being used by the build
in the comments of the first layer created.
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
We want to shrink the size of the import when importing pkg from
buildah. This should help us shrink the size of the golang bindings
in podman.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
If UID and GID mappings are specified, the container has a
mapped-layer, whose diffID is not computed when created.
Committing the image fails due to lack of diffID. This fix
computes diffID at creating an image source if a layer
doesn't have a diffID (UncompressedDigest).
This fix also tests if a container with UID and GID mappings
can be committed.
Signed-off-by: Hironori Shiina <shiina.hironori@fujitsu.com>
Golang built in functions like os.Create and others print the name of
the file system object when they fail. Wrapping them a second time
with the file system object, makes the error message look like crap
when reported to the user.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Avoid forcing the timestamps on content in new layers to the current
time when a timestamp is not specified.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
We recieved feedback on the --omit-timestamp that
users would rather specify the timestamp seconds
rather then just use EPOCH.
This PR removes --omit-timestamp from buildah bud
since this has never been released.
We also hide --omit-timestamp from buildah commit
and allow users to continue to use it, but it conflicts
with --timestamp.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Use the copier package to rework how we handle ADD and COPY.
When evaluating cache for content that's being copied/added in, switch
from (digest the data, check for a cache entry, then maybe copy the data
and create the new layer) to (copy the data and create the new layer,
digesting as we go, check for a cache entry, either commit or discard
the new layer).
Use the copier package for ADD, COPY, and for ensuring that a specified
directory exists in the working container's rootfs.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Currently you can only do deterministic builds with commit command
this change will cause the metadata in the container image to be
epoch 0.
Next step is to save the data in the tar balls as 0.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Remove these error checks since the instanceDigest is not used
and can just be ignored.
Signed-off-by: zvier <liuzekun0524@163.com>
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
For the sake of the Docker output format, always set the ContainerID
field, and if we have a non-nil Config, set ContainerConfig to the same
values.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1989
Approved by: rhatdan
Update the blobcache implementation to match API changes to the image
library.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1902
Approved by: rhatdan
If Squash is set, then we're only committing one layer, so if EmptyLayer
is set, which would normally tell us to skip committing the final layer,
we should ignore the EmptyLayer flag.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1839
Approved by: TomSweeneyRedHat
Don't try to use zstd to compress layers until we know what MIME type to
use to describe layers that are compressed with zstd.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1723
Approved by: TomSweeneyRedHat
Fix handling of ID mapping for COPY: when copying from other containers,
use their mappings, and when copying from the host, use host mappings.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1630
Approved by: TomSweeneyRedHat
When building an image with multiple layers, go back to committing
images for instructions for which we previously wouldn't bother
committing an image, but create them without adding a new layer.
This violates some assumptions that we currently make elsewhere, as it's
possible for an image that's derived from a base image to add no layers
relative to the base image, when previously it was always the case that
we'd add at least one whenever we committed it.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1539
Approved by: rhatdan
Instead of passing in an increasing number of the fields of
CommitOptions to makeImageRef(), just pass the whole thing.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1539
Approved by: rhatdan
Always set a parent ID when we go to commit an image, whether it's as
part of build-using-dockerfile or our "commit" CLI. Coerce the parent
image's ID directly into the value that we use instead of digesting it
again.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1538
Approved by: vrothberg
Set the CreatedBy field of the new image's new history item to the shell
if we don't have a different value to set.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
By default, a field called "createdAt" is written into the image manifest.
However, even if my image build process is perfectly deterministic, the image
sha256 hash will be different every time, even if my sources are exactly the
same. In many cases it is desirable that the same input results in the exactly
same output.
This commit introduces the flag --omit-timestamp to the commit command. If set to
true, the timestamp is set to epoch 0, instead of the current timestamp (which
causes different results every time commit is invoked).
Signed-off-by: Johannes Brüderl <johannes.bruederl@gmail.com>
Closes: #1294
Approved by: nalind
Add configuration methods for adding entries which will show up in a
committed image's history, both before and after the new layer that we
add while committing the image. Expose them from the CLI in the form of
a new --add-history option for the "add", "config", "copy", and "run"
commands.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1300
Approved by: rhatdan
NOTE: This is a very minimal update; blobinfocache.go should be
substantially reworked to benefit from the cache instead of decompressing
layers to get the DiffID values.
Also removes some HasBlob checks in the blobinfocache test suite which don't have
a direct equivalent in the new API, and makes the test suite independent
of host's signature policy.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Closes: #1218
Approved by: rhatdan
Add API hooks for designating locations to be used as blob caches when
pulling and pushing images. When we commit read-only copies of
container layers for use in images, if we're using blob caching, store a
copy of the layer in the blob cache directory so that it can be found.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1149
Approved by: rhatdan
Make sure that when attempting to diagnose an error, if we encounter an
error during the diagnostic attempt, we return the original error rather
than the error encountered in trying to diagnose it. Log that one.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1072
Approved by: rhatdan
Update github.com/containers/storage and github.com/containers/image,
and modify containerImageRef.NewImageSource() to account for the change
that the ReadCloser that store.Diff() returns now holds a lock until
it's closed, and because that's the same lock that store.PutLayer()
takes, we can't read more than one layer's contents at a time.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1024
Approved by: nalind
Nalin Dahyabhai
Sascha Grunert
Aditya R
Daniel J Walsh
Kurt Kartaltepe
Giuseppe Scrivano
Ashley
Diego
Hironori Shiina
zvier
Miloslav Trmač
Sascha Grunert
Johannes Brüderl
Valentin Rothberg
TomSweeneyRedHat
umohnani8