What type of PR is this?
/kind feature
What this PR does / why we need it:
It implements --link for COPY and ADD instructions and enables the creation of
cachable layers that can be reused independently across builds.
Follows buildkit `--link` specifications
How to verify it
bats tests/bud.bats
Which issue(s) this PR fixes:
Fixes#4325
Does this PR introduce a user-facing change?
Yes, gives extra functionality to Containerfiles
Signed-off-by: Joshua Arrevillaga <2004jarrevillaga@gmail.com>
Add a --timestamp flag to the "add" and "copy" CLIs, along with a
corresponding field in AddAndCopyOptions.
When a timestamp is set, we'll force the timestamp on data copied in to
be the specified value while reading it, so that the content will have
the specified datestamp in the rootfs and when the image is committed.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Correctly report back error when attempting to create `Tmpdir` for a
given url source.
Also remove superfluous `isGitTag` from define/types.go since `git
fetch` correctly resolves by provided reference.
Closes: https://github.com/containers/podman/issues/25679
Signed-off-by: flouthoc <flouthoc.git@gmail.com>
Ensure that the temporary directory that we create is never itself the
top-level directory of the content that we're downloading, in case it's
an archive which includes a "." with weird permissions.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
runc/libcontainer/userns package moved to a new home; the old package
will be made obsoleted. Switch to the new package.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
When collecting items we matched using the passed-in globs during Add(),
be sure that globbing performed by Get() will match the items exactly,
by escaping special matching characters.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add CertPath and InsecureSkipTLSVerify flags to AddAndCopyOptions, and
connect the CLI flag values passed to `buildah add` and `buildah build`
so that Builder.Add() gets those.
Add MaxRetries and RetryDelay fields to AddAndCopyOptions, and connect
them to the values passed on the command line to `buildah add` and
`buildah build`.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
The build breaks trying to build libcontainer/userns which no longer
builds on FreeBSD. Fortunately we only need this for
userns.RunningInUserNS so this change moves that call to a linux-only
file and adds a stub for FreeBSD.
[NO NEW TESTS NEEDED]
Signed-off-by: Doug Rabson <dfr@rabson.org>
Make sure that `add` and `copy` handle relative paths given as input
correctly, both with and without a context directory being specified.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
In golang 1.19, `io/ioutil` is fully deprecated preventing Buildah from
compiling. Replace all calls with equivalent calls from the `os`
package.
Signed-off-by: Chris Evich <cevich@redhat.com>
Podman adds an Error: to every error message. So starting an error
message with "error" ends up being reported to the user as
Error: error ...
This patch removes the stutter.
Also ioutil.ReadFile errors report the Path, so wrapping the err message
with the path causes a stutter.
Signed-off-by: Daniel J Walsh dwalsh@redhat.com
[NO NEW TESTS NEEDED]
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
We now use the golang error wrapping format specifier `%w` instead of
the deprecated github.com/pkg/errors package.
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
Adding sources from URL using `ADD` instruction adds reponse to build
container even if it receives bad HTTP response, following behaviour is
not in parity with `docker` or `buildkit`.
Following commit ensures that `ADD` where source is external URL fails
on build step if we get bad HTTP response.
Example: Following containerfile should fail while building
```Dockerfile
FROM registry.fedoraproject.org/fedora:36
ADD https://mirror.init7.net/fedora/fedora/linux/releases/36/Server/x86_64/iso/Fedora-Server-netinst-x86_64-36-1.5.foo /
```
**Fixes: BZ#2102140**
Signed-off-by: Aditya R <arajan@redhat.com>
Users have accidently had a .containerignore or .dockerignore
file in their context directly which blocked all content.
Currently we report that no globs matched, but do not
identify where the globs came from.
This change is an attempt to add this data to the error.
Example: https://github.com/containers/buildah/issues/3318
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
When copying a single file to the workdir make sure to create the workdir
and not to overwrite it as file.
Fixescontainers/podman#10671
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
In Docker if you are copying more then one object, and
one of them is successful, then the command is successful. Currently in
buildah each glob has to be successful. This PR matches Buildah to
Docker.
Fixes: https://github.com/containers/podman/issues/9594
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Expand conformance test coverage for COPY --chown, and correct our
behavior when the argument is a single number: instead of assuming the
number is the UID and GID = 0, the GID should be the same as the UID.
This means that we can't quite use the same function for parsing the
arguments for COPY --chown and USER, as they evidently have different
defaults for the case where the argument is a single number.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Always create the destination directory first when ADDing or COPYing
content into a container, then extract contents into it using the
destination directory as the chroot instead of the container's root
directory.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When a COPY command was being targeted at an empty new directory,
the copy would fail. It could be "worked around" by putting a dummy
file into the directory.
Addresses: #2964
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
We want to shrink the size of the import when importing pkg from
buildah. This should help us shrink the size of the golang bindings
in podman.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
When a COPY command was being targeted at an empty new directory,
the copy would fail. It could be "worked around" by putting a dummy
file into the directory.
Addresses: #2964
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
Prior to 1.16, when ADDing contents to a working container, if we were
being run by an unprivileged user using a user namespace, content that
was a device node would be ignored.
Add a flag in copier.PutOptions that tells copier.Put() to ignore
entries that are either a device, or a hard link to a device.
Make buildah.Add() set the IgnoreDevices flag in PutOptions when
libcontainer says we're running in a user namespace.
Together, these two changes should restore the earlier behavior.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Incorporate the value of the --chown option when digesting content added
from a URL, so that build caching will take it into account. Add tests
to ensure that layer caching takes into account the changed ownership of
items that we're adding or copying.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Golang built in functions like os.Create and others print the name of
the file system object when they fail. Wrapping them a second time
with the file system object, makes the error message look like crap
when reported to the user.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
When a directly-named (or globbed) source directory for ADD or COPY is
marked for exclusion by .dockerignore, check if its name is a prefix for
any exceptions in the .dockerignore file, and if it is, check the
directory for things we need to include anyway.
This will miss exceptions where the pattern uses a wildcard for anything
but the final component.
When adding items, count items that are actually passed over the tar
pipe, rather than items scanned, so that we can correctly diagnose not
having found anything that we needed to copy under a directory that
would otherwise have been excluded.
In copierHandlerGet(), just don't discount any globbed directories that
are excluded.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When extracting archives that are added using ADD, don't override
permissions and ownership information. We regressed on this when we
switched to using the copier package to handle them.
Add a conformance test to prevent regressions on this.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add() is supposed to handle relative paths when the ContextDir value
passed to it is not set, but it hasn't been doing that correctly since
it was overhauled. Correct it to recognize relative paths at the
function start, when ContextDir is not set, by combining them with the
current directory to convert them to absolute paths.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
COPY --from was incorrectly discarding ownership information on files
copied from other layers, which unlike content copied from the build
context, should not default to being owned by 0:0.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Use the copier package to rework how we handle ADD and COPY.
When evaluating cache for content that's being copied/added in, switch
from (digest the data, check for a cache entry, then maybe copy the data
and create the new layer) to (copy the data and create the new layer,
digesting as we go, check for a cache entry, either commit or discard
the new layer).
Use the copier package for ADD, COPY, and for ensuring that a specified
directory exists in the working container's rootfs.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Don't start digesting the contents of any file that we end up skipping.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #2072
Approved by: giuseppe
When generating the list of exclusions to process .dockerignore
contents, don't include .dockerignore if we don't have a .dockerignore
file in the context directory. That way, if the file doesn't exist, and
the caller didn't pass in any patterns, we get no patterns instead of
just one ".dockerignore" pattern, and we can hit the faster copy path.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #2072
Approved by: giuseppe
Joshua Arrevillaga
Nalin Dahyabhai
Kir Kolyshkin
flouthoc
Jan Rodák
danishprakash
Doug Rabson
Jean-Francois Roy
Chris Evich
Daniel J Walsh
Sascha Grunert
Aditya R
Giuseppe Scrivano
dependabot[bot]
Paul Holzinger
Jakub Guzik
OpenShift Merge Robot
TomSweeneyRedHat
yangfeiyu20102011