Add copier.Ensure() and copier.ConditionalRemove(), for controlling the
permissions and datestamps we set on multiple items we create with one
call (along with any parents), and selectively removing multiple items
with one call.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add a --timestamp flag to the "add" and "copy" CLIs, along with a
corresponding field in AddAndCopyOptions.
When a timestamp is set, we'll force the timestamp on data copied in to
be the specified value while reading it, so that the content will have
the specified datestamp in the rootfs and when the image is committed.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
This function always returns nil as the first parameter, which makes
unparam linter sad.
Rather than adding //nolint:unparam, let's just move nil to actual
returns.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
These:
> copier/copier.go:1664:2: QF1003: could use tagged switch on hdr.Typeflag (staticcheck)
> if hdr.Typeflag == tar.TypeReg {
> ^
> pkg/parse/parse.go:708:4: QF1003: could use tagged switch on arr[1] (staticcheck)
> if arr[1] == "local" {
> ^
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Handle glob patterns with "**" path components by expanding "**" to the
set of subdirectories and calling filepath.Glob() on the results.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When we get a tried-to-write-to-closed-pipe error while encoding
something for a coprocess, try to capture error output from the
coprocess and add it to the error message, to hopefully catch a flake
we're seeing in CI.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
This allows podman to import the copier package on the remote client
without needing the full libimage package.
Based on Miloslav's work: https://github.com/containers/podman/pull/19718
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Fixes golangci-lint message:
`tar.TypeRegA has been deprecated since Go 1.11 and an alternative has
been available since Go 1.1: Use TypeReg instead.`
Signed-off-by: Chris Evich <cevich@redhat.com>
In golang 1.19, `io/ioutil` is fully deprecated preventing Buildah from
compiling. Replace all calls with equivalent calls from the `os`
package.
Signed-off-by: Chris Evich <cevich@redhat.com>
When noting that a non-symlink has setuid/setgid/sticky bits, switch
from using "syscall" package bits and syscall.Chmod() to using "os"
package bits and os.Chmod(), and if the item's a directory, record the
updated mode information in the "directoryModes" map that we'll use to
reset its permissions later.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
This call to os.Mkdir got missed the first time I tried to add the
workaround for the FreeBSD EISDIR bug since I was only testing with
'buildah run'. Trying to use 'buildah add' triggers the same bug whan
trying to extract an archive into a container's filesystem.
[NO NEW TESTS NEEDED]
Signed-off-by: Doug Rabson <dfr@rabson.org>
Podman adds an Error: to every error message. So starting an error
message with "error" ends up being reported to the user as
Error: error ...
This patch removes the stutter.
Also ioutil.ReadFile errors report the Path, so wrapping the err message
with the path causes a stutter.
Signed-off-by: Daniel J Walsh dwalsh@redhat.com
[NO NEW TESTS NEEDED]
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
If errors for which os.IsExist() or os.IsNotExist() would have returned
true have been wrapped using fmt.Errorf()'s "%w" verb, os.IsExist() and
os.IsNotExist(), not having been retrofitted to use errors.Is(), will
return false.
Use errors.Is() to check if an error is an os.ErrExist or os.ErrNotExist
error instead of calling os.IsExist() or os.IsNotExist().
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
We now use the golang error wrapping format specifier `%w` instead of
the deprecated github.com/pkg/errors package.
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
On FreeBSD, os.Mkdir can return EISDIR if the target is the root. As a
workaround, treat EISDIR as a non-error similarly to EEXISTS.
[NO NEW TESTS NEEDED]
Signed-off-by: Doug Rabson <dfr@rabson.org>
Similar to the `NoOverwriteDirNonDir` one, add an option that disables
non-directories from being overwritten by directories.
Required-for: containers/podman/issues/14420
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
do not attempt to load NSS modules at init() time but do it only when
it is needed before a chroot.
[NO NEW TESTS NEEDED]
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Since we started calling into the standard library's io/fs package
directly, we effectively made Go 1.16 our minimum Go version, so we
don't need to keep the workaround for compiling with Go 1.12.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add StripSetuidBit/StripSetgidBit/StripStickyBit flags to
copier.PutOptions, that are interpreted similarly to their counterparts
in copier.GetOptions.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Try to improve our ability to write to directories that aren't
writable. If we encounter an EPERM error while attempting to create an
item, attempt to temporarily make writable the directory that we're
writing the item to, and restore its permissions on our way out.
The error usually isn't seen when run as UID 0, whether in a user
namespace or not, which is usually how we're called, but running the
unit tests as an unprivileged user will verify it.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
it is needed to fix a failing test.
commit 985eec5391 changed the behavior
since the directory is not opened first before adding it.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Should speed up most walks escpecially if they don't need to
stat every directory entry.
[ NO NEW TESTS NEEDED]
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
When checking if something that we want to overwrite with a directory is
already a directory or not, use lstat instead of stat. If it's a
symbolic link, it's not a directory.
This is a subtle behavior change, but it's in line with docker build.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When we attempt to remove a directory to make way for a non-directory as
part of extracting content, use RemoveAll() instead of Remove().
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When processing a directory tree, only descend into a directory that is
marked for exclusion if its path is literally a prefix of an exception
pattern.
Subtly, but in a way that's compatible with docker, this means that if
we exclude directory "subdir", but we've been told to also include
"**/file" (with an exclusion pattern of "!**/file"), we won't descend
into "subdir" and find a file named "subdir/file", because "**/file"
doesn't start with "subdir/".
More generally, exclusion patterns that start with "!" which include any
wildcards before their final component technically won't be treated
correctly.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Set extended attributes on files _after_ setting their ownership, so
that security-sensitive attributes ("security.capability" among them)
won't get quietly cleared from under us if we set them before calling
chown().
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
During a copy operation which descends through a directory tree,
It's possible for a referenced file to become inaccessible (by unlink
or permission change or whatever). During the walk of paths to copy,
an `Lstat()` is run on each item, and any error passed into the handler
function to deal with. Subsequently, if there is no error, the file
is examined for inclusion/exclusion by the handler.
Unfortunately, this introduces a TOCTOU race condition for files which
become inaccessible even if they would otherwise be excluded. For
example a file or directory under /proc or /sys (which frequently and
unpredictably change). This was the original cause encountered during
podman integration testing.
It's impractical to actually fix this race at the file-level, without
introducing negative effects to any source-container operations. It's
also questionably useful to offer a command-line option to offload the
choice to the user. Instead, follow the behavior of the `tar` command
for this situation: Issue a warning to the user, and ignore the
problematic item (don't copy it).
Also add a test resembling the podman test which originally caught this
race. While not reliable, it does introduce a non-zero chance of
hitting the race condition - and handling the new warning properly.
Signed-off-by: Chris Evich <cevich@redhat.com>
In Docker if you are copying more then one object, and
one of them is successful, then the command is successful. Currently in
buildah each glob has to be successful. This PR matches Buildah to
Docker.
Fixes: https://github.com/containers/podman/issues/9594
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Add a NoCrossDevice flag to GetOptions, telling it to ignore
subdirectories on devices different than the top reference directory
that we start from, i.e., ignore the contents of mounted filesystems.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add an IgnoreUnreadable flag to copier.GetOptions to suppress errors
from copier.Get() that would pass the os.IsPermission() test, if they're
encountered while attempting to read files or descend into directories.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Always create the destination directory first when ADDing or COPYing
content into a container, then extract contents into it using the
destination directory as the chroot instead of the container's root
directory.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add a NoDerefSymlinks flag to force items that are matched to the Globs
we're given to be treated as symlinks, rather than dereferencing them as
we would need to do for sources for ADD or COPY.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
openshift-merge-bot[bot]
Nalin Dahyabhai
Sascha Brawer
Kir Kolyshkin
David Negstad
Jan Rodák
Paul Holzinger
Chris Evich
Doug Rabson
Daniel J Walsh
Sascha Grunert
Valentin Rothberg
Giuseppe Scrivano
OpenShift Merge Robot