When generating a checksum for files mounted into container via
`--mount=type=bind` ignore their `ModTime`, `AccessTime` and
`ChangeTime` so we can maintain cache burst consistency with `COPY`
command.
Closes: https://github.com/containers/buildah/issues/6291
Signed-off-by: flouthoc <flouthoc.git@gmail.com>
When using `--mount=type=bind` or `--mount=type=cache` the hash or
digest of source in these flags should be added to image history so
buildah can burst cache if files on host or image which is being used as
source is changed.
Signed-off-by: flouthoc <flouthoc.git@gmail.com>
While processing `FROM <token> as final` executor populates baseMap as
it is without resolving or processing for any ARG values. Following
commit ensures that we process resolve any ARG variables with ARG values
while populating baseMap so it can be used later to check if stage is
reused.
Fixes: https://github.com/containers/buildah/issues/3939
Signed-off-by: Aditya R <arajan@redhat.com>
In prepare(), don't check if the image name that it's passed is a
pseudonym for the result of a stage in the Dockerfile. Its callers
already did that.
When execute() knows that the image it's told to use as a base is a
pseudonym for the result of another stage in the Dockerfile, force the
pull policy to "never" to prevent an error when --pull-always=true.
Make imagebuildah.Mount a type alias instead of its own type, since we
never needed it to be a distinct type.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
This PR removes the pkg/auth which brings in docker/docker
since it really is not needed, and was only there to help users
discover the settings of where the authfile was, when the environment
variables were set. Would almost never be of any value.
Move imagebuildah.BuildOptions to define.BuildOptions
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Golang built in functions like os.Create and others print the name of
the file system object when they fail. Wrapping them a second time
with the file system object, makes the error message look like crap
when reported to the user.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2220: Scope build args to a single stage in a multi-stage build r=rhatdan a=carbonin
<!--
Thanks for sending a pull request!
Please make sure you've read and understood our contributing guidelines
(https://github.com/containers/buildah/blob/master/CONTRIBUTING.md) as well as ensuring
that all your commits are signed with `git commit -s`.
-->
#### What type of PR is this?
<!--
Please label this pull request according to what type of issue you are
addressing, especially if this is a release targeted pull request.
Uncomment only one `/kind <>` line, hit enter to put that in a new line, and
remove leading whitespace from that line:
-->
/kind bug
#### What this PR does / why we need it:
This PR uses the changes made in https://github.com/openshift/imagebuilder/pull/151 to handle arguments the same way `docker build` does. In particular, it scopes arguments to the stage in which they are defined and only records arguments in a layer's history if they could have been used in that layer.
#### How to verify it
```
$ cat Dockerfile
FROM alpine
ARG THING
FROM alpine
RUN echo "$THING" > things
$ buildah bud --layers --build-arg THING=things
STEP 1: FROM alpine
STEP 2: ARG THING
--> Using cache 232af6ca4a94e52dbef13f6da08c62b4172eaff7ee2e93cab08aceb4b00e6f81
STEP 3: FROM alpine
STEP 4: RUN echo "$THING" > things
--> Using cache fad9788d65a3062cc823516c8fff73b39e914463c709149a2855cbea61a10abe
fad9788d65a3062cc823516c8fff73b39e914463c709149a2855cbea61a10abe
$ podman run --rm fad9788d65a3062cc823516c8fff73b39e914463c709149a2855cbea61a10abe cat things
things
```
The above `podman run` command should return an empty file
#### Which issue(s) this PR fixes:
<!--
Automatically closes linked issue when PR is merged.
Uncomment the following comment block and include the issue
number or None on one line.
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`, or `None`.
-->
Fixes#2210
#### Does this PR introduce a user-facing change?
<!--
If no, just write `None` in the release-note block below. If yes, a release note
is required: Enter your extended release note in the block below. If the PR
requires additional action from users switching to the new release, include the
string "action required".
For more information on release notes please follow the kubernetes model:
https://git.k8s.io/community/contributors/guide/release-notes.md
-->
Yes, it significantly changes the way arguments behave. While it's not a change to how `buildah` would be used. Images built with the same Dockerfile before and after this change could be very different.
```release-note
Altered the behavior of the `--build-arg` flag and `ARG` commands to mirror `docker build`.
In particular, the following behaviors have changed:
- An ARG is only available after its ARG command in the current stage.
- Previously, anything provided using the --build-arg flag could be accessed in any stage. After this change, accessing a build arg provided on the command line will require a corresponding `ARG` command in the stage before it is accessed.
- Additionally, "heading" args (ARG commands before the first FROM) also now require an additional ARG declaration in the stage to be accessed. Previously, they were accessible without the additional ARG command.
- A later ARG default value should override an earlier one in the same stage
FROM alpine
ARG FOO=foo
ARG FOO=bar
RUN echo "$FOO"
The above Dockerfile should print "bar". Previously, the behavior was the opposite, an arg was not changed once set.
Generally this makes buildah handle args as described in https://docs.docker.com/engine/reference/builder/#arg
```
Co-authored-by: Nick Carboni <ncarboni@redhat.com>
Previously, every build-arg was recorded in the image history
even if the arg was not accessible to that layer when the build
was run.
This commit fixes that by ensuring that args are only added to
history when they are in scope. This is tracked in the
imagebuilder.Builder struct which is now accessible through
the stage reference in the StageExecutor
Fixes#2210
Signed-off-by: Nick Carboni <ncarboni@redhat.com>
Stealing @nalind 's workaround to avoid refetching
content after a file read failure. Under the right
circumstances that could be a symlink to a file meant
to overwrite a good file with bad data.
Testing:
```
goodstuff
[1] 14901
127.0.0.1 - - [24/Mar/2020 20:15:50] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [24/Mar/2020 20:15:50] "GET / HTTP/1.1" 200 -
no FROM statement found
goodstuff
```
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
If PullOptions/BuilderOptions/CommitOptions/PushOptions includes a
MaxRetries value other than 0, retry operations except for (currently)
connection-refused, authentication, and no-such-repository/no-such-tag
errors, at a default-but-configurable interval of 5 seconds.
Set the default for `buildah pull/from/commit/push` to 3 retries at 2
second intervals.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
If the context directory was specified as a git repo with a
branch reference ala
```
buildah bud --layers -t test git://github.com/containers/skopeo#master
```
The internal `git clone` command executed by buildah would fail as the
`#master` branch specification needed to be removed and specified with
the `-b` option like:
```
git clone -b master git://github.com/containers/skopeo /var/tmp/buildah12332
```
rather than:
```
git clone git://github.com/containers/skopeo#master /var/tmp/buildah12332
```
Addresses #1934
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
Closes: #1941
Approved by: rhatdan
Some consumers of the docker command line API expect to be able to pass
a tar or a Dockerfile into the build command via stdin, which causes a
build failure when it hits 'buildah bud' or 'podman build', which calls
buildah. A good example of this is the `linuxkit pkg build $dir`
command, which will append a `-` to the build command.
To improve compatibility with `docker build` support for `-` is added to
mean "read from stdin".
While a user could pass /dev/stdin or /proc/self/fd/0, or a path to some
other character device, as such tricks are common when attempting to get
programs to read from stdin, `docker build` does not support this, thus
this leaves out a bit of complexity while improving 'docker build' cli
compatibility.
Signed-off-by: Sachi King <nakato@nakato.io>
Closes: #1870
Approved by: rhatdan
During the unpacking of remote context for '-'(stdin), child commit,
the possibility for the remote context processing to leave behind a
stale context directory in the temporary directory `/var/run` was called
out.
As the context is unpacked into a temporary directory, it should be safe
to remove the directory and all files under it. The files under the
path should only ever be the content of an unpacked tar, a checked out
git repository, or a lone "Dockerfile".
Signed-off-by: Sachi King <nakato@nakato.io>
Closes: #1870
Approved by: rhatdan
Git repositories can also be cloned via http{s}, so add additional
checks for a ".git" suffix.
Fixes: #1609
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Closes: #1611
Approved by: TomSweeneyRedHat
We currently prepend a "FROM" instruction to the full set of
instructions for any images which are referenced in "COPY --from"
instructions that we don't create during the build. Make sure that the
list doesn't include any duplicates.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1474
Approved by: vrothberg
imagebuildah.ReposToMap is only used by the CLI, so move it there.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1406
Approved by: TomSweeneyRedHat
If you do a
podman bud http://example.com/Dockerfile
podman currently assumes the file is an archive, if it is not an archive
we should assume it is a Dockerfile, download and create a Dockerfile in the
directory.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Closes: #687
Approved by: TomSweeneyRedHat
Use Errorf() from 'errors' rather than 'fmt' to help with stack traces.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #130
Approved by: rhatdan
Impove error reporting by wrapping all returned err functions with
error.Wrapf
Signed-off-by: Dan Walsh <dwalsh@redhat.com>
Closes: #124
Approved by: nalind
Signed-off-by: Dan Walsh <dwalsh@redhat.com>
Closes: #125
Approved by: nalind
Have imagebuildah reexport some constants and its own Mount type, to
reduce the number of our dependencies that a prospective consumer of
this package would also need to import directly.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #84
Approved by: rhatdan
Add a build-using-dockerfile command (alias: bud) which uses
openshift/imagebuilder to wrap parsing and dispatching, and runc (or
another OCI runtime) to handle RUN instructions.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #59
Approved by: rhatdan
flouthoc
Aditya R
Nalin Dahyabhai
Daniel J Walsh
bors[bot]
Nick Carboni
TomSweeneyRedHat
Sachi King
WanLinghao
Valentin Rothberg
Qi Wang