Commit Graph

1084 Commits

Author SHA1 Message Date
Aditya R ce8c567808
test: don't allow to override io.buildah.version
buildah now does not supports overriding default version label,
hence remove the test which expects the wrong behaviour.

See discussion here: https://github.com/containers/buildah/pull/4673#discussion_r1142063367

Signed-off-by: Aditya R <arajan@redhat.com>
2023-03-29 20:46:49 +05:30
Aditya R dfc9bb20e4
executor: only apply label on the final stage
Buildah currently applies labels to each intermediate image which is not
the right thing to do since others builds uses these intermediate image
and they can inherit the label even though they never had this
intention.

Only apply labels to the final build i.e last instruction of last stage
which will make sure that labels are only applied to final image and not
to the intermediate images.

Closes: https://github.com/containers/buildah/issues/4632

Signed-off-by: Aditya R <arajan@redhat.com>
2023-03-29 20:46:40 +05:30
Jack Wearden e84247b543
Merge branch 'main' into build-arg-file 2023-03-28 15:57:31 +01:00
Jack Wearden 3afa5ca74d build: accept arguments from file with --build-arg-file
Allows codifying build arguments into a structured file, perhaps one
named arguments.Containerargs

The build arg file also accepts comments starting #, so automated
tooling or CI/CD workflows can monitor arguments like versions to
ensure they are up-to-date.

Signed-off-by: Jack Wearden <jack@jackwearden.co.uk>
2023-03-25 11:04:23 +00:00
Giuseppe Scrivano 8e84bb7097
run_linux: Update heuristic for mounting /sys
change the heuristic to mount a fresh sysfs every time a new network
namespace is created.  This modification ensures the creation of the
sysfs when the network namespace is created, which is better than
sharing the one from the host.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=2164524

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2023-03-23 23:21:39 +01:00
Daniel J Walsh 61ba32647e
Merge pull request #4660 from danishprakash/img-id-len
imagebuildah: set len(short_image_id) to 12
2023-03-18 16:03:12 -04:00
Chris Evich 2088f6bd78
Fix requiring tests on Makefile changes
Changes to how things are built shouldn't trigger the
'pr-should-include-tests' check.

Signed-off-by: Chris Evich <cevich@redhat.com>
2023-03-16 10:26:41 -04:00
danishprakash b6f32e5766
imagebuildah: set len(short_image_id) to 12
* tests/bud.bats: add test

Signed-off-by: danishprakash <danish.prakash@suse.com>
2023-03-16 15:36:19 +05:30
Nalin Dahyabhai 499f1301f8 Re-enable conformance tests
Just ignore the new moby.buildkit.buildinfo.v1 field in the Docker
format.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2023-03-15 17:50:19 -04:00
Chris Evich fa145b0926
Skip conformance test failures with Docker 23.0.1
See issue https://github.com/containers/buildah/issues/4639

Signed-off-by: Chris Evich <cevich@redhat.com>
2023-03-14 17:18:34 -04:00
Aditya R 37afb5dbc7
run: add support for inline --network in RUN stmt
Buildah should allow clients to support inline --network in RUN stmts so users
can create isolate or expose a particular build containers.

```Dockerfile
FROM alpine
RUN --network=host wget google.com
RUN --network=none wget google.com
```

Closes: https://github.com/containers/buildah/issues/4230

Signed-off-by: Aditya R <arajan@redhat.com>
2023-03-14 20:13:49 +05:30
Aditya R db509859da
stage_executor: attempt to push cache only when cacheKey is valid
Buildah's layer caching does not generates cache key or uses a layer
from store if `RUN --mount` is used to make sure that always freshly
mounted content is used in the `RUN` instruction, in such case there is
no use of even pushing the cache since it will be never used by buildah.

Make sure that stage executor attempts to push cache only if cacheKey is
generated.

Closes: https://github.com/containers/buildah/issues/4647

Signed-off-by: Aditya R <arajan@redhat.com>
2023-03-09 16:59:03 +05:30
Daniel J Walsh 15114d97a1
Merge pull request #4634 from flouthoc/dont-warn-multi-arch
executor: don't warn about unused `TARGETARCH`,`TARGETOS`,`TARGETPLATFORM`
2023-03-02 08:40:03 -05:00
Aditya R 46ffaf9354
executor,copy: honor default ARG value while eval stage
COPY must honor processing any argument if any configured while
evaulating `--from=` statement.

Makes below Containerfile functional without any external `--build-arg`
value from CLI

```Dockerfile
ARG my_env=a

FROM alpine as stage_a
RUN /bin/true

FROM alpine
COPY --from=stage_${my_env} /bin/true true
```

Closes: https://github.com/containers/buildah/issues/4573

Signed-off-by: Aditya R <arajan@redhat.com>
2023-03-02 14:33:47 +05:30
Aditya R 9e6e15c314
tests/bud: remove unwated test
Expecting warning for the builtin defaults in not in sync with what
buildkit is doing or docker is doing so buildah should do the same.

Signed-off-by: Aditya R <arajan@redhat.com>
2023-03-02 11:37:12 +05:30
Aditya R 8a8f99e09b
executor: do not warn on builtin default args
Do not warn on builtin default args

Closes: https://github.com/containers/buildah/issues/4319

Signed-off-by: Aditya R <arajan@redhat.com>
2023-03-02 11:37:08 +05:30
Aditya R b9e4e29e90
executor: don't warn about unused TARGETARCH,TARGETOS,TARGETPLATFORM
Buildah warns about TARETARCH, TARGETOS, TARGETPLATFORM in a multi-arch
build however its not very useful for end-users and there are cases
where this can be left untouched so do not warn about test default build
args

Closes: https://github.com/containers/buildah/issues/4485

Signed-off-by: Aditya R <arajan@redhat.com>
2023-03-02 11:37:02 +05:30
Nalin Dahyabhai 0e707faa7b Ignore the base image's base image annotations
When initializing a stage in a multi-stage build that's based on the
end-result of an earlier stage, don't reset the base image name/digest
annotations.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2023-02-27 08:11:38 -05:00
Nalin Dahyabhai 0b7a7f80bb docs/tutorials/04: add defaults for Run()
In the tutorial, switch to calling `DefaultStoreOptionsAutoDetectUID()`
instead of `DefaultStoreOptions()`, which should figure things out
better.

In the tutorial, add an example of using Run(), where for API backward
compatibility reasons, we can't tell the difference between "grant no
capabilties by default" and "grant the default set of capabilities by
default".

The default isolation can be set automatically, so start doing that at
run-time, but have the tutorial look it up anyway because the tutorial
on the web will be newer than our current release for at least a while.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2023-02-21 08:59:43 -05:00
Nalin Dahyabhai 7baf73b2a5 imagebuildah.StageExecutor: suppress bogus "Pushing cache []:..."
When we changed the cache-from and cache-to options to take slices of
name values instead of single values, we didn't update logic that
checked if a value was set to handle cases where we had a non-nil slice
with no elements in it.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2023-02-17 13:15:27 -05:00
Aditya R be10b8d20c
executor: also add stage with no children to cleanupStages
While preparing list of cleanup stages, executor must add stage with no
children also to cleanupStages since a stage can also exist in the form
only `FROM base_image` i.e without any child nodes.

Ensures we leave no dandling build container for cases such as

```dockerfile
FROM quay.io/quay/busybox:latest AS busybox

FROM scratch

COPY --from=busybox /bin/busybox /bin/busybox
```

Closes: https://github.com/containers/buildah/issues/4572

Signed-off-by: Aditya R <arajan@redhat.com>
2023-02-17 09:55:33 +05:30
Ed Santiago 956c5155d5 Temporarily hard-skip bud-multiple-platform-values test
It is completely broken (see #4396) and is now causing failures
in Fedora gating tests:

   https://artifacts.dev.testing-farm.io/30e7b5bc-d162-4ae7-9a60-896f0186bf73/

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-02-15 11:50:50 -07:00
Daniel J Walsh 4dfb828909
Run codespell on codebase
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2023-02-09 09:05:03 -05:00
Aditya R ae697555cc
login: support interspersed args for password
Buildah must support accepting password from stdin agnostic of order
like `podman`

Example with buildah which should work

```console
$ cat password.txt | buildah login docker.io -u user --password-stdin
Error: too many arguments, login takes only 1 argument
```

Example with podman which already works
```console
$ cat password.txt | buildah login -u user --password-stdin docker.io
Login Succeeded
```

Closes: https://github.com/containers/buildah/issues/4557

Signed-off-by: Aditya R <arajan@redhat.com>
2023-02-06 12:43:31 +05:30
Aditya R 123ef1693b
conformance: use scratch for minimal test
Following test is just copying files it should not depend upon
`ubuntu:18.04` lets change that so CI never gets blocked.

Signed-off-by: Aditya R <arajan@redhat.com>
2023-02-01 16:18:43 +05:30
Daniel J Walsh 68d3b0ab76
Merge pull request #4531 from sstosh/test-proxy
tests: improve build-with-network-test
2023-01-25 08:24:05 -05:00
Toshiki Sonoda c9cbc6d7d2 tests: improve build-with-network-test
This test could be failed when we enable firewall.
To reduce the impact of firewall, we use "curl".

Signed-off-by: Toshiki Sonoda <sonoda.toshiki@fujitsu.com>
2023-01-25 16:20:09 +09:00
Ed Santiago fe0256d38f Flake 3710 has been closed. Reenable the test.
(No, I don't expect the issue to be fixed, but sometimes we need
incentives in order to pay attention to flakes)

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-01-23 05:56:39 -07:00
Aditya R ac7458e70d
stage_executor: while mounting stages use freshly built stage
When using `--mount=` in RUN instruction and source is a stage make sure
that freshly built stage is used if the stage selected in source was
just rebuilt.

Closes: https://github.com/containers/buildah/issues/4522

Signed-off-by: Aditya R <arajan@redhat.com>
2023-01-18 16:15:16 +05:30
Giuseppe Scrivano c0f6c6b7ad
[Itests: change the runtime-flag test for crun
crun might not print any debugging message, so change the runtime-flag
test to use --log=log-file and test it was created.

Closes: https://github.com/containers/buildah/issues/4503

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2023-01-17 17:37:53 +01:00
Nalin Dahyabhai 67ab55bbb7 bud: Consolidate multiple synthetic LABEL instructions
We handle --label command line arguments by appending LABEL instructions
to the Dockerfile contents before we parse it.  Previously, we were
appending a separate line for each label-value pair.  Consolidate them
for the sake of tools that arbitrarily limit the length of histories
that they're willing to accept in images (boo!).

Add a similar implementation for --env command line arguments.
Previously, we'd set them in the initial configuration for each stage
and also set them at commit-time, and that potentially overrode any
values that were explicitly in the stage itself, and which would have
affected RUN instructions.  Remove the set-at-commit-time logic so that
the history reflects what ends up in the image.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2023-01-09 16:45:10 -05:00
Aditya R 9fced965ee
build, secret: allow realtive mountpoints wrt to work dir
When working with `--mount=type=secret` allow `target`/`dst` to accept relative paths w.r.t to the configured work dir.

See detailed use-case here: https://github.com/containers/buildah/issues/4491

Closes: https://github.com/containers/buildah/issues/4491

**Steps to reproduce the issue from #4491:**

1. Create Dockerfile and Makefile:

Dockerfile:
```
FROM docker.io/ubuntu:22.04

WORKDIR /somedir

RUN --mount=type=secret,id=secret-foo,dst=secret1.txt --mount=type=secret,id=secret-bar,dst=secret2.txt \
     printf "PWD=%s\n" "$(pwd)" && ls -la && ls -la / && stat secret1.txt && stat secret2.txt && \
     cp secret1.txt /root/secret-foo.txt && \
     cp secret2.txt /root/secret-bar.txt
```

Makefile:
```
DOCKER ?= docker

.PHONY: build-container

build-container:
	rm -rf build
	mkdir build
	echo "secret:foo" >build/secret1.txt
	echo "secret:bar" >build/secret2.txt
	buildah --no-cache --secret id=secret-foo,src=build/secret1.txt --secret id=secret-bar,src=build/secret2.txt -t defanator/example:tag1 .
	podman run --rm -t -i defanator/example:tag1 cat /root/secret-foo.txt
	podman run --rm -t -i defanator/example:tag1 cat /root/secret-bar.txt
	podman rmi defanator/example:tag1
```
```
make
```

Signed-off-by: Aditya R <arajan@redhat.com>
2023-01-09 23:34:55 +05:30
Ed Santiago a75b263f75 system tests: remove unhelpful assertions
Regular primitive bats uses assertions like '[ $foo = something ]'.
These are worthless for debugging: when they fail, all you know
is that foo is not "something" but you don't know what foo _is_.

Find and replace those assertions with 'assert', which is
more informative. Instances found via:

   $ ack '^ *\[' tests/*.bats

There are many matches for 'test' (instead of '[') but those
mostly look like file-existence ones, which are less evil
than string-check tests. I'm leaving those be for now.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2023-01-05 08:20:25 -07:00
Aditya R 3566683896
buildah: add prune command and expose CleanCacheMount API
* Just like buildkit buildah must allow cleaning the buildcache and cache
generated on host by `--mount=type=cache` just like buildkit's `prune`
command.

* Also expose `CleanCacheMount` API so other tools like `podman` can use
  it.

See: https://github.com/moby/buildkit#cache
Closes: https://github.com/containers/buildah/issues/4486

Signed-off-by: Aditya R <arajan@redhat.com>
2023-01-05 17:32:42 +05:30
Daniel J Walsh f8eb73820a
Merge pull request #4452 from flouthoc/relative-mount
build, mount: allow realtive mountpoints wrt to work dir for `--mount=type=cache` and `--mount=type=bind`
2023-01-03 08:33:18 -05:00
Daniel J Walsh 60be7f2509
Add support for --group-add to buildah from
Allow containers running under buildah to use
--group-add keep-groups, so that they can inherit
access to the users groups.

Also allow users to add supplimental groups to the container.

Fixes: https://github.com/containers/buildah/issues/4476

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2022-12-22 11:53:00 -05:00
Aditya R 7a4702ae25
parse: default ignorefile must not point to symlink outside context
Drop support for remote use-cases when .containerignore or .dockerignore is a symlink pointing to arbitrary location on host.

[NO NEW TESTS NEEDED]
[NO TESTS NEEDED]

Signed-off-by: Aditya R <arajan@redhat.com>
2022-12-19 16:27:44 +05:30
Aditya R d9578d32cd
build, mount: allow realtive mountpoints wrt to work dir
When working with `--mount=type=bind` and `--mount=type=cache` allow
`target` to accept relative paths w.r.t to the configured work dir.

Closes: https://github.com/containers/buildah/issues/4309

Signed-off-by: Aditya R <arajan@redhat.com>
2022-12-15 16:30:34 +05:30
Daniel J Walsh c65d837641
Merge pull request #4443 from flouthoc/extend-cache-from-api
remote-cache: support multiple sources and destination for `--cache-to` and `--cache-from`
2022-12-12 14:41:48 -05:00
Chris Evich 46eea31588
Replace io/ioutil calls with os calls
In golang 1.19, `io/ioutil` is fully deprecated preventing Buildah from
compiling.  Replace all calls with equivalent calls from the `os`
package.

Signed-off-by: Chris Evich <cevich@redhat.com>
2022-12-06 14:29:32 -05:00
Aditya R ffed850367
remote-cache: support multiple sources and destinations
Buildah must support multiple sources for remote cache when using
`--cache-to` and `--cache-from` so users can distribute and collect
cache from various sources.

We also need for compat with buildkit api which in past broke `podman`
compat build API see: https://github.com/containers/podman/pull/16380

More discussion here: https://github.com/containers/podman/pull/16380

Signed-off-by: Aditya R <arajan@redhat.com>
2022-12-05 12:10:41 +05:30
Aditya R e6eb05f757
mount=type=cache: seperate cache parent on host for each user
`mount=type=cache` creates a common cache directory on host in temporary
directory, split this cache directory for each user invocation in order
to prevent overlapping of cache content when `buildah` is invoked by
different users on same host.

Signed-off-by: Aditya R <arajan@redhat.com>
2022-11-18 10:04:55 +05:30
Nalin Dahyabhai 3d755b5eb7 copier.Put(): clear up os/syscall mode bit confusion
When noting that a non-symlink has setuid/setgid/sticky bits, switch
from using "syscall" package bits and syscall.Chmod() to using "os"
package bits and os.Chmod(), and if the item's a directory, record the
updated mode information in the "directoryModes" map that we'll use to
reset its permissions later.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2022-11-09 15:38:40 -05:00
Daniel J Walsh d4c661a774
Add no-new-privileges flag
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2022-11-04 11:43:29 -04:00
Aditya R b2054360ae
imagebuildah:build with --all-platforms must honor args for base images
Commit:
217b2d524c
was pushed to `main` accidently while it incorrectly assumes that `ENV`
must be used while processing base image, however `ENV` should be only
evaluated inside `RUN` instruction and not for any containerfile
instruction such as `FROM <>`. This fixes change made in above commit.

Following commit ensures that imagebuildah considers `HeadingArgs`
and `UserArgs` when its evaluating base image to pull all images from
manifest list early when build is invoked with `--all-platforms`

Closes: https://github.com/containers/buildah/issues/4399

Signed-off-by: Aditya R <arajan@redhat.com>
2022-11-04 10:51:15 +05:30
Daniel J Walsh 217b2d524c
Expand args and env when using --all-platforms
Fixes: https://github.com/containers/buildah/issues/4399

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2022-11-03 16:57:15 -04:00
Daniel J Walsh a429bbddd5
Merge pull request #4366 from flouthoc/multi-stage-name-heading-args
executor: honor default `ARG` values while evaluating base image name.
2022-11-03 14:24:38 -04:00
Ed Santiago 1091222b27 Stop using ubi8
First: because we have a bunch of tests that rely on the image
manifest list, and ubi8 is not under our control: we've already
seen where ubi8 gets changed without our knowledge, requiring
almost a week of scrambling to get things fixed again.

Second, the registry it lives on is unreliable.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2022-11-03 09:18:53 -06:00
Ed Santiago cec864147c remove unnecessary (hence misleading) rmi
A recently-added test included an unnecessary "rmi --prune".
Unnecessary code causes brain fatigue in maintainers trying
to understand its purpose. Kill it.

I've confirmed that tests pass under buildah root & rootless
and podman local & remote.

Signed-off-by: Ed Santiago <santiago@redhat.com>
2022-11-02 07:29:00 -06:00
Aditya R a237085fe0
executor: honor default ARG value while eval base name
While PR https://github.com/containers/buildah/pull/3947 added support
for evaluating `--build-args` in base image names for a builds but it
missed processing default value if any. So for scenarios where `ARG`
already has a default value in Containerfile via `ARG key=value` but
was not specified with `--build-arg key=value` the processing ignored
the default value. Following commit just adds support for that.

Makes below Containerfile functional without any external `--build-arg`
value from CLI

```Dockerfile
ARG my_env=a

FROM alpine as stage_a
RUN /bin/true

FROM stage_${my_env} as stage_b
RUN /bin/true
```

Closes: https://github.com/containers/buildah/issues/4312

Signed-off-by: Aditya R <arajan@redhat.com>
2022-11-01 11:11:28 +05:30