buildah/tests/helpers.bash

207 lines
6.6 KiB
Bash
Raw Normal View History

#!/bin/bash
BUILDAH_BINARY=${BUILDAH_BINARY:-$(dirname ${BASH_SOURCE})/../buildah}
IMGTYPE_BINARY=${IMGTYPE_BINARY:-$(dirname ${BASH_SOURCE})/../imgtype}
TESTSDIR=${TESTSDIR:-$(dirname ${BASH_SOURCE})}
STORAGE_DRIVER=${STORAGE_DRIVER:-vfs}
PATH=$(dirname ${BASH_SOURCE})/..:${PATH}
BATS tests - extensive but minor cleanup This started off as bug fixes necessary to get BATS tests actually working on RHEL8 (they weren't). It grew. I will defend my actions in the first comment post. Primary change: import some helpers from podman BATS tests, most importantly 'run_buildah' and 'is'. The vast majority of the changes you'll see here are of the form: - run buildah ... - [ $status = 0 ] - [ check $output ] + run_buildah ... ! automatically checks status + is "$output" "..." Also: chmod'ed some files -x. Necessary because rpmbuild tries to be oh-so-clever about requirements, and when it sees an executable file with a shebang line like '#!env bats' it helpfully adds 'Requires: /usr/bin/bats' to the rpm, which then fails to install because RHEL8 does not have bats. Also: refactored duplicate code in a few places, by writing and invoking module-specific helper functions. Also: changed a handful of 'buildah's to run_buildah, in order to get error checking and debug logging. Also: added descriptive reasons to many "skip"s. Also: selinux test: some tweakery to make it run on production system (context is different if /usr/bin/buildah is chcon'ed appropriately). I can't get this test to pass on Fedora from a build dir, and I'm actually not convinced that this test has ever passed, but let's see what CI shows. Also: selinux test: skip broken test (#1465). Also: version test: skip parts of it if running w/o sources. Tests are now passing as root on RHEL8; rootless has numerous failures which I don't believe are related to this PR. Signed-off-by: Ed Santiago <santiago@redhat.com> Closes: #1472 Approved by: TomSweeneyRedHat
2019-04-02 05:56:29 +08:00
# Default timeout for a buildah command.
BUILDAH_TIMEOUT=${BUILDAH_TIMEOUT:-300}
BATS tests - extensive but minor cleanup This started off as bug fixes necessary to get BATS tests actually working on RHEL8 (they weren't). It grew. I will defend my actions in the first comment post. Primary change: import some helpers from podman BATS tests, most importantly 'run_buildah' and 'is'. The vast majority of the changes you'll see here are of the form: - run buildah ... - [ $status = 0 ] - [ check $output ] + run_buildah ... ! automatically checks status + is "$output" "..." Also: chmod'ed some files -x. Necessary because rpmbuild tries to be oh-so-clever about requirements, and when it sees an executable file with a shebang line like '#!env bats' it helpfully adds 'Requires: /usr/bin/bats' to the rpm, which then fails to install because RHEL8 does not have bats. Also: refactored duplicate code in a few places, by writing and invoking module-specific helper functions. Also: changed a handful of 'buildah's to run_buildah, in order to get error checking and debug logging. Also: added descriptive reasons to many "skip"s. Also: selinux test: some tweakery to make it run on production system (context is different if /usr/bin/buildah is chcon'ed appropriately). I can't get this test to pass on Fedora from a build dir, and I'm actually not convinced that this test has ever passed, but let's see what CI shows. Also: selinux test: skip broken test (#1465). Also: version test: skip parts of it if running w/o sources. Tests are now passing as root on RHEL8; rootless has numerous failures which I don't believe are related to this PR. Signed-off-by: Ed Santiago <santiago@redhat.com> Closes: #1472 Approved by: TomSweeneyRedHat
2019-04-02 05:56:29 +08:00
function setup() {
suffix=$(dd if=/dev/urandom bs=12 count=1 status=none | od -An -tx1 | sed -e 's, ,,g')
TESTDIR=${BATS_TMPDIR}/tmp${suffix}
rm -fr ${TESTDIR}
mkdir -p ${TESTDIR}/{root,runroot}
}
function starthttpd() {
pushd ${2:-${TESTDIR}} > /dev/null
go build -o serve ${TESTSDIR}/serve/serve.go
HTTP_SERVER_PORT=$((RANDOM+32768))
./serve ${HTTP_SERVER_PORT} ${1:-${BATS_TMPDIR}} &
HTTP_SERVER_PID=$!
popd > /dev/null
}
function stophttpd() {
if test -n "$HTTP_SERVER_PID" ; then
kill -HUP ${HTTP_SERVER_PID}
unset HTTP_SERVER_PID
unset HTTP_SERVER_PORT
fi
true
}
function teardown() {
stophttpd
rm -fr ${TESTDIR}
}
function createrandom() {
dd if=/dev/urandom bs=1 count=${2:-256} of=${1:-${BATS_TMPDIR}/randomfile} status=none
}
function buildah() {
${BUILDAH_BINARY} --debug --registries-conf ${TESTSDIR}/registries.conf --root ${TESTDIR}/root --runroot ${TESTDIR}/runroot --storage-driver ${STORAGE_DRIVER} "$@"
}
function imgtype() {
${IMGTYPE_BINARY} -debug -root ${TESTDIR}/root -runroot ${TESTDIR}/runroot -storage-driver ${STORAGE_DRIVER} "$@"
}
BATS tests - extensive but minor cleanup This started off as bug fixes necessary to get BATS tests actually working on RHEL8 (they weren't). It grew. I will defend my actions in the first comment post. Primary change: import some helpers from podman BATS tests, most importantly 'run_buildah' and 'is'. The vast majority of the changes you'll see here are of the form: - run buildah ... - [ $status = 0 ] - [ check $output ] + run_buildah ... ! automatically checks status + is "$output" "..." Also: chmod'ed some files -x. Necessary because rpmbuild tries to be oh-so-clever about requirements, and when it sees an executable file with a shebang line like '#!env bats' it helpfully adds 'Requires: /usr/bin/bats' to the rpm, which then fails to install because RHEL8 does not have bats. Also: refactored duplicate code in a few places, by writing and invoking module-specific helper functions. Also: changed a handful of 'buildah's to run_buildah, in order to get error checking and debug logging. Also: added descriptive reasons to many "skip"s. Also: selinux test: some tweakery to make it run on production system (context is different if /usr/bin/buildah is chcon'ed appropriately). I can't get this test to pass on Fedora from a build dir, and I'm actually not convinced that this test has ever passed, but let's see what CI shows. Also: selinux test: skip broken test (#1465). Also: version test: skip parts of it if running w/o sources. Tests are now passing as root on RHEL8; rootless has numerous failures which I don't believe are related to this PR. Signed-off-by: Ed Santiago <santiago@redhat.com> Closes: #1472 Approved by: TomSweeneyRedHat
2019-04-02 05:56:29 +08:00
#################
# run_buildah # Invoke buildah, with timeout, using BATS 'run'
#################
#
# This is the preferred mechanism for invoking buildah:
#
# * we use 'timeout' to abort (with a diagnostic) if something
# takes too long; this is preferable to a CI hang.
# * we log the command run and its output. This doesn't normally
# appear in BATS output, but it will if there's an error.
# * we check exit status. Since the normal desired code is 0,
# that's the default; but the first argument can override:
#
# run_buildah 125 nonexistent-subcommand
# run_buildah '?' some-other-command # let our caller check status
#
# Since we use the BATS 'run' mechanism, $output and $status will be
# defined for our caller.
#
function run_buildah() {
# Number as first argument = expected exit code; default 0
expected_rc=0
case "$1" in
[0-9]) expected_rc=$1; shift;;
[1-9][0-9]) expected_rc=$1; shift;;
[12][0-9][0-9]) expected_rc=$1; shift;;
'?') expected_rc= ; shift;; # ignore exit code
esac
# Remember command args, for possible use in later diagnostic messages
MOST_RECENT_BUILDAH_COMMAND="buildah $*"
BATS tests - extensive but minor cleanup This started off as bug fixes necessary to get BATS tests actually working on RHEL8 (they weren't). It grew. I will defend my actions in the first comment post. Primary change: import some helpers from podman BATS tests, most importantly 'run_buildah' and 'is'. The vast majority of the changes you'll see here are of the form: - run buildah ... - [ $status = 0 ] - [ check $output ] + run_buildah ... ! automatically checks status + is "$output" "..." Also: chmod'ed some files -x. Necessary because rpmbuild tries to be oh-so-clever about requirements, and when it sees an executable file with a shebang line like '#!env bats' it helpfully adds 'Requires: /usr/bin/bats' to the rpm, which then fails to install because RHEL8 does not have bats. Also: refactored duplicate code in a few places, by writing and invoking module-specific helper functions. Also: changed a handful of 'buildah's to run_buildah, in order to get error checking and debug logging. Also: added descriptive reasons to many "skip"s. Also: selinux test: some tweakery to make it run on production system (context is different if /usr/bin/buildah is chcon'ed appropriately). I can't get this test to pass on Fedora from a build dir, and I'm actually not convinced that this test has ever passed, but let's see what CI shows. Also: selinux test: skip broken test (#1465). Also: version test: skip parts of it if running w/o sources. Tests are now passing as root on RHEL8; rootless has numerous failures which I don't believe are related to this PR. Signed-off-by: Ed Santiago <santiago@redhat.com> Closes: #1472 Approved by: TomSweeneyRedHat
2019-04-02 05:56:29 +08:00
# stdout is only emitted upon error; this echo is to help a debugger
echo "\$ $BUILDAH_BINARY $*"
run timeout --foreground --kill=10 $BUILDAH_TIMEOUT ${BUILDAH_BINARY} --debug --registries-conf ${TESTSDIR}/registries.conf --root ${TESTDIR}/root --runroot ${TESTDIR}/runroot --storage-driver ${STORAGE_DRIVER} "$@"
BATS tests - extensive but minor cleanup This started off as bug fixes necessary to get BATS tests actually working on RHEL8 (they weren't). It grew. I will defend my actions in the first comment post. Primary change: import some helpers from podman BATS tests, most importantly 'run_buildah' and 'is'. The vast majority of the changes you'll see here are of the form: - run buildah ... - [ $status = 0 ] - [ check $output ] + run_buildah ... ! automatically checks status + is "$output" "..." Also: chmod'ed some files -x. Necessary because rpmbuild tries to be oh-so-clever about requirements, and when it sees an executable file with a shebang line like '#!env bats' it helpfully adds 'Requires: /usr/bin/bats' to the rpm, which then fails to install because RHEL8 does not have bats. Also: refactored duplicate code in a few places, by writing and invoking module-specific helper functions. Also: changed a handful of 'buildah's to run_buildah, in order to get error checking and debug logging. Also: added descriptive reasons to many "skip"s. Also: selinux test: some tweakery to make it run on production system (context is different if /usr/bin/buildah is chcon'ed appropriately). I can't get this test to pass on Fedora from a build dir, and I'm actually not convinced that this test has ever passed, but let's see what CI shows. Also: selinux test: skip broken test (#1465). Also: version test: skip parts of it if running w/o sources. Tests are now passing as root on RHEL8; rootless has numerous failures which I don't believe are related to this PR. Signed-off-by: Ed Santiago <santiago@redhat.com> Closes: #1472 Approved by: TomSweeneyRedHat
2019-04-02 05:56:29 +08:00
# without "quotes", multiple lines are glommed together into one
if [ -n "$output" ]; then
echo "$output"
fi
if [ "$status" -ne 0 ]; then
echo -n "[ rc=$status ";
if [ -n "$expected_rc" ]; then
if [ "$status" -eq "$expected_rc" ]; then
echo -n "(expected) ";
else
echo -n "(** EXPECTED $expected_rc **) ";
fi
fi
echo "]"
fi
if [ "$status" -eq 124 -o "$status" -eq 137 ]; then
# FIXME: 'timeout -v' requires coreutils-8.29; travis seems to have
# an older version. If/when travis updates, please add -v
# to the 'timeout' command above, and un-comment this out:
# if expr "$output" : ".*timeout: sending" >/dev/null; then
echo "*** TIMED OUT ***"
false
BATS tests - extensive but minor cleanup This started off as bug fixes necessary to get BATS tests actually working on RHEL8 (they weren't). It grew. I will defend my actions in the first comment post. Primary change: import some helpers from podman BATS tests, most importantly 'run_buildah' and 'is'. The vast majority of the changes you'll see here are of the form: - run buildah ... - [ $status = 0 ] - [ check $output ] + run_buildah ... ! automatically checks status + is "$output" "..." Also: chmod'ed some files -x. Necessary because rpmbuild tries to be oh-so-clever about requirements, and when it sees an executable file with a shebang line like '#!env bats' it helpfully adds 'Requires: /usr/bin/bats' to the rpm, which then fails to install because RHEL8 does not have bats. Also: refactored duplicate code in a few places, by writing and invoking module-specific helper functions. Also: changed a handful of 'buildah's to run_buildah, in order to get error checking and debug logging. Also: added descriptive reasons to many "skip"s. Also: selinux test: some tweakery to make it run on production system (context is different if /usr/bin/buildah is chcon'ed appropriately). I can't get this test to pass on Fedora from a build dir, and I'm actually not convinced that this test has ever passed, but let's see what CI shows. Also: selinux test: skip broken test (#1465). Also: version test: skip parts of it if running w/o sources. Tests are now passing as root on RHEL8; rootless has numerous failures which I don't believe are related to this PR. Signed-off-by: Ed Santiago <santiago@redhat.com> Closes: #1472 Approved by: TomSweeneyRedHat
2019-04-02 05:56:29 +08:00
fi
if [ -n "$expected_rc" ]; then
if [ "$status" -ne "$expected_rc" ]; then
die "exit code is $status; expected $expected_rc"
fi
fi
}
#########
# die # Abort with helpful message
#########
function die() {
echo "#/vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv" >&2
echo "#| FAIL: $*" >&2
echo "#\\^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^" >&2
false
}
########
# is # Compare actual vs expected string; fail w/diagnostic if mismatch
########
#
# Compares given string against expectations, using '=~' to allow patterns.
BATS tests - extensive but minor cleanup This started off as bug fixes necessary to get BATS tests actually working on RHEL8 (they weren't). It grew. I will defend my actions in the first comment post. Primary change: import some helpers from podman BATS tests, most importantly 'run_buildah' and 'is'. The vast majority of the changes you'll see here are of the form: - run buildah ... - [ $status = 0 ] - [ check $output ] + run_buildah ... ! automatically checks status + is "$output" "..." Also: chmod'ed some files -x. Necessary because rpmbuild tries to be oh-so-clever about requirements, and when it sees an executable file with a shebang line like '#!env bats' it helpfully adds 'Requires: /usr/bin/bats' to the rpm, which then fails to install because RHEL8 does not have bats. Also: refactored duplicate code in a few places, by writing and invoking module-specific helper functions. Also: changed a handful of 'buildah's to run_buildah, in order to get error checking and debug logging. Also: added descriptive reasons to many "skip"s. Also: selinux test: some tweakery to make it run on production system (context is different if /usr/bin/buildah is chcon'ed appropriately). I can't get this test to pass on Fedora from a build dir, and I'm actually not convinced that this test has ever passed, but let's see what CI shows. Also: selinux test: skip broken test (#1465). Also: version test: skip parts of it if running w/o sources. Tests are now passing as root on RHEL8; rootless has numerous failures which I don't believe are related to this PR. Signed-off-by: Ed Santiago <santiago@redhat.com> Closes: #1472 Approved by: TomSweeneyRedHat
2019-04-02 05:56:29 +08:00
#
# Examples:
#
# is "$actual" "$expected" "descriptive test name"
# is "apple" "orange" "name of a test that will fail in most universes"
# is "apple" "[a-z]\+" "this time it should pass"
#
function is() {
local actual="$1"
local expect="$2"
local testname="${3:-FIXME}"
if [ -z "$expect" ]; then
if [ -z "$actual" ]; then
return
fi
expect='[no output]'
elif [ "$actual" = "$expect" ]; then
return
elif [[ "$actual" =~ ^$expect ]]; then
BATS tests - extensive but minor cleanup This started off as bug fixes necessary to get BATS tests actually working on RHEL8 (they weren't). It grew. I will defend my actions in the first comment post. Primary change: import some helpers from podman BATS tests, most importantly 'run_buildah' and 'is'. The vast majority of the changes you'll see here are of the form: - run buildah ... - [ $status = 0 ] - [ check $output ] + run_buildah ... ! automatically checks status + is "$output" "..." Also: chmod'ed some files -x. Necessary because rpmbuild tries to be oh-so-clever about requirements, and when it sees an executable file with a shebang line like '#!env bats' it helpfully adds 'Requires: /usr/bin/bats' to the rpm, which then fails to install because RHEL8 does not have bats. Also: refactored duplicate code in a few places, by writing and invoking module-specific helper functions. Also: changed a handful of 'buildah's to run_buildah, in order to get error checking and debug logging. Also: added descriptive reasons to many "skip"s. Also: selinux test: some tweakery to make it run on production system (context is different if /usr/bin/buildah is chcon'ed appropriately). I can't get this test to pass on Fedora from a build dir, and I'm actually not convinced that this test has ever passed, but let's see what CI shows. Also: selinux test: skip broken test (#1465). Also: version test: skip parts of it if running w/o sources. Tests are now passing as root on RHEL8; rootless has numerous failures which I don't believe are related to this PR. Signed-off-by: Ed Santiago <santiago@redhat.com> Closes: #1472 Approved by: TomSweeneyRedHat
2019-04-02 05:56:29 +08:00
return
fi
# This is a multi-line message, which may in turn contain multi-line
# output, so let's format it ourself, readably
local -a actual_split
readarray -t actual_split <<<"$actual"
printf "#/vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv\n" >&2
printf "#| FAIL: $testname\n" >&2
printf "#| expected: '%s'\n" "$expect" >&2
printf "#| actual: '%s'\n" "${actual_split[0]}" >&2
local line
for line in "${actual_split[@]:1}"; do
printf "#| > '%s'\n" "$line" >&2
done
printf "#\\^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n" >&2
false
}
#######################
# expect_line_count # Check the expected number of output lines
#######################
#
# ...from the most recent run_buildah command
#
function expect_line_count() {
local expect="$1"
local testname="${2:-${MOST_RECENT_BUILDAH_COMMAND:-[no test name given]}}"
local actual="${#lines[@]}"
if [ "$actual" -eq "$expect" ]; then
return
fi
printf "#/vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv\n" >&2
printf "#| FAIL: $testname\n" >&2
printf "#| Expected %d lines of output, got %d\n" $expect $actual >&2
printf "#| Output was:\n" >&2
local line
for line in "${lines[@]}"; do
printf "#| >%s\n" "$line" >&2
done
printf "#\\^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n" >&2
false
}
BATS tests - extensive but minor cleanup This started off as bug fixes necessary to get BATS tests actually working on RHEL8 (they weren't). It grew. I will defend my actions in the first comment post. Primary change: import some helpers from podman BATS tests, most importantly 'run_buildah' and 'is'. The vast majority of the changes you'll see here are of the form: - run buildah ... - [ $status = 0 ] - [ check $output ] + run_buildah ... ! automatically checks status + is "$output" "..." Also: chmod'ed some files -x. Necessary because rpmbuild tries to be oh-so-clever about requirements, and when it sees an executable file with a shebang line like '#!env bats' it helpfully adds 'Requires: /usr/bin/bats' to the rpm, which then fails to install because RHEL8 does not have bats. Also: refactored duplicate code in a few places, by writing and invoking module-specific helper functions. Also: changed a handful of 'buildah's to run_buildah, in order to get error checking and debug logging. Also: added descriptive reasons to many "skip"s. Also: selinux test: some tweakery to make it run on production system (context is different if /usr/bin/buildah is chcon'ed appropriately). I can't get this test to pass on Fedora from a build dir, and I'm actually not convinced that this test has ever passed, but let's see what CI shows. Also: selinux test: skip broken test (#1465). Also: version test: skip parts of it if running w/o sources. Tests are now passing as root on RHEL8; rootless has numerous failures which I don't believe are related to this PR. Signed-off-by: Ed Santiago <santiago@redhat.com> Closes: #1472 Approved by: TomSweeneyRedHat
2019-04-02 05:56:29 +08:00
function check_options_flag_err() {
flag="$1"
[ "$status" -eq 1 ]
[[ $output = *"No options ($flag) can be specified after"* ]]
}