mirror of https://github.com/goharbor/harbor.git
Merge fb142cc598 into c004f2d3e6
This commit is contained in:
Cecilie
GitHub
commit
39311ec34e
|
|
@ -0,0 +1,107 @@
|
|||
# Harbor Backup and Restore Scripts (Contrib)
|
||||
|
||||
**Warning:** These scripts are provided as-is in the `contrib/backup-restore` directory. They are not officially maintained or supported by the Harbor project. Use them at your own risk and ensure you understand their functionality before running them in a production environment.
|
||||
|
||||
These scripts (`harbor-backup` and `harbor-restore`) are provided as a convenience for backing up and restoring your Harbor instance. They aim to back up the following components:
|
||||
|
||||
* Harbor Database (PostgreSQL)
|
||||
* Container Registry Data
|
||||
* Chart Museum Data (if enabled)
|
||||
* Redis Data (if enabled)
|
||||
* Secret Keys
|
||||
* Harbor Configuration (`harbor.yml`)
|
||||
|
||||
### Features
|
||||
Compared to the scripts the harbor project used to have in their repo this set of scripts is more robust in its error handling and also offers features
|
||||
for not packing the backup into a tarball. This makes it easy to rsync the whole backup directory to a secondary/standby node and restore there.
|
||||
|
||||
rsync is used extensively by the script. by leaving the files in the backup directory between runs the downtime for backup is greatly reduced at the
|
||||
expense of disk space usage.
|
||||
|
||||
Supports logging of status messages directly to syslog
|
||||
|
||||
## Prerequisites
|
||||
|
||||
* **Docker:** These scripts rely on the `docker` command-line interface to interact with Harbor's containers. Ensure Docker is installed and accessible on the machine where you run these scripts.
|
||||
* **Sufficient Permissions:** You'll need appropriate permissions (e.g., `sudo` or being in the `docker` group) to run Docker commands and perform file system operations.
|
||||
* **Stopped Harbor Instance:** You must stop your Harbor instance completely before running the `harbor-backup` or `harbor-restore` script to avoid data inconsistencies.
|
||||
|
||||
## Usage
|
||||
|
||||
### Backup (`harbor-backup`)
|
||||
|
||||
1. **Download the Scripts:** Place the `harbor-backup` script in a location accessible from your Harbor instance. Within the Harbor repository, this would typically be under `contrib/backup-restore/`.
|
||||
|
||||
2. **Make it Executable:**
|
||||
```bash
|
||||
chmod +x harbor-backup
|
||||
```
|
||||
|
||||
3. **Run the Backup Script:**
|
||||
```bash
|
||||
./harbor-backup [OPTIONS]
|
||||
```
|
||||
3. **Stop Harbor:** Ensure your Harbor instance is completely stopped before proceeding with the backup.
|
||||
|
||||
4. **Options:**
|
||||
* `--docker-cmd <command>`: Specify the Docker command to use (default: `docker`).
|
||||
* `--db-image <image>`: Specify the Harbor database image to use for the temporary backup container (default: auto-detected). It's generally recommended to let it auto-detect.
|
||||
* `--db-path <path>`: Harbor DB data path (default: `/data/database`). Adjust if your deployment uses a different path.
|
||||
* `--registry-path <path>`: Registry data path (default: `/data/registry`). Adjust if your deployment uses a different path.
|
||||
* `--chart-museum-path <path>`: Chart Museum data path (default: `/data/chart_storage`). Adjust if your deployment uses a different path.
|
||||
* `--redis-path <path>`: Redis data path (default: `/data/redis`). Adjust if your deployment uses a different path.
|
||||
* `--secret-path <path>`: Secret data path (default: `/data/secret`). Adjust if your deployment uses a different path.
|
||||
* `--config-path <path>`: Harbor configuration file path (default: `/etc/harbor/harbor.yml`). Adjust if your deployment uses a different path.
|
||||
* `--backup-dir <path>`: Directory where the backup will be stored (default: `harbor_backup`).
|
||||
* `--no-archive`: Do not create a `tar.gz` archive of the backup directory. The backup will remain as a directory structure in `$BACKUP_DIR/harbor`.
|
||||
* `--use-syslog`: Use syslog for logging output.
|
||||
* `--log-level <level>`: Set the logging level (default: `INFO`, options: `DEBUG`, `INFO`, `NOTICE`, `WARNING`, `ERROR`, `CRITICAL`, `ALERT`, `EMERGENCY`).
|
||||
* `--help`: Display this help message.
|
||||
|
||||
5. **Backup Location:** By default, the backup will be created in a directory named `harbor_backup` in the current working directory. If the `--no-archive` option is not used, the final backup will be a compressed tarball named `harbor_backup.tar.gz` within the `harbor_backup` directory.
|
||||
|
||||
### Restore (`harbor-restore`)
|
||||
|
||||
1. **Download the Scripts:** Place the `harbor-restore` script in a location accessible from your Harbor instance. Within the Harbor repository, this would typically be under `contrib/backup-restore/`.
|
||||
|
||||
2. **Make it Executable:**
|
||||
```bash
|
||||
chmod +x harbor-restore
|
||||
```
|
||||
|
||||
3. **Stop Harbor:** Ensure your Harbor instance is completely stopped before proceeding with the restore.
|
||||
|
||||
4. **Run the Restore Script:**
|
||||
```bash
|
||||
./harbor-restore [OPTIONS]
|
||||
```
|
||||
|
||||
5. **Options:** The restore script accepts similar options to the backup script, allowing you to specify the Docker command, database image, data paths, and the backup directory.
|
||||
|
||||
* `--backup-dir <path>`: **Crucially**, this should point to the directory containing your Harbor backup (either the `harbor` subdirectory extracted from the tarball or the `harbor_backup` directory if `--no-archive` was used).
|
||||
* `--no-archive`: Use this option if your backup is already extracted into the `$BACKUP_DIR/harbor` directory. If your backup is a `tar.gz` file, **do not** use this option; the script will attempt to extract it.
|
||||
|
||||
*(Other options like `--docker-cmd`, `--db-image`, `--db-path`, `--registry-path`, `--chart-museum-path`, `--redis-path`, `--secret-path`, `--config-path`, `--use-syslog`, and `--log-level` function similarly to the backup script.)*
|
||||
|
||||
6. **Restore Process:** The script will:
|
||||
* Start a temporary database container.
|
||||
* Extract the backup archive (if not using `--no-archive`).
|
||||
* Drop and recreate existing Harbor databases.
|
||||
* Restore the database content from the backed-up SQL files.
|
||||
* Synchronize the registry, chart museum, Redis, and secret data directories.
|
||||
* Restore the Harbor configuration file.
|
||||
* Clean up the temporary database container.
|
||||
|
||||
7. **Restart Harbor:** Once the restore script completes successfully, you can restart your Harbor instance.
|
||||
|
||||
## Important Notes
|
||||
|
||||
* **Backup Consistency:** For a consistent backup, it's recommended to stop your Harbor instance or at least ensure minimal write activity during the backup process.
|
||||
* **Database Image Tag:** In production environments, it's advisable to use a specific tag for the `--db-image` option in both the backup and restore scripts to ensure consistency.
|
||||
* **Custom Deployments:** If you have a highly customized Harbor deployment with data stored in non-default locations, you **must** use the appropriate command-line options to point the scripts to the correct paths.
|
||||
* **Testing:** Always test the backup and restore process in a non-production environment before relying on it for critical data.
|
||||
* **Unsupported:** Remember that these scripts are provided in the `contrib/backup-restore/` directory. They may not be actively maintained, and you might encounter issues. Contributions and improvements from the community are welcome.
|
||||
|
||||
## Contributing
|
||||
|
||||
If you find issues or have improvements to these scripts, feel free to submit pull requests to the Harbor project in the `contrib/backup-restore/` directory.
|
||||
|
|
@ -0,0 +1,375 @@
|
|||
#!/bin/bash
|
||||
#
|
||||
# Harbor Backup Script
|
||||
#
|
||||
|
||||
# Default values
|
||||
DOCKER_CMD="docker"
|
||||
HARBOR_DB_IMAGE="$(docker images goharbor/harbor-db --format "{{.Repository}}:{{.Tag}}" | head -1)"
|
||||
HARBOR_DB_PATH="/data/database"
|
||||
REGISTRY_DATA_PATH="/data/registry"
|
||||
CHART_MUSEUM_PATH="/data/chart_storage"
|
||||
REDIS_DATA_PATH="/data/redis"
|
||||
SECRET_PATH="/data/secret"
|
||||
BACKUP_DIR="harbor_backup"
|
||||
BACKUP_FILE="harbor_backup.tar.gz"
|
||||
HARBOR_YML="/etc/harbor/harbor.yml"
|
||||
# Default log level (can be overridden by environment variable)
|
||||
LOG_LEVEL="INFO"
|
||||
# Whether to syslog (can be overridden by environment variable)
|
||||
USE_SYSLOG=false
|
||||
POSTGRES_UID=999
|
||||
POSTGRES_GID=999
|
||||
|
||||
|
||||
# Log levels (syslog-compatible)
|
||||
declare -A LOG_LEVELS=(
|
||||
[DEBUG]=7
|
||||
[INFO]=6
|
||||
[NOTICE]=5
|
||||
[WARNING]=4
|
||||
[ERROR]=3
|
||||
[CRITICAL]=2
|
||||
[ALERT]=1
|
||||
[EMERGENCY]=0
|
||||
)
|
||||
|
||||
|
||||
usage() {
|
||||
echo "Usage: $0 [OPTIONS]"
|
||||
echo ""
|
||||
echo "Backs up a Harbor instance."
|
||||
echo ""
|
||||
echo "Options:"
|
||||
echo " --docker-cmd <command> Docker command (default: docker)"
|
||||
echo " --db-image <image> Harbor DB image (default: auto-detected)"
|
||||
echo " --db-path <path> Harbor DB data path (default: /data/database)"
|
||||
echo " --registry-path <path> Registry data path (default: /data/registry)"
|
||||
echo " --chart-museum-path <path> Chart Museum data path (default: /data/chart_storage)"
|
||||
echo " --redis-path <path> Redis data path (default: /data/redis)"
|
||||
echo " --secret-path <path> Secret data path (default: /data/secret)"
|
||||
echo " --config-path <path> Harbor configuration file path (default: /etc/harbor/harbor.yml)"
|
||||
echo " --backup-dir <path> Backup directory (default: harbor_backup)"
|
||||
echo " --no-archive Do not create a tarball"
|
||||
echo " --use-syslog Use syslog for logging"
|
||||
echo " --log-level <level> Log level (default: INFO, options: DEBUG, INFO, NOTICE, WARNING, ERROR, CRITICAL, ALERT, EMERGENCY)"
|
||||
echo " --help Display this help message"
|
||||
exit 0
|
||||
}
|
||||
|
||||
# exit immediately if any command fails, an unset variable is used, or a pipe fails
|
||||
set -euo pipefail
|
||||
|
||||
# Parse command-line arguments
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--docker-cmd)
|
||||
DOCKER_CMD="$2"
|
||||
shift 2
|
||||
;;
|
||||
--db-image)
|
||||
HARBOR_DB_IMAGE="$2"
|
||||
shift 2
|
||||
;;
|
||||
--db-path)
|
||||
HARBOR_DB_PATH="$2"
|
||||
shift 2
|
||||
;;
|
||||
--registry-path)
|
||||
REGISTRY_DATA_PATH="$2"
|
||||
shift 2
|
||||
;;
|
||||
--chart-museum-path)
|
||||
CHART_MUSEUM_PATH="$2"
|
||||
shift 2
|
||||
;;
|
||||
--redis-path)
|
||||
REDIS_DATA_PATH="$2"
|
||||
shift 2
|
||||
;;
|
||||
--secret-path)
|
||||
SECRET_PATH="$2"
|
||||
shift 2
|
||||
;;
|
||||
--config-path)
|
||||
HARBOR_YML="$2"
|
||||
shift 2
|
||||
;;
|
||||
--backup-dir)
|
||||
BACKUP_DIR="$2"
|
||||
shift 2
|
||||
;;
|
||||
--no-archive)
|
||||
NO_ARCHIVE=true
|
||||
shift
|
||||
;;
|
||||
--use-syslog)
|
||||
USE_SYSLOG=true
|
||||
shift
|
||||
;;
|
||||
--log-level)
|
||||
LOG_LEVEL="$2"
|
||||
shift 2
|
||||
;;
|
||||
--help)
|
||||
usage
|
||||
;;
|
||||
*)
|
||||
echo "Unknown option: $1"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
# log() will prepend datetime and log the message to stderr and optionally to syslog
|
||||
# if the optional log level (defaults to INFO) is equal to higher than
|
||||
# $LOG_LEVEL
|
||||
log() {
|
||||
local level
|
||||
local message
|
||||
|
||||
if [[ $# -eq 1 ]]; then # Check if only one argument is provided
|
||||
level="INFO" # Default level if not provided
|
||||
message="$1"
|
||||
elif [[ $# -eq 2 ]]; then # Check if two arguments are provided
|
||||
level="$1"
|
||||
message="$2"
|
||||
else
|
||||
echo "Usage: log [LEVEL] MESSAGE" >&2
|
||||
return 1 # Return error code
|
||||
fi
|
||||
|
||||
# Check if logging level is valid
|
||||
if [[ ! "${LOG_LEVELS[$level]+_}" ]]; then
|
||||
echo "Invalid log level: $level" >&2 # Log error to stderr
|
||||
level="INFO" # Fallback to INFO
|
||||
fi
|
||||
|
||||
# Check minimum log level
|
||||
if (( ${LOG_LEVELS[$level]} > ${LOG_LEVELS[$LOG_LEVEL]} )); then
|
||||
return # Don't log if below the threshold
|
||||
fi
|
||||
|
||||
local formatted_message="[$(date '+%Y-%m-%d %H:%M:%S')] [$level] $message"
|
||||
|
||||
echo "$formatted_message"
|
||||
|
||||
if [[ "$USE_SYSLOG" == "true" ]]; then
|
||||
local syslog_priority="local0.${level,,}" # Convert level to lowercase for syslog
|
||||
logger -t "harbor-backup" -p "$syslog_priority" "$message"
|
||||
fi
|
||||
}
|
||||
|
||||
|
||||
# Launch and wait for database
|
||||
launch_db() {
|
||||
if [ -n "$($DOCKER_CMD ps -q)" ]; then
|
||||
log CRITICAL "There are running containers, please stop and remove before backup"
|
||||
exit 1
|
||||
fi
|
||||
log "Starting database container for backup"
|
||||
docker run -d --name harbor-db -v "${BACKUP_DIR}":/backup -v "${HARBOR_DB_PATH}":/var/lib/postgresql/data "${HARBOR_DB_IMAGE}" "postgres"
|
||||
wait_for_db_ready
|
||||
}
|
||||
|
||||
# Wait for database to be ready
|
||||
wait_for_db_ready() {
|
||||
TIMEOUT=12
|
||||
set +e
|
||||
while [ $TIMEOUT -gt 0 ]; do
|
||||
docker exec harbor-db pg_isready | grep "accepting connections"
|
||||
if [ $? -eq 0 ]; then
|
||||
break
|
||||
fi
|
||||
TIMEOUT=$((TIMEOUT - 1))
|
||||
sleep 5
|
||||
done
|
||||
set -e
|
||||
if [ $TIMEOUT -eq 0 ]; then
|
||||
log CRITICAL "Harbor DB cannot reach within one minute."
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Dump databases
|
||||
dump_database() {
|
||||
log "Backing up databases"
|
||||
# Get list of databases
|
||||
databases=$(docker exec harbor-db psql -t -c "SELECT datname FROM pg_database WHERE datistemplate = false;")
|
||||
|
||||
if [[ -z "$databases" ]]; then
|
||||
log WARNING "No user databases found to backup."
|
||||
return 0 # Return success if no user databases are found
|
||||
fi
|
||||
|
||||
for db in $databases; do
|
||||
log "- $db"
|
||||
dump_file="${BACKUP_DIR}/harbor/db/${db}.sql"
|
||||
local pg_dump_error=$(${DOCKER_CMD} exec harbor-db bash -c "( pg_dump -U postgres ${db} || echo Exit: \$? >&2 ) 2>&1 > ${dump_file}" )
|
||||
local pg_dump_status="${pg_dump_error##*$'\n'}"
|
||||
if [[ "$pg_dump_status" =~ Exit:[[:space:]]*[1-9][0-9]* ]] ; then
|
||||
log ERROR "Failed to backup database $db: $pg_dump_error"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
done
|
||||
}
|
||||
|
||||
# Backup registry data
|
||||
backup_registry() {
|
||||
log "Backing up registry data"
|
||||
if ! rsync -a --delete "${REGISTRY_DATA_PATH}/" "${BACKUP_DIR}/harbor/registry/"; then
|
||||
log ERROR "Failed to backup registry data using rsync"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
}
|
||||
|
||||
# Backup chart museum data
|
||||
backup_chart_museum() {
|
||||
if [ -d "${CHART_MUSEUM_PATH}" ]; then
|
||||
log WARNING "Deprecated feature detected - Backing up chartmuseum data"
|
||||
if ! rsync -a --delete "${CHART_MUSEUM_PATH}/" "${BACKUP_DIR}/harbor/chart_storage/"; then
|
||||
log ERROR "Failed to backup chartmuseum data using rsync"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
# Backup redis data
|
||||
backup_redis() {
|
||||
if [ -d "${REDIS_DATA_PATH}" ]; then
|
||||
log "Backing up redis"
|
||||
if ! rsync -a --delete "${REDIS_DATA_PATH}/" "${BACKUP_DIR}/harbor/redis/"; then
|
||||
log ERROR "Failed to backup redis data using rsync"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
log "Not backing up redis ${REDIS_DATA_PATH} not found"
|
||||
fi
|
||||
}
|
||||
|
||||
# Backup secrets
|
||||
backup_secret() {
|
||||
# Pre 1.8
|
||||
OLD_SECRET_PATH=`basename "${SECRET_PATH}"`
|
||||
if [ -f "${OLD_SECRET_PATH}/secretkey" ]; then
|
||||
log INFO "Backing up secretkey (pre 1.8)"
|
||||
if ! cp "${OLD_SECRET_PATH}/secretkey" "${BACKUP_DIR}/harbor/secret/"; then
|
||||
log ERROR "Failed to backup secretkey"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
if [ -f "${OLD_SECRET_PATH}/defaultalias" ]; then
|
||||
log INFO "Backing up defaultalias (pre 1.8)"
|
||||
if ! cp "${OLD_SECRET_PATH}/defaultalias" "${BACKUP_DIR}/harbor/secret/"; then
|
||||
log ERROR "Failed to backup defaultalias"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# 1.8 and later (Using rsync for directory)
|
||||
if [ -d "${SECRET_PATH}" ]; then
|
||||
log INFO "Backing up secrets (1.8+)"
|
||||
if ! rsync -a --delete "${SECRET_PATH}/" "${BACKUP_DIR}/harbor/secret/"; then
|
||||
log ERROR "Failed to backup secrets"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
}
|
||||
|
||||
# Backup config
|
||||
backup_config() {
|
||||
if [ -f "${HARBOR_YML}" ]; then
|
||||
log "Backing up configuration file"
|
||||
cp "${HARBOR_YML}" "${BACKUP_DIR}/harbor/"
|
||||
else
|
||||
log "Not backing up configuration file (${HARBOR_YML} not found)"
|
||||
fi
|
||||
}
|
||||
|
||||
# Clean up database container
|
||||
clean_db() {
|
||||
log "Clean up temporary database container for backup"
|
||||
docker stop harbor-db
|
||||
docker rm harbor-db
|
||||
}
|
||||
|
||||
# Create tar archive
|
||||
create_tarball() {
|
||||
if [ -z "${NO_ARCHIVE}" ]; then
|
||||
log "Creating tarball ${BACKUP_DIR}/${BACKUP_FILE}"
|
||||
cd "${BACKUP_DIR}"
|
||||
tar -czvf --remove-files "${BACKUP_DIR}/${BACKUP_FILE}" harbor
|
||||
rm -rf harbor
|
||||
else
|
||||
log "Not creating tarball"
|
||||
fi
|
||||
}
|
||||
|
||||
create_backup_dir(){
|
||||
if [[ -e "${BACKUP_DIR}/harbor/db" ]] ; then
|
||||
log "Removing existing db backup directory ${BACKUP_DIR}/harbor/db"
|
||||
rm -rf "${BACKUP_DIR}/harbor/db"
|
||||
|
||||
fi
|
||||
log "Setting correct ownership and permissions"
|
||||
mkdir -p "${BACKUP_DIR}/harbor/db"
|
||||
mkdir -p "${BACKUP_DIR}/harbor/registry"
|
||||
mkdir -p "${BACKUP_DIR}/harbor/chart_storage"
|
||||
mkdir -p "${BACKUP_DIR}/harbor/redis"
|
||||
mkdir -p "${BACKUP_DIR}/harbor/secret"
|
||||
|
||||
chown "${POSTGRES_UID}:${POSTGRES_GID}" "${BACKUP_DIR}/harbor/db"
|
||||
chown 0:0 "${BACKUP_DIR}/harbor/secret"
|
||||
chown 0:0 "${BACKUP_DIR}/harbor/registry"
|
||||
chown 0:0 "${BACKUP_DIR}/harbor/chart_storage"
|
||||
chown 0:0 "${BACKUP_DIR}/harbor/redis"
|
||||
chown 0:0 "${BACKUP_DIR}/harbor"
|
||||
|
||||
chmod 700 "${BACKUP_DIR}/harbor/db"
|
||||
chmod 700 "${BACKUP_DIR}/harbor/secret"
|
||||
chmod 755 "${BACKUP_DIR}/harbor/registry"
|
||||
chmod 755 "${BACKUP_DIR}/harbor/chart_storage"
|
||||
chmod 755 "${BACKUP_DIR}/harbor/redis"
|
||||
chmod 755 "${BACKUP_DIR}/harbor"
|
||||
}
|
||||
|
||||
#
|
||||
# main
|
||||
#
|
||||
|
||||
log "Starting backup of harbor"
|
||||
|
||||
# Create backup directory
|
||||
create_backup_dir
|
||||
|
||||
# Trap signals for cleanup
|
||||
trap clean_db EXIT ERR INT TERM
|
||||
|
||||
# Launch container for database backup
|
||||
launch_db
|
||||
|
||||
# Dump the databases
|
||||
dump_database
|
||||
|
||||
# Back up registry files
|
||||
backup_registry
|
||||
|
||||
# Back up chartmuseum files
|
||||
backup_chart_museum
|
||||
|
||||
# Back up redis files
|
||||
backup_redis
|
||||
|
||||
# Back up secrets
|
||||
backup_secret
|
||||
|
||||
# Back up configuration files
|
||||
backup_config
|
||||
|
||||
# Create archive (if not disabled)
|
||||
create_tarball
|
||||
|
||||
log "Ending backup of harbor"
|
||||
|
|
@ -0,0 +1,335 @@
|
|||
#!/bin/bash
|
||||
#
|
||||
# Harbor Restore Script
|
||||
#
|
||||
|
||||
# Default values (match the backup script)
|
||||
DOCKER_CMD="docker"
|
||||
HARBOR_DB_IMAGE="$(docker images goharbor/harbor-db --format "{{.Repository}}:{{.Tag}}" | head -1)" # Use a specific tag in production
|
||||
HARBOR_DB_PATH="/data/database"
|
||||
REGISTRY_DATA_PATH="/data/registry"
|
||||
CHART_MUSEUM_PATH="/data/chart_storage"
|
||||
REDIS_DATA_PATH="/data/redis"
|
||||
SECRET_PATH="/data/secret"
|
||||
BACKUP_DIR="harbor_backup"
|
||||
BACKUP_FILE="harbor_backup.tar.gz"
|
||||
HARBOR_YML="/etc/harbor/harbor.yml"
|
||||
NO_ARCHIVE="false"
|
||||
# Default log level
|
||||
LOG_LEVEL="INFO"
|
||||
USE_SYSLOG="false"
|
||||
POSTGRES_UID=999
|
||||
POSTGRES_GID=999
|
||||
|
||||
# Log levels (syslog-compatible)
|
||||
declare -A LOG_LEVELS=(
|
||||
[DEBUG]=7
|
||||
[INFO]=6
|
||||
[NOTICE]=5
|
||||
[WARNING]=4
|
||||
[ERROR]=3
|
||||
[CRITICAL]=2
|
||||
[ALERT]=1
|
||||
[EMERGENCY]=0
|
||||
)
|
||||
|
||||
usage() {
|
||||
echo "Usage: $0 [OPTIONS]"
|
||||
echo ""
|
||||
echo "Restores a Harbor instance."
|
||||
echo ""
|
||||
echo "Options:"
|
||||
echo " --docker-cmd <command> Docker command (default: docker)"
|
||||
echo " --db-image <image> Harbor DB image (default: auto-detected)"
|
||||
echo " --db-path <path> Harbor DB data path (default: /data/database)"
|
||||
echo " --registry-path <path> Registry data path (default: /data/registry)"
|
||||
echo " --chart-museum-path <path> Chart Museum data path (default: /data/chart_storage)"
|
||||
echo " --redis-path <path> Redis data path (default: /data/redis)"
|
||||
echo " --secret-path <path> Secret data path (default: /data/secret)"
|
||||
echo " --config-path <path> Harbor configuration file path (default: /etc/harbor/harbor.yml)"
|
||||
echo " --backup-dir <path> Backup directory (default: harbor_backup)"
|
||||
echo " --no-archive Do not unpack a tarball (source file tree already in place in ${backup_dir}/harbor)"
|
||||
echo " --use-syslog Use syslog for logging"
|
||||
echo " --log-level <level> Log level (default: INFO, options: DEBUG, INFO, NOTICE, WARNING, ERROR, CRITICAL, ALERT, EMERGENCY)"
|
||||
echo " --help Display this help message"
|
||||
exit 0
|
||||
}
|
||||
|
||||
# exit immediately if any command fails, an unset variable is used, or a pipe fails
|
||||
set -euo pipefail
|
||||
|
||||
# Parse command-line arguments
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--docker-cmd)
|
||||
DOCKER_CMD="$2"
|
||||
shift 2
|
||||
;;
|
||||
--db-image)
|
||||
HARBOR_DB_IMAGE="$2"
|
||||
shift 2
|
||||
;;
|
||||
--db-path)
|
||||
HARBOR_DB_PATH="$2"
|
||||
shift 2
|
||||
;;
|
||||
--registry-path)
|
||||
REGISTRY_DATA_PATH="$2"
|
||||
shift 2
|
||||
;;
|
||||
--chart-museum-path)
|
||||
CHART_MUSEUM_PATH="$2"
|
||||
shift 2
|
||||
;;
|
||||
--redis-path)
|
||||
REDIS_DATA_PATH="$2"
|
||||
shift 2
|
||||
;;
|
||||
--secret-path)
|
||||
SECRET_PATH="$2"
|
||||
shift 2
|
||||
;;
|
||||
--config-path)
|
||||
HARBOR_YML="$2"
|
||||
shift 2
|
||||
;;
|
||||
--backup-dir)
|
||||
BACKUP_DIR="$2"
|
||||
shift 2
|
||||
;;
|
||||
--no-archive)
|
||||
NO_ARCHIVE="true"
|
||||
shift
|
||||
;;
|
||||
--use-syslog)
|
||||
USE_SYSLOG="true"
|
||||
shift
|
||||
;;
|
||||
--log-level)
|
||||
LOG_LEVEL="$2"
|
||||
shift 2
|
||||
;;
|
||||
--help)
|
||||
usage
|
||||
;;
|
||||
*)
|
||||
echo "Unknown option: $1"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
# log() will prepend datetime and log the message to stderr and optionally to syslog
|
||||
# if the optional log level (defaults to INFO) is equal to higher than
|
||||
# $LOG_LEVEL
|
||||
log() {
|
||||
local level
|
||||
local message
|
||||
|
||||
if [[ $# -eq 1 ]]; then # Check if only one argument is provided
|
||||
level="INFO" # Default level if not provided
|
||||
message="$1"
|
||||
elif [[ $# -eq 2 ]]; then # Check if two arguments are provided
|
||||
level="$1"
|
||||
message="$2"
|
||||
else
|
||||
echo "Usage: log [LEVEL] MESSAGE" >&2
|
||||
return 1 # Return error code
|
||||
fi
|
||||
|
||||
# Check if logging level is valid
|
||||
if [[ ! "${LOG_LEVELS[$level]+_}" ]]; then
|
||||
echo "Invalid log level: $level" >&2 # Log error to stderr
|
||||
level="INFO" # Fallback to INFO
|
||||
fi
|
||||
|
||||
# Check minimum log level
|
||||
if (( ${LOG_LEVELS[$level]} > ${LOG_LEVELS[$LOG_LEVEL]} )); then
|
||||
return # Don't log if below the threshold
|
||||
fi
|
||||
|
||||
local formatted_message="[$(date '+%Y-%m-%d %H:%M:%S')] [$level] $message"
|
||||
|
||||
echo "$formatted_message"
|
||||
|
||||
if [[ "$USE_SYSLOG" == "true" ]]; then
|
||||
local syslog_priority="local0.${level,,}" # Convert level to lowercase for syslog
|
||||
logger -t "harbor-restore" -p "$syslog_priority" "$message"
|
||||
fi
|
||||
}
|
||||
|
||||
# Wait for database to be ready
|
||||
wait_for_db_ready() {
|
||||
local TIMEOUT=12
|
||||
set +e
|
||||
while [ $TIMEOUT -gt 0 ]; do
|
||||
docker exec harbor-db pg_isready | grep "accepting connections"
|
||||
if [ $? -eq 0 ]; then
|
||||
break
|
||||
fi
|
||||
TIMEOUT=$(( $TIMEOUT - 1 ))
|
||||
log "... waiting for database"
|
||||
sleep 5
|
||||
done
|
||||
set -e
|
||||
if [ $TIMEOUT -eq 0 ]; then
|
||||
log CRITICAL "Harbor DB cannot reach within one minute."
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Launch and wait for database
|
||||
launch_db() {
|
||||
if [ -n "$($DOCKER_CMD ps -q)" ]; then
|
||||
log CRITICAL "There are running containers, please stop and remove before backup"
|
||||
exit 1
|
||||
fi
|
||||
log "Starting database container for restore"
|
||||
docker run -d --name harbor-db -v "${BACKUP_DIR}/harbor/db":/backup -v "${HARBOR_DB_PATH}":/var/lib/postgresql/data "${HARBOR_DB_IMAGE}" "postgres"
|
||||
wait_for_db_ready
|
||||
}
|
||||
|
||||
# Restore database
|
||||
restore_database() {
|
||||
log "Restoring databases"
|
||||
|
||||
databases=$(ls "${BACKUP_DIR}/harbor/db"/*.sql | xargs -n1 basename | cut -d '.' -f 1) # Find .sql files
|
||||
|
||||
for db in $databases; do
|
||||
log "- $db"
|
||||
dump_file="/backup/${db}.sql"
|
||||
${DOCKER_CMD} exec harbor-db psql -U postgres -d template1 -c "drop database ${db}; exit \$?"
|
||||
${DOCKER_CMD} exec harbor-db psql -U postgres -d template1 -c "create database ${db}; exit \$?"
|
||||
${DOCKER_CMD} exec harbor-db sh -c "psql -U postgres -d ${db} > /dev/null < ${dump_file}; exit \$?"
|
||||
done
|
||||
|
||||
}
|
||||
|
||||
# Restore registry data
|
||||
restore_registry() {
|
||||
if [ -d "${BACKUP_DIR}/harbor/registry" ]; then
|
||||
log "Restoring registry data"
|
||||
if ! rsync -a --delete "${BACKUP_DIR}/harbor/registry/" "${REGISTRY_DATA_PATH}/"; then
|
||||
log ERROR "Failed to restore registry data using rsync"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
log WARNING "No registry data found (${BACKUP_DIR}/harbor/registry/), skipping restore"
|
||||
fi
|
||||
}
|
||||
|
||||
# Restore chart museum data
|
||||
restore_chart_museum() {
|
||||
if [ -d "${BACKUP_DIR}/harbor/chart_storage" ]; then
|
||||
log WARNING "Deprecated feature detected - Restoring chartmuseum data"
|
||||
if ! rsync -a --delete "${BACKUP_DIR}/harbor/chart_storage/" "${CHART_MUSEUM_PATH}/"; then
|
||||
log ERROR "Failed to restore chartmuseum data using rsync"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
# Restore redis data
|
||||
restore_redis() {
|
||||
if [ -d "${BACKUP_DIR}/harbor/redis" ]; then
|
||||
log "Restoring redis"
|
||||
if ! rsync -a --delete "${BACKUP_DIR}/harbor/redis/" "${REDIS_DATA_PATH}/"; then
|
||||
log ERROR "Failed to restore redis data using rsync"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
log "No redis backup found, skipping restore."
|
||||
fi
|
||||
}
|
||||
|
||||
# Restore secrets
|
||||
restore_secret() {
|
||||
if [ -d "${BACKUP_DIR}/harbor/secret" ]; then
|
||||
log "Restoring secrets"
|
||||
if ! rsync -a --delete "${BACKUP_DIR}/harbor/secret/" "${SECRET_PATH}/"; then
|
||||
log ERROR "Failed to restore secrets"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
log "No secrets backup found, skipping restore."
|
||||
fi
|
||||
}
|
||||
|
||||
|
||||
|
||||
restore_config() {
|
||||
if [ -f "${BACKUP_DIR}/harbor/harbor.yml" ]; then
|
||||
log "Restoring configuration file"
|
||||
cp "${BACKUP_DIR}/harbor/harbor.yml" "${HARBOR_YML}"
|
||||
else
|
||||
log "No configuration backup found, skipping restore."
|
||||
fi
|
||||
}
|
||||
|
||||
# Clean up database container
|
||||
clean_db() {
|
||||
log "Clean up temporary database container for backup"
|
||||
docker stop harbor-db
|
||||
docker rm harbor-db
|
||||
}
|
||||
|
||||
|
||||
extract_tarball() {
|
||||
if [ ${NO_ARCHIVE} == "false" ]; then
|
||||
if [ -f "${BACKUP_DIR}/${BACKUP_FILE}" ]; then
|
||||
cd "${BACKUP_DIR}"
|
||||
if [ -d harbor ] ; then
|
||||
log "Removing existing ${BACKUP_DIR}/harbor"
|
||||
rm -rf harbor
|
||||
fi
|
||||
log "Extracting tarball ${BACKUP_DIR}/${BACKUP_FILE}"
|
||||
tar -xzf "${BACKUP_DIR}/${BACKUP_FILE}"
|
||||
else
|
||||
log ERROR "No tarball found, Exiting."
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
log "Not extracting tarball"
|
||||
fi
|
||||
}
|
||||
|
||||
|
||||
#
|
||||
# main
|
||||
#
|
||||
|
||||
log "Starting restore of harbor"
|
||||
|
||||
# Create backup directory if it doesn't exist (for extracted backups)
|
||||
if [[ ! -d "$BACKUP_DIR" ]]; then
|
||||
mkdir -p "$BACKUP_DIR"
|
||||
fi
|
||||
|
||||
# Trap signals for cleanup
|
||||
trap clean_db EXIT ERR INT TERM
|
||||
|
||||
# Launch container for database backup
|
||||
launch_db
|
||||
|
||||
# Extract the tarball (if it exists)
|
||||
extract_tarball
|
||||
|
||||
# Restore the databases
|
||||
restore_database
|
||||
|
||||
# Restore the registry
|
||||
restore_registry
|
||||
|
||||
# Restore the Chart Museum
|
||||
restore_chart_museum
|
||||
|
||||
# Restore Redis
|
||||
restore_redis
|
||||
|
||||
# Restore the secrets
|
||||
restore_secret
|
||||
|
||||
# Restore the configuration
|
||||
restore_config
|
||||
|
||||
log "Ending restore of harbor"
|
||||
Loading…
Reference in New Issue