pax_global_header00006660000000000000000000000064147675524420014532gustar00rootroot0000000000000052 comment=0b44a5e4e2784fe056a7f6933e0ffc02fcb8261e autopostgresqlbackup-2.5/000077500000000000000000000000001476755244200156625ustar00rootroot00000000000000autopostgresqlbackup-2.5/Changelog.md000066400000000000000000000066531476755244200201050ustar00rootroot00000000000000# Changelog ## Version 2.5 * fix runtime environment by creating backup directories separately (Closes: [#43](https://github.com/k0lter/autopostgresqlbackup/issues/43) ## Version 2.4 * Fix runtime environment (backup directories created too early) (Closes: [#43](https://github.com/k0lter/autopostgresqlbackup/issues/43) ## Version 2.3 * Add `REPORT_ERRORS_ONLY` (default: `yes`) configuration variable. Set it to `no` permits to receive a backup report even if there are no errors. (Closes: [#35](https://github.com/k0lter/autopostgresqlbackup/issues/35)) * Add cron and systemd examples * Manpage is by now built from Documentation.md using pandoc ## Version 2.2 * Fix manpage issue ## Version 2.1 * Default configuration file /etc/default/autopostgresqlbackup is now deprecated in favor of /etc/autodbbackup.d * Add support for MySQL/MariaDB (using the `DBENGINE` configuration parameter). * Add a command line option `-c` to specify an alternate config file or directory (see [Documentation](/Documentation.md)). * Add manpage * Fix hypotetical shell injection by crafting special database names * Variable `OPT` (used with pg_dump) is renamed to `PGDUMP_OPTS` and a new variable `PGDUMPALL_OPTS` is available (used for dump globals with pg_dumpall) (Closes: [#12](https://github.com/k0lter/autopostgresqlbackup/issues/12)) * Fix stderr capture and check return code while running pg_dump/pg_dumpall commands * Add `MIN_DUMP_SIZE` configuration variable to raise a warning when a backup file size is below this limit (Closes: [#15](https://github.com/k0lter/autopostgresqlbackup/issues/15)) * Various bug fixes ## Version 2.0 * Huge code cleanup and refactoring (Closes: [#2](https://github.com/k0lter/autopostgresqlbackup/issues/2)) ### Added features * Compressing and/or encrypting dumps on the fly (Closes: [#1](https://github.com/k0lter/autopostgresqlbackup/issues/1)) * The day of the week for the weekly backups is now configurable (1 for Monday, 0 disable weekly backups) (Closes: [#1](https://github.com/k0lter/autopostgresqlbackup/issues/1), [#9](https://github.com/k0lter/autopostgresqlbackup/issues/9)) * The day of the month for the monthly backups is now configurable (instead of 1 by default, 0 disable the monthly backups) (Closes: [#1](https://github.com/k0lter/autopostgresqlbackup/issues/1), [#8](https://github.com/k0lter/autopostgresqlbackup/issues/8)) * Support for any compression tool that supports to read data to be compressed from stdin and outputs it to stdout (Closes: [#3](https://github.com/k0lter/autopostgresqlbackup/issues/3), [#6](https://github.com/k0lter/autopostgresqlbackup/issues/6)) * Daily, weekly and monthly dumps keeped are now configurable (by default, 14 daily, 5 weekly and 6 monthly backups) * Switch encryption from OpenSSL to GnuPG, see [details](https://github.com/k0lter/autopostgresqlbackup#openssl-encryption) (Closes: [#10](https://github.com/k0lter/autopostgresqlbackup/issues/10)) ### Removed features * It's no longer possible to dump all databases in a single file * Copying the lastest dump in the latest/ directory is no longer supported * Specifying the databases names to dump during the montly backup is no longer supported * It is no longer supported to send backup files by email (MAILCONTENT=files). It was anyway probably not a good idea to send backup files by email (for various reasons: file size, privacy, data leaks, etc.) but I guess that it could easily be implemented in a POSTBACKUP script if needed. autopostgresqlbackup-2.5/Documentation.md000066400000000000000000000223061476755244200210200ustar00rootroot00000000000000## NAME autopostgresqlbackup -- Automated tool to make periodic backups of databases ## SYNOPSIS `autopostgresqlbackup [OPTIONS]` ## DESCRIPTION AutoPostgreSQLBackup is a shell script (usually executed from a cron job or a systemd timer) designed to provide a fully automated tool to make periodic backups of databases. AutoPostgreSQLBackup extract databases into flat files (compressed or not, encrypted or not) in a daily and/or weekly and/or monthly basis. AutoPostgreSQLBackup supports multiple databases engines (PostgreSQL and MySQL by now). ## OPTIONS `-h` displays command line help `-d` Run in debug mode (no mail sent) `-c` Configuration file or directory (default: `/etc/autodbbackup.d/`) When a directory is used, the `*.conf` files will be processed sequentially. It allows one to backup multiple databases servers with distinct settings : - database servers with distinct engines - PostgreSQL cluster with instances running on multiple ports Note: if no configuration file or directory is passed as argument but `/etc/default/autopostgresqlbackup` exists, it will be used for backward compatibility. ## ENCRYPTION Encryption (asymmetric) is now done with GnuPG, you just need to add the public key (armored or not) you want to encrypt the data to in the file pointed by the `ENCRYPTION_PUBLIC_KEY` configuration setting. Export your public key: `gpg --export 0xY0URK3Y1D --output mypubkey.gpg` or `gpg --export --armor 0xY0URK3Y1D --output mypubkey.asc` then copy mypubkey.asc or mypubkey.gpg to the path pointed by the `ENCRYPTION_PUBLIC_KEY` configuration setting and set the `ENCRYPTION` setting to yes. ## DECRYPTION In order to decrypt a previously encrypted database dump: `gpg --decrypt --output backup.sql.gz backup.sql.gz.enc` ## OPENSSL ENCRYPTION Starting from version 2.0 encryption with OpenSSL is no longer supported as it was discovered[1] (but also known for quite some time[2]) that encrypting large files with OpenSSL silently fail[3] and that decrypting these files is close to be impossible[4]. * [1] https://github.com/k0lter/autopostgresqlbackup/issues/10 * [2] https://github.com/cytopia/mysqldump-secure/issues/21 * [3] https://github.com/openssl/openssl/issues/2515 * [4] https://github.com/imreFitos/large_smime_decrypt ## CONFIGURATION ### MAILADDR Email Address to send errors to. If empty errors are displayed on stdout. **default**: `root` ### REPORT_ERRORS_ONLY Send email to `MAILADDR` only if there are errors **default**: `yes` ### DBENGINE Database engine **default**: `postgresql` *supported values*: `postgresql` or `mysql` ### USERNAME Username to access the database server **default**: `""` (empty, the username to use is automatically defined depending on `DBENGINE`: `postgres` for PostgreSQL and `root` for MySQL) ### SU_USERNAME By default, on Debian systems (and maybe others), only `postgres` user is allowed to access PostgreSQL databases without password. In order to dump databases we need to run pg_dump/psql commands as `postgres` with su. This setting makes possible to run backups with a substitute user using `su`. If empty, `su` usage will be disabled) **default**: `""` (empty, not used) *Only while using PostgreSQL database engine* ### PASSWORD Password to access then Database server While using PostgreSQL database engine, in order to use a password to connect to database create a file `~/.pgpass` containing a line like this: `hostname:*:*:dbuser:dbpass` replace `hostname` with the value of `DBHOST`, `dbuser` with the value of `USERNAME` and `dbpass` with the password. While using MySQL database engine, if password is not set mysqldump will try to read credentials from `~/.my.cnf` if file exists. **default**: `""` (empty) ### DBHOST Host name (or IP address) of database server. Use `localhost` for socket connection or `127.0.0.1` to force TCP connection. **default**: `localhost` ### DBPORT Port of database server. While using PostgreSQL database engine, it is also used if `DBHOST` is `localhost` (socket connection) as socket name contains port. **default**: `""` (empty, the port to use is automatically defined depending on `DBENGINE`: `5432` for PostgreSQL and `3306` for MySQL) ### DBNAMES Explicit list of database(s) names(s) to backup If you would like to backup all databases on the server set `DBNAMES="all"`. If set to `"all"` then any new databases will automatically be backed up without needing to modify this settings when a new database is created. If the database you want to backup has a space in the name replace the space by a `%20` (`"data base"` will become `"data%20base"`). **default**: `all` **example**: `"users pages user%20data"` ### DBEXCLUDE List of databases to exclude if `DBNAMES` is not set to `all`. **default** : `""` (empty) **example**: `"pages user%20data"` ### GLOBALS_OBJECTS Virtual database name used to backup global objects (users, roles, tablespaces). **default**: `postgres_globals` *Only while using PostgreSQL database engine* ### BACKUPDIR Backup directory **default**: `/var/backups` ### CREATE_DATABASE Include or not `CREATE DATABASE` statements in dabatbases backups. **default**: `yes` *supported values*: `yes` or `no` ### DOWEEKLY Which day do you want weekly backups? (1 to 7 where 1 is Monday). When set to 0, weekly backups are disabled. **default**: `7` (Sunday) ### DOMONTHLY Which day do you want monthly backups? When set to 0, monthly backups are disabled. **default**: `1` (first day of the month) ### BRDAILY Backup retention count for daily backups, older backups are removed. **default**: `14` (14 days) ### BRWEEKLY Backup retention count for weekly backups, older backups are removed. **default**: `5` (5 weeks) ### BRMONTHLY Backup retention count for monthly backups, older backups are removed. **default**: `12` (12 months) ### COMP Compression tool. It could be gzip, pigz, bzip2, xz, zstd or any compression tool that supports to read data to be compressed from stdin and outputs them to stdout). If the tool is not in `${PATH}`, the absolute path can be used. **default**: `gzip` ### COMP_OPTS Compression tools options to be used with `COMP` **default**: `""` (empty) **example**: `COMP="zstd" COMP_OPTS="-f -c"` ### PGDUMP pg_dump path (relative if present in `${PATH}` or absolute) **default**: `""` (if empty `pg_dump` will be used) *Only while using PostgreSQL database engine* ### PGDUMPALL pg_dumpall path (relative if present in `${PATH}` or absolute) **default**: `""` (if empty `pg_dumpall` will be used) *Only while using PostgreSQL database engine* ### PGDUMP_OPTS Options string for use with pg_dump (see [pg_dump](https://www.postgresql.org/docs/current/app-pgdump.html) manual page). **default**: `""` (empty) *Only while using PostgreSQL database engine* ### PGDUMPALL_OPTS Options string for use with pg_dumpall (see [pg_dumpall](https://www.postgresql.org/docs/current/app-pg-dumpall.html) manual page). **default**: `""` (empty) *Only while using PostgreSQL database engine* ### MY mysql path (relative if present in `${PATH}` or absolute) **default**: `""` (if empty `mysql` will be used) *Only while using MySQL database engine* ### MYDUMP mysqldump path (relative if present in `${PATH}` or absolute) **default**: `""` (if empty `mysqldump` will be used) *Only while using MySQL database engine* ### MYDUMP_OPTS Options string for use with mysqldump (see [mysqldump](https://dev.mysql.com/doc/refman/8.3/en/mysqldump.html) manual page). **default**: `""` (empty) *Only while using MySQL database engine* ### EXT Backup files extension **default**: `sql` ### PERM Backup files permission **default**: `600` ### MIN_DUMP_SIZE Minimum size (in bytes) for a dump/file (compressed or not). File size below this limit will raise a warning. **default**: `256` ### ENCRYPTION Enable encryption (asymmetric) with GnuPG. **default**: `no` *supported values*: `yes` or `no` ### ENCRYPTION_PUBLIC_KEY Encryption public key (path to the key) **default**: `""` (empty) ### ENCRYPTION_SUFFIX Suffix for encyrpted files **default**: `.enc` ### PREBACKUP Command or script to execute before backups **default**: `""` (empty, not used) ### POSTBACKUP Command or script to execute after backups **default**: `""` (empty, not used) ## AUTHORS Originally developed by Aaron Axelsen with Friedrich Lobenstock contributions. Almost fully rewritten by Emmanuel Bouthenot (version 2.0 and higher). ## LICENSE AND COPYRIGHT This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. ## CONTRIBUTIONS Contributions are welcome on the project page: https://github.com/k0lter/autopostgresqlbackup/pulls ## BUGS Bug reports are welcome on the project page: https://github.com/k0lter/autopostgresqlbackup/issues ## SEE ALSO `pg_dump`(1), `pg_dumpall`(1), `mysqldump`(1) and the project page https://github.com/k0lter/autopostgresqlbackup/ autopostgresqlbackup-2.5/Makefile000066400000000000000000000003441476755244200173230ustar00rootroot00000000000000NAME=autopostgresqlbackup SECTION=8 all: man man: ${NAME}.${SECTION} ${NAME}.${SECTION}: Documentation.md pandoc \ --standalone \ --to man \ Documentation.md \ -o ${NAME}.${SECTION} clean: rm -f ${NAME}.${SECTION} autopostgresqlbackup-2.5/Readme.md000066400000000000000000000030201476755244200173740ustar00rootroot00000000000000# AutoPostgreSQLBackup AutoPostgreSQLBackup is a shell script (usually executed from a cron job) designed to provide a fully automated tool to make periodic backups of databases (supports PostgreSQL and MySQL/MariaDB). AutoPostgreSQLBackup extract databases into flat files in a daily, weekly or monthly basis. Version 2.2 adds support for MySQL/MariaDB. Version 2.0 is a full rewrite. It supports: * Email notification * Compression on the fly * Encryption on the fly * Rotation (daily and/or weekly and/or monthly) * Databases exclusion * Pre and Post scripts * Local configuration ## Usage On Debian (or derived): Install: `apt install autopostgresqlbackup` That's it! ## Documentation See [the documentation](/Documentation.md). ## Manual page Man page is build from [the documentation](/Documentation.md) using pandoc using the Makefile. `make man` ## History * 2023: Almost full rewrite with better error handling and new features (see Changelog.md for details) * 2019: Creation of a fork/standelone project on Github (https://github.com/k0lter/autopostgresqlbackup) * Since 2011: Various patches (fixes and new features) were added in the Debian package * 2011: AutoPostgreSQLBackup was included in Debian * 2005: AutoPostgreSQLBackup was written by Aaron Axelsen (with some contributions of Friedrich Lobenstock) * Project webpage was http://autopgsqlbackup.frozenpc.net/ (offline) ## Authors * Emmanuel Bouthenot (Current maintainer) * Friedrich Lobenstock (Contributions) * Aaron Axelsen (Original author) autopostgresqlbackup-2.5/autopostgresqlbackup000077500000000000000000000616741476755244200221100ustar00rootroot00000000000000#!/usr/bin/env bash # {{{ License and Copyright # https://github.com/k0lter/autopostgresqlbackup # Copyright (c) 2005 Aaron Axelsen # 2005 Friedrich Lobenstock # 2013-2023 Emmanuel Bouthenot # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA # }}} # {{{ Constants PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/postgres/bin:/usr/local/pgsql/bin HOMEPAGE="https://github.com/k0lter/autopostgresqlbackup" NAME="AutoPostgreSQLBackup" # Script name VERSION="2.5" # Version Number DATE="$(date '+%Y-%m-%d_%Hh%Mm')" # Datestamp e.g 2002-09-21 DNOW="$(date '+%u')" # Day number of the week 1 to 7 where 1 represents Monday DNOM="$(date '+%d')" # Date of the Month e.g. 27 # }}} # {{{ Variables # Configuration file or directory CONFIG="/etc/autodbbackup.d" # Legacy configuration file path (for backward compatibility) CONFIG_COMPAT="/etc/default/autopostgresqlbackup" # Email Address to send errors to. If empty errors are displayed on stdout. MAILADDR="root" # Send email only if there are errors REPORT_ERRORS_ONLY="yes" # Database engines supported: postgresql, mysql DBENGINE="postgresql" # Only while using PostgreSQL DB Engine SU_USERNAME="" # Username to access the Database server USERNAME="" # Password to access then Database server PASSWORD="" # Host name (or IP address) of the Database server. DBHOST="localhost" # Port of Database server. DBPORT="" # List of database(s) names(s) to backup. DBNAMES="all" # List of databases to exclude DBEXCLUDE="" # Virtual database name used to dump global objects (users, roles, tablespaces) GLOBALS_OBJECTS="postgres_globals" # Backup directory BACKUPDIR="/var/backups" # Include CREATE DATABASE statement CREATE_DATABASE="yes" # Which day do you want weekly backups? DOWEEKLY=7 # Which day do you want monthly backups? DOMONTHLY=1 # Backup retention count for daily backups. BRDAILY=14 # Backup retention count for weekly backups. BRWEEKLY=5 # Backup retention count for monthly backups. BRMONTHLY=12 # Compression tool. COMP="gzip" # Compression tools options. COMP_OPTS= # pg_dump path (pg_dump will be used if empty) PGDUMP= # pg_dumpall path (pg_dumpall will be used if empty) PGDUMPALL= # Options string for use with all_dump (see pg_dump manual page). PGDUMP_OPTS= # Options string for use with pg_dumpall (see pg_dumpall manual page). PGDUMPALL_OPTS= # mysql path (mysql will be used if empty) MY= # mysqldump path (mysqldump will be used if empty) MYDUMP= # Options string for use with mysqldump (see myqldump manual page). MYDUMP_OPTS= # Backup files extension EXT="sql" # Backup files permission PERM=600 # Minimum size (in bytes) for a dump/file (compressed or not). MIN_DUMP_SIZE=256 # Enable encryption (asymmetric) with GnuPG. ENCRYPTION="no" # Encryption public key (path to the key) ENCRYPTION_PUBLIC_KEY= # Suffix for encyrpted files ENCRYPTION_SUFFIX=".enc" # Command or script to execute before backups PREBACKUP= # Command or script to execute after backups POSTBACKUP= # Debug mode DEBUG="no" # Encryption prerequisites GPG_HOMEDIR= # Database connection arguments CONN_ARGS=() # Hostname HOSTNAME="$(uname -n)" if [[ "${HOSTNAME}" != *.* ]]; then HOSTNAME="$(hostname --fqdn)" fi # Return Code RC=0 # }}} # {{{ log{,ger,_info,_debug,_warn,_error}() logger() { local fd line severity reset color fd="${1}" severity="${2}" reset= color= if [ -n "${TERM}" ]; then reset="\e[0m" case "${severity}" in error) color="\e[0;91m" ;; warn) color="\e[0;93m" ;; debug) color="\e[0;96m" ;; *) color="\e[0;94m" ;; esac fi while IFS= read -r line ; do printf "%s|%s|%s\n" "${fd}" "${severity}" "${line}" >> "${LOG_FILE}" if [ "${DEBUG}" = "yes" ]; then if [ "${fd}" = "out" ]; then printf "${color}%6s${reset}|%s\n" "${severity}" "${line}" >&6 elif [ "${fd}" = "err" ]; then printf "${color}%6s${reset}|%s\n" "${severity}" "${line}" >&7 fi fi done } log() { echo "$@" | logger "out" "" } log_debug() { echo "$@" | logger "out" "debug" } log_info() { echo "$@" | logger "out" "info" } log_error() { echo "$@" | logger "err" "error" } log_warn() { echo "$@" | logger "err" "warn" } # }}} # {{{ arg_encode() arg_encode() { while read -r arg ; do echo "${arg}" | sed \ -e 's/%/%25/g' \ -e 's/ /%20/g' \ -e 's/\$/%24/g' \ -e 's/`/%60/g' \ -e 's/"/%22/g' \ -e "s/'/%27/g" \ -e "s/#/%23/g" \ -e 's/=/%3D/g' \ -e 's/\[/%5B/g' \ -e 's/\]/%5D/g' \ -e 's/!/%21/g' \ -e 's/>/%3E/g' \ -e 's/ >(logger "err" "error") } # }}} # {{{ compression() compression () { if [ -n "${COMP_OPTS}" ]; then IFS=" " read -r -a comp_args <<< "${COMP_OPTS}" log_debug "Compressing using '${COMP} ${comp_args[*]}'" "${COMP}" "${comp_args[@]}" 2> >(logger "err" "error") else log_debug "Compressing using '${COMP}'" "${COMP}" 2> >(logger "err" "error") fi } # }}} # {{{ postgresqldb_init() postgresqldb_init () { if [ -z "${DBPORT}" ]; then DBPORT="5432" fi CONN_ARGS=(--port "${DBPORT}") if [ "${DBHOST}" != "localhost" ]; then CONN_ARGS+=(--host "${DBHOST}") fi if [ -z "${USERNAME}" ]; then USERNAME="postgres" fi CONN_ARGS+=(--username "${USERNAME}") if [ -z "${PGDUMP}" ]; then PGDUMP="pg_dump" fi if [ -z "${PGDUMPALL}" ]; then PGDUMPALL="pg_dumpall" fi } # }}} # {{{ postgresqldb_list() postgresqldb_list () { local cmd_prog cmd_args raw_dblist dblist dbexcl databases cmd_prog="psql" cmd_args=(-t -l -A -F:) if [ "${#CONN_ARGS[@]}" -gt 0 ]; then cmd_args+=("${CONN_ARGS[@]}") fi log_debug "Running command: ${cmd_prog} ${cmd_args[*]}" raw_dblist=$( if [ -n "${SU_USERNAME}" ]; then if ! su - "${SU_USERNAME}" -c "${cmd_prog} ${cmd_args[*]}" 2> >(logger "err" "error"); then log_error "Running (as user '${SU_USERNAME}' command '${cmd_prog} ${cmd_args[*]}' has failed" fi elif ! "${cmd_prog}" "${cmd_args[@]}" 2> >(logger "err" "error"); then log_error "Running command '${cmd_prog} ${cmd_args[*]}' has failed" fi ) read -r -a dblist <<< "$( printf "%s\n" "${raw_dblist}" | \ sed -E -n 's/^([^:]+):.+$/\1/p' | \ arg_encode | \ tr '\n' ' ' )" log_debug "Automatically found databases: ${dblist[*]}" if [ -n "${DBEXCLUDE}" ]; then IFS=" " read -r -a dbexcl <<< "${DBEXCLUDE}" else dbexcl=() fi dbexcl+=(template0) log_debug "Excluded databases: ${dbexcl[*]}" mapfile -t databases < <( comm -23 \ <(IFS=$'\n'; echo "${dblist[*]}" | sort) \ <(IFS=$'\n'; echo "${dbexcl[*]}" | sort) \ ) databases+=("${GLOBALS_OBJECTS}") log_debug "Database(s) to be backuped: ${databases[*]}" printf "%s " "${databases[@]}" } # }}} # {{{ postgresqldb_dump() postgresqldb_dump () { local db_name cmd_prog cmd_args pg_args db_name="${1}" if [ -n "${PGDUMP_OPTS}" ]; then IFS=" " read -r -a PGDUMP_ARGS <<< "${PGDUMP_OPTS}" else PGDUMP_ARGS=() fi # pg_dumpall options if [ -n "${PGDUMPALL_OPTS}" ]; then IFS=" " read -r -a PGDUMPALL_ARGS <<< "${PGDUMPALL_OPTS}" else PGDUMPALL_ARGS=() fi if [ "${db_name}" = "${GLOBALS_OBJECTS}" ]; then cmd_prog="${PGDUMPALL}" cmd_args=(--globals-only) pg_args=("${PGDUMPALL_ARGS[@]}") else cmd_prog="${PGDUMP}" if [ -n "${SU_USERNAME}" ]; then cmd_args=("'${db_name}'") else cmd_args=("${db_name}") fi pg_args=("${PGDUMP_ARGS[@]}") if [ "${CREATE_DATABASE}" = "yes" ]; then pg_args+=(--create) fi fi if [ "${#CONN_ARGS[@]}" -gt 0 ]; then cmd_args+=("${CONN_ARGS[@]}") fi if [ "${#pg_args[@]}" -gt 0 ]; then cmd_args+=("${pg_args[@]}") fi log_debug "Running command: ${cmd_prog} ${cmd_args[*]}" if [ -n "${SU_USERNAME}" ]; then if ! su - "${SU_USERNAME}" -c "${cmd_prog} ${cmd_args[*]}" 2> >(logger "err" "error"); then log_error "Running (as user '${SU_USERNAME}' command '${cmd_prog} ${cmd_args[*]}' has failed" fi elif ! "${cmd_prog}" "${cmd_args[@]}" 2> >(logger "err" "error"); then log_error "Running command '${cmd_prog} ${cmd_args[*]}' has failed" fi } # }}} # {{{ mysqldb_init() mysqldb_init () { CONN_ARGS=() if [ -z "${DBPORT}" ]; then DBPORT="3306" fi if [ "${DBHOST}" != "localhost" ]; then CONN_ARGS+=(--host "${DBHOST}") fi if [ "${DBPORT}" != "3306" ]; then CONN_ARGS+=(--port "${DBPORT}") fi if [ -z "${USERNAME}" ]; then USERNAME="root" fi CONN_ARGS+=(--user "${USERNAME}") if [ -n "${PASSWORD}" ]; then CONN_ARGS+=(--password "${PASSWORD}") fi if [ -z "${MY}" ]; then MY="mysql" fi if [ -z "${MYDUMP}" ]; then MYDUMP="mysqldump" fi } # }}} # {{{ mysqldb_list() mysqldb_list () { local cmd_prog cmd_args raw_dblist dblist dbexcl databases cmd_prog="${MY}" cmd_args=(--batch --skip-column-names --execute 'SHOW DATABASES;') if [ "${#CONN_ARGS[@]}" -gt 0 ]; then cmd_args+=("${CONN_ARGS[@]}") fi log_debug "Running command: ${cmd_prog} ${cmd_args[*]}" raw_dblist=$( if ! "${cmd_prog}" "${cmd_args[@]}" 2> >(logger "err" "error"); then log_error "Running command '${cmd_prog} ${cmd_args[*]}' has failed" fi ) read -r -a dblist <<< "$( printf "%s\n" "${raw_dblist}" | \ arg_encode | \ tr '\n' ' ' )" log_debug "Automatically found databases: ${dblist[*]}" if [ -n "${DBEXCLUDE}" ]; then IFS=" " read -r -a dbexcl <<< "${DBEXCLUDE}" else dbexcl=() fi dbexcl+=(information_schema performance_schema mysql) log_debug "Excluded databases: ${dbexcl[*]}" mapfile -t databases < <( comm -23 \ <(IFS=$'\n'; echo "${dblist[*]}" | sort) \ <(IFS=$'\n'; echo "${dbexcl[*]}" | sort) \ ) log_debug "Database(s) to be backuped: ${databases[*]}" printf "%s " "${databases[@]}" } # }}} # {{{ mysqldb_dump() mysqldb_dump () { local db_name cmd_prog cmd_args my_args db_name="${1}" if [ -n "${MYDUMP_OPTS}" ]; then IFS=" " read -r -a MYDUMP_ARGS <<< "${MYDUMP_OPTS}" else MYDUMP_ARGS=() fi cmd_prog="${MYDUMP}" cmd_args=("${db_name}") my_args=("${MYDUMP_ARGS[@]}") my_args+=(--quote-names --events --routines) if [ "${CREATE_DATABASE}" = "no" ]; then my_args+=(--databases) else my_args+=(--no-create-db) fi if [ "${#CONN_ARGS[@]}" -gt 0 ]; then cmd_args+=("${CONN_ARGS[@]}") fi if [ "${#my_args[@]}" -gt 0 ]; then cmd_args+=("${my_args[@]}") fi log_debug "Running command: ${cmd_prog} ${cmd_args[*]}" if ! "${cmd_prog}" "${cmd_args[@]}" 2> >(logger "err" "error"); then log_error "Running command '${cmd_prog} ${cmd_args[*]}' has failed" fi } # }}} # {{{ db_init() db_init () { case ${DBENGINE} in postgresql) postgresqldb_init ;; mysql) mysqldb_init ;; *) log_error "Unsupported database engine ${DBENGINE}, check DBENGINE configuration parameter" return 1 ;; esac } # }}} # {{{ db_list() db_list () { case ${DBENGINE} in postgresql) postgresqldb_list ;; mysql) mysqldb_list ;; *) log_error "Unsupported database engine ${DBENGINE}, check DBENGINE configuration parameter" return 1 ;; esac } # }}} # {{{ db_dump() db_dump () { case ${DBENGINE} in postgresql|mysql) ${DBENGINE}db_dump "${1}" ;; *) log_error "Unsupported database engine ${DBENGINE}, check DBENGINE configuration parameter" return 1 ;; esac } # }}} # {{{ db_purge() db_purge() { local dumpdir db when count line dumpdir="${1}" db="${2}" when="${3}" count="${4}" # Since version >= 2.0 the dump filename no longer contains the week number # or the abbreviated month name so in order to be sure to remove the older # dumps we need to sort the filename on the datetime part (YYYY-MM-DD_HHhMMm) log_info "Rotating ${count} ${when} backups..." log_debug "Looking for '${db}_*' in '${dumpdir}/${when}/${db}'" find "${dumpdir}/${when}/${db}/" -name "${db}_*" | \ sed -E 's/(^.+([0-9]{4}-[0-9]{2}-[0-9]{2}_[0-9]{2}h[0-9]{2}m).*$)/\2 \1/' | \ sort -r | \ sed -E -n 's/\S+ //p' | \ tail -n "+${count}" | \ xargs -L1 rm -fv | \ while IFS= read -r line ; do log_info "${line}" done } # }}} # {{{ dump() dump() { local db_name dump_file comp_ext db_name="${1}" dump_file="${2}" if [ -n "${COMP}" ]; then comp_ext=".comp" case "${COMP}" in gzip|pigz) comp_ext=".gz" ;; bzip2) comp_ext=".bz2" ;; xz) comp_ext=".xz" ;; zstd) comp_ext=".zstd" ;; esac dump_file="${dump_file}${comp_ext}" fi if [ "${ENCRYPTION}" = "yes" ]; then dump_file="${dump_file}${ENCRYPTION_SUFFIX}" fi if [ -n "${COMP}" ] && [ "${ENCRYPTION}" = "yes" ]; then log_debug "Dumping (${db_name}) +compress +encrypt to '${dump_file}'" db_dump "${db_name}" | compression | encryption > "${dump_file}" elif [ -n "${COMP}" ]; then log_debug "Dumping (${db_name}) +compress to '${dump_file}'" db_dump "${db_name}" | compression > "${dump_file}" elif [ "${ENCRYPTION}" = "yes" ]; then log_debug "Dumping (${db_name}) +encrypt to '${dump_file}'" db_dump "${db_name}" | encryption > "${dump_file}" else log_debug "Dumping (${db_name}) to '${dump_file}'" db_dump "${db_name}" > "${dump_file}" fi if [ -f "${dump_file}" ]; then log_debug "Fixing permissions (${PERM}) on '${dump_file}'" chmod "${PERM}" "${dump_file}" fsize=$(stat -c '%s' "${dump_file}") if [ ! -s "${dump_file}" ]; then log_error "Something went wrong '${dump_file}' is empty" elif [ "${fsize}" -lt "${MIN_DUMP_SIZE}" ]; then log_warn "'${dump_file}' (${fsize} bytes) is below the minimum required size (${MIN_DUMP_SIZE} bytes)" fi else log_error "Something went wrong '${dump_file}' does not exists (error during dump?)" fi } # }}} # {{{ setup() setup() { # Using a shared memory filesystem (if available) to avoid # issues when there is no left space on backup storage if [ -w "/dev/shm" ]; then LOG_DIR="/dev/shm" fi LOG_PREFIX="${LOG_DIR}/${NAME}_${DBHOST//\//_}-$(date '+%Y-%m-%d_%Hh%Mm')" LOG_FILE="${LOG_PREFIX}.log" LOG_REPORT="${LOG_PREFIX}.report" HOST="${DBHOST}:${DBPORT}" if [ "${DBHOST}" = "localhost" ]; then HOST="${HOSTNAME}:${DBPORT} (socket)" fi } # }}} # {{{ prepare_backupdir() prepare_backupdir() { # Create required directories if [ ! -e "${BACKUPDIR}" ]; then # Check Backup Directory exists. mkdir -p "${BACKUPDIR}" fi if [ ! -e "${BACKUPDIR}/daily" ]; then # Check Daily Directory exists. mkdir -p "${BACKUPDIR}/daily" fi if [ ! -e "${BACKUPDIR}/weekly" ]; then # Check Weekly Directory exists. mkdir -p "${BACKUPDIR}/weekly" fi if [ ! -e "${BACKUPDIR}/monthly" ]; then # Check Monthly Directory exists. mkdir -p "${BACKUPDIR}/monthly" fi } # }}} # {{{ cleanup() cleanup() { # Cleanup GnuPG home dir if [ -d "${GPG_HOMEDIR}" ]; then rm -rf "${GPG_HOMEDIR}" fi # Clean up log files rm -f "${LOG_FILE}" "${LOG_REPORT}" } # }}} # {{{ setup_io() setup_io() { exec 6>&1 # Link file descriptor #6 with stdout. # Saves stdout. exec 7>&2 # Link file descriptor #7 with stderr. # Saves stderr. exec > >( logger "out") exec 2> >( logger "err") } # }}} # {{{ cleanup_io() cleanup_io() { exec 1>&6 6>&- # Restore stdout and close file descriptor #6. exec 2>&7 7>&- # Restore stdout and close file descriptor #7. } # }}} # {{{ reporting() reporting() { local exitcode subject exitcode=0 if grep -q '^err|' "${LOG_FILE}"; then exitcode=1 fi if [[ ( "${DEBUG}" = "no" ) && ( ${exitcode} = 1 || "${REPORT_ERRORS_ONLY}" = "no" ) ]]; then ( if [ ${exitcode} = 1 ]; then printf "*Errors/Warnings* (below) reported during backup on *%s*:\n\n" "${HOST}" grep '^err|' "${LOG_FILE}" | cut -d '|' -f 3- | \ while IFS= read -r line ; do printf " | %s\n" "${line}" done fi printf "\n\nFull backup log follows:\n\n" grep -v '^...|debug|' "${LOG_FILE}" | \ while IFS="|" read -r fd level line ; do if [ -n "${level}" ]; then printf "%8s| %s\n" "*${level}*" "${line}" else printf "%8s| %s\n" "" "${line}" fi done printf "\nFor more information, try to run %s in debug mode, see \`%s -h\`\n" "${NAME}" "$(basename "$0")" ) > "${LOG_REPORT}" if [ -n "${MAILADDR}" ]; then subject="report" if [ ${exitcode} = 1 ]; then subject="issues" fi mail -s "${NAME} ${subject} on ${HOSTNAME}" "${MAILADDR}" < "${LOG_REPORT}" else cat "${LOG_REPORT}" fi fi return ${exitcode} } # }}} # {{{ usage() usage() { cat <&2 exit 1 esac done # }}} # {{{ I/O redirection(s) for logging setup_io # }}} # {{{ Setup runtime settings setup # }}} # {{{ Config file loading CONFIG_N=0 if [ -d "${CONFIG}" ]; then CONFIG_N=$(find "${CONFIG}" -type f -iname '*.conf' | wc -l) fi if [ -f "${CONFIG_COMPAT}" ]; then log_debug "Loading config '${CONFIG_COMPAT}' (for backward compatibility)" # shellcheck source=/dev/null . "${CONFIG_COMPAT}" setup elif [ "${CONFIG_N}" -gt 0 ]; then CMD="$(readlink -f "${0}")" CMD_ARGS=() if [ "${DEBUG}" = "yes" ]; then CMD_ARGS+=(-d) fi cleanup_io cleanup find "${CONFIG}" -type f -iname '*.conf' -print0 | \ xargs -0 -L1 "${CMD}" "${CMD_ARGS[@]}" -c exit $? elif [ -f "${CONFIG}" ]; then log_debug "Loading config '${CONFIG}'" # shellcheck source=/dev/null . "${CONFIG}" setup else log_error "${NAME}: config file or directory '${CONFIG}' does not exists or directory '${CONFIG}' does not contains any configuration files." reporting cleanup_io cleanup exit 1 fi # }}} # {{{ Create backup directories prepare_backupdir # }}} # {{{ PreBackup # Run command before we begin if [ -n "${PREBACKUP}" ]; then log_info "Prebackup command output:" ${PREBACKUP} | \ while IFS= read -r line ; do log " ${line}" done fi # }}} # {{{ main() log_info "${NAME} version ${VERSION}" log_info "Homepage: ${HOMEPAGE}" log_info "Backup of Database Server (${DBENGINE}) - ${HOST}" if [ -n "${COMP}" ]; then if ! command -v "${COMP}" >/dev/null ; then log_warn "Disabling compression, '${COMP}' command not found" unset COMP fi fi if [ "${ENCRYPTION}" = "yes" ]; then if [ ! -s "${ENCRYPTION_PUBLIC_KEY}" ]; then log_warn "Disabling encryption, '${ENCRYPTION_PUBLIC_KEY}' is empty or does not exists" ENCRYPTION="no" elif ! command -v "gpg" >/dev/null ; then log_warn "Disabling encryption, 'gpg' command not found" ENCRYPTION="no" else gpg_setup if ! keyinfo="$(gpg --quiet --homedir "${GPG_HOMEDIR}" "${ENCRYPTION_PUBLIC_KEY}" 2>/dev/null)"; then log_warn "Disabling encryption, key in '${ENCRYPTION_PUBLIC_KEY}' does not seems to be a valid public key" ENCRYPTION="no" if command -v "openssl" >/dev/null && openssl x509 -noout -in "${ENCRYPTION_PUBLIC_KEY}" >/dev/null 2>&1; then log_warn "public key in '${ENCRYPTION_PUBLIC_KEY}' seems to be in PEM format" log_warn "Encryption using openssl is no longer supported: see ${HOMEPAGE}#openssl-encryption" fi else keyfp="$(echo "${keyinfo}" | sed -E -n 's/^\s*([a-z0-9]+)\s*$/\1/pi')" keyuid="$(echo "${keyinfo}" | sed -E -n 's/^\s*uid\s+(\S.*)$/\1/pi' | head -n1)" log_info "Encryption public key is: 0x${keyfp} (${keyuid})" fi fi fi log_info "Backup Start: $(date)" if [ "${DNOM}" = "${DOMONTHLY}" ]; then period="monthly" rotate="${BRMONTHLY}" elif [ "${DNOW}" = "${DOWEEKLY}" ]; then period="weekly" rotate="${BRWEEKLY}" else period="daily" rotate="${BRDAILY}" fi db_init # If backing up all DBs on the server if [ "${DBNAMES}" = "all" ]; then DBNAMES="$(db_list)" fi for db_enc in ${DBNAMES} ; do db="$(echo "${db_enc}" | arg_decode)" log_info "Backup of Database (${period}) '${db}'" backupdbdir="${BACKUPDIR}/${period}/${db_enc}" if [ ! -e "${backupdbdir}" ]; then log_debug "Creating Backup DB directory '${backupdbdir}'" mkdir -p "${backupdbdir}" fi db_purge "${BACKUPDIR}" "${db_enc}" "${period}" "${rotate}" backupfile="${backupdbdir}/${db_enc}_${DATE}.${EXT}" dump "${db}" "${backupfile}" done log_info "Backup End: $(date)" log_info "Total disk space used for ${BACKUPDIR}: $(du -hs "${BACKUPDIR}" | cut -f1)" # }}} # {{{ PostBackup # Run command when we're done if [ -n "${POSTBACKUP}" ]; then log_info "Postbackup command output:" ${POSTBACKUP} | \ while IFS= read -r line ; do log " ${line}" done fi # }}} # {{{ cleanup I/O redirections cleanup_io # }}} # {{{ Reporting reporting RC=${?} # }}} # {{{ Cleanup and exit() cleanup exit ${RC} # }}} # vim: foldmethod=marker foldlevel=0 foldenable autopostgresqlbackup-2.5/examples/000077500000000000000000000000001476755244200175005ustar00rootroot00000000000000autopostgresqlbackup-2.5/examples/mariadb.conf000066400000000000000000000011541476755244200217470ustar00rootroot00000000000000 # DB engine DBENGINE="mysql" # Backup directory BACKUPDIR="/var/backups/autodbbackup/mariadb" # Include CREATE DATABASE in backups? CREATE_DATABASE="no" # Which day do you want weekly backups? (1 to 7 where 1 is Monday) # When set to 0, weekly backups are disabled DOWEEKLY=0 # Which day do you want monthly backups? # When set to 0, monthly backups are disabled DOMONTHLY=0 # Backup retention count for daily backups, older backups are removed. BRDAILY=7 # Compression tool COMP="zstd" # mariadb path (like "mysql" for MySQL) MY="mariadb" # mariadb-dump path (like "mysqldump" for MySQL) MYDUMP="mariadb-dump" autopostgresqlbackup-2.5/examples/mysql.conf000066400000000000000000000007631476755244200215220ustar00rootroot00000000000000 # DB engine DBENGINE="mysql" # Backup directory BACKUPDIR="/var/backups/autodbbackup/${DBENGINE}" # Include CREATE DATABASE in backups? CREATE_DATABASE="no" # Which day do you want weekly backups? (1 to 7 where 1 is Monday) # When set to 0, weekly backups are disabled DOWEEKLY=0 # Which day do you want monthly backups? # When set to 0, monthly backups are disabled DOMONTHLY=0 # Backup retention count for daily backups, older backups are removed. BRDAILY=7 # Compression tool COMP="zstd" autopostgresqlbackup-2.5/examples/postgresql.conf000066400000000000000000000010661476755244200225550ustar00rootroot00000000000000 # DB engine DBENGINE="postgresql" # Substitute user to run dump commands SU_USERNAME="postgres" # Backup directory BACKUPDIR="/var/backups/autodbbackup/${DBENGINE}" # Include CREATE DATABASE in backups? CREATE_DATABASE="no" # Which day do you want weekly backups? (1 to 7 where 1 is Monday) # When set to 0, weekly backups are disabled DOWEEKLY=0 # Which day do you want monthly backups? # When set to 0, monthly backups are disabled DOMONTHLY=0 # Backup retention count for daily backups, older backups are removed. BRDAILY=7 # Compression tool COMP="zstd" autopostgresqlbackup-2.5/services/000077500000000000000000000000001476755244200175055ustar00rootroot00000000000000autopostgresqlbackup-2.5/services/cron/000077500000000000000000000000001476755244200204465ustar00rootroot00000000000000autopostgresqlbackup-2.5/services/cron/autopostgresqlbackup000077500000000000000000000003521476755244200246560ustar00rootroot00000000000000#!/bin/sh # # autopostgresqlbackup daily cron task # if [ -d /run/systemd/system ]; then # Skip in favour of systemd timer. exit 0 fi if [ -x /usr/sbin/autopostgresqlbackup ]; then /usr/sbin/autopostgresqlbackup fi exit 0 autopostgresqlbackup-2.5/services/systemd/000077500000000000000000000000001476755244200211755ustar00rootroot00000000000000autopostgresqlbackup-2.5/services/systemd/autopostgresqlbackup.service000066400000000000000000000003721476755244200270430ustar00rootroot00000000000000[Unit] Description=autopostgresqlbackup daily service Documentation=man:autopostgresqlbackup(8) After=postgresql.target mysql.target [Service] Type=oneshot ExecStart=/usr/sbin/autopostgresqlbackup ProtectHome=true ProtectSystem=full PrivateTmp=true autopostgresqlbackup-2.5/services/systemd/autopostgresqlbackup.timer000066400000000000000000000002001476755244200265110ustar00rootroot00000000000000[Unit] Description=autopostgresqlbackup daily timer [Timer] OnCalendar=daily Persistent=true [Install] WantedBy=timers.target