VMWare copy and keep thin provisioning scripts & register the VM on a backup VM server

Post date: Mar 31, 2017 1:54:59 PM

I use ghettovcb for backing up my VM Hosts to my file servers via NFS. I do this as I can't quickly and easily copy them directly to my backup VM host(s). As VMWare refuses to acknowledge that it's users want and haven been asking for remote access to datastores since the beginning of time. So here is my stop gap solution that only uses the tools built into VMWare (like ghettovcb) to copy from a NFS mount to the backup VM host, Register the latest good backup & unregister the old copy of that same server

VM Host ( ghettovcb via cron )=> NFS Share => Backup VM Host (My Scripts Via Cron)

This script does the fallowing.

1. Look for all STATUS.ok files found in the directory structure that is using the default ghettovcb setup (aka VM_Hostname/VM_Hostname-DATE ).

Copy over all files in that directory to a corresponding location on the "Backup VM Host"
Edit the Display names of all backups to read " Date - VM Hostname "
1. Copy the vmdk using vmkfstools to keep or covert the backup to a thin provisioned version
2. verify and log everything along the way
3. only keep X copies on the backup VM HOST
4. Auto removes Moved copied from NFS host
5. Only copys the STATUS.ok file the backup VM Host after everything else is done and verified aka no reported errors)

copy_vm_to_backup.sh

## config vars

max_copies=10

vmstore_backup="backups"

vmstore_restore="datastore1"

log="/vmfs/volumes/${vmstore_backup}/scripts/logs/esxi-backups.$(date +%s).log"

debug=1

vmkfstools="$(which vmkfstools)"

## functions

logit() {

if [ $debug -eq 1 ]; then

echo "$1"

fi

echo "$1" >> "${log}"

}

logit "========= Starting Copy ========="

if [ ! -d "${log}" ]; then

mdkir -p "${log}"

if [ $? -eq 0 ]; then

debug=1

logit "ERROR : mdkir -p '${log}'"

else

logit "Created Logs dir : '${log}'"

fi

## the work

find /vmfs/volumes/${vmstore_backup}/ -type d -maxdepth 1 -mindepth 1 | while read vmdirs; do

# Reset skip this vm

vmskip=0

if [ -d "${vmdirs}" ]; then

vmname=$(basename "${vmdirs}")

logit "Found dir : ${vmdirs}"

logit "VM name : ${vmname}"

if [ "${vmdirs}" != "scripts" ]; then

find "${vmdirs}" -type f -iname "STATUS.ok" | while read vmbackup; do

logit "Found Completed backup file : ${vmbackup}"

vmbackupdir=$(dirname "${vmbackup}")

vmrestoredir=$(dirname "${vmbackup/${vmstore_backup}/${vmstore_restore}}")

logit "vmbackupdir : ${vmbackupdir}"

logit "vmrestoredir : ${vmrestoredir}"

if [ -e "${vmrestoredir}/BACKUP.err" ]; then

logit "Removing old ERROR file"

rm -f "${vmrestoredir}/BACKUP.err"

fi

logit "Looking into : ${vmbackupdir}"

find "${vmbackupdir}" -type f | while read vmbackupfiles; do

vmbackupfilesdir=$(dirname "${vmbackupfiles}")

vmbackupfilesname=$(basename "${vmbackupfiles}")

vmbackupfilesextension="${vmbackupfilesname##*.}"

vmrestorefiles="${vmbackupfiles/$vmstore_backup/$vmstore_restore}"

logit "Found file : ${vmbackupfiles}"

if [ ! -d "${vmrestoredir}" ]; then

logit "Creating dir : ${vmrestoredir}"

mkdir -p "${vmrestoredir}"

if [ $? -ne 0 ]; then

vmskip=1

logit "ERROR : mkdir -p '${vmrestoredir}'"

fi

if [ $vmskip -eq 0 ]; then

case "${vmbackupfilesname}" in

"STATUS.ok") ## Ignore the STATUS.ok file

ignore=1

;;

*"-flat.vmdk") ## Ignore any -flat vhd files

logit "checking if flat file is an orphan. : '${vmrestorefiles}' or '${vmrestorefiles/-flat.vmdk/.vmdk}.converted'"

if [ -e "${vmrestorefiles}" ]; then

logit "Found restored flat file"

if [ ! -e "${vmrestorefiles/-flat.vmdk/.vmdk}.converted" ]; then

logit "Found no converted flag though : '${vmrestorefiles/-flat.vmdk/.vmdk}.converted'"

if [ -e "${vmrestorefiles/-flat.vmdk/.vmdk}.convert.log" ]; then

if [ $(grep -c '100% done.' "${vmrestorefiles/-flat.vmdk/.vmdk}.convert.log") -eq 1 ]; then

logit "Found convert.log & it reads the conver was 100%. Creating converted flag file : '${vmrestorefiles/-flat.vmdk/.vmdk}.converted'"

touch "${vmrestorefiles/-flat.vmdk/.vmdk}.converted"

else

logit "ERROR : No converted flag found, convert file does NOT read 100% done. Leaving file"

fi

else

logit "Found no converted log : '${vmrestorefiles/-flat.vmdk/.vmdk}.convert.log'"

fi

if [ -e "${vmrestorefiles/-flat.vmdk/.vmdk}.converted" ]; then

logit "Found left over flat file"

if [ ! -e "${vmbackupfiles/-flat.vmdk/.vmdk}" ]; then

logit "Removing left over flat (orphan)"

rm -f "${vmbackupfiles}"

if [ $? -ne 0 ]; then

logit "ERROR : Removing left over flat (orphan) : rm -f '${vmbackupfiles}'"

fi

;;

*".vmdk") ## Convert to thin (or at least keep thin provisioning)

vmrestorefilesflat="${vmrestorefiles%.*}-flat.vmdk"

if [ ! -e "${vmrestorefiles}.converted" ]; then

logit "Looking for any preveous convert attemps : '${vmrestorefiles}' or '${vmrestorefilesflat}'"

if [ -e "${vmrestorefiles}" ] || [ -e "${vmrestorefilesflat}" ]; then

if [ -e "${vmrestorefiles}" ]; then

logit "Removing preveous convert attemp : '${vmrestorefiles}'"

rm -f "${vmrestorefiles}"

if [ $? -ne 0 ]; then

logit "ERROR : removing preveous convert attemp : rm -f '${vmrestorefiles}'"

fi

if [ -e "${vmrestorefilesflat}" ]; then

logit "Removing preveous convert attemp (flat) : '${vmrestorefilesflat}'"

rm -f "${vmrestorefilesflat}"

if [ $? -ne 0 ]; then

logit "ERROR : removing preveous convert attemp : rm -f '${vmrestorefilesflat}'"

fi

logit "Convert : Start : $(date)"

logit "Converting : '${vmbackupfiles}' => '${vmrestorefiles}'"

$vmkfstools -d thin -i "${vmbackupfiles}" "${vmrestorefiles}" > "${vmrestorefiles}.convert.log"

if [ $? -eq 0 ] && [ $(grep -c '100% done.' "${vmrestorefiles}.convert.log") -eq 1 ]; then

logit "Convert : End : $(date)"

touch "${vmrestorefiles}.converted"

logit "Created : ${vmrestorefiles}.converted"

logit "Looking for vmdk file : '${vmbackupfiles}'"

if [ -e "${vmbackupfiles}" ]; then

echo "Removing Converted VHD : ${vmbackupfiles}"

rm -f "${vmbackupfiles}"

fi

logit "Looking for flat file : '${vmbackupfilesdir}/${vmbackupfilesname%.*}-flat.vmdk'"

if [ -e "${vmbackupfilesdir}/${vmbackupfilesname%.*}-flat.vmdk" ]; then

echo "Removing Converted Flat VHD : ${vmbackupfiles%.*}-flat.vmdk"

rm -f "${vmbackupfilesdir}/${vmbackupfilesname%.*}-flat.vmdk"

fi

else

logit "Convert : End : $(date)"

logit "ERROR : $vmkfstools -d thin -i '${vmbackupfiles}' '${vmrestorefiles}'"

vmskip=1

fi

else

logit "Already Convereted found : '${vmrestorefiles}.converted'"

if [ $(grep -c '100% done.' "${vmrestorefiles}.convert.log") -eq 1 ]; then

logit "Converted log reports 100% complete : grep -c '100% done.' '${vmrestorefiles}.convert.log'"

logit "Looking for left over vmdk file : '${vmbackupfiles}'"

if [ -e "${vmbackupfiles}" ]; then

echo "Removing Converted VHD : ${vmbackupfiles}"

rm -f "${vmbackupfiles}"

fi

logit "Looking for left over flat file : '${vmbackupfilesdir}/${vmbackupfilesname%.*}-flat.vmdk'"

if [ -e "${vmbackupfilesdir}/${vmbackupfilesname%.*}-flat.vmdk" ]; then

echo "Removing Converted Flat VHD : ${vmbackupfiles}"

rm -f "${vmbackupfilesdir}/${vmbackupfilesname%.*}-flat.vmdk"

fi

;;

*".vmx") ## Copy and modify base config

vmbackupdate="${vmrestoredir%%_*}"

vmbackupdate="${vmbackupdate##*/}"

vmbackupdate="${vmbackupdate/${vmname}-/}"

cp -f "${vmbackupfiles}" "${vmrestorefiles}.org"

if [ $? -eq 0 ]; then

logit "Copied Config : ${vmrestorefiles}.org"

cat "${vmrestorefiles}.org" | sed -e 's/^displayName\ \=\ \".*"$/displayName\ \=\ \"'"${vmbackupdate} ${vmname}"'"/' > "${vmrestorefiles}"

if [ $? -eq 0 ]; then

logit "Mod'ed Config : ${vmrestorefiles}"

rm -f "${vmbackupfiles}"

else

logit "ERROR : Mod'ing Config : ${vmrestorefiles}"

vmskip=1

fi

else

logit "ERROR : Copying Config : ${vmbackupfiles} => ${vmrestorefiles}.org"

vmskip=1

fi

;;

*) ## Default copy any non known file

if [ ! -d "${vmrestoredir}" ]; then

mkdir -p "${vmrestoredir}"

if [ $? -ne 0 ]; then

logit "ERROR : mkdir -p '${vmrestoredir}'"

vmskip=1

fi

if [ -d "${vmrestoredir}" ]; then

cp -f "${vmbackupfiles}" "${vmrestorefiles}"

if [ $? -eq 0 ]; then

logit "Copied MISC file : ${vmbackupfiles}"

rm -f "${vmbackupfiles}"

else

logit "ERROR : Coping MISC file : cp -f '${vmbackupfiles}' '${vmrestorefiles}'"

vmskip=1

fi

else

logit "ERROR : Coping MISC file : no dir '${vmrestorefiles}'"

vmskip=1

fi

;;

esac

fi

if [ $vmskip -eq 1 ]; then

touch ${vmrestoredir}/BACKUP.err

fi

done

if [ $(find "${vmbackupdir}" -type f | wc -l) -eq 1 ] && [ ! -e "${vmrestoredir}/BACKUP.err" ]; then

logit "Found only remaning file : $(find '${vmbackupdir}' -type f)"

touch "${vmrestoredir}/STATUS.ok"

if [ $? -eq 0 ]; then

logit "Created STATUS.ok file : ${vmrestoredir}/STATUS.ok"

rm -f "${vmbackupdir}/STATUS.ok"

if [ $? -ne 0 ]; then

logit "ERROR : unable to delete backup source STATUS.ok : rm -f '${vmbackupdir}/STATUS.ok'"

fi

else

logit "ERROR : Creating STATUS.ok : touch '${vmrestoredir}/STATUS.ok'"

fi

if [ $? -eq 0 ]; then

rm -f "${vmbackupdir}/STATUS.ok"

fi

vmbackupdir="${vmbackupdir//../}"

case "${vm2delete}" in

*[!/]*/)

vmbackupdir=${vmbackupdir%"${vmbackupdir##*[!/]}"}

;;

esac

if [ -d "vmbackupdir" ]; then

if [ "${vmbackupdir}" != "/vmfs/volumes/${vmstore_backup}" ] && [ "${vmbackupdir}" != "/vmfs/volumes" ] && [ "${vmbackupdir}" != "/vmfs" ]; then

rm -rf "${vmbackupdir}"

if [ $? -eq 0 ]; then

logit "Removed : '${vmbackupdir}'"

else

logit "ERROR : rm -rf '${vmbackupdir}'"

fi

else

logit "ERROR : Found remaning files : $(find ${vmbackupdir} -type f)"

fi

done

fi

else

logit "Found misc file : ${vmdirs}"

fi

done

logit "======= Start Old Removal ======="

find /vmfs/volumes/${vmstore_restore}/ -type d -maxdepth 1 -mindepth 1 -not -iname ".*" | while read vmbase; do

foundvms=$(find "${vmbase}" -type f $ -name "STATUS.ok" -o -name "BACKUP.err" $ | wc -l)

logit "Old Copies Count : ${foundvms} for ${vmbase}"

if [ $foundvms -gt $max_copies ]; then

logit "Over Max Copies : ${vmbase} by $(($foundvms - $max_copies))"

ls -1cr "${vmbase}" | head -n $(($foundvms - $max_copies)) | while read vmdelete; do

vm2delete="${vmbase}/${vmdelete}/"

vm2delete="${vm2delete//../}"

case "${vm2delete}" in

*[!/]*/)

vm2delete=${vm2delete%"${vm2delete##*[!/]}"}

;;

esac

logit "Removing Old Copy : ${vmbase}/${vmdelete}"

if [ "${vm2delete}" != "/vmfs/volumes/${vmstore_restore}" ] && [ "${vm2delete}" != "/vmfs/volumes" ] && [ "${vm2delete}" != "/vmfs" ]; then

echo rm -rf "${vm2delete}"

if [ $? -eq 0 ]; then

logit "Removed : ${vm2delete}"

else

logit "ERROR : Removing Old Copy : '${vmbase}/${vmdelete}'"

fi

done

fi

done

logit "======= Completely Done ========="

This script does all the registering and un-registering of the backups.

register_guests.sh

vmstore_backup="datastore1"

find "/vmfs/volumes/${vmstore_backup}/" -type d -maxdepth 1 -mindepth 1 -not -iname "scripts" | while read vmname; do

ls -1r "${vmname}"/*/STATUS.ok | head -n 1 | while read vmstatusfile; do

## Setup some vars

vmpath=$(dirname "${vmstatusfile}")

vmversion=${vmpath##*/}

vmname=${vmpath%/*}

vmname=${vmname##*/}

vmdate=${vmpath##*/}

vmdate=${vmdate/${vmname}-/}

vmdate=${vmdate%_*} #2017-03-27

vmdisplayname="${vmname}_${vmdate}"

# See if this VM is already reg.

if [ $(vim-cmd vmsvc/getallvms | grep -c -i "${vmname}") -eq 0 ]; then

# if not reg it

echo "Adding : ${vmversion}"

find "${vmpath}" -iname "*.vmx" -exec vim-cmd solo/registervm '{}' \;

else

# test is this version is already reged, if not unreg the existing and reg this opne

if [ $(vim-cmd vmsvc/getallvms | grep -c "${vmversion}") -eq 0 ]; then

# get vmid of exiting vm guest

vmid=$(vim-cmd vmsvc/getallvms | grep -i "${vmname}" | awk '{print $1}')

# chec kthat it is not on. if it is, we must be using it and SHOULD NOT remove it

if [ $(vim-cmd vmsvc/power.getstate ${vmid} | grep -c "Powered off") ]; then

vim-cmd vmsvc/unregiste ${vmid}

# check that it was removed

if [ $(vim-cmd vmsvc/getallvms | grep -c -i "${vmname}") -eq 0 ]; then

# add the new one

echo "Adding : ${vmversion}"

find "${vmpath}" -iname "*.vmx" -exec vim-cmd solo/registervm '{}' \;

fi

else

echo "Skipping : ${vmversion}"

fi

done