Scripting‎ > ‎

VMWare copy and keep thin provisioning scripts & register the VM on a backup VM server

posted Mar 31, 2017, 6:54 AM by Chris Franklin   [ updated Mar 31, 2017, 7:17 AM ]
I use ghettovcb for backing up my VM Hosts to my file servers via NFS. I do this as I can't quickly and easily copy them directly to my backup VM host(s). As VMWare refuses to acknowledge that it's users want and haven been asking for remote access to datastores since the beginning of time. So here is my stop gap solution that only uses the tools built into VMWare (like ghettovcb) to copy from a NFS mount to the backup VM host, Register the latest good backup & unregister the old copy of that same server


            VM Host ( ghettovcb via cron )=> NFS Share => Backup VM Host (My Scripts Via Cron)
                        


This script does the fallowing.
  1. Look for all STATUS.ok files found in the directory structure that is using the default ghettovcb setup (aka VM_Hostname/VM_Hostname-DATE ).
  2. Copy over all files in that directory to a corresponding location on the "Backup VM Host"
  3. Edit the Display names of all backups to read " Date - VM Hostname "  
  4. Copy the vmdk using vmkfstools to keep or covert the backup to a thin provisioned version
  5. verify and log everything along the way
  6. only keep X copies on the backup VM HOST
  7. Auto removes Moved copied from NFS host
  8. Only copys the STATUS.ok file the backup VM Host after everything else is done and verified aka no reported errors)

copy_vm_to_backup.sh


## config vars
max_copies=10
vmstore_backup="backups"
vmstore_restore="datastore1"
log="/vmfs/volumes/${vmstore_backup}/scripts/logs/esxi-backups.$(date +%s).log"
debug=1
vmkfstools="$(which vmkfstools)"


## functions
logit() {
 if [ $debug -eq 1 ]; then
  echo "$1"
 fi
 echo "$1" >> "${log}"
}

logit "========= Starting Copy ========="

if [ ! -d "${log}" ]; then
 mdkir -p "${log}"
 if [ $? -eq 0 ]; then
  debug=1
  logit "ERROR : mdkir -p '${log}'"
 else
  logit "Created Logs dir : '${log}'"
 fi
fi

## the work
find /vmfs/volumes/${vmstore_backup}/ -type d -maxdepth 1 -mindepth 1 | while read vmdirs; do
 # Reset skip this vm
 vmskip=0

 if [ -d "${vmdirs}" ]; then
  vmname=$(basename "${vmdirs}")
  logit "Found dir : ${vmdirs}"
  logit "VM name : ${vmname}"
  if [ "${vmdirs}" != "scripts" ]; then
   find "${vmdirs}" -type f -iname "STATUS.ok" | while read vmbackup; do
    logit "Found Completed backup file : ${vmbackup}"
    vmbackupdir=$(dirname "${vmbackup}")
    vmrestoredir=$(dirname "${vmbackup/${vmstore_backup}/${vmstore_restore}}")
    logit "vmbackupdir  : ${vmbackupdir}"
    logit "vmrestoredir : ${vmrestoredir}"

    if [ -e "${vmrestoredir}/BACKUP.err" ]; then
     logit "Removing old ERROR file"
     rm -f "${vmrestoredir}/BACKUP.err"
    fi

    logit "Looking into : ${vmbackupdir}"
    find "${vmbackupdir}" -type f | while read vmbackupfiles; do
     
     vmbackupfilesdir=$(dirname "${vmbackupfiles}")
     vmbackupfilesname=$(basename "${vmbackupfiles}")
     vmbackupfilesextension="${vmbackupfilesname##*.}"
     vmrestorefiles="${vmbackupfiles/$vmstore_backup/$vmstore_restore}"
     logit "Found file : ${vmbackupfiles}"

     if [ ! -d "${vmrestoredir}" ]; then
      logit "Creating dir : ${vmrestoredir}"
      mkdir -p "${vmrestoredir}"
      if [ $? -ne 0 ]; then
       vmskip=1
       logit "ERROR : mkdir -p '${vmrestoredir}'"
      fi
     fi

     if [ $vmskip -eq 0 ]; then
      case "${vmbackupfilesname}" in
       "STATUS.ok") ## Ignore the STATUS.ok file 
        ignore=1
        ;;
       *"-flat.vmdk") ## Ignore any -flat vhd files
        logit "checking if flat file is an orphan. : '${vmrestorefiles}' or '${vmrestorefiles/-flat.vmdk/.vmdk}.converted'"
        if [ -e "${vmrestorefiles}" ]; then
         logit "Found restored flat file"
         if [ ! -e "${vmrestorefiles/-flat.vmdk/.vmdk}.converted" ]; then
          logit "Found no converted flag though : '${vmrestorefiles/-flat.vmdk/.vmdk}.converted'"
          if [ -e "${vmrestorefiles/-flat.vmdk/.vmdk}.convert.log" ]; then
           if [ $(grep -c '100% done.' "${vmrestorefiles/-flat.vmdk/.vmdk}.convert.log") -eq 1 ]; then
            logit "Found convert.log & it reads the conver was 100%. Creating converted flag file : '${vmrestorefiles/-flat.vmdk/.vmdk}.converted'"
            touch "${vmrestorefiles/-flat.vmdk/.vmdk}.converted"
           else
            logit "ERROR : No converted flag found, convert file does NOT read 100% done. Leaving file"
           fi
          else
           logit "Found no converted log : '${vmrestorefiles/-flat.vmdk/.vmdk}.convert.log'"
          fi
         fi

         if [ -e "${vmrestorefiles/-flat.vmdk/.vmdk}.converted" ]; then
          logit "Found left over flat file"
          if [ ! -e "${vmbackupfiles/-flat.vmdk/.vmdk}" ]; then
           logit "Removing left over flat (orphan)"
           rm -f "${vmbackupfiles}"
           if [ $? -ne 0 ]; then
            logit "ERROR : Removing left over flat (orphan) : rm -f '${vmbackupfiles}'"
           fi
          fi
         fi

        fi
        ;;
       *".vmdk") ## Convert to thin (or at least keep thin provisioning)
        vmrestorefilesflat="${vmrestorefiles%.*}-flat.vmdk"
        
        if [ ! -e "${vmrestorefiles}.converted" ]; then

         logit "Looking for any preveous convert attemps : '${vmrestorefiles}' or '${vmrestorefilesflat}'"         
         if [ -e "${vmrestorefiles}" ] || [ -e "${vmrestorefilesflat}" ]; then

          if [ -e "${vmrestorefiles}" ]; then
           logit "Removing preveous convert attemp : '${vmrestorefiles}'"
           rm -f "${vmrestorefiles}"
           if [ $? -ne 0 ]; then
            logit "ERROR : removing preveous convert attemp : rm -f '${vmrestorefiles}'"
           fi
          fi

          if [ -e "${vmrestorefilesflat}" ]; then
           logit "Removing preveous convert attemp (flat) : '${vmrestorefilesflat}'"
           rm -f "${vmrestorefilesflat}"
           if [ $? -ne 0 ]; then
            logit "ERROR : removing preveous convert attemp : rm -f '${vmrestorefilesflat}'"
           fi
          fi

         fi


         logit "Convert : Start : $(date)"
         logit "Converting : '${vmbackupfiles}' => '${vmrestorefiles}'"
         $vmkfstools -d thin -i "${vmbackupfiles}" "${vmrestorefiles}" > "${vmrestorefiles}.convert.log"
         if [ $? -eq 0 ] && [ $(grep -c '100% done.' "${vmrestorefiles}.convert.log") -eq 1 ]; then
          logit "Convert : End : $(date)"
          touch "${vmrestorefiles}.converted"
          logit "Created : ${vmrestorefiles}.converted"

          logit "Looking for vmdk file : '${vmbackupfiles}'"
          if [ -e "${vmbackupfiles}" ]; then
           echo "Removing Converted VHD : ${vmbackupfiles}"
           rm -f "${vmbackupfiles}"
          fi
 
          logit "Looking for flat file : '${vmbackupfilesdir}/${vmbackupfilesname%.*}-flat.vmdk'"
          if [ -e "${vmbackupfilesdir}/${vmbackupfilesname%.*}-flat.vmdk" ]; then
           echo "Removing Converted Flat VHD : ${vmbackupfiles%.*}-flat.vmdk"
           rm -f "${vmbackupfilesdir}/${vmbackupfilesname%.*}-flat.vmdk"
          fi
 
         else 
          logit "Convert : End : $(date)"
          logit "ERROR : $vmkfstools -d thin -i '${vmbackupfiles}' '${vmrestorefiles}'"
          vmskip=1
         fi
        else
         logit "Already Convereted found : '${vmrestorefiles}.converted'"
         if [ $(grep -c '100% done.' "${vmrestorefiles}.convert.log") -eq 1 ]; then 
          logit "Converted log reports 100% complete : grep -c '100% done.' '${vmrestorefiles}.convert.log'"
          logit "Looking for left over vmdk file : '${vmbackupfiles}'"
          if [ -e "${vmbackupfiles}" ]; then
           echo "Removing Converted VHD : ${vmbackupfiles}"
           rm -f "${vmbackupfiles}"
          fi

          logit "Looking for left over flat file : '${vmbackupfilesdir}/${vmbackupfilesname%.*}-flat.vmdk'"
          if [ -e "${vmbackupfilesdir}/${vmbackupfilesname%.*}-flat.vmdk" ]; then
           echo "Removing Converted Flat VHD : ${vmbackupfiles}"
           rm -f "${vmbackupfilesdir}/${vmbackupfilesname%.*}-flat.vmdk"
          fi
          
         fi
        fi
        ;;
       *".vmx") ## Copy and modify base config
        vmbackupdate="${vmrestoredir%%_*}"
        vmbackupdate="${vmbackupdate##*/}"
        vmbackupdate="${vmbackupdate/${vmname}-/}"
        cp -f "${vmbackupfiles}" "${vmrestorefiles}.org"
        if [ $? -eq 0 ]; then
         logit "Copied Config : ${vmrestorefiles}.org"
         cat "${vmrestorefiles}.org" | sed -e 's/^displayName\ \=\ \".*"$/displayName\ \=\ \"'"${vmbackupdate} ${vmname}"'"/' > "${vmrestorefiles}"
         if [ $? -eq 0 ]; then
          logit "Mod'ed Config : ${vmrestorefiles}"
          rm -f "${vmbackupfiles}"
         else
          logit "ERROR : Mod'ing Config : ${vmrestorefiles}"
          vmskip=1
         fi
        else
         logit "ERROR : Copying Config : ${vmbackupfiles} => ${vmrestorefiles}.org"
         vmskip=1
        fi
        ;;
       *) ## Default copy any non known file
        if [ ! -d "${vmrestoredir}" ]; then
         mkdir -p "${vmrestoredir}"
         if [ $? -ne 0 ]; then
          logit "ERROR :  mkdir -p '${vmrestoredir}'"
          vmskip=1
         fi
        fi
        if [ -d "${vmrestoredir}" ]; then
         cp -f "${vmbackupfiles}" "${vmrestorefiles}"
         if [ $? -eq 0 ]; then
          logit "Copied MISC file : ${vmbackupfiles}"
          rm -f "${vmbackupfiles}"
         else 
          logit "ERROR : Coping MISC file : cp -f '${vmbackupfiles}' '${vmrestorefiles}'"
          vmskip=1
         fi
        else 
         logit "ERROR : Coping MISC file : no dir '${vmrestorefiles}'"
         vmskip=1
        fi
        ;;
      esac
     fi

     if [ $vmskip -eq 1 ]; then
      touch ${vmrestoredir}/BACKUP.err
     fi

    done

    if [ $(find "${vmbackupdir}" -type f | wc -l) -eq 1 ] && [ ! -e "${vmrestoredir}/BACKUP.err" ]; then     
     logit "Found only remaning file : $(find '${vmbackupdir}' -type f)"

     touch "${vmrestoredir}/STATUS.ok"
     if [ $? -eq 0 ]; then
      logit "Created STATUS.ok file : ${vmrestoredir}/STATUS.ok"
      rm -f "${vmbackupdir}/STATUS.ok"
      if [ $? -ne 0 ]; then
       logit "ERROR : unable to delete backup source STATUS.ok : rm -f '${vmbackupdir}/STATUS.ok'"
      fi
     else
      logit "ERROR : Creating STATUS.ok : touch '${vmrestoredir}/STATUS.ok'"
     fi 

     if [ $? -eq 0 ]; then
      rm -f "${vmbackupdir}/STATUS.ok"
     fi

     vmbackupdir="${vmbackupdir//../}"
     case "${vm2delete}" in
      *[!/]*/)
       vmbackupdir=${vmbackupdir%"${vmbackupdir##*[!/]}"}
      ;;
     esac
                         
     if [ -d "vmbackupdir" ]; then
      if [ "${vmbackupdir}" != "/vmfs/volumes/${vmstore_backup}" ] && [ "${vmbackupdir}" != "/vmfs/volumes" ] && [ "${vmbackupdir}" != "/vmfs" ]; then
       rm -rf "${vmbackupdir}"
       if [ $? -eq 0 ]; then
        logit "Removed : '${vmbackupdir}'"
       else
        logit "ERROR : rm -rf '${vmbackupdir}'"
       fi 
      fi 
     fi

    else 
     logit "ERROR : Found remaning files : $(find ${vmbackupdir} -type f)"
    fi

   done
  fi
 else
  logit "Found misc file : ${vmdirs}"
 fi

done

logit "======= Start Old Removal ======="

find /vmfs/volumes/${vmstore_restore}/ -type d -maxdepth 1 -mindepth 1 -not -iname ".*" | while read vmbase; do
 foundvms=$(find "${vmbase}" -type f \( -name "STATUS.ok" -o -name "BACKUP.err" \) | wc -l)
 logit "Old Copies Count : ${foundvms} for ${vmbase}" 

 if [ $foundvms -gt $max_copies ]; then
  logit "Over Max Copies : ${vmbase} by $(($foundvms - $max_copies))"

  ls -1cr "${vmbase}" | head -n $(($foundvms - $max_copies)) | while read vmdelete; do

   vm2delete="${vmbase}/${vmdelete}/"
   vm2delete="${vm2delete//../}"

   case "${vm2delete}" in 
    *[!/]*/) 
     vm2delete=${vm2delete%"${vm2delete##*[!/]}"}
     ;; 
   esac
   logit "Removing Old Copy : ${vmbase}/${vmdelete}"
   
   if [ "${vm2delete}" != "/vmfs/volumes/${vmstore_restore}" ] && [ "${vm2delete}" != "/vmfs/volumes" ] && [ "${vm2delete}" != "/vmfs" ]; then
    echo rm -rf "${vm2delete}"
    if [ $? -eq 0 ]; then
     logit "Removed : ${vm2delete}"
    else
     logit "ERROR : Removing Old Copy : '${vmbase}/${vmdelete}'"
    fi
   fi
   
  done

 fi
done

logit "======= Completely Done ========="


This script does all the registering and un-registering of the backups.

register_guests.sh

vmstore_backup="datastore1"

find "/vmfs/volumes/${vmstore_backup}/" -type d -maxdepth 1 -mindepth 1 -not -iname "scripts" | while read vmname; do
 ls -1r "${vmname}"/*/STATUS.ok | head -n 1 | while read vmstatusfile; do

  ## Setup some vars
  vmpath=$(dirname "${vmstatusfile}") 
  vmversion=${vmpath##*/} 

  vmname=${vmpath%/*}
  vmname=${vmname##*/} 

  vmdate=${vmpath##*/}
  vmdate=${vmdate/${vmname}-/}
  vmdate=${vmdate%_*} #2017-03-27

  vmdisplayname="${vmname}_${vmdate}"

  # See if this VM is already reg.
  if [ $(vim-cmd vmsvc/getallvms | grep -c -i "${vmname}") -eq 0 ]; then
   # if not reg it
   echo "Adding : ${vmversion}"
   find "${vmpath}" -iname "*.vmx" -exec vim-cmd solo/registervm '{}' \;
  else
   # test is this version is already reged, if not unreg the existing and reg this opne
   if [ $(vim-cmd vmsvc/getallvms | grep -c "${vmversion}") -eq 0 ]; then
    # get vmid of exiting vm guest
    vmid=$(vim-cmd vmsvc/getallvms | grep -i "${vmname}" | awk '{print $1}')
    # chec kthat it is not on. if it is, we must be using it and SHOULD NOT remove it
    if [ $(vim-cmd vmsvc/power.getstate ${vmid} | grep -c "Powered off") ]; then
     vim-cmd vmsvc/unregiste ${vmid}
     # check that it was removed
     if [ $(vim-cmd vmsvc/getallvms | grep -c -i "${vmname}") -eq 0 ]; then
      # add the new one
      echo "Adding : ${vmversion}"
      find "${vmpath}" -iname "*.vmx" -exec vim-cmd solo/registervm '{}' \;
     fi
    fi
   else
    echo "Skipping : ${vmversion}"
   fi
  fi

 done

done




Comments