macOS - status-file not created if run-before script fails?

[HealsCodes]

I'm not sure if it is intentional (or maybe even an issue with my specific configuration) but it seems that the status-file is only ever written on successful backup?

Background:

• three profiles in my yaml file: base, local, cloud - the later two inherit common settings from base
• status-file: /private/var/tmp/restic_{{ .Profile.Name }}.status.json set in base
• run-before scripts for local and cloud

Both of my active profiles check via a run-before script if they should run or exit early and in either case I don't see the status-file being created if it was missing nor being updated if it existed from a previous successful run.

---

For the sake of "it's a me issue" here's a slightly redacted version of my resticprofile:

# yaml-language-server: $schema=https://creativeprojects.github.io/resticprofile/jsonschema/config.json

version: "2"

# global settings
global:
  default-command: snapshots
  initialize: true
  log: 'syslog:'

# -- template variables used by all profiles --
# {{ $get_passwd := "/usr/bin/security find-generic-password -s resticprofile -w -a"}}
#
profiles:
  base:
    status-file: "/private/var/tmp/restic_{{ .Profile.Name }}.status.json"
    password-command: "{{ $get_passwd }} restic-repository-{{ .Profile.Name }}"

    backup:
      source:
        - "/private/var/tmp/restic_{{ .Profile.Name }}/Users/XXXX"
      exclude-caches: true
      exclude-file:
        - restic-excludes.all

      run-before:
        - 'sudo -E {{ .ConfigDir }}/restic-apfs-snapshot create -v /System/Volumes/Data -m "/private/var/tmp/restic_{{ .Profile.Name }}"'

      run-finally:
        - sudo -E {{ .ConfigDir }}/restic-apfs-snapshot remove

    retention:
      keep-last: 24
      keep-daily: 30
      keep-weekly: 52
      keep-monthly: 12
      prune: true
      after-backup: true

  # local backups to an external disk
  # -- template variables used by the profile --
  # {{ $DISK_UUID := "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX" }} # external SSD UUID
  # {{ $PART_UUID := "YYYYYYYY-YYYY-YYYY-YYYY-YYYYYYYYYYYY" }} # backup partiton UUID
  local:
    inherit: base
    no-cache: true

    repository: "local:/Volumes/Backup Repo/restic"

    run-before:
      # check if the external drive is connected
      - 'diskutil info  {{ $DISK_UUID }} 2>&1 &>/dev/null'
      # attempt to mount the backup volume
      - 'diskutil mount {{ $PART_UUID }}'

    run-after-fail:
      - 'echo -n "$ERROR_COMMANDLINE" | grep -q {{ $DISK_UUID }}
         || osascript -e "display notification \"$ERROR_STDERR\"
            with title \"Restic Backup Failed\" subtitle \"$ERROR\""'

    run-finally:
      - 'diskutil umount {{ $PART_UUID }}
         || diskutil info {{ $PART_UUID }} | grep -qE "Mounted: *No"'

    backup:
      schedule: "*:00"
      schedule-permission: "auto"
      schedule-ignore-on-battery: true

    check:
      read-data-subset: '{{ len (printf "a%*s" .Now.Weekday "") }}/7'
      schedule: "*-*-* 12:15:00"
      schedule-permission: "auto"
      schedule-ignore-on-battery: true

  cloud:
    inherit: base
    repository: "s3:https://s3.xxxxxxxx.wasabisys.com/xxxxxxxx"
    initialize: true
    cleanup-cache: true

    run-before:
      # check if we're online at all
      - 'ping -c3 -q heals.codes 2>&2 >/dev/null'
      # check if we're connected to a preferred WiFi network
      - 'networksetup -listpreferredwirelessnetworks "en0"
          | sed -n "2s/^\t//p" | grep -qE "(aaaaa|bbbbbb|cccccc)"'
      # fetch credentials for Wasabi
      - 'echo AWS_ACCESS_KEY_ID="`{{ $get_passwd }} restic-aws-access-key-{{ .Profile.Name }}`" >> {{env}}'
      - 'echo AWS_SECRET_ACCESS_KEY="`{{ $get_passwd }} restic-aws-secret-key-{{ .Profile.Name }}`" >> {{env}}'

    run-after-fail:
      - 'echo -n "$ERROR_COMMANDLINE" | grep -q "(ping|networksetup)"
         || osascript -e "display notification \"$ERROR_STDERR\"
            with title \"Restic Backup Failed\" subtitle \"$ERROR\""'

    retention:
      keep-daily: 30
      keep-weekly: 8
      keep-monthly: 6

    backup:
      exclude-file__APPEND:
        - restic-excludes.cloud
      schedule: "8,10,12,14,16,18,20:30"
      schedule-permission: "auto"
      schedule-ignore-on-battery: true

    check:
      read-data-subset: '{{ len (printf "a%*s" .Now.Weekday "") }}/7'
      schedule: "*-*-* 12:45:00"
      schedule-permission: "auto"
      schedule-ignore-on-battery: true

[creativeprojects]

Hi,

The status file is only created once the profile command runs

If you do start a profile command (check, forget, backup, copy) on an empty directory (and assuming you keep initialize to false) you will get a status file with an error.

In your case I'm assuming the run-before returns an error and prevents any other command from running.

This is something I should fix at some point but it's rather complicated actually 😢

In the meantime, in my monitoring, I always throw an alert on the latest run of the status file being too old (>24h if you run a backup every day) so I never miss a backup failure of any reason.

I should add this information to the documentation 👍🏻

[HealsCodes]

If you do start a profile command (check, forget, backup, copy) on an empty directory (and assuming you keep initialize to false) you will get a status file with an error.

Would that be true even with my run-before returning an error? I had initialize set to true initially when running my first backups setting it to false wouldn't be an issue now!

In your case I'm assuming the run-before returns an error and prevents any other command from running.

Yeah, that's the case - I'm using the profile run-* hooks to check for conditions that prevent any of the commands (like being on a blacklisted Wifi for cloud.* or not having the backup disk connected for local.* - it felt like a sensible approach vs. duplicating the same checks for each of the commands.

(A little background to the while issue: I'd like to write a status bar application for my mac that gives me a glance at the status of my resticprofile backups. I could manage that with additional run- or even http- pings but thought "hey there's a nice status file I can simply parse!)

[creativeprojects]

(A little background to the while issue: I'd like to write a status bar application for my mac that gives me a glance at the status of my resticprofile backups. I could manage that with additional run- or even http- pings but thought "hey there's a nice status file I can simply parse!)

I'd be quite interested in using such a little app actually 😆