Tag: Mac

  • Community Projects: Part 2

    DiskEncrypter – a shared shell script helps the world

    A Community Code Evolution Story: From Simple Utility to Security Tool

    Introduction

    What started as a straightforward 515-line bash script in October 2022 has evolved into a, uh, slightly more complicated, 1,125-line “enterprise-grade” encryption enforcement system.

    This is the story of how a simple external drive encryption script was shared with the community and provided a starting point for others to create their own “enhanced” utility which can now be one piece of a security solution that protects users from data loss, encourages users to encrypt external drives, and provides a framework to enforce an encryption policy.

    This DiskEncrypter_Enhanced.sh script is also a journey that spans nine major versions, each addressing real-world problems discovered through testing and user feedback. Along the way, I learned valuable lessons about user experience, data safety, and the importance of preventing accidental data loss—especially when it comes to irreplaceable photos and videos on camera cards.

    Chapter 1: The Beginning – DiskEncrypter.sh v1.0 (October 2022)

    The Original Vision

    Created by Thijs Xhaflaire, the original `DiskEncrypter.sh`script had a clear mission: automatically detect unencrypted external drives and prompt users to encrypt them. It was designed for macOS enterprise environments where data security compliance required all removable media to be encrypted.

    What It Did Well

    The original script handled the basics competently:

    – Detected external APFS, HFS+, and ExFAT/FAT volumes

    – Prompted users with a swiftDialog interface

    – Encrypted APFS volumes directly

    – Converted HFS+ volumes to APFS before encryption

    – Erased and reformatted ExFAT/FAT volumes as encrypted APFS

    – Offered a “mount as read-only” option for users who didn’t want encryption

    The Hidden Limitations

    But as deployments grew, limitations became apparent:

    1. Single-Volume Processing**

    If you inserted a disk with three partitions, the script would only process the first one. The other two? Invisible to the system.

    “`bash

    Original approach

    if [[ $StorageType =~ “Apple_APFS” ]]; then

        # Process APFS

    elif [[ $StorageType =~ “Apple_HFS” ]]; then

        # Process HFS+

    elif [[ $StorageType =~ “Microsoft Basic Data” ]]; then

        # Process ExFAT

    fi

    Only ONE branch executed, then script exits!

    “`

    2. No Logging Infrastructure**

    Troubleshooting was a nightmare. Basic `echo` statements went nowhere when run as a LaunchDaemon.

    3. No Testing Capability

    Want to test the script? Hope you don’t mind encrypting real drives, because there was no dry-run mode.

    4. Generic Error Messages

    “FileVault is disabled on disk4s1” – Great, but which drive is that? What’s it called?

    5. The Re-Prompt Problem

    User mounts a drive as read-only. Five minutes later, they insert another drive. The script triggers again and asks about BOTH drives, including the one already mounted read-only. Every. Single. Time.

    Chapter 2: The First Major Overhaul – v2.0 Enhanced (December 2025)

    ### Modernization for macOS 15+ Sequoia

    When macOS 15 (Sequoia) shipped, several compatibility issues emerged. This sparked a complete reimagining of the script’s architecture.

    ### The Two-Phase Architecture

    The biggest change was moving from a linear “process-and-exit” model to a sophisticated two-phase system:

    **Phase 1: Discovery**

    “`bash

    # Scan ALL drives

    # Detect ALL partitions

    # Check ALL volume types independently

    # Build a queue of unencrypted volumes

    “`

    **Phase 2: Processing**

    “`bash

    # Process each queued volume sequentially

    # Track what was encrypted

    # Show comprehensive summary at end

    “`

    This solved the multi-volume problem instantly. Now a disk with three APFS volumes, two HFS+ partitions, and an ExFAT partition would have ALL of them discovered and processed.

    ### The Multi-Volume Victory

    Here’s a real example of the difference:

    **Original v1.0 (Single partition found):**

    “`

    Processing /dev/disk4

    Found: disk4s1 (APFS volume “Old”)

    Encrypted: disk4s1

    Exit.

    “`

    **Enhanced v2.0 (All partitions found):**

    “`

    Discovery Phase:

    – disk29s1 “Old” (APFS, unencrypted) → Added to queue

    – disk29s2 “New” (APFS, unencrypted) → Added to queue

    – disk29s3 “Lucky” (APFS, unencrypted) → Added to queue

    – disk28s3 “Untitled 2” (HFS+, unencrypted) → Added to queue

    Processing Phase:

    [1/4] Encrypting “Old” (disk29s1)… Done

    [2/4] Encrypting “New” (disk29s2)… Done

    [3/4] Encrypting “Lucky” (disk29s3)… Done

    [4/4] Converting and encrypting “Untitled 2” (disk28s3)… Done

    Summary: 4 volumes encrypted in this session

    “`

    Volume Names Everywhere

    Every dialog, every log message, every notification now showed friendly volume names alongside technical IDs:

    “`

    Before:”Processing disk4s1″

    After:”Processing ‘MyBackups’ (disk4s1)”

    “`

    Users could finally understand what was happening to which drive.

    The LaunchDaemon Dialog Fix

    A critical bug emerged: when run as a LaunchDaemon (root), swiftDialog couldn’t accept keyboard input. Password fields were useless. The fix used `launchctl asuser` to run dialogs in the user’s GUI session context:

    Now dialogs appeared correctly with proper focus and keyboard interaction.

    Chapter 3: Production Readiness – v2.1 (December 2025)

    ### The Logging Revolution

    v2.1 introduced a professional four-level logging system:

    | Level | Output | Use Case |

    |——-|——–|———-|

    | **0 – Minimal** | Errors only | Production (silent success) |

    | **1 – Normal** | Errors + key operations | Standard deployment |

    | **2 – Verbose** | + detailed progress | Troubleshooting |

    | **3 – Debug** | Everything | Development/diagnosis |

    **Dual-destination logging** sent output to both console and macOS unified logging:

    “`bash

    log_info() {

        if [[ $LOG_LEVEL -ge 1 ]]; then

            echo “[$(get_timestamp)] INFO: $*”

            logger -p user.info “DiskEncrypter [INFO]: $*”

        fi

    }

    “`

    Now you could watch logs in real-time:

    “`bash

    log stream –predicate ‘eventMessage CONTAINS[c] “DiskEncrypter”‘ –info

    “`

    ### Dry-Run Mode: Test Without Fear

    “`bash

    sudo ./DiskEncrypter_Enhanced.sh –dry-run –log-level 3

    “`

    This became invaluable for:

    – Testing in production environments

    – Training new IT staff

    – Validating configuration changes

    – Demonstrations

    Every disk operation was logged but not executed:

    “`

    [DRY RUN] Would execute: diskutil apfs encryptVolume disk4s1 -user disk

    INFO: DRY RUN: Encryption would start for disk4s1

    “`

    ### Command-Line Arguments

    For the first time, you could override plist settings from the command line:

    “`bash

    # Debug logging without changing plist

    ./DiskEncrypter_Enhanced.sh -l 3

    # Dry-run with verbose logging

    ./DiskEncrypter_Enhanced.sh –dry-run -l 2

    “`

    Priority: **Command-line → Plist → Default**

    Chapter 4: The Bug Hunt – v2.2 (December 2025)

    ### The Critical Read-Only Bug

    A critical bug was discovered: the script couldn’t detect read-only volumes, causing users to be re-prompted endlessly.

    **The Bug:**

    “`bash

    # Wrong field name!

    volumeMountInfo=$(diskutil info “$VolumeID” | grep “Read-Only Volume:”)

    if [[ “$volumeMountInfo” == “Yes” ]]; then

        # This NEVER executed!

    fi

    “`

    **Why It Failed:**

    “`bash

    $ diskutil info disk4s2 | grep “Read-Only Volume:”

    # (no output – field doesn’t exist!)

    $ diskutil info disk4s2 | grep “Volume Read-Only:”

       Volume Read-Only:          Yes (read-only mount flag set)

    # ^^^ The actual field name!

    “`

    **The Fix:**

    “`bash

    volumeMountInfo=$(diskutil info “$VolumeID” | grep “Volume Read-Only:”)

    if [[ “$volumeMountInfo” =~ ^Yes ]]; then

        log_info “Volume $VolumeID ($volumeName) is mounted read-only, skipping”

        continue

    fi

    “`

    This simple fix eliminated the annoying re-prompt problem.

    ### NTFS Support Added

    Windows NTFS volumes were now recognized and handled:

    “`bash

    if [[ $StorageInfo =~ “Microsoft Basic Data” ]] ||

       [[ $StorageInfo =~ “Windows_FAT” ]] ||

       [[ $StorageInfo =~ “DOS_FAT” ]] ||

       [[ $StorageInfo =~ “Windows_NTFS” ]]; then  # NEW!

    “`

    Chapter 5: The Safety Revolution – v2.3 (December 2025)

    ### The Security Gap

    A realization hit: unencrypted drives were mounted read/write while waiting for user decision. This created a window where data could accidentally be written to unencrypted media.

    ### Auto Read-Only Mounting

    v2.3’s solution was elegant: automatically mount ALL unencrypted volumes as read-only immediately upon detection, before showing any dialog.

    **New Function:**

    “`bash

    mountReadOnly() {

        local VolumeID=$1

        local volumeName=$2

        log_info “Auto-mounting volume as read-only: $VolumeID ($volumeName)”

        # Unmount first

        diskutil unmountDisk “$VolumeID” 2>/dev/null

        # Mount as read-only

        diskutil mount readOnly “$VolumeID”

        return 0

    }

    “`

    **Updated Workflow:**

    “`

    v2.2:

    1. Drive detected → Mounted read/write ⚠️

    2. Dialog shown

    3. If “Mount read-only” → Remount

    v2.3:

    1. Drive detected → Auto-mounted read-only ✅

    2. Dialog shown (drive already safe)

    3. If “Keep Read-Only” → Already done!

    “`

    ### Updated User Messages

    Dialogs now explicitly stated the protection status:

    “`

    This volume has been mounted as read-only for your protection.

    To write files, you must encrypt the disk.

    “`

    Button labels changed:

    – **Before:** “Mount as read-only”

    – **After:** “Keep Read-Only” (more accurate)

    ### Comprehensive User Documentation

    v2.3 introduced `USER_GUIDE.md`, a 22KB end-user safety manual with:

    – Clear explanation of the three button options

    – Decision guide chart by drive type

    – Step-by-step backup workflows

    – Critical warnings for ExFAT/FAT32 (data loss risk)

    – Password management best practices

    **Example Safe Workflow:**

    “`

    WRONG:

    ❌ Insert ExFAT drive with photos → Click “Encrypt” → ALL PHOTOS DELETED

    CORRECT:

    ✅ Click “Keep Read-Only”

    ✅ Copy all photos to Mac

    ✅ Verify photos copied correctly

    ✅ Eject drive

    ✅ Re-insert drive

    ✅ NOW click “Encrypt”

    ✅ Photos safe on Mac, drive encrypted

    “`

    Chapter 6: The Camera Card Protection – v2.4 (December 2025)

    ### The Data Loss Catastrophe Waiting to Happen

    v2.3 was safer, but a critical vulnerability remained: users could still choose to “Erase and Encrypt” ExFAT/FAT32 drives, which contained camera cards with irreplaceable wedding photos, vacation videos, and professional photography.

    **The Risk Scenario:**

    “`

    1. Wedding photographer inserts SD card (ExFAT) with 500 photos

    2. Dialog appears: “Erase and Encrypt” option available

    3. Photographer thinks “encrypt” = “protect my photos”

    4. Clicks button, enters password

    5. ❌ ALL 500 WEDDING PHOTOS DELETED FOREVER

    “`

    This was a bad situation waiting to happen.

    ### The Bold Decision: Remove the Erase Option

    v2.4 made a controversial choice: **completely remove the encrypt option for ExFAT/FAT/NTFS volumes.**

    This volume cannot be encrypted without erasing all data.

    To protect your data from accidental loss, encryption is not

    offered for this disk type (ExFAT/FAT/NTFS).

    Why encryption is not offered:

    • Encrypting this volume type requires complete erasure

    • All existing data would be permanently lost

    • This protection prevents accidental data loss on camera cards,

      USB drives, and other portable media

    To encrypt this drive:

    1. Back up all data to a secure location

    2. Use Disk Utility to erase and format as APFS

    3. Then encryption can be applied without data loss”

    <

    Chapter 7: The UX Polish – v2.4.1 (December 2025)

    ### The Scrolling Problem

    v2.4’s dialogs had too much text—users had to scroll to see everything. This defeated the purpose of clear communication.

    **The Problem:**

    “`

    Main Dialog:

    [15+ lines of text explaining everything]

            ▼ SCROLL REQUIRED ▼

    “`

    ### The Infobox Solution

    v2.4.1 leveraged swiftDialog’s `–infobox` parameter to split content:

    “`bash

    # Concise main message (5 lines)

    customMessage=”Non-encryptable volume: **\”$volumeName\”** ($VolumeID)

    File System: **$fsType**

    $subTitleNonEncryptable”

    # Detailed infobox (collapsible)

    infoboxMessage=”### Why Encryption Is Not Offered

    • Encrypting this volume type requires **complete erasure**

    • All existing data would be **permanently lost**

    • This protection prevents accidental data loss

    ### To Encrypt This Drive (If Needed)

    **Step 1:** Back up all data to a secure location

    **Step 2:** Open Disk Utility and erase the drive

    **Step 3:** Format as **APFS** (Mac only)

    **Step 4:** Re-insert for automatic encryption

    ⚠️ **Warning:** Only proceed if you have backed up all data!”

    Chapter 8: The Feedback Loop Fix – v2.4.3

    ### The Duplicate Dialog Bug

    Just when everything seemed perfect, a new bug emerged in production:

    **The Problem:**

    “`

    1. User inserts USB drive

    2. Script displays dialog

    3. User clicks “Eject”

    4. Script executes: diskutil unmountDisk disk4

    5. ❌ Unmount event triggers LaunchDaemon AGAIN

    6. ❌ Script runs concurrently with first instance

    7. ❌ Dialog appears TWICE

    8. ❌ User must click “Eject” again

    “`

    The LaunchDaemon was monitoring both mount AND unmount events, creating a feedback loop.

    The Two-Pronged Solution

    v2.4.3 implemented two complementary mechanisms:

    **1. Lock File Mechanism (Prevents concurrent execution)**

    “`bash

    LOCK_FILE=”/var/run/diskencrypter.lock”

    **2. Processed Volumes Tracking (Prevents re-processing)**

    “`bash

    PROCESSED_VOLUMES_FILE=”/var/tmp/diskencrypter_processed.txt”

    COOLDOWN_SECONDS=30

    Chapter 9: The Spaces In Between – v2.4.5

    A few more big fixes happened in v2.4.4 and v2.4.5 which addresses spaces not allowed in the password field and the password hint field and suffice it to say that regex was the issue. And testing was required. And a second fix for the first fix when the installer was discovered to be setting an old default in the management preferences. Ooops!

    Chapter 10: Lessons Learned

    ### 1. Users Don’t Understand Technical Terms

    “Erase and encrypt” sounds like “protect my data” to non-technical users. The v2.4 solution of removing the option entirely was controversial but necessary.

    **Key Learning:** When data loss is possible, don’t rely on warnings—remove the dangerous option.

    ### 2. Edge Cases Are Real Cases

    The read-only field name bug (v2.2) seemed minor until users reported constant re-prompts. The duplicate dialog bug (v2.4.3) only appeared in production when LaunchDaemon unmount events triggered re-execution.

    **Key Learning:** Real-world testing reveals issues that synthetic tests miss.

    ### 3. Good UX Is Iterative

    v2.4’s dialogs had too much text. v2.4.1’s infobox solution reduced reading time by 66%. Sometimes the best improvement is reduction, not addition.

    **Key Learning:** Watch real users interact with your interface. Simplify ruthlessly.

    ### 4. Logging Saves Lives (and Debugging Time)

    The v2.1 four-level logging system with dual output (console + system logger) paid dividends in every subsequent version. Debug output that would have taken hours to add per-bug was already there.

    **Key Learning:** Invest in logging infrastructure early. Future you will thank present you.

    ### 5. Backward Compatibility Matters

    Every version maintained 100% backward compatibility with configuration files and user workflows. APFS/HFS+ encryption worked identically across all versions.

    **Key Learning:** Add features, don’t break existing deployments.

    ### 6. Concurrent Execution Is Harder Than It Looks

    The feedback loop bug (v2.4.3) required two complementary solutions: lock files for mutual exclusion AND processed volume tracking for cooldowns.

    **Key Learning:** Concurrent execution requires multiple layers of protection.

    Chapter 11: The Final Product

    ### What Disk Encrypter Enhanced v2.4.5 Delivers

    **For IT Administrators:**

    – Comprehensive logging with 4 verbosity levels

    – Dry-run mode for safe testing

    – Command-line arguments for flexibility

    – Automatic log rotation (30-day retention)

    – Lock file protection against race conditions

    – Processed volume tracking with configurable cooldown

    **For End Users:**

    – Auto read-only mounting (immediate protection)

    – Clear, friendly volume names in all dialogs

    – No risk of accidental data loss on camera cards

    – Clean, professional dialog layout (no scrolling)

    – One dialog per user decision (no duplicates)

    – Educational content explaining technical concepts

    **For Compliance Officers:**

    – Complete audit trail in system logs

    – Encryption enforcement for compatible volumes

    – Safe handling of non-compatible media

    – User acknowledgment tracking

    – Session-based encryption reporting

    Chapter 12: Looking Forward

    ### What’s Next?

    The evolution from v1.0 to v2.4.5 represents maturity, but there’s always room for improvement:

    **Potential Future Enhancements:**

    – Web-based dashboard for monitoring encryption across enterprise

    – Centralized reporting to MDM systems

    – Email notifications for IT admins

    – Custom encryption policies per volume type

    – Integration with company password managers

    – Support for FileVault-encrypted APFS containers on external drives

    ### The Philosophy Going Forward

    The journey from v1.0 to v2.4.5 taught us three guiding principles:

    1. **User safety over feature completeness** – Removing the ExFAT erase option protected users from themselves

    2. **Comprehensive logging over simplicity** – The debugging investment paid dividends

    3. **Iterative refinement over big-bang releases** – Each version solved real problems discovered in the field

    ## Conclusion

    What started as a 515-line utility script in October 2022 has evolved into a 1,325-line enterprise-grade security solution. Along the way, we learned that good software is never “done”—it evolves through real-world use, user feedback, and a commitment to continuous improvement.

    The DiskEncrypter journey demonstrates that the best solutions emerge from:

    – Listening to users (the read-only re-prompt problem)

    – Protecting users from themselves (the camera card protection)

    – Obsessive attention to detail (the dialog UX refinement)

    – Comprehensive testing (the feedback loop bug discovery)

    Today, DiskEncrypter_Enhanced.sh v2.4.5 stands as a cautionary tale to what iterative refinement can achieve: a tool that not only enforces security policy but actively prevents data loss, provides excellent user experience, and operates reliably in production environments.

    The code journey continues.

    **Technical Stats:**

    – **Total Development Time:** ~12 days of Xmas coding (December 3-15, 2025)

    – **Lines of Code:** 515 → 1,125 (+157%)

    – **Functions:** 2 → 25+ (+1,150%)

    – **Major Versions:** 9 (v2.0, v2.1, v2.2, v2.3, v2.4, v2.4.1, v2.4.3, v2.4.4, v2.4.5)

    – **Bug Fixes:** 4 critical issues resolved

    – **Compatibility:** macOS 15+ (Sequoia) and macOS 26+

    **Documentation:**

    – Evolution Guide: 1,844 lines

    – User Guide: 650+ lines

    **Acknowledgments:**

    – Original script by Thijs Xhaflaire (October 2022)

    swiftDialog by Bart Reardon

    – Testing and feedback from the MacAdmin community

    **License:**

    The DiskEncrypter_Enhanced.sh script is distributed for enterprise use. See individual script files for license details.

  • Community Projects: Part 1

    SetDefaultAppsX – A Community-Driven Evolution

    From Enterprise Lock-In to Universal macOS Tool

    When Scott Kendall released his SetDefaultApps script in December 2025, it solved a real problem: giving users a friendly GUI with swiftDialog to set their default applications using scriptingOSX’s utiluti for file types, URLs, and protocols on macOS. It worked beautifully—but only if you had Jamf Pro.

    That’s where the community stepped in.

    The MDM-specific Problem

    Scott’s original script was tightly coupled to Jamf Pro’s infrastructure. It relied on policy triggers for installing dependencies:


    jamf policy -trigger install_SwiftDialog
    jamf policy -trigger install_SymFiles
    jamf policy -trigger install_utiluti

    For enterprise Mac administrators already using Jamf, this was perfect. For everyone else—small businesses, education labs, home users, or shops using different MDM solutions—it was a non-starter.

    The X Factor: SetDefaultAppsX

    I took Scott’s excellent foundation and worked to make it a truly standalone tool. The “X” represents both the removal of dependencies and the cross-platform (MDM-agnostic) nature of the new version.

    Major Transformations

    1. Self-Contained Installation

    Instead of calling out to Jamf policies, SetDefaultAppsX now downloads swiftDialog directly from GitHub, verifies the package signature against Bart Reardon’s Team ID, and installs it automatically:


    expectedDialogTeamID="PWA5E9TQ59"
    LOCATION=$(curl -s https://api.github.com/repos/bartreardon/swiftDialog/releases/latest | awk -F '"' '/browser_download_url/ {print $4}')
    curl -L "$LOCATION" -o /tmp/swiftDialog.pkg
    # Verify signature before installation
    teamID=$(/usr/sbin/spctl -a -vv -t install "/tmp/swiftDialog.pkg" 2>&1 | awk '/origin=/ {print $NF}' | tr -d '()')

    No MDM required. No manual downloads. Just works.

    (Real time update: Writing this blog post made me add a feature to do the same with ScriptingOSX’s utiluti. Now the script checks and downloads both.)

    2. Hardware Detection That Actually Works

    Scott’s original used system_profiler, which sounds reasonable—until you run it in certain contexts where it returns “Unknown” due to , uh, issues. System_profiler cli is powerful, but it can time out and not always return the values I wanted. It could also be some script shenanigans but in any case.

    For my scripting needs the fix was simple but crucial: switch to sysctl queries that always work:


    # CPU Detection - Direct kernel query
    MAC_CPU=$(/usr/sbin/sysctl -n machdep.cpu.brand_string 2>/dev/null)

    # RAM Detection – Also via kernel
    MAC_RAM=$(/usr/sbin/sysctl -n hw.memsize 2>/dev/null | awk ‘{printf “%.0f GB”, $1/1024/1024/1024}’)

    The result? Instead of seeing “Unknown” or generic “chip_type”, I would see “Apple M3” on my MacBook Air or the full Intel CPU model for an Intel Mac. And it’s 20-30x faster.

    3. No Sudo Required

    The original required administrators to create directories in `/Library/Application Support` before users could run the script. Either management tools created this beforehand or users ran a script with sudo which was not ideal or workable for non-admins. So, SetDefaultAppsX includes automatic fallback:


    if [[ ! -d "${SUPPORT_DIR}" ]]; then
    echo "WARNING: Application Support directory not found"
    echo "Falling back to /Users/Shared/SetDefaultAppsX"
    SUPPORT_DIR="/Users/Shared/SetDefaultAppsX"
    # Automatically create writable directories
    /bin/mkdir -p "${SUPPORT_DIR}"
    fi

    Users can run the script immediately without any preparation. For enterprise deployments, there’s an optional `PrepareSetDefaultAppsX.sh` that sets up system-wide directories, but it’s truly optional.

    4. Modern Icon System

    Again, in my hacking of Scott’s perfectly working script, I ran into some required banner images and so instead of relying on file system icon resources that might not exist, SetDefaultAppsX uses SF Symbols:


    OVERLAY_ICON="SF=xmark.circle,weight=medium,colour1=#000000,colour2=#ffffff"

    Always available, always renders perfectly, and customizable.

    The X2 Portable Edition: Platypus-Ready

    But I didn’t stop there. I had an idea that users could run an app easily so SetDefaultAppsX2 takes portability even further—it’s designed for packaging as a standalone application using Platypus.

    The key difference is local binary detection:


    # Get the directory where the script is located
    SCRIPT_DIR="${0:a:h}"

    # Check for binaries in script directory first
    if [[ -x “${SCRIPT_DIR}/utiluti” ]]; then
    UTI_COMMAND=”${SCRIPT_DIR}/utiluti”
    elif [[ -x “/usr/local/bin/utiluti” ]]; then
    UTI_COMMAND=”/usr/local/bin/utiluti”
    fi

    This means you can package the script along with the `dialog` and `utiluti` binaries into a single app bundle with Platypus. Users double-click the app, and everything just works—no installation, no command line, no dependencies.

    Perfect for:

    • Quick distribution to non-technical users
    • Testing environments
    • Portable USB installations
    • Labs where users can’t install software system-wide

    Community Contributions Flow Both Ways

    The best part? Scott has been incorporating some of these improvements back into his Jamf-specific version. The hardware detection fixes and error handling enhancements benefit both the enterprise and standalone versions.

    This is open-source collaboration at its finest: Scott provided the excellent foundation and deep integration expertise, the community contributed cross-platform portability, and both versions improve together.

    The Technical Wins

    Let’s talk numbers:

    • Performance: 3-4x faster startup (sysctl vs system_profiler)
    • Reliability: 100% success rate for hardware detection (up from ~60%)
    • Portability: Works on any Mac, any MDM, or no MDM
    • Security: Package signature verification via Team ID
    • User Experience: No sudo required, automatic fallback, clear error messages

    What You Get

    Three versions for different needs:

    1. SetDefaultApps.sh – Scott’s original Jamf-integrated version
    2. SetDefaultAppsX.sh – MDM-agnostic standalone version
    3. SetDefaultAppsX2.sh – Portable version ready for Platypus packaging

    All three share the same excellent user interface powered by swiftDialog, the same UTI handling via utiluti, and the same goal: make setting default apps friendly and accessible.

    Getting Started

    The simplest possible workflow:

    # 1. Run the script
    ./SetDefaultAppsX.sh

    That’s it. The script downloads swiftDialog if needed, creates directories automatically, and presents users with a beautiful interface to set their default apps.

    For Platypus app building with X2:

    – Include dialog and utiluti binaries in your app bundle
    – Point Platypus to SetDefaultAppsX2.sh
    – Users get a double-clickable app with zero dependencies

    Credits Where Due

    – **Scott Kendall**: Original script author, Jamf integration expert
    – **Bart Reardon**: swiftDialog creator (the UI magic behind it all)
    – **scriptingOSX**: utiluti tool for UTI management
    – **The Community**: Testing, feedback, and collaborative improvements

    The Open Source Philosophy

    This is what makes the Mac admin community special. Scott could have kept his script locked down or enterprise-only. Instead, he shared it, accepted community modifications, and even pulled improvements back into his version.

    The result? Better tools for everyone—whether you’re managing 10,000 Macs with Jamf or helping your family set up their MacBooks.

    What’s Next?

    The scripts are stable and production-ready, but there’s always room for improvement:

    – Auto-installation of utiluti from GitHub releases (Done!)
    – Built-in default banner images
    – Dark mode support
    – Multi-language interface
    – Configuration file support for organizations

    But the foundation is solid: a truly portable, MDM-agnostic tool for one of macOS’s most user-requested features.

    Try it yourself: The full source code, documentation, and evolution guide are available in the project repository. Whether you need the Jamf version, the standalone version, or the portable Platypus version, there’s a SetDefaultApps that fits your workflow.

    Because good tools should be accessible to everyone, not just those with enterprise MDM budgets.

    *Special thanks to Scott Kendall for creating the original script and being open to community contributions, ScriptingOSX (Armin Briegel) for utiliti and to Bart Reardon for swiftDialog —the best thing to happen to Mac admin UIs in years. *

  • Don’t stop!

    Using a REST API to code a bunch of useful apps

    Background: I released some scripts to help users manage Archiware P5 servers on code.matx.ca with a first blog post as a background to the scripting approaches of cli vs REST API and then I discussed my cute Platypus built apps and my first swift/swiftUI Mac apps… so of course now I’m going to discuss two new Mac apps that use a REST API to explore a P5 Archive.

    I’m building tools to help me and my clients work with Archiware’s amazing and awesome P5 Archive product. It’s great. It archives, it restores, it has a great web UI, and it’s always getting better. So why build apps? Sometimes you want something different, in my case my clients wanted spreadsheets. Yup. Data in a sheet. To look at. So I poked at P5’s databases via cli (see P5 Archive Manager) and with the REST API. Here are the results, two new apps P5 Archive Overview and P5 Archive Search.

    https://code.matx.ca/ is code on GitHub + Mac apps that help manage data in Archiware P5

    Why API? And not cli

    Usually the cli (command line interface) is perfect for working in Terminal and in shell scripts or other programming languages. Using an API or application programming interface, allows different software applications to communicate and share data with each other. Instead of cli commands and arguments programs make requests using specific methods like GET or POST to retrieve or send data. See: API on Wikipedia

    Using an API for my Mac apps means they could use a protocol like HTTP, and make a request using GET (to retrieve data).

    The magic parts of an API

    • API Client: The application making the request.
    • Endpoints: Specific URLs where the API can be accessed.

    For the P5 Archive Overview app we need to use the specific API Endpoint for archive overview detailed by Archiware here which lucky for us is a simple call for data which our app can display, save as json, format as csv (for the spreadsheet!!) and stash in a SQLite database for historical searches.

    However, the P5 Archive Search app has to make many calls to walk through the index tree which is an inventory by path of the files archived. So we ask the user to name the storage they want to search and we make a breadcrumb through the storage, storing everything in SQLite as well as saving json and csv snippets of what we find. A lot more API calls but perfect for an app to do in the background.

    The P5 Archive Search app queries the P5 Archive Index Inventory REST API:

    “`

    GET http://{server}:{port}/rest/v1/archive/indexes/{archive-index}/inventory/{path}

    “`

    Example:

    “`

    http://192.168.1.100:8000/rest/v1/archive/indexes/Default-Archive/inventory/Volumes/BigStorageSMB

    Made for Macs, macOS 14 and up. Bring your own jq

    When running macOS 15 and up jq is installed for free and can help make the csv files form the json, but if you’re running macOs 14 then you need to install it with homebrew, MacPorts, or on your own.

    One funny story during the early testing, I had to fix the jq detection in my first version because it totally missed reading the unix path correctly and yeah, thanks to a friend on the MacAdmins Slack who had an issue when trying it out. I was able to go back and fix that. Friends help friends make better code.

    I’m not done making apps, and I’ll keep tweaking these three I have so far and making new ones from the scripts I have. The goal is to manage data, get a better understanding of the data in the archives and let the clients and owners of the data to know what they have.

    Stay tuned.

  • Use the SOFA feed to check if SimpleMDM devices needs updates

    I wrote a “simple” bash script to check SimpleMDM device list by API and check if any devices need updates and/or are compatible with the latest macOS. Of course, it will output some CSVs for fun and profit. Send to clients, managers, security professionals and be well.

    Note: It was a quick hack and for reasons I made 3 output CSVs for testing various presentations of the data that combines the full SimpleMDM device list and matches the macOS with available updates and max supported versions. There may be errors or omissions. Please test. Use and modify. I know I will. This is a test. Just a test.

    The script is in my GitHub repo

    Fetching SimpleMDM device list...
    Downloading SOFA feed...
    ✅ Exported:
      → Full device CSV: /Users/Shared/simplemdm_devices_full_2025-07-30.csv
      → Outdated devices CSV: /Users/Shared/simplemdm_devices_needing_update_2025-07-30.csv
      → Supported macOS per model: /Users/Shared/simplemdm_supported_macos_models_2025-07-30.csv
    ✅ Export complete.
    

    References:

    SOFA MacAdmins Getting Started

    https://sofa.macadmins.io/getting-started.html

    https://github.com/macadmins/sofa/tree/main/tool-scripts

    SimpleMDM API docs

    https://api.simplemdm.com/v1#retrieve-one-dep-device

    squirke1977 / simpleMDM_API

    https://github.com/squirke1977/simpleMDM_API/blob/master/device_details.py

  • Dynamic Groups – SimpleMDM tricks and tips part2

    When we last left our hero the big news was the discovery custom attributes and running scripts to test for certain conditions in SimpleMDM, like “is the firewall on” to post in the main dashboard was all the excitement, this year we present “dynamic groups” which in combination with custom attributes or by itself ups the game to the next level. Keep up!

    What if we wanted to know what is the current version of XProtect across the Mac fleet? and what if this wasn’t collected by default by MDM tool, in my case, SimpleMDM. Well, I can write a script to collect this info, for my purposes I’ve chosen to use silnite from Howard Oakley of eclectic light co fame and write the version number to a custom attribute. The next step is use SimpleMDM’s new dynamic groups (in preview, at the time of this blog post), and then I can watch the result filter in with a special group watching for “is matching this version” or the opposite “is not this version”. Just depends on what you want to act on or how you want to see the information. The new dynamic groups is the exciting part. I’m sooo excited.

    The custom attribute

    Screenshot

    Setting up a custom attribute of “XProtectV: and a default value of “Version Unknown” should be done before the script runs. If I get the default result then the script didn’t run or some other reason.

    The code

    #!/bin/bash
    LOG_DIR="/Users/Shared"
    DATE=$(date +"%Y-%m-%d_%H-%M-%S")
    LOG_FILE="$LOG_DIR/silcheck-log-$DATE.txt"
    /usr/local/bin/silnite aj > "/Users/Shared/silnite-xprotectv-$DATE.json"
    XPROTECTV=$(/usr/bin/plutil -extract XProtectV raw "/Users/Shared/silnite-xprotectv-$DATE.json")
    echo "$XPROTECTV" | tee -a "$LOG_FILE"
    

    The simple script writes a log into /Users/Shared just because I want to and uses the silnite binary to write out the XProtect info and plutil to extract the info from the json Note: you could also use jq in latest macOS 15 but this way is more compatible across macOS versions for now. The XProtect version is saved as an attribute which SimpleMDM picks up and reports back to base.

    The dynamic group

    Screenshot

    The filter headings are a little cut off in the screenshot but it basically says choose from all devices, refer to the custom attribute I set of XprotectV and makes sure the value equals the latest (at blog post writing) 5297 and further filter results for devices last seen in the last day. If I had switched it to the not equal to version 5297 I would see all the devices not up to date. And it’s easy to change on the fly. Easier than refreshing the main device dashboard page to see these results as I was trying to do previously and that method made it hard to further filter.

    The exciting part

    Yes the best part is to set up a job in SimpleMDM that runs the scripts on the devices to refresh the value of XProtect (I have it set to recurring as well) and then watch the results roll into a dynamic group which has its members populate as the scripts runs and results report back. Easey peasy.

    Screenshot

    Addendum:

    Adding an example screenshot to show how you can change the filter from matches an exact value of XProtect, in this example, to “not equal to” to see all the devices that haven’t upgraded yet. It’s as easy as changing the filter and clicking on “staging filter changes” button. Et voilà !

    Updated: May 16, 2025 – 19h00 local time

  • Backup Fast, Restore Quicker

    Backing up is nice, restoring is better. Slow backups, mean slow restores. Make good decisions, and backup only the files you want to keep to the fastest storage you have.

    When working with a fast fibre channel or Thunderbolt SAN your first choice for fastest backup destinations is a Thunderbolt RAID. I recommend to have this onsite with an off site LTO and/or cloud disaster recovery setup (a replicated SAN or shared storage system is nice to have too).

    A built-in option to copy Xsan files is cvcp (cv stands for centravision).

    cvcp -vxy /Volumes/TSAN/folder /Volumes/GammaRAID/backups

    cvcp is fast. Really fast. And cli commands are scriptable. A very smart person (Jasper Siegers) wrote a script called cvcpSync which combined the power of rsync and cvcp. It was awesome. But there are limits to the best of scripts. For my clients I use Archiware P5 with large SAN and other shared storage to simplify the number of things which need to be monitored. One dashboard to monitor tape or cloud backups, tape archives and sync to nearline RAIDs or NAS.

    With a recently Thunderbolt SAN deployment with Accusys T-Share I set up the Accusys Gamma Carry as a backup destination. I set up Archiware P5 to do the backup. It was fast. How fast? Over 1Gb/s. Fast backups are also fast restores. With the Gamma Carry I can run a backup then carry it off site. It’s an option as part of a complete backup strategy.

    Archiware P5 backup 1.6TB in 53 minutes

    (Luckily I have almost 2 TB of video from my Cycliq bike cameras to test backups. Sadly, after my last bike vs car incident I felt obliged to buy bike cameras for my safety. I edit small fun rides when I can. Sometimes traffic near-accidents too. Please be kind, don’t kill cyclists.)

    Archiware P5 backup of a Thunderbolt SAN to a Thunderbolt Gamma Carry RAID

    Note: In my tests I tested backup to a nearline RAID. I also like to use tape drives. LTO tape is another recommended option for backups or archives. Cloud or other offsite replication is also recommended if possible but is the slowest of all the options. Good to have slow and fast options, offsite and on premise, though any practical solution should be affordable and useful to help decision makers take the steps to preserve data and ultimately their own business.

    LTO vs Cloud backup comparison: For LTO backups to one LTO7 drive I normally see 1TB in under 2 hours versus some recent cloud backups I did using rclone which took 9 hours for 1TB. Remember: restore times will equal your backup times. Want to restore 100TB? Got a spare 900 hours? 38 days for cloud restore vs 8 days with one LTO7 drive (much faster if you have more than one drive). Even faster if you restore from a Thunderbolt RAID. Only 2.5 days. Think about it.

    Testing equipment:

    Hardware: T-Share SAN, Gamma Carry Thunderbolt RAID

    Software: Archiware P5

  • Do you want to build a Thunder SAN?

    Thunderbolt Xsan in a box. I’ve written about the Accusys T-share in 2020 (and in 2015 when I first found this cool tech). What’s different now? New year, new macOS. And a new challenge: can we build Xsan only using Terminal? No apps. It’s the journey that counts, right? One nerd’s journey to make an Xsan with macOS 11 Big Sur cli. Destination adventure with family fun, next stop a blinking cursor on a command line prompt.

    make Xsan

    make —Xsan —-bigger

    reboot

    Sudo make me an Xsan sandwich. I wish it were that easy! Stick around for the two or three commands you do need.

    Xsan goes Terminal

    Important commands for using Xsan have always been cvadmin and cvlabel (cv is short for centravision the original creators) but more recently xsanctl and slapconfig are important for creating the SAN and the OD (Open Directory) environment. Read the man pages, search the web, read some help documents. This blog is for entertainment and occasional learnings.

    Xsan Commands: where are they?

      /System/Library/Filesystems/acfs.fs/Contents/bin
    	cvlabel			sncfgremove
    cvaffinity cvmkdir sncfgtemplate
    cvcp
    cvmkfile sncfgtransform
    cvdb
    cvmkfs sncfgvalidate
    cvdbset cvupdatefs sndiskmove
    cverror cvversions snfsdefrag
    cvfsck fsm snlatency
    cvfsck_compat fsmpm snlicense
    cvfsdb has_snfs_label snprodalert35chk
    cvfsid mount_acfs snquota
    cvgather sncfgconvert wingather
    cvgather_fsm sncfgedit xsanctl
    cvgather_multipath sncfginstall xsand
    cvgather_sum sncfgquery xsandaily

    Lots of interesting cv (CentraVision) and sn (StorNext) commands in macOS (this list is from 10.15 Catalina). Besides binaries, what else is there? Examples. A ton of example files:

    /System/Library/Filesystems/acfs.fs/Contents/examples/

    cvlabels.example fsnameservers.example rasexec.example
    cvpaths.example fsports.example rvio.example
    fsmlist.example nss_cctl.example snfs_metadata_network_filter.json.example

    Just the facts. The Xsan basics

    If you don’t have a fibre channel switch and fibre channel hardware RAIDs do not worry. You can build a useful Thunderbolt based Xsan with a little bit of effort. Just a little bit of peril It’s not too perilous, don’t worry.

    Apple includes Xsan for free in macOS. Xsan is Apple’s fork Quantum’s StorNext SAN software. Want large fast storage made for Final Cut Pro editors, just add Xsan. Download Server.app from the Mac App Store and make your Xsan. Easy peasey. Right?

    Why? Why are we doing this? Nothing beats fibre channel or Thunderbolt SAN speed for editing. Network attached storage (NAS) at 1GbE is barely usable. NAS at 10GbE is much better but still has road blocks for editors. Fibre channel or Thunderbolt with a big enough raid behind your SAN then life is great. Xsan can be shared by a small or media sized team of editors, producers and assistants.

    Oh, ok. There is one problem. Apple did a major upgrade of Xsan (now version 7!) in macOS 11 Big Sur but apparently they took out the Xsan config in Server.app. (Note: This is what I was told early on and what seemed to be confirmed by Apple’s recent Xsan cli guide. It turns out that Xsan’s disappearance in Server.app to not be totally correct). Xsan is there in Server.app if you upgrade to macOS Big Sur but when you install Server on a clean macOS there is no Xsan visible in the app. Hmm. What do we do? Apple published a very nice handy guide about how to build Xsan in Terminal. So let’s get started. This is fun.

    Accusys T-Share is a Thunderbolt SAN. Connect Macs with Thunderbolt cable.

    What do we need? 1) Hardware raid. Ok check I have an Accusys T-Share. It’s a raid with Thunderbolt switch built in. 2) Mac. Ok I have a Mac Mini. 3) A network. Some cables, a switch and a DNS server. Ok I have a new raspberry Pi. That’s perfect.

    Raspberry Pi 400 (the amazing linux computer shaped like a keyboard).

    Step 1. Hardware raid. With the Accusys T-Share I just have to plug in some clients with a Thunderbolt 3 cable. Let’s fill the RAID with drives. I picked two different sizes. One group of larger disks for a data LUN (main production storage) and two smaller disks for a raid mirror to be used as metadata storage.

    Step 2. A Mac running macOS Big Sur 11.5.2. Download the Accusys Mac installer on your Intel Mac (M1 is not supported with the T-Share yet as of this blog post).

    Step 3. The network. Ok. This is the fun part. Let’s set up a DNS server. Ok, how do we do that? Remember that raspberry Pi you bought yourself for Christmas but never opened because you have been so busy and well you know life. Ok just me? Well, that one. Let’s use a raspberry Pi. A small inexpensive Linux computer. Install dns masq. It’s perfect for this.

    The raid. Not only a great movie it’s the central part of this production media network for creatives. Once the drives are in the raid we have to make raid sets which become LUNs for Xsan. RAID5/6 for the data LUN and RAID1 (mirror) for the metadata LUN.

    Read the label. Using Xsan cvlabel

    Normally after we create RAID sets in the hardware raid utility we would open up Server.app and label the LUNs for Xsan use. But since we are now hardcore SAN architects we can use Terminal and the cvlabel the command to do this the hard way. Well, it’s not that hard but it can be intimidating the first few times. It’s much easier to label new LUNs than stare at a broken production SAN that has lost its labels. StorNext fun times. More about in another blog post.

    Whether using Server.app in the good old days or cvlabel to label your LUNs now you should all be familiar with the command to list available LUNs. For larger SANs that won’t mount the first thing I’d check is see if the LUNs are all there. You don’t want a SAN to mount if it’s missing an important piece of itself.

    cvlabel -l

    This command lists available LUNs. It’s handy to know. Do this before trouble arises and you will be a cool dude when trouble happens. It does that occasionally. Prepare for the worst, hope for the best, IT motto.

    To create labels for newly created RAID arrays use cvlabel to output a text file of the unlabelled LUNs, make some minor changes then label those LUNs. Create the template files first:

    cvlabel -c

    Edit the file. I like nano. Maybe you like vim. Or BBEdit. Or text edit. Change the name of LUNs from CVFS_unknown to whatever you like. I like to name LUNs based on the hardware they originate from so that I can find them, remove them, fix them or whatever I need to do for troubleshooting. Trust me. It’s a good idea.

    cvlabel ~/Desktop/cvlabel
    *WARNING* This program will over-write volume labels on the devices specified in the file "/Users/xavier/Desktop/cvlabel". After execution, the devices will only be usable by the Xsan. You will have to re-partition the devices to use them on a different file system.
    Do you want to proceed? (Y / N) ->
    Requesting disk rescan .

    Congratulations this is the hardest part. You’ve labeled the RAID arrays as usable LUNs for Xsan. Ok, just kidding that’s not the hardest part. Have you ever heard of Open Directory? Do you fear LDAP and DNS? Well, maybe you should. It’s always DNS. Just saying.

    DNS (domain name system) is just a fancy word for a list of IP addresses and host names. Using the raspberry Pi with dns masq installed we can populate the list of hosts for the Xsan and then we are golden. Hopefully if we did it right. Turns out we can make mistakes here too. Don’t use “.local” domain names. I did. It was late. I blame being tired. Changing them to “.lan” worked better.

    Next up we finally create an Xsan in terminal. Or do we? let’s check the hostname first. It’s always DNS.

    scutil —get HostName 

    CrazyMac.local

    scutil --set HostName XsanMac.lan.

    And now we make very big Xsan using the Xsan guide example

    xsanctl createSan 'VIDEOSAN' --account localadmin --pass 72DERjx1 --user localadmin --cert-auth-name videocert --cert-admin-email administrator@example.com

    It was at this point that it started falling apart. It was late. I had messed up my DNS with “.local” and the Xsan wouldn’t go past this basic OD setup. I did what I always do and reach out to my Xsan colleagues and I got some curious feedback. “What do you mean Xsan isn’t in macOS Big Sur Server.app?” Hmm. I don’t see it on a fresh install. On an upgrade from 10.15 Catalina I do. So, uh, Where is it? And then it was revealed. In the View menu. Advanced. Ugh. It’s right there. Almost staring right at me. When I opened the app it said it couldn’t create an Xsan with my “.local”. That was helpful. Fixed that and Xsan with my pre-labeled LUNs was super quick to set up.

    Xsan configuration in Server.app. “Ignore ownership” is the best thing ever for creatives. Trust me,

    I’ll have to play with the cli set up again soon. Because there were some strange formatting it recommended to me when I tried some variations of the xsanctl createSan. I’ll dig into another day when I have more sleep. Ha ha.

    There’s a lot of useful commands in macOS Big Sur Xsan which was upgraded to v7. You can check which version of Xsan you have in macOS with the cvversions command.

    In Catalina (macOS 10.15.7)

    File System Server:
    Server Revision 5.3.1 Build 589[63493] Branch Head BuildId D
    Built for Darwin 19.0 x86_64
    Created on Tue Jun 22 21:08:03 PDT 2021
    Built in /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/XsanFS/XsanFS-630.120.1/buildinfo

    In Big Sur (macOS 11.5.2)

    File System Server:
    Server Revision 7.0.1 Build 589[96634] Branch Head BuildId D
    Built for Darwin 20.0 x86_64
    Created on Wed Jun 23 00:32:35 PDT 2021
    Built in /System/Volumes/Data/SWE/macOS/BuildRoots/d7e177bcf5/Library/Caches/com.apple.xbs/Sources/XsanFS/XsanFS-678.120.3/buildinfo

    There’s a lot of cool new binaries in Xsan v7. We will dig into those next post. For now enjoy this and go forth make some Xsan volumes with Thunderbolt or fibre channel storage. It’s fun.

  • Hello Big Sur! See ya later Monterey

    I am so happy to install macOS Big Sur 11.5.1, now that it is a ready for production. Have fun with macOS Monterey those of you on the bleeding edge. For media professionals using Xsan in production storage environments August is a great month to update to the soon to be yesterday’s bad boy Mr. Big Sur.

    Server.app v.5.10 in macOS Catalina 10.15.7

    Upgrading to a new major version of macOS can be fraught with peril for a fleet of mac devices but it is potentially fatal for a production SAN environment. That is why we wait. We want a nice stable storage system for our Final Cut Pro editors and other media creatives so it is safe to be one version behind. Less drama that way. We prefer our dramas to be on AppleTV+

    Watch TV Upgrade Xsan

    It is not boring to watch AppleTV+ while upgrading Xsan

    The Xsan upgrade to Big Sur was pretty much not exciting except for one funny roadblock that I had set up myself last as a kind of booby trap for “future me”. More about that later. First the boring stuff. The last few weeks have been very busy updating and re-writing documentation in Pages.app and running multiple redundant full and incremental LTO backups with Archiware P5, syncing to nearline archives, and archiving finalized projects to the LTO shelf in paradise (sounds more exciting when you put it that way don’t you think?). Updating and re-writing documentation can sound like a waste of time but “future you” will appreciate what “past you” was doing today. And today I had fun updating Xsan to macOS Big Sur. Now I must write down all my thoughts before I each too much vegan vanilla ice cream and slip into a food coma.

    “Planning for disasters, while hoping for none” is the IT mantra. We planned hard and we were ready to restore Xsan from Time Machine, if we had to. Not a joke. The server is backed up by Time Machine. The data is backed up to LTO, nearline archives racked and stacked in a server room and on redundant thunderbolt RAIDs which are parked on electric trucks ready to blast off at the earliest sign of danger. Well, everything except for the last part. Would be nice. And cloud backups for those clients that want them. Plan for the worst, pay for what you can to keep your business operational and lessen the impact of mechanical failures, human oopsies, or ransomware. Sysadmins are indistinguishable from malware sometimes, but we mean well. More seriously, humans makes mistakes and break things (that, me!) but ransomware is real and my elaborate backup and archive planning has saved a few customers this year.

    Ok, now for the fun part. How to upgrade an Xsan to macOS Big Sur (11.5.1). Maybe go read last year’s blog post on upgrading the Xsan to macOS Catalina 10.15.6 which was detailed and thorough. Or read Apple’s new Xsan Management Guide. It’s got all the fundamentals explained.

    Xsan volumes are typically made of up fibre channel RAID arrays. Nice icon!

    Preparation is key. Be prepared. Get ready. Psych yourself up. I used Greg Neagle’s installinstallmacos.py to download macOS Big Sur as a disk image and had that and the App Store’s Server.app downloaded beforehand and not be dependent on internet access (production SANs are not always internet accessible). It is both true and not true that you can setup Xsan in Big Sur with the Server.app. It is true you need the Server.app for an upgrade from macOS Catalina 10.15.7 but if you’re starting from scratch in macOS 11 you will be building your Xsan in Terminal. Have fun! (We will cover this in a future post).

    Download macOS Big Sur and the Server.app. Keep old copies zipped up. Cvlabel is nice too

    Server.app manages only three (3) services for an Xsan upgrade: Profile Manager, Open Directory and Xsan. In macOS Big Sur new setups of Server.app Xsan is gone. Why they haven’t taken out Profile Manager and not kept Xsan instead made me scratch my head. No one in their right mind is using Profile Manager to install or manage profiles, they’re using commercial MDM vendors. But Xsan in macOS Big Sur (11) is not only production ready storage SAN awesome it has been upgraded to be compatible with Quantum’s Stornext 7 (previously it was only v.5)

    Profile Manager does not belong here. Long Live Xsan!!

    Installing macOS 11 Big Sur and upgrading Xsan to v7 is compatible (in my testing) with macOS 10.14 Mojave, 10.15 Catalina and of course macOS 11 Big Sur. If you don’t believe me check out this not updated in forever Apple’s compatibility chart.

    Ok, by this time you get the idea I’m an expert, right? I’m ready to upgrade. But I run into my first real road block. And I have only myself to blame. I can’t launch the macOS Big Sur install app. It is blocked. “Contact your administrator”?! I am the sysadmin. Oh, ok. That’s me. What have I done now? I installed Hannes Juutilainen’s Big Sur Blocker last year, that’s what.

    Of course I installed that. With Munki. On all my Mac clients that were upgraded to macOS Catalina. (And of course my Xsan controller has Munki!). But no worries, let me read up on my last year’s blog post about it to figure out how I installed it, there must be a launch daemon or something.

    this is not how I expected it to go

    Hmm, no didn’t mention there. And where is that pesky launch daemon that I can unload and get to this Big Sur install. Oh? It’s a launch agent. Unloaded. Hmm, still no. Ok, delete the app from /usr/local/bin, hmm, nope. ok kill the app process. Ok, now we can install macOS Big Sur. Sorry for the delay. I had told Munki to uninstall the bigsurblocker app and it did for every other Mac, I swear, really. It did.

    Please proceed with the macOS Big Sur install

    So ready for macOS Big Sur. Oh wait, we noticed that you’re running Server.app and well, we don’t do a lot of the same things anymore in the new Server.app so maybe this is a warning.

    Warning. We noticed that you’re running Server.app and we don’t do those fun things anymore.

    So a lot of progress bars and stuff. See my last upgrade blog post and it’s the same as installing macOS Big Sur on any Mac, except this Mac Mini is running an Xsan production SAN environment with a lot of RAID arrays in a server rack or two. Ok, yeah, just run the installer.

    We noticed that Server app is no longer server app.

    After macOS Big Sur is installed zip up your older server.app and drag in your new one (or use that fancy App Store app to do it for you if you’re lazy). Click a bunch of buttons (see all my old blog posts) and launch the new Server.app.

    Profile Manager is updating. No one cares.

    So we have to wait while the bag of scripts that is Profile Manager gets updated but no one uses it but it’s the most important app in Server.app now, no I am not bitter why do you ask. Xsan is awesome.

    Xsan is off. Don’t panic.

    Xsan is off. Don’t panic. Where’s my towel? Panic now!

    Time to restore from your old Xsan configuration. Wheee…..

    Xsan restore configuration.

    Activate your Xsan and carry on upgrading all your Mac clients. Note: I did test macOS Mojave 10.14, macOS 10.15 Catalina and of course macOS 11.5.1 Big Sur Xsan clients. All worked.

    Xsan on. Power up.

    Upgrading Xsan with macOS Big Sur is easy if you’re going from macOS Catalina. Starting from scratch is another story to be covered in another blog post. Also not covered is certificate issues from self-signed certs breaking when I upgraded my Munki and MunkiReport server. That’s definitely another blog post. It’s just a webserver. Just. A. Web. Server. What is so hard? haha

    Technical Errata:

    With more than one Xsan controller it used to be recommended to upgrade the secondary before the primary but it is now best practise to upgrade the primary first to maintain the sanity of the OD data.

    Xsan Upgrade Step by Step:

    Clone the controllers. (+ Time Machine backups)
    Turn off the clients.
    Stop the Xsan Volume.
    Run cvfsck on the volume.
    **Upgrade the primary.
    Confirm the secondary can see the primary.
    *Upgrade the secondary.
    Confirm the secondary can see the primary.
    Check SAN access on both controllers.

    Upgrade the clients as desired.

  • A Mesh VPN For Everyone

    How to use Tailscale (wireguard based) mesh VPN to connect everything

    What is Tailscale? It’s a mesh VPN based on the wireguard open source project. It’s a secure network to connect your own devices no matter where they are.

    Tailscale is free to use with one account and up to 100 devices, which is enough to see how well this can work to connect up servers, storage and desktops. They have paid plans for teams and enterprise.

    Tailscale macOS app icon

    macOS and iOS

    To start, download Tailscale on your Mac or iPhone then find your IP address. Once you are signed in and have your IP address you can connect easily between devices. For example, on your iPhone open the Tailscale app and see your installed devices. Click on your Mac and the IP will be copied into the clip board. Use this to connect with app such as Secure shellfish for SSH or VNC viewer for remote login. When you’re in the same network it’s impressive, but when you’re on a different network, separated far away, It’s magic.

    The real test for me was to install Tailscale on some backup servers I manage to make it more secure and more convenient to access them. I had used a variety of remote control for business services and well, Tailscale is easier, quicker and much more awesome. All the other software I tried was much less awesome.

    After using Tailscale mainly for remote control, I tested Tailscale to securely connect my remote Macs to my own MunkiReport server. I use Munki and MunkiReport to manage Macs and having Tailscale allows me to securely connect endpoints to the server without opening up ports on my router. MunkiReport allows me to detect malware (with DetectX plugin) or check on backup jobs with Archiware P5 backup software (using a module I wrote) or a multitude of other diagnostics such as disk space free, apps installed, and all kinds of great hardware and software metrics. So much reporting. And MunkiReport doesn’t need Munki specifically, so if Tailscale is installed for remote control why not report on everything else.

    DSM Package Center: Tailscale (and Archiware P5) app on Synology NAS

    Synology DSM

    Having Tailscale installed in all the Synology NAS I manage in various physical locations allows me to securely connect to all of these NAS from anywhere. With remote work using a NAS is a great way to sync data between locations. Synology has a lot of great built-in tools to make this happen and a very robust quick connect feature combined with ddns, and let’s encrypt certificates to support it. After setting up a few to sync one location to another I was constantly getting notifications of IPs being blocked on my firewall. I had to open a port on my firewall to let in the ssh / rsync traffic through and despite a strong set of firewall rules with a geo block there were still connection failures and password attempts. Using Tailscale I can now have a P5 server set up on one Synology NAS connecting to the Tailscale IP of other remote units and it can easily backup, sync or archive the data from the various locations.

    To install the Synology Tailscale package check out this GitHub page. Download the app then side load it (manual install in package center). To enable it you will have to have ssh on, a user with permission to use it, and one command to type.

    sudo tailscale up

    In one case I didn’t have SSH enabled on the remote unit so I remoted into a Mac on the same network, enabled an admin user, turned on ssh with a time limit on the account, and then logged in. Once the above command is run you will get a link to a website to authenticate the device with your account.

    Linux (CentOS)

    I have also installed and tested Tailscale on a Linux (CentOS) storage server. In my case a Jellyfish which has a ZFS volume shared over direct 10GbE for Final Cut Pro video editors using nfs or smb. the Jellyfish works well on premise, but wouldn’t it be nice to capture camera cards to the remote storage server via Tailscale? Oh yes it would. And what about playing back some of the video files via VLC on your iPhone! Or Files.app! Yes, to all the above. All made possible with Tailscale. And a huge shout out for their great documentation. Installing Tailscale on CentOS was super simple. Add a yum repo, install, tailscale, and then bring the service up. Couldn’t be easier.

    sudo yum-config-manager --add-repo https://pkgs.tailscale.com/stable/centos/7/tailscale.repo
    
    sudo yum install tailscale

    Shared Devices

    A small, but very exciting, feature was added part way through my testing of Tailscale which made it infinitely more awesome, shared devices. The concept is you are authenticated to your devices and can see in the Tailscale app all the IPs to connect to, but what if you could share one device (computer, server, NAS) with another person? Well, now you can. In the Tailscale admin console choose a device and send a share link to someone, they then will see this devices in their device list as shared. Home users can set up Tailscale to access all their own devices, but now can also choose to share access with a device in particular. For example, if you create an account, open a service (file sharing) and send a share link then the other person will login with the account you create and access the one thing you want them to. Maybe it’s a smb share to drop files. Works great for video collaboration and other kinds of teams.

    There’s a whole lot more you can do with Tailscale (and wireguard) mesh VPN but I hope this gives you all some ideas to start with.

  • Xsan Upgrade and Big Sur Prep. Hello Catalina!

    Big Sur summer testing time!

    Summer time is beta testing time. A new macOS beta cycle with Big Sur is upon us. Test early, and test often. With all the excitement of Big Sur in the air, it’s time to look at Catalina.

    Our day to day production Xsan systems do not run beta software, not even the latest version of macOS, they only run tested and safe versions of macOS. I always recommend being a revision behind the latest. Until now that meant macOS 10.14 (Mojave). With the imminent release of macOS Big Sur (is it 10.16 or macOS 11?) then it’s time to move from 10.14.6 Mojave to 10.15.6 Catalina. It must be safe now, right? 

    Background

    Xsan is Apple’s based Storage Area Network (SAN) software licensed from Quantum (see StorNext), and since macOS 10.7 aka Lion it has been included with macOS for free (it was $1,000 per client previously!).

    Ethernet vs Fibre Channel vs Thunderbolt

    A SAN is not the same as a NAS (Network attached storage) or DAS (direct attached storage). A NAS or other network based storage is often 10GbE and can be quite fast and capable. I will often use Synology NAS with 10GbE for a nearline archive (a second copy of tape archive) but can also use it as a primary storage with enough cache. Lumaforge’s Jellyfish is another example of network based storage.

    Xsan storage is usually fibre channel based and even old 4GB storage is fast because … fibre channel protocol (FCP) is fast and the data frames are sent in order unlike TCP. It is more common to see 8GB or 16Gb fibre channel storage these days (though 32GB is starting to appear). And while fibre channel is typically what you use for Xsan you can also use shared Thunderbolt based storage like the Accusys A16T3-Share. I have tested a Thunderbolt 2 version of this hardware with Xsan and it works very well. I’m hoping to test a newer Thunderbolt 3 version soon. Stay tuned.

    Xsan vs macOS Versions

    We’ve discussed all the things that the Xsan is not and now what is it? Xsan is often created from multiple fibre channel RAID storage units but the data is entirely dependent on the Xsan controller that creates the volume. The Xsan controller is typically a Mac Mini but can be any Mac with Server.app (from Apple’s App Store). The existence of any defined Xsan volumes depends on the sanity of its SAN metadata controllers. If the SAN controllers die and the configuration files go with it then your data is gone.  POOF! I’ve always said that Xsan is a shared hallucination, and all the dreamers should dream the same dream. To make sure of this we always recommend running the same version of macOS on the Mac clients as well as the servers (the Xsan controllers). And while the Xsan controllers should be the same or at a higher macOS version level it can sometimes be the opposite in practise. To be sure what versions of macOS are interoperable we can check with Apple’s Xsan controllers and clients compatibility chart and Xsan versions included in macOS for the rules and exceptions. Check the included version of Xsan on your Mac with the cvversions command

    File System Server:
      Server  Revision 5.3.1 Build 589[63493] Branch Head BuildId D
       Built for Darwin 17.0 x86_64
       Created on Sun Dec  1 19:58:57 PST 2019
       Built in /BuildRoot/Library/Caches/com.apple.xbs/Sources/XsanFS/XsanFS-613.50.3/buildinfo

    This is from a Mac running macOS 10.13

    Host OS Version:
     Darwin 17.7.0 Darwin Kernel Version 17.7.0: Sun Dec  1 19:19:56 PST 2019; root:xnu-4570.71.63~1/RELEASE_X86_64 x86_64

    We see similar results from a newer build below:

    File System Server:
      Server  Revision 5.3.1 Build 589[63493] Branch Head BuildId D
       Built for Darwin 19.0 x86_64
       Created on Sun Jul  5 02:42:52 PDT 2020
       Built in /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/XsanFS/XsanFS-630.120.1/buildinfo

    This is from a Mac running macOS 10.15.

    Host OS Version:
     Darwin 19.6.0 Darwin Kernel Version 19.6.0: Sun Jul  5 00:43:10 PDT 2020; root:xnu-6153.141.1~9/RELEASE_X86_64 x86_64

    Which tells us that the same version of Xsan are included with macOS 10.13 and 10.15 (and indeed is the same from 10.12 to 10.15). So we have situations with Xsan controllers running 10.13 and clients running 10.14 are possible even though macOS versions are a mismatch, the Xsan versions are the same. There are other reasons for keeping things the macOS versions the same: troubleshooting, security, management tools, etc  To be safe check with Apple and other members of the Xsan community (on MacAdmins Slack).

    Backups are important

    Do not run Xsan or any kind of storage in production without backups. Do not do it. If your Xsan controllers die then your storage is gone. Early versions of Xsan (v1 especially) were unstable and the backups lesson can be a hard one to learn. All later versions of Xsan are much better but we still recommend backups if you like your data. Or your clients. (Clients are the people that make that data and pay your bills). I use Archiware P5 to make tape backups, tape archives, nearline copies as well as workstation backups. Archiware is a great company and P5 is a great product. It has saved my life (backups are boring, restores are awesome!).

    P5-Restore-FCPX.png

    Xsan Upgrade Preparation

    When you upgrade macOS it will warn you that you have Server.app installed and you might have problems. After the macOS upgrade you’ll need to download and install a new version of Server.app. In my recent upgrades from macOS 10.13 to macOS 10.15 via 10.14 detour I started with Server.app 5.6, then install 5.8 and finally version 5.10.

    After the macOS upgrade I would zip up the old Server.app application and put in place the new version which I had already downloaded elsewhere. Of course you get a warning about removing the Server app

     

    Xsan-ServerApp-ZipRemovalDetected.png

    Install the new Server app then really start your Xsan upgrade adventure.

    Serverapp-setup.png

    Restore your previous Xsan setup.

    If everything goes well then you have Xsan setup and working on macOS 10.15.6 Catalina

    Xsan-Catalina-Upgrade-Success