So apparently I’ve missed the big story of Apple introducing BSD licensed open rsync in macOS 15 Sequoia back in the Fall of 2024. (Thanks to Rich Trouton’s recent blog post).
Open Rysnc in macOS 15.4
Openrsync it’s new. And I didn’t notice it, because I’ve been using a newer Apple silicon compiled rsync (see RSKgroup) because Apple hadn’t updated the included rsync 2.6.9 in decades and it was becoming a problem. It seems the big break of only shipping the newer openrsync happened in macOs 15.4 because until 15.4 Apple gave the option to use either the SAMBA or the new BSD licensed version with an environment variable.
I use rsync a lot, for all my hand rolled artisanal backup scripts and the occasional push and sync between storage. Of course, for serious backups and archives I use Archiware P5 not the least because I need one dashboard to track backups, syncs, and archives, and more importantly I need to use LTO (tape backups) and Archiware is the best. As for useful terminal commands, rsync is often my go to for a lot of edge cases and quick storage comparisons.
macOS 15 Sequoia. Check your firewall checking scripts please
If anyone is following along with my attempt to re-create MunkiReport in SimpleMDM then you’ll be happy to know the space madness is still strong and macOS 15 has made one tiny thing break, my firewall checking script.
My firewall checking script began life as a simple check of the status in the alf pref file but that file no longer exists in macOS 15.
See this Knowledge base article which lists in bug fixes that the file no longer exists and that the socketfilterfw binary be used instead, except that doesn’t work when Macs are managed.
Application Firewall settings are no longer contained in a property list. If your app or workflow relies on changing Application Firewall settings by modifying /Library/Preferences/com.apple.alf.plist, then you need to make changes to use the socketfilterfw command line tool instead.
Yes, my Macs are managed with MDM and yes I have a profile to enable the firewall but no I don’t trust it so can I check please with another method. Trust but verify.
So thanks to some friends in the MacAdmins Slack I stole the idea from tuxudo to check firewall in macOS 15 using system profiler, because he had re-written the MunkiReport module already and so there I go again, stealing from MunkiReport and all the hard work they do.
After playing with the output of system_profiler a bit I looked at the “Mode”
/usr/sbin/system_profiler SPFirewallDataType -detailLevel basic |grep Mode
Mode: Allow all incoming connections
Stealth Mode: No
Of course I could write some nice code to clean this up or instead I switched to searching for “Limit” and if there’s no hit on that there’s no limit (translated: firewall is not enabled“)
And if there is a limit then the firewall is enabled.
Mode: Limit incoming connections to specific services and applications
Simple. Good enough to add to my SimpleMDM script to run and populate the value to the custom attribute and update my dashboard. And my crazy mission to build everything into SimpleMDM dashboard is still… madness …. but also quite fun.
Custom Attributes in SimpleMDM are a way to assign values in a few different cases. I will show one use case, scripting, and one example: checking the firewall.
I wanted to see the status of the macOS firewall in the device view dashboard. That’s so simple, right? Well, I wanted to see it at a glance for every device, and not have to go into each device entry to see if the firewall was enabled.
Write a script:
#!/bin/bash
# Check firewall status
firewall=$(defaults read /Library/Preferences/com.apple.alf globalstate)
if [ "$firewall" = "1" ]; then
echo "Firewall is enabled"
elif [ "$firewall" = "0" ]; then
# Set firewall status
#defaults write /Library/Preferences/com.apple.alf globalstate -int 0
echo "Firewall is NOT enabled"
else
echo "Unable to determine firewall status"
fi
Note: This is my script. This seems to work. If you have other working examples let me know.
Add it to SimpleMDM scripts
Add your script to the scripts section. Check the “Enable attribute support” check box.
Add a custom attribute
Set up a custom attribute that your script will populate with its variable later. I set up one for the firewall.
Create a job
Your script will need to run (once, or scheduled) to populate the value into the variable and into the custom attribute. Choose what script runs where on what Macs. And choose the custom attribute.
And choose the custom attribute.
Note: The cancel job if not started is helpful if your devices are not responding. And is a premonition to issues you may have with this feature and might give some flashbacks to the ancient way of using scripts in ARD (Apple Remote Desktop) to try to make changes, back in the days before MDM or good configuration management tools ie. munki puppet chef salt etc
Dashboard Devices
Add your custom attribute to the viewable columns in the Devices dashboard and your life will be full of joy. Seeing at a glance your scripts output variable as a custom attribute.
And now I just have to recreate everything in MunkiReport as a custom attribute and then I’ll be good.
Script debugging.
Running scripts is all well and good until your devices don’t check in and don’t run the scripts for whatever reason. Rebooting the Mac helps. Refreshing the inventory in SimpleMDM helps (maybe) and well, you’ll see it’s like the old ARD scripts run ad hoc and you’ll wish for better tools like fully functional DDM (declarative device management) which is like configuration management of the days of old. Incorporate MunkiReport and Fleet’s osquery tools and save me the trouble of doing piecemeal.
Enjoy the script output in the custom attributes for now and send me your awesome ideas for what to script next.
Navigating a database to get what you want, that is the goal. Do you love it? No, but you do need to do it? Yes, to get the data. Remember the goal.
Many popular (and many lesser known) applications use SQL, and SQLite in particular, to store data. That’s fine. That’s great. But unfortunately on occasion you need to go spelunking to find data you want and get it out. This is not a blog post about how much I love SQL (structured query language), because I do not love it. This is also not a blog post about how SQL is awesome, because I can’t say that. But what I hope to share are some tips and tricks for getting in and out with the data you want.
If you’re not a fan of SQL and you’ve got a need for DB data then this app will let you open a database and explore. This is a great app because you can see the tables and what’s in the Db which will no doubt help you late explore in Terminal or in a script. I personally need a visual map sometimes before I jump in. Exploring the Db in this app will also show you the arcane commands necessary to do the same in Terminal. You will be in awe of whomever decided to create this complicated series of commands which makes long insane Unix commands seem logical in comparison.
You can use DB Browser to export a csv (comma separated values) for a spreadsheet or as JSON (JavaScript Object Notation) which all the cool kids like these days. Better start loving this. More on JSON and APIs in a future blog post. Its not XML, but it makes you wish it was.
Use Terminal – Type the commands by yourself
In Terminal we can tell sqlite we want to export a csv file of everything. Add a header and tell it to be in csv mode then SELECT everything.
sqlite> .header on
sqlite> .mode csv
sqlite> .output export.csv
sqlite> .quit
Export just some the data as a CSV
sqlite3 /path/to/the/database.db SQLite version 3.40.1 2022-12-28 14:03:47 Enter “.help” for usage hints. sqlite> .header on sqlite> .mode csv sqlite> .output JustTheSelectFieldsPlease.csv sqlite> SELECT label, timeDated, fancyList, sillyList, boringFiles, indexName FROM tableName; sqlite> .quit
Automate and Make a Script
#!/bin/bash
sqlite3 /path/to/the/database.db <<EOF
.header on
.mode csv
.output JustTheSelectFieldsPlease.csv
SELECT label, timeDated, fancyList, sillyList, boringFiles, indexName FROM tableName;
.quit
EOF
UPDATE: You must check out Datasette!
After posting this I was reminded of an app called Datasette which is truly remarkable and awesome. It’s also a pip install thing but I’ve been using the standalone Mac app which has everything self-contained.
With Datasette it’s easy to load SQLite DBs directly and filtering out the tables I want by easily set conditions, which makes exporting a workable JSON or CSV file quite easy in one step. There’s also a small and lightweight web app called Datasette Lite to make installing and running Datasette extremely simple. Datasette has plugins too. A lot. More to say on those in a later post.
I was chatting with Per Olofsson on a recent episode of the MacDevOps podcast about some recent fixes with relocatable Python he did for MunkiReport version 5.7.0 and I happened to mention how much I love my MR dashboards with custom hot keys. He is a long time user of MunkiReport but hadn’t heard that you could make custom dashboards and I couldn’t remember where I had heard of it or even how I made them. Pretty typical of tech these days I think. You learn something, you make something and then you move to the next thing and forget what you were doing or how you did it. Well, thanks to documentation we can share the knowledge and spread the love.
Custom Dashboards
The MunkiReport wiki actually has a short entry which explains how to make a custom Dashboards. Basically, add some YAML files in the dashboards folders and you’re done. Follow the Read Me file for instructions. Pro Tip: Use the Widget Gallery in MR to find useful pieces to build into your dashboards. Note: I’ve added these custom dashboards to my local folder which is set in my “.env” to be outside of the main munkireport folder so it easier to update across version upgrades.
In the old days we used to forward ports. On your router the traffic for a server or service went to a port (where a number represents a service, some which are defined, but can be arbitrary) and to a destination IP address. Well, wouldn’t you know it, if ssh is port 22 or web traffic is on port 80 then everyone and their port scanner comes knocking. So then your firewall is tested, and then auto-ban and geo-block and emails go out. What if we could avoid that and not open (or forward) any port to make services work across the internet?
Tailscale is a mesh-VPN which uses WireGuard to securely establish a mesh (point to point) VPN of your devices. Suddenly your iPhone can securely send files to your Mac or raspberry Pi across the world. How cool is that? In today’s advanced lesson: you can backup and sync your Synology NAS using Archiware P5.
Step 1: Setting up Tailscale on Synology
It honestly used to be harder than this, these days you can simply add the Tailscale package via the Synology package center app and you’re done. Almost. There’s one more step.
Step 2: Set up Outgoing VPN access via Tailscale which requires editing some files (which necessitate Terminal and remote login access). This only has to be done once but future updates may require fixes. This was tested in DSM 7. Pro tip: only allow remote access to a restricted and time limited account so you don’t leave it on accidentally.
Step 3. Install Archiware P5 on Synology NAS
Using Archiware P5 to Backup and Sync your NAS is a good thing if you’re already using Archiware P5 to backup and sync all the other things, then at least you have only one dashboard to look at. I use P5 with my clients to backup their shared storage to LTO and it makes sense to backup all the things no matter where they are with P5 also. With Synology NAS package center it’s a simple one-click install for P5. Add your P5 clients to your P5 server via Tailscale and you’ve got a secure setup.
This post is just a quick overview of using Tailscale to set up your P5 clients (which is your Synology NAS in this case).
How to deploy applications using munki and simplemdm
You want to deploy apps to Macs but you also want to keep them up to date, fear not, we have a way. If you are using SimpleMDM for Mac management but hate the way MDMs deploy applications then listen up it’s easy(*) to set up Munki and use the power Autopkg to deploy and update all your apps. Note: SimpleMDM also offers a short list of curated apps to deploy without any extra setup but these instructions are for those who want to choose the apps they want to deploy. If that’s you then read on.
Managed Software Centre is the AppStore for all your apps you want your Macs to have
SimpleMDM: The basics
Macs are enrolled into SimpleMDM, then assigned to Groups. Groups have profiles assigned to them to enforce and escrow FileVault or set other policies. Simple enough, right?
Ok, what about apps?
SimpleMDM Category setting for a Munki’s Managed Software Centre
When you have apps in your Catalog you can assign a Munki category to the applications to make it show up in a nice group using Managed Software Centre (the client facing app).
With Apps in your Catalog you can manage them with Assignment Groups which are created as Munki (or not-Munki aka Standard). Next select Managed or Self-Serve, two concepts which make sense to Munki admins. One set of apps is required and will be installed without asking, and the other group is presented to the end user to choose as needed (they’re optional).
API key options. Allow Munki plugin access
API key
How do we get applications we want into SimpleMDM? Two ways exist. Import them manually and deploy via MDM or setup up Autopkg. For this we need the API key. Note: Only the munki plugin permissions are needed. Put the key into the Autopkgr.app SimpleMDM integration or set them as an environment variable and use autopkg in Terminal.
Autopkgr app choose autopkg recipes to use
Select recipes using Autopkgr (Linde Group) from the curated list of recipes created by IT Admins around the world or create your own recipes. What used to be a painstakingly difficult process by hand is now much easier with Recipe Robot by Elliot Jordan to help fish out the AppCast / Sparkle / Download URLs and transform into a nice autopkg recipe to be used by Munki (and ingested into SimpleMDM).
autopkg run -v Postlab.munki.recipe -k MUNKI_REPO_PLUGIN="SimpleMDMRepo" -k MUNKI_REPO="" -k extract_icon=True
MunkiImporter
Using API key provided by environment variable.
MunkiImporter: Using repo lib: MunkiLib
MunkiImporter: plugin: SimpleMDMRepo
Managed Software Centre notification
Managed Software Centre
Once Macs are enrolled and added to a SimpleMDM Group with the Munki assignment then the Managed Software Centre app will allow users to use the Self-Serve portal to install optional apps. Managed apps will install invisibly in the background.
The beauty of this integration is that Munki is awesome and works well. It is battle tested by many companies and organizations around the world. Using autopkg and its recipes to check for updates allows for a seamless automation of new apps into your catalog and then onto your fleet. Updated Macs are happy Macs.
Backing up is nice, restoring is better. Slow backups, mean slow restores. Make good decisions, and backup only the files you want to keep to the fastest storage you have.
When working with a fast fibre channel or Thunderbolt SAN your first choice for fastest backup destinations is a Thunderbolt RAID. I recommend to have this onsite with an off site LTO and/or cloud disaster recovery setup (a replicated SAN or shared storage system is nice to have too).
A built-in option to copy Xsan files is cvcp (cv stands for centravision).
cvcp is fast. Really fast. And cli commands are scriptable. A very smart person (Jasper Siegers) wrote a script called cvcpSync which combined the power of rsync and cvcp. It was awesome. But there are limits to the best of scripts. For my clients I use Archiware P5 with large SAN and other shared storage to simplify the number of things which need to be monitored. One dashboard to monitor tape or cloud backups, tape archives and sync to nearline RAIDs or NAS.
With a recently Thunderbolt SAN deployment with Accusys T-Share I set up the Accusys Gamma Carry as a backup destination. I set up Archiware P5 to do the backup. It was fast. How fast? Over 1Gb/s. Fast backups are also fast restores. With the Gamma Carry I can run a backup then carry it off site. It’s an option as part of a complete backup strategy.
Archiware P5 backup 1.6TB in 53 minutes
(Luckily I have almost 2 TB of video from my Cycliq bike cameras to test backups. Sadly, after my last bike vs car incident I felt obliged to buy bike cameras for my safety. I edit small fun rides when I can. Sometimes traffic near-accidents too. Please be kind, don’t kill cyclists.)
Archiware P5 backup of a Thunderbolt SAN to a Thunderbolt Gamma Carry RAID
Note: In my tests I tested backup to a nearline RAID. I also like to use tape drives. LTO tape is another recommended option for backups or archives. Cloud or other offsite replication is also recommended if possible but is the slowest of all the options. Good to have slow and fast options, offsite and on premise, though any practical solution should be affordable and useful to help decision makers take the steps to preserve data and ultimately their own business.
LTO vs Cloud backup comparison: For LTO backups to one LTO7 drive I normally see 1TB in under 2 hours versus some recent cloud backups I did using rclone which took 9 hours for 1TB. Remember: restore times will equal your backup times. Want to restore 100TB? Got a spare 900 hours? 38 days for cloud restore vs 8 days with one LTO7 drive (much faster if you have more than one drive). Even faster if you restore from a Thunderbolt RAID. Only 2.5 days. Think about it.
Thunderbolt Xsan in a box. I’ve written about the Accusys T-share in 2020 (and in 2015 when I first found this cool tech). What’s different now? New year, new macOS. And a new challenge: can we build Xsan only using Terminal? No apps. It’s the journey that counts, right? One nerd’s journey to make an Xsan with macOS 11 Big Sur cli. Destination adventure with family fun, next stop a blinking cursor on a command line prompt.
make Xsan
make —Xsan —-bigger
reboot
Sudo make me an Xsan sandwich. I wish it were that easy! Stick around for the two or three commands you do need.
Xsan goes Terminal
Important commands for using Xsan have always been cvadmin and cvlabel (cv is short for centravision the original creators) but more recently xsanctl and slapconfig are important for creating the SAN and the OD (Open Directory) environment. Read the man pages, search the web, read some help documents. This blog is for entertainment and occasional learnings.
Lots of interesting cv (CentraVision) and sn (StorNext) commands in macOS (this list is from 10.15 Catalina). Besides binaries, what else is there? Examples. A ton of example files:
If you don’t have a fibre channel switch and fibre channel hardware RAIDs do not worry. You can build a useful Thunderbolt based Xsan with a little bit of effort. Just a little bit of peril It’s not too perilous, don’t worry.
Apple includes Xsan for free in macOS. Xsan is Apple’s fork Quantum’s StorNext SAN software. Want large fast storage made for Final Cut Pro editors, just add Xsan. Download Server.app from the Mac App Store and make your Xsan. Easy peasey. Right?
Why? Why are we doing this? Nothing beats fibre channel or Thunderbolt SAN speed for editing. Network attached storage (NAS) at 1GbE is barely usable. NAS at 10GbE is much better but still has road blocks for editors. Fibre channel or Thunderbolt with a big enough raid behind your SAN then life is great. Xsan can be shared by a small or media sized team of editors, producers and assistants.
Oh, ok. There is one problem. Apple did a major upgrade of Xsan (now version 7!) in macOS 11 Big Sur but apparently they took out the Xsan config in Server.app. (Note: This is what I was told early on and what seemed to be confirmed by Apple’s recent Xsan cli guide. It turns out that Xsan’s disappearance in Server.app to not be totally correct). Xsan is there in Server.app if you upgrade to macOS Big Sur but when you install Server on a clean macOS there is no Xsan visible in the app. Hmm. What do we do? Apple published a very nice handy guide about how to build Xsan in Terminal. So let’s get started. This is fun.
Accusys T-Share is a Thunderbolt SAN. Connect Macs with Thunderbolt cable.
What do we need? 1) Hardware raid. Ok check I have an Accusys T-Share. It’s a raid with Thunderbolt switch built in. 2) Mac. Ok I have a Mac Mini. 3) A network. Some cables, a switch and a DNS server. Ok I have a new raspberry Pi. That’s perfect.
Raspberry Pi 400 (the amazing linux computer shaped like a keyboard).
Step 1. Hardware raid. With the Accusys T-Share I just have to plug in some clients with a Thunderbolt 3 cable. Let’s fill the RAID with drives. I picked two different sizes. One group of larger disks for a data LUN (main production storage) and two smaller disks for a raid mirror to be used as metadata storage.
Step 2. A Mac running macOS Big Sur 11.5.2. Download the Accusys Mac installer on your Intel Mac (M1 is not supported with the T-Share yet as of this blog post).
Step 3. The network. Ok. This is the fun part. Let’s set up a DNS server. Ok, how do we do that? Remember that raspberry Pi you bought yourself for Christmas but never opened because you have been so busy and well you know life. Ok just me? Well, that one. Let’s use a raspberry Pi. A small inexpensive Linux computer. Install dns masq. It’s perfect for this.
The raid. Not only a great movie it’s the central part of this production media network for creatives. Once the drives are in the raid we have to make raid sets which become LUNs for Xsan. RAID5/6 for the data LUN and RAID1 (mirror) for the metadata LUN.
Read the label. Using Xsan cvlabel
Normally after we create RAID sets in the hardware raid utility we would open up Server.app and label the LUNs for Xsan use. But since we are now hardcore SAN architects we can use Terminal and the cvlabel the command to do this the hard way. Well, it’s not that hard but it can be intimidating the first few times. It’s much easier to label new LUNs than stare at a broken production SAN that has lost its labels. StorNext fun times. More about in another blog post.
Whether using Server.app in the good old days or cvlabel to label your LUNs now you should all be familiar with the command to list available LUNs. For larger SANs that won’t mount the first thing I’d check is see if the LUNs are all there. You don’t want a SAN to mount if it’s missing an important piece of itself.
cvlabel -l
This command lists available LUNs. It’s handy to know. Do this before trouble arises and you will be a cool dude when trouble happens. It does that occasionally. Prepare for the worst, hope for the best, IT motto.
To create labels for newly created RAID arrays use cvlabel to output a text file of the unlabelled LUNs, make some minor changes then label those LUNs. Create the template files first:
cvlabel -c
Edit the file. I like nano. Maybe you like vim. Or BBEdit. Or text edit. Change the name of LUNs from CVFS_unknown to whatever you like. I like to name LUNs based on the hardware they originate from so that I can find them, remove them, fix them or whatever I need to do for troubleshooting. Trust me. It’s a good idea.
cvlabel ~/Desktop/cvlabel
*WARNING* This program will over-write volume labels on the devices specified in the file "/Users/xavier/Desktop/cvlabel". After execution, the devices will only be usable by the Xsan. You will have to re-partition the devices to use them on a different file system.
Do you want to proceed? (Y / N) ->
Requesting disk rescan .
Congratulations this is the hardest part. You’ve labeled the RAID arrays as usable LUNs for Xsan. Ok, just kidding that’s not the hardest part. Have you ever heard of Open Directory? Do you fear LDAP and DNS? Well, maybe you should. It’s always DNS. Just saying.
DNS (domain name system) is just a fancy word for a list of IP addresses and host names. Using the raspberry Pi with dns masq installed we can populate the list of hosts for the Xsan and then we are golden. Hopefully if we did it right. Turns out we can make mistakes here too. Don’t use “.local” domain names. I did. It was late. I blame being tired. Changing them to “.lan” worked better.
Next up we finally create an Xsan in terminal. Or do we? let’s check the hostname first. It’s always DNS.
scutil —get HostName
CrazyMac.local
scutil --set HostName XsanMac.lan.
And now we make very big Xsan using the Xsan guide example
It was at this point that it started falling apart. It was late. I had messed up my DNS with “.local” and the Xsan wouldn’t go past this basic OD setup. I did what I always do and reach out to my Xsan colleagues and I got some curious feedback. “What do you mean Xsan isn’t in macOS Big Sur Server.app?” Hmm. I don’t see it on a fresh install. On an upgrade from 10.15 Catalina I do. So, uh, Where is it? And then it was revealed. In the View menu. Advanced. Ugh. It’s right there. Almost staring right at me. When I opened the app it said it couldn’t create an Xsan with my “.local”. That was helpful. Fixed that and Xsan with my pre-labeled LUNs was super quick to set up.
Xsan configuration in Server.app. “Ignore ownership” is the best thing ever for creatives. Trust me,
I’ll have to play with the cli set up again soon. Because there were some strange formatting it recommended to me when I tried some variations of the xsanctl createSan. I’ll dig into another day when I have more sleep. Ha ha.
There’s a lot of useful commands in macOS Big Sur Xsan which was upgraded to v7. You can check which version of Xsan you have in macOS with the cvversions command.
In Catalina (macOS 10.15.7)
File System Server: Server Revision 5.3.1 Build 589[63493] Branch Head BuildId D Built for Darwin 19.0 x86_64 Created on Tue Jun 22 21:08:03 PDT 2021 Built in /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/XsanFS/XsanFS-630.120.1/buildinfo
In Big Sur (macOS 11.5.2)
File System Server: Server Revision 7.0.1 Build 589[96634] Branch Head BuildId D Built for Darwin 20.0 x86_64 Created on Wed Jun 23 00:32:35 PDT 2021 Built in /System/Volumes/Data/SWE/macOS/BuildRoots/d7e177bcf5/Library/Caches/com.apple.xbs/Sources/XsanFS/XsanFS-678.120.3/buildinfo
There’s a lot of cool new binaries in Xsan v7. We will dig into those next post. For now enjoy this and go forth make some Xsan volumes with Thunderbolt or fibre channel storage. It’s fun.
I am so happy to install macOS Big Sur 11.5.1, now that it is a ready for production. Have fun with macOS Monterey those of you on the bleeding edge. For media professionals using Xsan in production storage environments August is a great month to update to the soon to be yesterday’s bad boy Mr. Big Sur.
Server.app v.5.10 in macOS Catalina 10.15.7
Upgrading to a new major version of macOS can be fraught with peril for a fleet of mac devices but it is potentially fatal for a production SAN environment. That is why we wait. We want a nice stable storage system for our Final Cut Pro editors and other media creatives so it is safe to be one version behind. Less drama that way. We prefer our dramas to be on AppleTV+
Watch TV Upgrade Xsan
It is not boring to watch AppleTV+ while upgrading Xsan
The Xsan upgrade to Big Sur was pretty much not exciting except for one funny roadblock that I had set up myself last as a kind of booby trap for “future me”. More about that later. First the boring stuff. The last few weeks have been very busy updating and re-writing documentation in Pages.app and running multiple redundant full and incremental LTO backups with Archiware P5, syncing to nearline archives, and archiving finalized projects to the LTO shelf in paradise (sounds more exciting when you put it that way don’t you think?). Updating and re-writing documentation can sound like a waste of time but “future you” will appreciate what “past you” was doing today. And today I had fun updating Xsan to macOS Big Sur. Now I must write down all my thoughts before I each too much vegan vanilla ice cream and slip into a food coma.
“Planning for disasters, while hoping for none” is the IT mantra. We planned hard and we were ready to restore Xsan from Time Machine, if we had to. Not a joke. The server is backed up by Time Machine. The data is backed up to LTO, nearline archives racked and stacked in a server room and on redundant thunderbolt RAIDs which are parked on electric trucks ready to blast off at the earliest sign of danger. Well, everything except for the last part. Would be nice. And cloud backups for those clients that want them. Plan for the worst, pay for what you can to keep your business operational and lessen the impact of mechanical failures, human oopsies, or ransomware. Sysadmins are indistinguishable from malware sometimes, but we mean well. More seriously, humans makes mistakes and break things (that, me!) but ransomware is real and my elaborate backup and archive planning has saved a few customers this year.
Xsan volumes are typically made of up fibre channel RAID arrays. Nice icon!
Preparation is key. Be prepared. Get ready. Psych yourself up. I used Greg Neagle’s installinstallmacos.py to download macOS Big Sur as a disk image and had that and the App Store’s Server.app downloaded beforehand and not be dependent on internet access (production SANs are not always internet accessible). It is both true and not true that you can setup Xsan in Big Sur with the Server.app. It is true you need the Server.app for an upgrade from macOS Catalina 10.15.7 but if you’re starting from scratch in macOS 11 you will be building your Xsan in Terminal. Have fun! (We will cover this in a future post).
Download macOS Big Sur and the Server.app. Keep old copies zipped up. Cvlabel is nice too
Server.app manages only three (3) services for an Xsan upgrade: Profile Manager, Open Directory and Xsan. In macOS Big Sur new setups of Server.app Xsan is gone. Why they haven’t taken out Profile Manager and not kept Xsan instead made me scratch my head. No one in their right mind is using Profile Manager to install or manage profiles, they’re using commercial MDM vendors. But Xsan in macOS Big Sur (11) is not only production ready storage SAN awesome it has been upgraded to be compatible with Quantum’s Stornext 7 (previously it was only v.5)
Profile Manager does not belong here. Long Live Xsan!!
Installing macOS 11 Big Sur and upgrading Xsan to v7 is compatible (in my testing) with macOS 10.14 Mojave, 10.15 Catalina and of course macOS 11 Big Sur. If you don’t believe me check out this not updated in forever Apple’s compatibility chart.
Ok, by this time you get the idea I’m an expert, right? I’m ready to upgrade. But I run into my first real road block. And I have only myself to blame. I can’t launch the macOS Big Sur install app. It is blocked. “Contact your administrator”?! I am the sysadmin. Oh, ok. That’s me. What have I done now? I installed Hannes Juutilainen’s Big Sur Blocker last year, that’s what.
Of course I installed that. With Munki. On all my Mac clients that were upgraded to macOS Catalina. (And of course my Xsan controller has Munki!). But no worries, let me read up on my last year’s blogpost about it to figure out how I installed it, there must be a launch daemon or something.
this is not how I expected it to go
Hmm, no didn’t mention there. And where is that pesky launch daemon that I can unload and get to this Big Sur install. Oh? It’s a launch agent. Unloaded. Hmm, still no. Ok, delete the app from /usr/local/bin, hmm, nope. ok kill the app process. Ok, now we can install macOS Big Sur. Sorry for the delay. I had told Munki to uninstall the bigsurblocker app and it did for every other Mac, I swear, really. It did.
Please proceed with the macOS Big Sur install
So ready for macOS Big Sur. Oh wait, we noticed that you’re running Server.app and well, we don’t do a lot of the same things anymore in the new Server.app so maybe this is a warning.
Warning. We noticed that you’re running Server.app and we don’t do those fun things anymore.
So a lot of progress bars and stuff. See my last upgrade blog post and it’s the same as installing macOS Big Sur on any Mac, except this Mac Mini is running an Xsan production SAN environment with a lot of RAID arrays in a server rack or two. Ok, yeah, just run the installer.
We noticed that Server app is no longer server app.
After macOS Big Sur is installed zip up your older server.app and drag in your new one (or use that fancy App Store app to do it for you if you’re lazy). Click a bunch of buttons (see all my old blog posts) and launch the new Server.app.
Profile Manager is updating. No one cares.
So we have to wait while the bag of scripts that is Profile Manager gets updated but no one uses it but it’s the most important app in Server.app now, no I am not bitter why do you ask. Xsan is awesome.
Time to restore from your old Xsan configuration. Wheee…..
Xsan restore configuration.
Activate your Xsan and carry on upgrading all your Mac clients. Note: I did test macOS Mojave 10.14, macOS 10.15 Catalina and of course macOS 11.5.1 Big Sur Xsan clients. All worked.
Xsan on. Power up.
Upgrading Xsan with macOS Big Sur is easy if you’re going from macOS Catalina. Starting from scratch is another story to be covered in another blog post. Also not covered is certificate issues from self-signed certs breaking when I upgraded my Munki and MunkiReport server. That’s definitely another blog post. It’s just a webserver. Just. A. Web. Server. What is so hard? haha
Technical Errata:
With more than one Xsan controller it used to be recommended to upgrade the secondary before the primary but it is now best practise to upgrade the primary first to maintain the sanity of the OD data.
Xsan Upgrade Step by Step:
Clone the controllers. (+ Time Machine backups) Turn off the clients. Stop the Xsan Volume. Run cvfsck on the volume. **Upgrade the primary. Confirm the secondary can see the primary. *Upgrade the secondary. Confirm the secondary can see the primary. Check SAN access on both controllers.
