Thunderbolt Xsan: Set up a T-SAN

Setting up your very own Xsan at home… What could be more exciting? Nothing like SAN storage to cure those stacks of hard drive blues. Don’t have a spare fibre channel switch or fibre channel storage at home? No problem Grab some thunderbolt storage from Accusys and join the fun.

I am testing the A12T3-Share 12-drive desktop Thunderbolt RAID solution to build my Xsan. Accusys also have a 16 drive rack mounted raid storage box if you want to install a nice pro set up in the server room you have tucked neatly in your home office. Ha ha. Seriously, the 12 drive unit is whisper quiet and would be a great addition to any home lab or production storage setup. I mean, aren’t we all doing video production at home these days? And even if we are doing a proxy workflow in the clouds, we still need to store the original footage somewhere before it goes to LTO tape, or backed up in the clouds (hopefully another cloud). A few years ago I tested the Accusys 16 drive Thunderbolt 2 unit and it worked perfectly with my fibre channel storage but this time I am testing the newest Thunderbolt 3 unit. Home office test lab is GO!

It is a pretty straight forward setup but I ran into some minor issues that anyone could run into and so I want to mention them and save you all the frustration by learning from my mistakes. Always be learning. That’s my motto. Or “break things at home not in production”, but if your home is production now, then break things fast and learn very quickly.

First step is to download the software for the RAID and you’ll find it on the Accusys website.

(I found the support downloads well organized but still a bit confusing as to what i needed)

The installer is not signed which in our security conscious age is a little concerning, but examining the package with Suspicious package should allay any concerns.

The installer installs the RAIDGuard X app which you will need to configure the RAID.

Of course, RAIDGuard X needs a Java Runtime Environment to run. Why is this still a thing? Hmm…

RAIDGuardX will allow you to configure your connected Thunderbolt hardware.

Configure the array as you like. I only had four drives to test with. Just enough for RAID5.

Choose your favourite RAID level. I picked RAID5 for my 4 drives.

The first gotcha that got me was this surprisingly simple and easy to overlook section. “Assign LUN automatically” asks you to choose which port that LUN (the configured RAID) will be assigned to. If you don’t check anything like I didn’t in my first run through then you configure a RAID5 array that you’ll never see on your connected Mac. Fun, right? Ha ha.

Xsan requires a sacrifice…. I mean, a LUN (available RAID array). Check your Fibre Channel in System Information. Yes, this is from the thunderbolt storage. Hard to believe, but it’s true!

Setting up enterprise grade SAN storage requires a trip to the Mac App Store. Server.app

Open Server.app, enable Xsan, create a new volume and add your LUN from the Accusys Thunderbolt array. Set the usage to “any” (metadata and data) since this is a one LUN test setup.

Pro tip: connect your Xsan controller to your Open Directory server. Ok, just kidding. You don’t have an OD server in your home office? Hmm… Create an entry in /etc/hosts instead.

If you’ve set up your SAN volume then you will see it listed in the Finder.

Easy shareable SAN storage is possible with thunderbolt RAID arrays from Accusys. No more Fibre channel switches needed. Small SAN setups are possible for creative teams without a server room. This setup was a quiet 12 drive RAID and a Mac mini. Add some Thunderbolt cables. There are four thunderbolt 3 connections and you can add more with an additional RAID. Up to 8 connections with one of them for the Mac Mini running the SAN. Not bad at all. And Xsan is free. Add a Server app from the App Store, but the Xsan client is free and built-in (Xsan has been included with macOS since 10.7 so many years ago). Fibre channel protocol (even through Thunderbolt) is faster than network protocols and great for video production. Fast and shareable storage at home. Or in your office. Thunderbolt Xsan. T-SAN.

Xsan Upgrade and Big Sur Prep. Hello Catalina!

Big Sur summer testing time!

Summer time is beta testing time. A new macOS beta cycle with Big Sur is upon us. Test early, and test often. With all the excitement of Big Sur in the air, it’s time to look at Catalina.

Our day to day production Xsan systems do not run beta software, not even the latest version of macOS, they only run tested and safe versions of macOS. I always recommend being a revision behind the latest. Until now that meant macOS 10.14 (Mojave). With the imminent release of macOS Big Sur (is it 10.16 or macOS 11?) then it’s time to move from 10.14.6 Mojave to 10.15.6 Catalina. It must be safe now, right? 

Background

Xsan is Apple’s based Storage Area Network (SAN) software licensed from Quantum (see StorNext), and since macOS 10.7 aka Lion it has been included with macOS for free (it was $1,000 per client previously!).

Ethernet vs Fibre Channel vs Thunderbolt

A SAN is not the same as a NAS (Network attached storage) or DAS (direct attached storage). A NAS or other network based storage is often 10GbE and can be quite fast and capable. I will often use Synology NAS with 10GbE for a nearline archive (a second copy of tape archive) but can also use it as a primary storage with enough cache. Lumaforge’s Jellyfish is another example of network based storage.

Xsan storage is usually fibre channel based and even old 4GB storage is fast because … fibre channel protocol (FCP) is fast and the data frames are sent in order unlike TCP. It is more common to see 8GB or 16Gb fibre channel storage these days (though 32GB is starting to appear). And while fibre channel is typically what you use for Xsan you can also use shared Thunderbolt based storage like the Accusys A16T3-Share. I have tested a Thunderbolt 2 version of this hardware with Xsan and it works very well. I’m hoping to test a newer Thunderbolt 3 version soon. Stay tuned.

Xsan vs macOS Versions

We’ve discussed all the things that the Xsan is not and now what is it? Xsan is often created from multiple fibre channel RAID storage units but the data is entirely dependent on the Xsan controller that creates the volume. The Xsan controller is typically a Mac Mini but can be any Mac with Server.app (from Apple’s App Store). The existence of any defined Xsan volumes depends on the sanity of its SAN metadata controllers. If the SAN controllers die and the configuration files go with it then your data is gone.  POOF! I’ve always said that Xsan is a shared hallucination, and all the dreamers should dream the same dream. To make sure of this we always recommend running the same version of macOS on the Mac clients as well as the servers (the Xsan controllers). And while the Xsan controllers should be the same or at a higher macOS version level it can sometimes be the opposite in practise. To be sure what versions of macOS are interoperable we can check with Apple’s Xsan controllers and clients compatibility chart and Xsan versions included in macOS for the rules and exceptions. Check the included version of Xsan on your Mac with the cvversions command

File System Server:
  Server  Revision 5.3.1 Build 589[63493] Branch Head BuildId D
   Built for Darwin 17.0 x86_64
   Created on Sun Dec  1 19:58:57 PST 2019
   Built in /BuildRoot/Library/Caches/com.apple.xbs/Sources/XsanFS/XsanFS-613.50.3/buildinfo

This is from a Mac running macOS 10.13

Host OS Version:
 Darwin 17.7.0 Darwin Kernel Version 17.7.0: Sun Dec  1 19:19:56 PST 2019; root:xnu-4570.71.63~1/RELEASE_X86_64 x86_64

We see similar results from a newer build below:

File System Server:
  Server  Revision 5.3.1 Build 589[63493] Branch Head BuildId D
   Built for Darwin 19.0 x86_64
   Created on Sun Jul  5 02:42:52 PDT 2020
   Built in /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/XsanFS/XsanFS-630.120.1/buildinfo

This is from a Mac running macOS 10.15.

Host OS Version:
 Darwin 19.6.0 Darwin Kernel Version 19.6.0: Sun Jul  5 00:43:10 PDT 2020; root:xnu-6153.141.1~9/RELEASE_X86_64 x86_64

Which tells us that the same version of Xsan are included with macOS 10.13 and 10.15 (and indeed is the same from 10.12 to 10.15). So we have situations with Xsan controllers running 10.13 and clients running 10.14 are possible even though macOS versions are a mismatch, the Xsan versions are the same. There are other reasons for keeping things the macOS versions the same: troubleshooting, security, management tools, etc  To be safe check with Apple and other members of the Xsan community (on MacAdmins Slack).

Backups are important

Do not run Xsan or any kind of storage in production without backups. Do not do it. If your Xsan controllers die then your storage is gone. Early versions of Xsan (v1 especially) were unstable and the backups lesson can be a hard one to learn. All later versions of Xsan are much better but we still recommend backups if you like your data. Or your clients. (Clients are the people that make that data and pay your bills). I use Archiware P5 to make tape backups, tape archives, nearline copies as well as workstation backups. Archiware is a great company and P5 is a great product. It has saved my life (backups are boring, restores are awesome!).

P5-Restore-FCPX.png

Xsan Upgrade Preparation

When you upgrade macOS it will warn you that you have Server.app installed and you might have problems. After the macOS upgrade you’ll need to download and install a new version of Server.app. In my recent upgrades from macOS 10.13 to macOS 10.15 via 10.14 detour I started with Server.app 5.6, then install 5.8 and finally version 5.10.

After the macOS upgrade I would zip up the old Server.app application and put in place the new version which I had already downloaded elsewhere. Of course you get a warning about removing the Server app

 

Xsan-ServerApp-ZipRemovalDetected.png

Install the new Server app then really start your Xsan upgrade adventure.

Serverapp-setup.png

Restore your previous Xsan setup.

This slideshow requires JavaScript.

If everything goes well then you have Xsan setup and working on macOS 10.15.6 Catalina

Xsan-Catalina-Upgrade-Success

I don’t get High — Sierra!

Friends don’t let friends install macOS High Sierra in production. Don’t get High, Sierra.

macOS 10.13 was released on Sep 25, 2017, and almost two months later with only one point release update, it’s still too new for production. Download it on a test machine or two or more, test it with your apps and systems, file bug reports and radars, but for the love of all that is Python and Monty! don’t run it on your production Xsan. Well, at least not yet. Wait until next year. Or as long as you can. Or until the new iMac Pro is released with 10.13 pre-installed or wait until they ship the new Final Cut Pro X 10.4 that may or may not require macOS High Sierra.

With that out of the way, I’ve just upgraded the production Xsan to … macOS Sierra. Yes, macOS 10.12.6 is stable and it’s a good time to install last year’s macOS release. Time to say good bye to macOS el Capitan 10.11.6, we hardly knew ya. Besides guaranteed security updates, stability and the annoying newness of a changed macOS, what else is there? In Xsan v5 they introduced a new “ignore permissions” checkbox for your Xsan volumes. Looking forward to that feature in production. No more Munki onDemand nopkg scripts to run chmod. No more tech support requests for folders, files, FCP X projects that won’t open because someone else used it, owns it, touched it. We’ll see how that pans out. I’ll let you know.

Upgrading Xsan to v5

Step 1. Back up your data

You’re doing this, right? I’m using Archiware P5 Backup to backup the current projects to LTO tape. I’m using Archiware P5 sync to sync the current Xsan volumes to Thunderbolt RAIDs, and using Archiware P5 Archive (and Archive app) to archive completed projects to the LTO project archive. That’s all I need to do, right?

Step 2. Back up your servers

Don’t forget the servers running your SAN! I use Apple’s Time Machine to backup my Mac Mini Xsan controllers. External USB3 drive. I also use another Mac Mini in target disk mode with Carbon Copy Cloner to clone the server nightly. (Hat tip to Alex Narvey, a real Canadian hero). And of course I grab the Xsan config with hdiutil and all the logs with cvgather. Because, why not?! For Archiware P5 backup server I also have a python scripts to backup everything, another scripts to export a readable list of tapes, and BackupMinder to rotate the backups. Add some rsync scripts and you’re golden.

 

Step 3. Upgrade the OS

Unmount the Xsan volume on your clients or shut them down, disconnect the fibre channel. Do something like that. Stop your volume. Download the macOS Sierra installer from the App Store. Double click upgrade. Wait. Or use Munki. I loaded in the macOS 10.12.6 installer app into Munki and set it up as an optional install to make this portion of the upgrade much quicker and cleaner.

In my case after the OS was upgraded I checked the App Store app for any Apple updates (you can also use Munki’s Managed Software Center to check) and of course there were some security updates. In this case the security upgrade hung on a slow network connection and the server crashed. Server down! I had to restore from Time Machine backup to the point where I just upgraded the server. It took some extra time  but it worked (can’t wait for next year’s mature APFS / Time Machine and restoring from snapshots instead).

Step 4. Upgrade Server

After macOS is upgraded you’ll need to upgrade the Server.app or just upgrade the services used by Server (even those not used by Server get upgraded).

Step 5. Upgrade the Xsan

Bur first we have to restore the Xsan config. Don’t panic! It may invoke bad memories of data loss and restoring from backups. Xsan PTSD is real.

Restore-previous-Xsan.png

Step 6. Upgrade the rest

Next you have to upgrade the Xsan volumes.

Xsan-volume-needs-upgrade

New version of Xsan, ch-ch-changes! Ignore permissions check box will remount the xsan with the “no-owners” flag. Let’s test this out.

 

Upgrade the OS and Server app on the backup controller. Upgrade the OS on the clients using Munki or App Store if you like doing it the hard way. Ha Ha.

Step 7. Enjoy

Plug those Thunderbolt to Fibre adapters back in, mount those Xsan volumes and be happy.

Step 8. Wait for the complaints

The next day the editors walked in and went straight to work with Final Cut Pro X. No one noticed anything. Xsan upgraded. Workstation macOS upgraded. Everything appeared to be the same and just worked. Thankless task but well worth it.

 

Reference: Apple’s iBook guide here

 

 

My Thunderbolt Nightmare

It was a dark and stormy night of cables and capacitors when suddenly I heard the door knocking, or was something falling of a shelf? I was in a cramped server room, if you’d call it that, and I was day dreaming, sorry, night terrorizing, of days gone past when I worked in nice big well ventilated server rooms with proper enterprise gear. Oh wait, did I really dream that? Did it really happen? Maybe it was less well ventilated and there were cables strewn about the tall 42U shelves and sometimes we found a Mac hidden underneath spaghetti. Sometimes. I vaguely remember the long shiny metal servers, they talked to me, they sang, a whiny pitch of whale song. Dream on, dream on.

Now. Today. Apple Music on my iPhone plays every single Arcade Fire album in a long playlist, in order. And I follow the white rabbit of Thunderbolt cables. This is my thunderbolt nightmare. Dead drive in a Thunderbolt Promise Pegasus unit, web ticket filed for registered hardware. Legacy. That’s the word they used. Where’s Marshall McLuhan when you need a proper redux of the shit storm you’re in? Thunderbolt 3 uses USB-C and everything is possible. Can’t wait to step into that confusing identity crisis. OK, back to the present day when I stared at the red blinking drive, a replacement drive from not long ago dead again, sitting in the last row of a now legacy Pegasus R6 unit. RAID 5, the most dangerous kind, this is what stood between me and uncertainty. The worst kind of RAID. Well, not as bad RAID 0. Raid nothing. Raid 5 is one bad drive away from a bad day. Backups? Hmm, I got those, I got plenty of those, but I don’t want to be tested today. No, not today. Not this bloody day.

I open the Pegasus utility and the GUI wants an update. Hmm, that’s not in autopkg, I think. Why is out of date? Munki let me down. I start to drift, to side shift into adding newer better recipes to autopkg, to tweaking my Munki repo, to what sessions would be awesome at the next MacDevOps:YVR conference. Gee whiz, I love open source, and everyone in the Mac Admins community…. Snap out of! I slap myself in the face. I was hallucinating. Stay on task. I update the Pegasus utility. I stare at the critical reports from one of the three R6 units attached to this Mac Mini server. Did I say server? But it’s so small, so little. It works. It’s magical, kinda neat. Until you stare too close at the back. The Thunderbolt cables go from the Mac Mini to the first Pegasus unit to the SANlink fibre channel adapter to the LTO 6 tape library to the next Pegasus utility to the second SANlink adapter to a third and final Pegasus RAID unit. What’s is going on? Where does this cable go? Let me just follow it to the next jumping off point. My brain slows, the lack of oxygen in this cold machine room start to affect my thinking. I lose my way.

I download the report for the Pegasus unit. I had to unlock a pretty neat lock icon and click on the save report. I upload it to the web support and add it to the ticket. Tech support gets backs to me in a day and said all is good, and to carry on. I can’t. The drive is dead. What are they not seeing? It’s right in front of me. I download the report again. Again the same response. Fine. It’s time to stop messing around and pop open Terminal. Loading up promiseutil I check out the options and switches and get into an argument with myself about the currently valid optionals of letters and numbers that are required. I check my notes, online knowledge base, and try again. It’s broken. It doesn’t work. Stumbling around the command line typing imprecisely incorrect statements gets nowhere fast. I realize that there’s no way for the cli utility to properly change its focus to the broken unit with the busted drive. Both the GUI and the binary are stuck on the one R6 unit and won’t see what’s in front of my face.

I call tech support. This is humiliating. This was supposed to be easy. Drive dead, drive reported, drive ordered, drive replaced, then no one the wiser. Data saved, not dead. Backups not tested. Not today. No, not today. Tech support treats me like the imaginary newbie IT people sometimes treat everyone with. He repeats his instructions to me. He is polite. Download the report. I can’t. It won’t work. Unplug the unit. Plug it into something else. I can’t. The cables. The Thunderbolt cables are everywhere. It’s magical, and daisy-chained, and stuck. “Can I remote in and see?” he asks, hoping to resolve this quickly. Sorry. That’s impossible. Even if I thought it was a good idea. I remind him that I have a dead drive. That’s why I called. I want to get a replacement drive. “Sorry sir that legacy unit is not under support most likely,” I know that. I realize that now. That I wasted my time. It happens sometimes. The truth is staring at you. You need a mirror to see. “You need to order compatible drive from the compatibility list.” I am a well spring of emotions. I thank him. I am nice. He was polite. But now I know what I need to do. Oh wait, what? Order a drive now! Order two.

McLuhan never had a chance to evaluate Thunderbolt storage technology but the insane genius and simplicity of Thunderbolt reduced expensive enterprise fibre channel storage to the dust bin. Magical SAN for video editing with a Mac Mini and Thunderbolt RAIDs. Cheap enough to buy with a departmental credit card, fewer meetings to attend, more films to shoot and edit. Backup, archive, repeat. McLuhan would have no doubt reminded me that the tetrad of technology would have flipped Thunderbolt on its head, Fibre channel never went into a dust bin, but was firmly relegated to well cooled storage room, and long ago legacy drives in the enterprise units are humming a long while the cursing wind and emotions swell over the Thunderbolt mountain. Fibre channel just became cool again. Retro smart.

Best of 2015: Archiware P5 Archive app

Announced late in 2015 the Archiware P5 Archive app is a revolution for editors who want to control the archive and restore process. No longer the job of the IT Admin, editors can select files or folders on their SAN volume (or anywhere) and send them to the tape archive.

The Archive app is a brilliantly simple app that allows the right-click services action in OS X, or in another words a it’s a GUI app that presents a contextual menu that knows to how to the talk to your P5 Archive server. When the files are safely on tape the original files on the filesystems are replaced with stub files that can be used to start the restore process.

Requirements: Archiware P5 server with the Archive module setup with an Archive plan. Add to that the P5 Archive App which is installed on the clients.

Note: At the moment all archiving goes over the LAN by default, so if you have a fast SAN then you set up the P5 Archive app client settings as “localhost” instead of their actual client name. That means that when it goes to archive the file, the server knows that the files exist on the SAN at a known path (which is the same on the client and the server).

And now for some detailed steps and screenshots.

  1. Archiving completed projects

Choose the completed project folder and right-click. Select “Archive to P5”.

Note 1: If you want to restore files choose the folder that was archived and right-click. Choose “Restore from P5”.

Note 2: Restoring individual files that have been archived is possible by double-clicking the files with the “.p5a” extension, but it will be much faster to select an entire folder to restore than many individual files.

Note 3: For either archive or restore to work the P5 Archive app needs to be installed.

Note 4: To avoid having a services sub menu keep the contextual-menu items to four.

Right-click folder to archive

2. Archiving app status

When you are archiving or restoring files the Archive app will show you the status of your request. It will also show you the status of other jobs running on the P5 server. This is to let you know why perhaps your archive or restore is taking a long time (it’s possibly waiting for access to the tape drive and it currently busy backing up or archiving something else).

P5 Archive app Running jobs status

The P5 Archive app offers you three operations “cancel job”, “list items” and “get report”. The last two are great when you want to examine a completed job, for example. If you want to find out what files were archived in the particular job choose “list items”.

3. Restoring files

Archived files will have either one of or both of, 1) a”.p5a” file extension and 2) a P5 Archive app icon.

Folders and FCP X project bundles (which are folders) do not get the “.p5a” extension, but FCP X projects have the the icon.

p5a-icon.png

Note 1: Files can also be restored by the admin through the P5 web interface. They can be restored in place or to any other location that is required.

Note 2: On the P5 server jobs that are sent to archive or restored from tape show up as “cli job” with the tapes in use.  Actual files or folders involved need to be noted from the P5 Archive app not the P5 web admin console. Otherwise checking the P5 web restore tab will files actually archived (that can be restored).

That’s enough for the quick overview of this great new app. One of the best things in 2015.

For more information on Archiware’s new P5 Archive app check out their website:

P5 Archive app

2015 in review

The WordPress.com stats helper monkeys prepared a 2015 annual report for this blog, but nobody wants to read no stinkin’ reports so let me just sum it all up: Xsan, Munki, Thunderbolt, Archives. Or is that all one word? Thunderbolt Xsan Munki Archives! That’s better.

Here’s an excerpt from the report that no one will read:

The concert hall at the Sydney Opera House holds 2,700 people. This blog was viewed about 20,000 times in 2015. If it were a concert at Sydney Opera House, it would take about 7 sold-out performances for that many people to see it.

Surprisingly, or not, that opera would be about Xsan. Yes, Apple’s Xsan is still alive, and Apple even added new features with OS X 10.11 El Capitan. I’m still building Xsan shared storage SANs and upgrading old ones to new versions. That was one of the good news stories of 2015 for me.

You can build an Xsan with one or two Mac Minis and add your storage of choice. That used to mean more often than not the fibre channel storage from Promise. A great choice for larger deployments, the x30 Vtraks are solid.

But the real shocker for me in 2015 was stumbling upon the Accusys Thunderbolt SAN RAID, the A16T2-Share. For more than half off the price of a similar fibre channel storage RAID here’s a magical box powered by unicorns that has four (4) Thunderbolt connections. Plug one Thunderbolt cable into that Mac Mini, format the raid, setup Apple’s Xsan, and then plug the other three (3) Thunderbolt cables into iMacs, Mac Pro, MacBook Pros or any Xsan clients. Wow. Awesome.

Suddenly we have a game changer. An affordable SAN storage RAID for real block-level storage. Now more than ever we can afford to have true collaborative workflows for video editors and anyone in the creative. If you need to work together with fast connections to a shared pool then building an Xsan got much more attractive.

Disclaimer: I got a chance to test the Accusys A16T2-Share. And I would be crazy to recommend something without testing it thoroughly. This was used for several weeks by video editors in production. It was much faster than our 4GB fibre channel storage, of course, but it was also faster than our 8GB FC storage. Speed tests showed we got close to 1GB/sec, and even when it was 97% full we got 700MB/sec. Sa-weet.

I look forward to seeing what Accusys bring to NAB in 2016. What new box will they show up with? I hope for more than 4 client ports and faster Thunderbolt 3. Only 82 more sleeps till we all find out.

Apple’s Xsan and Accusys Thunderbolt storage A16T2-Share were big stars of 2015, but what else stood out? The two other bright shiny lights were Archiware’s new P5 Archive app, and Vidispine’s VidiXplore cloud based MAM. More on those in posts to follow. Both of these products have transformed workflows for editors. Stay tuned!

Thunderbolt SAN talk at Mac Admin meet up

Big thanks to Ross at Ping Identity for organizing and Jamf for sponsoring the Mac Admin meetup on September 9, 2015.

We filled the tiny meeting room and we will have to expand to the larger conference room (or theatre) next time. It was a well attended meetup with much discussion of the earlier day’s Apple announcements, new OS X “El Capitan” and iOS 9 changes and how this affects management products like Casper which have had to move the binary because of the new SIP implementation in OS X.

I opened up the meet up with a presentation on Storage, SANs and the new Accusys Thunderbolt SAN A16T2-Share product.

The goal of my presentation was to give a quick overview of SAN technology as I’ve seen it change over the last 10 years: from Fibre Channel, to iSCSI to PCIe and Thunderbolt based. The last change to Thunderbolt based SANs is the most interesting for small video production workgroups or anyone that likes working on small scale shared projects but needs a decent bandwidth at an affordable price. Block level storage (SANs) is straight forward storage tech for users and applications to interact with without having to negotiate network protocols (AFP, SMB, or NFS). It’s never been quite that affordable until now.

Having built a lot of Fibre Channel based SANs for media and entertainment companies and post-production editors in corporate environments I know how awesome and fast and solid these SANs are. Lots of editors and clients can hit a large SAN and it won’t blink. Thirty or Sixty users is not unusual. But not everyone believes in fibre channel or the idea of pulling fibre cables. It is surprisingly a large stumbling block to building large SANs, “no, we don’t want fiber cables”. True, sometimes clients have objected to gigabit Ethernet too, but that’s another story.

I found that iSCSI, especially with the DDP units I’ve set up, has been a great alternative to fibre channel. Not fiber cables to pull. Just use the CAT6 cables already in place. Great Ethernet based SANs using 1 x or 2 x CAT6 cables per client, or even 10G. Works well. Very well indeed. It’s been great for smaller (and larger) clients who want a great Ethernet iSCSI SAN solution without needing fibre channel cables, switches, HBAs, Thunderbolt adapters, etc.

That’s why when I stumbled across the Accusys Thunderbolt storage I was kinda really excited. No fibre channel to Thunderbolt adapters. Just use Thunderbolt cables. Brilliant! Finally a solution for small workgroups. And there’s so many video groups sprouting out of every corporate office, or boutique VFX or post-production shops that have been struggling with small NAS solutions that were not meant for video production. Now you can get that SAN that you’ve wanted, you can really get that block-level storage at an affordable price. Instead of working locally and copying raw footage and finished products  back and forth across slow network links they can work in a small video group with high speed storage. Sa-weet. (Can you tell I’m excited?).

I’ll include the presentation PDF here as a link if anyone is interested. I’ve added a link at the end from Accusys on how to build an Xsan with the A16T2-Share. Yes, Xsan from Apple still exists and is bundled with the OS for free. Building a SAN is pretty easy and everyone can do it. Don’t forget your backups though.

Lastly, anyone interested in attending any meetings for the upcoming MacDevOps:YVR (June 16-17, 2016) drop me a note. I added the email in the presentation document.

MatX_SAN_Accusys-Thunderbolt_2015

Move over El Capitan, hello Yosemite!

With all this talk about El Capitan, Apple’s as of yet unreleased version 10.11 of OS X, and its wondrous new features in Xsan, I think it might be time to upgrade to last year’s breakthrough version of OS X, Yosemite. Sure, you might be excited by the press releases for the built-in DLC in El Capitan but seriously sane folks stay 1-year behind the bleeding nose upgrades provided by Apple. So if OS X 10.11 is all the rage before its released it must be time to seriously consider upgrading that working Xsan running OS X 10.8 or OS X 10.9.

In my case, I upgraded a working Xsan running on Mac Minis and OS X 10.8.5. Here are some screenshots from the process. As always think worked better than I could have expected, and it is a much easier process that one expects. But stay sharp kids, danger lurks when you wake the dreamer…. Upgrading a SAN is serious business and doing anything like this without proper backups is taking your life in your own hands. In my case, full disk backups on Promise Pegasus RAIDs and full tape backups using Archiware P5.

Download the Yosemite installer form the App Store. Install. Download the new Server.app from the App Store. Install. Now upgrade your Xsan. That’s it. You’re done. No surprises, aren’t you happy? Ha ha. I’m kidding. The fun is just getting started.

If you’re actually following along, this isn’t a step by step recipe. Go to Apple’s site and read this Kbase and check out the migration guide.

Restore Xsan

Restore Xsan

Step 1 is to launch the new Server.app, find Xsan Admin. Just kidding, it isn’t there. Enable Xsan, and choose to Restore a previous SAN configuration. That wasn’t hard. High five! Actually, we’re not done yet. Set up OD now. Go!

Step 2. Set up your Xsan controller as an Open Directory (OD) master. Does’t matter if it’s joined to another domain, Xsan keeps itself organized in OD, so you need it.

Set up OD

Set up OD

Step 3. Admire your upgraded SAN, “how lovely the flowers do smell…. life is good.”

XSAN LIST

Xsan list

Step 4. Where did my Xsan admin go? Where do I add clients? Where are my clients? Huh? What? Why did I upgrade a perfectly working SAN to this version? Ha ha.

Take it all in, take a good look at what you’ve done to your Xsan. What? Just so the editors could have the latest version of Final Cut Pro (v.10.2.1) which is only compatible with OS X 10.10.4. I see what you’ve done Apple, very clever indeed. Hmm…

Click on the “Save configuration profile” button and download the profile somewhere. Use this to set up the SAN on your clients. Distribute via Profile Manager or install it manually. Up to you. I haven’t gotten it to work with Munki quite yet. Installing it requires the admin password for the Xsan controller. How convenient.

When you client is configured you’ll see a Profile in System Preferences. Remove it and your client is un-configured. No more Xsan.prefpane to list volumes and mount or unmount them. Nope. That would be too easy. Learn to love “xsanctl”, as in “xsanctl mount Xsan”. Read some xsanctl tips in this Kbase

Step 5. Set up a backup Xsan controller. You have one of those, right? In my case, I had a client which I wanted to promote to be a controller.  But first what to do about its status a client of the Xsan?

backup cannot be client

backup cannot be client

Open Server.app, enable Xsan, join current Xsan as a backup controller and set up a replica OD. Confirm, confirm, confirm. Think about what you’re doing, then do it!

confirm OD replica

confirm

Apple wizards are the best wizards, uh, i mean Setup Assistants. No wizards here…. So, you’ve setup a backup Xsan controller, and OD replica, and now look in Server.app. How amazing is that… wait, what? Where’d my Xsan volumes go? Huh? Where are the controllers? Weird. Very strange. Not comforting at all.

Xsan 4 no SAN list crop 122815

The Xsan window eventually shows the volumes and controllers, bur geez, almost gave me a heart attack. It’s not like I never seen Xsan go bad before. Xsan 1 nightmare still haunt me. They do. Backups. Need more backups. Archiware P5 Backups, do it now!

OK, you’ve survived the uncertainty of Xsan upgrades…. But wait more minute… cat the fsnameservers (no, it’s not the name of a band, it’s a command). Check it out. Holy smokes, batman. Xsan 4 by default will set your metatadata network to the public LAN, something that’d would be laughed at years ago, but they do it now by default. Of course, upgrading our SAN kept out metadata network the same. But strangely the Xsan backup controller is set to use the public for metadata when the primary controller is not. WTF.

Change your metadata network. Read the Kbase, and once again wield xsanctl like a boss.

Xsan 4 in OS X 10.10 (Yosemite)

Apple released Yosemite (OS X 10.10) today.  The big news for me is the built-in version of Xsan is v.4. But don’t get too excited and upgrade your OS without some planning (and backups). If your systems are in production then please leave them as is. Install OS X 10.10 on a test system first. Install a test Xsan and play with that. Don’t test in production. ‘Nough said.

What you need to know is, if you upgrade your Mac to 10.10 then it is officially incompatible with Xsan 3. You can NOT have Xsan 3 (10.9) clients on a 10.10 Xsan, and I don’t think that 10.10 (Xsan 4) clients will work on a Xsan 3 based SAN. There may be a hack to get incompatible versions working together but that’s left to imaginative tinkerers and not useful for production where deadlines are involved.

I’ve done some basic testing with Xsan 4 and it does away with the Xsan Admin app, all setup is done in the Server.app. Also, it depends on Open Directory (and DNS, of course). If there is no OD master set up then it will create one (same with DNS). If you have OD then join your Xsan controllers to it as replicas or else they will create a new OD master on the first Xsan controller and a replica on the second. You were warned.

To configure the clients you export a config profile and install it on the clients, or alternatively you can enrol the Xsan controller in MDM (Profile Manager, for example) and push out the config to the clients.

I have not tested Xsan 4 with StorNext but I expect there is compatibility, as usual.

In Summary:

More testing is needed, but strictly speaking Xsan 4 is not going to work with Xsan 3 and vice versa. If an Xsan 3 (10.9 client) is part of Xsan 4 (10.10) then it may work but commands and configs will not come across (unmount / mount the volume, the volume is destroyed stop looking for it, etc).

And now for some screenshots of the actual setup.

Step 1. Install Server. Turn on Xsan and get ready to rumble.

Screen Shot 2014-10-15 at 2.02.06 PM

Step 2. Change your name. If you’re using dot-local change it.

Change-dot-local-name-Xsan4

Step 3. Set up valid DNS

Setup-DNS-if-you-dont-have-none

Step 4. Set up a new SAN

Set-up-new-SAN

Step 5. Choose a SAN name

Choose-SAN-Name

Step 6. Configure Users and Groups (OD)

Config-users-groups

Step 7. Choose your organization name

OD-name

Step 8. Create the Xsan volume

Add-Xsan-volume2

Step 9. Add LUNs to your storage

Edit-storage-pool-add-LUNs

Step 10. Save a configuration profile

Save-mobile-config

Step 11. Deploy config to clients

Use MDM or manually deliver the file to your clients.

Stay tuned.