Apple released Yosemite (OS X 10.10) today. The big news for me is the built-in version of Xsan is v.4. But don’t get too excited and upgrade your OS without some planning (and backups). If your systems are in production then please leave them as is. Install OS X 10.10 on a test system first. Install a test Xsan and play with that. Don’t test in production. ‘Nough said.
What you need to know is, if you upgrade your Mac to 10.10 then it is officially incompatible with Xsan 3. You can NOT have Xsan 3 (10.9) clients on a 10.10 Xsan, and I don’t think that 10.10 (Xsan 4) clients will work on a Xsan 3 based SAN. There may be a hack to get incompatible versions working together but that’s left to imaginative tinkerers and not useful for production where deadlines are involved.
I’ve done some basic testing with Xsan 4 and it does away with the Xsan Admin app, all setup is done in the Server.app. Also, it depends on Open Directory (and DNS, of course). If there is no OD master set up then it will create one (same with DNS). If you have OD then join your Xsan controllers to it as replicas or else they will create a new OD master on the first Xsan controller and a replica on the second. You were warned.
To configure the clients you export a config profile and install it on the clients, or alternatively you can enrol the Xsan controller in MDM (Profile Manager, for example) and push out the config to the clients.
I have not tested Xsan 4 with StorNext but I expect there is compatibility, as usual.
In Summary:
More testing is needed, but strictly speaking Xsan 4 is not going to work with Xsan 3 and vice versa. If an Xsan 3 (10.9 client) is part of Xsan 4 (10.10) then it may work but commands and configs will not come across (unmount / mount the volume, the volume is destroyed stop looking for it, etc).
And now for some screenshots of the actual setup.
Step 1. Install Server. Turn on Xsan and get ready to rumble.
Step 2. Change your name. If you’re using dot-local change it.
Step 3. Set up valid DNS
Step 4. Set up a new SAN
Step 5. Choose a SAN name
Step 6. Configure Users and Groups (OD)
Step 7. Choose your organization name
Step 8. Create the Xsan volume
Step 9. Add LUNs to your storage
Step 10. Save a configuration profile
Step 11. Deploy config to clients
Use MDM or manually deliver the file to your clients.
Stay tuned.
DAM SAN 
I have a mavericks Xsan and my macbook pro running yosemite mounted the xsan volumes automatically – it would appear that a yosemite client can mount a mavericks xsan volume.
That’s good to know. I will test this also today. I needed another test Mac in my Xsan lab. I found out 10.9 clients worked with 10.8 Xsan controllers when a client upgraded by themselves. It’s not recommended but what may work and what are officially supported can be different. I do wonder about what does not gets passed through since the communication channels are different now. Xsan 4 stores config data in OD. Was this mac laptop an upgrade with your old Xsan settings already configured or a brand new (clean) install ?
It was upgraded so I’m sure it see’s the SAN volumes because the settings were already in place – although the drives mount there is no longer an xsan control panel – not sure if terminal commands will work for unmounting/mounting volumes. Interestingly it didn’t mount san volumes until the developer release reached gold master.
Were you ever able to test 10.10 clients on a 10.9 SAN? The old best practice was to never run newer versions of the OS on older MDCs.
The new best practice is everyone Xsan 4 if anyone is Xsan 4. That is, if you upgrade clients to Yosemite then upgrade the MDCs. Unofficially you can join new clients to older MDCs but it’s not recommended.
It’s a shame yosemite server doesn’t support older clients – I use 2 old xserve clients to share volumes using NFS – which if your tests are correct I won’t be able to do if I upgrade.
We will have too see the long term impacts are … And what we can get away with in terms of older clients connecting. This kinda puts the nail in the coffin for older Xserves. They’re stuck at 10.7.
Where did you get the information about the incompatibility? Do you have any links/references to the official requirements, if so I would love to see them?
I upgraded a 10.9 Xsan 3 client to 10.10 and verified it mounted Xsan volumes fine (but this was 10.10 Public Preview so maybe it is not working in 10.10.0 anymore).
Ok found it myself: http://support.apple.com/kb/HT200135
This is interesting, so now the first time we get a new Mac with 10.10 all MDC’s and other Xsan Clients has to be updated. Great..
Let’s hope it works anyway and that the article is “supported” not “verified working”.
Yes, they’ve changed a lot of things in the new Xsan 4. Precisely what’s changed is the method of communication with the clients. Xsan 4 SANs require Xsan 4 client to understand the commands being sent. I heard initially from the dev forums that I had access to, but now this is released and wrstuden has explained it on Xsanity: http://www.xsanity.com/story/2014/notes-xsan4
do you still need to set up a osx dns server when there is already a windows dns server?
also i’m running into performance issues when setting up xsan. I ran performance tests on the xsan managed volume vs a mounted volume. Both volumes are from the same san and set of disks. xsan volumes pushing about 170mbps, and the none manage (san lun mounted using disk utility) is pushing 850mbps. Any idea?
If you have your own DNS server (Windows or otherwise) then no, you will not need another one. It will set up an OD master for its own use (if it’s not already an OD master).
The speed of a LUN mounted directly that is not being managed by a SAN should always be faster but that seems like quite a difference. What kind of files or workload are you testing it with? What is the rest of the infrastructure set up like?
+++ tests are being done on the xsan server itself and i’m using blackmagic disk benchmark
Try a real application, like Final Cut or Premiere to see what real world performance is like. But in general the fine grain control with volume creation seems to be lacking in the new Xsan setup. More testing needed. I’m not in a hurry to upgrade any production set up already running on 10.9 or 10.8.
wow your quick in response! appreciate that very much… yes it is OD master, and connected to 8GB brocade fiber switches. SAN is flash along with SAS disks. I too am confuse why such drastic performance numbers. Luns that is not managed by xsan is off the charts, while manage luns has such poor numbers. just doesnt make sense
by the way. do you know where to get documentation on xsan4?
also cant seem to post screenshots here, so figure i post a video.
I’m also going through XSAN issues on Yosemite. It’s true (for me) that Yosemite will work with an older XSAN client. However, the fiber connection isn’t active and you would only be connecting to the SAN via CAT5. So, there would be a dramatic speed difference. I’m hoping Mac developers are working on a fix for getting XSAN 4 compatible with the older version so that the fiber connectivity is being utilized and updating to Yosemite won’t be an issue.
What do you mean you’ve gotten Yosemite working with older clients? Do you mean older OS X clients with Yosemite Xsan controller? Getting Yosemite clients working with older OS X Xsan controllers with some hacking.
I mean I can access my san in Yosemite, via Cat5. Before, when I was on Mavericks – I would use xsan to connect to our DVS San Server via 2 NICs; Fiber and Cat5 (the fiber carries the data and the cat5 is for metadata.) Once I updated to yosemite, initially it seems to not work because the XSAN control isn’t even on the control panel and the san wasn’t getting mounted automatically. But if you open up the shared folders on the network, I was able to connect to the SAN. I notice the speed difference immediately because everything would take longer to load (I work with large video files in our SAN.) So, this leads me to believe that I’m accessing my SAN only through the CAT5 and not through the fiber. I’m sorry If I’m not explaining this clearly enough, I just an editor with a very shallow IT networking experience.
Sounds like you’re accessing it as a network share not directly as a fibre attached volume. Perhaps a controller or client is resharing the volume via cat5/6 Ethernet LAN. To get proper SAN access you’ll have to downgrade or hack your configs.
XSAN 4 is FUNDAMENTALLY different that previous versions. The metadata is now stored in LDAP open directory. This is such a change that we might as well be talking about XSAN 1 all over again.
This is why compatibility with previous versions and with StorNext have been broken.
Apple has also taken is a step further and disallowed XSAN 3.1 and older clients from running on 10.10 OS X clients. This smells like a business move, as there have been very few client restrictions in the past.
Also I am seeing issues copying large files (above 4 GB) from 10.10 to a StorNext volume over copper. I am getting “The item can’t be copied because it is too large for the volume’s format.”
These files copy fine in 10.9.
I think Apple definitely broke compatibility on purpose.
Yes it is very different. The importance of OD in the new structure is why older clients cannot be used.
Not sure why your files are not being transferred. Yosemite is new. We’re you using Samba or NFS to transfer the file? What version of samba?
The stornext share has Windows 2012 and windows 2008 hosts. Windows 2012 should work every version of samba out there, as far as I know.
I’m wondering why Apple blocked the installation of XSAN 3.1 clients onto 10.10?
I loved Xsan 4 up until this week. I had everything running just fine two weeks ago. My connection is iSCSI over dual 10GbE to a 45drives server running Centos 7. I was able to get it all set up so that the user home folders were based on the server, and I had a backup on the primary MDC. Everyone was able to log in, files were being transferred and checked out with no issues, and I had data transfer speeds nearing 700MB/s. I went on vacation for a week, and one user decided he would leave himself logged in for the entire week. Not sure if that had anything to do with it, but when I came back, the whole SAN was destroyed and my users and groups were gone. Now, my Xsan volume automatically mounts on my primary MDC, but Xsan in the Server.app shows no volumes. When I try to create a volume, it gives me a “cannot create nil from object” error. Also, none of my clients can mount the volume, which I assume is because the MDC doesn’t see the volume in Server.app.
I have it set up as a primary OD controller for my 10GbE network, and DNS primary as well. I’m thinking I may have to cut off any other networks, and maybe use the Centos server as a proxy to the rest of the world. I know iSCSI isn’t the best way in the world to connect, but until this week, it worked just fine. Any thoughts?
Any news on what’s happened to your setup? What does cvadmin shows (vs. what Server.app/Xsan admin shows)?
I ended up rebuilding the whole thing. Fortunately, I didn’t lose any data, as I was able to back it up onto an external RAID. However, it took nearly a week to get it going again.
Personally, I think that my server was trying to receive DNS information from our corporate domain controllers and name servers, and it erased the DNS records inside my server. I’ve since removed it from my corporate network, but I have requested DNS records to be made on my corporate name servers that will point to my server by name. Since Server.app relies heavily on DNS, I’m crossing my fingers that this helps.
DNS is extremely important to Xsan, and if there was a mixup between corporate DNS servers and DNS info running on the Xsan controllers then bad things can happen. I think getting your proper DNS entries in the corporate domain controllers is the best way forward. When I set up Xsan on someone else’s network I do what I can to get them to put the proper entries into corp DNS. When I run the Xsan and the network then I set up my own DNS servers, as needed. Good luck.
I’m interested to know more about your iSCSI – Xsan setup. Have it written up anywhere? I know it’s possible but my iSCSI SAN setups have all been limited to DDP storage servers from Ardis; my Xsan stuff is a lot different.
I haven’t done any write-up on it, but it was pretty simple.
I’m using a 45Drives Storinator running Centos 7 as my server. I have three RAID 5 sets of 14 drives and one hot spare each, and then I put the three RAID 5 sets into a RAID 0 configuration. The drive is formatted to XFS+. Then, I created two iSCSI target images, one at 65TB, and one at 1TB. The server also has two 10GbE ports that I bonded.
The switch I’m using is the new Netgear XS712T, which supports link aggregation.
I have three of the new black Mac Pros running Yosemite. Each Mac Pro has a Promise Sanlink 2 attached, which allows Thunderbolt 2 to dual 10GbE and supports link aggregation. As long as the aggregation is set on both the computer and the switch, it connects easily.
Each of the Mac Pros has GlobalSan installed for the iSCSI initiator. I tried a few others, but GlobalSan has proven to be very reliable.
The main Server.app is running on one of the Mac Pros. Once I connected the iSCSI target, Finder wanted to format it, but I ignored it and had Server.app format it in the XSan pane. With the 65TB target set for video, and the 1TB target set for metadata and journaling, XSan picked it up right away and started working with it.
I then setup my users, and used your suggestions to configure the clients. I had trouble with the authconfig file on each of the other Macs, but once I had them set, they were good.
My second and third Mac Pro computers are on my corporate network with the 1GbE ports, but the 10GbE ports are set specifically for the XSan network. Until recently, my Server.app Mac Pro has been solely on my XSan network, and my 1GbE ports have been turned off. I also have a 12TB RAID 5 drive connected to the Server.app Mac Pro for backup. I have 65TB on my iSCSI, but I’ve only used 7TB so far, and that’s all stuff from the last two or three years. I try to keep things clean.
Performance has been great. I can play a 1080i ProRes 422 file from one machine while recording it on another in realtime 1080i ProRes 422. I also have the home folders hosted on my XSan drive so that when a user logs in to a different machine, they have all the same settings.
Overall, this has been very fun to set up, and I’ve learned a lot. I admit I perused your blog a lot while setting this up. I think most of what I learned came from here.