Root Me Baby One More Time!

UPDATE: Apple has posted a security update. 2017-001

Root-a-pocalyse. Root down. Root a toot toot. Many funny tweets today about a very serious issue. A bug was discovered in macOS 10.13 that enabled anyone to login with a root account. With no password. Wow. Seriously. Yeah, that’s bad.

Bug discovered by Lemi Orhan Ergin.

I tested by clicking on the lock icon in System Preferences. Normally this requires an admin account. I was able to authenticate with “root” and no password. This actually also set root to no password. You can choose a password here and this makes it for you. How convenient. You can also login to the Mac via the login window. With root. And no password. Crazy.

If your Mac is off it’s safe. Not joking. If your FileVault protected drive is encrypted and your mac is turned off then you’re good. If you Mac is turned on and you’ve logged in at least once (or at least decrypted the drive on boot) then you’re not safe.

What can you do? Change the root password and set the shell to false. Until Apple fixes this. Should be anytime now. Or soon.

dscl . -passwd /Users/root “random or very secure password here”

dscl . -create /Users/root UserShell /usr/bin/false

Read a comprehensive explanation on Rich Trouton’s site:  Der Flounder blog

 

I don’t get High — Sierra!

Friends don’t let friends install macOS High Sierra in production. Don’t get High, Sierra.

macOS 10.13 was released on Sep 25, 2017, and almost two months later with only one point release update, it’s still too new for production. Download it on a test machine or two or more, test it with your apps and systems, file bug reports and radars, but for the love of all that is Python and Monty! don’t run it on your production Xsan. Well, at least not yet. Wait until next year. Or as long as you can. Or until the new iMac Pro is released with 10.13 pre-installed or wait until they ship the new Final Cut Pro X 10.4 that may or may not require macOS High Sierra.

With that out of the way, I’ve just upgraded the production Xsan to … macOS Sierra. Yes, macOS 10.12.6 is stable and it’s a good time to install last year’s macOS release. Time to say good bye to macOS el Capitan 10.11.6, we hardly knew ya. Besides guaranteed security updates, stability and the annoying newness of a changed macOS, what else is there? In Xsan v5 they introduced a new “ignore permissions” checkbox for your Xsan volumes. Looking forward to that feature in production. No more Munki onDemand nopkg scripts to run chmod. No more tech support requests for folders, files, FCP X projects that won’t open because someone else used it, owns it, touched it. We’ll see how that pans out. I’ll let you know.

Upgrading Xsan to v5

Step 1. Back up your data

You’re doing this, right? I’m using Archiware P5 Backup to backup the current projects to LTO tape. I’m using Archiware P5 sync to sync the current Xsan volumes to Thunderbolt RAIDs, and using Archiware P5 Archive (and Archive app) to archive completed projects to the LTO project archive. That’s all I need to do, right?

Step 2. Back up your servers

Don’t forget the servers running your SAN! I use Apple’s Time Machine to backup my Mac Mini Xsan controllers. External USB3 drive. I also use another Mac Mini in target disk mode with Carbon Copy Cloner to clone the server nightly. (Hat tip to Alex Narvey, a real Canadian hero). And of course I grab the Xsan config with hdiutil and all the logs with cvgather. Because, why not?! For Archiware P5 backup server I also have a python scripts to backup everything, another scripts to export a readable list of tapes, and BackupMinder to rotate the backups. Add some rsync scripts and you’re golden.

 

Step 3. Upgrade the OS

Unmount the Xsan volume on your clients or shut them down, disconnect the fibre channel. Do something like that. Stop your volume. Download the macOS Sierra installer from the App Store. Double click upgrade. Wait. Or use Munki. I loaded in the macOS 10.12.6 installer app into Munki and set it up as an optional install to make this portion of the upgrade much quicker and cleaner.

In my case after the OS was upgraded I checked the App Store app for any Apple updates (you can also use Munki’s Managed Software Center to check) and of course there were some security updates. In this case the security upgrade hung on a slow network connection and the server crashed. Server down! I had to restore from Time Machine backup to the point where I just upgraded the server. It took some extra time  but it worked (can’t wait for next year’s mature APFS / Time Machine and restoring from snapshots instead).

Step 4. Upgrade Server

After macOS is upgraded you’ll need to upgrade the Server.app or just upgrade the services used by Server (even those not used by Server get upgraded).

Step 5. Upgrade the Xsan

Bur first we have to restore the Xsan config. Don’t panic! It may invoke bad memories of data loss and restoring from backups. Xsan PTSD is real.

Restore-previous-Xsan.png

Step 6. Upgrade the rest

Next you have to upgrade the Xsan volumes.

Xsan-volume-needs-upgrade

New version of Xsan, ch-ch-changes! Ignore permissions check box will remount the xsan with the “no-owners” flag. Let’s test this out.

 

Upgrade the OS and Server app on the backup controller. Upgrade the OS on the clients using Munki or App Store if you like doing it the hard way. Ha Ha.

Step 7. Enjoy

Plug those Thunderbolt to Fibre adapters back in, mount those Xsan volumes and be happy.

Step 8. Wait for the complaints

The next day the editors walked in and went straight to work with Final Cut Pro X. No one noticed anything. Xsan upgraded. Workstation macOS upgraded. Everything appeared to be the same and just worked. Thankless task but well worth it.

 

Reference: Apple’s iBook guide here

 

 

Be a NoMAD!

 

NoMAD stands for “no more AD” and has nothing to do with a nomadic lifestyle, nomads, ronin or other wandering IT professionals. Sorry.

NoMAD allows you to stop binding Macs to a corporate domain and instead get your kerberos tickets as needed. Connect to those file shares, change your password, and other fun tasks, without being stuck on the domain and constantly resetting your keychain from the insanity of password retention policies.

NoMAD-intro

Using Autopkg and Autopkgr to feed trusted apps into your Munki repo you can easily deploy NoMAD to your fleet of Macs.

And for bonus points you can add your preference settings as “updates for” NoMAD in Munki. One such add on is a setting for an auto mounting sharepoint.

Name your file: “menu.nomad.shares.plist” and open up your favourite text editor.

<?xml version=”1.0″ encoding=”UTF-8″?>
<!DOCTYPE plist PUBLIC “-//Apple//DTD PLIST 1.0//EN” “http://www.apple.com/DTDs/PropertyList-1.0.dtd”&gt;
<plist version=”1.0″>
<dict>
<key>Shares</key>
<array>
<dict>
<key>AutoMount</key>
<true/>
<key>ConnectedOnly</key>
<true/>
<key>Groups</key>
<array/>
<key>LocalMount</key>
<string></string>
<key>Name</key>
<string>Corp_Share</string>
<key>Options</key>
<array/>
<key>URL</key>
<string>smb://winserver5000/Corp_Share</string>
</dict>
</array>
<key>Version</key>
<string>1</string>
</dict>
</plist>

Create a package with munkipkg and add this to Munki. Set the package as an update for Munki and as your NoMAD agent gets installed your updates for NoMAD go with it.

More tips and tricks in the future.

 

MDOYVR 2018

MacDevOps:YVR 2018 tickets are on sale now. Buy one for everyone in your MacAdmin family.

Seems like just the other day we were hanging out with our friends who came from all over the world to talk Open Source and macOS management, and now we can do it all again!

Tickets are on sale now.

MacDevOps:YVR is the place for Mac Admins interested in integrating DevOps into their IT practise. Developers and IT (Ops) working together to build a better world.

Join us at MacDevOps:YVR 2018, our annual conference, for two days of learning and networking in Vancouver, BC, Canada. With speakers from a diverse group of companies, this year’s conference will be the best place to talk about Open Source projects that matter to the community. Learn from your peers, and connect with fellow Mac Admins.

We will be discussing: munki, imagr, autopkg, chef, puppet and all your favourite Open Source projects. This year we will be discussing MDM and all the changes in macOS. We’re planning another hack night because it was so much fun last year, and if you are interested in a particular workshop topic let us know.

Learn more at https://mdoyvr.com

And because we’re always learning from every conference we’ve organized we’re trying something different this year: tiered pricing for tickets. We want everyone to join us and we want to make it fair for independents, students and others who want to be there. At the same time we want to pay the bills and support a diverse group of speakers and attendees who might not be able to attend due to lack of funds.

We’ve created three tickets: corporate (if your work is paying), independent (if you’re buying you’re own ticket), and education (students and those who work in schools). Last, but not least, the Donation ticket is for those who want to contribute to our financial aid fund. Help those who want to speak and/or attend but need some help.

Ticket sales: https://www.eventbrite.com/e/mdoyvr2018-tickets-38821491125

Setting up Secure Munki

So you’ve set up Munki to deploy software to your Macs by following the basic set up here: Set up Munki, and now you want to set it up more securely.

You need two things. 1) a cert and 2) a secure repo

  • TRUST US

The optimal situation is a trusted secure certificate for your server from a reputable certificate authority, if you don’t have that, or want to use the self-signed certificate your server has then your Munki Mac clients will need to trust this certificate.

Export out the cert from Server Admin if you’re using that to manage your Mac mini server. Place this cert file on your clients (using ARD, or other methods) then use the security command to get the Mac clients to trust this cert.

security add-trusted-cert -d -r trustRoot -k “/Library/Keychains/System.keychain” “/private/tmp/name-of-server.cer”

REFERENCE: Rich Trouton’s blog goes into more detail and details a way to script this.

  •  SECURE IT

Use htpasswd to add a password to your Munki repo.

htpasswd -c .htpasswd munki

Edit the htaccess info

AuthType Basic
AuthName "Munki Repository"
AuthUserFile /path/to/your/munki/repo_root/.htpasswd
Require valid-user

Encode this password for Munki:

python -c 'import base64; print "Authorization: Basic %s" % base64.b64encode("USERNAME:PASSWORD")'
Authorization: Basic VVNFUk5BTUU6UEFTU1dPUkQ=

Push out this password to your Munki clients with ARD (or use some other method)

defaults write /Library/Preferences/ManagedInstalls.plist AdditionalHttpHeaders -array “Authorization: VVNFUk5BTUU6UEFTU1dPUkQ=”

Change the Munki RepoURL on all your clients to use the new secure URL

defaults write /Library/Preferences/ManagedInstalls SoftwareRepoURL “https://munkiserver/munki_repo&#8221;

REFERENCES:

Consult the Munki Wiki for: Basic authentication setup for Munki 

Ala Siu’s excellent write on securing munki

Notes:

Consider using a server made for securing Munki, like the Squirrel server from the MicroMDM project. More on this in another blog post.

Consider using certificate from a known reputable certificate authority such as Let’s Encrypt (the Squirrel server above automates the setup with Let’s Encrypt).

Further:

Another project which seeks to combine all these open source projects in the Munki ecosystem is Munki in a Box. There’s a secure branch of this project which setups a basic authentication as well but while it aims to simplify setting up a secure Munki it may be a bit confusing to set up at first glance. Test, and test again.

 

 

My Thunderbolt Nightmare

It was a dark and stormy night of cables and capacitors when suddenly I heard the door knocking, or was something falling of a shelf? I was in a cramped server room, if you’d call it that, and I was day dreaming, sorry, night terrorizing, of days gone past when I worked in nice big well ventilated server rooms with proper enterprise gear. Oh wait, did I really dream that? Did it really happen? Maybe it was less well ventilated and there were cables strewn about the tall 42U shelves and sometimes we found a Mac hidden underneath spaghetti. Sometimes. I vaguely remember the long shiny metal servers, they talked to me, they sang, a whiny pitch of whale song. Dream on, dream on.

Now. Today. Apple Music on my iPhone plays every single Arcade Fire album in a long playlist, in order. And I follow the white rabbit of Thunderbolt cables. This is my thunderbolt nightmare. Dead drive in a Thunderbolt Promise Pegasus unit, web ticket filed for registered hardware. Legacy. That’s the word they used. Where’s Marshall McLuhan when you need a proper redux of the shit storm you’re in? Thunderbolt 3 uses USB-C and everything is possible. Can’t wait to step into that confusing identity crisis. OK, back to the present day when I stared at the red blinking drive, a replacement drive from not long ago dead again, sitting in the last row of a now legacy Pegasus R6 unit. RAID 5, the most dangerous kind, this is what stood between me and uncertainty. The worst kind of RAID. Well, not as bad RAID 0. Raid nothing. Raid 5 is one bad drive away from a bad day. Backups? Hmm, I got those, I got plenty of those, but I don’t want to be tested today. No, not today. Not this bloody day.

I open the Pegasus utility and the GUI wants an update. Hmm, that’s not in autopkg, I think. Why is out of date? Munki let me down. I start to drift, to side shift into adding newer better recipes to autopkg, to tweaking my Munki repo, to what sessions would be awesome at the next MacDevOps:YVR conference. Gee whiz, I love open source, and everyone in the Mac Admins community…. Snap out of! I slap myself in the face. I was hallucinating. Stay on task. I update the Pegasus utility. I stare at the critical reports from one of the three R6 units attached to this Mac Mini server. Did I say server? But it’s so small, so little. It works. It’s magical, kinda neat. Until you stare too close at the back. The Thunderbolt cables go from the Mac Mini to the first Pegasus unit to the SANlink fibre channel adapter to the LTO 6 tape library to the next Pegasus utility to the second SANlink adapter to a third and final Pegasus RAID unit. What’s is going on? Where does this cable go? Let me just follow it to the next jumping off point. My brain slows, the lack of oxygen in this cold machine room start to affect my thinking. I lose my way.

I download the report for the Pegasus unit. I had to unlock a pretty neat lock icon and click on the save report. I upload it to the web support and add it to the ticket. Tech support gets backs to me in a day and said all is good, and to carry on. I can’t. The drive is dead. What are they not seeing? It’s right in front of me. I download the report again. Again the same response. Fine. It’s time to stop messing around and pop open Terminal. Loading up promiseutil I check out the options and switches and get into an argument with myself about the currently valid optionals of letters and numbers that are required. I check my notes, online knowledge base, and try again. It’s broken. It doesn’t work. Stumbling around the command line typing imprecisely incorrect statements gets nowhere fast. I realize that there’s no way for the cli utility to properly change its focus to the broken unit with the busted drive. Both the GUI and the binary are stuck on the one R6 unit and won’t see what’s in front of my face.

I call tech support. This is humiliating. This was supposed to be easy. Drive dead, drive reported, drive ordered, drive replaced, then no one the wiser. Data saved, not dead. Backups not tested. Not today. No, not today. Tech support treats me like the imaginary newbie IT people sometimes treat everyone with. He repeats his instructions to me. He is polite. Download the report. I can’t. It won’t work. Unplug the unit. Plug it into something else. I can’t. The cables. The Thunderbolt cables are everywhere. It’s magical, and daisy-chained, and stuck. “Can I remote in and see?” he asks, hoping to resolve this quickly. Sorry. That’s impossible. Even if I thought it was a good idea. I remind him that I have a dead drive. That’s why I called. I want to get a replacement drive. “Sorry sir that legacy unit is not under support most likely,” I know that. I realize that now. That I wasted my time. It happens sometimes. The truth is staring at you. You need a mirror to see. “You need to order compatible drive from the compatibility list.” I am a well spring of emotions. I thank him. I am nice. He was polite. But now I know what I need to do. Oh wait, what? Order a drive now! Order two.

McLuhan never had a chance to evaluate Thunderbolt storage technology but the insane genius and simplicity of Thunderbolt reduced expensive enterprise fibre channel storage to the dust bin. Magical SAN for video editing with a Mac Mini and Thunderbolt RAIDs. Cheap enough to buy with a departmental credit card, fewer meetings to attend, more films to shoot and edit. Backup, archive, repeat. McLuhan would have no doubt reminded me that the tetrad of technology would have flipped Thunderbolt on its head, Fibre channel never went into a dust bin, but was firmly relegated to well cooled storage room, and long ago legacy drives in the enterprise units are humming a long while the cursing wind and emotions swell over the Thunderbolt mountain. Fibre channel just became cool again. Retro smart.

Screen recording and other tricks

QuickTime has a neat little trick that some may not know about, it can record your screen.

QT-ScreenRecord

Use it to record a how-to video how to navigate System Preferences, or how to use Final Cut Pro, or record a MacDevOps:YVR talk.

The first two MacDevOps:YVR conferences needed to be converted to a suitable format for YouTube and using QuickTime screen recording + Soundflower is the way I chose to do it.

Note: Soundflower is needed to redirect the audio to QuickTime. Screenrecording with QuickTime does not capture the audio without Soundflower.

MacDevOps Screen recording steps

  1. Install Soundflower (Soundflower-2.0b2.dmg)

https://github.com/mattingalls/Soundflower/releases/tag/2.0b2

  1. Set audio output to SoundFlower 2chSoundFlower
  2. Set QT screen recording to Soundflower audio QT-ScreenRecord-SoundFlower-2ch
  3. Start screen recording (select screen area)
  4. Play website audio / video (Safari / Other )
  5. Stop both. Edit and trim QT video as needed
  6. Upload to YouTube
  7. Tag video (mdoyvr, yvr, MacDevOps, MacAdmin, MacIT), put in proper playlist
  8. Publish

Addendum:

I own Rogue Amoeba’s excellent Audio Hijack application and have used this app for audio capture (podcast interviews, etc), but I couldn’t get it to work in this case. It might have also required their Rogue Amoeba’s Loopback app which I did not own. Since I’ve used Soundflower previously I used it here in this case.