Thunderbolt Storage in the field: Part 1 – The Gamma Carry test

I’m testing the new Accusys Gamma Carry thunderbolt storage in a series of blog posts with real life situations. Filming on location and editing with a remote team requires a combination of good workflow, great apps, and excellent storage. The Gamma Carry is small sturdy box meant for on set and on location editing and camera off loading. It also survives going back and forth between your office and your home office, and wherever you need to go. Disclaimer: I was not paid to write this review and this blog post is an ongoing field test with storage required by every day editors making films using local thunderbolt storage and the cloud. This is their story.

In the beginning….

The start of every creative editorial project is choosing the tools needed for the job, and working as team to make it happen. Making a film requires the hard work of everyone, not the least the IT / Tech who supports the crew and prepares the gear. On set the DIT (digital imaging technician) copies camera cards to multiple hard drives and backup devices before going back to the office to copy them to LTO (archive tape storage), but before the DIT can do their job the IT / Tech has to set up the RAID and design the best and safest backup and archive workflow.

Gamma Carry Overview

The Gamma Carry is an 8 drive external RAID box and it really small and sturdily built. There is a solid metal handle built into the top case and the drives carriers where you put the drives are metal and very solid. This entire unit is built to be solid and protect the drives it contains. The drive tray have a pin to lock the track in place when inserted as well as a thumb screw to lock them in. All drives must be attached with provided screws which is not as convenient as some others with quick plastic tray mounts, but this metal cage for your drive is solid and feels safe for a RAID that will probably be transported everywhere. Keep the data safe!

There are other little touches which are well thought out. The blue red green blinky lights are perfect for any Christmas holiday party and tell you important information about the state of your RAID at a glance but you with one button push turn off all the blinky lights. And keep working in the dark editorial suite not bothered by anything blinking. There’s a mute button and an extra port on the back with 60W to charge your laptop via the RAID. Very useful.

Gamma Carry The Setup

The set up of the Gamma Carry software wise is identical to the Accusys A12 T-Share which I set up recently and once you have the Accusys Mac installer then you have the RaidGuardX app. And the usual IT caveats apply, the software is not signed so you will have to right click on it to open it up. Hopefully they will sign this and notarize their software to make this easier for end users. Also there is a requirement for JRE to run RaidGuardX which means downloading and installed Java on your Mac. Also not optimal but only necessary for the RAID setup.

RaifGuardX detects the Gamma Carry

The set up of the Gamma Carry is the same as the Accusys A12 T-Share and a raid array you built with RaidGuardX will be recognized in the Gamma Carry. That could be a good thing, or not. I did have some excitement when it recognized the array I had built previously (I used the same drives form the Xsan setup). And this was the next step to resolve, because tp create a new array, or delete the array then create a new one I have to do one more thing. Since they were used with an Xsan I had a LUN label that identified them as such and had to remove this LUN label before proceeding. Occasionally we see this issue when re-using drives that had once made up a RAID which was part of an Xsan.

Delete the existing RAID array…. button grayed out.

Xsan to the resuce.

To see the RAID arrays avaialable when building an Xsan you can use the cvlabel command to list them. You can also use it to remove this label. WARNING: Do not do this when connect to an Xsan or Stornext storage network. Unless you know what you are doing. You are warned. This is dangerous. Removing a LUN label can bring down the entire SAN. That’s it. Now you know.

sudo cvlabel -l                                      

/dev/rdisk3 [ACCUSYS Gamma Carry      366] acfs-EFI "accusys"Sectors: 46881814495. Sector Size: 512.  Maximum sectors: 46881814495. Stripebreadth: 0.


sudo cvlabel -u "accusys"           

*WARNING* This program will remove the volume label from the
          device specified (accusys).

          After execution, the devices will not be usable by the
          Xsan. You will have to relabel the
          device to use it on the Xsan.


Do you want to proceed? (Y / N) -> y
Requesting disk rescan .% 
                                                                                     sudo cvlabel -l                          

/dev/rdisk3 [ACCUSYS Gamma Carry      366] unknown  Sectors: 46881814495. Sector Size: 512.  Stripebreadth: 0.

So all is good again now we can create new RAID arrays now that the Xsan LUN label was removed. Back to work! Once the drive is set up in Disk Utility as a new volume then you’re ready to go. In this case I added one more drive and created a 5 drive RAID5 set and formatted as HFS+. In my testing this was fast enough and would be faster if all filled with drives or SSDs. There are variations of this hardware with SSD ports instead of 1 or 2 drive bays to allow quicker ingest of SSDs which have camera footage on them.

Speed test of a 5 drive RAID5. Fast enough!

I then set up 48 hours of drive copies via Hedge for some testing of the RAID hardware. Thanks to Hedge Connect I get notified when the large copy finished. In this I was copying Thunderbolt 2 and 3 external hard drives (thunderbolt but hard drive and not that fast) to the Gamma Carry to be copied. The source drives are slow. Copying from SSDs would be way faster.

And of course Hedge found some minor warnings with these old drives. People don’t like LTO archival tape and want to have stacks of hard drives, but that data on those hard drives won’t last forever. Keep your important data in three places on two different kinds of media (tape, cloud, drives etc). This test of mine was to copy off old drives and use the Gamma to re-organize and re-sort for a new edit project. Now to back up to the cloud and set up Postlab projects with postlab drive for the proxies. In the mean time the editors can get the original footage when I carry over the Gamma Carry thunderbolt RAID.

This is part 1 and after some more testing I will publish some real world tests and experiences.

Thunderbolt Xsan: Set up a T-SAN

Setting up your very own Xsan at home… What could be more exciting? Nothing like SAN storage to cure those stacks of hard drive blues. Don’t have a spare fibre channel switch or fibre channel storage at home? No problem Grab some thunderbolt storage from Accusys and join the fun.

I am testing the A12T3-Share 12-drive desktop Thunderbolt RAID solution to build my Xsan. Accusys also have a 16 drive rack mounted raid storage box if you want to install a nice pro set up in the server room you have tucked neatly in your home office. Ha ha. Seriously, the 12 drive unit is whisper quiet and would be a great addition to any home lab or production storage setup. I mean, aren’t we all doing video production at home these days? And even if we are doing a proxy workflow in the clouds, we still need to store the original footage somewhere before it goes to LTO tape, or backed up in the clouds (hopefully another cloud). A few years ago I tested the Accusys 16 drive Thunderbolt 2 unit and it worked perfectly with my fibre channel storage but this time I am testing the newest Thunderbolt 3 unit. Home office test lab is GO!

It is a pretty straight forward setup but I ran into some minor issues that anyone could run into and so I want to mention them and save you all the frustration by learning from my mistakes. Always be learning. That’s my motto. Or “break things at home not in production”, but if your home is production now, then break things fast and learn very quickly.

First step is to download the software for the RAID and you’ll find it on the Accusys website.

(I found the support downloads well organized but still a bit confusing as to what i needed)

The installer is not signed which in our security conscious age is a little concerning, but examining the package with Suspicious package should allay any concerns.

The installer installs the RAIDGuard X app which you will need to configure the RAID.

Of course, RAIDGuard X needs a Java Runtime Environment to run. Why is this still a thing? Hmm…

RAIDGuardX will allow you to configure your connected Thunderbolt hardware.

Configure the array as you like. I only had four drives to test with. Just enough for RAID5.

Choose your favourite RAID level. I picked RAID5 for my 4 drives.

The first gotcha that got me was this surprisingly simple and easy to overlook section. “Assign LUN automatically” asks you to choose which port that LUN (the configured RAID) will be assigned to. If you don’t check anything like I didn’t in my first run through then you configure a RAID5 array that you’ll never see on your connected Mac. Fun, right? Ha ha.

Xsan requires a sacrifice…. I mean, a LUN (available RAID array). Check your Fibre Channel in System Information. Yes, this is from the thunderbolt storage. Hard to believe, but it’s true!

Setting up enterprise grade SAN storage requires a trip to the Mac App Store. Server.app

Open Server.app, enable Xsan, create a new volume and add your LUN from the Accusys Thunderbolt array. Set the usage to “any” (metadata and data) since this is a one LUN test setup.

Pro tip: connect your Xsan controller to your Open Directory server. Ok, just kidding. You don’t have an OD server in your home office? Hmm… Create an entry in /etc/hosts instead.

If you’ve set up your SAN volume then you will see it listed in the Finder.

Easy shareable SAN storage is possible with thunderbolt RAID arrays from Accusys. No more Fibre channel switches needed. Small SAN setups are possible for creative teams without a server room. This setup was a quiet 12 drive RAID and a Mac mini. Add some Thunderbolt cables. There are four thunderbolt 3 connections and you can add more with an additional RAID. Up to 8 connections with one of them for the Mac Mini running the SAN. Not bad at all. And Xsan is free. Add a Server app from the App Store, but the Xsan client is free and built-in (Xsan has been included with macOS since 10.7 so many years ago). Fibre channel protocol (even through Thunderbolt) is faster than network protocols and great for video production. Fast and shareable storage at home. Or in your office. Thunderbolt Xsan. T-SAN.

From Camera to the Clouds: the very real story of Hedge and Postlab

Note: I want to explain how our current workflow for editing remotely. I am always testing new tools and methods, so workflows change all the time. This is a snapshot in time of what we are trying now. So far it works.

Hedge

We use Hedge to copy camera cards to multiple drives on set (or after a shoot if on location) and then we use Hedge once more to copy one of these drives to the office shared storage (Apple’s Xsan).

Why use Hedge? A nice simple app which hides its complexity well. Hedge has an easy interface to copy multiple sources (camera cards, usually) to multiple destinations (two external drives, or two SAN locations etc), and it does it well. It verifies, and double checks its work and leaves receipts. What was copied when. This is very nice and very useful for troubleshooting. It also has an API which made it easy to build an app that configures Hedge for its current task, and AppleScript support for extending automations after specified actions.

Kyno and Postlab

We are using two other tools in our remote ingest workflow currently: Kyno from Lesspain software for rewrapping and converting camera footage and Postlab, the remote collaboration tool for Final Cut Pro (and Premiere Pro). Testing with other tools is always ongoing and during a recent test of the workflow we also tried EditReady from Divergent Media.

The Workflow (so far)

While we are exploring various workflow automations we are currently doing the following steps manually.

  1. Hedge to copy camera cards two external drives on set, and then Hedge copy the drive to Xsan
  2. Making re-wrapped in MOV files from the original camera MXF files using Kyno and then
  3. Making H264 MOV 4K proxies in Kyno
  4. Uploading finished proxies to Postlab drive using Hedge
  5. Set up FCPX production and new FCPX library from template connected to proxies in Postlab drive

Hedge and Postlab

Hedge is super useful. Two times good. Hedge and Postlab are best friends. And the UI on both shows the simple aesthetic shared by the developers. Three panes. Source / Start to Destination / Projects. Whether you are copying Proxies to Postlab Drive or accessing your editing projects in Postlab the apps will guide you through.

Copying the Proxies with Hedge to Postlab Drive.

Details. Rewrap and Proxies

Workflows will depend on your goals, and your available tools. In this case we are using a Canon camera and ingesting MXF files. In order to edit with small Proxies in FCPX but also be able relink to original (and larger) files easily we need to in our case re-wrap the original MXF to QuickTime MOV container.

Right click on a clip in Kyno to rewrap to Mov.

Originals. Not Proxies.

And to be clear we are treating these in FCPX as “new” originals not as actual FCPX proxies. With the rewrapped MOV files we make transcoded H264 files which are swapped 1 for 1 with the original. When we need to export a final 4K version we can relink to the original 4K source and export easily.

Proxies. Not originals

The transocded H264 4K proxies we made in Kyno were 15x smaller than the original re-wrapped Mov files. We had almost 600GB in originals and 37GB for the 4K H264 proxies!!

Postlab Pro Tips

Working with Postlab pro tip #1 –> keep those FCPX libraries light. Keep all media and cache files out of the library. We knew that and we had Storage Locations set to outside of the library but one new issue came up when the libraries grew really big and we realized the editors were making multiple sequences, not backups, but versions. Now we are trying to work around this habit with Postlab itself. You can check in a version of the library and duplicate library for an alternate version. Modifications of old habits are always tough but technical reasons may force a change in habits here. We will see. Postlab pro tip #2 –> Keep your cache large and fast. By default the Postlab cache is your local drive and only 20GB. If you have a fast SSD or an external drive then move that cache and increase the size. It will help. Trust me.

Kyno vs EditReady

Another small issue we encountered in testing was that we could make the rewrapped Mov files in Kyno or in EditReady and both were fine. The only objection the editors had was that in Kyno we could keep the folder structure of the original camera cards and they felt that this lent some confidence to being able to track the files to the camera card folders if any media was missing or misplaced. The EditReady files kept the original names but they were all in one folder. As the tech I see no issue with FCPX handling these files since we’d be ingesting all the finished proxy files and all the files were named by the camera. Editors should be able to tell which reel the clips were from by the clip name and that’s all you need technically, but you can’t win every argument with an editor. As the tech you need to test alternative tools and methods and see what works technically but also see what can be accepted to work in the way the editors want to work. Changes to workflow are some of the hardest to make, making a system that is used, actually used, by the editors is the goal.

Errata

Errors. If you get them, how do you know? This was one area where I could comment on both Kyno and EditReady. I am spoiled by Hedge and it’s nice reports when it is done copying. And Postlab which has a Help menu :collect logs for support button, very nice. If your software tool is going to process a lot of files (rewrapping then transcoding) I want to know if there were errors. EditReady popped up a window to what had succeeded or failed and Divergent Media support told me to look in the logs for any issues encountered. Not great. While Kyno has a separate jobs window which shows jobs done or failed. But still no report. I would like a receipt or report or log at the end with files converted or failed to convert. It would help troubleshooting any issues when they arise. Tech support for both companies is great and responsive. Thanks again. And I’ll keep sending in feature requests.

Testing. More Testing. And Teamwork.

We are testing this workflow in production with a real project and getting feedback from the team. So far the proxies have proven to be easy to make, quick to upload to Postlab drive, simple to use in FCPX in Postlab. Assembling the cut and editing are going well. We will find out about the colour process when we get to that stage and relink to the originals. Stay tuned.

Thanks!

Thanks to Felipe Baez / cr8ivebeast for his assistance on this part of the workflow. We were having trouble relinking to the original MXF and he gave us the excellent tip to rewrap then in Kyno then make the smaller proxies. Works like a charm. Thank you Felipe! Here’s a link to a video Felipe made showing a similar procedure using Compressor to transcode and then relink in FCPX and it goes to show you that there are lot of ways to do things and to keep trying, and experimenting. You might learn a thing or two.

Minecraft Server for My Kids and My Sanity

Summer time or anytime is a good time to run a minecraft server. And when I am not troubleshooting IT networks, planning SAN storage upgrades, running a DevOps for Dummies bookclub and the MDOYVR podcast then I like to upgrade my minecraft server.

Every time there is an update to the java client there is demand from my users (uh, I mean, my kids) to immediately stop all other work (hey kids, I’m working here! let Dad work) and upgrade the minecraft server.

Like all other IT domains where there are variety of solutions and software fixes to problems, it would seem that Minecraft has official server downloads as well as the unofficial artisanal craft versions. I’ve tried a few, and some out of desperation… there was an incident with netherite blocks and the server wouldn’t start anymore but the Ppaer minecraft server fixed the issues!

The normal routine is that when an official release comes out the other versions may not be up to date as quick, so it’s back to the official versions.

Download the official Minecraft Server

Or try the Paper Minecraft Server

See also Michael Lynn’s two part family harmony blog series which started me on this road to keep the kids happy and maintain family happiness.

Xsan Upgrade and Big Sur Prep. Hello Catalina!

Big Sur summer testing time!

Summer time is beta testing time. A new macOS beta cycle with Big Sur is upon us. Test early, and test often. With all the excitement of Big Sur in the air, it’s time to look at Catalina.

Our day to day production Xsan systems do not run beta software, not even the latest version of macOS, they only run tested and safe versions of macOS. I always recommend being a revision behind the latest. Until now that meant macOS 10.14 (Mojave). With the imminent release of macOS Big Sur (is it 10.16 or macOS 11?) then it’s time to move from 10.14.6 Mojave to 10.15.6 Catalina. It must be safe now, right? 

Background

Xsan is Apple’s based Storage Area Network (SAN) software licensed from Quantum (see StorNext), and since macOS 10.7 aka Lion it has been included with macOS for free (it was $1,000 per client previously!).

Ethernet vs Fibre Channel vs Thunderbolt

A SAN is not the same as a NAS (Network attached storage) or DAS (direct attached storage). A NAS or other network based storage is often 10GbE and can be quite fast and capable. I will often use Synology NAS with 10GbE for a nearline archive (a second copy of tape archive) but can also use it as a primary storage with enough cache. Lumaforge’s Jellyfish is another example of network based storage.

Xsan storage is usually fibre channel based and even old 4GB storage is fast because … fibre channel protocol (FCP) is fast and the data frames are sent in order unlike TCP. It is more common to see 8GB or 16Gb fibre channel storage these days (though 32GB is starting to appear). And while fibre channel is typically what you use for Xsan you can also use shared Thunderbolt based storage like the Accusys A16T3-Share. I have tested a Thunderbolt 2 version of this hardware with Xsan and it works very well. I’m hoping to test a newer Thunderbolt 3 version soon. Stay tuned.

Xsan vs macOS Versions

We’ve discussed all the things that the Xsan is not and now what is it? Xsan is often created from multiple fibre channel RAID storage units but the data is entirely dependent on the Xsan controller that creates the volume. The Xsan controller is typically a Mac Mini but can be any Mac with Server.app (from Apple’s App Store). The existence of any defined Xsan volumes depends on the sanity of its SAN metadata controllers. If the SAN controllers die and the configuration files go with it then your data is gone.  POOF! I’ve always said that Xsan is a shared hallucination, and all the dreamers should dream the same dream. To make sure of this we always recommend running the same version of macOS on the Mac clients as well as the servers (the Xsan controllers). And while the Xsan controllers should be the same or at a higher macOS version level it can sometimes be the opposite in practise. To be sure what versions of macOS are interoperable we can check with Apple’s Xsan controllers and clients compatibility chart and Xsan versions included in macOS for the rules and exceptions. Check the included version of Xsan on your Mac with the cvversions command

File System Server:
  Server  Revision 5.3.1 Build 589[63493] Branch Head BuildId D
   Built for Darwin 17.0 x86_64
   Created on Sun Dec  1 19:58:57 PST 2019
   Built in /BuildRoot/Library/Caches/com.apple.xbs/Sources/XsanFS/XsanFS-613.50.3/buildinfo

This is from a Mac running macOS 10.13

Host OS Version:
 Darwin 17.7.0 Darwin Kernel Version 17.7.0: Sun Dec  1 19:19:56 PST 2019; root:xnu-4570.71.63~1/RELEASE_X86_64 x86_64

We see similar results from a newer build below:

File System Server:
  Server  Revision 5.3.1 Build 589[63493] Branch Head BuildId D
   Built for Darwin 19.0 x86_64
   Created on Sun Jul  5 02:42:52 PDT 2020
   Built in /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/XsanFS/XsanFS-630.120.1/buildinfo

This is from a Mac running macOS 10.15.

Host OS Version:
 Darwin 19.6.0 Darwin Kernel Version 19.6.0: Sun Jul  5 00:43:10 PDT 2020; root:xnu-6153.141.1~9/RELEASE_X86_64 x86_64

Which tells us that the same version of Xsan are included with macOS 10.13 and 10.15 (and indeed is the same from 10.12 to 10.15). So we have situations with Xsan controllers running 10.13 and clients running 10.14 are possible even though macOS versions are a mismatch, the Xsan versions are the same. There are other reasons for keeping things the macOS versions the same: troubleshooting, security, management tools, etc  To be safe check with Apple and other members of the Xsan community (on MacAdmins Slack).

Backups are important

Do not run Xsan or any kind of storage in production without backups. Do not do it. If your Xsan controllers die then your storage is gone. Early versions of Xsan (v1 especially) were unstable and the backups lesson can be a hard one to learn. All later versions of Xsan are much better but we still recommend backups if you like your data. Or your clients. (Clients are the people that make that data and pay your bills). I use Archiware P5 to make tape backups, tape archives, nearline copies as well as workstation backups. Archiware is a great company and P5 is a great product. It has saved my life (backups are boring, restores are awesome!).

P5-Restore-FCPX.png

Xsan Upgrade Preparation

When you upgrade macOS it will warn you that you have Server.app installed and you might have problems. After the macOS upgrade you’ll need to download and install a new version of Server.app. In my recent upgrades from macOS 10.13 to macOS 10.15 via 10.14 detour I started with Server.app 5.6, then install 5.8 and finally version 5.10.

After the macOS upgrade I would zip up the old Server.app application and put in place the new version which I had already downloaded elsewhere. Of course you get a warning about removing the Server app

 

Xsan-ServerApp-ZipRemovalDetected.png

Install the new Server app then really start your Xsan upgrade adventure.

Serverapp-setup.png

Restore your previous Xsan setup.

This slideshow requires JavaScript.

If everything goes well then you have Xsan setup and working on macOS 10.15.6 Catalina

Xsan-Catalina-Upgrade-Success

TCC troubleshooting

Download Howard Oakley’s Taccy app

Read Howard Oakley’s blog post on Catalina and privacy protection

Read Apple’s profile reference doc with respect to Privacy Preferences Policy Control payload

Read Rich Trouton’s guide to creating privacy pref policy profiles

This snippet (from MacAdmins slack) shows tcc in the logs if that is the issue:

log stream –debug –predicate ‘subsystem == “com.apple.TCC” AND eventMessage BEGINSWITH “AttributionChain”‘

Drop it! MunkiReport Db hacking

SQLite3 Db hacking for MunkiReport

Making modules for MunkiReport is easier than ever. Seriously.

please make:module

It’s easier than ever to make modules for MunkiReport (check out the recent MDOYVR MunkiReport workshop) and since the heavy lifting is done you can concentrate on the business logic (what makes sense) and the commands to execute or the scripts to run (python, shellI, etc). Worry about actionable data and less about the tables and views.

If you testing in production (which you should never do, always test is a testing environment) then you may happen to change a module (tables and fields etc) but keep the module name the same. This will confuse your database and you will need to erase it from the db to continue. In SQL speak this is “Drop table”. (You could also delete the munkireport Db and start again, but this is for those crazy enough to test in production and may want to keep the other data).

Sqlite commands for Munkireport

  1. Maintenance mode (current way)
sudo ./please down
Application is now in maintenance mode.

Old way –> sudo touch /Users/Shared/MunkiReport/munkireport-php/storage/framework/down

 2. Edit MunkiReport db

sudo /usr/bin/sqlite3 /path/to/MR/app/db/db.sqlite

 3. Exit maintenance mode (current way):

sudo ./please up
MunkiReport is now live.

Old way –> sudo rm /Users/Shared/MunkiReport/munkireport-php/storage/framework/down

Note: if you forget to get out of maintenance mode then clients can’t check in

“ERROR: Server error: MunkiReport is in maintenance mode, try again later.”

4. Migrate Db

sudo ./please migrate

Please use please migrate (or migrate db in web admin) if making changes to a module or else. That is, if you’re crazy to do this in production.

Server An error occurred while processing: \fancy_module_processor
Server Error: SQLSTATE[HY000]: General error: 1 no such table: fancy_module (SQL: select * from "fancy_module" where ("serial_number" = D09TP1QLH1K3) limit 1)

5. SQL hacking:

If you’re testing in prod and change a module’s fields but keep the same module name this will confuse your database and you will need to erase the entire db and start again or just erase this module from the db to continue. In SQL speak this is called “Drop table”.

A. List tables

For tables, the type field will always be ‘table’ and the name field will be the name of the table. So to get a list of all tables in the database, use the following SELECT command:

SELECT name FROM sqlite_master
WHERE type=’table’
ORDER BY name;

B. Drop table

Go through the list of tables and confirm the one you want to drop. Then do it. You’re backed up anyway, right? I mean, the data will come back when the clients check in again. So don’t worry.

Drop table to remove “fancy module” table from Db

DROP TABLE fancy_module;

3. Exit

.quit

REFERNCE:

MunkiReport WIKI – https://github.com/munkireport/munkireport-php/wiki

Jon Crain’s module making blog series – https://joncra.in/2018/11/30/creating-munkireport-modules.html

SQLite FAQ – https://www.sqlite.org/faq.html#q5

Notarize it!

Apple’s notarization service allows Apple to verify apps distributed outside of the App Store system. If you make your own apps to distribute to customers, clients, family or friends then you will have to notarize them by submitting them to Apple. This avoids painful dialog boxes in macOS 10.15 Catalina that prevent your app from launching by default.

NotarizeYourApps-Apple-Oct2019

Notarization The Hard Way

I’d been putting off notarizing my apps created for my clients for three reasons,

1)  it isn’t a strict necessity because most users are on macOS 10.14 Mojave,

2) I use Munki to distribute and install software which bypasses the requirements, and

3) I’m lazy

But it is only a matter of time before this would be a strict requirement and necessity. Also the relaxed requirements for notarization of apps was about to change again in February 2020 and I said this is the moment to do something. What now? Check with Rich Trouton and his blog Der Flounder.

codesign –force –options runtime –deep –sign “Developer ID Application: Name (#H7373736)” “/Applications/Cool-App.app/”

Rich Trouton is the modern major general of documentation and a super awesome dude. His blog Der Flounder has documented this process and now it was time to revisit this. Step by step recipes well explained with comments. What’s not to love?! Well, I didn’t get far because I missed some ingredients. Signing the app failed. I couldn’t notarize it without signing it. Hmm…

error: The specified item could not be found in the keychain.

It didn’t work the hard way, so let’s try it another way.

Notarization Made Easy

A very awesome app from Late Night Software called SD Notary can help make this process go smoothly. Their app detects if you have the right cert to run this process. Something which I thought I had, but did not.

Certificates, Identifiers & Profiles

It’s no accident I got tripped up in the same place with the cli and with the SD Notary app to notarize my app. I was missing the correct certificate. When I tried to codesign as a first step that’s when I got an error that I puzzled over for a minute.

The SD Notary app stopped me also at the first step because it said it couldn’t find a Developer Signing ID. And that with the command line error finally made me realize I’d missed something. And here I thought creating the app specific password was the hard part. (It wasn’t hard, but you have to look in the right place!).

A quick run back to the Apple developer site and a trip to the “certs identifier and profiles” section to create a new “Developer ID Application” cert (I had the installer one previously) solved that. I also had some trouble creating an app specific password, mostly because I was looking in the wrong place (in my dev account, not my apple ID account) but that got sorted.

Notarize-Apple-CreateNewCertificate

The “Developer ID Application” is what I needed.. Of course to get this I need to generate a cert signing request. There’s always a few steps. But once these are done then you’re good to go.

Once the proper Application type cert is in place, and the app specific password then you’re able to notarize via cli or an app like SD Notary. I tested this in Terminal:

xcrun altool --notarize-app --primary-bundle-id "com.apple.automator.Cool-App" --username "memyself@email.com" --password "really-cool-passw0rd" --file "/Applications/Cool-App.app.zip" 

No errors uploading '/Applications/Cool-App.app.zip'.
RequestUUID = 12345f-567e-476f-a229-6789cef906b

And in less than 3 minutes I received an email declaring it done. “Your Mac software was successfully notarized.

Then I went back to SD Notary and tried again. It was also successful and after selecting the app the entire process of signing, zipping, submitting to Apple, then stapling was done seamlessly.

SDNotary-Stapling

Hope that makes sense to someone. And the next time I notarize an app I will be able to do it seamlessly thanks to the help of everyone who has provided documentation and cool apps. Cheers.

References:

SD Notary app — Notarizing made easy

Rich Trouton’s Der Flounder blog

Apple dev docs

Howard Oakley’s Eclectic blog

FCP7 to FCPX

If you used classic Final Cut Pro 7 for years then eventually moved to FCPX now what do you do when you want to restore an old project? Read on…

FCP7 to FCPX

In the beginning we set up an older iMac that had been sitting around and already had macOS 10.12 and Final Cut Pro 7 (and even an early version of FCPX). We used this iMac to open up old FCP7 projects from our projects archive which were restored from LTO tape archive created by Archiware P5.

This process of restoring from tape archive back to the SAN then copying to an external drive to attach to this older iMac to convert worked but was cumbersome and not convenient. Opening old projects in FCP7 and then exporting out the XML was easy. Using SendToX to convert to FCPX XML was also easy. But getting the project to this old Mac off the main network was a drag.

Retroactive app

Use Retroactive app to install Final Cut Pro 7

Then one day I heard of this project that allowed to install iTunes on macOS 10.15 (Catalina) which only had the new Music app. Weird flex, but OK. Reading further it also allowed FCP7 to be installed on macOS 10.14 (Mojave)! Now this was a useful revelation. The app is called Retroactive and it would be very useful to us. Now FCP7 could be installed on the same Mac as FCPX. It would then have access to the network and the SAN where do all our editing and where we restore archive from LTO archive. Awesome.

The best part was that we moved from a dedicated old iMac running macOS 10.12 to a newer iMac Pro with macOS 10.14 on the Xsan and can run FCP7 thanks to that new app that makes it work. And then FCP7 to XML to sendtoX to FCPX is not too bad.

We also used Kyno to drill down into all the restored projects to identify en masse all the restored footage that it incompatible with FCPX. Renamed and then reconverted. All is well again for now. Archive restored, FCP7 projects converted to FCPX. Yeah, happy times.

Kyno batch rename dialog box

Kyno batch rename

Kyno FCPX incompatible files reanaming converting

Kyno convert and transcode

We had one minor snag in the process. Some of the restored projects didn’t use FCP7 they used early versions of FCPX with their events and projects folders separated (not the current library structure). Latest version of FCPX 10.4 did not know what to do with these projects that were also some times stored on sparse disk images (oh how the Xsan did not like these projects at the time). There was a menu for a while to convert these projects but it was now gone. What to do?

Back to the old iMac and we used FCPX 10.2.3 to convert these projects from 10.0 version to 10.2.3 library which can then be converted to the latest FCPX 10.4 format. Almost easier to convert FCP7 to FCPX in one shot but it worked and we were happily editing old projects in the latest version of FCPX.

fcpx 10.2.3 dialog box to update projects

fcpx 10.23. update projects and events dialog

Editing old projects in new FCPX

We have a way to restore old projects from LTO tape thanks to Archiware P5, a way to identify, rename and bulk convert old footage in an easy fashion thanks to Kyno and now also a way to convert FCP7 with SendToX and Retroactive to make it more seamless.

Hope this helps anyone else if want to do the same thing good luck.

Automate those apps. Get some robot love 🤖 ❤️!

If only one person needs an application then I think about using Munki to deploy that app. If more than one person should have it then Munki is definitely the way to automate app deployment. And really, if you’re going to take the time to download an app from a website, mount a disk image or un-pack a ZIP archive, run an installer, type an admin password, close that installer … then for the love of all that is good just put the app into your Munki repo and be done with it. Automate it.

Using Munki to solve problems makes sense. Automation helps everyone in this case. But if you’re putting in one off applications into your Munki repo more often than you need to, you need to get those apps into Autopkg. Using Autopkg recipes to download the latest apps and put them into your Munki repo automatically is an automation love fest, but if your apps don’t have recipes what are you going to do? Manually add your apps to Munki? No way. We need a robot 🤖❤️. Recipe robot, that is.

Using Recipe Robot we can build Autopkg recipes for most apps then add the recipes to the Autopkg community to enjoy. Everyone wins.

I recently created recipes for two important apps in my media workflow: Kyno and Hedge. I’ll show an example of this workflow using Recipe Robot and Munki Admin to demonstrate the workflow.

Step 1. Feed the robot.

Drag and and drop the app you want to create your Autopkg recipes.

RecipeRobot-FeedMe

Step 2. Watch the robot do it’s work

RecipeRobot-start

Step 3. Robot is done. Recipes made.

RecipeRobot-Done

Various type of recipes can be made. I chose download and munki because those are what I am using to automate adding apps to my Munki repo. But there are other options: jss, Filewave, or “install” for example.

reciperobot-options.jpg

Step 4. Run those Recipes

You can use your recipes locally with Autopkg. Run them in Terminal or use Autopkgr , a very nice GUI app for automating the collection and scheduling of recipes. Note: Autopkg and Munki can all be run via cli (command line interface) but for this demo we are showing the GUI apps that are there provided by outstanding members of the community. Many Thanks to them and the contributors to their projects.

Autopkgr-notification

Autopkgr app can send notifications in macOS, emails, or post to your Slack group.

Step 5. See the recipes, Use them wisely

MunkiAdmin-Recently ChangedPKGS

Here is an example of newly imported Kyno and Hedge apps in our Munki repo (via Munki Admin GUI).

MunkiAdmin-Description

Add a display name, choose which catalogs the apps will reside in, and check that the description will help explain what the app is.

References:

Elliot Jordan – Autopkg talk at MacDevOps:YVR

https://youtu.be/Q_cvgGtJ71M

Elliot Jordan – Recipe Robot talk at MacDevOps:YVR

https://youtu.be/DgjO1mfMHtI