I’m testing the new Accusys Gamma Carry thunderbolt storage in a series of blog posts with real life situations. Filming on location and editing with a remote team requires a combination of good workflow, great apps, and excellent storage. The Gamma Carry is small sturdy box meant for on set and on location editing and camera off loading. It also survives going back and forth between your office and your home office, and wherever you need to go. Disclaimer: I was not paid to write this review and this blog post is an ongoing field test with storage required by every day editors making films using local thunderbolt storage and the cloud. This is their story.

In the beginning….

The start of every creative editorial project is choosing the tools needed for the job, and working as team to make it happen. Making a film requires the hard work of everyone, not the least the IT / Tech who supports the crew and prepares the gear. On set the DIT (digital imaging technician) copies camera cards to multiple hard drives and backup devices before going back to the office to copy them to LTO (archive tape storage), but before the DIT can do their job the IT / Tech has to set up the RAID and design the best and safest backup and archive workflow.

Gamma Carry Overview

The Gamma Carry is an 8 drive external RAID box and it really small and sturdily built. There is a solid metal handle built into the top case and the drives carriers where you put the drives are metal and very solid. This entire unit is built to be solid and protect the drives it contains. The drive tray have a pin to lock the track in place when inserted as well as a thumb screw to lock them in. All drives must be attached with provided screws which is not as convenient as some others with quick plastic tray mounts, but this metal cage for your drive is solid and feels safe for a RAID that will probably be transported everywhere. Keep the data safe!

There are other little touches which are well thought out. The blue red green blinky lights are perfect for any Christmas holiday party and tell you important information about the state of your RAID at a glance but you with one button push turn off all the blinky lights. And keep working in the dark editorial suite not bothered by anything blinking. There’s a mute button and an extra port on the back with 60W to charge your laptop via the RAID. Very useful.

Gamma Carry The Setup

The set up of the Gamma Carry software wise is identical to the Accusys A12 T-Share which I set up recently and once you have the Accusys Mac installer then you have the RaidGuardX app. And the usual IT caveats apply, the software is not signed so you will have to right click on it to open it up. Hopefully they will sign this and notarize their software to make this easier for end users. Also there is a requirement for JRE to run RaidGuardX which means downloading and installed Java on your Mac. Also not optimal but only necessary for the RAID setup.

The set up of the Gamma Carry is the same as the Accusys A12 T-Share and a raid array you built with RaidGuardX will be recognized in the Gamma Carry. That could be a good thing, or not. I did have some excitement when it recognized the array I had built previously (I used the same drives form the Xsan setup). And this was the next step to resolve, because tp create a new array, or delete the array then create a new one I have to do one more thing. Since they were used with an Xsan I had a LUN label that identified them as such and had to remove this LUN label before proceeding. Occasionally we see this issue when re-using drives that had once made up a RAID which was part of an Xsan.

Xsan to the resuce.

To see the RAID arrays avaialable when building an Xsan you can use the cvlabel command to list them. You can also use it to remove this label. WARNING: Do not do this when connect to an Xsan or Stornext storage network. Unless you know what you are doing. You are warned. This is dangerous. Removing a LUN label can bring down the entire SAN. That’s it. Now you know.

sudo cvlabel -l                                      

/dev/rdisk3 [ACCUSYS Gamma Carry      366] acfs-EFI "accusys"Sectors: 46881814495. Sector Size: 512.  Maximum sectors: 46881814495. Stripebreadth: 0.


sudo cvlabel -u "accusys"           

*WARNING* This program will remove the volume label from the
          device specified (accusys).

          After execution, the devices will not be usable by the
          Xsan. You will have to relabel the
          device to use it on the Xsan.


Do you want to proceed? (Y / N) -> y
Requesting disk rescan .% 
                                                                                     sudo cvlabel -l                          

/dev/rdisk3 [ACCUSYS Gamma Carry      366] unknown  Sectors: 46881814495. Sector Size: 512.  Stripebreadth: 0.

So all is good again now we can create new RAID arrays now that the Xsan LUN label was removed. Back to work! Once the drive is set up in Disk Utility as a new volume then you’re ready to go. In this case I added one more drive and created a 5 drive RAID5 set and formatted as HFS+. In my testing this was fast enough and would be faster if all filled with drives or SSDs. There are variations of this hardware with SSD ports instead of 1 or 2 drive bays to allow quicker ingest of SSDs which have camera footage on them.

I then set up 48 hours of drive copies via Hedge for some testing of the RAID hardware. Thanks to Hedge Connect I get notified when the large copy finished. In this I was copying Thunderbolt 2 and 3 external hard drives (thunderbolt but hard drive and not that fast) to the Gamma Carry to be copied. The source drives are slow. Copying from SSDs would be way faster.

And of course Hedge found some minor warnings with these old drives. People don’t like LTO archival tape and want to have stacks of hard drives, but that data on those hard drives won’t last forever. Keep your important data in three places on two different kinds of media (tape, cloud, drives etc). This test of mine was to copy off old drives and use the Gamma to re-organize and re-sort for a new edit project. Now to back up to the cloud and set up Postlab projects with postlab drive for the proxies. In the mean time the editors can get the original footage when I carry over the Gamma Carry thunderbolt RAID.

This is part 1 and after some more testing I will publish some real world tests and experiences.

Download Howard Oakley’s Taccy app

Read Howard Oakley’s blog post on Catalina and privacy protection

Read Apple’s profile reference doc with respect to Privacy Preferences Policy Control payload

Read Rich Trouton’s guide to creating privacy pref policy profiles

This snippet (from MacAdmins slack) shows tcc in the logs if that is the issue:

log stream –debug –predicate ‘subsystem == “com.apple.TCC” AND eventMessage BEGINSWITH “AttributionChain”‘

SQLite3 Db hacking for MunkiReport

Making modules for MunkiReport is easier than ever. Seriously.

please make:module

It’s easier than ever to make modules for MunkiReport (check out the recent MDOYVR MunkiReport workshop) and since the heavy lifting is done you can concentrate on the business logic (what makes sense) and the commands to execute or the scripts to run (python, shellI, etc). Worry about actionable data and less about the tables and views.

If you testing in production (which you should never do, always test is a testing environment) then you may happen to change a module (tables and fields etc) but keep the module name the same. This will confuse your database and you will need to erase it from the db to continue. In SQL speak this is “Drop table”. (You could also delete the munkireport Db and start again, but this is for those crazy enough to test in production and may want to keep the other data).

Sqlite commands for Munkireport

1. Maintenance mode (current way)

sudo ./please down
Application is now in maintenance mode.

Old way –> sudo touch /Users/Shared/MunkiReport/munkireport-php/storage/framework/down

 2. Edit MunkiReport db

sudo /usr/bin/sqlite3 /path/to/MR/app/db/db.sqlite

 3. Exit maintenance mode (current way):

sudo ./please up
MunkiReport is now live.

Old way –> sudo rm /Users/Shared/MunkiReport/munkireport-php/storage/framework/down

Note: if you forget to get out of maintenance mode then clients can’t check in

“ERROR: Server error: MunkiReport is in maintenance mode, try again later.”

4. Migrate Db

sudo ./please migrate

Please use please migrate (or migrate db in web admin) if making changes to a module or else. That is, if you’re crazy to do this in production.

Server An error occurred while processing: \fancy_module_processor
Server Error: SQLSTATE[HY000]: General error: 1 no such table: fancy_module (SQL: select * from "fancy_module" where ("serial_number" = D09TP1QLH1K3) limit 1)

5. SQL hacking:

If you’re testing in prod and change a module’s fields but keep the same module name this will confuse your database and you will need to erase the entire db and start again or just erase this module from the db to continue. In SQL speak this is called “Drop table”.

A. List tables

For tables, the type field will always be ‘table’ and the name field will be the name of the table. So to get a list of all tables in the database, use the following SELECT command:

SELECT name FROM sqlite_master
WHERE type=’table’
ORDER BY name;

B. Drop table

Go through the list of tables and confirm the one you want to drop. Then do it. You’re backed up anyway, right? I mean, the data will come back when the clients check in again. So don’t worry.

Drop table to remove “fancy module” table from Db

DROP TABLE fancy_module;

3. Exit

.quit

REFERNCE:

MunkiReport WIKI – https://github.com/munkireport/munkireport-php/wiki

Jon Crain’s module making blog series – https://joncra.in/2018/11/30/creating-munkireport-modules.html

SQLite FAQ – https://www.sqlite.org/faq.html#q5

Apple’s notarization service allows Apple to verify apps distributed outside of the App Store system. If you make your own apps to distribute to customers, clients, family or friends then you will have to notarize them by submitting them to Apple. This avoids painful dialog boxes in macOS 10.15 Catalina that prevent your app from launching by default.

Notarization The Hard Way

I’d been putting off notarizing my apps created for my clients for three reasons,

1)  it isn’t a strict necessity because most users are on macOS 10.14 Mojave,

2) I use Munki to distribute and install software which bypasses the requirements, and

3) I’m lazy

But it is only a matter of time before this would be a strict requirement and necessity. Also the relaxed requirements for notarization of apps was about to change again in February 2020 and I said this is the moment to do something. What now? Check with Rich Trouton and his blog Der Flounder.

codesign –force –options runtime –deep –sign “Developer ID Application: Name (#H7373736)” “/Applications/Cool-App.app/”

Rich Trouton is the modern major general of documentation and a super awesome dude. His blog Der Flounder has documented this process and now it was time to revisit this. Step by step recipes well explained with comments. What’s not to love?! Well, I didn’t get far because I missed some ingredients. Signing the app failed. I couldn’t notarize it without signing it. Hmm…

error: The specified item could not be found in the keychain.

It didn’t work the hard way, so let’s try it another way.

Notarization Made Easy

A very awesome app from Late Night Software called SD Notary can help make this process go smoothly. Their app detects if you have the right cert to run this process. Something which I thought I had, but did not.

Certificates, Identifiers & Profiles

It’s no accident I got tripped up in the same place with the cli and with the SD Notary app to notarize my app. I was missing the correct certificate. When I tried to codesign as a first step that’s when I got an error that I puzzled over for a minute.

The SD Notary app stopped me also at the first step because it said it couldn’t find a Developer Signing ID. And that with the command line error finally made me realize I’d missed something. And here I thought creating the app specific password was the hard part. (It wasn’t hard, but you have to look in the right place!).

A quick run back to the Apple developer site and a trip to the “certs identifier and profiles” section to create a new “Developer ID Application” cert (I had the installer one previously) solved that. I also had some trouble creating an app specific password, mostly because I was looking in the wrong place (in my dev account, not my apple ID account) but that got sorted.

The “Developer ID Application” is what I needed.. Of course to get this I need to generate a cert signing request. There’s always a few steps. But once these are done then you’re good to go.

Once the proper Application type cert is in place, and the app specific password then you’re able to notarize via cli or an app like SD Notary. I tested this in Terminal:

xcrun altool --notarize-app --primary-bundle-id "com.apple.automator.Cool-App" --username "memyself@email.com" --password "really-cool-passw0rd" --file "/Applications/Cool-App.app.zip" 

No errors uploading '/Applications/Cool-App.app.zip'.
RequestUUID = 12345f-567e-476f-a229-6789cef906b

And in less than 3 minutes I received an email declaring it done. “Your Mac software was successfully notarized.”

Then I went back to SD Notary and tried again. It was also successful and after selecting the app the entire process of signing, zipping, submitting to Apple, then stapling was done seamlessly.

Hope that makes sense to someone. And the next time I notarize an app I will be able to do it seamlessly thanks to the help of everyone who has provided documentation and cool apps. Cheers.

References:

SD Notary app — Notarizing made easy

Rich Trouton’s Der Flounder blog

Apple dev docs

Howard Oakley’s Eclectic blog