Category: Archiving

  • Archiware P5 scripts

    Background: I manage a few backup and archive servers for clients and these run the Archiware P5 suite of software (archive, backup and sync). To help manage these servers over the years I’ve written some P5 monitoring tools for Watchman and MunkiReport as well worked on helper scripts using the cli tools or more recently the API.

    In an effort to share some simple examples of what is possible I have organized a few samples from my GitHub repos on the code.matx.ca page with some useful descriptions and text about usage and purpose of the scripts. They are hosted on in repo here:

    https://github.com/macvfx/Archiware

    I have more scripts of general and specific interest in my repo to P5 users or anyone managing files. See this post on find scripts.

    The P5 code toolbox

    The following P5 scripts are just a few examples and I have more to share, if there’s interest. I created most of these simple tools initially to run on the P5 server directly but I have since created, for my clients, versions which run from anywhere. Also, in some cases, a few scripts have now been built into easy to use Mac applications where it makes sense. If you want some help and you want to hire me to help with these things please reach out.

    The scripts are in three categories: 

    1) P5 archive intelligence (all archive jobs from Db exported as a spreadsheet, or get recent archive jobs via REST API),

    2) P5 housekeeping (make all full tapes read only, show all appendable archive tapes), and 

    3) P5 info backup (export all volumes into one csv, and export all volumes inventory as TSV with barcode as the name)

    P5 Archive Intelligence

    What do I mean by “archive intelligence”? Simply, I want to know about everything I’ve archived. One should consider the P5 Archive server as the ultimate source for all things archived but in some cases my clients don’t use the P5 server directly, or they want the information organized differently, like in a spreadsheet. And while the Archiware P5 suite of software is ever evolving, growing and adding features (even some lovely visual dashboards in v.7.4) I have been attempting to solve the perennial question of “what do we have archived?” in better and more useful ways.

    P5 Information Backup

    Related to archive intelligence is knowing what is in my P5 archive system entirely. I modified a provided shell script from the Archiware cli manual to output a csv of a list of all P5 volumes in the tape library (aka jukebox) so that I might know what is in the system at all times, and even if my P5 server is not running I have a record of every tape. This is one of the scripts I run with my periodic and backup workflows but more on my own special P5 backups (backups of the P5 Db and other metadata) in another post. The Archiware provided P5 volume list script inspired my own script to list full and appendable archive tapes which I have as a one-click desktop app for my clients. When they want to restore something P5 will tell them what to put in the tape library but if you have a lot of tapes maybe you want to know what to remove and so I have a list of candidates (ie take out the full tapes, and leave the appendable archive tapes). Helpful, yes.

    P5 Inventory

    There is a P5 cli command to export out the complete inventory and depending on how long you’ve been archiving and how much is in your archive this tool can take a long time to export a list of every file ever. And because of my mostly non-advanced super skills some times I’d find this process would time out. (There are ways around this but that’s another post). Basically, Too much archive! When it didn’t error out I had a big file… so one day a friend of mine suggested we use Jupyter notebook and yes, use Python!, to do some data analysis. A really fun project, great tool, but this is a hard problem to solve. We made a thing, it worked for a while then I wanted to find a better way. People liked my bar graphs and total amount archived but they also wanted spreadsheets. So let’s give them what they want.

    Two (or three) approaches:

    • cli
    • api
    • db

    I like lower case letters which are acronyms but let’s explore further.

    cli

    Using the cli (command line interface) usually in a shell script (but also in clever Mac apps) typically requires running the script with the Archiware P5 cli (nsdchat) locally on the P5 server and certainly this is what I did when I was testing scripts and various tools. It makes sense if you’re administering a server and you remote in (ssh or screenshare) and that’s where you start. After I wile I discovered the trick to make these cli based scripts run from anywhere which was handy if I wanted to connect to all my p5 servers at once in a script or have my client use an app on their desktop which talks to the P5 server. More on awsock in another post.

    An example of a cli script using nsdchat I have a script for taking the inventory contents of each archive tape (LTO) and writing its contents to a TSV file (tab separated values) which is like a CSV (comma separated values).

    nsdchat Volume names

    Give me a list of p5 volumes (ie tapes)and tell me which one are archive tapes and that are readonly and what is the barcode.

    api

    Ok instead of a cli command dependent on the nsdchat binary installed somewhere we are doing http (web) magic with the API (application programming interface) — a set of known commands in a path based on GET, PUT, POST, DELETE. The API has a different way of doing things than the cli but you can ask a lot of the same basic questions.

    In my api-archive-overview script I am sending one command then using jq to select elements to organize the info into a csv (spreadsheet). This example is set to run locally but this is easily modifiable to run from anywhere. For one client I have a script that talks to every P5 server, each in a different city and asks them all what they’ve been archiving then organizes it all into one spreadsheet. It’s fun, and it’s helpful.

    db

    The best for last. I mentioned above my attempts to use the inventory command which itself goes to all the relavant databases and gathers all the requested data about every archive file, its size and when it was archived etc. Yeah, that’s one way to do it. I’ve shown two examples above for the cli and api but a third is to just talk to the database directly. This is an advanced technique and should only be attempted by an expert. Ok, I’m kidding. As long as you’re not writing to the database and only reading from it, this is pretty safe. What you do with the info is another magic trick and one which I’ve been working on. Two db examples are dump all jobs in a csv file and the second, dump only archive jobs. I’ve got a more advanced script which takes the data and uses sql commands to organize into a csv of how much data archived per day per week per month per year and totals, which is nice for some people who like spreadsheets and want to know about everything ever done. Caution: once you look into the Db you’ll see a lot of things, and sorting through it takes time. I found when making more advanced and selective scripts that the cli jobs used by the very old P5 Archive app (by Andre Aulich) for example showed up as system jobs not archive jobs, so you have to be careful if you want to include those. Have fun.

    P5 Housekeeping

    Finally, some housekeeping scripts are included in the example repo, like the script to make all archive tapes that are “full” to be marked “read only” which is handy is you also have a script to only export the contents of each tapes from archive tapes marked readonly so many little scripts to do little things.

    P5 Archive Prep scripts

    There’s another category of scripts which I haven’t elaborated on but I do have a few examples in my repo. I do have scripts to prepare or examine files and folders going to be archived to LTO with P5 Archive. These scripts do various things like check for trailing space in the name or check file name length but maybe the most important ones I have are scripts which take the path of the archived projects and create html maps, file size directory listings and spreadsheets (again!) of the exif data of all files to keep for future. Clients do refer to the archive stub files (p5c) but they also find it handy to see the directory map and the file size of archived items without going into the p5 server. I’m not trying to replicate the P5 server, or replace it, but this falls into p5 housekeeping and p5 information backup.

    That’s enough for now. If you’ve been reading and following along then let me know if you any questions or want any help with a P5 or other related projects. If you have better ways to do these things feel free to share. My scripts are always evolving and I love to learn.

    Reference:

    For more info on Archiware P5 scripting and building code to interact with it I’d recommend checking out the main P5 manual, as well as the CLI (command line interface) manual, and the API documentation, knowledge base (support). As well as the sample scripts and the Archiware blog, and the video series generally.

  • Add header here

    Or remove it, up to you,

    Had some fun creating a longer script to add a text header to some shell scripts, then because I wrote the wrong thing to all my shell scripts I had some more fun tweaking my script to find and remove this header. I’ve added it to my GitHub repo with a couple of other scripts based on the find command, one of my favourite unix tools since it is so handy.

    The script that should be a Unix one-liner: add (or remove) a header

    Some of the other example scripts based on find might be of interest to some, such as the

    File Name Check

    Especially important with certain filesystems (certain encrypted filesystems) with file name “length limits”. So why not check for these files and zip them up and put them aside for safe keeping. In practise, the only files which push this limit are downloaded (purchased) from stock photos sites and write the file name with every keyword. Nice, but why can’t we have standard metadata handling these days? (I can dream!)

    Archiware P5

    The last two scripts I made with Archiware P5 in mind, as I manage many servers for clients with P5 Archive and I really do love this software and the team. More Archiware P5 inspired scripts are in other repos here or on my main P5 code site

    Find A Trailing Space

    In this case, besides it just being nice to clean up folder names with invisible trailing spaces before the archive job it was also necessary when using the P5 Companion (desktop) app which will not archive a top-level folder with a space at the end of the name.

    Make (Only) One Thumbnail

    This script makes only one image thumbnail per RDC folder, as they normally have a lot of R3D files which are part of the same shot. Also, I don’t want a lot of duplicates and only one is enough.

    And while yes technically P5 Archive can make thumbnails and proxy videos when it is archiving (and I do use this feature) making proxies of RED files is an intensive process for older computers which means taking a long time, so pre-processing these R3D files ahead of time on faster computers can make the final archive job quicker. As part of some pre-processing before archiving to LTO (or wherever) is making sure some formats like R3D (aka RED) files have a thumbnail which will then end up in the P5 Archive created Archive index.

  • Good bye Rsync, hello open rsync

    So apparently I’ve missed the big story of Apple introducing BSD licensed open rsync in macOS 15 Sequoia back in the Fall of 2024. (Thanks to Rich Trouton’s recent blog post).

    Open Rysnc in macOS 15.4

    Openrsync it’s new. And I didn’t notice it, because I’ve been using a newer Apple silicon compiled rsync (see RSKgroup) because Apple hadn’t updated the included rsync 2.6.9 in decades and it was becoming a problem. It seems the big break of only shipping the newer openrsync happened in macOs 15.4 because until 15.4 Apple gave the option to use either the SAMBA or the new BSD licensed version with an environment variable.

    I use rsync a lot, for all my hand rolled artisanal backup scripts and the occasional push and sync between storage. Of course, for serious backups and archives I use Archiware P5 not the least because I need one dashboard to track backups, syncs, and archives, and more importantly I need to use LTO (tape backups) and Archiware is the best. As for useful terminal commands, rsync is often my go to for a lot of edge cases and quick storage comparisons.

    Apple’s open source GitHub for Rsync

    Open rsync was written by Kristaps Dzonsons in 2019 and is part of Open BSD but should run on all BSD or Linux distributions.

  • “I love SQL” and other lies you tell yourself

    Navigating a database to get what you want, that is the goal. Do you love it? No, but you do need to do it? Yes, to get the data. Remember the goal.

    Many popular (and many lesser known) applications use SQL, and SQLite in particular, to store data. That’s fine. That’s great. But unfortunately on occasion you need to go spelunking to find data you want and get it out. This is not a blog post about how much I love SQL (structured query language), because I do not love it. This is also not a blog post about how SQL is awesome, because I can’t say that. But what I hope to share are some tips and tricks for getting in and out with the data you want.

    Use an app – DB Browser for SQLite

    https://sqlitebrowser.org/dl/

    If you’re not a fan of SQL and you’ve got a need for DB data then this app will let you open a database and explore. This is a great app because you can see the tables and what’s in the Db which will no doubt help you late explore in Terminal or in a script. I personally need a visual map sometimes before I jump in. Exploring the Db in this app will also show you the arcane commands necessary to do the same in Terminal. You will be in awe of whomever decided to create this complicated series of commands which makes long insane Unix commands seem logical in comparison.

    You can use DB Browser to export a csv (comma separated values) for a spreadsheet or as JSON (JavaScript Object Notation) which all the cool kids like these days. Better start loving this. More on JSON and APIs in a future blog post. Its not XML, but it makes you wish it was.

    Use Terminal – Type the commands by yourself

    In Terminal we can tell sqlite we want to export a csv file of everything. Add a header and tell it to be in csv mode then SELECT everything.

    sqlite> .header on

    sqlite> .mode csv

    sqlite> .output export.csv

    sqlite> .quit

    Export just some the data as a CSV

    sqlite3 /path/to/the/database.db
    SQLite version 3.40.1 2022-12-28 14:03:47
    Enter “.help” for usage hints.
    sqlite> .header on
    sqlite> .mode csv
    sqlite> .output JustTheSelectFieldsPlease.csv
    sqlite> SELECT label, timeDated, fancyList, sillyList, boringFiles, indexName FROM tableName;
    sqlite> .quit

    Automate and Make a Script

    #!/bin/bash
    sqlite3 /path/to/the/database.db <<EOF
    .header on
    .mode csv
    .output JustTheSelectFieldsPlease.csv
    SELECT label, timeDated, fancyList, sillyList, boringFiles, indexName FROM tableName;
    .quit
    EOF

    UPDATE: You must check out Datasette!

    After posting this I was reminded of an app called Datasette which is truly remarkable and awesome. It’s also a pip install thing but I’ve been using the standalone Mac app which has everything self-contained.

    With Datasette it’s easy to load SQLite DBs directly and filtering out the tables I want by easily set conditions, which makes exporting a workable JSON or CSV file quite easy in one step. There’s also a small and lightweight web app called Datasette Lite to make installing and running Datasette extremely simple. Datasette has plugins too. A lot. More to say on those in a later post.

  • How To Securely Sync Your Synology NAS with P5

    Use Tailscale Mesh-VPN with P5 Backup and Sync

    In the old days we used to forward ports. On your router the traffic for a server or service went to a port (where a number represents a service, some which are defined, but can be arbitrary) and to a destination IP address. Well, wouldn’t you know it, if ssh is port 22 or web traffic is on port 80 then everyone and their port scanner comes knocking. So then your firewall is tested, and then auto-ban and geo-block and emails go out. What if we could avoid that and not open (or forward) any port to make services work across the internet?

    Tailscale is a mesh-VPN which uses WireGuard to securely establish a mesh (point to point) VPN of your devices. Suddenly your iPhone can securely send files to your Mac or raspberry Pi across the world. How cool is that? In today’s advanced lesson: you can backup and sync your Synology NAS using Archiware P5.

    Step 1: Setting up Tailscale on Synology

    It honestly used to be harder than this, these days you can simply add the Tailscale package via the Synology package center app and you’re done. Almost. There’s one more step.

    Step 2: Set up Outgoing VPN access via Tailscale which requires editing some files (which necessitate Terminal and remote login access). This only has to be done once but future updates may require fixes. This was tested in DSM 7. Pro tip: only allow remote access to a restricted and time limited account so you don’t leave it on accidentally.

    Step 3. Install Archiware P5 on Synology NAS

    Using Archiware P5 to Backup and Sync your NAS is a good thing if you’re already using Archiware P5 to backup and sync all the other things, then at least you have only one dashboard to look at. I use P5 with my clients to backup their shared storage to LTO and it makes sense to backup all the things no matter where they are with P5 also. With Synology NAS package center it’s a simple one-click install for P5. Add your P5 clients to your P5 server via Tailscale and you’ve got a secure setup.

    This post is just a quick overview of using Tailscale to set up your P5 clients (which is your Synology NAS in this case).

  • Backup Fast, Restore Quicker

    Backing up is nice, restoring is better. Slow backups, mean slow restores. Make good decisions, and backup only the files you want to keep to the fastest storage you have.

    When working with a fast fibre channel or Thunderbolt SAN your first choice for fastest backup destinations is a Thunderbolt RAID. I recommend to have this onsite with an off site LTO and/or cloud disaster recovery setup (a replicated SAN or shared storage system is nice to have too).

    A built-in option to copy Xsan files is cvcp (cv stands for centravision).

    cvcp -vxy /Volumes/TSAN/folder /Volumes/GammaRAID/backups

    cvcp is fast. Really fast. And cli commands are scriptable. A very smart person (Jasper Siegers) wrote a script called cvcpSync which combined the power of rsync and cvcp. It was awesome. But there are limits to the best of scripts. For my clients I use Archiware P5 with large SAN and other shared storage to simplify the number of things which need to be monitored. One dashboard to monitor tape or cloud backups, tape archives and sync to nearline RAIDs or NAS.

    With a recently Thunderbolt SAN deployment with Accusys T-Share I set up the Accusys Gamma Carry as a backup destination. I set up Archiware P5 to do the backup. It was fast. How fast? Over 1Gb/s. Fast backups are also fast restores. With the Gamma Carry I can run a backup then carry it off site. It’s an option as part of a complete backup strategy.

    Archiware P5 backup 1.6TB in 53 minutes

    (Luckily I have almost 2 TB of video from my Cycliq bike cameras to test backups. Sadly, after my last bike vs car incident I felt obliged to buy bike cameras for my safety. I edit small fun rides when I can. Sometimes traffic near-accidents too. Please be kind, don’t kill cyclists.)

    Archiware P5 backup of a Thunderbolt SAN to a Thunderbolt Gamma Carry RAID

    Note: In my tests I tested backup to a nearline RAID. I also like to use tape drives. LTO tape is another recommended option for backups or archives. Cloud or other offsite replication is also recommended if possible but is the slowest of all the options. Good to have slow and fast options, offsite and on premise, though any practical solution should be affordable and useful to help decision makers take the steps to preserve data and ultimately their own business.

    LTO vs Cloud backup comparison: For LTO backups to one LTO7 drive I normally see 1TB in under 2 hours versus some recent cloud backups I did using rclone which took 9 hours for 1TB. Remember: restore times will equal your backup times. Want to restore 100TB? Got a spare 900 hours? 38 days for cloud restore vs 8 days with one LTO7 drive (much faster if you have more than one drive). Even faster if you restore from a Thunderbolt RAID. Only 2.5 days. Think about it.

    Testing equipment:

    Hardware: T-Share SAN, Gamma Carry Thunderbolt RAID

    Software: Archiware P5

  • Best of 2018: FCPX and iMac Pro

    Part of a series of blog post on the “Best of 2018”

    Part 1: the iMac Pro and FCPX

    The year started off with the new iMac Pro and Final Cut Pro X 10.4. Both new hardware and software were released in December 2017. New awesome hardware and software to start of 2018.

    FCPX and the iMac Pro have proven themselves to be a great combination that has been amazing for FCPX editors everywhere. The new colour grading tools and other enhancements were warmly received in FCP X 10.4. The power of the iMac Pros was not exaggerated. Excellent pro hardware.

    FCPX works great on a MacBook Pro and internal storage, with Apple’s Xsan and fibre channel or with Lumaforge Jellyfish 10GbE over NFS. I worked with all different setups in 2018 and happy to report that editors kept editing and left the storage and backup worries to me (and I didn’t worry since I’ve got Archiware P5 watching my back).

    Working with the Jellyfish I installed the P5 Linux agent to backup and archive to tape. Getting the Jellyfish to back up to my P5 server running on a Mac Mini couldn’t have been easier. Through the year I worked with Archiware to make improvements in the P5 Archive app so that my editor clients can archive and restore more easily on their own. Works well and look forward to working more closely with both companies to help make awesome setups for FCPX editors and creative professionals everywhere.

    NAB and FCPX

    The week before NAB 2018, Apple announced a new version of Final Cut Pro X with support for closed captions, and the brand new ProRes RAW codec.

    NAB in April is always a busy month with announcements from all companies in the media production and media asset management world and Apple’s public talk at NAB showing off new features so soon after their last major release was unexpected but very warmly received.

    Of course there was one more major event in the 2018, in November there was the FCPX Creative Summit.

    I attended this year and it was awesome. Apple released a brand new version with 3rd party integration in the form of extensions. This is huge. This will be amazing for FCPX editors who want to stay in FCPX and do their editing work but integrate with other apps.

    What was the FCPX creative summit?

    ⁃ rendez-vous in Cupertino with Final Cut Pro editors, studio owners, plugin authors, creative apps vendors

    ⁃ Visit to Apple HQ. With Apple Pro Apps engineers, QA, managers and everyone involved.

    ⁃ In depth discussion of the next version of FCPX extensions which allow third party integration deep into the app for example: Frame IO for review and approve or Keyflow Pro or Cat DV media asset management apps.

    ⁃ Great team of people organizing. This event had multiple tracks and lots of great sessions for everyone. Well done. Enjoyed it immensely. Everyone using Final Cut Pro or involved in this creative universe should be there.

    2018 was great year for pro hardware and software. The iMac Pro and the constant stream of FCPX updates kept us grinning from ear to ear. Great stuff. Awesome year.

    Next up: best conferences of 2018

  • Updating the P5 client on the Jellyfish

    You’ve successfully installed Archiware’s P5 backup and archiving software on your backup server following my previous blog posts and after it has run smoothly for a while you decided to upgrade the version of P5 on your server, but how do you do this on the Lumaforge Jellyfish storage? I’m glad you asked.

    There are a couple of ways to update your P5 agent, and I will show you the built in way in Archiware’s P5 software. Surprisingly after many years of using P5 I have never used this method before. I’ve been using Munki for years to upgrade all software on my Mac clients including P5 and on Linux and Solaris servers I’ve just done it by hand. Install over top of the previous version and voila upgrade! But what if you didn’t want to ssh in as root and just install over top, what if there was a better way? I present to you the official “Update client” dialog box. It’s nice.

    Update-p5-jellyish-1

    Updating client software assumes you’ve set up clients in the P5 server clients section, This is needed when you want to use these server agents to designate their attached storage as a backup, archive or sync source. And also, this assumes you’ve updated your server.

    P5 client update Jellyfish 2 Screen Shot 2018-08-06 at 4.40.03 PM

    During the update process there are some nice dialog boxes to let you know what is happening.

    P5 client update Jellyfish 4 Screen Shot 2018-08-06 at 4.44.34 PM

    And afterwards you can test your client with a Ping test.

    P5 client update Jellyfish 3 Screen Shot 2018-08-06 at 4.44.27 PM

    Success! Looks like we’ve updated our client successfully. How wonderful. And no need to mess about in Terminal with a root shell. No telling what kind of trouble we could get into with those elevated privileges…. much safer this way.

    Thanks Archiware for making this great software. I depend on it every day.

     

     

  • P5 on the Jellyfish: Archiving Gotchas

    TL;DR

    Using Archiware P5 to Archive files to tapes is awesome, but watch out for little things you might miss, such as the path to the files and backing up your Archive Db.

    P5 Archive on the Jellyfish

    Using P5 Archive with the Lumaforge Jellyfish is a great way to preserve your digital archives. See this post for how to set up P5 on the Jellyfish

    Using Archiware P5 for archiving makes sense. You want your completed projects and original camera footage on LTO tape. But how do you do archives? There are several different ways, and there be gotchas.

    P5 Archive vs P5 Archive app

    Using P5 Archive to manually archive completed projects to LTO tape is a process of logging into the server via a web browser and selecting the the project folder you want to archive to tape.

    The completed project folder could be on the storage visible to the server or it could be storage the client sees. And that can make a difference. Where the storage is mounted is different on a Mac vs Linux. Its’ the difference between “/Volumes” and “/mnt”.

    The same Jellyfish storage, either SMB or NFS, when seen on a Mac is mounted by default at “/Volumes” (this can be changed but for most people leave it at the default). But when archiving the storage via a Jellyfish client you will get “/mnt” path.

    p5-smb-test2.png

    Using the P5 Archive app, which is a Mac only companion application to P5 Archive, to run the archives you will see the storage archived as “/Volumes”.

    This first Archiving gotcha is if you’re archiving the Jellyfish storage with the web application of P5 Archive you will have to find your footage and restore from the “/mnt” path vs if you’re archiving from the P5 Archive app which is running from a Mac and will see and store the footage using the “/Volumes” path.

    All this to say that using both ways to archive may double up your footage in your archive which may be unintended. And from a restore in the web browser finding your footage may be confusing if you’re used to seeing it mounted in “/Volumes” and you actually find it under “/mnt”.

    Note: the reason to use the P5 Archive app is because of the simplicity of right-clicking files in the finder which are on your storage and telling them to archive right then and there. Files are copied to tape then the original files on the storage are replaced with stub files. Right-click again to restore. Simple.

    p5-archive-app-job-monitor.png

    Backup your Archive!

    Don’t forget to backup your archives. Or rather, your archive Db. A more recent addition is the ability to automate the backups on the Archive index, so don’t forget to enable it.

    In the managed index section, choose your Archive index.

    Set the target client where the backups are going and the backup directory. Choose a time and don’t forget to enable it (check the checkbox and hit apply before closing the windows).

    Note: Repeat this setup for each Archive index you want to backup.

    Archive Backup db setup3.png

    Monitoring your Archive!

    Don’t forget to enable email notifications for your P5 server to get your inbox full of status notifications and errors and other important stuff. But if you want to cut down on email notifications or you have multiple P5 servers (many different clients, perhaps), then you might want to check out Watchman Monitoring and the P5 plugin that is built-in). Find out easily when your tape pools are getting low, the tape drives needs to be cleaned, the support maintenance needs renewing etc. All in one dashboard. How convenient!

    Maybe everything is going well…

    Watchman-P5-info.png

    Or maybe not!

    Archiware-P5-Jobs-Watchman-tapes-required.png

     

  • Install P5 on the Jellyfish

    TL;DR

    You can easily install Archiware P5 backup and archive software on a Lumaforge Jellyfish storage server. Once you’ve done that you can backup to tape or disk or the cloud directly or through another P5 server. Backups are good. Archive are good. Restores are better.

    P5 install on the Jellyfish (Linux) How-To:

    Note: Thank you to Lumaforge’s CTO Eric Altman who gave me some basic instructions to get me going.

    Step One: Download the latest Linux P5 rpm file 

    http://p5.archiware.com/download

    p5-Linux-rpm.png

    Copy the downloaded rpm file to the root folder of your SMB or NFS file share.

     

    Step Two: Install the rpm file

    Open Terminal and ssh into your Jellyfish. Login as root or as another appropriate user.

    yum localinstall /mnt/Primary/ShareSMB/awpst554.rpm

     

    Step Three: Browse to server on port 8000 to test that the server is up

    e.g. https://jellyfish:8000

    Or in Terminal and ssh into your Jellyfish and ping your P5 server

    cd /usr/local/aw 
    
    ./ping-server
    
    Pinging PresStore application servers...
    
      lexxsrv pid: 4840 (server is running)
    
      lexxsrv url: http://127.0.1.1:8000/login 
    
    Pinged 1 from 1 application servers.

     

    Step Four: Decide if the Jellyfish storage will be a P5 client or a server.

    Note: If configuring the Jellyfish storage as the main P5 server you may wish to set up a user that only has access to the shared volumes.

    For my set up the Jellyfish storage is going to act as a P5 client to a main P5 server on a Mac mini (yes, they are useful for something). The Mac mini is this case is the P5 server and is attached to theOverland tape library via a Promise SANlink2 Thunderbolt Fibre Channel adapter.

    NEOs-T24-large-new.jpg

    macmini-ports.png

     

    Step Five: Set up the Jellyfish storage as a P5 client

    Log into your P5 server and add the Jellyfish by the IP known to the P5 server. In this case the P5 server is connected via 1GB to the Jellyfish in Port 1.

    P5 clients jellyfish setup1.png

    Note: You could also choose to plug into the Jellyfish via a 10GB port, but in my setup these 10GB ports are reserved for the edit stations. You should choose what’s appropriate for your setup.

    P5 clients jellyfish setup2.png

    Resource utilization of P5 on the server is low, topping off generally at 1GB of RAM at peak usage. While this does technically take resources from ZFS caching, the impact should be super minimal.

    In my observations the CPU never spiked too high while both serving NFS and SMB mount points to multiple Final Cut Pro X workstations even with backups or archive jobs going to tape at the same time.

    jellyfish-cpu-resources-graph.png

    More Jellyfish P5

    See the follow up post on Archiving gotchas with the Jellyfish here