Let’s say you work in the media and entertainment industry, perhaps in Post production, and maybe, just maybe, you shoot a lot of digital film (R3D, Arri RAW, XDCAM, etc), and just maybe you have a SAN. And maybe you’re lucky and you set up a nice backup system using Archiware P5, for example. What about archive? What about finished projects, what do you do about that? Even more importantly, what do you do about the camera archives? That’s the digital film footage that comes in and gets copied to the SAN before creative work begins… it seems like important stuff. Very important to backup, and even better, to archive! How can we set up an automatic archive in Archiware P5?
DEFINITION: What is the difference between backup and archive? In the media and entertainment industry I would define backup as a continuous data protection of live data on a production volume, while archive is a copy of a finished project or original media that will be removed from the production volume and must kept safe for future retrieval. Backups will roll over (a new full backup every week, or month) and if that is all we had, then footage or projects that are done and gone off the SAN would be then lost. Archive separates out the finished projects or source material as needing an independent safekeeping. P5 Archive also has the option of creating a mini-MAM type database of proxy files that can be easily viewed in a web browser for quick identification of files to be restored.
So, how do we set up an automatic archive in Archiware P5? We want it to be automatic so we don’t have to think about it, since manually archiving like backups can be forgotten. If we set up a watch folder on the SAN then we just have to instruct everyone to drop their camera masters and other source files into the folder when they copy to the SAN and P5 will automatically archive them to an incremental tape archive. Wow. That’s awesome.
Tips and Tricks: If you edit with Final Cut Pro X then I recommend using it to make Camera Archives (a verified copy of the footage from the original card or drive) then placing this in the watch folder. If you’re using Adobe’s Premiere workflow, then Prelude can make a verified copy as well, but not in the same way exactly. This will be the subject of another post. Stay tuned.
What does an automatic archive look like?
P5 Archive watch folder
Requirements:
- Pool (designated tapes, or a disk, for archive)
- Index (could be the archive default index, or a unique new index)
- Plan (an archive plan specifies the pool and index used as well as the what and how)
Tips and Tricks: If you’re generating QT previews then you’ll want to move your index off of the default, or else you might fill up your system hard drive with movie files and then your backup server won’t boot. No, I never done this. OK, trust me it will happen 🙂
Tips and Tricks: Make sure to set up a backup job of the Archive index. This is a safety measure. The archive index is not saved to the tape in the same way backup jobs information. You need to create a backup job specifically to save your archive index. But you’re already running a backup job to backup your Archiware main index, right? Uh huh, thought so.
Tips and Tricks: When creating the archive pool set it at 512KB media block size for faster archiving of big video files.
Archive Index:
Use the advanced options to create a new Archive Index, select its location and optionally create new additional fields to help searching for archived projects.
Archive options:
Incremental or full? QT previews, yes or no? For an automatic archiving set up I suggest incremental archive and no QT previews. Using previews is up to you (and your disk space available), so plan accordingly.
If you have any questions or need help setting this up please do not hesitate to contact me. In the meantime, I’ll work on a nice PDF doc to summarize the setup. Download a demo of Archiware P5 and give it a whirl. You’ll see that it’s super easy to create an automatic archive location on your SAN and your editors will thank you when you can restore their files that they need (when they need it!).
DAM SAN 
Thanks for sharing this, very enlightening.
I am thinking since weeks about nearly the exact workflow. I have a question concerning restoring projects from Tape to SAN: Does the P5 incremental Backup still work when the Projektfiles are restored to the San and afterwards copied into the P5 watchfolder?
My concerns are that the creation/change Date of the files will be changed while copying/moving them several times. So P5 will assume they are “changed” and will start to backup the whole Footage instead only the Premiere Projekt Files. Backups would take much longer since the footage size is about 85% of the whole Projekt
Do you now how P5 exactly handles the indexing of files ? By creation date, size ect?
Thanks for sharing this, very enlightening.
I am thinking since weeks about nearly the exact workflow. I have a question concerning restoring projects from Tape to SAN: Does the P5 incremental Backup still work when the Projektfiles are restored to the San and afterwards copied into the P5 watchfolder?
My concerns are that the creation/change Date of the files will be changed while copying/moving them several times. So P5 will assume they are “changed” and will start to backup the whole Footage instead only the Premiere Projekt Files. Backups would take much longer since the footage size is about 85% of the whole Projekt
Do you now how P5 exactly handles the indexing of files ? By creation date, size ect?
I use the watch folder for clients to drop Camera archives / footage from camera cards etc and I use the new P5 Archive app to allow editors to archive and restore projects themselves in the Finder by selecting either a project folder to archive or by selecting the stub file / project folder to restore. (Blog post with details coming soon).
To answer your question on what criteria is used to determine whether the files will be backed up again in an incremental archive plan (full archive is everything always), then Incremental archive will consider the file path, permissions, creation and modification dates. Incremental plans can save space by only archiving changed files but if you want to keep everything together in one place on a tape re-archiving it all is an option. Of course, most choose save a bit of space. It adds up quickly.
I added a blog post about the P5 Archive app and discuss some workflows / use cases for video editors:
https://macvfx.wordpress.com/2016/01/29/best-of-2015-archiware-p5-archive-app/