A blog about archives, storage and everything

Tag: swift

  • More features, more fun

    While all my post production and media colleagues from around the world gather for NAB to discuss cool new storage hardware, and awesome backup and archive software (shout out to Archiware !!) I have been coding steadily at home working on my Media Trust Tools. Today I’m releasing v2.2. build 2 with many new features to help media professionals load up their camera cards and walk away, knowing they can Copy, Trust then Verify the results.

    Coding Features and Building Products

    What a fun learning experience it has been these last few months becoming a software developer and product manager.

    My apps solve real world client problems, so keeping the features aligned with their workflow is the goal. You need to capture (ingest) camera footage on a lot of different camera cards after a film shoot, and you need to do a lot of it, and copy it everywhere safely and securely. That takes a special kind of software. Multi-source and multi-destination aware with ways of proving and certifying the copy afterwards. Show me the receipts!

    While coding non stop with a little help from my AI friends I’ve been extending and expanding my existing apps and cli workflows into native macOS apps for my clients. Testing in production is crucial, proving that your code and apps work and make sense for end users. Along the way I’ve learned about creating shared code frameworks to optimise apps that share logic and features, creating tests in the code so original features don’t regress when new features are added, and a lot about Swift and Xcode and AI helper tools (Anthropic Claude is the best, while OpenAI’s Codex is not far behind).

    Coding Tools Are Overconfident

    In 5 months of Swift coding and building and testing Mac apps to do media copy things, it can be struggle to remain focussed on the specific features your clients need while accommodating various workflows. I found the trick is to iterate a lot, making small changes, only adding one or two features at a time, using git feature branches to test, using local git repos and remote GitHub main repos. And for sanity, lots of features and goals and next steps documentation to keep some ideas for tomorrow while I focus on today’s testing goals. Using Claude and Codex to tech audit each other helps, and review documentation vs actual code.

    “The bug fixes you declared fixed were over confident in their declarations” said one AI tool of the fixes it documented to patch the bugs it created.

    Adding to my apps to help manage Archiware P5 archives, I now have tools for camera card ingest workflows (CopyTrust) and MHL (media hash list) verification (MHL Verify, Drop Verify) and simpler sanity checks for copies made with other apps (Folder Copy Compare).

    I’ll keep adding features when requested by my end users and keep testing in production. Right now with v.2.2 build 2 just released I can test the new relay chain feature which allows users to copy to the first and fastest storage then copy verified copies onwards to additional storage. With sending to Archiware P5 archives on LTO automatically in the future. Keep that planning list up to date of features to be considered once testing is happy these current features work properly. One thing at a time.

    See the Archiware P5 apps and Final Cut Pro helper apps at code.matx.ca and more details for the Media Trust Tools specifically in GitHub.

  • Don’t stop!

    Using a REST API to code a bunch of useful apps

    Background: I released some scripts to help users manage Archiware P5 servers on code.matx.ca with a first blog post as a background to the scripting approaches of cli vs REST API and then I discussed my cute Platypus built apps and my first swift/swiftUI Mac apps… so of course now I’m going to discuss two new Mac apps that use a REST API to explore a P5 Archive.

    I’m building tools to help me and my clients work with Archiware’s amazing and awesome P5 Archive product. It’s great. It archives, it restores, it has a great web UI, and it’s always getting better. So why build apps? Sometimes you want something different, in my case my clients wanted spreadsheets. Yup. Data in a sheet. To look at. So I poked at P5’s databases via cli (see P5 Archive Manager) and with the REST API. Here are the results, two new apps P5 Archive Overview and P5 Archive Search.

    https://code.matx.ca/ is code on GitHub + Mac apps that help manage data in Archiware P5

    Why API? And not cli

    Usually the cli (command line interface) is perfect for working in Terminal and in shell scripts or other programming languages. Using an API or application programming interface, allows different software applications to communicate and share data with each other. Instead of cli commands and arguments programs make requests using specific methods like GET or POST to retrieve or send data. See: API on Wikipedia

    Using an API for my Mac apps means they could use a protocol like HTTP, and make a request using GET (to retrieve data).

    The magic parts of an API

    • API Client: The application making the request.
    • Endpoints: Specific URLs where the API can be accessed.

    For the P5 Archive Overview app we need to use the specific API Endpoint for archive overview detailed by Archiware here which lucky for us is a simple call for data which our app can display, save as json, format as csv (for the spreadsheet!!) and stash in a SQLite database for historical searches.

    However, the P5 Archive Search app has to make many calls to walk through the index tree which is an inventory by path of the files archived. So we ask the user to name the storage they want to search and we make a breadcrumb through the storage, storing everything in SQLite as well as saving json and csv snippets of what we find. A lot more API calls but perfect for an app to do in the background.

    The P5 Archive Search app queries the P5 Archive Index Inventory REST API:

    “`

    GET http://{server}:{port}/rest/v1/archive/indexes/{archive-index}/inventory/{path}

    “`

    Example:

    “`

    http://192.168.1.100:8000/rest/v1/archive/indexes/Default-Archive/inventory/Volumes/BigStorageSMB

    Made for Macs, macOS 14 and up. Bring your own jq

    When running macOS 15 and up jq is installed for free and can help make the csv files form the json, but if you’re running macOs 14 then you need to install it with homebrew, MacPorts, or on your own.

    One funny story during the early testing, I had to fix the jq detection in my first version because it totally missed reading the unix path correctly and yeah, thanks to a friend on the MacAdmins Slack who had an issue when trying it out. I was able to go back and fix that. Friends help friends make better code.

    I’m not done making apps, and I’ll keep tweaking these three I have so far and making new ones from the scripts I have. The goal is to manage data, get a better understanding of the data in the archives and let the clients and owners of the data to know what they have.

    Stay tuned.