Category: learning

  • Don’t stop!

    Using a REST API to code a bunch of useful apps

    Background: I released some scripts to help users manage Archiware P5 servers on code.matx.ca with a first blog post as a background to the scripting approaches of cli vs REST API and then I discussed my cute Platypus built apps and my first swift/swiftUI Mac apps… so of course now I’m going to discuss two new Mac apps that use a REST API to explore a P5 Archive.

    I’m building tools to help me and my clients work with Archiware’s amazing and awesome P5 Archive product. It’s great. It archives, it restores, it has a great web UI, and it’s always getting better. So why build apps? Sometimes you want something different, in my case my clients wanted spreadsheets. Yup. Data in a sheet. To look at. So I poked at P5’s databases via cli (see P5 Archive Manager) and with the REST API. Here are the results, two new apps P5 Archive Overview and P5 Archive Search.

    https://code.matx.ca/ is code on GitHub + Mac apps that help manage data in Archiware P5

    Why API? And not cli

    Usually the cli (command line interface) is perfect for working in Terminal and in shell scripts or other programming languages. Using an API or application programming interface, allows different software applications to communicate and share data with each other. Instead of cli commands and arguments programs make requests using specific methods like GET or POST to retrieve or send data. See: API on Wikipedia

    Using an API for my Mac apps means they could use a protocol like HTTP, and make a request using GET (to retrieve data).

    The magic parts of an API

    • API Client: The application making the request.
    • Endpoints: Specific URLs where the API can be accessed.

    For the P5 Archive Overview app we need to use the specific API Endpoint for archive overview detailed by Archiware here which lucky for us is a simple call for data which our app can display, save as json, format as csv (for the spreadsheet!!) and stash in a SQLite database for historical searches.

    However, the P5 Archive Search app has to make many calls to walk through the index tree which is an inventory by path of the files archived. So we ask the user to name the storage they want to search and we make a breadcrumb through the storage, storing everything in SQLite as well as saving json and csv snippets of what we find. A lot more API calls but perfect for an app to do in the background.

    The P5 Archive Search app queries the P5 Archive Index Inventory REST API:

    “`

    GET http://{server}:{port}/rest/v1/archive/indexes/{archive-index}/inventory/{path}

    “`

    Example:

    “`

    http://192.168.1.100:8000/rest/v1/archive/indexes/Default-Archive/inventory/Volumes/BigStorageSMB

    Made for Macs, macOS 14 and up. Bring your own jq

    When running macOS 15 and up jq is installed for free and can help make the csv files form the json, but if you’re running macOs 14 then you need to install it with homebrew, MacPorts, or on your own.

    One funny story during the early testing, I had to fix the jq detection in my first version because it totally missed reading the unix path correctly and yeah, thanks to a friend on the MacAdmins Slack who had an issue when trying it out. I was able to go back and fix that. Friends help friends make better code.

    I’m not done making apps, and I’ll keep tweaking these three I have so far and making new ones from the scripts I have. The goal is to manage data, get a better understanding of the data in the archives and let the clients and owners of the data to know what they have.

    Stay tuned.

  • Swiftly make an app

    This is a blog post about an app, there are many of them but this is mine. How I made a swift app in Xcode but took the long way ’round to get there. And an even longer post to tell the story.

    P5 Archive Manager, an app I created with AI tools to help check files archived with Archiware P5

    I recently posted about some scripts I released on github to help other admins using Archiware P5 archive software to manage their servers and the data in the archive vault.

    Claude: “How can I help you today?”

    Using free credits in various AI tools will get you pretty far, but how far, really? Can you make an app? A useful app? Maybe, yes. Encouraged to transform some simple shell script projects and make a “proper” swift / swiftUI based Mac app I started using Claude as a test. The result is two apps for checking Archiware P5’s archive with a drag and drop tool that verifies files with local or remote servers. Code and apps on GitHub.

    # Path to nsdchat
    chatcmd="/usr/local/aw/bin/nsdchat -c"

    Working with Microsoft’s Visual Studio Code app with co-pilot (tied to my GitHub account) and occasional ChatGPT coding sessions, when co-pilot ran out of free credits, I was surprised that I made a lot of progress on some more complicated scripts.

    ChatGPT: “Ask anything”

    It all started with shell scripts to help me automate some tasks with managing data and eventually I wanted to create a Mac app (or two) for my clients and me to use. I like scripts, but I have so many. And solving a problem in terminal and sifting through all those amazing scripts became more and more complicated, so maybe I needed a nice app that you can click and launch and be done with it to do that one thing. Single minded apps for single purpose objectives.

    Platypus created app. Check if files are archived by drag and drop

    I asked my friendly AI super tool to suggest a way to make an app from a shell script to see if it had any ideas, and it did have a few, although Mac specific ones were not plentiful. Looking through a list of possible methods, it suggested I use the excellent and awesome Platypus app. Now that’s a name I haven’t heard of it a while.

    Platypus app UI. Pick a script, choose an interface and create an app.
    #!/bin/sh
    
    alias nuke="/Applications/Nuke5.2v1/Nuke5.2v1.app/Nuke5.2v1"
    export NUKE_PATH="/Volumes/XSAN/District9/Foundry/Nuke/"
    export OFX_PLUGIN_PATH="/Volumes/XSAN/District9/Foundry/OFX/"
    
    /Applications/Nuke5.2v1/Nuke5.2v1.app/Nuke5.2v1

    I used it many years back in VFX when working with Nuke and other pipeline tools but now I really needed it. Lucky for me it is still around and still works great. So I built a bunch of small Mac apps wrapped around simple shell scripts. Many of my scripts for P5 acted on a path (ie what folder of files do you need to examine) and making drag and drop app in Platypus was incredible easy. Add swiftDialog in the mix and you get nice messages to communicate progress.

    Platypus app with embedded files, including swiftDialog.

    These Platypus born simple apps worked well but I wanted to sign and notarize them and honestly that’s always fun in part because Apple, like every company, changes the way to do it all the time so I often stumbled at this step and looked for more helper apps and guidance.

    I previously used SD Notary Tool and while I had success in the past I got stuck somewhere in the process and couldn’t figure it out. Then I remembered someone had posted in the MacAdmins Slack about a cool new app they built called Signaro to help sign apps. I had looked at it and was initially confused (lots of buttons and options!) but now I needed it. So I tried again and it really helped.

    Signaro app for signing Mac apps, and notarizing and the whole app distribution workflow

    At first I was stuck in the same step, and it was the app specific password that was my first main issue. I couldn’t figure it out and so re-created it with Apple and still nothing, then one more time and for some reason it worked. Success!

    Small side story: I did find a small bug in Signaro when entering the AppleID and App password that I reported and it was fixed right away. In the process of submitting the bug I chatted with the author and I learned about the keychain profile option to make this step easier so thanks again for the helpers in this community.

    xcrun notarytool store-credentials MacVFX
    
    This process stores your credentials securely in the Keychain. You reference these credentials later using a profile name.

    Another confusing part of signing and notarizing is that there are so many certificates you need so I was super pleased to have Signaro identify that I was using an incorrect cert and suggested how to fix the issue. The step by step process and the clear messaging is a big win in this app. It’s a Swiss army knife for signing, notarizing, making DMGs, and all the steps for preparing your app for distribution. Huge shout out to the author.

    Signaro app certificate check.

    So I’m pretty setup I have all the free AI coding tools to smash on my bash shell scripts, I have Platypus for creating the Mac app, bundled swiftDialog and other assets, and lastly a smooth signing and notarizing workflow with Signaro to distribute the apps. What else do I need? What about a native swift app built in Xcode?

    Again talking with the friendly author of Signaro he suggested why not make a swift Xcode app with some AI super tool? Hmm, I thought. Ok, and 3 hours later I’d built something that didn’t work and my “free credits” had run out. Surprisingly, Xcode has its own AI tool helper built-in to macOS 26 and I found it quite useful to fix small issues.

    Xcode has built in AI tools to fix small mistakes.

    When I ran into my first coding road block and free AI credit exhaustion, I also realized I needed lunch and some fresh air, even maybe a break from coding at my desk. As I took a short break I realized I could troubleshoot the old fashioned way and figure out what was not working while I waited for more “free credits” to re-materialize (or I could sign up!). After food and rest it only took me a few minutes to realize that my app did in fact work correctly, but I was testing it incorrectly!

    So maybe you need fresh air and a break from the keyboard occasionally. Truth time. I finally did sign up for a month to test Claude out some more and it is great, but it also gets tired and reaches a maximum length of a conversation. Everyone needs fresh air.

    Even with a simple app, and before running out of credits, you reach max length of conversation.

  • MDOYVR 2018

    MacDevOps:YVR 2018 tickets are on sale now. Buy one for everyone in your MacAdmin family.

    Seems like just the other day we were hanging out with our friends who came from all over the world to talk Open Source and macOS management, and now we can do it all again!

    Tickets are on sale now.

    MacDevOps:YVR is the place for Mac Admins interested in integrating DevOps into their IT practise. Developers and IT (Ops) working together to build a better world.

    Join us at MacDevOps:YVR 2018, our annual conference, for two days of learning and networking in Vancouver, BC, Canada. With speakers from a diverse group of companies, this year’s conference will be the best place to talk about Open Source projects that matter to the community. Learn from your peers, and connect with fellow Mac Admins.

    We will be discussing: munki, imagr, autopkg, chef, puppet and all your favourite Open Source projects. This year we will be discussing MDM and all the changes in macOS. We’re planning another hack night because it was so much fun last year, and if you are interested in a particular workshop topic let us know.

    Learn more at https://mdoyvr.com

    And because we’re always learning from every conference we’ve organized we’re trying something different this year: tiered pricing for tickets. We want everyone to join us and we want to make it fair for independents, students and others who want to be there. At the same time we want to pay the bills and support a diverse group of speakers and attendees who might not be able to attend due to lack of funds.

    We’ve created three tickets: corporate (if your work is paying), independent (if you’re buying you’re own ticket), and education (students and those who work in schools). Last, but not least, the Donation ticket is for those who want to contribute to our financial aid fund. Help those who want to speak and/or attend but need some help.

    Ticket sales: https://www.eventbrite.com/e/mdoyvr2018-tickets-38821491125

  • Mac IT training

    I’ve been asked about Mac IT sysadmin training ideas, and I thought perhaps sharing some initial ideas might help other aspiring Mac IT sysadmins.

    It’s hard to recommend a training site or specific place for Mac IT training these days. It used to be that you could sign up for a Unix SysAdmin or bash scripting class and that would be most helpful. And those both are helpful but Mac IT specific technologies are discussed at Mac IT conferences such as MacTech in LA (and periodically in other cities), PSU Mac Admins (Penn State), MacIT (San Jose), MacSysAdmin in Sweden, MacDeploy in Calgary and MacDevOps:YVR in Vancouver. As well as Apple sponsored camps for ACNs (Apple Consultants). Some of these conferences post their video online for free (PSU, MacSysAdmin and MacDevOps) so these are great resources to learn from.

    Online learning: there are resources such as mail lists which have lots of helpful Mac Admins: mac-enterprise (MACENTERPRISE@LISTS.PSU.EDU), munki-dev (munki-dev@googlegroups.com), munki-discussion (munki-discussion@googlegroups.com), autopkg (autopkg-discuss@googlegroups.com) and the Slack MacAdmins channel (http://macadmins.org) and perhaps lots of other places I’m forgetting.

    The most important is local meetups, and right now we’re not hosting anything in Vancouver. But I do want to start some local meetups here to encourage fellow Mac Admins to help each other out learning MacDevOps skills and getting into the greater Mac IT sysadmin community. There are great meetups in many cities across North America and the world. Finding out where they are means looking at afp548.com for info, or check out Watchman Monitoring’s new Mac Meetup site: http://www.macmeetups.com/find/, or check on Jamf Nation (https://jamfnation.jamfsoftware.com/index.html). Lots of options.
    Here in Vancouver we are also contemplating some more in depth workshops for the next MacDevOps:YVR conference and perhaps sooner than that. How to get started with Mac IT technologies and how to build a beautiful Mac Admins community by being a part of it, and sharing what you know with others.
    Anything you all want to learn specifically? What do you want to know about? Let me know.