SOP - Standard Organisation { of } Pictures
In an earlier post I mentioned using AI to create a script/program to organize photos taken from my or my SO's phone into usable albums. Today, we're going to discuss why I created that script, and my attempt at gluing systems together so that I can have organized, backed up, and available photos, with little to no effort. If you don't care about the why, skip to the break, where I start from the beginning of the process.
First we must address the Google in the room. "Joe, Google Photos does all this stuff and more; why would you roll your own and not just 'Google It'?"
If you ask that question, I feel like you didn't read the terms of service for Google Photos, and just said "Yes, I agree". Specifically, I'm referring to this paragraph:
Through this licence, you provide Google with permission to use
your intellectual property rights in your content (such as copyright
and trademark) and any 'droits immatériel' in your content (such as
image rights).
and the following sections (Shortened for brevity sake, but feel free to read the whole thing on your own time) :
This licence is:
worldwide, which means that it’s valid anywhere in the world....
royalty-free, which means that there are no monetary fees for
this licence
This licence allows Google to:
...
sublicense these rights to:...
our contractors who’ve signed agreements with us thatare consistent with these terms, only for the limitedpurposes described in the Purpose section below
This licence is for the limited purpose of:
....
operating and improving the services,....
using content that you’ve shared publicly to promote the services.
So if I understand this correctly, by me using the services, like Google Photo, Google gets to take my images, use them where ever they want too to "promote the service" (i.e. Ads) and pay me nothing for the trouble.
Now, I am fully aware that I am not a professional photographer, and that the likelihood of any one of my images being selected as a good promotional image is so far from reality I might as well be casting fireballs. It still unsettles me that I could fly into Germany for Oktoberfest, and see my SO plastered on a billboard with a big "Google Photos" brand on a particularly good vacation photo from years past.
Now, that's only ONE reason to not use Google Photos, but even that should cover why I don't want to just 'Google it'. So moving onto how I'm doing it.
The first thing I needed to do was to find a replacement for my current process, which isn't google photos, and is pretty stupid, so I'm not going to go into what I currently do.
After some research, I kept reading good reviews of Immich, and decided to give it a try. I figured worst case I could tear it down and basically be back to where I am now. So I setup an Immich server, Then the question became:
"How do I want to use Immich?"
I thought about manually uploading pictures and organizing them and tagging them and ....sigh, this is gonna be a time sink isn't it? I'm gonna have to slog through a bunch of steps each time my SO goes on a photo spree? I knew my SO wasn't going to do that, as I am the only reason they organize at all. (I have cleared multiple explorer-crashing directories, trying to get backups of the files when I started this years ago). A quick web search revealing that Immich had an API, and that it supported uploading both pictures, and the creation of albums. BOOM, problem solved mitigated.
So now with an API, I can setup an automated system to gather the photos, organize them, and upload them to Immich, without my intervention. Saves me time, energy, and mental state, Where do I start?
To start this adventure, we need to gather the photos from mine and my SO's phone, automatically. Luckily, I had already setup a phone backup several months ago, via a third-party rsync-like app. This meant that I had access to both sets of photos without involving the phones themselves (to an extent)[*].
The next step was to decide how I wanted to group the photos into albums. Should it be by day? or maybe location? or should I copy the groupings from the phone? (when those existed). After some thought I settled on a combination of location and timestamp, with an override of phone groupings. summing up:
If the photos are already grouped, applied a day timestamp and upload as it. Done.
If the photos are not grouped, group by location within a 0.5km[**] radius and within a 2 hour time frame[**]. Using a service find the location name and create an album labeled "YYYY_MM_DD_Location_Name", Upload, Done.
Once i had the Idea "set in mud"[***] I decided that I should use AI to do the work, so I could practice my AI prompting and because I was not looking forward to figuring out the API for locations.
The first iteration was surprisingly good at first glance. It picked up the pictures, calculated the center of all their locations, and used that for the for both grouping and location look up. The biggest issue that I've encountered, and am still fighting is that the script doesn't know what "level" of location detail to stop at. So I end up with albums labeled "YYYY_MM_DD_City_State" instead of "YYYY_MM_DD_Local_Attraction". This has manifested itself a couple of time across several different types of things, Attractions, Parks, Museums, etc. Since I kinda figured that the first iteration wasn't gonna be good, I included a hard review stage. This way I can go back through, validate the albums names (adjusting as needed, and logging for the next attempt), and then rerun the script (with an optional parameter) to have it upload to Immich. Eventually I would like to phase out this stage, but at this time, I don't trust that I won't get a mislabeled album.
Great, We now have pulled all the photos, and organized them into albums, time to upload right? Wrong. As I said, my SO is either bad at organizing or doesn't care enough to try, so instead of a reasonable payload of a couple dozen pictures at most a day, My SO has a camera folder of about 30 GB. I'm not gonna try to upload 30 GB of mostly duplicate images to get the handful of new ones. Thus I included in the AI script a duplication check. I have it run the files through Czkawka with the Immich albums folder as a reference, so I can clear most of the images, without having to send 30GB over the Immich API.
Finally, after gathering, duplication check, and organization, we're ready to upload the images to Immich. This is the last step in the AI script. It first verifies that I want to upload at all, and if not it exits. If I do want to upload or I run the script with the "--Immich-only" flag, it loops through the existing base directory and uploads the albums(folders) and images, with the images tagged correctly. It specifically reads the directory and doesn't try to store the information between the first stages and this stage because If I had modified one of the albums (E.g. YYYY_MM_DD_City_State -> YYYY_MM_DD_Local_Attraction), I didn't want the script failing out for "missing" albums, or uploading the incorrectly labeled album. That allows me the freedom to alter album names, or even remove albums that are tagged incorrectly, or are not something to be putting on the Immich server.
There's still quite a bit of work to be done, but the script is at a point that I believe it to be "Stable", so any changes will likely be small changes to accommodate the AI lack of ability to decide on the "level" of detail for locations. If your interested, take a look, but I make no guarantees other then "it works on my machine".
[*] For Future Reference the app is FolderSync Pro
[**] These values have changed over time, and may not be currently accurate.
[***] I change the "Set in Stone" saying to reflect how malleable the plan is, with sand being the most, and granite/diamond being the least (depending on day)