How to train local AI with own photos

[mlavicka]

I have a lot of old scanned in photos that I have added descriptions, tagged people and properly dated.  I have a lot more that I haven't yet.  I have looked over the information on how to install and use a local AI for AutoTagging, but I would like to further train the model with my own photos' data first.  The thought is that it could better use the photos where I have already identified people, places, and dates, to help with the ones where I haven't.

Does anyone have any suggestions on how to use Ollama with iMatch for AutoTagging, but first training the model with my own library of photos?

[Mario]

The internet has many tutorials on how to train or fine-tune models. This is quite complex, requires usually very fast hardware and a lot of know-how.

Fine-tuning an existing model like Gemma3 is the way to go in your case, I guess. I have not looked into fine-tuning myself yet, so I cannot give you instructions for how to do it.

Google for fine-tune Gemma 3 for some good pointers.

[mlavicka]

Thanks.  I will give it a try.  Ideally, I can get it to work like iMatch face recognition, where I can train it on some subset of photos to learn people, location/buildings, and likely time frame when photo was taken (like MyHeritage PhotoDater).  Then scan the rest of the photos to suggest updated info.

[Mario]

If you manage to do it, let us know how. This is an interesting topic, but, at least at the moment, something that's not easy to do.

[mastodon]

Yes, I have the same exact situaton, with more than 2200 scanned files, and counting. Very interesting topic.

[monochrome]

I actually tried implementing something like this using my old IMception IMatch app as a base. 

The plan was to have a trainable image classifier with multiple fine-tuned models. If you wanted to auto-classify images you'd create some set of categories, assign images to them, and then tell "IMception 2" that you wanted to auto-classify stuff "like that". It would then use the images in the categories to train a classifier, and then go to town on the rest of your database, or the categories you pointed it at. (Yes, there would be a way to limit which images would be classified and so on.)

For your use case, you'd set up a training category hierarchy, like "KnownDates" with child categories for each year or decade or whatever, fill them with known dated photos, and use that for training. IMception 2 would run a small Python web service in the background and run training jobs continuously or on demand, keeping your models and photos up to date.

It basically failed on these things: 

1. I didn't have a pressing enough need. It was a cool idea but I didn't have a use case important enough for me to do it. Related to this is the fact that with some few exceptions I don't categorize my images, and so my categories are of the kind that is easy to do assignments to manually - why bother with a best-effort AI when you can clickity-click and get it perfect yourself even faster?
2. The old knowledge management problem. I saw my category tree being populated by machine-controlled categories (known good, classified, curated) and it was just a very unpleasant experience UI-wise. My categories are curated, not really a dumping ground for best-effort AI classifications.
3. Classification was, as always, good but not great. Enough for a rough sorting, but not enough for being in my category tree. (And obviously nothing I'd ever want to write back to the image metadata.)

With the addition of "AI tags" to IMatch it may be time to revisit the idea, though. The AI tags provide a storage space where the knowledge management problem maybe can be solved.

I still don't know what I'd use it for, though.