AI Traits Tags - Where are they stored?

[bekesizl]

Where are the AI Traits Tags stored?

I seem to have lost the custom tags assigned in my database.
I also seem to have lost the prompt settings for those tags.
It is not a big loss for the moment, but it would be good to see, where/why those data has gone.

I don't remember doing anything with those tags. 
The data-driven for them categories are still there, but they refer to a completely different tag.

I updated to the newest IMatch version soon after it was released, but haven't used AI since approximately a month ago.

[Mario]

Your question is slightly unclear.
Do you mean the data stored in Traits Tags oder the Trait Tags you have created?
What is missing?

Where are the AI Traits Tags stored?

To quote the help system:

"AI tags work like any other metadata tag, except that they are stored in the database only and can be created and deleted by the user. " IMatch Traits

The data you store in Trait Tags is stored safely in the database.

You can see a list of all trait tags you have created via Managing AI Tags

You can search in trait tags via the File Window search bar and the Filter Panel (e.g. a Valuer Filter based on the trait tag you want to use).

Each AutoTagger setting you create can use any number of trait tags.

[bekesizl]

I meant in the main DB or in the settings DB?

The list in the Edit - Preferences - Auto Tagger - Custom Tags was completely empty.
I have defined 3 Custom Traits there.

The custom settings for Ollama and LM Studio were there, but when I open them, I get following message. Now the Custom Traits are not empty, my prompts are there.

But when I look at the Category "AutoTagger" created and populated by IMatch and at some files I played with, I only see values for the built in AI tags, but not for my custom ones.

I also have two data-driven categories defined.
One was for AI.HasText. It is now referring "JPEG::HDRGainInfo\6\HDRGainCurveSize\0", something I never heard of and surely not have set.
The other is AI.PeopleOnImage and it is now referring "JPEG::HDRGainInfo\10\HDRGainCurve\0".
They had some files assinged to them before, I am sure.

I switched the settings DB some time ago to an older one maybe.
I also had IMatch disappear (close the main window all of a sudden), probably due to free memory reasons.
It was although always starting clear after such cases.
I will take care of the memory issue soon.

[Mario]

The AI Tag configuration is stored in the IMatch database, like all other tags IMatch imports from ExifTool or creates itself, like the photools.com.LTID tag. They have their own tag set and unique tag keys.

When you open the Tag Selector (e.g. in VarToy) do you see any AI tags:

[bekesizl]

OK. For test I loaded a DB backup from 3rd of May. It is most probably before the last update.
Opened it with the updated IMatch.

Here the Tag Selector in the VarToy App shows only the 3 built in tags.


I then opened the Preferences and I can clearly see there two of the tags I defined, prompt is present for both.


And here is the definition of the Data-driven Category with the tag I have never heard of:


I guess, that this might be some bug, but I don't know what I have done to get this.
Should I upload the DB somewhere?

[Tveloso]

I switched the settings DB some time ago to an older one maybe.

Would this cause a problem if the older Settings DB is from a prior IMatch Version?...(or will IMatch just convert that DB just as it does with the Main DB?)

[Mario]

@bekesizl

In your AutoTagger setting, no tag is selected for storage.
Your database has no AI tags.
AI tags you select for Traits are created when you close the dialog. The only known ways to remove AI tags are:

- You delete the last trait in the last setting that uses the tag. IMatch asks you.
- You delete the tag from Database > Tools > Manage AI...

I have never "lost" an AI tag and no other user ever reported this problem. I have no idea how an AI tag could get lost. There is only one "DeleteAITag" function in the IMatch engine and it is used in the two situations reported above.

If you find out when this happens, let me know. If this is something that is reproducible, we can fix it.
I assume the database diagnosis finds no problems?

@Tveloso Installing an older settings database should do no harm, IMatch should upgrade settings when opening / accessing the settings. It should be avoided, though. NOT a good practice.

Since all/some of the settings might have been migrated, triggering other changes in features and the database. Installing an older setting database may bring older settings back, which then need to be migrated, maybe clashing with migrations already performed. I don't test such forced roll-backs.

[bekesizl]

I am sure I haven't done any of those two things to remove a tag.

I just checked and the field in the dialog you mention is read-only and is filled automatically, so I am sure I haven't deleted it either. They disappeared somehow.

I have done Database Diagnosis last weekend multiple time ant it was showing only some warnings about writeback and faces without thumbnails, that I usually see. No errors.

I will observe this and hope its a one time glitch.
My laptop will get extra RAM tomorrow and I hope those sudden "crashes" will also disappear.
I am not playing with the settings DB, but I am not 100% sure what I have done on the transition from PC to laptop to finally transfer also my settings to my new laptop.
But if these tags are stored in the main DB, it must have been something with this that happened.

Lets hope it only affected me at this one time.

[Mario]

Maybe you copied the wrong database when migrating between computers?

The trait tag field is empty because no trait tag exists (anymore). It is read-only, the name is derived from the name of the trait.
Unless you setup a trait to fill a normal XMP tag, e.g. Headline.

More RAM is good. IMatch was designed and implemented on a PC with 4 GB RAM, but 8 or 16 GB are standard today, for good reasons (Windows needs a lot of RAM, IMatch can use more RAM for better performance etc.). Good that you can add more RAM to your laptop, many of the latest models had the RAM soldered on, which makes it hard to add more.

[bekesizl]

I am sure that this didn't happen with the database migration.
Migration of the main DB was done before starting with AutoTagger.

RAM upgrade is needed to be able to run IMatch in parallel with Photolab and Radiant Photo, Mylio and maybe a webbrowser.
16GB is not very much today. 32GB should be more than enough.

[bekesizl]

My custom AI tags disappeared again.

Last weekend I had a database failure and restored multiple pack&go-s until I reached one that had an errorless Database Diagnostics run.
I don't know what caused the failure. DB is on an internal SSD, only the cache is on an external SSD.
I have only on idea what might have cause the breakdown, I will open another post for it.

I will open my older pack&go-s via extract and open DB only to look for my custom tags.
If I find one where this information is intact, will it be possible to Export them via the Text exporter and import them in the current database?

[Mario]

What doe you mean by "database failure"? The database system reporting physical damage to the database file?
This is a very, very rare error and. usually, caused by hardware issues, network issues (for databases on a server), power failures or "bad" drivers which don't restore state correctly when Windows goes to sleep and wakes up again. And, sometimes, virus checkers blocking the IMatch process, and so the database system, from finishing writing data to the disk.

Do you have created "exceptions/exclusion" for IMatch and the folder (!) containing the database in your virus checker?

You can export AI tags via the text exporter and re-import them into another database, like any other metadata tag.

[bekesizl]

What doe you mean by "database failure"? The database system reporting physical damage to the database file?
This is a very, very rare error and. usually, caused by hardware issues, network issues (for databases on a server), power failures or "bad" drivers which don't restore state correctly when Windows goes to sleep and wakes up again. And, sometimes, virus checkers blocking the IMatch process, and so the database system, from finishing writing data to the disk.

Yes, it has been this ugly one and I don't really see the cause. DB is on an internal SSD in a 1 year old laptop.

Do you have created "exceptions/exclusion" for IMatch and the folder (!) containing the database in your virus checker?

Not at the time, but I created them now.


The problem is, that I have gone back with restoring my backups until 2025.05.12 and 2025.05.16 and there is nothing inside the DB.
The tag selector is not listing my custom tags.
The AutoTagger settings list them but with empty tag field.
The LM Studio log clearly shows activity and I have only been using it with Autotagger, as the LMStudio logs in my next post show.
Still everything is gone from the database and I recall no activity that should cause this.

The restore from yesterday were to a completely different folder and I only opened the MainDB file with IMatch after closing the other DB.

[bekesizl]

Attached the LMStudio logs that clearly show, that AutoTagger was working on my files.

[Mario]

You forgot to attach the IMatch log file.

The LM studio log does only show that AutoTagger or some other application was using it. It does not show the data AutoTagger has received, if there were any errors or problems etc.

A log file in debug logging mode from an IMatch session where AutoTagger fails to store data in AI tags would provide some data to work with. No user ever reported that AI tags were lost, and why only AI tags? They are stored alongside all other tags in the database, why would this issue be so selective? And repeated physical database damage on top...

[bekesizl]

Unfortunately there is no IMatch log file as I don't know when the tags get lost.
When I run autotagger they are there. 
Also after I exit and restart IMatch they are still there. 
But they get lost somewhere on the way. 

Physical damage was a one time problem last weekend or start of this week. 

The problem is, that none of the from pack&go extracted DBs from before contain any of the custom tags.

If I upload a pack&go somewhere could you check if you see them or it is a problem with my computer? 

[Mario]

But they get lost somewhere on the way. 

Keep debug logging enabled and do backups of your database multiple times a day (just copy the database file into a new folder while IMatch is not running).

Run database diagnosis frequently (at least daily).

Make a data-driven category for the AI tags (e.g. for AI.description) so you can tell if there is data in the tag when you open IMatch and refresh this category.

This way you will notice (hopefully) when something wipes all AI tags from your database. I have no clue which operation could do this. The 3 default AI tags cannot be deleted from the database, only trait tags can. 

Looking at the database will probably tell me nothing you don't know. But I can have a look. Send me a link to my email #support with a link back to this thread (I get many emails every day).

[bekesizl]

I sent you links for two databases per e-mail.
Debug logging is enabled, but I haven't added the AI Traits tags to the DB again.

The 3 default AI tags are in the database and also contain data.


There are some formula categories and data-driven categories created by me when the Traits data was still there. I haven't checked this time where they are pointing. Last time they were pointing to some unkown JPG tags, see earlier posts.


Last time I recreated the AI Traits tags, they were showing duplicated in the Tag selector dialog, when I remember right until an IMatch restart.

[Mario]

Last time I recreated the AI Traits tag

You cannot create AI.description, AI.keywords and AI.location, these tags are managed by the database automatically. They are created once and cannot be deleted.

What exactly do you do?

[bekesizl]

I recreated only the "deleted" AI.HasText, AI.PeopleOnImage and the other AI Traits tags.
The standard AI tags were only populated by AutoTagger.

[Mario]

So, keep debug logging on, make at least daily copies of your database, and pay attention to if and when AI tags become empty again. Only then you will have a chance to figure out the reason. I have never seen this, no other user ever reported this, and this makes this very difficult. What you did before this happens (again) will be really important. I have no clue where even start to look for this.