AI Autotagger stalls in 'media & folders' view

[mity!]

Beginning (I think) with v2025.1.10 I can observe and reproduce this behaviour with AutoTagger:

• I select an bunch of files in the 'media & folders' view (typically a whole folder containing about 100 files) and start AutoTagger. AutoTagger is configured for OpenAI which worked fine so far (up to v2025.1.8 probably but at least v2025.1.6).
• All files get the "AI in progress icon". First file will complete within seconds (AI icon vanishes) and progress shows "1 of x processed". And then the progress stalls.
• I can see that 2nd an 3rd file have AI Tags received, even the AutoTagger category is already set, but AI icon is yet shown and AutoTagger progress shows continuously "1 of x".
• When I switch to any other tab (categories, timeline...) progress starts to go on again and AutoTagging finishes within some minutes. This even happens if the other tab shows the same images as the 'media & folders' tab.

I don't know if it has to do with this behavior but perhaps it is some more clue to you: While AutoTagger is running I can see a lot of activity in updating categories and metadata visualization. So in categories tab many categories constantly update (switching continuously between 0 and the recent number of assigned images). I have no auto-driven AuoTagger categories for the AI keywords configured.

Also in the file view of 'media & folders' there is some kind of "pumping" during AutoTagger is running, meaning that all the images icons (for flags, bookmarks, GPS etc.) get slightly bigger and shrink again to their normal size. I then realized that even the slider in the menu bar for the size of the preview images slides slightly to the right for a moment during this "refresh". But the size of my previews does not change.

[Mario]

Always include a ZIPped log file (preferably in debug logging mode) when you report such issues.
See log file for more information.
This gives us a minimum of information to work with, e.g. responses sent by OpenAI, headers with rate limit data etc.

I've made a test, with a set of 50 and 120 files and ran AutoTagger on them using OpenAI.
All files were processed without any problem.

Maybe the info in your log file tells us more. Maybe OpenAI just asked for a pause via a rate limit or something. Many possible reasons.

Do you see the "AutoTagger: Files with Problems" notification?
Was IMatch busy ingesting files or doing face recognition at the time (See Info & Activity Panel). These operations pause  AutoTagger until finished.

If this happens again, secure the log (Help menu > Support > Copy application logfile) and reopen the database. AutoTagger should continue after a minute or so.

That IMatch is updating categories and collections and metadata while AutoTagger is running is normal. Metadata changes, @Keywords must be updated, the Workflow categories are possible affected (depending on which tags you use etc.).

[mity!]

Well...

Wanted to produce a debug log just now. So started IMatch (v2025.1.10). First OpenAI access today, so no issue with tier 1 limits possible. Selected 160 images, started AutoTagger - fault as described (AutoTagger stalls with "1 of 160" and in fact three images already with AI keywords - tagging goes on when switching to another tab).

"Ahhh, forgot to change to debug mode." Switched to debug mode when Imatch told me about new version. Okay, installed v2025.1.12.

Next try with v2025.1.12. Debug on. Selected next folder with 167 images. Started AutoTagger and... no problems - proceeded all images within 2 minutes or so. Selected next folder - and next folder: no faults any more.

I will keep an eye on it... 

[Jingo]

Ok.. so, I just had a similar thing happen to me today... AI process never kicked off though.

1 - I opened IMatch and imported a new folder of images
2 - I then selected 13 images and used the Autotagger for a single image run.  Thumbnail received the AI symbol.

I waited and waited and waited but the autotagger process never would start.

3 - I switched to Debug logging and chose another image to Autotag.
4 - The thumbnail also got the AI symbol but nothing happened... log didn't show anything related to AI.. just FolderSweepers.
5 - I switched to the Categories tab and then back to Media & Folders... and the autotagger kicked right on and did its thing.

Log attached (only partially in Debug mode)... hope it helps narrow this down to something. 

On a side note - I think it would be great if we had an "Autotagger" panel or something that shows what is going on with the autotagger... I'm never sure if things are not running because of a rate limiter, broken process or if IMatch is just busy doing something else. I always want to see what is going on "under the hood".. and the Info & Activity panel just doesn't provide those details all the time.

[Mario]

The log reports that the last IMatch session did not close correctly. Did you have a crash or something?

I see the following entry:

An AutoTagger run is requested at 05.11 19:41:53 for one file. The request is enqueued.
Cache files are loaded. Data is requested from IMWS, GPS coordinates are loaded for files, more cache files are loaded, more data is requested from IMWS, more cache files loaded,  File Window reloaded.

Then comes the interesting bit:

05.11 19:43:47 AIRateLimiter: Request rate limit expired for [900] gemini-2.0-flash-lite. Was 1899-12-30T00:00:00.0
[117029]AIRateLimiter: Token rate limit expired for [900] gemini-2.0-flash-lite. Was 1899-12-30T00:00:00.0

The 1899 timestamp is the "unknown/unset" timestamp used to indicate that the rate limit was not set before and it will be ignored.

05.11 19:43:47+    0 [3E50] 02  I> AutoTagger: [117029]

AutoTagger is starting to process a JPG file loading the image from the cache/disk

At 05.11 19:43:48 AutoTagger sends the first request to Gemini, then follow 5 other requests, the last one at 05.11 19:44:10.
Then follow some internal updates, collection recalcs, metadata is searched (e.g. for categories), files are loaded into the Viewer or QVP etc.

At 05.11 19:41:53 you ran AutoTagger for a file and IMatch enqueued it.
At 05.11 19:43:47 the update queue is temporarily disabled and re-enabled. This is not unusual and happens often when things change that might affect the update queue.
At 05.11 19:43:47  AutoTagger starts processing. (Enabling the queue triggers processing)

So what happened between  05.11 19:41:53 and  05.11 19:43:47?
Many files were loaded from the cache (does not affect the queue), Quick View Panel loading images, data requested from IMWS several times (app open? Map Panel open), all normal stuff when you navigate in a File Window, use apps, use the QVP etc.

I cannot tell why the queue did not process the AutoTagger entry immediately. The call to AddEntries at 05.11 19:41:53  adds the request to the queue and immediately triggers the responsible background thread(s) for the type of entry. The threads fetch the next entry for "their" type from the queue (if any) and process it.

The Info & Activity Panel shows the number of entries. Did it show a 1 and the number never changed?

We need some case or steps to reproduce this. I have worked with AutoTagger for many hours yesterday, processing my own new images and running some tests and I have not experienced this.

I think it would be great if we had an "Autotagger" panel or something that shows what is going on with the autotagger... I'm

We have. When the AutoTagger is processing (or IMatch is indexing files, or processing faces), the Info & Activity Panel, the IMatch status bar and the Dashboard show progress indicators, with progress and an estimate for the remaining time.

There is not much more IMatch could reveal. Constantly running background tasks like category updates, prefetch, folder monitoring etc. is not presented in the UI. Would make no sense.

[Jingo]

Hmm... it just happened again.

All I did was:
1 - select a single photo and Alt-F7
2 - opened the info&activity Panel and noticed the 1 item queued (missed this before because I was looking for "Autotagger" Queued or something - should have read the help I suppose [which I typically do]).
3 - waited a bit.. nothing happened

4 - selected another photo and Alt-F7
5 - Queue # did go to 2.. but nothing runs.

6 - After waiting a couple of minutes - I selected a 3rd photo, did an F7 (instead of Alt-F7) and all three images immediately processed (and now I see the Autotagger status bar in the I&A panel).  I also see the Purge-Stats right before the Autotagger seems to run.. coincidence?

Debug log (yesterdays and today since IMatch remained on) attached again.  I hope it helps....

[Mario]

Which rate limits have you entered for Gemini (E > P > AutoTagger)?
Check https://ai.google.dev/gemini-api/docs/rate-limits

[Jingo]

I am using these (defined on that page):

You cannot view this attachment.

You cannot view this attachment.

[Mario]

That's the free tier, right?
There is no SLA, which means Google can refuse requests for any reason. But we would see that in the log as errors.

I've worked with AT today again, using OpenAI, Mistral, Gemini and Gemma 3 with LM Studio. It has not stalled once and I'm working in the Media & Folders View.

When I have some extra time, I will analyze yet another log file. I've analyzed the two million row logfile from Mees yesterday and your's today and now I'm a bit sick of it.

Since this is reported only by you, the OP is from February, this is probably something with your database or the operations you perform.

[Jingo]

Thx Mario - yeah.. free tier but I too would expect to see some "communication" with the API if this was a refusal/rejection... from an IMatch persepective, it just looks like the requests don't occur until some other "process" causes the system to trigger it. 

If I can do anything to try and troubleshoot or reproduce again, please let me know... 

For what's its worth, I just closed and relaunched IM, chose a different folder, selected a file, Alt-F7 and the system immediately processed the request.  Oddness... 

[zuiko]

I'm seeing something similar although under difference circumstances.  I'm using a paid tier from OpenAI and currently my internet is very unstable due to problems with my provider. 

My connection will randomly go down for 1-2 minutes and when that happens IMatch correctly reports that it is having problems communicating with OpenAI.  However, when my connection comes back IMatch never restarts processing the queue.  I've left it for hours and nothing moves.  Re-opening the database gets the queue going again.

Maybe cutting your internet connection for a few minutes while processing the queue will duplicate the problem.

[Mario]

However, when my connection comes back IMatch never restarts processing the queue. 

I have just fixed a glitch last Sunday that may be responsible for this. See release note 2814 on https://www.photools.com/release-notes/.

[Mario]

So far I did not see anything helpful in the log. This is what you did:

Code Select Expand

05.11 20:23:50+  I> AutoTagFilesInQueue: RunOnce for file [117067] and 92 selected files.
05.11 20:23:50+  M> >  0 CIMEngineUpdateQueue::AddEntries
05.11 20:23:50+  M> >  0 CIMEngineAIAutoTagger::AutoTagFiles
05.11 20:23:50+  M>  <  0 CIMEngineUpdateQueue::AddEntries
05.11 20:23:50+  M>  <  0 CIMEngineAIAutoTagger::AutoTagFiles

05.12 07:40:16+  I> AutoTagFilesInQueue: RunOnce for file
05.12 07:40:16+  M> >  0 CIMEngineUpdateQueue::AddEntries
05.12 07:40:16+  M>  <  0 CIMEngineUpdateQueue::AddEntries

05.12 07:40:44+  I> AutoTagFilesInQueue: RunOnce for file
05.12 07:40:44+  M> >  0 CIMEngineUpdateQueue::AddEntries
05.12 07:40:44+  M>  <  0 CIMEngineUpdateQueue::AddEntries

05.12 07:45:55+  I> AutoTagFilesInQueue: RunOnce for file [117145] and 1 selected files.
05.12 07:45:55+  M> >  0 CIMEngineUpdateQueue::AddEntries
05.12 07:45:55+  M> >  0 CIMEngineAIAutoTagger::AutoTagFiles
05.12 07:45:55+  M>  <  0 [16ms] CIMEngineUpdateQueue::AddEntries
05.12 07:45:55+  M>  <  0 CIMEngineAIAutoTagger::AutoTagFiles
The first run worked.
The second and third run did not work.
The forth run worked again.

With "work" I mean that the queue fired up an AutoTagger queue straight away. The AddEntries function adds a new entry to the queue and triggers the responsible threads to do the  work.

I see no active rate limits, the queue is not disabled or paused. Nothing that would give me a hint about why this does not work for you. I cannot reproduce it so far.

[Jingo]

Well.. thanks for trying Mario.. I'll keep monitoring things to see if anything I do can trigger it.  Changing to another tab or closing/restarting IMatch tends to fix to it so that is an easy thing to do.

Appreciate your efforts!!

[Mario]

Just reopen the database. This will "unstuck" things.
I can fix this when I can repro it.

[Jingo]

I know you can't fix it if you can't reproduce it.. but this seems to be happening to me a lot lately. 

Just happened again to me during an Autotag session (2nd request from Alt-F7 - the first one ran immediately). 

Here are the steps - debug log attached too:

• Select 9 images
• Hit Alt-F7 (seems to be a key part of this?) and runOnce for the selected images

          5.29 16:10:31  AutoTagFilesInQueue: RunOnce for file and 9 selected files in log

• Wait 30 seconds and nothing happens
• Check Info Panel and see 1 entry in the queue but DB idle
• Wait another minute .. nothing happens - just foldersweeper activity
• Click the next photo - log shows next photo loaded but Autotagger still not running
• Finally - click the Category Tab and then click back to Media & Folders tab... 

        Autotagger finally starts running....
           05.29 16:13:28 UpdateQueue enabled: 1 by thread 4756

So.. something is causing the UpdateQueue to not enable...? Perhaps something is not updating when Alt-F7 is used?

You cannot view this attachment.
You cannot view this attachment.

Hope the log helps ... screenshots too.  Thx! - Andy.

[Stenis]

However, when my connection comes back IMatch never restarts processing the queue. 

I have just fixed a glitch last Sunday that may be responsible for this. See release note 2814 on https://www.photools.com/release-notes/.

Is this something you are planning to release or is this question settled at least for Open AI with what Jingo has written about Gemini in this tread. Even Open AI had choking settings for AI "Rate Limits" in Autotagger? I did not see the same though at least in my case for the Google Gemini which had much higher values by default i think.

After changing the default settings for OpenAI I don´t seem to see these problems anymore so what is the fix you have done in this new version when it seems to have gone with a change of "Rate Limits". I´m using a paid plan and not the free one.

[Mario]

This is a release note for a yet unreleased version. I ship updates about every 8 weeks.

[Jingo]

Interesting... I missed this "fix" in the thread and wonder if it *is* the resolution to my issue as well?  

I don't have any issues with internet connections (rock solid Fiber optic) but if it doesn't retry for some reason like say rate limits, that could indeed be my issue because the entry does not process off the queue unless I force IMatch to do something like switching tabs or restarting the program.

I'll hold off on further testing until the update is released... Thx!

[Jingo]

Just installed the latest release and tested this out again.  I did a bunch of successful AutoTagger operations in a row and was very excited to call this one fixed.... unfortunately, the very last Alt-F7 I did produced the dreaded Hang/Queue again... AI would not process the image from the queue until I forced it by switching tabs again.  Bummer!

Debug Log attached.

[Mario]

Can you please attach this info to your open bug report? This way we have things in one place:

https://www.photools.com/community/index.php?msg=106902

[Jingo]

Can you please attach this info to your open bug report? This way we have things in one place:

https://www.photools.com/community/index.php?msg=106902

Whoops!  I didn't realize this wasn't my post.. that's what I get for searching through to try and find it!  Rookie Mistake  

All set now...

[Jingo]

So.. this continues to happen for me and I think I'd like to try and build a new database to rule that out.  Stay tuned for a test run on a new database!

[Mario]

Also consider the panels you have open, maybe custom apps running etc. If something happens only on one PC, the trick is to find out what makes this PC so special. I have never experienced this, I tried to reproduce it several times, but I just don't see this effect...and this means I cannot fix when something's not working.

[Jingo]

Also consider the panels you have open, maybe custom apps running etc. If something happens only on one PC, the trick is to find out what makes this PC so special. I have never experienced this, I tried to reproduce it several times, but I just don't see this effect...and this means I cannot fix when something's not working.

I'll try a clean workspace on my current DB first just in case something is interacting with it... no apps.. no panels open other than Media & Folders.

[Mario]

If you can reproduce it, keep the logging in debug logging mode (Help menu > Support) and retain the log file. Hopefully I can then see something in the log that's causing this.

[Jingo]

Also.. sorry.. posting in wrong thread again!  DOH!!!