IMatch crashes when running face recognition

Started by fisketjon, June 09, 2025, 11:45:50 AM

Previous topic - Next topic

fisketjon

First I tried to start face recognintion on all images in my Master->Canon Versioning collection 50.000+ images with "small faces", but the program crashed when I was moving the mouse pointer to the view menu trying to select to view the info & activity panel.
After a couple of restarts, and succeding to open the info & activity panel, I saw that it was expected to take 15 hours and chose to clear the file queue.
Then I chose a much smaller group of files, approximately 5.000, but IMatch still crashes.

When I restart IMatch it says "running face recognition" in the lower right of the window, but suddenly it just crashes.
Doesn't it check one image at the time? Shuoldn't it be possible to add 5.000 images to be checked?


Mario

When you say IMatch crashes, does it produce a DUMP file: The Debug Dump File ?
If no DUMP file is produced, the crash is caused by something outside of IMatch.

IMatch processes many files in parallel with face recognition, trying to utilize all available processor cores, within the limits of the current Performance Profile.

Face recognition is very robust and stable. I regularly run it with batches of 5,000 or 10,000 files on my workstation PC and laptop.

However, since it can utilize all processor cores for an extended period, it tends to reveal stability issues with the computer. Sometimes, computers fail under prolonged load, with all kinds of effects. We have seen this a number of times, especially related to face recognition. The PCs in question just could not cope, overheating or producing random errors after a short while.
The same PC can do Office or video watching or web browsing all day - no problem. But running all cores at 80% or more for a few minutes makes it crash.

You did not include a log file from an IMatch session where the crash happened, so we have nothing to work with (log file).

I suggest you change the IMatch performance profile (see link above) to "Balanced" or even "Low". This reduces the load face recognition produces, which maybe fixes this load-induced problem.

fisketjon

No Debug Dump File, but there are 6 entries in the Event Viewer under Windows Logs->ApplicationFaulting application name: IMatch2025x64.exe, version: 25.3.0.1, time stamp: 0x6810b0fd
Faulting module name: ntdll.dll, version: 10.0.26100.4061, time stamp: 0x1d4ecf98
Exception code: 0xc0000374
Fault offset: 0x000000000011e0e5
Faulting process id: 0x7EA0
Faulting application start time: 0x1DBD9229536A3F8
Faulting application path: C:\Program Files\photools.com\imatch6\IMatch2025x64.exe
Faulting module path: C:\WINDOWS\SYSTEM32\ntdll.dll
Report Id: 693804c0-c449-46f8-b426-df68988f076d
Faulting package full name:
Faulting package-relative application ID:

- <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
- <System>
  <Provider Name="Application Error" Guid="{a0e9b465-b939-57d7-b27d-95d8e925ff57}" />
  <EventID>1000</EventID>
  <Version>0</Version>
  <Level>2</Level>
  <Task>100</Task>
  <Opcode>0</Opcode>
  <Keywords>0x8000000000000000</Keywords>
  <TimeCreated SystemTime="2025-06-09T09:43:01.6529127Z" />
  <EventRecordID>31317</EventRecordID>
  <Correlation />
  <Execution ProcessID="16604" ThreadID="39848" />
  <Channel>Application</Channel>
  <Computer>nfpc2024</Computer>
  <Security UserID="S-1-5-21-1880448694-1230180008-1552581263-1002" />
  </System>
- <EventData>
  <Data Name="AppName">IMatch2025x64.exe</Data>
  <Data Name="AppVersion">25.3.0.1</Data>
  <Data Name="AppTimeStamp">6810b0fd</Data>
  <Data Name="ModuleName">ntdll.dll</Data>
  <Data Name="ModuleVersion">10.0.26100.4061</Data>
  <Data Name="ModuleTimeStamp">1d4ecf98</Data>
  <Data Name="ExceptionCode">c0000374</Data>
  <Data Name="FaultingOffset">000000000011e0e5</Data>
  <Data Name="ProcessId">0x7ea0</Data>
  <Data Name="ProcessCreationTime">0x1dbd9229536a3f8</Data>
  <Data Name="AppPath">C:\Program Files\photools.com\imatch6\IMatch2025x64.exe</Data>
  <Data Name="ModulePath">C:\WINDOWS\SYSTEM32\ntdll.dll</Data>
  <Data Name="IntegratorReportId">693804c0-c449-46f8-b426-df68988f076d</Data>
  <Data Name="PackageFullName" />
  <Data Name="PackageRelativeAppId" />
  </EventData>
  </Event>

Mario

This is not that helpful.
The faulting module reported is "C:\WINDOWS\SYSTEM32\ntdll.dll", which is a part of Windows.
No IMatch log file (see log file) to see what's going on.
Did you change the performance profile as I suggested?
Which kind of computer is this? Laptop / Notebook?

fisketjon

I found IMatch2025x64.exe.nnnnn.dmp files in AppData/Local/Crashdumps

I have set the Performance Profile to Low

Desktop PC


The latest entry in the IMATCH_LOG_BACKUP.TXT was made 23:28:29
The latest entry in the Windows Log was made 23:31:31

Mario

QuoteI found IMatch2025x64.exe.nnnnn.dmp files in AppData/Local/Crashdumps
When IMatch crashes, an internal crash handler is called which lets Windows produce a DUMP file as explained here: The Debug Dump File

If this does not happen, thew crash happened in something "outside" of IMatch, like a WIC codec, graphic card driver or something similar IMatch uses indirectly. Or the Windows/your virus checker has terminated IMatch for whatever reason. If the crash handler in IMatch is not executed, it could be anything. And, from experience, when IMatch is running face recognition, it's usually a system instability triggering Windows.

Telling me the time of the last entry in IMatch log file is fine. It would be much better if you could just ZIP and attach the log files to your posts so I can actually have a look. Maybe there are warnings or errors logged, pointing at the source of the problem.

fisketjon


Mario

The options are below the post editor. If you only use the quick post/reply editor, you need to click on preview once to see all the options. See https://wiki.simplemachines.org/smf/SMF2.1:Posting for detailed help.


 You cannot view this attachment.

fisketjon

#8
Attached the latest dump & log files

Mario

#9
I need the IMATCH6_LOG.txt. The *backup* file is from the previous session (not the crashed one).
It shows a normal IMatch start and initialization, a File Window loading, the folder monitoring system starting. No faces are processed.

When IMatch crashes again, ZIP the log file and attach it here before before restarting IMatch. Switch IMatch to debug logging via Help menu > Support > Debug Logging to tell IMatch to produce a more detailed log file. We need all the info we can get.

The small DMP file included in the ZIP does not tell me much. The crash happens somewhere in the bowels of NTDLL.DLL, which is a part of Windows. The top of the stack may indicate that the crash happened while IMatch was reading an image file (?) but it is inconclusive. A debug log file will tell me more.

fisketjon

#10
Set the performance option back to the default to make the crash come sooner.
The zip file is 9.4MB which is more than I am allowed to attach.

I have put the zip file on my web server at

http://fisketjon.no/dumps.zip

Mario

QuoteSet the performance option back to the default to make the crash come sooner.
This is interesting, since it tells me that this is most likely an system instability under prolonged heavy load problem.

The log file contains a number of warnings, e.g.

06.10 15:45:33+    0 [11D54] 01  W> PTRaw: Failed to delete temp file C:\Users\njalf\AppData\Local\Temp\14302045315724.imt with 32  'V:\develop\IMatch5\src\ptpicore\PlugIns\ptpipip\PTLibRaw.cpp(368)'
06.10 15:45:33+    0 [12B80] 01  W> PTRaw: Failed to delete temp file C:\Users\njalf\AppData\Local\Temp\14302045315724.imt with 32  'V:\develop\IMatch5\src\ptpicore\PlugIns\ptpipip\PTLibRaw.cpp(368)'
06.10 15:45:50+    0 [12B80] 01  W> PTRaw: Failed to delete temp file C:\Users\njalf\AppData\Local\Temp\14303717111942.imt with 32  'V:\develop\IMatch5\src\ptpicore\PlugIns\ptpipip\PTLibRaw.cpp(368)'
06.10 15:45:53+    0 [137A8] 01  W> PTRaw: Failed to delete temp file C:\Users\njalf\AppData\Local\Temp\1430401875436.imt with 32  'V:\develop\IMatch5\src\ptpicore\PlugIns\ptpipip\PTLibRaw.cpp(368)'


Error 32 means "Error 32: The process cannot access the file because it is being used by another process."

Which virus checker do you use? Try to make an exception/exclusion for the folder (!) containing the database and for the 
"C:\Program Files\photools.com\imatch6\IMatch2025x64.exe" executable. 

If IMatch cannot delete temporary files it has created, the most likely "other process" locking them is a virus checker or similar "protection" software.

Looking at the end of the log file, IMatch was happily loading images from the cache, extracting face vectors, storing stuff back into the database. No errors, no warnings (except the ones above). About 1 second per file on average, with many files processed in parallel.

Your system reports 24 cores, which is a beefy machine. But maybe it's not stale under load...? Maybe try the "Minimal" performance profile. Here IMatch massively limits how many processor cores it uses.

fisketjon


Not 24, but 12 cores



I'm using Microsoft Defender 

Mario

Quote from: fisketjon on June 10, 2025, 09:01:16 PMNot 24, but 12 cores
24 with HT.

Add an exclusion to Defender and see if this changes anything. Defender gets rarely in the way, but it has done so in the past.