Trying to create a working database for 3 days

[andyk]

I’m currently using the trial to see if iMatch will work for my needs. I really want to like it, but it crashes almost every time i try to create a database. I’ve been trying for 3 days, with new database searches taking 3+ hours. I have a huge library of photos. I have been a photographer for 20 years. I’ve broken it down to smaller chunks and was able to get one database created successfully. I tried to add to it and it crashed again. It crashes at the end of adding the new images and won’t reopen with that database, even after an attempted repair of the database. I’ve created a second database that is smaller than the first and it crashes. I’m starting to think there is something in that folder that is causing the crash.

What is my next step? Should i post the log here? Is there a maximum amount of photos that can be added to a database?

Thanks,
andy

[Mario]

This sounds strange.
Usually, even if IMatch crashes when ingesting a badly corrupted file, IMatch starts again and continues with the ingest process without any problems.

You did not give us much to work with in this post...

"What do you mean with "huge library" of images? 200,000 files? 2 million files?
On what kind of computer to you run IMatch?
Which file formats are you using?
On which kind of disk did you create the database?
Are your files on a local or locally attached disk or on a network server or NAS?

Please always ZIP and attach the log file (see log file) from the session where you experienced problems.
The log file tells us how IMatch is performing, what it was doing when the problem happened etc.

Note: Before you repeat your test, switch IMatch to debug logging (Help menu > Support).
When IMatch crashes again, make a copy of the log file before restarting IMatch. ZIP and attach to your reply.

IMatch also produces a DUMP File in the case of a crash, to help with the error analysis.
Does it create this file on your machine?

You write "and won’t reopen with that database".
What do you mean by that? Does IMatch display an error message (and which) when you attempt to open the database?
Does IMatch just close again when you open the database (which produces useful info in the log file and usually also a DUMP file)

It is not uncommon that IMatch crashes when a new user ingests 200,000 files, produced over a period of 30 years, with a dozen cameras and manipulated with 20 different applications.
Corrupted files may crash ExifTool or the WIC codec or IMatch.
But usually IMatch just continues up with the next file after you restart it.
That it even fails to load the database indicates a very severe issue.
The log file will tell us more.

[andyk]

You are correct, i didn’t provide much for the post, just looking to figure out where to start and what the normal limitations are. Your response was helpful. I’ll setup the debug logging and provide a zipped log file next time it crashes if can’t figure out the problem.

Since my post, i have been ingesting chunks of folders. I have an idea of where the problem folder is and have successfully been importing stuff around it. I’m closing in on that area. Can i merge databases, or will i need to start over after i’ve identified the problem?

The total library will probably be in the 200-400k files range.

In addition to whatever files are automatically selected, i’ve added NEF and DNG files.

The computer is an old i5-4670K with 16GB of ram running Windows 10. It’s my file storage computer with around 10TB on about 6 HDDs. The 128GB SSD runs Windows and the apps, but all the cache and databases are on a separate drive. All disk locations are on local drives.

As for the inoperable databases... When i open iMatch, it does fine. When i select the bad database and attempt to open it, it closes without loading anything or providing any error. If i can’t figure it out, i’ll look for the dump file and post that along with the log file.

Thank you for some direction.

[Mario]

When IMatch immediately closes while loading a database, something is definitely broken.
It may be the database (rarely) or it may be something in a setting related to the database that was damaged due to the crash (also rarely).
Since this is so rare, I don't have much to work with. The log file from such a session (copy it before restarting IMatch) will tell me more.

200-400K files is not much of a challenge for IMatch, once the files have been indexed.
If you can identify a folder with files causing the problem, very good.

Some general tips:
There is no merge for IMatch databases.

1.  I recommend yo make a backup of the database after each "chunk" of files. Just Ctrl+C, Ctrl+V the file with the .imd5 extension in Windows Explorer.
Then continue.

2. Make sure the database is on your fastest disk. A SSD is preferred, if you have enough free space on it. But a 128 GB SSD is quite small so I don't know.

3. Drives connected via USB sometimes have "issues" under prolonged loaded, as produced by IMatch when it keeps them very busy while indexing hundreds of thousands of images.
You can use the Process Control (Advanced Setting) options to reduce the number of parallel threads of execution IMatch uses.
Set all four options to 2 to reduce the processor utilization and disk load. This often does wonders when a computer becomes unstable under prolonged full utilization. The i5-4670K has four cores and a value of 2 will reduce the utilization by half(ish).

For all else, without a log file, I cannot tell you more.