Conversion fails

Started by densac, January 21, 2016, 09:31:20 PM

Previous topic - Next topic

densac

Attempting to convert a version 3 database to 5 but it fails every time somewhere towards the end of processing metedata..the back end of the conversion log says this:

ADMIN: Log file data removed and put into a text attachment. Posting this amount of 'nonsene' data fills up the community search engine with junk and makes it less useful for all users. Please always attach log files.

and the windows event log:

Faulting application name: IMDBConverter5.exe, version: 5.6.0.10, time stamp: 0x569e1955
Faulting module name: ucrtbase.dll, version: 10.0.10240.16384, time stamp: 0x559f3aa5
Exception code: 0xc0000409
Fault offset: 0x0007d84b
Faulting process id: 0xc8
Faulting application start time: 0x01d1546936df7b7b
Faulting application path: C:\Program Files (x86)\photools.com\IMatch5\IMDBConverter5.exe
Faulting module path: C:\WINDOWS\SYSTEM32\ucrtbase.dll
Report Id: 36420c60-e941-4383-bd73-819bcd6a61f7
Faulting package full name:
Faulting package-relative application ID:

Does this information give a clue to the problem?

[attachment deleted by admin]

Mario

Is this a log file from database conversion?
Can you please attach the full log file and also the database converter log?
The help for the database converter lists the name of these files and where to fetch them.

Tip: ZIP them to reduce their site by 80% to 90%.

IMatch is allocated a whooping 3.5 GB of memory during the conversion. I cannot say more at this time because you included only a smart part of the log file. I cannot even tell if your database has 10,000 or 300,000 files...

Also, a crash happens in the ucrtbase.dll DLL, which is a DLL provided by Microsoft. This may be a side effect.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

densac

End of the conversion logfile attached...

[attachment deleted by admin]

Mario

The database converter produces 3 log files.
Please open the database converter help by pressing <F1> while the converter is open or search for convert in the IMatch help index.
Press <Ctrl>+<F> and search for log files.

I need the complete log files, not just a snippet from the end. There is important information at the beginning of the log file. I cannot even say if you convert 5000 files or 500,000 and you are giving me almost no information.

If you want help, please supply the information I request. If the log files are too large to attach, upload them to your cloud space or send them to me via email. See the link below in my signature for details.


Your system is running out of memory.

On 14:28:29 the converter is using only 548 MB. IMatch is importing metadata from your files.
The last files processed are

01.22 14:28:35+    0 [31F4] 02  I>                   [269126] 182 tags 'D:\RAW Files\150706\200\150706-113721-200-1234.CR2'
01.22 14:28:35+    0 [31F4] 02  I>                   [269127] 182 tags 'D:\RAW Files\150706\200\150706-113722-200-1235.CR2'
01.22 14:28:35+    0 [31F4] 02  I>                   [269128] 182 tags 'D:\RAW Files\150706\200\150706-113739-200-1236.CR2'
01.22 14:28:35+    0 [31F4] 02  I>                   [269129] 182 tags 'D:\RAW Files\150706\200\150706-113739-200-1237.CR2'
01.22 14:28:35+    0 [31F4] 02  I>                   [269130] 182 tags 'D:\RAW Files\150706\200\150706-113740-200-1238.CR2'
01.22 14:28:35+    0 [31F4] 02  I>                   [269131] 178 tags 'D:\RAW Files\150808\6400\150808-220450-6400-5276.CR2'
01.22 14:28:35+    0 [31F4] 02  I>                   [269132] 178 tags 'D:\RAW Files\150808\6400\150808-221525-6400-5306.CR2'
01.22 14:28:35+    0 [31F4] 02  I>                   [269133] 178 tags 'D:\RAW Files\150808\6400\150808-221623-6400-5313.CR2'

On 14:29:02 (3 seconds later) IMatch is using 3.2 GB (!) and is running out of memory.
It seems that there is some metadata in these files or some other conditions which causes IMatch to allocate a massive amount of memory.

I suggest you temporarily remove these files and retry the conversion. Since you still don't send me the full log file yet, I cannot say if IMatch was running for 3 minutes or 10 hours, how many files were processed etc.

If the conversion takes long, do it manually.
Create a new database in IMatch 5, add your files in batches.
This way you have more control over the process, and when IMatch runs into the same problem again, all files added before will remain in the database.
You can then remove the problematic files and add all other files until completed.

If everything has been added successfully, export the categories for your in IMatch 3 and import them into your new IMatch 5 database.
If you also have used properties (Title and Description), you can also export them in IMatch 3 and import them into IMatch 5.
Let us know when you need help with that.

-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

lightchaser

... I agree with Mario. Also had big problems to migrate my database with about 200.000 images. Did it the way Mario suggested and it worked just fine.
Franz

http://www.xdreamphoto.com/

densac

Still using 3.6.0.114 2 years later :-\. Have tried many conversion attempts with different elements of my database but the conversion always fails on the 'out of memory' problem. The database in question is 307258 images in size, the greatest percentage of which reside off-line on 398 DVD's/Blue Rays. This makes it impractical to create a new database and add all my images to it. I even tried removing the on-line images and converting the off-line images on their own and, though this operation is faster than converting the full database, it still terminates on the 'out of memory' error. Strange thing is that when 5 was initially released my first conversion was successful but I wasn't ready to move forward with it at that point...

Mario

No support is available anymore for IMatch 3 or IMatch 5.

The conversion would not be faster than adding your 300K files to a new 64-bit IMatch database.
The slowest part is the metadata read, and this requires all files to be on-line anyway.

If you have 400 DVDs that makes only 1.8 TB in storage.
I know than in earlier days DVD was the only viably and affordable bulk media. But these times are over for a long time.

A fast 2 TB 3.5 hard disk with USB 3.0 costs about 70 US$.
Take 2 to have a full mirror backup.
(I use this for backup and they are just great).

DVDs are known for bitrot after a few years and if your DVDs are older than 3 to 5 years you have to copy them to new DVDs anyway to avoid degradation.
If you copy all your DVDs on a modern 3.5 inch external USB 3 disk, you win a lot of speed and data security. And of course IMatch will just happily scan in all your files from that disk.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook