Issue of thumbnails and Quick view cached images

[Lucio.B]

Since  i started to use a Sony camera I have problems with cached images.
Sometimes thumbnails and Quick Vie display a banded mixture of two files.
This happens either with ARW or DNG copies of the same file.
If I try a  normal rescan nothing happens, with a cache only rescan i get the Quick View image fixet but not the thumbnail, the only way to get everything fixed is to run a forced rescan. Next time I open the sane directory i still have some new issues.
The whole thing is quite time consuming.
Examples attached.
Thanks, regards.

[Mario]

How strange. This does not look like a mix of images, but more like JPEG decompression errors or some sort? 

The log file contains many warnings about bad IFD directories in DNG files, e.g. 

Warning: [minor] Bad IDC_IFD SubDirectory start - A:\archivio foto\2025\07\street\A7_01997.DNG

This may indicate an issue with the metadata / EXIF record stored in the DNG file.

When you do a forced rescan (which updates both the cache and the thumbnail) it works?
Then you experience maybe a "stress" issue? Your system reports 16 processors and IMatch will utilize them all when ingesting files. Maybe this causes too much stress on your system and produces random errors? Maybe this somehow only affects Sony files or DNG files? IMatch uses the official Adobe DNG library to read DNG files via LibRaw, maybe the Adobe code has issues with the DNG flavor produced by the SONY cam? No idea.

If you have a set of files that produce this error when IMatch is indexing them, that would be great!

Copy the files from your test into a new folder (in Windows Explorer, not IMatch) and then add that folder to the database. Does the problem happen again? Awesome.

Use the "Remove Folder from Database" command in the folder's context menu.
Go to Edit menu > Preferences > Application: Search for "profile" and set the performance profile setting to Low (Default is Performance). Then add the folder again to the database. The Low performance profile will put less stress on your PC and maybe the result is now different?

When you have a set of images that always produces this problem, upload them to your cloud space (Dropbox, OneDrive, GoogleDrive....) and send a public link to me via email (see support email address) and mention topic 15342 in the email so I know what the email is about.

[Lucio.B]

When you do a forced rescan (which updates both the cache and the thumbnail) it works?

Yes it works

Copy the files from your test into a new folder (in Windows Explorer, not IMatch) and then add that folder to the database. Does the problem happen again? Awesome.

If I copy the faulty files to a new directory from inside iMatch the problem persists.
If I do it from Windows File Manager then I add the folder to the database the files look normal. Tested on a 5 files set.

Go to Edit menu > Preferences > Application: Search for "profile" and set the performance profile setting to Low (Default is Performance). Then add the folder again to the database. The Low performance profile will put less stress on your PC and maybe the result is now different?

Yes, if i follow your indication the result is different and the files in that directory look normal. Does this procedure affects much iMatch performance?

When you have a set of images that always produces this problem, upload them to your cloud space

Ok I've done it.

I forgot to tell you that, as I don't use Adobe Lightroom, I make the file conversion directly from Adobe DNG converter. Do you think that changing the compatibility in Adobe DNG converter with an older version could help?

Thanks Mario, regards.

[Mario]

If I copy the faulty files to a new directory from inside iMatch the problem persists.

This is to be expected. When you copy files within IMatch, IMatch also copies the thumbnail and cache images, it does not re-generate the. This is much better for performance.

If I do it from Windows File Manager then I add the folder to the database the files look normal. Tested on a 5 files set.

Yes, if i follow your indication the result is different and the files in that directory look normal. Does this procedure affects much iMatch performance?

These two tests indicate that there is a problem with the PC under prolonged, heavy load. IMatch processes many files in parallel when it scans folders for new or update files, and, rarely, this causes "stress" issues with some PCs, which become unstable and produce all kinds of random errors. The screen shots look like there was an issue writing the JPG files, causing corruption of some sorts. I have not seen this particular problem before, though.

If setting the performance profile to a lower setting solves the problem, good. Unless you process tens of thousands files regularly, you should not notice much of a difference. Maybe your PC is even faster now, when it is less "stressed".

I forgot to tell you that, as I don't use Adobe Lightroom, I make the file conversion directly from Adobe DNG converter. Do you think that changing the compatibility in Adobe DNG converter with an older version could help?

I pretty much doubt this. IMatch processes DNG files with the latest version of the official Adobe DNG software development kit which is supposed to handle all over now 50 DNG variants in use.

[Mario]

I have downloaded the sample images, thanks.
I've tried to reproduce the problem you were experiencing by adding the files to an IMatch database.
I tried this on my workstation PC, on my notebook and on a fresh Windows 11 install in a virtual machine.

All images loaded fine, without any issues or corruption of thumbnails or cache images.
ExifTool reports  "Warning: [minor] Bad IDC_IFD SubDirectory start" for each image, but we know this already.

What I find interesting is that the official Windows WIC codec for DNG reports 

Thumbnail: Codec 'DNG Decoder'
(GetThumbnail failed (88982F44 The bitmap codec does not support a thumbnail.).) 0x0 pixel in 0 ms.
Preview: Codec 'DNG Decoder'
(GetPreview failed (88982F81 The operation is unsupported.).) 0x0 pixel in 0 ms.
Full resolution: Codec 'DNG Decoder'
() 1024x683 pixel in 16 ms.

in IMatch WIC Diagnostics. It seems to be only able to extract a small 1024 x 683 pixel image as the "full resolution" image. Strange. So much for DNG and standard.

LibRaw, which utilizes the Adobe DNG SDK code I've included in IMatch 2025+ shows 

Preview: (Format:1, Dimensions: 1024x683) in 0ms. Result: 0
Original Image: (6048x4024 6048x4024) in 2984ms. Result: 0

which looks better.

Since the preview size is smaller than the cache size I have configured under Edit menu > Preferences > Cache, IMatch pulls the full RAW data from the DNG file, which causes the images to look a bit flat and dark. I can remedy that by opening the image in DNG converter, apply a development setting and save the image again as DNG with as 100% preview.

That's all I can say. The images seem to work just fine, even on my notebook and in a rather slow Windows 11 VM.

[Lucio.B]

Thank you for your prompt answer.
So I will check my machine behaviour with a low performance setting for the next weeks as I'll be shooting some thousands of pictures.
One more thing, I've noticed that in Adobe DNG converter preferences i have as default: JPEG Preview <medium size>  and Embed fast load data unchecked. (see attachment)
Do you think these setting could interfere with cache image building/previewing?
Regards

[Mario]

The medium size is only 1024 pixels. Which means that IMatch usually cannot use this (the default cache size is 100%, see The Cache) and IMatch will use Adobe DNG and LibRaw routines to "develop" the full RAW data and use the result for creating the thumbnail and cache image. The results are way worse than what a RAW processor or even the DNG Converter can produce.

If you have a DNG-based workflow and you must not conserve disk space, embedding 100% previews in the DNG converter is always better. 1024 pixel is really not much.

The fast load data is something Adobe added to support their own applications. It adds another "preview" which allows Lr and other Adobe software to process the files faster. Some kind of proxy, I believe. You must ask Adobe about the details.