Why not default to iMatch Libraw 'YES'? ;)

Started by MrPete, May 08, 2025, 01:24:32 PM

Previous topic - Next topic

MrPete

I just finished diagnosing some new thumbnail blanks, due to WIC errors. Learning more than I ever wanted to know about that mess... I learned:
  • Nikon no longer has a NEF Codec available for download :(
  • The old one we had installed now gets errors in some situations in iMatch, producing (generic icon-only) blank thumbnails (error: wic copypixels failed 80070057 -- giving a 0x0 thumbnail in the file window)
  • The Microsoft NEF WIC codec gives fuzzy thumbnails in the file window, and in reality it is based on LibRaw but hasn't been updated in years. :(
  • iMatch's use of LibRaw works great, and produces good clean thumbnails

Given the above, why not default the iMatch LibRaw setting to YES instead of NO? ;)

Mario

#1
LibRaw is the default for all new installations since IMatch 2025.

QuoteNikon no longer has a NEF Codec available for download
I've read that on Nikon Rumors.
A bad move for Nikon customers. But they were the last to offer one.
I wonder why Nikon/Canon/Sony/Olympus not cooperate with the LibRaw developer, provide him with technical data and some annual funding. I hope Microsoft does so, too.

If LibRaw dies, the only way to work with RAW is Adobe/DxO/C1. Affinity uses LibRaw.

Another argument for making DNG a requirement for all cameras. If Android/Apple can do it for billions of devices, Canon/Nikon/... can surely use DNG as well. Requires pressure by paying customers and probably some laws.

mopperle

If LibRaw would die, only options would be Adobe and C1. DxO is for me and many others no option since they refuse to support DNG files from modern smartphones. E.G. Apple got its ProRAW format which is packed into a DNG file. But DxO support for modern smartphones stopped some years ago and they constantly ignore customer requests.
This and some other reasons made me stick to LrC. Users not willing to spend money on a RAW developer then need to stick to software they get for free from the camera vendor,. Like Canon DPP or Nikon Studio NX etc.

Mario

I dunno why they don't support DNG.
The implementation is open source and cross-platform. They can fetch the Bayer matrix from the DNG and do their normal RAW processing as they do with other RAW formats. RAW files and DNG files share the TIFF file structure.

I've spent some effort and time to include the Adobe DNG toolkit in IMatch 2025. Together with it and LibRaw IMatch supports the DNG variants with JPGXL previews, which Adobe introduced for Lightroom. For "reasons".

I'm absolutely no fan of being dependent on large companies or one-person projects like LibRaw for a simple thing as looking at the images my cameras and phones produce. I use Android smart phones, so DNG is the standard. And I will consider this when I purchase my next digital camera.

MrPete

@Mario Just back from travels, and discovered I have more to learn... and perhaps an anomaly in iMatch:
  • Imported a bunch of new (D500) NEF's
  • Yet again, no thumbnails visible, photos show fine in Viewer, WIC Diagnostics has no complaint, iMatch LibRaw was Yes. Hmmmm...

I dug in deeper than before. My assumption: I'm confused; maybe something is getting confused in this computer. ;)

I hope what follows is helpful. This WIC thing is pretty crazy. 

Bottom line up front:
* By removing ALL removable photo codecs, and turning iMatch LibRaw OFF, and doing a Force Update, the thumbnails were properly rebuilt. 
* In this scenario, iMatch uses "Microsoft Camera Raw Decoder" which is apparently built in to Win10 and is fully up to date?! See debug log.
* Doing any form of Cache Rebuild or Rescan Now->Update Cache only -- does NOT rebuild the thumbnails
    (Mario, is this expected?)
* Doing Rescan Now->Force Update -- DOES rebuild the thumbnails

Without going through contortions, I have a choice:
A) Don't install any special codecs. Just use what's in Windows, which includes up-to-date WIC codec for NEF.
        - Windows Explorer doesn't see NEF files, but iMatch DOES as noted above
        - Hope that MS will eventually fix Win Explorer??
B) iMatch LibRaw may work properly if I use Force Update on newly imported files? I can test that next time I have some (it won't be long ;) )
C) Install updated Nikon NEF Codec 1.31.4 once iMatch is able to support it (See Note C below)
        - Then "everything" will work (LOL... if you believe that... ;) )

Things I learned:
* WIC Diagnostics ignores the "iMatch LibRaw=YES" setting. 
       Is that intentional? Is there a way to test LibRaw=Yes?

* It pays to put online info in time sequence. There's a LOT of outdated info being quoted by "experts"

* Even if I uninstall every codec pack possible (See Note A below), Windows 10 still has three WIC-related dll's, installed as part of Windows (See Note B below), visible to WIC Diagnostics as two WIC codecs that support NEF:
   Microsoft Raw Image Image Decoder
   Microsoft Camera Raw Decoder
* With just those preinstalled Windows codecs, iMatch properly creates thumbnails as described above. But Windows Explorer is unhappy.
* Oh, confirmed: Do NOT bother with the Microsoft Store Raw Image Extension. It's old, outdated, and incompatible with all modern cameras... Looks like MS is supporting a built-in set of codecs, and keeping that up to date.

Note A: How to uninstall as many codec packs as possible:
- Control Panel-> Programs: NEF Codec (Nikon)
- Settings->Apps: Raw Image Extension (MS)
- Settings->Apps->Apps & Features: Camera Codec Pack (MS)

Note B: Codec DLL's in %windows%\system32 and SysWow64
These files remain after all NEF-related WIC codecs are uninstalled (well, the ones I can see.)

These are all accessed by Windows, iMatch, etc. (Ver/dates are those found on my up-to-date Win10 desktop):

 windowscodecs.dll 10.0.19041.5794 04/07/2025 (has timestamped digital sig)
windowscodecsext.dll 10.0.19041.3636 12/03/2023
windowscodecsraw.dll 10.0.19041.3636 10/20/2023 (timestamped sig) -- "MS camera codec pack"

I see no way to uninstall these, nor are they marked as installed separately from Windows.

Note C: Nikon NEF Codec
FWIW, Nikon has supported NEF in their own WIC-compatible codec for years. Supports all recent Windows versions, 32 and 64 bit. The latest (from October 2024) solves a reported security vulnerability. It supports all recent Nikon cameras.
https://download.nikonimglib.com/archive7/5BLiL00geYdA06vPEVw70TjsbQ23/S-NEFCDC-013104WF-ALLIN-ALL___.exe

That codec does work in Windows Explorer and MS apps. iMatch WIC Diagnostics fails with it. I don't know why. 
Mario, I'm not sure if it's worth you working out whatever the issue is. At least it IS a vendor-supplied codec that works properly with Microsoft systems.




Mario

#5
Quote* Doing any form of Cache Rebuild or Rescan Now->Update Cache only -- does NOT rebuild the thumbnails
    (Mario, is this expected?)
Yes. You are rebuilding the cache. This command only exists to allow users to rebuild cache images after they changed cache options. Cache does not affect thumbnails.


Quote* WIC Diagnostics ignores the "iMatch LibRaw=YES" setting.
      Is that intentional? Is there a way to test LibRaw=Yes?
WIC diagnostics always tests WIC and LibRaw and outputs results for both.


QuoteNote A: How to uninstall as many codec packs as possible:
I guess 99.99% of all users only have the standard WIC codecs that are part of WIndows. Most of them are based on LibRaw now anyway, but on a very old version.

If you still use Windows 10, you are using older WIC codecs. Microsoft no longer provides feature updates for Windows 10, only security patches until October (?) this year. Then Windows 10 goes out of support.


QuoteNote C: Nikon NEF Codec

As far as I know, Nikon has discontinued their own codecs. The link you post points to a very old unsupported version with probably a ton of security problems and is not recommended to install.

I have no problems processing Nikon NEF files (D500 and many other models) with IMatch and LibnRaw.
Just checked. Can you send me a D500 NEF that fails with LibRaw? What does WICDiagnosis report for a failed DNG? Can you attach the diagnosis output?

sybersitizen

#6
Quote from: Mario on June 13, 2025, 08:25:39 AMAs far as I know, Nikon has discontinued their own codecs. The link you post points to a very old unsupported version with probably a ton of security problems and is not recommended to install.
I was waiting for you to weigh in on this.

I'm still on Win10 and I do a lot of shooting with the old Nikon V3 and V1. I've had the FastPictureViewer codecs (which you also recommend against) installed for years so I can see thumbnails and open them in File Explorer. They don't work for new Nikon cameras, but that's not important to me.

What sort of security risks exist with the old Nikon codecs that were mentioned, or with the FastPictureViewer codecs?

What is a better option that will work with my old NEFs (and old ARWs) in File Explorer?

As you say, it won't be long before I upgrade to Win11. What then? Is NEF and ARW support built in?

maxbelloni

#7
Quote from: sybersitizen on June 13, 2025, 06:27:46 PM
Quote from: Mario on June 13, 2025, 08:25:39 AMAs far as I know, Nikon has discontinued their own codecs. The link you post points to a very old unsupported version with probably a ton of security problems and is not recommended to install.
I was waiting for you to weigh in on this.
I'm still on Win10 and I do a lot of shooting with the old Nikon V3 and V1. I've had the FastPictureViewer codecs (which you also recommend against) installed for years so I can see thumbnails and open them in File Explorer. They don't work for new Nikon cameras, but that's not important to me.
What sort of security risks exist with the old Nikon codecs that were mentioned, or with the FastPictureViewer codecs?
What is a better option that will work with my old NEFs (and old ARWs) in File Explorer?
As you say, it won't be long before I upgrade to Win11. What then? Is NEF and ARW support built in?
AFAIK the version 1.31.4 pointed by @MrPete above is anyway the last version Nikon released around october 2024

Mario

#8
What sort of security risks exist with the old Nikon codecs that were mentioned, or with the FastPictureViewer codecs?

I don't track Nikon WIC codecs or what they do with them. I know that they discontinued their codecs a while ago (I even updated my WIC knowledge base article in my blog at the time).

FastPictureViewer codeds are of of support for many years. The last blog article is from 2015 and the forum is dead.
They are sill for sale, though. They cause many issues with modern RAW formats and uninstalling them is the usual way to solve these problems.

IMatch defaults to LibRaw, which has the best available support for all RAW formats (https://www.libraw.org/supported-cameras). Most of the RAW WIC codecs included in Windows 11 are based on LibRaw now (an older version, though). IMatch uses a very recent version of LibRaw.

QuoteWhat is a better option that will work with my old NEFs (and old ARWs) in File Explorer?

Windows Explorer (if this is what you mean) uses WIC codecs to render RAW files. If the WIC codecs shipped with W10 don't support your RAW format, you're out of luck. Windows Explorer is not the right tool to look at RAW files anyway.

I recommend XN View MP (https://www.xnview.com/en/) which supports many formats, including RAW (it uses LibRaw, I believe). Free for personal use (use the donation link to donate 5 bucks to the developer) 30€ for a professional license. Very good, very affordable. Don't bother with Windows Explorer.

DO NOT install WIC Codec Packs or similar from dubious sources. Hackers use them to spread malware.

QuoteAs you say, it won't be long before I upgrade to Win11. What then? Is NEF and ARW support built in?

I can only repeat myself again. Using undocumented and proprietary RAW formats like NEF or ARW is risky.
Nobody knows if and which RAW formats Microsoft supports in their WIC codecs. It's surely not a priority for them.

The entire world depends on Adobe and the 2 or 3 other companies which have a financial incentive large enough to spend time reverse engineering undocumented RAW formats and lenses. If they don't want anymore, all your precious RAW files suddenly become binary blobs without value. Or maybe you can use the (usually crappy) software your camera vendor might provide - for some time at least.

The digital camera market has changed massively over the last years. It has become what it was before the hype - a hobby market for people who care about image quality and photography. That smart phones have become so good at taking photos helped this trend.

This also means that there is less money to make from the market, for companies like Adobe, DxO and CaptureOne. Their user base should have shrunk a lot.

This does not affect me much, though. IMatch was always a nice product and I don't have a fancy company building, large marketing expenses or shareholders to keep happy.

We are very lucky to have Alex and his LibRaw project. He spends tons of times to support new RAW formats. I have bought at least two licenses of FastRawViewer to support him.

Without WIC codecs and LibRaw, IMatch could not open or display RAW formats. I don't have the financial and personell resources to reverse engineer RAW formats from Nikon, Canon, Sony, Fuji and others. As far as I know, none of these companies provides WIC codecs or developer programs anymore, or technical data about their ever-changing RAW formats.

The vast majority of RAW files these days are produced by Apple and Android smart phones.
And these vendors did the right thing, settling on DNG as their format of choice. DNG is an ISO standard (at least until Adobe again breaks it because they want something new for one of their products) and the best bet for making RAW files save for the future. It is superior in features and compatibility over all the proprietary and undocumented RAW formats camera vendors seem to insist on for whatever reason (Leica/Panasonic excluded, they use DNG).

IMatch embeds the latest DNG toolkit version made available by Adobe, and combines it with LibRaw to provide access to hundreds of RAW and DNG variants in use.

If you have an image that does not display in IMatch when LibRaw (photools.com RAW processing) is enabled, send me a sample.

sybersitizen

Quote
QuoteWhat is a better option that will work with my old NEFs (and old ARWs) in File Explorer?

Windows Explorer (if this is what you mean) uses WIC codecs to render RAW files.

It is called File Explorer on my system.

File Explorer.jpg

I had only a few simple questions, but guess I can get the answers I was seeking somewhere else because they don't actually relate directly to IMatch.

Mario

Windows File Explorer, to my knowledge, uses WIC codecs to render RAW files.
Did you try XN View? It does all what Windows Explorer does + excels at displaying and working with images.

Jingo

I cannot recommend XYPlorer enough for a File Explorer replacement.  I've been using it for decades now and the feature set is mind blowing... great for photo browsing as well with mouse down blow up, preview windows and full thumbnail display even for RAW files.

sybersitizen

Again, I don't need a replacement for File Explorer, and I know about WIC codecs and third party codecs.

My real question was about the 'ton of security problems' statement made earlier. Sure, downloading something from a questionable source can be a security problem if that's the meaning. But I've been using my FastPictureViewer codecs for years, so is there some security problem in continuing to use them now or in the future? Is there a security problem in downloading and using codecs from Nikon's own servers? If so, what is the vulnerability? That's all.

Mario

If software such as the Nikon WIC codecs is discontinued, no longer maintained, removed from the Nikon website and relocated to an Archive server, I would not consider using it. These WIC codecs operate under your user account and have the same permissions as you do. Any undetected or unfixed security issue could potentially be exploited by malicious actors.

I understand that you have files stored in a Nikon NEF variant not supported by the WIC codecs included with Windows 10, which itself is approaching end-of-support this year.

In my test library, I have about a dozen D-500 NEF files. Upon checking, all of them display thumbnails in Windows Explorer. This indicates that the WIC codecs included in Windows 11 have been updated or are now based on LibRaw, which supports many more RAW formats than the older WIC codecs provided with Windows 10. IMatch also handles these files.

If you remain on Windows 10 and prefer not to use free software like XnView, it seems your only option is to install the legacy WIC codecs from the Nikon archive.

sybersitizen

As indicated, I won't be on Win10 much longer anyway, so I'll see how things go with Win11.

Mario

Quote from: sybersitizen on June 14, 2025, 05:00:59 PMAs indicated, I won't be on Win10 much longer anyway, so I'll see how things go with Win11.
D-500 files work here just fine, with only the standard WIC codecs installed. Should work for you too.