New RAW format

Started by DigPeter, December 10, 2019, 12:58:47 PM

Previous topic - Next topic

DigPeter

I have just obtained the new(ish) Canon Powershot G5X II.  The RAW format for this is CR3.  This is covered in IMv2019.8.4 under CR2, but they are not compatible with each other.  It consequently does not render well in IM.

Mario

#1
IMatch knows about CR3 or a long time. This is not a new extension or format.
These files are processed via the installed WIC codec.

What does "not render well" mean?
Not showing? All grayscale? All red? Colors wrong? Error message? Only thumbnail extracted? Preview extraction fails? RAW instruction fails?...

I wrote the same stuff sentence multiple times over the past days: Use the built-in WIC Diagnostics in IMatch to check if the installed WIC codec on your system handles the file.  This will give us important info. Attach the result to your reply. Upload a sample image somewhere and include a link. Having a file here will allow me to try this out myself.

If WIC does not handle the file update the WIC codec. If you have Windows 10 and the integrated WIC codec does not handle your format, ask Microsoft or Canon for an update. Canon is a lazy cheater and stopped providing their customers with WIC codecs years ago - to save a couple of developer hours per year, probably.

If the Windows WIC codec does not handle your files and neither does LibRaw, we're out of luck for now.
IMatch uses either WIC, or if that fails, falls back to LibRaw. If both fails, we can only wait for an update.

That's the risk of using undocumented and proprietary file formats. Especially for Canon user, where the camera vendor produces new format variants all the time but is to cheap-assed to provide their paying customers with a working WIC codec.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

DigPeter

Thank you Mario.  The attachment demonstrates what I mean by not rendering well.  These were shot with Raw CR3/jpg setting.  The left half shows the jp render, the right the poor CR3 render.

IM's WIC diagnostic shows CR3, but it would appear not to be functioning.

The camera in question is Canon PowerShot G5X Mk II, which shoots CR3.  LibRaw website does not show this camera, only the G5X, which I believe shoots CR2.  But fast viewer claims to include CR3 rendering. 




Mario

1. The image on the right looks like a badly upsampled thumbnail or a too small preview scaled up.

2. Attach the WIC diagnosis log file.
This would tell me if Windows can actually extract a preview or RAW or only the thumbnail for your file. Which would explain 1.

3. Attach a screen shot of the Edit > Preferences > Cache dialog.

Maybe the embedded preview in some of your files is too small and IMatch thus must fall-back to developing the full RAW.
In that case, whatever the WIC codec or LibRaw returns is "it" what IMatch uses.
The results may be vastly different from the optimized and processed preview your camera stores in the file.

4. Upload some sample images and send a link to support email address. Include a link to this topic in your email.

5. Try the latest version of Windows WIC codecs perhaps. They often produce better results but are not installed automatically. Windows 10 May required.
See https://www.microsoft.com/en-us/p/raw-image-extension/9nctdw2w1bh8?activetab=pivot:overviewtab for details.

-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

DigPeter

Quote from: Mario on December 11, 2019, 08:08:34 PM
1. The image on the right looks like a badly upsampled thumbnail or a too small preview scaled up.

2. Attach the WIC diagnosis log file.
This would tell me if Windows can actually extract a preview or RAW or only the thumbnail for your file. Which would explain 1.
I have attached a WIC dialog text file.  If this is not what  you require.could you please say how to get it.

Quote3. Attach a screen shot of the Edit > Preferences > Cache dialog.
Attached

QuoteMaybe the embedded preview in some of your files is too small and IMatch thus must fall-back to developing the full RAW.
In that case, whatever the WIC codec or LibRaw returns is "it" what IMatch uses.
The results may be vastly different from the optimized and processed preview your camera stores in the file.
I have LibRaw, but not sure how to apply it to IMatch.

Quote4. Upload some sample images and send a link to support email address. Include a link to this topic in your email.
Link sent to your support address.

Quote5. Try the latest version of Windows WIC codecs perhaps. They often produce better results but are not installed automatically. Windows 10 May required.
See https://www.microsoft.com/en-us/p/raw-image-extension/9nctdw2w1bh8?activetab=pivot:overviewtab for details.
I have installed this but it does not appear to make any difference.

Mario

#5
1. None of the WIC codecs installed on your system understands the file format you are using.

WIC Result: It looks like no WIC codec is installed which can handle this file.

2. IMatch automatically falls back to LibRaw (you don't need to do anything special). LibRaw also fails to load the files.

Error opening file: -2 (Unsupported file format or not RAW file)

As far as I know, I've included the latest version of LibRaw when shipping the last IMatch update. It does not change very often.
I here assume that the file is not damaged and you can open it in, e.g. Lightroom or another RAW processor?

I suggest you contact Canon about a WIC codec which is able to understand the Canon RAW format your camera is producing. Or contact Microsoft and ask for an update of the WIC codec for Canon RAW formats.

Unfortunately, this is one of the drawbacks when you rely on undocumented and proprietary RAW formats. When Canon cooks up something new, it may take months before Windows / LibRaw catch up. And since Canon does not provide good support and stopped developing their own WIC codec (probably to save 10K or 20K $ per year for a few developer weeks) all you can do it wait. I don't develop the Windows WIC codecs and I don't develop LibRaw. And I don't reverse-engineer proprietary RAW formats anymore.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

DigPeter

Thank you Mario.  I cannot understand why Canon would not support its products.  Not being able to process RAW files with commonly used apps is poor marketing.   DxO can convert CR3 files, but not the latest Corel AfterShotPro.  I do use Abobe Cloud apps, so have only the last stand-alone LR, v5.

RogueMD

FYI, for this version of the MS Raw Image Extension:
https://www.microsoft.com/en-us/p/raw-image-extension/9nctdw2w1bh8?activetab=pivot:overviewtab

In the description:
The current version of the extension supports a long list of cameras, however some Raw formats like .CR3 and .GPR are not currently supported.

Mario

#8
The latest version of LibRaw (Released in November) claims to have support for CR3 files.
I will see if I can build this and include it in IMatch 2020. FastPictureViewer codecs, albeit old and no longer supported, have CR3 support too, apparently.

I'd really customers would force Canon to ship a proper WIC codec. They know best how to treat their files, and it cannot be that expensive to use their existing RAW processing code and wrap it ina  proper WIC codec. Nikon provides a WIC codec for all their cameras. Canon should too. But users never complain to them or they are just ignored.

Send one of your problem CR3 files to support email address so I can check it and add it to my test library. A photo of a wall or so is OK.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

DigPeter

#9
Quote from: Mario on December 13, 2019, 06:36:52 PM

I'd really customers would force Canon to ship a proper WIC codec. They know best how to treat their files, and it cannot be that expensive to use their existing RAW processing code and wrap it ina  proper WIC codec. Nikon provides a WIC codec for all their cameras. Canon should too. But users never complain to them or they are just ignored.

I have done that.

In the absence of the official codec, Canon do provide a free app "Digital Photo Professional 4.11.0" [LINK REMOVED BY ADMIN].

ADMIN:
Please don't use tinyurl here in the community. tinyurl is often used for malicious purposes. Since you don't see where the URL points at, it can be anything.
There is no need for tinyurls here. Plenty of space and everybody should see the real target of links posted.

Mario

Canon offers several software products at https://www.canon-europe.com/support/camera_software/#EOSDPP as a replacement for a WIC codec.

Why they don't offer a WIC codec escapes me. Especially given the fact that Canon cameras produced such a massive number of different RAW formats over the years.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

DigPeter