Author Topic: Workflow with DXO (Imatch 2020 with PL 4)  (Read 494 times)

WebEngel

  • Full Member
  • **
  • Posts: 248
Workflow with DXO (Imatch 2020 with PL 4)
« on: November 11, 2020, 08:23:54 PM »
Hi all,

I have been using Lightroom for a decade but have now switched to DXO.  I am still struggling to establish a clean workflow.  All related threads in this forum seem decades old (except one which did not cover my questions), so I am starting a new one.  Questions below indicated with emoticon  :-\.

Here is where I am:

Raw+Jpg: I always shoot Raw+Jpg. In Imatch, Master is JPG, while the Raw file and DXO's DOP as well as any XMP files are buddies.  All have the same filename for simplicity reasons, so it is Test.Jpg, Test.ARW, Test.ARW.dop, and if need be Test.xmp.  All are in the same folder.

(Why is Jpg Master?  I want to view Jpgs in full-res for rating and for viewing. For 10% of my images, I immediately process the Raw file to create a better Jpg file, replacing the out-of-camera Jpg.  For another 20% I do that later, possibly half a year later.  For another 20% I keep the Raw in case but never process it. For 50% of my images, I delete the raw file after a few years without ever having used it.  So having the Jpg as master ensures I always use the same master file -- the raw file is only there for the Raw converter.  Also, I use Imatch as well on a laptop which only has the Jpgs but not the Raw files.)

Rate/Meta/Category: Step one of my workflow is to cull, rate and generate metadata (GPS and other stuff) and rename files.  That metadata then sits in Imatch.  I can write it to the Jpg or an XMP sidecar.  I also assign the jpg to a category.

Raw processing: When I decide a picture (Test.jpg) is worth processing (i.e. the 30% mentioned before), I go to the Media&Folders view in Imatch, disable the "Hide buddy files" filter, double-click on the corresponding raw file (Test.Arw).     :-\ (I am pretty sure this can be simplified in Imatch given how powerful it is, can anybody just let me know the name of the functionality to launch the raw file from the jpg file?  RTFM comments are fine).

Export and replace in Imatch: When I am done in DXO, I export the file, thereby replacing the master file in Imatch.  Unfortunately, DXO does not seem to pick the metadata up from the xmp sidecar.  So when DXO generates the new Jpg file, it does not contain the metadata (except rating perhaps).  Of course, Imatch is smart enough to realize this is the same pic, so all categories and metadata are still there.

Face recog: Only the face annotations in Imatch are wrong if the new file is cropped.  Is there a way to automate this  :-\ (algorithm: if file dimensions have changed and if the file had face annotations before, rerun face recognition)

set label based on new meta: When the new Jpg file is digested by Imatch, I manually set it to status "green".  Given the jpg has "DXO" in metadata field "Software", Imatch can probably do that automatically  :-\ (algorithm: if file has changed and if {File.MD.Exif::Main\305\Software\0} contains "DXO", set status to green  - what is the name of the Imatch functionality to do that?)

How to keep metadata in Jpg:-\ How do I get the metadata into the Jpg file so that people with whom I exchange pictures (or Android devices) can view it as well?  Should I do "Metadata write-back" from Imatch?  Or is there a way to tell DXO to use the metadata in the appropriate way and create the new Jpg with the metadata in the first place?  Certainly, DXO is far behind Imatch in terms of metadata, so it is probably best to not even try DXO with metadata.   :-\ In this case, does it make sense at all to have XMP files?

Updating metadata only after processing the raw file in DXO is not an option for me: I need to update metadata immediately after taking the pictures, because I may have forgotten details later.  But I need to be able to process raw files and generate new Jpgs much later, even years later.  And I need to be able to process files again even later ("damn, why did I crop the picture this way, this looks ugly").

As you can see, I do get along, it works somehow, but it surely can be improved.  What would you do differently?  How would you automate the manual tasks?  Again, pointing me to the name of the functionality or the help page is sufficient, I can easily work from there.

Thanks for any help.

martin

JohnZeman

  • Global Moderator
  • *****
  • Posts: 1348
  • I'm too damn old to act my age.
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #1 on: November 11, 2020, 11:28:52 PM »
I'm also using IMatch and DxO PhotoLab 4 plus I also throw Affinity Photo into the workflow.

Rather than trying to answer your questions I'll just tell you what my workflow is, perhaps doing that may answer some of your questions.

For starters my workflow is quite different than yours.
I only shoot raw, not raw+jpg, and in IMatch my raw files are my masters.

I use FastRawViewer to send the raw file to IMatch which then imports it.
Next using IMatch I rename the raw to a YYYY_MMDD_HHMMSS.SSS format, categorize it, and enter all of my metadata, which IMatch saves to XMP sidecars.

When I'm ready to process the raw image I use an IMatch favorite to send the selected raw file(s) to PhotoLab.

I do not enter or change any metadata in PhotoLab at all, I simply use PhotoLab as a raw processor.

Once I'm done processing the raw in PhotoLab, which saves its edits to a sidecar .dop file that sits in the folder right beside the raw and XMP sidecar file, I export the raw as a temporary 16 bit tiff.  Then I open the tiff in Affinity Photo and do my final optimization.

Finally, I export the optimized tiff as a hi-res JPG and save that to IMatch, also storing it in the same folder as the original raw file.  At this point the TIFF file can be discarded.

So in the folder I have the raw file, the XMP sidecar file, the .dop buddy file, and the optimized JPG, all sitting next to each other.

In IMatch I'm using file relations and stacking to group and keep those 4 files together.

The end result is the raw is the master but in IMatch I see the optimized JPG version of the raw instead of the original raw image while the JPG file itself is hidden inside the stack.  I can easily disable this if I want but I usually don't unless I have a good reason to.

Finally using file relations IMatch automatically propagates (copies) the metadata and categories from the raw to the jpg so essentially both the raw and the jpg have the same metadata and category assignments.  It's all automatic in IMatch once you get it set up.

This system works very well for me and hopefully it might give you some ideas or at least some food for thought.

sinus

  • Global Moderator
  • *****
  • Posts: 4153
  • IMatch-User since 2001 (IMatch 3.6)
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #2 on: November 12, 2020, 11:39:48 AM »

Thanks for any help.

martin

Martin, first I wanted answer you, but then I created an own thread (in this forum-section) to not somehow hijack your thread.
But maybe it gives you also some ideas, in the end it is you who made me write something about a workflow again.  ;D
Good luck anyway.
Best wishes from Switzerland! :-)
Markus

akirot

  • Full Member
  • **
  • Posts: 179
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #3 on: November 12, 2020, 04:49:41 PM »
@JohnZeman:

Just curious:
Quote
Finally, I export the optimized tiff as a hi-res JPG and save that...
This "hires JPG" is it a 1:1 export or do you scale to a certain size?

WebEngel

  • Full Member
  • **
  • Posts: 248
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #4 on: November 12, 2020, 05:24:47 PM »
Thanks, John

When I'm ready to process the raw image I use an IMatch favorite to send the selected raw file(s) to PhotoLab.

Are you referring to application favorites or External Tool Favorites (as described here: https://www.photools.com/help/imatch/#fav_basics.htm)

Finally using file relations IMatch automatically propagates (copies) the metadata and categories from the raw to the jpg so essentially both the raw and the jpg have the same metadata and category assignments.  It's all automatic in IMatch once you get it set up.

Are you saying that the Jpg has all metadata, so that if you use Windows to copy it to a new PC, you can see the metadata there?  And are you using the versioning commands to propagate metadata to the JPG version?

It seems that metadata handling is indeed easier if you make the Raw file the master.  Do you ever delete Raw files and keep the Jpgs?  If yes, does it work, or does it create problems?  This is the main use case which made me go for JPG=Master.


JohnZeman

  • Global Moderator
  • *****
  • Posts: 1348
  • I'm too damn old to act my age.
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #5 on: November 12, 2020, 05:48:59 PM »
@JohnZeman:

Just curious:
Quote
Finally, I export the optimized tiff as a hi-res JPG and save that...
This "hires JPG" is it a 1:1 export or do you scale to a certain size?

It's always a full sized 1:1 export at 85% quality.

The only time I need to reduce the size and quality from that is when I'm going to send a copy of the photo to facebook or someone else.  And when I do that I drag and drop the original raw file into the IMatch Image Batch Processor.  The Batch Processor doesn't reduce the raw, instead it makes a reduced sized copy of the JPG version.

JohnZeman

  • Global Moderator
  • *****
  • Posts: 1348
  • I'm too damn old to act my age.
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #6 on: November 12, 2020, 06:01:17 PM »

Are you referring to application favorites or External Tool Favorites (as described here: https://www.photools.com/help/imatch/#fav_basics.htm)

The External Tool Favorites.
A partial screenshot of my favorites panel is attached.

Are you saying that the Jpg has all metadata, so that if you use Windows to copy it to a new PC, you can see the metadata there?  And are you using the versioning commands to propagate metadata to the JPG version?

It seems that metadata handling is indeed easier if you make the Raw file the master.  Do you ever delete Raw files and keep the Jpgs?  If yes, does it work, or does it create problems?  This is the main use case which made me go for JPG=Master.

Yes to your first 2 questions, no to the last one.
When I decide to delete an image I delete the entire works, the master and the raw.  However that's pretty rare for me to do since I cull all of the photos before and during the import new images process.

I do not actually propagate all metadata from a master raw image to a JPG version, just the metadata and categories I want copied.  For example I do not copy image orientation from a master raw to a JPG version.

jch2103

  • Oldtimer
  • ****
  • Posts: 2058
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #7 on: November 12, 2020, 06:20:53 PM »
Just a note: PhotoLab is pretty good about copying pertinent metadata from the raw file to the output jpg (i.e., things like Description, Author, location, keywords, etc.). So if you use IMatch to add metadata to the raw file, you should also see it in the PL output jpg (and tiff, etc., I believe).

My workflow is similar to John Z's. It helps keep things fairly simple.
John

WebEngel

  • Full Member
  • **
  • Posts: 248
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #8 on: November 16, 2020, 09:07:35 AM »
Hello

It seems that the workflow is indeed easier if the raw file is the master.  Unless you delete raw files (and not the corresponding jpg), this seems the better option.

If raw files are deleted as part of the workflow, like in my case, it is a difficult story:

If the raw file is the master, you delete the master
If the jpg file is the master, you replace the master with a new master (when you process the file in DXO).

Maybe i need to setup a test db to see which approach fits best.  Anybody else who tried to delete master jpg files?

martin

DigPeter

  • Super Hero
  • ****
  • Posts: 1211
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #9 on: November 16, 2020, 01:57:53 PM »
Hello
If raw files are deleted as part of the workflow, like in my case, it is a difficult story:
Why do you delete raw files?  I think most people keep them on their ex-camera state, in case reprocessimg is needed and a safeguard against loss or corruption of the jpeg.

Jingo

  • Super Hero
  • ****
  • Posts: 1658
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #10 on: November 16, 2020, 02:24:17 PM »
Hello
If raw files are deleted as part of the workflow, like in my case, it is a difficult story:
Why do you delete raw files?  I think most people keep them on their ex-camera state, in case reprocessimg is needed and a safeguard against loss or corruption of the jpeg.

To be completely honest - since I have gone back and re-edited a RAW file only about a handful of times over the years...  if space wasn't so cheap, I too might consider deleting the RAW files after conversion. 

My JPG's are backed up to 2 locations daily as is my IMatch db so there is no concern about corruption, etc... in the end, it really is just a choice about space, complexity of workflow and ultimate use of the photos.. for most amateurs, JPG's are simply enough and with camera and software technology at a place it is at, taking JPG only might become a real option in the near future.

jch2103

  • Oldtimer
  • ****
  • Posts: 2058
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #11 on: November 16, 2020, 08:13:07 PM »
Hello
If raw files are deleted as part of the workflow, like in my case, it is a difficult story:
Why do you delete raw files?  I think most people keep them on their ex-camera state, in case reprocessimg is needed and a safeguard against loss or corruption of the jpeg.

DxO 4 is a case in point of why it's good to keep the raw files: The new DeepPRIME feature makes such a difference in some of my old high_ISO/small sensor shots that I'm going back to reprocess a number of them.
John

JohnZeman

  • Global Moderator
  • *****
  • Posts: 1348
  • I'm too damn old to act my age.
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #12 on: November 17, 2020, 04:39:57 AM »
DxO 4 is a case in point of why it's good to keep the raw files: The new DeepPRIME feature makes such a difference in some of my old high_ISO/small sensor shots that I'm going back to reprocess a number of them.

I agree.
One of my favorite pastimes is going back in time to optimize and reprocess raw photos I'd taken several years before.

DigPeter

  • Super Hero
  • ****
  • Posts: 1211
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #13 on: November 17, 2020, 01:19:55 PM »
Hello
If raw files are deleted as part of the workflow, like in my case, it is a difficult story:
Why do you delete raw files?  I think most people keep them on their ex-camera state, in case reprocessimg is needed and a safeguard against loss or corruption of the jpeg.
It is what works for each individual.  I keep my raws on separate media from processed jpegs, so there is an additional backup - belt and braces.

Erik

  • Sr. Member
  • **
  • Posts: 286
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #14 on: November 19, 2020, 12:27:39 AM »
Hi all,

I have been using Lightroom for a decade but have now switched to DXO.  I am still struggling to establish a clean workflow.  All related threads in this forum seem decades old (except one which did not cover my questions), so I am starting a new one.  Questions below indicated with emoticon  :-\.

Here is where I am:

Raw+Jpg: I always shoot Raw+Jpg. In Imatch, Master is JPG, while the Raw file and DXO's DOP as well as any XMP files are buddies.  All have the same filename for simplicity reasons, so it is Test.Jpg, Test.ARW, Test.ARW.dop, and if need be Test.xmp.  All are in the same folder.

(Why is Jpg Master?  I want to view Jpgs in full-res for rating and for viewing. For 10% of my images, I immediately process the Raw file to create a better Jpg file, replacing the out-of-camera Jpg.  For another 20% I do that later, possibly half a year later.  For another 20% I keep the Raw in case but never process it. For 50% of my images, I delete the raw file after a few years without ever having used it.  So having the Jpg as master ensures I always use the same master file -- the raw file is only there for the Raw converter.  Also, I use Imatch as well on a laptop which only has the Jpgs but not the Raw files.)

Rate/Meta/Category: Step one of my workflow is to cull, rate and generate metadata (GPS and other stuff) and rename files.  That metadata then sits in Imatch.  I can write it to the Jpg or an XMP sidecar.  I also assign the jpg to a category.

Raw processing: When I decide a picture (Test.jpg) is worth processing (i.e. the 30% mentioned before), I go to the Media&Folders view in Imatch, disable the "Hide buddy files" filter, double-click on the corresponding raw file (Test.Arw).     :-\ (I am pretty sure this can be simplified in Imatch given how powerful it is, can anybody just let me know the name of the functionality to launch the raw file from the jpg file?  RTFM comments are fine).

I should probably read through everyone's responses, but I read through your email and wanted to add some thoughts where I think you could be helped.  I use DXO albeit quite differently (mostly because I have RAW's as masters).

I get why you are using JPG as the master, but what I think I am confused is why you are using the RAW as a buddy and not as a version.  I think setting the RAW file as a version will allow the metadata (XMP) to link up between the JPG and RAW.  You would have to play with the settings, but what would happen is that you'd want to make sure the RAW file is set up to XMP record from the JPG master.  This is a bit counter-intuitive, but assuming the JPG already matches from the original process of the RAW file, it should work.  The RAW files may not have much of an XMP record to begin with, and by using RAW's as buddy files, they are not picking up an XMP record from the JPG files.

You'll want to test some of this out within IMatch and inspecting the XMP records for the RAW and JPG files.

Once you have this set up, you can use the version panel or just open a version set to find the RAW and double-click it.

Quote
Export and replace in Imatch: When I am done in DXO, I export the file, thereby replacing the master file in Imatch.  Unfortunately, DXO does not seem to pick the metadata up from the xmp sidecar.  So when DXO generates the new Jpg file, it does not contain the metadata (except rating perhaps).  Of course, Imatch is smart enough to realize this is the same pic, so all categories and metadata are still there.

I think fixing the version may help this, but I'm not sure.  You probably need to make sure DxO shows an XMP record for the RAW file (I think version 4 allows you to inspect the metadata).  You might also need to make sure that it saves metadata (all of it) to the exported file.  Ultimately, the issue would be a DxO issue unless your RAW file doesn't have the XMP record to begin with.

This part is hard for me because I use the RAW files as masters and I've not had the issue you have, and if the exported JPG doesn't have an XMP record the file will get one from IMatch detecting a new version

Quote
Face recog: Only the face annotations in Imatch are wrong if the new file is cropped.  Is there a way to automate this  :-\ (algorithm: if file dimensions have changed and if the file had face annotations before, rerun face recognition)

I'm not sure on this one.  Problem is that I think IMatch doesn't see your file as different aside from maybe metadata.  The only possibility is to watch for whether IMatch is picking up the file as changed, triggering an update, and maybe leverage that trigger in an app.  If you have a collection for updated files, you can probably also look to that.  Automating may be a question you'll want to ask in the general forum since it isn't necessarily DxO specific.

Quote
set label based on new meta: When the new Jpg file is digested by Imatch, I manually set it to status "green".  Given the jpg has "DXO" in metadata field "Software", Imatch can probably do that automatically  :-\ (algorithm: if file has changed and if {File.MD.Exif::Main\305\Software\0} contains "DXO", set status to green  - what is the name of the Imatch functionality to do that?)

I guess the question would be how stuck are you on using a label, or would you be ok to using a category, which might be easier and definitely contained to your database.  Labels end up part of the XMP record.  If you are open, it opens up some easy solutions.  For instance, you could use a data driven category based on the {File.MD.Exif::Main\305\Software\0} variable.  You could set a color for the data driven category specific to DXO in its properties.  You'll have to read the manual, but this would be automatic no matter what.  Otherwise, you might need an app that can conditionally set the XMP label field to green when the Software field is DXO and operates when files are updated.

Quote
How to keep metadata in Jpg:-\ How do I get the metadata into the Jpg file so that people with whom I exchange pictures (or Android devices) can view it as well?  Should I do "Metadata write-back" from Imatch?  Or is there a way to tell DXO to use the metadata in the appropriate way and create the new Jpg with the metadata in the first place?  Certainly, DXO is far behind Imatch in terms of metadata, so it is probably best to not even try DXO with metadata.   :-\ In this case, does it make sense at all to have XMP files?

Updating metadata only after processing the raw file in DXO is not an option for me: I need to update metadata immediately after taking the pictures, because I may have forgotten details later.  But I need to be able to process raw files and generate new Jpgs much later, even years later.  And I need to be able to process files again even later ("damn, why did I crop the picture this way, this looks ugly").

As you can see, I do get along, it works somehow, but it surely can be improved.  What would you do differently?  How would you automate the manual tasks?  Again, pointing me to the name of the functionality or the help page is sufficient, I can easily work from there.

Thanks for any help.

martin

I think this end discussion speaks to a need to understand how metadata works with the various file types.  JPG images really should only operate with XMP records embedded in them.  IMatch will not work right if it finds external XMP records AND internal XMP records.  The raw files on the other hand will likely only operate with XMP records.  And that is all fine.  IMatch will handle that well, and generally so will DxO. 

To share files with people so they can see the metadata, it needs to be in the JPG.  If that is the desire, I would use the metadata write back options in IMatch.  However, because you are using JPG files as masters but then exporting what amounts to new masters, you NEED DxO to export metadata otherwise you'll lose it.

This is actually the danger in the method you are using for cataloging your files.  While you are treating the JPG file as the master in IMatch you are actually using the RAW file as the master in DxO.  You are creating a chicken and egg scenario because the exported file becomes the master for the RAW file you generated it from.  What comes first?  You could lose data that way.  I'll post some thoughts regarding how I handle my workflow in a separate post.

Erik

  • Sr. Member
  • **
  • Posts: 286
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #15 on: November 19, 2020, 12:39:57 AM »
Hello

It seems that the workflow is indeed easier if the raw file is the master.  Unless you delete raw files (and not the corresponding jpg), this seems the better option.

If raw files are deleted as part of the workflow, like in my case, it is a difficult story:

If the raw file is the master, you delete the master
If the jpg file is the master, you replace the master with a new master (when you process the file in DXO).

Maybe i need to setup a test db to see which approach fits best.  Anybody else who tried to delete master jpg files?

martin

You'll have to see the end of my extensive response to your original post.  I think the process of making the jpg a master and then replacing it with a new master (with the RAW file in between) is something that is the most dangerous of situations in terms of metadata, especially if the metadata needs to propagate from one file to another. 

An alternative scenario in your case may be for the exported file to be an additional version (different file name).  You could create a new version setup (with the same JPG master) so that the JPG that is exported is the Visual for the version stack.  For instance you might have files like follows:

Original.JPG
Original.RAW
Original-dxo.JPG

Original.JPG would be the master for both sets and would contribute metadata to the RAW file and the exported JPG (as you set it up). 

Now what you can do is ultimately delete any of the files.  You want to be careful about deleting the full set, but deleting a master on its own doesn't necessarily hurt anything.  The versions won't lose their metadata because it only gets updated when the master file's is changed or edited (or you directly edit it).  Thus, you could remove the RAW file later on.

Now as mentioned a few posts back, the recent addition of the deepPRIME noise reduction in PL 4 is a good reason to keep RAW files, but that's just a matter of preference.

Mario

  • IMatch Developer
  • Administrator
  • *****
  • Posts: 26787
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #16 on: November 19, 2020, 12:41:53 AM »
Quote
IMatch will not work right if it finds external XMP records AND internal XMP records.

By standards, JPEG files use embedded XMP metadata (in the file).
Also by standards, an XMP sidecar file belongs to all files with the same file name as the XMP file in the same folder.

IMatch is aware of this and merges XMP metadata from the external sidecar file with XMP metadata found embedded in the JPG file - by default considering the embedded XMP metadata to be more precious.

You can thank the other software companies for all the confusion.
Every RAW processor out there deals with embedded XMP data, XMP data in sidecar files and other important metadata like EXIF and GPS as they see fit. Nobody really cares that much.
Proper metadata management may be critical - but it does not make a good source for fancy ads or magazine and blog reviews.
And most users are unaware of all these potential problems until they switch their RAW processing software, "DAM" or platform...
« Last Edit: November 19, 2020, 01:25:34 AM by Mario »

Erik

  • Sr. Member
  • **
  • Posts: 286
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #17 on: November 19, 2020, 01:22:58 AM »
Just to add a perspective, my workflow with PL 4 (And really DxO for a while) relies predominently on IMatch.

In IMatch, the RAW files are my masters.  If I shoot in RAW+, the jpgs become the visual proxy (and are their own version set as opposed to buddy).  All the DxO sidecars are buddy files in IMatch.

For each session in DxO, I create a collection in PL.  Usually, I'll use IMatch's pins to create the collection, which I then select all and drag over to PL.  I typically work on a set of files at once, and this allows me to work in PL without being confined to folders (or its extremely rudimentary library).  Once I have files selected, I usually close IMatch (to avoid indexing intermediate files).

I process my files and export the files as either TIFF or JPG (depending on whether I have further edits).  The files out of DxO are named in the format "Original-dxo" (The original part is more complicated, but the key is the -dxo).

The finished file will come back into IMatch as a new version to the original RAW.  In IMatch, I have categories for software used, so I'll quickly go through the newly imported files and the RAW files and categorize the files for their software.  The categories are color coded, so I quickly see what images have been processed.

Now the other thing I have set up is that my database goes back to a period when my originals were JPGs or TIFFs from cameras that could only do that at best (or scanners).  Consequently, the version setups I have in IMatch are such that if there is no RAW file, a JPG or TIFF can be a master.  This works because my versions always have a suffix tacked on with a "-xxx", and it would allow for a situation where a RAW file could be deleted if I wanted to while keeping its versions with one of the versions taking over as a master.  I rarely do this since hard drive space has gotten so cheap.

Erik

  • Sr. Member
  • **
  • Posts: 286
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #18 on: November 19, 2020, 01:27:54 AM »
Quote
IMatch will not work right if it finds external XMP records AND internal XMP records.

By standards, JPEG files use embedded XMP metadata (in the file).
Also by standards, an XMP sidecar file belongs to all files with the same file name as the XMP file in the same folder.

IMatch is aware of this and merges XMP metadata from the external sidecar file with XMP metadata found embedded in the JPG file - by default considering the embedded XMP metadata to be more precious.

You can the other software companies for all the confusion.
Every RAW processor out there deals with embedded XMP data, XMP data in sidecar files and other important metadata like EXIF and GPS as they see fit. Nobody really cares that much.
Proper metadata management may be critical - but it does not make a good source for fancy ads or magazine and blog reviews.
And most users are unaware of all these potential problems until they switch their RAW processing software, "DAM" or platform...

I guess i shouldn't have said it wouldn't work right.  The user may get unexpected results.  In my experience when the situation accidentally occurred, IMatch overwrote the XMP record from one with the other and didn't merge it, so data was lost.  Granted that was many versions ago now (probably version 5 time frame).  It was not pleasant, but it was a limited situation and a case of these other companies deciding to create XMP data their own way. 

Mario

  • IMatch Developer
  • Administrator
  • *****
  • Posts: 26787
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #19 on: November 19, 2020, 09:07:34 AM »
IMatch always merged embedded XMP and sidecar data when importing XMP data. The merge happens during ingest and the results are stored in the database. This is the metadata you see in the Metadata Panel. During ingest, the embedded XMP is merged 'on top' of the XMP from the sidecar. When you write-back, IMatch stores XMP data into the JPEG, as is is standard.

Aubrey

  • Super Hero
  • ****
  • Posts: 1187
  • IMatch user since June 2008
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #20 on: November 19, 2020, 03:57:45 PM »
Once I'm done processing the raw in PhotoLab, which saves its edits to a sidecar .dop file that sits in the folder right beside the raw and XMP sidecar file, I export the raw as a temporary 16 bit tiff.  Then I open the tiff in Affinity Photo and do my final optimization.
John,
are you aware that you can go straight to Affinity from DxO PL4?
In DxO File|Export to Application...
Select: photo.exe from C:\Program Files\Affinity\Photo

Next time just export to application, it keeps the affinity photo info.

Aubrey.

JohnZeman

  • Global Moderator
  • *****
  • Posts: 1348
  • I'm too damn old to act my age.
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #21 on: November 19, 2020, 04:31:28 PM »
John,
are you aware that you can go straight to Affinity from DxO PL4?
In DxO File|Export to Application...
Select: photo.exe from C:\Program Files\Affinity\Photo

Next time just export to application, it keeps the affinity photo info.

Aubrey.

Thanks Aubrey.  Yes, I have explored that option and I would be doing it that way if my IMatch versions were in TIF format instead of JPG.

But exporting as a 16 bit TIF from PhotoLab directly to Affinity Photo puts a big unneeded intermediate TIF file in my IMatch database when all I really need there is the final JPG version of it.

If PhotoLab had a way I could export to application but specify a different output folder that is not in IMatch then I would be exporting to application.

WebEngel

  • Full Member
  • **
  • Posts: 248
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #22 on: November 19, 2020, 09:14:42 PM »
PhotoLab is pretty good about copying pertinent metadata from the raw file to the output jpg (i.e., things like Description, Author, location, keywords, etc.). So if you use IMatch to add metadata to the raw file, you should also see it in the PL output jpg (and tiff, etc., I believe).

True.  I was missing some metadata, but it seems I was using inappropriate fields ({File.MD.XMP::tiff\Artist\Artist\0} does not get propagated, but other artist fields do.  Will need to research what to use).  In a short test today, I did not notice any other field missing.

Alright, this already solves one of my problems which is: How do I get the metadata into the Jpgs!

WebEngel

  • Full Member
  • **
  • Posts: 248
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #23 on: November 19, 2020, 09:38:28 PM »
Hello
If raw files are deleted as part of the workflow, like in my case, it is a difficult story:
Why do you delete raw files?

Let me explain my workflow a bit more:
All images get a rating and a label.  Rating means quality (the better the photo and the more unique, the more stars).  Rating refers to the potential--an underexposed picture still may get a high rating, as will a picture that will only be good after cropping.  Label "blue" means "needs processing", e.g., because of a wrong exposure or wrong white balance.  "Pink" means image is technically correct and does not need processing.  Half of the pictures are neither pink nor blue as I will later decide whether I still need to process them.

If a picture has 1 star and label pink, I will never process the raw file. I simply dont have the time, and if the Jpg already suffices for me, why should I waste my time in DXO slightly tuning exposure and shadows for an anyway mediocre image?

I NEVER delete raw files for images with 4 or 5 stars.  In fact, just this week I re-processed some old high-ISO RAWs with the DXO's amazing DeepPrime.

Also I don't delete raw files with "blue" labels even if they have only one star, because I may process them some day in the future in order to significantly improve them.  Well, in reality, I hardly ever do that, and if the picture 5 years later still bores me, I delete the raw even if it could benefit from a bit more exposure.

Does this make sense to you guys?

Do you guys really process ALL of your pictures?  I mean I can understand if you don't delete Raw files because storage is so cheap.  Just like not deleting e-mails as long as they don't have attachments.

I do understand that pros probably need to process every picture, because you cannot deliver a technically incorrect pic to a client, and you don't keep pictures that you are not going to deliver.  Make sense.  But I am an amateur.  An image may be a nice memory, but I dont' care whether the highlights need a bit more contrast or not unless the picture is really good.

Hope it explains it.

WebEngel

  • Full Member
  • **
  • Posts: 248
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #24 on: November 19, 2020, 09:55:52 PM »
I get why you are using JPG as the master, but what I think I am confused is why you are using the RAW as a buddy and not as a version.

That is probably a VERY good question.  I started with Imatch 15 years ago and configured it in a way it would work for me.  Maybe I though it would belong this way.  This included writing JPG metadata to XMP records, thereby leaving MWG-compliance--which sets up Mario everytime he hears it (kidding).

Maybe if I make the Raw file a version, I can update the Jpg and at the same time write the XMP record.

I will probably have to create a brand new test DB to test all of that.  Will keep you updated.

Thanks a lot for this suggestion.

martin

Mario

  • IMatch Developer
  • Administrator
  • *****
  • Posts: 26787
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #25 on: November 19, 2020, 10:28:08 PM »
When you change metadata for a master, it is propagated to versions during write-back.
See Propagation

jch2103

  • Oldtimer
  • ****
  • Posts: 2058
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #26 on: November 19, 2020, 10:41:12 PM »
Do you guys really process ALL of your pictures?  I mean I can understand if you don't delete Raw files because storage is so cheap.  Just like not deleting e-mails as long as they don't have attachments.
No. I use IMatch to select which images I want to process (based on initial Rating in IMatch), and often don't process all of those (I mark rejects in DxO before processing a set of images). Like others who have already discussed their workflow above, the RAW file is the master and the JPG output is a version.

When I first started using DxO many years ago, one of the key things that attracted me was the efficient workflow. For many images I just use a 'standard' preset, so those images can be processed without further changes as part of the batch of selected images. For others, most of my adjustments are relatively minor and often done in groups (e.g., apply ClearView Plus to a set of landscape images). In the end, it doesn't take that much time and avoids complicated issues of jpgs as masters, etc.
John

JohnZeman

  • Global Moderator
  • *****
  • Posts: 1348
  • I'm too damn old to act my age.
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #27 on: November 20, 2020, 12:17:00 AM »

Do you guys really process ALL of your pictures?

I do.  But then I'm not a pro and I'm retired so I have time to process each individual image.  In fact I enjoy taking my time to do it. :)

sinus

  • Global Moderator
  • *****
  • Posts: 4153
  • IMatch-User since 2001 (IMatch 3.6)
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #28 on: November 20, 2020, 08:27:46 AM »
Do you guys really process ALL of your pictures?  I mean I can understand if you don't delete Raw files because storage is so cheap.  Just like not deleting e-mails as long as they don't have attachments.
No. ...

So do I.
I do not process all of my pictures, maybe, hmmm, 10-50% of them (depends on the event), and of course I delete the very bad images.
But I do not delete a RAW, what is not that bad to delete.
Storage is cheap, and IMatch handles my 300'000 files simply very good and quick enough.  :)
Best wishes from Switzerland! :-)
Markus

WebEngel

  • Full Member
  • **
  • Posts: 248
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #29 on: November 20, 2020, 11:26:55 PM »
Couldn't get the interface done, so asking again?

The External Tool Favorites.

@John, are you just transferring one file over?  Or multiple?  I managed to get one but not multiple.

For each session in DxO, I create a collection in PL.  Usually, I'll use IMatch's pins to create the collection, which I then select all and drag over to PL.  I typically work on a set of files at once, and this allows me to work in PL without being confined to folders (or its extremely rudimentary library).

@Erik, are you referring to "New project" in PL?  That's the only thing I found.

If yes, have you managed to use a command?  I haven't.  All I got is drag&drop.  Which is a bit tricky because then I need to have the raw files selected.  With a command, I could use the external tool as described by John above, and click on the Jpg and still transfer the Raw to DxO.  Also, Drag&Drop requires me to resize my windows.  Not bad but not extremely comfortable.

Martin

jch2103

  • Oldtimer
  • ****
  • Posts: 2058
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #30 on: November 20, 2020, 11:42:10 PM »
(This isn't a reply from Erik or John Z.)

I created a Favorite/Application item for PhotoLap (drag and drop the desktop icon into Favorite/Application). If you then drag and drop one or more image files from the IMatch File Window onto the Favorite, PhotoLab will open with a new External Selections project in PhotoLab. Very simple and straightforward. No need to create special tags/collections for the images in IMatch, unless you want to.
John

JohnZeman

  • Global Moderator
  • *****
  • Posts: 1348
  • I'm too damn old to act my age.
Re: Workflow with DXO (Imatch 2020 with PL 4)
« Reply #31 on: November 21, 2020, 12:04:33 AM »
Yep, that's the way I do it too.  I just drag the desktop shortcut created by PhotoLab during the program installation to my IMatch favorites.  With that I can use IMatch to send one or several images to PhotoLab.

Maybe you might want to check the properties of the shortcut you're using in your IMatch favorite?  Below is the Target config in the properties of mine.

Code: [Select]
"C:\Program Files\DxO\DxO PhotoLab 4\DxO.PhotoLab.exe" -mode=openwith
As an aside once I have sent the files to PhotoLab I'll usually assign those files to a PhotoLab Project.  That way I don't have to process all of those files in the current PhotoLab session, I can take my time and process them as I want.  (I usually delete the external selections in PhotoLab, I prefer the versatility of the Projects instead).
« Last Edit: November 21, 2020, 12:38:11 AM by JohnZeman »