F_P Propagation Script

[Ferdinand]

**Please test this script on test (duplicate) images before you use it on production images.  It's still a fairly new script.**

Since IMatch 5.1.14 was released most of this script is redundant.  That is, most of what this script can do, IMatch can now do.  (I wrote this script to propagate to only selected versions rather than all versions, but IMatch can now do that.)

You still use this script if you want to:
.  Copy only keywords but not the rest of XMP; or
.  Copy XMP (all of it or just keywords) and keep existing keywords in the versions being propagated to; or
.  Use some combination of propagation options not available in IMatch (i.e. you use this script as a base and hack its code).

This script enables you to select some versions and propagate XMP, EXIF and IPTC to only those versions.  The script itself does not propagate things that exist only in the database, such as bookmarks, flags, dots, pins, attributes, annotations - that's too much work and the native propagation is fine for that, since it doesn't change the files and their sidecars.  If you also want to propagate those things then you can select the script's first option - to trigger IMatch's own propagation.  This will cause IMatch to propagate whatever is configured in file relations to the selected versions. 

(I made one exception to the above - this script will also propagate categories, but since IMatch 5.1.14 this isn't necessary - you can use the option to trigger native propagation to do that.)

This script assumes that you have file relations configured and working.  It uses file relations to find the master, so if the versions you select don't have the little blue versions icon under the thumbnail then the script won't work (and you'll get error messages in the scripting output window).

To run the script you select the versions that to want to propagate to.  (Actually, you can select any files you like to run the script on, but any that are not versions are ignored and a warning is printed to the output window.)  The script locates the master for each version, it triggers Imatch propagation to the selected versions if that option is enabled, uses ExifTool to copy whichever of XMP, EXIF or IPTC you have enabled, and propagates any categories you have selected and their subcats. 





Trigger propagation to these versions as per file relations - As it says, before any other actions this script can trigger Imatch to use its file relations settings to propagate metadata and collections to the selected versions.  It's up to you to ensure that there isn't any overlap between what IMatch propagates and what this script does.  I'm not sure what takes precedence if there is overlap.

Copy XMP - There are options to clear XMP and to copy it.  I find that clearing it before a copy is a good precaution.  You can copy all XMP or just keywords.  There are options to exclude ratings and labels from XMP copy - an often requested feature - and if you select this option then ratings and labels are excluded from the XMP clear command as well.  You can also choose to retain existing keywords in the selected versions rather than replace them as part of the copy, and they are retained even if you select XMP Clear.

This script will search for an XMP sidecar to copy XMP from, and if it can't find it then it will use the file.  However you can force it to use the file by using the "Ignore XMP sidecars and copy XMP from master file" option.  In the case of CRW it looks for a .THM if it can't find the XMP sidecar. 

Copy EXIF and IPTC - Like XMP, you can copy EXIF and IPTC, and it's a good idea to clear them first.  But given the MWG rules, it is better to populate IPTC from XMP than to copy it, and so the script has  option to run the xmp2iptc.args file.

You can assign masters and versions with successful propagations to a category for checking the outcome.

There are some built-in inclusions and exclusions:
.  Hierarchical keywords are included in the XMP clear and in the XMP copy
.  Lightroom and ACR edit settings (xmp-crs) are excluded from the XMP copy
.  XMP orientation (xmp-tiff:Orientation) is excluded from the XMP copy, since it can cause issues
.  GPS (gps:all) is included in the copy
.  Only standard file types - JPG, TIFF, PSD, DNG can be propagated to - other file types are skipped
.  Versions with more than one master (rare but possible) are skipped.
.  If you select category A to propagate then all its subcats will be included in the propagation.

[attachment deleted by admin]

[sinus]

Hi Ferdinand

Thanks for your effort and for your very good explanations.
I downloaded the script and will it try, if I have the time.
But I am sure, that it is a good script. 

[Erik]

I will definitely be downloading this script and giving it a try. 

I do hope those other feature requests get implemented, but in the meantime, I think this is a reasonable way of handling things until either of those feature requests gets implemented, if they do.

[Ferdinand]

I thought you'd be interested Erik.  As always, please test this first on test images.  I'll add a note to that effect to the initial post.

Out of interest, what else would you want to propagate?  I'm giving some thought to grafting on the code from my synchronisation script that propagates categories, because that's the only other thing I propagate, and it would certainly simplify my workflow.  And it's always going to be hard to use the native propagation for XMP as I don't want to propagate labels, and Mario has resisted more options in the file relations propagation settings.

[Ferdinand]

I decided that I was prepared to set aside an hour or two to simplify my workflow, by adding the ability to propagate categories to this script.  The updated version 1.1 is attached to the first post in this thread.

[cytochrome]

Hello Ferdinand,

I tried out the script and must confess I don't get it to work, maybe because I don't understand what it should do.

What I did:

- test folder with a NEF and a subfolder .\ASP with a JPG derivative
- I have NEF to JPG relations but here I did not activate them (no F4-R and F4-P ==> no master/version icons)
- I selected the JPG, chose to propagate XMP (they are in the NEF) and my categoty Quoi/réunion (What/meeting)
- run the script ==> nothing hoppens

the script output:
============
10/08/2014 12:38:42: File 1 of 1: This file is either a master or not a version and so was skipped:
   F:\Images\Reserve_Util\ASP\_DSC5504_ASP.jpg

10/08/2014 12:40:33: File 1 of 1: This file is not a standard file type and so was skipped:
   F:\Images\Reserve_Util\_DSC5504.NEF
==============

So it found the master but decided it was not a master and decided also that the JPG version was not a version...

Sure I don't get how it works or what it should do.

If I select the NEF instead it dosen't wotk either and produces:

==========
10/08/2014 14:57:17: File 1 of 1: This file is not a standard file type and so was skipped:
   F:\Images\Reserve_Util\_DSC5505.NEF
============

Francis

[Ferdinand]

First of all, in checking this I found that I had left in a line of debugging code that will print stuff to the output window.  Bother!  I've just uploaded a fresh version of 1.1 without it.

- I have NEF to JPG relations but here I did not activate them (no F4-R and F4-P ==> no master/version icons)

The script uses file relations to find the master.  It can't work without them.  I've added a note to this effect in the first post.  I thought that this was obvious, but perhaps only to me. 

- I selected the JPG, chose to propagate XMP (they are in the NEF) and my categoty Quoi/réunion (What/meeting)
- run the script ==> nothing hoppens

the script output:
============
10/08/2014 12:38:42: File 1 of 1: This file is either a master or not a version and so was skipped:
   F:\Images\Reserve_Util\ASP\_DSC5504_ASP.jpg

10/08/2014 12:40:33: File 1 of 1: This file is not a standard file type and so was skipped:
   F:\Images\Reserve_Util\_DSC5504.NEF
==============

In addition to the fact that file relations were not operational, this means that you must have selected both the NEF and the JPG, rather than just the JPG, even though you thought you didn't.  These error messages are only generated for the selected files.

If I select the NEF instead it doesn't work either and produces:

==========
10/08/2014 14:57:17: File 1 of 1: This file is not a standard file type and so was skipped:
   F:\Images\Reserve_Util\_DSC5505.NEF
============

Nope, that won't work - you have to select versions, that's the whole point of the script.  I'll try to make that clearer.

[cytochrome]

OK, Ferdinand, thanks for the clarification.

I read again your first post and finally understood (late bloomer) what the script does, and I see the advantage to not touch all versions when only one needs to be modified: faster backup. But for those writing xmp/iptc to the image files the benefits are only marginal, it will anyway open the master raw and this one jpg/tif. Also, I have background file relation/propagation set to ON most of the time.

There is a second point: "The script will search for an XMP sidecar for the master as the source to copy XMP from, and if one doesn't exist it uses the master file itself. " This is generally smart but not for my workflow. My ingester (PM5) does not write metadata to the RW2, so it produces xmp sidecars. My settings in IM are to not allow read/write of xmp sidecars. So first thing on import of Panasonic raw into iM I run an ECP preset to update the RW2 from the sidecar. and more than often I forget to delete them. This is no problem for IM since it does not know about sidecars, but your script would find them and update my xmp with old metadata..

This is a very special case so don't waste time on this, I will stick with the IM version propagation even if it costs some extra file copying when I backup my image library.

Francis

[Ferdinand]

I have added an option for your case.  See the top post for new attachments and edited text.

I don't understand this comment: "But for those writing xmp/iptc to the image files the benefits are only marginal, it will anyway open the master raw and this one jpg/tif.".  If this is a reference to this script, there's no writing to the RAW.  Only standard file types are written to, never RAW.

The script is intended for the situation where the RAW is unchanged but you add a new version.  I think there's a performance benefit in this instance.  Also for the situation where you want to propagate XMP excluding ratings and/or labels.

[Erik]

I thought you'd be interested Erik.  As always, please test this first on test images.  I'll add a note to that effect to the initial post.

Out of interest, what else would you want to propagate?  I'm giving some thought to grafting on the code from my synchronisation script that propagates categories, because that's the only other thing I propagate, and it would certainly simplify my workflow.  And it's always going to be hard to use the native propagation for XMP as I don't want to propagate labels, and Mario has resisted more options in the file relations propagation settings.

I haven't tested it out, yet, but it sounds like it works as I'd expect, and I always test on backup images.  I'm hoping to get back to IM this week.  Regular work has been zapping my photography time, lately.

[Mario]

@Ferdinand:

On a side node: IMatch 5.1.14 contains a Propagate method which allows you to propagate from a Master to all or selected versions. I've added this because I did some work in how IMatch propagates and detects versions. We will also have a "Propagate" command which allows you to propagate to the selected version(s) in the File Window and Version Panel.

Not sure if this makes your script easier or not.

[cytochrome]

I have added an option for your case.  See the top post for new attachments and edited text.

I don't understand this comment: "But for those writing xmp/iptc to the image files the benefits are only marginal, it will anyway open the master raw and this one jpg/tif.".  If this is a reference to this script, there's no writing to the RAW.  Only standard file types are written to, never RAW.

The script is intended for the situation where the RAW is unchanged but you add a new version.  I think there's a performance benefit in this instance.  Also for the situation where you want to propagate XMP excluding ratings and/or labels.

I thought the script had to open the raw to read the in file xmp  and this would take longer than for the sidecar, which is certainly silly...

Lets see how the new propagation scheme works in IM 5.1.14 and then I have a rich choice...

Francis

[Ferdinand]

I thought the script had to open the raw to read the in file xmp  and this would take longer than for the sidecar, which is certainly silly...

I'm not sure, but I guess it probably would take longer to read from the image rather than the sidecar, but I suspect not that much longer.  And in any case it's something you have to do once per propagation no matter how you do it.  It's writing to the image which is much slower, and that's where this script is faster than 112, since it only writes where it's asked to.

[Ferdinand]

Thank you Mario for implementing these new features.       I've had both these requests in for some time and although I admit that I have dropped the occasional hint, my own view has been that bug fixes had a higher priority, which is why I wrote this script.

On a side node: IMatch 5.1.14 contains a Propagate method which allows you to propagate from a Master to all or selected versions.

This certainly makes the script easier and more powerful, but that said ...

We will also have a "Propagate" command which allows you to propagate to the selected version(s) in the File Window and Version Panel.

This new command makes this script almost redundant, which as I said in the first post, is just fine with me. 

I say "almost" because if you want to propagate XMP but not ratings or labels, then you still need this script.  There is a feature request that covers this (there may be others).  I'd be keen to support the ability to propagate XMP without ratings or labels.

[Mario]

admit that I have dropped the occasional hint, my own view has been that bug fixes had a higher priority,

They have. But a) I can fix bugs only for so long at a time, and I was fixing bugs in the propagation and related features. Since I had to acquaint myself again with all the code, adding some features along the way was OK. I always try 50%/25%/25% (bug fix, feature requests, new stuff).

[Ferdinand]

I've incorporated some of the enhancements in IMatch 5.1.14 into this script.

This script is mostly redundant from V5.1.14 onwards, but there are a few people who will find it useful to do things that IMatch can't, so I kept it alive.

I've rewritten the documentation completely.  If you're thinking of using this script please read it.

Thank you to Mario for making it mostly redundant.  And I really mean that!!!   

Thank you to Phil Harvey for advice on how to retain existing keywords in the version when copying XMP from the master to the version.

[Ferdinand]

I assume almost no-one is using this script, since there's only 7 downloads and the script is almost entirely redundant.  I ask because I may have found a bug, whereby if you run the script on more than 50 files the script appears to work, but in fact doesn't do anything.  This is a bit confusing, as it appears that at some stage it must have worked, but it wasn't today and I can't understand why the change in behaviour.  There seems to be a very easy fix.  If anyone is interested I will post the fix or a new version.

[ubacher]

I think I might soon have a use for this script.

If I do not keep the raw development settings (from ACR) within IM and I want to copy these from a NEF
file to a (derived) dng file I need a routine which takes the XMP record from the XMP file and copies it (or the cr part)
to the dng.

[Mario]

This may break your file. LR will use the settings found in the DNG, but since they really belong to a NEF, you may experience all sorts of troubles. Probably. I would suggest you ask Adobe before you do something like this.

[Ferdinand]

The simple change to make is to edit the 9th line of code from the bottom and change
'Hnd = OpenProcess(SYNCHRONIZE, 0, Pid)
to
Hnd = OpenProcess(SYNCHRONIZE, 0, Pid)
by removing the ' at the start.

I can't imagine why I would have made this change.

Re the DNG, I would not have thought it wise to copy the ACR settings to a developed DNG, although that said, LR puts these settings in a developed JPG.  But I wonder what happens if you copy the settings to a DNG created by the Adobe DNG converter, since it's still sort of RAW?  I guess there's no harm in trying on a test file.  But surely it's safer to copy and paste the settings from the NEF to the DNG inside LR?

[ubacher]

When IM kept the camera raw tags the propagation from a NEF (XMP) file to the newly created dng file
(converted using dngconverter) worked just fine.  PS then sees the settings which I had used when converting the
NEF file when I open the dng file.

I considered the alternative of leaving camera raw tags enabled and every now and then disable it temporarily to run
diagnostics to purge.

[Ferdinand]

I don't understand, but good luck.