Keywords won't stick after being assigned

[lnh]

Being such a novice I'm not sure if I'm seeing a bug or some obvious thing isn't configured right. I have pairs of raw and jpg files (no versions defined at this time), and when I keyword using the keyword panel I can't get the keywords to stay assigned to the raw files. Moments after doing a write back, most, but not all of the keywords disappear. I've tried assigning each file with the same keywords together, individually and doing the copy function in the keyword panel as described in the help file. An example of the last attempt can be seen in this uploaded video at: http://youtu.be/D9uxuYFr4KM

This is happening to many of my files, but not all of them. Is some setting wrong?

[Mario]

The typical reason for this is that the RAW file has an IPTC record with keywords.
On import, IMatch imports the existing IPTC keywords into XMP. On write-back, IMatch is forbidden to update the IPTC in the RAW, which creates two different sets of keywords (the old keywords in IPTC, your new keywords in XMP).

You can check this by looking at your RAW file in the ExifTool Command Processor. Select the file, press <F9> followed by <E>. Select the "List Metadata" template and press <F9>. Search the output for IPTC data, especially keywords.


Solution:

a) Either allow IMatch to update the existing IPTC data in your RAW file (Edit > Preferences > Metadata 2: File Formats)
or
b) Remove the legacy IPTC record from your RAW file. There is a preset for that in the ExifTool Command Processor. But first verify that this is the problem.

[lnh]

I took a look at the file with ExifToolGUI:

1) No IPTC data
2) XMP data is present which wasn't in the pre-import file. It appears to be placed there by IMatch.

XMPToolkit---Image::ExifTool 9.68
Subject----EVENT*graduation
HierarchicalSubject----EVENT|graduation
Creator Tool----photools.com IMatch 5.1.0.10 (Windows)

3) If I look at the XMP sidecar file, there is an entry:

HierarchicalSubject----EVENT|graduation*PEOPLE|Moya Hallstein*PEOPLE|Moya Hallstein|cousin*PEOPLE|Moya Hallstein|female*PEOPLE|Zoe Borden*PEOPLE|Zoe Borden|female*PEOPLE|Zoe Borden|niece

The sidecar seems to have the right data, but the main file doesn't. I didn't think that IMatch wrote to the RAW files at all.

[Mario]

By default IMatch reads and writes XMP data in sidecar files for all RAW formats.

It imports embedded XMP data and merges it with XMP data in sidecar files, where the embedded XMP data in the image is considered more important and thus overrides data from the sidecar file. You should not have two competing XMP records.

Check your configuration under Edit > Preferences > Metadata 2.
See this FAQ for details: https://www.photools.com/community/index.php?topic=2624.0
This FAQ is for ratings, but it's about multiple XMP records for the same file.

[lnh]

Mario,

Thank you. Clearing out the xmp section of the raw file solved the problem. The question is, how did I make IMatch write to the xmp section of the raw file in the first place?

I've looked through various settings and don't see where it can do this (I haven't changed any settings like this from my memory).

[lnh]

Without question, IMatch is writing to the xmp section of raw files. Doesn't do it all the time, but when it does all the stuff around keywords and categories starts acting wacky. Keywords that you may have previously had associated with a file and deleted will reappear and stuff like that. Cleaning out the xmp part of the raw allows you to make it work again. Feels like a bug.

[Mario]

Without question, IMatch is writing to the xmp section of raw files. Doesn't do it all the time

What RAW files are you using?
What are your metadata settings?
What are your metadata file format settings for the file format in question?
Which operations do you perform to make IMatch embed XMP in the RAW file?
..

[lnh]

Without question, IMatch is writing to the xmp section of raw files. Doesn't do it all the time

What RAW files are you using?
What are your metadata settings?
What are your metadata file format settings for the file format in question?
Which operations do you perform to make IMatch embed XMP in the RAW file?
..

1) Panasonic RW2 raw files from a GM-1
2)
In Metadata: Checked: Keyword Export-Write hierarchical keywords / Write path elements. Keyword Import: Don't replace existing hierarchical keywords.
In Metadata2:
Basic: MWG Compliance-Yes
Protection: Protect unwritten metadata-Yes / Protect Rating and Label-No
XMP Export: Allow to create XMP files-Yes
Settings: Ignore minor metadata errors-Yes / Preserve date/time of original file-No / Keep existing XMP-Yes
3) In File Format Metadata Options:
RW2: Use default settings is checked (Write IPTC-No / Write EXEF-No / Allow create IPTC etc-No / XMP sidecare file-Default / Use data in THM files-No)
4) I haven't been able to figure out what operations I'm doing which is making it embed. The IMatch signature is in the raw files, so IMatch is doing it.  I've been playing around with different Thesaurus entries and how I'm setting up different schema around categories and keywords to figure out what works best for me. Sometimes I'll have deeper hierarchies and other times I've switched back to a less deep but wider setup. Somewhere in all that experimentation things have gotten mucked up on occasion. If I can figure out something more specific, I'll let you know.

[joel23]

Without question, IMatch is writing to the xmp section of raw files. Doesn't do it all the time

What RAW files are you using?
What are your metadata settings?
What are your metadata file format settings for the file format in question?
Which operations do you perform to make IMatch embed XMP in the RAW file?
..

See my latest bug reports (here and here) when an embedded XMP record for RAW is created (sometimes only with ModifyDate, sometimes with ModifyDate and Keywords), even when sidecars are preferred.

[Mario]

Please keep ExifTool output panel open all times and report what's written there.
We need to find out which operation you perform to cause write-back to RAW files.

[jch2103]

I'm having this problem now too. See also this thread: https://www.photools.com/community/index.php?topic=3074.0.
After I did a forced re-scan of my ~40,000 images database, I suddenly had about 10,000 images with a pending yellow pencil (they were's there before).

I cleared most of them by clicking the yellow pencil in batches, but have a couple of thousand left that didn't want to clear after repeated clicks of the yellow pencil.

I isolated one image and made a copy of it. For this particular image, I had used GeoSetter to set GPS coordinates and keywords based on Country/State/City of the location. Apparently (?), either Geosetter or IMatch wrote XML metadata to the raw NEF file despite my setting defaults to use XMP sidecars. I don't know why this would have happened or why the problem only showed up after my forced re-scan (a year after I loaded these images into IMatch.

Here are the steps I followed for the original NEF:

Yellow pencil showed XMP:dc\Subject was pending.
Clicked yellow pencil three times in succession (waiting for processing to stop after each one). Yellow pencil remained.
Ran ECP 'Delete XML Metadata' on original NEF.
Yellow pencil re-appeared; I clicked once, yellow pencil remained, clicked again and yellow pencil finally went away (I hope for good this time).

Here's a link to the original NEF and XMP and to an untouched NEF/XMP copy, along with the ExifTool output text:
https://www.sugarsync.com/pf/D6380252_70755938_6711992
I've also attached screen shots of settings for Metadata, Metadata 1 and for NEF files.

Let me know if you need more information.


Edit: I was able to clear most of the remaining 'pencil' files with single clicks, but at the moment I have about 65 that will need additional treatment (hopefully the method used above). The files I was able to clear with one click had hierarchical keywords pending; the remaining ones all show XMP:dc\Subject pending. I believe I only used Geosetter for a small number of these; rather, I think the only metadata editing was done with IMatch (Creator Tool was 5.0.0.34, 40 and 106).


[attachment deleted by admin]

[joel23]

Apparently (?), either Geosetter or IMatch wrote XML metadata to the raw NEF file despite my setting defaults to use XMP sidecars.

When you have configured Geosetter in the right manner and it still happens, it's IMatch which creates embedded XMP, overriding the settings for RAW. It just ignores XMP files when certain actions were launched, like copying metadata or rescanning images, especially when IPTC exists and "write IPTC" is set to Yes. See my former post in here and its links.

That for my RAW are r/o - which sometimes produces some other side effects, but those are (easier) to handle

why the problem only showed up after my forced re-scan (a year after I loaded these images into IMatch.

than to handle those oddities after months or years.

[Mario]

If some of the remaining files had metadata added with an early Beta, many things can happen. Metadata handling, mapping and production has changed many times since then. Pushing the data twice into the file usually resolves the problem, unless a rare infinite keyword loop has been constructed by the user, where each import generates 'new' hierarchical or flat keywords because of the file's contents, the settings made under E > P > Metadata / Metadata 2 and the thesaurus contents. In such cases only a file-by-file analysis help, basically looking at what IMatch writes via the Output panel and then looking at the file itself.

[jch2103]

I finally got the last of the remaining files written. For the very last ones I ran the ECP command to delete XMP data in the file, followed by two clicks on the yellow pencil. I have no idea how XMP data was written into my NEF files. Hopefully, this issue is now resolved. But then again, I didn't expect to suddenly find 10,000 files that needed metadata writes...

[Mario]

Double-check your per-file format settings under Edit > Preferences > Metadata 2 for the NEF/NR2 formats.

[jch2103]

Double-check your per-file format settings under Edit > Preferences > Metadata 2 for the NEF/NR2 formats.

See above jpg attachment: https://www.photools.com/community/index.php?topic=3025.msg20720#msg20720
All default.