Metadata Write-Back

Started by jln, September 15, 2025, 11:11:21 PM

Previous topic - Next topic

jln

I am having the problem that after having iMatch write metadata back to a file, iMatch continues to tell me that metadata needs to be written back. When I tell the program to write back metadata for all pending files, and then repeat the command, it will tell me that there are still pending files. Sometimes the number of files goes up; sometimes it goes down.

I realize similar problems have been discussed before and that it is mentioned in the help file. I understand that there is a problem with the metadata, but I am unsure how to handle it. A likely factor is my doing poor job on my part with the creation of keywords, particularly hierarchical keywords. I'm not sure how to fix that. Changing the keywords in the Thesaurus does not appear to change files already assigned keywords.

The problem seems to arise with old pictures originally taken with film and pictures taken with early digital cameras. The early cameras I used included a Nikon E2500 and a Nikon 7200. I edited them with DXO and used iMatch to assign keywords and sometimes map coordinates.

I am attaching a file showing my two young adult children taken with the Nikon E2500. When I hover over the Pencil icon, the only metadata item the window says needs to be written is XMP::dc\Subject. The Data Analyst Tool indicates the following errors and warnings:

Metadata Analyst Results. Version 2025.5.2. 9/15/2025 4:34:26 PM
File analyzed: G:\Adam & Margot in Suits\DSCN0317.JPG
Errors: 1
Warnings: 5

Warning: [System] File has unwritten metadata (pending write-back).<br/>The metadata loaded from the image and the data in the database may not match.
Warning: [Metadata] Warnings: 'Invalid PrintIM header'
Warning: [EXIF] Offset TimeOriginal missing. 'Date Subject Created' may have no time zone offset.
Warning: [EXIF] Offset TimeDigitized missing. 'Create Date' may have no time zone offset.
Warning: [XMP] [ExifIFD]:UserComment and [XMP-dc]:Description (embedded) mismatch.
Error: [Keywords] Flattened hierarchical XMP (embedded) keywords don't match XMP keywords.

Is there a way to fix this? Should I be concerned? Repeated asking iMatch to write back the metadata doesn't help.


Mario

This is a not uncommon problem when you deal with files containing metadata mess. I covered this in the Metadata Problems and Pitfalls help topic, in the Files Pending Again After Write-Back

For some reason, IMatch/ExifTool are unable to make the EXIF user command and the XMP description match (these should be identical after write-back).

The same is true for hierarchical and flat keywords. Which is a more common problem than the problem with the description.
Depending on how you have configured IMatch to flatten hierarchical keywords (Edit menu > Preferences > Metadata) and the keywords already existing in the image, IMatch may be unable to automatically synchronize keywords during write-back. If a second write-back does not help, manual user intervention is required.

I was able to fix the problem using these steps:

1. Select the image and in the Metadata Panel, click the pen in front of the Description and Hierarchical Subject tags. This marks them as modified.

2. Click on the green save check mark icon in the ribbon.

3. Write back the file by clicking the pen. The pen comes back.

4. Write back the file again. The pen is now off.

Under Edit > Preferences > Metadata, I use these settings:

You cannot view this attachment.

Please try these steps at your end and let us know if they worked. If not, please include screen shots of your E > P > Metadata and E >  P > Metadata 2 settings.

Note: I have removed the image you posted after downloading it, because it contains private data like names, faces and locations. It is not a good idea to post such information in a public place. You can send me images like this in private to my email (see support email address) and include a link to the community topic next time.

jln

It didn't seem to work. Just to be sure I am doing what you said, in the Metadata panel, under the "Core Data" section, I select the pen beside "Description Writer" and the pen beside "Hierarchical Keywords [8]." Then I click the green check arrow at the top of the Metadata panel. Then I click the pen in the image, wait for it come back, and click it again. It comes back again. 

In my Metadata preferences, I did not have "Write path elements" checked, but checking it did not change things. I am attaching screenshots of my two Metadata preferences.

Mario

Not Description Writer, Description.
Write path elements must not be checked if you don't want. Whatever works for you.

Try again, this time marking description as changed, not description writer.
When the pen comes back, point at it to see which tags IMatch needs wants to write.

jln

I can't find "Hierarchical Subject" tag. I can find "Hierarchical Keywords," but there are two: one under "XMP Adobe Lightroom" and the other under "Core Data." "Description" is under Core Data, but the field is empty. It doesn't seem to matter which I choose, although if I choose both, it results in the set of keywords appearing twice.

Two screenshots attached. The longer one is what I see when I put the pointer on the pen immediately after clicking the green check mark. The shorter one is what I see when I put the pointer on the pen both after clicking the pen the first time and clicking it the second time.

Mario

Hierarchical Subject = Hierarchical Keywords.
It's named hierarchical Subject (and Subject) in the XMP / IPTC standards, but nobody understand this. So most software just names it keywords in the UI, and everybody knows what is meant.

Hierarchical Keywords can appear multiple times in the MD Panel, depending on which layout you use.
XMP-dc:Subject (flat keywords) is part of the Dublin Core namespace. XMP-lr:hierarchicalSubject is part of the Adobe Lightroom XMP namespace. And there are also keywords in IPTC (this time even named keywords).

Yes. Metadata is complicated. Which is why IMatch tries to make it look simple for users.

It seems that IMatch struggles to synchronize hierarchical and flat keywords (XMP-dc:Subject) for your images. Which is what I mentioned in my firs reply and also list in the Keyword Problems section in the Metadata Pitfalls help topic.

IMatch cannot successfully flatten hierarchical keywords for the image you have used for your test.
Since I did not see these problems for the image you have provided for testing, and this is the same file (!), try this:

I the MD panel, click the pen in front of

You cannot view this attachment.

to mark them as modified. Write back. If the pen comes back after write-back and it shows again XMP-dc:Subject, Your metadata settings, existing metadata in the image and/or thesaurus prevents IMatch from synchronizing the keywords in the image. In this case, we must use brute force.

Make a backup of the image, just in case.
Then select the image, open the ExifTool Command Processor with <F9>,<E> and run the this preset to delete the XMP-dc:Subject

# im-warn
-overwrite_original_in_place
-xmp-dc:Subject=
-charset
filename=UTF8
{Files}

Save the preset under a name like "Delete XMP Subject" for later reuse.

Close the ECP and click the pen icon for hierarchical Keywords again and write back. If the pen comes back, write back again. This should do it.


Stenis

I don't even use hierarchich keywords but I also have these problems all the time. To force an update using the three key command does not seem to solve the problem because often the yellow pens pop up again in a few seconds.

Mario

#7
Quote from: Stenis on September 18, 2025, 11:00:42 AMI don't even use hierarchich keywords but I also have these problems all the time. To force an update using the three key command does not seem to solve the problem because often the yellow pens pop up again in a few seconds.
Read and follow the recommendations in this help topic to solve these issues: Metadata Problems and Pitfalls

Pay close attention to your keyword flattening settings and your thesaurus. The "pen comes back with XMP-dc:Subject in the tooltip" problem is always caused by a mismatch between hierarchical and flat keywords.

If IMatch cannot resolve this with two write-backs, manual correction is required. Not even IMatch can repair all metadata issues by itself.

Run the Metadata Analyst to check the files for problems and get base data for deciding how to resolve the metadata issues.

Just posting "I have this problem" is not very helpful.
If you want help with this, provide a sample file which exhibits the problem, your metadata and metadata 2 settings (screen shots), your thesaurus and the analysis report from Metadata Analyst (use the GREEN BUTTON to copy the info into the clipboard). This gives us a minimum of information to work with.

In almost all cases, the problem is caused by metadata mess in the image, or wrong settings for keyword flattening or the thesaurus.

Tveloso

Quote from: Mario on September 18, 2025, 10:02:16 AMIf the pen comes back, write back again. This should do it.
I routinely have this behavior as well.  After performing a Write-back on a set of files, a small subset of them come back as pending, with XMP-dc:Subject as the only tag that needs to be written.  The second write back always takes care of it.

Quote from: jln on September 15, 2025, 11:11:21 PMChanging the keywords in the Thesaurus does not appear to change files already assigned keywords.
There is an option to update existing files with the changes you have made in the Thesaurus, as described here:

Updating Keywords in the Database from Thesaurus Changes
--Tony

jch2103

I also routinely have the issue, and the second write back does take care of it, but it's an extra step. 
John

Mario

When you routinely have this problem, maybe some analysis would be helpful?

Before the write-back, does Metadata Analyst show any problems?
After the first write-back, when the pen comes back, what is the output of the Metadata Analyst now?
Two write backs mean, basically:

+ Write hierarchical keywords and sync to flat keywords.
+ Re-import the result.
+ The re-import produces new/different flat keywords, since thesaurus lookups did not work, so the file becomes pending again.
+ Now the "new" flat keywords are written in the second pass, and things are finally in-sync.
This usually depends on the keyword settings and the thesaurus.

Doing a post and pasts analysis with the Metadata Mechanic may shed some light on this and help you figure out why this happens.

Stenis

Quote from: Mario on September 18, 2025, 11:34:41 AM
Quote from: Stenis on September 18, 2025, 11:00:42 AMI don't even use hierarchich keywords but I also have these problems all the time. To force an update using the three key command does not seem to solve the problem because often the yellow pens pop up again in a few seconds.
Read and follow the recommendations in this help topic to solve these issues: Metadata Problems and Pitfalls

Pay close attention to your keyword flattening settings and your thesaurus. The "pen comes back with XMP-dc:Subject in the tooltip" problem is always caused by a mismatch between hierarchical and flat keywords.

If IMatch cannot resolve this with two write-backs, manual correction is required. Not even IMatch can repair all metadata issues by itself.

Run the Metadata Analyst to check the files for problems and get base data for deciding how to resolve the metadata issues.

Just posting "I have this problem" is not very helpful.
If you want help with this, provide a sample file which exhibits the problem, your metadata and metadata 2 settings (screen shots), your thesaurus and the analysis report from Metadata Analyst (use the GREEN BUTTON to copy the info into the clipboard). This gives us a minimum of information to work with.

In almost all cases, the problem is caused by metadata mess in the image, or wrong settings for keyword flattening or the thesaurus.

It can absolutely be a metadata mess in the images since many was originally edited with PhotoMechanic.

The strange is though that it is so random. In some ways they get updated but I never know when and how. I just press Shift+Alt+S and hopes for the best because I have felt that is all I can do.

jch2103

Quote from: Mario on September 18, 2025, 06:12:14 PMWhen you routinely have this problem, maybe some analysis would be helpful?

Before the write-back, does Metadata Analyst show any problems?
After the first write-back, when the pen comes back, what is the output of the Metadata Analyst now?
Two write backs mean, basically:

+ Write hierarchical keywords and sync to flat keywords.
+ Re-import the result.
+ The re-import produces new/different flat keywords, since thesaurus lookups did not work, so the file becomes pending again.
+ Now the "new" flat keywords are written in the second pass, and things are finally in-sync.
This usually depends on the keyword settings and the thesaurus.

Doing a post and pasts analysis with the Metadata Mechanic may shed some light on this and help you figure out why this happens.
I ran tests on some images; no problems with write-backs. I'll keep monitoring and will report back if I find issues. 
John

jch2103

I spoke too soon. I had an image that had no errors in Metadata Analyst, then I added hierarchical keywords. I hadn't noticed that the image later had a rewrite checkmark for 'Tags to Write: XMP::dc\Subject'. After the rewrite, the checkmark went away of course. 
 
Keywords

  •  XMP (sidecar) keywords found.
  • 'damselfly'; 'spreadwing damselfly'
  •  XMP (sidecar) hierarchical keywords found.
  • 'animals|insect|damselfly|spreadwing damselfly'
  •  Flattened hierarchical XMP (sidecar) keywords don't match XMP keywords.
  • animals;insect;damselfly;spreadwing damselfly

John