Why does iMatch want to write XMP::Lightroom/HierarchicalSubject?

[afx]

Just simply importing images leads to a pen icon indicating unsaved metadata being shown for all images.
The mouse-over text indicates that iMatch wants to write back XMP::Lightroom/HierarchicalSubject.

How can I get rid of this?
Looking at the metadata preferences I don't see anything that would make me think this is configurable.

thx
afx

[Mario]

Your files contain keywords in IPTC or XMP::dc:subject which are not yet in the hierarchical keywords, or the rules you have configured under Edit > Preferences > Metadata for keyword import and mapping produce "new" hierarchical keywords.

Tip: Setup a custom metadata panel layout which shows IPTC::keywords and XMP:hierarchicalSubject. This makes it easier to tell if and which  hierarchical keywords are produced on import.

After the first write-back the hierarchical keywords and the IPTC:Keywords and XMP:dc:subject are synchronized.

[Ferdinand]

Greetings AFX.  Welcome back to the forum - it's good to have you here.  (For those that don't know, AFX is an old-timer from the Bibble5 / Aftershot forums).

I think the problem you're having will be a common one and I'm not sure what to suggest to Mario that he should do about it.

Not sure if you have had a chance to get across hierarchical keywords in V5 (note: not categories).  If you assign a hierarchical keyword Where|France|Paris to an image then IMatch will want to write this to XMP::Lightroom/HierarchicalSubject in the file.  It will also want to write some or all of these nodes to flat keywords in the file.  You can control which nodes are written via metadata preferences > keyword export.

Now I know you haven't done this yet.  What IMatch is doing to you is reading in your flat keywords.  From the help file: "If you import an image file for the first time and the file has no hierarchical keywords, IMatch fills the hierarchical keywords from the flat keywords in the file, optionally trying to lookup the keywords in the thesaurus to produce the proper hierarchies".    But IMatch then sees that the XMP::Lightroom/HierarchicalSubject is empty, and so wants to write these keywords to this field.  Since keywords (unlike categories) are always stored in the file (or sidecar), there's a writeback that's initiated.  It won't also initiate a writeback to the flat keyword fields since the keywords are already there.

I wonder if there should be an option to turn this import completely off, if that's what people want - at the moment I don't think there is.  I think it's going to be quite common for people to have flat keywords, and will find all these writebacks queued up when they ingest images and before they've gotten across this whole keyword thing worked out.  If they enable immediate writeback then there's going to be a lot of background activity and sluggishness.

[afx]

Looks like Ferdinand explains the default quite well, thx!

To my knowledge I have not configured anything explicitly for the hierarchical keywords.
Hmm, is the write hierarchical keywords toggle on or off by default? I see it on here right now.

Me thinks the shipped default should in no way ever trigger a write, but now I am not sure whether I clicked that thing or not.

I am currently only toying around, not letting iMatch write back anything.

@Ferdinand: Working more with LR than AS right now, but have both in the mix and I can't stand the DB of either.... I just wish iM5 was further ahead in development...

cheers
afx

[Mario]

Me thinks the shipped default should in no way ever trigger a write, but now I am not sure whether I clicked that thing or not.

The IMatch defaults have been carefully chosen to be compliant with the Metadata Working Group standard and the typical workflow in a professional environment.

This means, for instance, that keywords must be synchronized between IPTC:keywords, XMP:dc:Subject and XMP:lr:hierarchicalSubject. IMatch may also add or update other XMP values when files are imported, depending on the existing IPTC, GPS and EXIF data and the import options chosen. All this will produce new  XMP data which is not stored in the file, and IMatch therefore marks the files as pending. You don't need to write back the new data if you don't want.

[afx]

Changing files without the user having told the program to do so by default is broken by design.
Writing Lightroom hierarchies into files that come from Lightroom without them sounds just plain silly.

And what is the default for the write hierarchical keywords toggle in the preferences?

cheers
afx

[Mario]

Changing files without the user having told the program to do so by default is broken by design.

IMatch does not change the file - unless you write back metadata or you enable the automatic write-back.
It only produces a MWG-compliant metadata record from the contents of your file, and puts this record into the database. Since this XMP record does not match the existing XMP record in the file (or the file has no XMP record), it indicates this with the write-back icon.

And what is the default for the write hierarchical keywords toggle in the preferences?

Do you mean the options which control if and how IMatch converts hierarchical keywords into flat IPTC/XMP keywords in Edit > Preferences > Metadata? If so, please see the corresponding help which explains all the options in detail. Just press <F1> while you have the configuration dialog open, or use th search function in the Help to find all related information.

IMatch internally always works with hierarchical keywords.

And if you are irritated by the Lightroom hierarchical keywords: That's the only (so-so) standard we currently have for working with hierarchical keywords in XMP. The MWG hierarchical keywords added recently is bloated and will cause great grief for typical IMatch users with dozens or hundreds of keywords and deep hierarchies. I'm still looking into it and may support it in a later edition of IMatch 5.

[Ferdinand]

Changing files without the user having told the program to do so by default is broken by design.

Just to clarify what Mario said - the install default has "Writeback changes to metadata immediately" set to off.  Which is why the user needs either to turn this on and flush the changes or click the yellow pencil to trigger writeback, or do so from the commands menu.  So files are not actually changed until this happens.  If you don't want all these queued writebacks then you can clear them from Database > Database Tools > Clear Processing Queues.

Writing Lightroom hierarchies into files that come from Lightroom without them sounds just plain silly.

I don't understand this.  Part of my problem was that I never input metadata in a RAW converter - only IMatch or Geosetter.  So to test this I put virgin RAW and JPG into LR and assigned keywords to both.  I assigned A, B|C, and D|E|F.  I then imported into IMatch5 and what I think were default metadata settings (esp both options under keyword export should be checked).  There was no writeback because everything was already in the right place.  My understanding and my experiments indicated that LR (5.3) fills in the hierarchical field when you enter either a flat or hierarchical keyword, as well as the flat fields.

Which is what I thought - Mario designed this for Adobe compatibility.  If you've entered keywords via LR and are getting write-backs queued immediately after ingesting in IM5 then something is not right.  Care to post a demo file and / or a screenshot of your metadata preferences?

P.s.  re RAW converters, I've got one of everything at the moment, but LR is used mostly.  I rarely use ASP.  Sometimes I still use B5 for NEF B&W.

[afx]

Changing files without the user having told the program to do so by default is broken by design.

IMatch does not change the file - unless you write back metadata or you enable the automatic write-back.

So how many users will understand this?
They see the pencil icon and will just write back to perfectly fine files, that are then suddenly changed with might trigger reactions in other parts of their processing chain.
Or in my case, I have a useless pencil icon because it tells nonsense I have no use for but not where I changed metadata so a writeback is needed.
(still assuming write hierarchical keywords is on by default, the question is still open, see below)

And what is the default for the write hierarchical keywords toggle in the preferences?

Do you mean the options which control if and how IMatch converts hierarchical keywords into flat IPTC/XMP keywords in Edit > Preferences > Metadata? If so, please see the corresponding help which explains all the options in detail. Just press <F1> while you have the configuration dialog open, or use th search function in the Help to find all related information.

The screenshot in the Help shows this option being on, the text implies it is off.

So what is the default state?

And if you are irritated by the Lightroom hierarchical keywords: That's the only (so-so) standard we currently have for working with hierarchical keywords in XMP. The MWG hierarchical keywords added recently is bloated and will cause great grief for typical IMatch users with dozens or hundreds of keywords and deep hierarchies. I'm still looking into it and may support it in a later edition of IMatch 5.

So why should users without any hierarchical keywords suddenly end up with flat keywords in a hierarchy?

cheers
afx

[afx]

Writing Lightroom hierarchies into files that come from Lightroom without them sounds just plain silly.

I don't understand this.  Part of my problem was that I never input metadata in a RAW converter - only IMatch or Geosetter.

For my personal case, keywords get entered in various stages of my processing queue. And iMatch is the last program that sees the files.

And then consider press professionals. 90% of them use PhotoMechanic to enter keywords as it is by far the most efficient program to do so.

So to test this I put virgin RAW and JPG into LR and assigned keywords to both.  I assigned A, B|C, and D|E|F.  I then imported into IMatch5 and what I think were default metadata settings (esp both options under keyword export should be checked).  There was no writeback because everything was already in the right place. 

Yes, as you have put hierarchical keywords in the file.
I don't. They are just not portable enough. After all, my hierarchical keywords might look completely different from the hierarchy system used by agencies A & B or whoever gets the files.

I love hierarchical categories for my internal use, but not for anything that is sent out. There those schemes are just a portability PITA.

My understanding and my experiments indicated that LR (5.3) fills in the hierarchical field when you enter either a flat or hierarchical keyword, as well as the flat fields.

As evident by my files, if I do not use hierarchies in LR, LR will not set the hierarchy field. And most of my keywords come in via AS presets still and LR just uses them but does not muck about with them.

cheers
afx

[Ferdinand]

I am sympathetic to your view that people who just want to use the flat keywords fields in IMatch 5 and not the hierarchical XMP field can't do so.  However if all this import stuff was turned off, then you don't have a way to manage your keywords in IMatch 5, since it's all done using the hierarchical field as the base, whether your keywords are hierarchy or not.  And IMatch 5 does this because this is what LR does:

My understanding and my experiments indicated that LR (5.3) fills in the hierarchical field when you enter either a flat or hierarchical keyword, as well as the flat fields.

As evident by my files, if I do not use hierarchies in LR, LR will not set the hierarchy field. And most of my keywords come in via AS presets still and LR just uses them but does not muck about with them.

This is not my experience.  Have you checked this using Exiftool?  My point was that if you enter flat keywords A, B, C, D, E in LR, then LR will fill in both the flat keywords fields plus XMP::Lightroom/HierarchicalSubject field with these flat keywords.  So my understanding is that you will be using this field whether you know it or not and whether you like it or not.  The advantage of IMatch 5, for those who use and Adobe workflow, is that it will keep the flat and hierarchical fields in sync, which is what the Adobe software will expect.

My point also was that if you had done this in LR, then you should therefore not get writebacks queued up when you ingest in IMatch 5.  So my question is, why did you?

If you really want to keep the hierarchical field empty, and just use the flat fields, then I have to say that IMatch 5 and all the power it has to manage keywords (e.g. thesaurus) may not be for you.  But you'd also need to stay away from LR/ACR.

[afx]

This is not my experience.  Have you checked this using Exiftool?  My point was that if you enter flat keywords A, B, C, D, E in LR, then LR will fill in both the flat keywords fields plus XMP::Lightroom/HierarchicalSubject field with these flat keywords.

You are correct. I just set up a completely new DB with iMatch and rechecked. Both with the "Write Hierarchical Keywords" checked and unchecked.
And I have absolutely no clue why I got the writeback pencil in the older DB. 
The second fluke effect that I run into now.  ;-(

(By the way, why check XMP files with exiftool? Editors are faster ;-)

The question whether that option is on or of by default is still open, the manual is not clear on this.

My point also was that if you had done this in LR, then you should therefore not get writebacks queued up when you ingest in IMatch 5.  So my question is, why did you?

Absolutely no clue, see above.

cheers
afx

[Ferdinand]

(By the way, why check XMP files with exiftool? Editors are faster ;-)

For RAW files, that's right, and in fact ExifTool won't tell you anything since LR won't write to the file.  However if you edit JPGs in LR I think you'll need to examine the internals of the file.  It's not something that I do - I was just trying to cover all bases.