Repeated writebacks

[jch2103]

Recently, I updated ~ 1000 images with new keywords. Since then, IMatch has been repeatedly rescanning these files. None of them are in the pending updates category. This situation may have occurred before but if so I don't remember the cure. I've tried variations on rescanning the affected folders (force update/reload metadata), but this doesn't seem to have fixed the issue. I've run Database Diagnostics and Compact and Optimize without changing the problem. Log file attached.

Any ideas on what to try next?

[Mario]

In 99% of all cases, repeated write-backs are caused by mis-matching keywords in IPTC and XMP. Did you check for that?
We have explained and discussed this dozens of times here in the community. One of the reasons why I invented the Metadata Analyst App for IMatch 2020.

Check for legacy IPTC data in your files with the ExifTool Command Processor.
Compare the keywords in IPTC with XMP. If there is a mismatch, IMatch may need to write two times to fix it.
If there is a real mess, manually deleting the IPTC keywords (or all legacy IPTC data) in the image with the ECP and then performing a write-back in IMatch fixes the problem.

[jch2103]

In 99% of all cases, repeated write-backs are caused by mis-matching keywords in IPTC and XMP. Did you check for that?

Maybe I'm in the 1%...

No legacy IPTC data in the files being rescanned. See example output from ExifTool Command Processor. Just in case,  I also tried deleting IPTC metadata in file via ECP (-overwrite_original_in_place -IPTC:All= {Files}); it not surprisingly reported image file unchanged. I didn't think these files should have had any legacy IPTC metadata, as I've been careful to avoid having it added by any editing tools.

I've also attached another copy of the IM log.

Any other ideas to try?


Also, what is 'IMATCH6_LOG_MIGRATE.TXT'? I don't recall seeing it in %temp% before.

[Mario]

The migrate log is created when IMatch migrates a database. It's probably old? How often do you clean the temp folder 

You say IMatch is rescanning the files. I misread that as repeating write-backs somehow, sorry.
IMatch is rescanning files when the data and time on disk is different from the date and time of the file in the database (last modified on disk timestamp as reported by the operating system).

For example, in your log, IMatch is scanning the folder  D:\John\Pictures\D600\2013_1206
Several files has a newer "last modified" on disk timestamp, e.g.  DSC_0120_DxO.jpg or DSC_1033_DxO.jpg.
This means that the "last modified" timestamp reported for these files is newer than the timestamp stored in the IMatch database.  So the files were modified and IMatch is correct when it rescans them. The files seem to process OK, and there is also propagation performed. Maybe you have some issues with metadata propagation?

I see two warnings about ExifTool failing to find files in the TEMP folder (IMatch produces temporary files with ExifTool in the TEMP folder for merging and then importing in the database).
Maybe disk full? Virus checker? Some other external issue causing this?
Try to reboot and maybe close other applications while you use IMatch, for testing.

There are also some reports about failure while creating cache files for some MOV files. This may be caused by very short movies (less than 2 seconds).
Search your log file for W> to find the file names of these mov files.

[jch2103]

I believe I figured out the source of the problem. My IM database resides on a networked desktop computer. I recently upgraded my copy of IMatch Anywhere in place, created a copy of my IM database on the desktop to use with IM Anywhere and started IMA. It had been running since March 25 when I installed it. Both IM and IMA point to the same images folders on the desktop.

It appears that when I added keywords to ~1000 files, I may have created an issue between the two running products (conflict of last modified dates for some files between the two databases?). When I stopped IMA, IM soon stopped rescanning (apologies, I should have been clearer with the subject of my post...).

I looked through the IMA Help but didn't spot anything that would help prevent this in the future. Perhaps I missed something?

The migrate log is created when IMatch migrates a database. It's probably old?

The migrate log is dated today. If there's no need to keep it, I'll delete it.

I see two warnings about ExifTool failing to find files in the TEMP folder
There are also some reports about failure while creating cache files for some MOV files. This may be caused by very short movies (less than 2 seconds).

Re files in the TEMP folder, I'll watch for this.
Re cache files, but not a problem (there's one less than two second mov file, several warnings about it).

[Rene Toepfer]

If there is a mismatch, IMatch may need to write two times to fix it.

Is there a setting that IM writes the keywords in IPTC with XMP twice by default? I have this behaviour always. Or maybe something else is broken. The settings for applying the writeback are as follows:


The Analyst shows the a.m. behaviour:

[Mario]

IM writes the keywords in IPTC with XMP twice by default? I

What does this mean?

IMatch may need two write-backs to sort out out-of-sync keywords issues caused by other software, as explained and documented here: Metadata Problems and Pitfalls and especially Keyword Problems

[Rene Toepfer]

I have imported new raw files out of cam into IM, without any keywords. The keywords will be set in IM. Before write-backs there is no error in metadata analyst. Then I do the first write-back and the analyst prompts above shown error in keywords. Afterwards the second write-backs this error is solved. So far so good and it is complete in line with your first reply in this thread. I understand this issue and know how to solve it. 
But if you know that in many cases this error occurs, wouldn't make sense an option to enable to make two write-backs in a row per file by default? Or didn't I have find it yet? 

[Mario]

If there are no keywords in the file, there should not be a keyword-related issue during write-back.
Such issues usually stem from out-of-sync flat and hierarchical keywords in the file, created by other software. And, often, legacy IPTC metadata is involved.

If you see this problem in other situations, the most likely source of the problem is your thesaurus and the keyword flatten settings configured under Edit > Preferences > Metadata.

If you let IMatch write only the leaf level or each path element separately into flat keywords, but your thesaurus does not contain these keywords, IMatch will be unable to map the flat keywords to hierarchical keywords, producing "new" keywords (there is no difference whether IMatch created the flat keywords or another software created the flat keywords).
Or something similarly complex along that line.

To diagnosis this, I would need a sample image that produces the problem, your thesaurus, your Edit > Preferences > Metadata settings, the keywords you added in IMatch when this happened during write-back and a very looong time slot to dig into this.

[Rene Toepfer]

If you let IMatch write only the leaf level or each path element separately into flat keywords, but your thesaurus does not contain these keywords, IMatch will be unable to map the flat keywords to hierarchical keywords, producing "new" keywords (there is no difference whether IMatch created the flat keywords or another software created the flat keywords).
Or something similarly complex along that line.

This is the case. I have some keywords which are not part of the Thesaurus especially if I used revers Geodecoding.
But before you dig deeper into this issue wouldn't it be easier to create an option for two write-backs in a row in Edit > Preferences > Metadata?

[Mario]

Two write-backs are an exception, indicating a problem.
I don't think an option to suppress this problem would be helpful.