Versioning and propagating question

[Mees Dekker]

My workflow starts with taking pictures in RAW. Then rename, rate and edit these pictures and save them as jpg in the same folder. Jpg is both buddy (for easy renaming and moving) and version (for propagating metadata). Write back RAW files. Metadata are propagated correctly. All is fine.

When I then want to make some of the jpg smaller (in order to send them via e-mail) I use the batchprocessor. Smaller jpgs are saved in the same folder, with the same filename but (25%) included in the name. Filerelations are set up in a way that this new smaller file becomes another version (but not a buddy) of the RAW master. However some metadata are propagated, some are not.

Since some of the metadata are "on write" attributes, I should write back the RAW file. But I did not touch the original RAW file (the second version is created from the already existing jpg) so there is nothing to write back for the RAW file. 

Can this be the reason for the partly incorrect propagating? If I manually propagate (F4,P) a couple of times, all metadata are propagated correctly, but this should be an automatic process (as per the help).  Do I miss something here?

[Mario]

Metadata is propagated when the master is written back. IMatch first writes back the master, then tells ExifTool to copy the data requested by the user in the version rule from the master to the version.

I understand that you have a rule that makes the 20% file a version of the master.
You create the 25% file via the Batch Processor. After the file has been created, it becomes a version, so metadata should be propagated from the master to the 25% file.

What do you mean by "partly incorrect"? Is metadata missing? Which?
Did you look at the 25% version in the ExifTool Command Processor? Does it contain the correct data?
What happens if you select the version and press Shift+Ctrl+F5 > Reload Metadata?

[Mario]

I think this is a caused by a timing issue.

The Batch Processor creates the file. It notices that the file is created in a folder already contained in the database and enqueues a request to rescan the files in the background processing queue.
IMatch scans the files and adds them to the database. The Relation Manager notices that the file is a version and creates the required links in the database. Then it checks all masters to see if there are masters that propagate metadata. If such masters are found, metadata is propagated.

In my tests I noticed that the version created by the BP shows in IMatch, and when I look at the metadata of the file with the ExifTool Command Processor, the data has been successfully propagated by the Relation Manager. But the file has not been rescanned / metadata reloaded afterwards, which makes the newly added data not visible in IMatch.

When I do a Shift+Ctrl+F5 > Reload Metadata, the propagated data shows up for the 25% version.
Can you check & verify this, please?

Somehow along these parallel executing and complex operations, IMatch "forgets" to reload the metadata after the propagation has completed. I will look into this for the next regular update.

[Mario]

I have fixed this by doing a rescan of the version.

NOTE: When you do this, and the master is pending write-back, creating a new version will not cause the master to be written back, the new version created by the BP will receive the data that is currently in the master file, not the data for the master only in the database, waiting for write-back!

A folder rescan which detects new files brings them into the database, but will never trigger other files to write back.
As soon as you write back the master, the version will be updated with the latest data. As usual.

[Mees Dekker]

Two tests:

• after editing and creating the jpg and not writing back the RAW file, I created the 25% file. Then wrote back all the files. "25%" files missed some metadata but after reloading, all the metadata were loaded.
• after editing and creating the jpg but writing back the RAW file and the jpfg file immediately (so no more pending write-back on neither the RAW nor the jpg), I created the 25% file. Even then the "25%" files missed some (not all) metadata but after reloading (with no write-back whatsoever), all the metadata were loaded and shown correctly.

I think you've nailed it. It is most likely a timing issue. 

Now that I know how to cope with this problem, I can live with it. But still I hope this can be solved in a future release.

[Mario]

This issue has been fixed for the upcoming 2025.5.2 release. See https://www.photools.com/release-notes/ for more information.

[Mees Dekker]

As always, top support that no other application I use can match.

[Mario]

As always, top support that no other application I use can match.

You're welcome. Let me know if you can still reproduce it. 

[Mees Dekker]

Alas, the "problem" seems to be still present. Only a manual full rescan after a manual write-back of the smaller (25%) file shows all the metadata.

[Mario]

Switch IMatch to debug logging, do whatever you do, then use Help > Support > Copy logfile, ZIP and attach.
I cannot reproduce this at all, and I have tried this on 3 computers.
Maybe I miss a step or do something else different. 
Does your BP preset use one of the metadata options?
Pre/Post actions and if so, which?

[Mees Dekker]

Here is my workflow, step by step
 

• Reboot computer (W11)
• Open Imatch, switch to debug mode
• Run Database diagnosis: no errors.
• Close IMatch (everything should be as pristine as possible)
• Open Imatch (in debug mode)
• Transfer 1 picture (CR3) from memorycard to local SSD drive (using Windows Explorer)
• Imatch automatically rescans folder, loads all metadata
• Add Copyright (IMatch favorite) and GPS (IMatch mappanel), rating not touched/changed.
• Write back CR3 file, IMatch performs automatic reload, all metadata are there (shown in filewindow tip)
• Open CR3 file and edit in PS
• Save edited file as jpg in same folder on local SSD
• Imatch automatically rescans folder, makes jpg file a buddy file and a version, reloads metadata
• Filewindow tip shows all required metadata
• Apply batchprocessor (for settings see screenshot) to make a jpg file smaller, add (25%) to file name.
• Imatch automatically rescans folder, makes smaller jpg file a version (but not buddy) of the original CR3, reloads metadata
• Smaller jpg file shows write-back pencil: only rating has to written back (why?) See screenshot
• Filewindow tip does not show camera make and model ({File.MD.Exif::Main\272\Model\0), nor processing software ({File.MD.Exif::Main\11\ProcessingSoftware\0})  These metadata are not present in metadata analysis
• Manually write back smaller jpgfile,  processing software is now shown in file window tip, make and model still not shown or present
• Manually propagate metadata from CR3 to versions (F4,P)
• Now make and model are present in metadata. All is fine at last.

Debug log file attached. Also settings that are used in the Batch processor. You cannot view this attachment.

Batchprocessor does not copy any metadata, but since the smaller file is a version of the original C3, these metadata should be propagated (like in the normal jpg). So my question: what could cause that some metadata are propagated to the smaller version, but some are not. You cannot view this attachment.

You cannot view this attachment.

[Mario]

I tried to reproduce this, following your exact steps.

First I made sure that background propagation etc. is enabled under Edit > Preferences > Background Processing:

You cannot view this attachment.

Fresh folder, with on CR3 files. Added title, headline, description, copyright info the the CR3.
Write back.
Open CR3 in Ps/ACR and save as JPG. Made sure that the "Copyright only" option is enabled in the ACR export dialog.
The resulting JPG has only the copyright info, no title etc.
IMatch detects the files, indexes it, makes it a version of the CR 3 and propagates metadata.
JPG now has the same title, description etc. as the CR3.

Create a BP preset that produces a 25% size copy of the source image in JPG format and stores it with the original file name + 25% appended.

In the BP Metadata tab only the default "Always copy..." option is enabled.

After running the BP, the new file is added to the database, detected as a version and metadata is propagated. The 25% copy now has the same metadata as the CR3 file. NO pending write-back. 

OK.

Deleted both the JPG and the 25 copy.
In Ps, open the file via ACR and use Export to produce a JPG file. In the export dialog, I use the "None" option for metadata.

IMatch picks up the new JPG file, makes it a version and propagates metadata.

Run the BP preset on the JPG this time.
Same result. The 25% file is written and then metadata is propagated from the CR3. No pending write-back.

Seems to work just fine. I've tried many different workflows and steps, e.g. also enabling metadata tags / groups in the BP, in case this somehow affects the propagation. But it worked just fine.

Anything else I can try?

[Mees Dekker]

Thanks for the effort you put in to find the cause of this "problem". Superior support (this cannot be stated too much).

I don't know what you could try more. What I found after several tests is that the metadata most likely are propagated to the 25% jpg, but not properly read. Because: when I skip manual propagating from the CR3 to the 25% jpg but only rescan it (manually) all seems fine.

So could this still be a timing issue? I.e. metadata of the 25% jpg are read before all metatadata were processed and/or written to the file?

Anyway, as stated above. I know to circumvent this. So no big deal.

[Mario]

I have  no idea.
When a new file is found, IMatch checks if there is a master for it (or if the new file is a potential master, if there are versions for it). And when a master is found, the metadata propagation is triggered and this then triggers the re-import of the new file to bring in the new metadata.

In your case, metadata is propagated correctly to the new file, but the re-import is not performed (?) or does not work correctly (?), which results in the metadata for the file in the database does not match the actual metadata in the file. A manual Shift+Ctrl+F5 > Reload metadata fixes that.

But I could so far not reproduce this, even after spending an hour or two on this.

[Mees Dekker]

Well, I'd say: put it to rest. More important issues deserve your time.

[Mario]

If you can find a another workflow or sequence of operation which causes this, let me know. I've tried your repro steps in several variations, but it did not fail. Maybe a change I did somewhere for the next release "fixes" this issue. On my PC's, IMatch is always in a fluid state, getting changes and additions done all the time.

I did try to repro your issue with a fresh install of the 2025.5.2 in a VM, though. Sometimes things just happen on one PC, or with one specific database.