Relation Rescan Issue

Started by David_H, August 22, 2025, 05:46:55 PM

Previous topic - Next topic

David_H

Quote from: Mario on August 22, 2025, 04:34:15 PMIMatch applies your file relation rules when new images come into the database, looking for masters (or versions, when the new file is a potential master). Make sure the corresponding options are enabled in Edit > Preferences > Indexing.

If this does not work, something is most likely wrong with your file relation setup.

Except the version is detected on a relation rescan of the master folder - so the detection rules should be correct.
Preferences->Background processing is set to automatically refresh relations, automatically propagate data and automatically search for masters.

Relation configuration is here You cannot view this attachment.

And a log of a session where it didn't detect automatically is here: You cannot view this attachment. (and there's a manual rescan which does pick it up at the end).

Mario

Since this is apparently a bug report and not a feature request, I have moved it accordingly.

Please provide the missing details, e.g. the folder hierarchy used for this test, in which folder you added a file that should have become a master/version and all other details you can thing off. Log files, even if in debug mode, do not contain this kind of information and I need to setup the same environment, same file names, same folder hierarchy, same file relation rules to attempt to reproduce this. 

David_H

#2
Quote from: Mario on August 22, 2025, 05:58:24 PMSince this is apparently a bug report and not a feature request, I have moved it accordingly.

Please provide the missing details, e.g. the folder hierarchy used for this test, in which folder you added a file that should have become a master/version and all other details you can thing off. Log files, even if in debug mode, do not contain this kind of information and I need to setup the same environment, same file names, same folder hierarchy, same file relation rules to attempt to reproduce this.

Folder Hierarchy :

j:\Photography\2025\08\2025_08_13\
IMG_7053.CR3

Only subfolder present initially was Processed
j:\Photography\2025\08\2025_08_13\Processed

Which contains JPG versions of all the CR3s (and an appropriate version rule). These are all correctly detected.
(eg j:\Photography\2025\08\2025_08_13\Processed\IMG_7053.JPG)


Added (none of the folders exists, yet) :
Altered\Prints\A4\IMG_7053_ET18100_DxO.jpg

Resulting in :
j:\Photography\2025\08\2025_08_13\Altered\Prints\A4\IMG_7053_ET18100_DxO.jpg

The folders, and the image are then added to IMatch, but the version is not detected.
When DxO starts to create the output image (but probably prior to the jpg being written), there will also be a new IMG_7053.CR3.dop file in the main folder, which may or may not be relevant (dop files are configured as a buddy to cr2/cr3 files).

Advanced Filesystem monitoring is on. A metadata template to apply to newly imported files is on (Set Copyright info) - this runs fine.


Mario

#3
You added a new folder which contained files that might become versions of the .CR3, correct?

You have

j:\Photography\2025\08\2025_08_13
  |-Processed

and then the folder

j:\Photography\2025\08\2025_08_13
  |-Processed
  |- Altered\Prints\A4

is created in the file system by DxO?

And then DxO stores the file  IMG_7053_ET18100_DxO.jpg in the A4 folder?

IMatch is running while this happens?  And adds the folder, the file but always fails to detect the version in the A4 folder?

I can already see that there might be timing issues with file system events, maybe DxO has the file locked, or uses a temporary file name while writing the jpg and then renaming it to the final name when the writing is successful.
Many applications use this pattern, e.g. most things from Adobe, sometimes Serif, not sure about DxO and I don't have a copy of this software running. Workflows/application behaviors like this may mess with relation detection. Timing issues would also explain why it works usually but not always.  This may be hard/impossible to reproduce, be aware.

Since F4,P works in your case after the file system has "settled", a "work-around" is in place.
I will have a look into this for one of the next releases.

David_H

Quote from: Mario on August 22, 2025, 06:32:14 PMYou added a new folder which contained files that might become versions of the .CR3, correct?

Correct.

Quote from: Mario on August 22, 2025, 06:32:14 PMYou have

j:\Photography\2025\08\2025_08_13
  |-Processed

and then the folder

j:\Photography\2025\08\2025_08_13
  |-Processed
  |- Altered\Prints\A4

is created in the file system by DxO?

And then DxO stores the file  IMG_7053_ET18100_DxO.jpg in the A4 folder?

IMatch is running while this happens?  And adds the folder, the file but always fails to detect the version in the A4 folder?


Correct; I have also just done some additional tests - I have tried both with IMatch running and not running, and copied a file that would match the rule directly into the folder (so as to rule out locked files, etc). The folder is already indexed by IMatch for these tests.
In both cases, the file is detected and indexed by IMatch, however the relation rules are not unless its run manually.


Quote from: Mario on August 22, 2025, 06:32:14 PMSince F4,P works in your case after the file system has "settled", a "work-around" is in place.
I will have a look into this for one of the next releases.


Thank you for looking into it; my hope was that the manual workaround (great for one or two folders, somewhat tedious for lots) could be automated via an IMWS endpoint - hence the original feature request.

Mario

Maybe when detecting the new file and learning that it has to look 3 levels up  from the folder containing the new file, and from there down into all child folders and files, IMatch decides to skip it. In a folder hierarchy with tens of thousand of files this could cause a real performance issue.

I don't have how this all works in my mind anymore. To complex, to long ago that I had to do the last change or analysis. I will look into this for one of the next releases and see if this is a bug or a "safety" feature.