OneDrive and IMatch update file issues

Started by jch2103, August 14, 2022, 11:03:02 PM

Previous topic - Next topic

jch2103

I'm not sure if this is a bug or a 'feature' of using OneDrive to store some of my (mostly phone) images.

Background
I use OneDrive to backup my cell phone photos and some other images. Most of the time, it works fine with IMatch, but I've been having problems reoccurring 'pencils' showing I need to re-write images (XMP::dc\Subject) for a small number of images. All of these have keywords, usually hierarchical, but so do most of my images that don't have this issue. I can't tell why some images are so affected and others not.

OneDrive is configured as a local folder on my desktop, synchronized with web and a couple of other local computers.

Specific Example
I took a photo with my phone from an airplane window. Subsequently I added metadata, including some hierarchical keywords. The image in IM now showed a 'write' icon, for XMP::dc\Subject. Multiple efforts to get the write to ''take' were unsuccessful (i.e., the write icon kept returning).

I then (in IM) cut the image from the OneDrive folder (Clipboard/Cut) and pasted (Clipboard/Paste) into a non-OneDrive folder. After I clicked the 'write' icon here, the pencil went away. After this, I used the same process to cut the image and paste back into the OneDrive folder. After an initial write, the icon went away and stayed away.

This suggests that OneDrive and IM have some kind of subtle timing issue that sometimes interferes with metadata updates in IM, yet other times does not. I can't tell what the specific issue, but in some cases it seems to be related to hierarchical keyword writes (which take longer??).

This isn't a severe problem, but it's annoying when it happens. I've attached a log file from the above IM session, along with a screen shot from Disc Pulse (I couldn't figure out how to write the session information). I've also attached one of the images which demonstrated this issue (after a successful write).

Any insights into how to prevent this issue would be appreciated.

John

jch2103

Info from Disk Pulse got left out in original post. Attached here.
John

Mario

Just in case, did you use the Metadata Analyst to check the data in the file.
Usually, when IMatch has to write-back multiple times, the problem are out-of-sync keywords between flat IPTC/XMP keywords and hierarchical keywords.
IMatch then either needs two write-backs to sync things or the user must manually correct the issue in tough cases.

I see nothing unusual in the log file, just a warning about ExifTool fixing the notoriously broken Microsoft proprietary XMP tags.

OneDrive works in mysterious and undocumented ways. If it decides to lock a file while IMatch/ExifTool is writing to it, there is nothing IMatch can do.
Writing metadata to a file requires multiple separate update operations, e.g. writing the XMP data, then mapping XMP data back into EXIF, GPS legacy IPTC. If OneDrive "kicks" in the middle and starts uploading the file, I guess this could cause an issue. There might be also cases where the files are actually not on your PC, but must be downloaded first by OneDrive first (depends on your settings). This is usually transparent, but who knows...

That all being said, if ExifTool fails to write to a file for some reason, this is logged in the log file. And also to the ExifTool output panel.
I don't see anything in the log file, which makes me think that this is probably not something caused by OneDrive, but by the data in the file. See my lead-in above.

jch2103

Yes, I did check the files with Metadata Analyst, but didn't save the results. Running MA after the initial write icon appears of course says pending metadata needing to be written. When I run it immediately after a write, nothing unusual appears (nothing about out-of-sync metadata), but after a period of time the write icon re-appears.

I'll update this post the next time the issue appears, with more documentation including MA data. Still, the issue seems to only occur while the files are in OneDrive folders, not when in 'regular' ones (and only for some files, not all). In the meantime, I've cleared all the pending write files by doing the 'cut from OneDrive-paste to 'regular' drive/write file on 'regular' drive/cut from 'regular' drive/paste into OneDrive' routine I mentioned in my original post. So, for the moment, all is good.
John

Mario

If there is an out-of-sync condition, the pen immediately re-appears after the write-back / re-ingest of the file.
If the pen appears after a short while, maybe the file on disk was modified / replaced on disk and IMatch has re-scanned it? Bringing in back old metadata?
This will show in the log-file, search for ET-Extract, which indicates that IMatch has reloaded metadata from a file.
Note: This will show after the write-back. If it shows again after some time, something has changed in the file system and IMatch has re-indexed the file.