Author Topic: File.DateTime is changed (to depart from Date Subject Created)  (Read 464 times)

Tveloso

  • Sr. Member
  • **
  • Posts: 340
The issue described in this post:

        https://www.photools.com/community/index.php?topic=11000.msg78413#msg78413

...is still happening for me.

But there’s a slight variation to the way that the metadata has been updated, so I thought a new bug report would be needed, rather than re-opening the one above.

Given the exact situation that Mario described in that post:

The problem in this specific situation you describe was that a) the date and time tags where marked as modified (by your previous use of the TimeWiz or the MD Panel), b) multiple files in that state where selected, c) you changed other metadata values in the MD panel, d) the previously updated date tags where, although unmodified by your last edit in the MD panel, checked again for modifications, not considering their multi-selection/not modified status.

...there are two methods I usually use to make the subsequent metadata changes, one of which does not produce the problem, and one that still does.

I periodically work with my Mother-in-Law on old scans, to get an approximate Date, and a Location.  When entering the location data, I usually just fill in only either City or Sublocation, and then after our session, I'll go back to fill in the remaining Location fields.

Method-1 involves the use of a Metadata Template (to populate CountryCode, Country, and State/Province), which is effectively the same as direct maintenance of the Location tags in the Metadata Panel.  The problem does not happen via this method...(and I believe that this is what was fixed for the original bug report above).

Method-2 is the use of the Metadata Panel's "merge" functionality.  The scenario is as follows:
  • The Photos we have been working on are in a Category called "Work with Mom" (and are actually in various folders)
  • A Stored Filter called "Incomplete Location Data" returns the photos we have just finished (because it returns all files that have at least one of CountryCode, Country, State/Province, City, and Location, but that also lack at least one of the first four)
  • A Value Filter (on either City or Location) is then also used to isolate all the files from that given location

So the scope is now a subset of the files we have just worked on, all from the same location. 

I know that there are other files in this Category from that same location, that were previously "fully updated" (so the FileWindow does not currently show them, since they do not meet the requirements of the "Incomplete Location Data" Filter).  I then uncheck the Stored Filter, and the scope grows to include both the files with "good" and "bad" location data, for that same location. 

Lets say there are now 50 files in the scope, with the earliest dated 7/01/1957 and the latest dated 9/01/1973, and the Sort Profile is set to Capture Time.  With the Metadata Panel set to the Location Layout I do the following:

  • Select the first file in the FileWindow (this file has complete location data)
  • Press Ctrl-A
  • In the Metadata Panel, click the pencil next to each Location field (which fills in all the Location values from the first file)
  • Click the Green Checkmark at the top of the Metadata Panel Toolbar

The Metadata Panel Layout that was in use at the time, didn't contain any Dates (I'm pretty sure that the Location Layout I used here is one that comes with IMatch, and not one that I created), but as soon as I clicked the Green Checkmark, all files in the FileWindow were now dated 7/01/1957. 

The original Date values are sill in XMP::photoshop\DateCreated\DateCreated\0, so I'm confident that a Write-Back will fix the issue, but it's a little disconcerting to see all the Dates get "overwritten".
--Tony

Mario

  • IMatch Developer
  • Administrator
  • *****
  • Posts: 31554
Re: File.DateTime is changed (to depart from Date Subject Created)
« Reply #1 on: April 02, 2022, 04:56:39 PM »
I don't follow, sorry.
IMatch never changes or sets these timestamps by itself.
They are set by ExifTool during import, mapped from EXIF metadata.
IMatch then builds the global File.DateTime as explained here: How IMatch uses Date and Time Information

hluxem

  • Sr. Member
  • **
  • Posts: 353
Re: File.DateTime is changed (to depart from Date Subject Created)
« Reply #2 on: April 02, 2022, 06:05:57 PM »
Quote
IMatch never changes or sets these timestamps by itself.
Unless there is a bug hidden somewhere.

I can't really add much to track down the issue, but this happens to me as well. I have never been able to reproduce it or see exactly when it happens. Just at some point there are a lot of thumbnails showing the same wrong date and time. Similar workflow, assigning location mostly using copy and paste from the metadata panel. Sometimes I have scanned images and assign time stamps with the TimeWiz, but that's not when the error happens.
I know that Mario can't fix this without more information.


Quote
The original Date values are still in XMP::photoshop\DateCreated\DateCreated\0, so I'm confident that a Write-Back will fix the issue, but it's a little disconcerting to see all the Dates get "overwritten".

I be careful with this, in some rare cases the wrong time was written back. If I see this happen, I usually try to force refresh the meta data, even if I loose some location data and have to assign them again.

Heiner

Mario

  • IMatch Developer
  • Administrator
  • *****
  • Posts: 31554
Re: File.DateTime is changed (to depart from Date Subject Created)
« Reply #3 on: April 02, 2022, 06:43:26 PM »
I have never experienced this.
IMatch does not update subject created / created. It uses it as the source for the global File.DateTime.

When I understand you correctly, File.DateTime is changed randomly, and shows a totally different time than subject created?
How much do they differ and where may the date come from which IMatch may have used instead of the date subject created?

IMatch sets File.DateTime when a file comes into the database.
IMatch updates the timestamp when metadata is saved back to the database import and date subject has been modified by the user.
This is an internal timestamp and not part of any metadata. IMatch can never "write-back" File.DateTime. It exists only in the database, as part of the file record.

Do your files have valid date subject created / created when you index them?
Which options have you enabled under Edit > Preferences > Metadata 2: XMP Import?
Does the Metadata Analyst show problems with these files?

Tveloso

  • Sr. Member
  • **
  • Posts: 340
Re: File.DateTime is changed (to depart from Date Subject Created)
« Reply #4 on: April 02, 2022, 08:39:09 PM »
Quote
The original Date values are still in XMP::photoshop\DateCreated\DateCreated\0, so I'm confident that a Write-Back will fix the issue, but it's a little disconcerting to see all the Dates get "overwritten".

I be careful with this, in some rare cases the wrong time was written back. If I see this happen, I usually try to force refresh the meta data, even if I loose some location data and have to assign them again.

Heiner
Thank you Heiner.

I tested one file, and in this case, the correct date was retained (and returned to File.Datetime) following the Writeback.  But I'll definitely keep an eye out for the wrong date being also written back.

I have about 150 files currently in this state (where the value in File.Datetime differs from what the Metadata Panel shows for Date Subject Created).  I selected one such file, which shows 1 July 1957 in File.DateTime:

   

...but still has the Date value originally entered in the Metadata Panel (of 10 May 1987):

   

The Tooltip shows that the Date Values are among the Metadata that will be written back:

   

Following the Writeback, the correct 1987 date was retained, and File.Datetime returned to displaying the correct Date.

When I understand you correctly, File.DateTime is changed randomly, and shows a totally different time than subject created?
How much do they differ and where may the date come from which IMatch may have used instead of the date subject created?
Yes, that's correct.  The value in File.DateTime for all files in the selection, was changed to match the value in the Focused File, when the Location Data was updated via the "merge" operation (clicking the pencil in front of each of the location fields in the Metadata Panel).

Do your files have valid date subject created / created when you index them?
Which options have you enabled under Edit > Preferences > Metadata 2: XMP Import?
Does the Metadata Analyst show problems with these files?
These are scans that did not have valid dates when indexed, so IMatch had to fall back to the FileSystem Dates.  An approximate Date was then entered in the Metadata Panel, and File.Datetime now displayed that updated Date value (no writeback done yet).

In a later IMatch Session, Location Data was changed for the files (by selecting multiple files, with the focused file having the desired location data), and when the location fields are "merged" into the target files, the Dates in those target files (in File.Datetime only) also change to what the focused file had.

Metadata-2 XMP Import has:
  • Use Native date/time Information: Yes
  • Mark file as pending writeback: No
  • Apply time-zone: No

The Metadata Analyst shows for one of the files with the issue:

    Metadata Analyst Results. Version 2021.15.2. 4/2/2022 2:15:59 PM
    File analyzed: \\vnetnas2\Media\Photos\ScannedPhotos\Scan20002.JPG
    Errors: 0
    Warnings: 1

    Warning: [System] File has unwritten metadata (pending write-back).
    The metadata loaded from the image and the data in the database may not match.

The file tested currently has:

    1958:04:06 00:00:00

...in XMP::photoshop\DateCreated\DateCreated\0, and:

    1957:07:01 00:00:00

...in File.Datetime.
« Last Edit: April 02, 2022, 08:41:22 PM by Tveloso »
--Tony

Mario

  • IMatch Developer
  • Administrator
  • *****
  • Posts: 31554
Re: File.DateTime is changed (to depart from Date Subject Created)
« Reply #5 on: April 03, 2022, 11:04:54 AM »
Mark file as pending writeback: No

Should be Yes.

Otherwise IMatch will use the File Date Time to set the XMP timestamps if no other date is available.
But it will not mark the tags as modified and not protect them or write them back.
This means that File.DateTime and these timestamps are in a bit of limbo.

You have "Changed Location data how, precisely"?
Where the XMP timestamps set at that point? Written back already or still pending?

Tveloso

  • Sr. Member
  • **
  • Posts: 340
Re: File.DateTime is changed (to depart from Date Subject Created)
« Reply #6 on: April 03, 2022, 06:42:07 PM »
You have "Changed Location data how, precisely"?
I switch to the Location Metadata Panel Layout, and select a range of files in the FileWindow (ensuring that the focused file contains the desired location data).  The location fields now say "(Multiple Values)". 

I then click the pencil icon next to each of the location fields (which brings the focused file's data into each field), and click the green checkmark icon in the Metadata Panel Toolbar.

Where the XMP timestamps set at that point?
Yes, the values in XMP::photoshop\DateCreated\DateCreated\0 were entered in a previous IMatch session (days prior)...and those values are retained, even after File.Datetime has been set to the same value for all files in the selection (and now differs from XMP DateCreated).

Written back already or still pending?
No writebacks done for the files yet.

Writeback was still pending (and included the Dates among the values to be written back) at the time that the location data was changed.

A writeback on one of the files in this state fixed the issue, and returned the value that was in Date Subject Created to File.Datetime (so they now agreed again following the writeback).

Mark file as pending writeback: No

Should be Yes.

Otherwise IMatch will use the File Date Time to set the XMP timestamps if no other date is available.
But it will not mark the tags as modified and not protect them or write them back.
This means that File.DateTime and these timestamps are in a bit of limbo.
I actually prefer not to have the dates that IMatch has had to fall back to, written back, when a file does not have any valid dates.  I think this is a valuable piece of information - that there are no "actual Dates" in the file's Metadata.  With that option set to Yes, IMatch will commit the incorrect (FileSystem) Date into the file.

Prior to this option being introduced, that was how IMatch always behaved (the date value shown in the XMP Date Subject Created was not written to the file unless the user changed it, or marked that existing date as modified)...and Mario, you had to deal with plenty of posts from users (including this one) who did not understand that...and considered it to be a bug when they did a writeback, without setting an explicit value for XMP Date Subject Created, and that Date then changed following the writeback.  I grew to appreciate that behavior ("Oh Yeah, those files don't have dates yet").

And when you made the subsequent change to use the oldest of the two FileSystem Dates when having to fall back (instead of using only Date Modified - which would always change with a write back, and confuse many users), the "changing date" behavior is now less likely to occur.

Also, in even older versions of IMatch (maybe prior to IMatch5?), IMatch did not "present" the Date value that was fallen back to, in the Metadata Panel - it simply left the field blank.  Many of the files I'm working with here, started out that way...perhaps that plays a role in manifesting this issue?

For example, I have a file that currently looks like this in the Metadata Panel:

   

I'll try to re-produce the (changing File.Datetime) behavior (with debug logging on), and will report back if I'm able to.  Heiner has said above that it is not readily reproduceable, so perhaps it will not happen this time...but if it does, I expect to see the following:
  • Enter a date in Date Created, and that value will now also appear in File.Datetime (i.e. File.Datetime will change from 14 Feb 2008, to the newly entered value).  good
  • Close and re-open IMatch (to simulate working on the file again at a later date)
  • Select this file, and one other file (having a different date), and click the pencil icon on the location fields - their values change from "(Multiple Values)" to the values of the Focused File. good
  • Click the Green Check in the Toolbar - the date in the thumbnail of the file in question changes to match the focused file (error)...while Date Created retains the previously entered Date value good
  • A Writeback will re-synch File.Datetime with Date Created (good)...(although Heiner has said that on rare occasions this does not happen - and its the incorrect File.Datetime value that is written back).
--Tony

Tveloso

  • Sr. Member
  • **
  • Posts: 340
Re: File.DateTime is changed (to depart from Date Subject Created)
« Reply #7 on: April 03, 2022, 08:35:36 PM »
Unfortunately I was not able to reproduce this during my test.

The File in question retained the same value in both File.Datetime and XMP DateCreated, when its location data was updated from another selected file with a different date.

But this test did not really faithfully duplicate everything from the "live incident" where File.Datetime was set to the same value for every file in the selection.  And it will likely be difficult to duplicate everything in that workflow...so best to just carry on, and keep an eye out for this.

Also unfortunately (or fortunately as the case may be), I'm also nearly finished working on this project with my Mother-in-Law...but I'll be sure to report back if this should happen again.
--Tony

Mario

  • IMatch Developer
  • Administrator
  • *****
  • Posts: 31554
Re: File.DateTime is changed (to depart from Date Subject Created)
« Reply #8 on: April 04, 2022, 10:10:30 AM »
I have tried to reproduce this, with varios sets of files.
Selecting multiple, using the pen to apply selected metadata tags to all selected files.
In no case, File.DateTime was changed (or the two XMP timestamps).

Since this seems to be pretty random, does not affect many users, and most likely depends on a specific combination of other factors, as you said above, there is little I can do.
I could play with this for a week without getting any result. I need a repro case.

If you or somebody else find steps or conditions to reproduce this, and I can then too, I will be happy to look into this and fix it.