Filtering .MOV Files that are the Video component of iOS Live Photos

[Tveloso]

When Photos from an Apple iOS Device, that have been taken using the Live Photo feature, are moved to a PC, they are "broken up" into their consitiuent parts: the Still image (in a .JPG or .HEIC File), and the Video (in a .MOV File).

I have tried several approaches in the past to distinguish between a "real" Video, and the Video component of a Live Photo (what I call a Live Photo Clip).

This describes the method I have arrived at for doing that, which hopefully can help someone.  It essentially involves configuring several Data-driven Categories, which can then be used in the Filter Panel (via a Category Filter), to filter for .MOV Files in a given Folder, that either are, or are not Live Photo Clips.

The Files of a Live Photo (the Still and the Video) have a "content identifier" UUID in common in their Metadata. The Tag where that UUID is stored varies across those files, and over time, has also varied within each File.  In particular, the tag where that UUID is stored in .MOV Files, has had eight slightly different names in my IMatch Database (presumably due to changes in ExifTool over time).  So the first step is to isolate all .MOV that have a value in one of those eight Tags.

In order to do that, I created a Category called LivePhoto Clips, and within that, separate Child Categories, each set up as a Data-driven Category based on one of the eight Tags I'm interested in:

       

For example, the QT-CID_1.01 Category is set up based on Tag QuickTime::ItemList\1.1\ContentIdentifier\0:

       

...and to not use the "Other" Category (so the child categories will not include files which do not have that Tag):

       

...and to replace all the unique UUIDs, with a single String of Text (please see the Help on Regular Expressions, and the Advanced Data-driven Category Features and Options for more details on what that Replace Mask is doing):

       

So those 8 DD Categories are based on these Tags:

        Name
        Assigned
        To Category  Based-on Tag
        QT-CID       QuickTime::ItemList\content.identifier\ContentIdentifier\0
        QT-CID_1.01  QuickTime::ItemList\1.1\ContentIdentifier\0
        QT-CID_1.02  QuickTime::ItemList\1.2\ContentIdentifier\0
        QT-CID_1.03  QuickTime::ItemList\1.3\ContentIdentifier\0
        QT-CID_1.05  QuickTime::ItemList\1.5\ContentIdentifier\0
        QT-CID_1.06  QuickTime::ItemList\1.6\ContentIdentifier\0
        QT-CID_1.07  QuickTime::ItemList\1.7\ContentIdentifier\0
        QT-CID_1.10  QuickTime::ItemList\1.10\ContentIdentifier\0

...but your Database may contain other variations of that Tag Name, so when using the Tag Selector to set up a given DD Category for one of those Tags, you should check the Show only tags with data CheckBox...this will determine how many of those ContentIdentifier Tags you actually have in your Database.

Now that we have the LivePhoto Clips gathered into a single parent Category, we can use that Category (recursively) in the Filter Panel, to filter any scope (i.e. any FileWindow) for the files that are in that Category.  We can set the Filter Panel to return Video Files:

        <Screenshot excluded due to limitation on attachment count>

...that are in one of the child categories of the LivePhoto Clips Categry:

        <Screenshot excluded due to limitation on attachment count>

That combination of Filters can then be saved under a Name, and re-used at a later time (as described in the Saving Filters section of the The Filter Panel Help Topic). 

For example, I can apply that saved Filter to check a Folder where I intend to store only "real" Videos, to ensure that it does not contain any Live Photo Clips:

        <Screenshot excluded due to limitation on attachment count>

...or with the Filter Panel's Invert option, I can use that same filter to check a Folder where I intend to store only Live Photo Clips, to ensure that it does not contain any "real" Videos:

       

[Mario]

Thanks for sharing 

Looks like Apple is making this quite difficult. They could just produce a standard XMP record + a custom Apple namespace where they could store proprietary data in clear text, like "video type": "Live Photo".
Or just use a file naming convention which makes it easy to tell what's what...

The tags you have to use have names which indicate that they depend on a specific version (firmware) of QuickTime (the numbers) and that the tags may be different for later phones or updated firmware.
As you said, a user who has to rely on these tags must keep that in mind and check new files for the existence of these tags.

I could ease the pain by adding another Metadata Tag ShortCodes which hides this problem by being based on several tags.
But I don't use Apple products and I have no deeper insight in their proprietary QuickTime tags. And if I make this a short code tag for IMatch, I would have to always keep up-to-date with whatever change Apple introduces affecting these proprietary tags. And since I'm neither an Apple user or an Apple developer, this would not work.

Phil lists 4 "live" photo tags on the QuickTime page: https://exiftool.org/TagNames/QuickTime.html
Two of them are also mentioned in the QuickTime developer documentation on the Apple web site. Maybe just checking for the existence of these tags?
I have no sample live photos in my test library (send some to @@support please) but I know that Apple offers a JavaScript tool for displaying live photos in a web browser (https://developer.apple.com/documentation/livephotoskitjs), which means IMatch could play them if there is sufficient interest

[Tveloso]

Thank you Mario.  I have sent in some sample LivePhoto Files.

Looks like Apple is making this quite difficult. They could just produce a standard XMP record + a custom Apple namespace where they could store proprietary data in clear text, like "video type": "Live Photo".
Or just use a file naming convention which makes it easy to tell what's what...

Yes indeed...I didn't see anything obvious when trying to identify this. 

Apple does usually name the component Files the same (IMG_nnnn.HEIC and IMG_nnnn.MOV), but I found that I could not really rely on that.  Sometimes "name collisions" happened, either as I was manipulating the files in windows (resulting in name changes for one of the pair, so that IMG_nnnn (1).HEIC now does not match IMG_nnnn.MOV), or Photos from others (whose phone is at a different number series) might match an existing "real" video, but they are not actually Live Photo pairs.

Phil lists 4 "live" photo tags on the QuickTime page: https://exiftool.org/TagNames/QuickTime.html
Two of them are also mentioned in the QuickTime developer documentation on the Apple web site. Maybe just checking for the existence of these tags?

When I started trying to definitively identify whether a given Video was actually a component of a Live Photo, I did try initially looking for some reference to "Live" using the ECP, but didn't find anything.  The four tags mentioned on the ExifTool QuickTime page do not appear to be present in my files.  Perhaps there's something more "under the hood" that you'll be able to see in the sample files I sent.

...but I know that Apple offers a JavaScript tool for displaying live photos in a web browser (https://developer.apple.com/documentation/livephotoskitjs), which means IMatch could play them if there is sufficient interest

That would be very cool if you could add the ability to play the Live Photos! 

I understand though, that you cannot devote time to something that not many users would need.  The professionals using IMatch probably don't care much for Mobile Phone Photos...on the other hand, maybe a sizable number of newer IMatch users might also be Apple users...

[Mario]

I have downloaded the sample images, thanks.

When you run ExifTool with the -ee option (extract embedded, search for -ee on https://exiftool.org/exiftool_pod.html) it outputs a ton of extra info for the file, tags in groups named [Doc1], [Doc2], ... [Doc59], each tag named "Sample Time", "Sample Duration", "Live Photo Info" and either a duration in seconds or a list of numbers of sorts.
You can see it in the ExifTool Command Processor using a preset like:

-G1
-all
-a
-ee
-charset
filename=UTF8
{Files}

IMatch does not use the -ee tag, because it produces dozens or even hundreds of additional tag elements, which need to be processed and then ignored. I did so far not see a sense in that, because the info is usually not useful for people. The only use I see is to let ExifTool extract them and then search the output for a tag named "Live Photo Info" to determine if the video file is in fact a live photo. I'm not sure if this is worth the effort to always pull out this data, process it and then produce synthetic photools.com tag from it.

Maybe add a feature request so we can see how many users would be interested in something like that.

[Tveloso]

Mario, thank you so much for taking the time to review the sample files, and for the instruction on obtaining the additional embedded data.

It does seem to be quite a bit of work for a feature that would only benefit IMatch users who are also Apple users...probably a small portion of the user base (but maybe a growing one?)

I'll go ahead and post the FR, just in case others might be interested.

[vsammy]

Hello Tveloso,

An interesting idea that you have implemented.

For me, iMatch is becoming more and more the only program for image management and viewing. However, it also bothers me that I always see the picture and the associated -mov file, especially with the live photos.
I also have thousands of live photos (I like to use it, especially when I take pictures of my kids, because by selecting the final picture afterwards it is very valuable this option).

The tag you specify (QuickTime::ItemList\content.identifier\ContentIdentifier\0) I also find in my associated .mov files when I look at the metadata. However, this tag is not shown to me in the DD category for selection. (When I enter it manually, the OK button does not appear at the bottom).

Do you have a tip how I could implement this?

Greetings, vsa

[Tveloso]

Hello vsammy.

Yes, IMatch is incredibly powerful, and gives us so many options to manage our files.

The tag you specify (QuickTime::ItemList\content.identifier\ContentIdentifier\0) I also find in my associated .mov files when I look at the metadata. However, this tag is not shown to me in the DD category for selection. (When I enter it manually, the OK button does not appear at the bottom).

Do you have a tip how I could implement this?

When you click the [...] button to the right of the Tag field in the Edit Data-driven Category dialog:

   

...then in the Search all Tags tab of the Tag Selector, search for "itemlist" in the Group, and "content" in the Tag, that should subset the list:

   

...to allow you to select that tag, and "bring it back" to the Tag field in the Edit DD Category dialog.

Since every MOV file that's a Live Photo will have a unique value in that tag, it's important to use the replace mask parameter as discussed in my earlier post (to prevent the creation of one child category for every MOV file in your database that's a Live Photo).

However, it also bothers me that I always see the picture and the associated -mov file, especially with the live photos.

I also like to be able to see just the still image, but sometimes want to see the still and the MOV side-by-side.  For this I use a folder structure that keeps the .MOV Files in a sub-folder of the folder of where the stills are:

    Photos
     |-Edits
     |-LivePhotoClips

...then with the help of the File Window's Show Hierarchy and Don't Group by Folder options, I can either see just the still, or both files (that's what I use the Filter for initially - to help me identify the files that must be moved into that LivePhotoClips sub-folder).

I have also started using IMatch's Versioning feature to make the MOV a version of the still (which relies on that folder structure - in the Where to search parameters - and on those Tags that "connect" the two files):

   

   

This way, in other Views, where both files appear, I can still selectively toggle the Version Stack to hide the MOV behind the still (note the rounded upper-left and lower-right corners, indicating a collapsed stack):

   

I still haven't settled on this though...since I don't actually propagate anything between the Master (still) and Version (MOV), I was thinking that maybe I should be using a "regular" Stack for this - but I'll need to read up on the IMatch Auto-Stacking feature. 

Like so many other things in IMatch, there are multiple ways to accomplish something, and I have changed the way I do things a number of times as I've learned more about IMatch, and from the suggestions I see here in this great community.

[sinus]

Tveloso,

Thanks for sharing!! 

[vsammy]

Hello Tveloso,

thank you very much for the great explanation. The pictures helped a lot. Interesting, I have just one of the tags (QuickTime::ItemList\content.identifier\ContentIdentifier\0)....good for me ;-)

vsa

[Tveloso]

Hello vsammy.

The UUID in the other variations of that tag have been "moving" into the QuickTime::ItemList\content.identifier\ContentIdentifier\0 tag here as well.  I think it may be a function of how ExifTool has delivered that tag to IMatch over time (i.e. the Tag Name ExifTool used for that same piece of data). 

I think I may eventually wind up with just that one Tag also, as I process my older Live Photos (I still have a lot to work on), and IMatch re-loads the Metadata for them, following a writeback.

Along the lines of changing how I do things as I learn more about IMatch...I actually just changed the Variable-based Data Driven Category being discussed here, into a Formula Category.

IMatch is so vast, and offers so many options, that I sometimes don't settle on the best tool for the job, and later realize (or am shown by regularly reading the posts here) that it's better and/or more appropriate to do something a different way.  That's the case here.  Since we only need to know that the files have a value in that "content identifier" tag (or tags in my case still), a Formula Category using the @MetadataTag formula is a much better way to go (and it calculates much faster).

I couldn't find a way to "convert" a Data-Driven Category into a Formula Category (there probably is one that I just couldn't figure out), so I just deleted my Categories, and re-created them, using the Formula:

   

The complete formula in that field is (including the quotes):

    "@MetadataTag[QuickTime::ItemList\content.identifier\ContentIdentifier\0,hasvalue]"

I recommend that you create your Life Photo Category that way as well...

[Mario]

IMatch imports a large list of "standard tags and groups" from ExifTool every time a new ExifTool version is shipped.
In addition to the classic IPTC, XMP, EXIF and GPS tags, there are about 10,000 or so tags for other formats, vendor-specific maker notes and stuffs. This is a bit of a moving target and changes with every ExifTool release - Phil adds new tags all the time.

In addition to this static list, ExifTool makes up tags on-the-fly when it encounters certain metadata. Often you can identify these tags because they contain numbers like 1,2,3.
Literally undocumented heaps of proprietary data like many QuickTime tags fall into this category.
IMatch adds them to the database when it encounters them for the first time, dynamically created new tag groups and tags.

Different MOV files, created with different devices, firmware versions, dates can and will have different metadata contents. QuickTime is an Apple standard, which means it is no standard.
If you rely on these proprietary tags for anything, prepare to revisit your formulas and data-driven categories frequently to see if they still do what you want. Just how it is.

There is no way to convert a data-driven category (with about 50 options and switches) to a formula-based category. Different beasts etc.

[vsammy]

Hello Tveloso,

great posting! Thanks again. I am learning and beeing fascinated of it! 

At least the View in iMatch is now "clean" of the Live-Clips. Haven't moved them to a subfolder, because in other Apps I need the Clips in the same folder.... ;-)

vsa

[vsammy]

Hello Tveloso,

I just remarked that all LivePhotos after 5/21 are not shown with the Formula-based category "@MetadataTag[QuickTime::ItemList\content.identifier\ContentIdentifier\0,hasvalue]"
Probably there is a new tag required?

vsa

[Tveloso]

Hello vsammy.

It's probably so, that some of the older files actually have that "LivePhoto UUID" stored in another variation of the ContentIdentifier TagName.

For me, the "main Name Reference" for that tag seems to have stayed consistent (for newer files):

Code Select Expand

QuickTime::ItemList\content.identifier\ContentIdentifier
...and I continue to see that for older files, where that data was stored in another variation of that tag, the data has been "migrating" into the above tag when I update them (and IMatch reads them back in following the Write-Back).

In fact, in my original post, all the LivePhoto Clips child categories I had created contained some files:

   

...but now, some of those Categories are empty:

    You cannot view this attachment.

...(so the files that used to be in Categories QT-CID_1.07 and QT-CID_1.10 have probably all migrated into Category QT-CID)

You may just need to create the appropriate other categories that you need to capture all instances of the TagName variation for that Data.  For me, 8 different Names were originally needed:

Code Select Expand

"@MetadataTag[QuickTime::ItemList\content.identifier\ContentIdentifier\0,hasvalue]"
"@MetadataTag[QuickTime::ItemList\1.1\ContentIdentifier\0,hasvalue]"
"@MetadataTag[QuickTime::ItemList\1.2\ContentIdentifier\0,hasvalue]"
"@MetadataTag[QuickTime::ItemList\1.3\ContentIdentifier\0,hasvalue]"
"@MetadataTag[QuickTime::ItemList\1.5\ContentIdentifier\0,hasvalue]"
"@MetadataTag[QuickTime::ItemList\1.6\ContentIdentifier\0,hasvalue]"
"@MetadataTag[QuickTime::ItemList\1.7\ContentIdentifier\0,hasvalue]"
"@MetadataTag[QuickTime::ItemList\1.10\ContentIdentifier\0,hasvalue]"

...but now I would only need 6 of them.

If you select a file that you know is a Live Photo Clip, that does not appear in your Category, and switch the Metadata Panel to the Browser Layout, then search for the text "cont", that should find the Tag that item is actually in in that file...and if you hover your cursor over the Tag, IMatch will give the Fully Qualified Name of that Tag in a ToolTip (and you can also right-click the Tag Name and select Copy as Variable to get that fully qualified Name into the Windows Clipboard):

    You cannot view this attachment.

For example, while the majority of my Live Photo Clips are now in the tag shown above, many are also in other variations of that tag:

   You cannot view this attachment.

But when building the additional child categories you'll need, you should probably Check che Tag Manager as discussed in one of the earlier posts in this topic, to identify all of the variations of the name that you need to create Categories for.  Just be sure to check the "Show only Tags with data" CheckBox, so you'll that see only the variations of the ContentIdentifier Tag that you actually have in your database.

And you should also ensure that your stored filter uses the option to recursively search the child categories (the two plusses):

    You cannot view this attachment.

[vsammy]

Thank you very much Tveloso,

that was exactly the solution. Now the filter works again and I can hide all .mov -Files which are Live Photos.

Right now it is the only solution for iphone-Photographers, who want to keep the Live Photos as they are (in same dictionary) and still see all the static Photos clearly in iMatch File Window.
 

Vsa

[Mario]

Apple could this make a lot easier for the benefit of their customers and other software. They won't, of course.