Need help with a query

[cassb]

Hi all -- I am in the process of adding geolocation data to all my older scanned photos that don't have it. So I'd like to make a query to find the photos I need to update.

I have a user-created attribute called "Location" that I created and manually populate for every photo.  It generally contains an physical address or a more general description of a physical location.

What I'd like to do is create a query to enumerate all photos by their "Location" attribute, with the name of the Location and the number of photos that have the same Location value.  I have never done a query in IMatch before, so I could use some pointers on how to get started with this.  I'd like to be able to do an exact match on the Location attribute for the photo count, OR a relative match. Something like:

Exact match groupings:
Location                                  Number of Photos
-----------------                        --------------------
Disneyland, Anaheim, CA          81
1234 Street Rd, Anywhere, MA  130
Boston, MA                              93

OR relative match groupings:
Location                                  Number of Photos
-----------------                        --------------------
Disneyland                              81
Anaheim                                 251
Anaheim, CA                           220
Boston, MA                             93
Boston                                   93
MA                                         259

Thank you!
Bryan

[Mario]

I suggest you create a data-driven category based on the Attribute variable for your custom Attribute.

Say I have an Attribute Set called "Notes" with an Attribute "People.".
To group files by this Attribute, I create a data-driven category below "My Categories" and name it "People Attribute". Use a name you like, it is not important.

Tip: I prefer to collect my "expensive" (aka computing-intensive) categories below a parent category with the "Direct assignments only" option enabled. This way, when I close the parent, the expensive child categories must not be calculated.



and the category itself is created like this (use the variable for your Location Attribute):



This creates one child category for each unique Attribute value (Location) and the category count tells me how many files are there with this Location.

[cassb]

OK Mario, thanks for the instructions - I'll try it out.   

Can I then use categories as a way to select the files so I can update their geolocation together?

[cassb]

Wow, Data-driven categories are awesome! I can create one for each manually-populated attribute and get an instant sorted list of all the values in that attribute.  It allows me to find spelling errors and duplicate but different values for that attribute.  Nice!

[cassb]

As I find and fix misspelled location attribute data, I am using Merge Mode so it shows all attribute records even if they are not identical.  Is there a way to then change all Location attributes to the same value on all records with one edit?  Or do I have to click on every Location attribute on each record and change them individually?

[Mario]

No. It would be easier if you would have used real location metadata like Location shown and/or Location created.

[cassb]

OK, thanks!  Yeah, these are mostly negatives and prints I scanned to digital files many years ago so there was no concept of real location metadata back then. That's why I'm storing this geolocation metadata into the files now. I'll just work through it and get it done.  Thanks for all your help and for your very nice software.  :-)

[Mario]

I cannot be sure, but you can import your location data from Attributes into "real" location metadata tags (See Metadata Panel with the "6. IPTC Location" layout) using a Metadata Template.
It depends on how you have structured/stored the location data in your Attributes. Basically, you use the variable representing your location Attribute and copy it into an IPTC/XMP location tags.
IPTC (XMP) uses World Region, Country Code, Country Name, State, City, Location, SubLocation etc., which gives you a lot to play with.

This would also allow IMatch to persist the location information in the metadata stored in the image or sidecar file.

[cassb]

I cannot be sure, but you can import your location data from Attributes into "real" location metadata tags (See Metadata Panel with the "6. IPTC Location" layout) using a Metadata Template.
It depends on how you have structured/stored the location data in your Attributes. Basically, you use the variable representing your location Attribute and copy it into an IPTC/XMP location tags.
IPTC (XMP) uses World Region, Country Code, Country Name, State, City, Location, SubLocation etc., which gives you a lot to play with.

This would also allow IMatch to persist the location information in the metadata stored in the image or sidecar file.

Oh, I see what you mean.  I have just been finding locations on the map and saving the GPS Lat/Long.  But there's also "Location" tags too?  I didn't know that.  Let me see if I can figure out what you said about importing attribute to metadata.  That would be helpful for sure.

My Location attribute is usually a standard address format:  123 Street Rd., Anytown, State

Or a more general location:  Town, State
Or known places like Glacier National Park, MT
Or a specific building like Kodak Park, 200 Ridge Road West, Rochester, NY

Maybe Reverse Geocoding can be used instead of importing from attributes. I'll see if that works.

https://www.photools.com/help/imatch/index.html#tools_revgeocoding.htm

(Nah, that didn't work... the GPS coordinates are too large to pinpoint individual home address on a street.  Back to attribute -> metadata.)

[Mario]

Reverse geocoding is usually the way to go. Depending on which service you use, it goes down to the exact location, house number etc. Services like HERE or Google Maps are used for navigation, after all.

The location you show is just a sample, part of the default layout.
Don't use it for working with locations. It's for viewing.

As I said, switch to the "6. IPTC Location" layout in the Metadata Panel. Location data is a structured and repeatable. The "6. IPTC Location" default layout makes working with location data very easy.

[cassb]

I'm manually updating the "6. IPTC Location" fields in the Metadata Panel by copy/paste from my Location attribute.  It's time consuming, but it works by selecting all images with the same Location attribute and copy/paste from there to metadata fields.

How about this -- I can populate the metadata location fields for all selected images. Then is it possible to transfer those metadata fields (like in format {Location, City, State}) to replace the Location attribute in all the selected images?

[Mario]

According to your description you have written location information as a regular text Attribute, using various formats. This could probably be parsed using variable functions and then stored in different IPTC location tags.
I don't know if working out the variables and formatting functions to automate this would be faster than doing it manually. I usually do such experiments only project work when there are tens or hundreds of thousands images to process and manual work is not an option.

When you have your location data in the real IPTC Location tags, you can use the Attribute importer to re-combine them using multiple variables and store the result back in your Location attribute, if you still need it.

Note that redundancy for things like metadata is never good. Redundancy in the form of storing location data for an image both the official Location metadata tags and in a Location attribute. You duplicate data and you will always have to take care to manually resolve differences when you chance one of the two location data items.

I see no real reason to keep (or again duplicate) the location data in Attributes. It's always better to use official XMP tags when available instead of custom-made Attribute records. IMatch can use the Location data to automatically organize your files in the "IMatch Standard Categories|Image|Location" category, it can search in it, sort by it, display it on the Map Panel etc.

[hluxem]

Like Mario said, it may be easier to just do that all manual, it really depends on how many different data you have to deal with. If it's a lot of data I find it easier to manipulate and clean up fields in Excel and reimport the data. 

If your data looks like this:

Disneyland, Anaheim, CA
1234 Street Rd, Anywhere, M
Boston, MA

You can export all attributes to an CSV file. Sort files for attributes, switch view to merge mode and then select all files. Select all attributes and files in merge mode, under the gear icon select export to cvs file. Open in Excel and use the formula =TEXTSPLIT(cellxy,","). That splits the single value in multiple and creates additional columns you will use for the standard location, city, state, county tags. As your tags are not consistent, in the next step you copy all cells and paste them back as values. Then you move cells as needed in the right column. Use the CSV import function and reimport the attributes. Not sure if you have to create the attributes for the location tags first or if you can create them during the import. Once you have the attributes for location, city, state and country you can use a meta data template to fill the tags.
Like fill tag {File.MD.XMP::iptcExt\LocationCreatedCity\LocationCreatedCity\0} with {File.AT.Image.City}. You can fill all tags for all files with one Metadata template.

Good luck,
Heiner

[cassb]

Thanks guys! Mario, I should be able to handle this manually -- there are only 20,743 images in my database. I hope I can work the locations to eliminate misspellings, etc., and then populate the metadata manually from my attribute as I'm doing now.  

I understand the issue with duplicated data fields. I can delete the Location attribute completely and use the metadata tags going forward. Originally (I think I bought IMatch in 1998?), my thought was to enter Location attribute on all images and then be able to export all my image attributes w/ file name to Excel so the data is available to my family when I'm gone. I doubt they would ever use or even open up IMatch on their own. So I can just use the metadata instead of the attribute for export. This is giving me something to do with my picture collection over the next year or so - no hurry. 

[Mario]

Having the location data, and things like a headline, keywords, and description stored in the image itself (in XMP metadata) makes your input and data independent from IMatch - as it should be. You can even see description, keywords and some other metadata in Windows Explorer and Mac Finder, if you upload your images to a web service or gallery it can "see" your data (more or less) and that's good.

The Attribute system in IMatch is designed to hold arbitrary and user-defined data that should not be stored in the image (or XMP sidecar file). And for interfacing with other systems and databases.

In general, if there is a place for the data you maintain in an official XMP namespace/tag and you don't consider the data to be secret, use the XMP tag. IMatch can do a lot with data in XMP, the data is persistent independently from IMatch, accessible by other applications and services.

[sinus]

...
(I think I bought IMatch in 1998?)

The answer from a KI:

Das Digital Asset Management (DAM)-Programm IMatch wurde erstmals im Jahr 1998 veröffentlicht. Es wurde von Mario M. Westphal entwickelt und ist seitdem kontinuierlich weiterentwickelt worden. IMatch richtet sich an professionelle und ambitionierte Amateurfotografen, kleine Fotoagenturen, Bibliothekare, Wissenschaftler, Architekten, Geologen sowie Unternehmen, Militärs und Regierungsanwender in über 60 Ländern .

But on the IMatch - Website I read "since 2001".

Only Mario knows it exactly

I think, I personally bought IMatch 2001 ... but to be honest, without beeing 100% sure.

[Mario]

The first baby steps of IMatch were indeed 1998. But at that time it was a tool to quickly find identical or visually similar images - based on an algorithm that works similar to the human eye - hence the name "Eye Match" => IMatch.

[Jingo]

Just an fyi: The Attribute Importer app can most likely be modified to become an Attribute Exporter quite easily... copying select attributes to select metadata.  If folks are interested, I'm happy to give it a go.!

[Mario]

Just an fyi: The Attribute Importer app can most likely be modified to become an Attribute Exporter quite easily... copying select attributes to select metadata.  If folks are interested, I'm happy to give it a go.!

You can do that with a Metadata Template already.

[cassb]

Thank you to everyone for your kind help.  Yes sinus, it may have been 2001 for me as well - I don't remember either.  I searched my old emails and found the oldest one from Mario in 2002 but I know I was using it before then. That is when I first began collecting negatives from my photo collection (I had my own b/w darkroom at home when I was 12 years old) and then getting my parents' negatives and slides, and scanning them all into the computer for the first time. Then I collected old photos from pre-1920 from my grandparents and scanned them in as well. Then I got my wife's and her parents' photos and scanned them. That's how I ended up with over 20,000 images in my personal collection.  And now of course, almost everyone has a camera (cell phone) with them at all times, so images are abundant and repetitive. ;-)

So my plan now is to work on getting location and GPS data into every image file where I know the location. Then I will be asking in the future how to copy my "Title" and "Description" and "People" and "Date" attributes into XMP tags.  Those could be copied with no editing or checking - Location is the most difficult.

I sure hope my descendants appreciate all this work.  lol


I also have digitized 8mm family movie film that I can catalogue some day. I spent over a year, after getting a frame-by-frame scan,  color-correcting every scene.

[Mario]

Then I will be asking in the future how to copy my "Title" and "Description" and "People" and "Date" attributes into XMP tags.

Very easy. You setup a Metadata Template that uses the variable representing your Attribute as the source, and the tag you want to copy the data into as the target.

Unless you want to process/format the data in your Attribute before it is copied into the XMP tag, this is really easy to do. Many examples in the IMatch help system.

[cassb]

Question:  When populating the location XMP data, I might have this kind of information:

A.E. Seaman Mineral Museum, Michigan Technological University
1400 Townsend Dr.
Houghton, MI

In the 4. IPTC Metadata fields, there is:

Location (1400 Townsend Dr.)
City (Houghton)
State (MI)
Country (USA)


Where would you put the building name "A.E. Seaman Mineral Museum, Michigan Technological University"?

[Mario]

I would consider this the location name or sub-location name.
Whatever you consider "more important" should probably go into the location tag, the other in the sub-location tag.
There are no rules or recommendations I'm aware of. Unless you work for clients or agencies, choose whatever fits best for you.

I usually use the sub-location to record more details about a location.
Location is the street name, sub-location may be "Museum of Modern Sausage Art" or something.

[cassb]

OK, I thought the same, Mario.  I'll need to add Sublocation to a panel layout.  I'll choose LocationShownSubLocation



But when I add it to 4 IPTC Metadata custom panel "Image", it shows a different name...?

[Mario]

There is location shown and location created in IPTC XMP. And there can even be multiple location records per image - to make things more complicated.

Some of the location tags are propagated to other metadata standards and tags. For example, XMP::iptcExt\LocationShownCity\LocationShownCity is mirrored into XMP::photoshop\City\City and Composite\MWG-City\City and there are many more.

When you fill out IPTC Location Shown metadata, it is mirrored into the Composite tags used by the 4. Image sample Metadata Panel layout. Just tried, works fine.

[cassb]

Right, I'm trying to add another field to the Image panel called Sublocation.  I want it to populate whatever XMP tag  holds the sub location, like maybe XMP::iptcExt\LocationShownSubLocation\LocationShownSubLocation.

But when I add that field to the Image panel under Composite\Location, it just mirrors what's in Composite\Location. How do I add and populate a unique field in the panel called SubLocation, different from the Location tag?

I need to read more on how IMatch handles XMP data and how to set up the layouts.  XMP is all new to me.

[Mario]

I don't see the location shown tag in your layout.
You need to add

XMP::iptcExt\LocationShownSublocation\LocationShownSublocation

when this is the tag you want to fill.

[cassb]

I did find it by searching all tags, but when I add it to the panel, the name shows as XMP IPTC Extension\Location (Shown) and mirrors what's already in the Composite\Location field.

Something doesn't seem to be working correctly.

[Mario]

It should mirror the composite tag. See my post: https://www.photools.com/community/index.php?msg=106470 from today which explains about mirrored tags appearing in multiple metadata formats.

Which problem are you trying to solve?

LocationShown is a structured tag, consisting of many individual tags.
When you don't search for tag keys, only the structured tag is shown, not it's individual parts.

[cassb]

OK, thank you for all your help.  I think I need to learn more about what XMP data actually is, how it's stored in an image file, and how I want to store the data I have into my images.

My brain is thinking "database fields" when we talk about XMP, so I imagine there are empty fields in every image file for "Location", "City", "State", "Sublocation", etc.  But I think I'm wrong. The fields are defined by the XMP tags which can be whatever the user wants to call them.  All I want to do is store the location address and location name/building name/attraction name/region name/etc. in two separate fields so that when I'm dead, someone else can view those values in the image file and know what the photo is showing, where it is, and what its name was at the time the picture was taken.

Then later, I will add the people names, date taken, description, etc. which you already showed me how to do.

[Mario]

I suggest you start here: Metadata for Beginners, which is a good introduction to what metadata is and how IMatch uses it.

[cassb]

GPS and Location Data

If your camera or smart phone records GPS coordinates in your files, IMatch can use this data to place the files in the Map Panel. This allows you to visualize where the images were taken and even to retrace an entire vacation or trip.

You can also use the Map panel to manually position/place images on the Map and to import track logs to automatically set the GPS coordinates of your files. Using reverse geocoding, IMatch can fill metadata like country, city and location automatically from GPS coordinates.

IMatch organizes your files automatically by their country, city and location metadata in a dedicated category hierarchy.
----------
Is there a way to insert "State" into the standard Location category hierarchy?

[Mario]

Is there a way to insert "State" into the standard Location category hierarchy?

Sure. You can modify the sample categories IMatch ships with to your liking.
Just add a State level to the Location data-driven category. Change the City level to state, then add a new City level below that.

See Data-driven Categories to learn how data-driven categories work and how you can add, remove and modify levels.

[cassb]

Mario, you are a very patient and kind person.

I read the Metadata for Beginners page and a couple associated pages.  I'm still trying to figure out how to get SubLocation as a unique field in the IPTC Metadata Image panel.

I attached a couple screen shots -- I added every "Location" field I could find in the iptcExt set.  Top one is LocationCreatedLocationName, second one is LocationCreatedSubLocation, and third one is LocationCreated.  All can hold unique values.

So do I just go with the one called "Location (Created)" and remember it's actually SubLocation?  Or possibly rename it in the panel?  The "Composite" fields I'm still not sure about either -- Composite\MWG-Location\Location is a IMatch manufactured field from Metadata Working Group fields? When it's written back, where does it actually get stored... as XMP data?

-- I apologize for all the questions and keeping you up late at night. But I need to sort all this out because once I get started storing the metadata, I'll go through all 20,000+ images and don't want to have to do it all again if I started out wrong.  ;-)

[Mario]

I'm still trying to figure out how to get SubLocation as a unique field in the IPTC Metadata Image panel.

When you manually enter location data, use the Metadata Panel with the "6. IPTC Location" layout. it manages both the Location Created and Location Shown tags and uses the structured tag editor to maintain all individual elements of these two tags.

I understand you want to show the XMP::iptcExt\LocationShownSublocation\LocationShownSublocation tag in another Metadata Panel layout?

Tip: When you hover the mouse over the caption in the Metadata Panel, the tooltip shows you to full tag key used:



To add the sublocation tag to any layout, just select it from the Tag Selector dialog:

Composite\MWG-Location\Location is a IMatch manufactured field from Metadata Working Group fields?

ExifTool creates Composite tags from a variety of sources. We try to avoid using them, because they have many side-effects and magick during write-back.

IMatch synchronizes this tag with "XMP::iptcExt\LocationShownSublocation\LocationShownSublocation" and "XMP::iptcCore\Location\Location".

[cassb]

Maybe I'm looking in the wrong place, but I don't see anything with the name "6. IPTC Location" on it.  This is what I'm looking at.

[Mario]

This looks like an old list of Metadata Panel layouts.
Did you not follow the advice in the release notes to re-import Metadata Panel layouts? Tsk, Tsk.
The "6 IPTC Location" layout is actually around since 2023.

See TODO New Default Metadata Panel Layouts in the IMatch 2023 (or now 2025) release notes.

I often rework layouts to accommodate changes in metadata standards and IMatch behavior. I then mention in the release notes that a re-import may be useful. The release notes are the only way to communicate that, I cannot reach out to every customer individually. And I just assume that users read the release notes. That's why I spend time and money to produce them.

[cassb]

OK, mea culpa.  :-)  I imported the panels and see it now, thank you.

I'll need to copy or re-enter all the locations I've manually entered into the Location Shown fields.  I think what I want is Location Created, which is where the camera physically was when the negative was exposed, correct?

Is there a way to move metadata from one field to another for a selection of images?

[Mario]

Location Created, which is where the camera physically was when the negative was exposed, correct?

Yes.
IMatch by fills both Shown and Created with the same data when you do reverse-geocoding and they are empty or have the same GPS coordinates. We've discussed the created/shown and when/what/why maybe a year or two ago (for IMatch 2023?) in a 2 mile long thread.

Is there a way to move metadata from one field to another for a selection of images?

Sure. Metadata Templates or the The Metadata Mechanic