The upcoming build 5.0.126 introduces an XMP-only workflow

[Mario]

The Metadata Working Group favors an XMP-only workflow for 'new' files.

For example, an application like IMatch should not create new classic IIM IPTC data unless the user explicitly demands it. Usually, unless your have to work with applications, tools or services which do not support XMP metadata, you don't need to deal with classic IPTC data anymore.

IMatch 5.0.126 uses this workflow by default, while giving you options to override this behavior on a file file extension basis if required.

For a typical RAW or JPEG file created by a camera, this means the following:

On import, IMatch produces rich XMP metadata from the EXIF (optionally also GPS) data contained in the image. IMatch by default favors XMP metadata and the default Metadata Panel layouts (and the Keyword Panel) show only XMP metadata. At write-back time, IMatch checks

1) whether XMP tags where updated which require synchronizing with IPTC/EXIF metadata
2) If the target file format supports IPTC/EXIF write-back, IMatch checks if the file already has an IPTC/EXIF record.
3) If the file has an IPTC record, it is updated from the XMP metadata
4) If the file has an EXIF/GPS record, it is updated from the XMP metadata

This means that files which have no classic IPTC record, don't get one during write-back. In these cases, IMatch only updates XMP metadata (embedded or in a sidecar file).

EXIF metadata is usually not affected by changes you make to XMP in the metadata panel - except image description maybe, and of course GPS data. Most of the files you work with will have EXIF metadata so IMatch will update it at write-back, but mostly if you make changes to GPS.

As before, you can configure if IMatch writes IPTC/EXIF/.... on a per-file format basis. You also can instruct IMatch to create classic IPTC metadata on a per file format basis, e.g. when you have to support applications or services which don't handle XMP metadata.

By default, IMatch disables IPTC creation for all file formats. So IMatch will update existing IPTC, but not create new IPTC records in your files. The general idea is to get rid of  the old IIM IPTC data and use only XMP metadata. Depending on whether you are a 'Pro' and have to fulfill certain metadata production rules or you are an 'amateur' with the freedom to do whatever you like, you have now better options to work with your files.

[cytochrome]

Within IMatch this iptc-less metadata handling works perfectly.

In fact, due to some problems in v.124 I deleted the iptc block before importing raw files into IM. And than everything was smooth (including diacritic characters!!).

The problem I see is that some external viewers like Irfanview don't display nothing apart from exif if iptc is absent. Maybe other external application which read iptc will have problems too.

If the read/write iptc and exif switch works it is a good and flexible solution.

Francis

[judophotos]

Mario,

As a contributor to Getty Images I have to submit my pictures using the Classic IPTC metadata setup. However, after submission I add more fields peculiar to my own workflow.

I can do that with Photo Mechanic after an event and I would hope that when they are catalogued all the images would be compatible in the way the metadata is handled. I do not want to duplicate the work in anyway shape or form as I have 275k images back to 1972 and growing in iMatch 3.6. For me, one large database is far better than several small ones that I have had in the past.

I am looking forward to migrating everything to v5 as soon as you are happy we can do so and as long as I can continue to have all the metadata compatible with Classic IPTC. Hopefully a tick here and there will be all that is needed to sync the IPTC and XMP.
David

[Mario]

IMatch 5 supports IPTC-based workflows as before. It just favors an XMP-only workflow. This avoids the many more or less subtle problems when converting between classic IPTC and XMP. And for users who never want to 'touch' there RAWs or who don't need to support classic IPTC, the workflow gets easier and quicker. The new behavior also implements the recommendations and requirements given in the latest MWG specification.

Of course, many stock and other photo agencies, press workflows, web sites etc. still rely on classic IPTC. This is why IMatch 5 still fully support it.

1. If your images already contain an IPTC record, IMatch will synchronize between XMP and IPTC automatically.
2. If you want IMatch 5 to create new IPTC records, you can enable this feature per file format.
3. If the 'allow create IPTC' option is off, IMatch only maintains the modern IPTC(Core) data inside the XMP record. It does not attempt to write classic IPTC data to the file.

The new option for example allows you to use classic IPTC in files you deliver, but use a cleaner XMP-only workflow with your RAW files. This also works in conjunction with versioning. Example:

You have RAW files and from these you create TIFF or JPG versions as deliverables for your clients. In IMatch you setup a .RAW->(JPEG,TIFF) version rule. You edit only XMP metadata and keywords in IMatch (the default). When you write back metadata, IMatch writes only XMP for the RAW, but additionally updates/adds classic IPTC to both the JPEG and TIFF. This way your RAW stays 'clean' and XMP-only, while you still deliver standardized IPTC metadata in the files you send to your agency. As long as they still require it.

[Ferdinand]

Can I just clarify what the default will be for the so-called standard formats, like JPG, TIFF, etc?  Are you saying that if there's no existing IPTC record in these formats then the default is that one won't be created?  If so then I am a little surprised by that.

[Mario]

You are not mistaken. The default is that no classic IPTC record is created by IMatch anymore when you update XMP metadata and IMatch writes back XMP. Already existing IPTC records are updated.

I think this is the better default than to create IPTC by default and make users disable it. Especially for new users which start with IMatch and use "current" files. If their files don't have classic IPTC records already, IMatch will not create them. This is in accordance with the MWG specification and avoids writing back to RAW files etc.

You can of course quickly re-enable this behavior on a per file format basis.

[cytochrome]

...
3. If the 'allow create IPTC' option is off, IMatch only maintains the modern IPTC(Core) data inside the XMP record. It does not attempt to write classic IPTC data to the file.
......
This way your RAW stays 'clean' and XMP-only, while you still deliver standardized IPTC metadata in the files you send to your agency. As long as they still require it.

This is fine. I am on a metadata in image file workflow, but he less I write to the Raw, the better. In file xmp with Core iptc is all that is needed.

Did you also consider the Exif problem? Especially the Exif:imageDescription is a trouble maker. I think that in a clean workflow only the Exif GPS tags should be updated in the master Raw. In JPG/TIFF there is no problem.

Francis

[Mario]

When IMatch is instructed to migrate from XMP to EXIF, it performs all the mappings required by the MWG (via ExifTool). I cannot just skip one of the EXIF tags because this could cause problems when re-mapping the data back to XMP on import.

You can disable EXIF writing on a per file format basis, so just exclude your RAW.

[judophotos]

Thanks Mario. This all sounds rather exciting.

My workflow starts with PhotoMechanic, which is a browser, but has an extraordinary amount of IPTC/XMP and EXIF ++ configurable fields available to use on the latest version 5. However, I stuck with the small IPTC classic optional setup.

Initially I was worried when on their forum there was some talk of doing away with the Classic layout. However, I moved to the larger metadata setup and easily structured it to work as an initial Classic setup that wrote to the derivative files as it would have done with just the Classic layout. Also I was able to personalize the field names but still retain their original correct identity as needed

It works fine and I am sure that is what you are working towards. I'm looking forward to using it when you are ready.
David

[Mario]

but has an extraordinary amount of IPTC/XMP and EXIF ++ configurable fields available

IMatch supports over 13,000 metadata tags from EXIF, IPTC, GPS, XMP, PDF, ID3, Office, movie and audio metadata formats....

Also I was able to personalize the field names but still retain their original correct identity as needed

In IMatch 5 you have an unmatched feature set for creating your own metadata panel layouts, including free choice of displayed 'field names', formats, sizes, colors, automatic highlighting and more. If you need more than IMatch offers with the standard Metadata Panel layout it ships with, check out the Metadata Panel Layout section in the help.

The same applies to File Windows in IMatch, which also give you a unique amount of features and option to configure your own custom layout. See File Window Layouts in the help.

I doubt that there is other software on the market which comes even close to what IMatch can do regarding customization and display of metadata.

[cnrgeorge]

Thank you Mario for this explicit explanation,

As one who was working only with iptc fields in imatch3, this helps explain some of my initial confusion in imatch5. For somebody like me (dealing with many images, but not a professional photographer well-versed in metadata), perhaps the eventual release can include an explicit help page something like "transitioning from iptc to xmp metadata: why and how"; an entry that would, for example, quickly appear if one searched the help files for "iptc" and would include much of the info. you have already given earlier in this thread.

Conrad.

[Mario]

Hi, Conrad

every page in the IMatch help has a link at the bottom where you can provide direct feedback, suggestions, ask for more details.
I suggest you use this because this will not only allow me to queue your request, but it will also tell me _where_ you looked for the information. This function just sends an email to me, which lists the specific help page for my info. You can include all kinds of info in the email, e.g. which keywords you used to search, etc.


The 'Default' metadata panel layout  shipped with IMatch 5 has been designed to show all the tags users know from their Adobe products (LR, Bridge, Photoshop) and thus the majority of users will feel at home immediately. Most other RAW software uses a very similar set of metadata fields.

Most of these fields are directly connected to a classic IPTC tag, and IMatch automatically imports IPTC data into XMP when it ingests files. This usually ensures that you can see all your classic IPTC data in the Metadata Panel without the need to change anything.

From your comment I assume that you did not see all "your" IPTC data in the Default Metadata Panel? What was missing? Did you use any non-standard or rarely used IPTC tags perhaps?

[DigPeter]

Geosetter and, at present IM5 put geographic locations in IPTC, though IM5 also shows them under Composite.  Will people who use Geosetter have to make an exception in metadata 2 prefs?

[Mario]

Composite takes are 'made up' by ExifTool.

When you reverse geo-code files in IMatch 5, IMatch 5 fills the official XMP tags with city, location etc. IMatch also works only with GPS coordinates in the XMP record. At write-back time, XMP maps geo-data back to EXIF/IPTC following the Metadata Working Group rules. This for example means that XMP City is mapped to IPTC city and so on. The same rules apply here as for all other IPTC/EXIF mapping.

[DigPeter]

When you reverse geo-code files in IMatch 5, IMatch 5 fills the official XMP tags with city, location etc. IMatch also works only with GPS coordinates in the XMP record. At write-back time, XMP maps geo-data back to EXIF/IPTC following the Metadata Working Group rules. This for example means that XMP City is mapped to IPTC city and so on. The same rules apply here as for all other IPTC/EXIF mapping.

Will people who use Geosetter to reverse geocode have to make an exception in metadata 2 prefs, to work with your XMP-only workflow?  I understand from the above that doing this through IM will work on default settings.  But at present I am applying geo-coords to images from gps tracks in Geosetter and it is convenient to reverse geocode at the same time. 

[Mario]

Does GeoSetter only update IPTC tags, but not the corresponding XMP tags?
In this case IMatch will fill XMP from IPTC on import, if you use the defaults.
If GeoSetter updates both IPTC and XMP, it will be automatic as well.

Can you give a precise sample so I understand the problem better.

[DigPeter]

Does GeoSetter only update IPTC tags, but not the corresponding XMP tags?
In this case IMatch will fill XMP from IPTC on import, if you use the defaults.
If GeoSetter updates both IPTC and XMP, it will be automatic as well.

Can you give a precise sample so I understand the problem better.

See attached jpg file.  The raw (rw2) file was geocoded by Geosetter and converted by lightroom 5.  In the jpg, Geo locations are in IPTC  and XMP.  But in the raw file, eixiftool only shows them in IPTC.  Also attached are the metadata of the raw file and the raw xmp file.  The raw file is 14MB, but I can put it Dropbox if you wish.

[attachment deleted by admin]

[Mario]

Attaching image files without zipping them strips/mods the metadata. Can you please upload the JPEG, RAW and XMP in a ZIP archive to my FTP or send it to my support email?

If the RAW has geo-data like city and location only in IPTC but not in XMP I would consider this an error. XMP and IPTC should always be synchronized. I'm not sure if IMatch can/should attempt to fix such issues.

[DigPeter]

Attaching image files without zipping them strips/mods the metadata. Can you please upload the JPEG, RAW and XMP in a ZIP archive to my FTP or send it to my support email?

If the RAW has geo-data like city and location only in IPTC but not in XMP I would consider this an error. XMP and IPTC should always be synchronized. I'm not sure if IMatch can/should attempt to fix such issues.

Sorry about that - zip folder attached.

Putting geopraphic locations in IPTC and not in XMP seems to be consistent behaviour by Geosetter.   

Under the new XMP only work flow, would setting metadata 2 prefs for my raw files to IPTC make the geographic data available? Or perhaps I should only use IM5 for reverse geocoding?  (Can I guess your answer?  )

[attachment deleted by admin]

[Ferdinand]

Putting geopraphic locations in IPTC and not in XMP seems to be consistent behaviour by Geosetter.   

IMHO this will depend on your Geosetter preference settings.  I think if you tell it not to use XMP sidecars for the file format in question and not to create internal XMP if it doesn't already exist, then you will get this effect.

[DigPeter]

Putting geopraphic locations in IPTC and not in XMP seems to be consistent behaviour by Geosetter.   

IMHO this will depend on your Geosetter preference settings.  I think if you tell it not to use XMP sidecars for the file format in question and not to create internal XMP if it doesn't already exist, then you will get this effect.

Thanks - I will look at that.

[DigPeter]

I now have the Geosetter settings as in the attached screen shot.  Don't create internal XMP data is now unchecked.  For the previous examples it was checked.  With the line unchecked, geo data are now shown in XMP as below.  It looks rather messy with the Country code and Location in one section, and City and Country in another.

XMP-iptc Core {Query will this be picked up in the new XMP workflow?} 
Country Code                    : GRC
Location                        : Kolimbetsaíika

XMP-exif
GPS Altitude                    : 473.999969 m
GPS Altitude Ref                : Above Sea Level
GPS Latitude                    : 36.906174° N
GPS Longitude                   : 22.273861° E
GPS Map Datum                   : WGS-84
GPS Date/Time                   : 2013:04:06 06:29:19
GPS Version                     : 2.2.0.0
GPS Version ID                  : 2.2.0.0

XMP-photoshop
City                            : Agios Nikolaos Messi
Country                         : Greece


[attachment deleted by admin]

[Ferdinand]

IIt looks rather messy with the Country code and Location in one section, and City and Country in another.

You can thank Adobe for this.  If you look at the file that ExifTool uses to map IPTC to XMP you will see how it's done:

C:\Program Files (x86)\photools.com\IMatch5\arg_files\iptc2xmp.args

These mappings have been determined by Phil Harvey to be the best match.  The IMatch "standard" tags hides all this messiness from you.

[DigPeter]

Thanks for the explanation, Ferdinand.

[Ferdinand]

It's a mystery to me why the IPTC<->XMP mappings had to be this messy and untidy, and I can only assume that it's Adobe's fault.

[Mario]

The Metadata Working Group specification explains all the mappings in detail, with lots of MUST, SHOULD, COULD, ...

http://www.metadataworkinggroup.com/specs/

It's all rather involved, I'm afraid. And we're (me) lucky to have such a powerful software like ExifTool to do all the migration work. It's tough enough when you work with the stream (following the MWG recommendations) but it's even harder if you want to manage your data differently. IMatch tries to support non-MWG workflows so users can do as they please, but I would not recommend it. You'll never know what Adobe may think about all this in two years time...

[Ferdinand]

I don't think that you can blame the point that Peter was making on the MWG.  They wouldn't have come up with things like XMP-photoshop on their own.  They must have been giving an ex-post imprimatur to whatever Adobe had done.  So in the absence of evidence to the contrary, I blame the complexity of the IPTC->XMP mappings on Adobe.

[DigPeter]

@Mario,
How will your XMP-only worflow impinge on the Geosetter IPTC fields described above?

[Mario]

IMatch imports existing IPTC data into XMP as before. No change. It just not longer creates classic IPTC data unless you explicitly enable it.