Counting "things" in repeatable tags (people, cars, animals, etc...)

[blackhead2]

As mentioned is some threads (https://www.photools.com/community/index.php?topic=7430.0 or https://www.photools.com/community/index.php?topic=7403.0) there are solutions for counting e.g the number of people that are an on a picture based on hierarchical keywords. The existing solutions have some drawbacks (e.g. performance depends on number and kind of other categories or manually copying of hierarchical keywords is necessary after adding keywords).

I don't know if counting things that are already tagged is really such a specific use case or if it was just not used in the past as no other DAM except from Imatch has the power to do this "on the fly"

So suppose you have hierarchical keywords like:

People|Adults|Max
People|Children|Moritz
Animals|Dogs|Race
Vehicles|Cars|Brand|Model

You can show pictures that exclusively show:
- Three specific people
- One specific person/model and a specific car / animal
- Two cars of a specific brand
- ...

A way to do that with a higher performance and automatically would be to add a regex filter for repeatable tags as it is already available for the file.categories variable. The advantage of doing this with repeatable tags instead of the file.categories variable is, that no additional category evaluation is necessary and therefore should have mostly a better performance.

Thus my request is to add a regex filter to repeatable tags similar to the one in file.categories so that {File.MD.XMP::Lightroom\hierarchicalSubject\HierarchicalSubject\0|"regex"|count:true} will only count entries that match the filter.

So if this also will help you feel free to +1

[mastodon]

I like to add "counting people by face tags". I know that might be a different approache for tha same problem.

[Mario]

I like to add "counting people by face tags". I know that might be a different approache for tha same problem.

Unclear, If a user does not map face tags to keyword? If a user maps face tags to keyword it's the same as post 1 I guess.
Still, determing all categories under a certian root for all files is an expensive operation.

[axel.hennig]

+1

[Jingo]

Was curious about existing functionality in other software/algorithms and ran across this software that automates counting rebar, pipes and sheets... too bad they don't have a batch mode (nor an automated way to do groups of images) because I wonder how it might work for faces (basically a similar shape across photos - kinda like pipes!). 

https://countingthings.com

[Mario]

Was curious about existing functionality in other software/algorithms and ran across this software that automates counting rebar, pipes and sheets... too bad they don't have a batch mode (nor an automated way to do groups of images) because I wonder how it might work for faces (basically a similar shape across photos - kinda like pipes!). 

https://countingthings.com

Did you follow our discussions about including 3rd party A.I. based technologies in IMatch? This not only covers faces (and hence the number of persons per image), but also buildins, animals, landmarks, sentiments and much more. I'm already making experiments in these areas for future IMatch versions...

[Jingo]

Yes.. I've read those threads and totally welcome that technology...  The product I mentioned is just an alternative that performs an interesting job... I wasn't aware that such programs exist to perform such specific tasks... I suppose if you were a rancher, you would know... but I don't need to count cattle from a drone photo on a daily basis!!

[Mario]

I've added a filter: function for variables working with metadata tags or Attributes.
The filter function takes a non case-sensitive regular expression and can be applied to both single-value tags and multi-value tags like keywords. For multi-value variables, each value is matched individually against the regular expression. This makes this quite powerful:

{File.MD.hierarchicalkeywords|filter:^Persons}

Returns all keywords starting with Person.

{File.MD.hierarchicalkeywords|filter:^Persons;hasvalue:Has Person Keyword}

Returns the text Has Person Keyword if the file has at least one keyword starting with Person.

{File.MD.hierarchicalkeywords|filter:^Persons\|;count:true}

Returns the number of persons in the file, by way of the Person| keywords assigned to it.

{File.AT.Notes.Note|filter:payment}

Returns the values of the Note Attribute of the Notes Attribute set. Filters out all values not containing the word payment somewhere.

[blackhead2]

  Many thanks! Looking forward to the next release. With the speed you're implementing new features, customizable software becomes a completely new meaning! 

[Mario]

With the speed you're implementing new features, customizable software becomes a completely new meaning! 

Some things are relatively quick to do.
I found the right place to add the regexp support quickly (I did some work in the same area last week and hence memory was fresh). The overhead for not using filters is minimal and there were no other side effects. Just the code, some testing and updating the documentation and release notes.
As soon as I do the next release build I will test how much faster this is than using the category variable with count. It should be quicker because finding all categories a file belongs to is epxensive, especially if categories need to be re-calculated first.

I like to include some small things in each release. I work months on major features without the users knowing 

[mastodon]

Oh, that works! Thank, Mario.

[jch2103]

... This makes this quite powerful:

{File.MD.hierarchicalkeywords|filter:^Persons}

Returns all keywords starting with Person.

This is good. Small technical question, though: The example uses 'Persons' but the explanation uses 'Person'. Did I miss some regex nuance?

[Mario]

No. I just mixed up Person and Persons. Too tired.

[jch2103]

No. I just mixed up Person and Persons. Too tired.

That's what I suspected (including the overworked Mario part). Just wanted to mention in case some users got confused.


Next time you're in Colorado, let me know and we can have a beer at one of our many microbreweries (or wine if you prefer)... (Lufthansa has direct flights from Munich to Denver.)

[Mario]

Next time you're in Colorado, let me know and we can have a beer at one of our many microbreweries (or wine if you prefer)...

Will do, thanks 

Since my Interrail days, spending some time in Spain and helping with the wine harvest (Spanish girl friend at the time), I'm a red wine drinker. Yummy!
Although, what some of the local and micro beer breweries in Germany create is really good stuff.
I'm just  not really exited about the 'TV beers' (as we call them here) - The big, multi-national beer companies which spend more money on advertising than on the product. And their product taste like that...

[jch2103]

Although, what some of the local and micro beer breweries in Germany create is really good stuff.
I'm just  not really exited about the 'TV beers' (as we call them here) - The big, multi-national beer companies which spend more money on advertising than on the product. And their product taste like that...

Agreed. See this about craft breweries in Colorado: https://www.bizjournals.com/denver/news/2017/05/12/how-many-craft-breweries-colorados-near-the-top.html. Not coincidentally, I live in Boulder.
Not that I reject red wine, of course.

[Mario]

I've made a few tests with the new variable

{File.MD.hierarchicalkeywords|filter:^Person\|;count:true}

to drive a data-driven category which groups files based on the number of persons shown.

If a file has the keywords Vacation, Sun, Summer, Fly, Person|Mike, Person|Lilly  the variable returns 2, because it only counts the keywords starting with ^Person|.

With this new formula and the internal IMatch mechanisms this is now an order of magnitude faster.

As expected, loading metadata and doing a bit of string manipulation is much faster than

a) determining all categories a file belongs to
b) Filter these categories based on a regular expression
c) Count them
d) Repeat for each file in the database (Yuck).

[sinus]

Although, what some of the local and micro beer breweries in Germany create is really good stuff.
I'm just  not really exited about the 'TV beers' (as we call them here) - The big, multi-national beer companies which spend more money on advertising than on the product. And their product taste like that...

Agreed. See this about craft breweries in Colorado: https://www.bizjournals.com/denver/news/2017/05/12/how-many-craft-breweries-colorados-near-the-top.html. Not coincidentally, I live in Boulder.
Not that I reject red wine, of course.

Just to mention, if we look "only" at countries (not states like Colorado), then it seems, that we in Switzerland are number one. 
But of course, statistics are statistics, some are correct, some not. 

https://www.tagesanzeiger.ch/wirtschaft/standardschweiz-hat-die-groesste-brauereidichte-weltweit/story/28608754

At least I like beer and I like red wine. Most I like champagne (or sparkling wine).
But if you link red wine with Love, well, then it is clear, that red wine wins!