Smarter face detection collections

[plastikman]

I would like to have additional smart collections based on the Face Detection (not recognition). This gives additional controls of quickly finding useful photos (e.g. a close-up portrait, which most often will be a face taking up a larger percentage of the image.

Some of my ideas, feel free to add:

• Has one face
• Has two faces
• Has three faces
• Group photo (e.g. more than 3 identified faces)
• Large face / close-up e.g. 1 face and taking up x% of image dimensions
• Small faces e.g. all faces under Y x Z identified area
• Faces in photo close together (x %)
• Faces in photo far apart (x%)
• Has no people[/i]
  

These would go in the collections tab from my point of view.

[mastodon]

Yes, that would be nice. I was wondering about this kind of feature.
It is very handy, if you have not made this kind of grouping/keywording after importing into the database, and now you want to do it with a lot of pictures.

[Mario]

These would go in the collections tab from my point of view.

Without judging on this request, just one or two things:

How often would a user really need this? Once? Once a week? After adding new images?

Some of your 'special collections' are immensely 'expensive' to calculate. Consider a typical database with 100,000 to 300,000 files and, say, 50,000 to 200,000 faces.
Keeping all these collections updated just for the odd chance that you really need them would be a real performance drag for the entire IMatch system.

[plastikman]

I think it would be super useful since you can easily filter out types of portraits and group photos that way, for which I now have to make another tag while I already tagged the faces so it is already known to the system. If this should be something that can be build or part of the standard collection is another story.

Apple Final Cut Pro X (Video editor) has this feature where you can scan for faces and it groups them not on name but on how many there are in the part of the video.

If you have a better idea to find close-up pictures of my daughter in my collection please let me know. And no, I don't want to start tagging all my images with so much details. I think there is so much potential for automatic metadata assignment that could make people's life so much easier.

See Apple Photos and Google Photos and all the default groupings they do based on categories, people, location, content type (e.g. selfie, panorama etc.). To me Metadata is still the elephant in the room... I wish camera makers would understand how much more useful photo cameras and applications could be with smarter metadata.

Also: automation prevents mistakes to an extent. Why? Because automation requires rules and they are applied systematically (e.g. define tag "Group portrait" for every picture with more than 3 named faces.)

[mastodon]

It should not be a data-driven category, and that wy it would not be calaculate intensive.

[Mario]

Data-driven categories can be extremely expensive, depending on how you create it, the number of levels and whether your database has just 50,000 files or 500,000...

[jch2103]

Just out of curiosity, how expensive are collections v data-driven categories? I'd guess they might be about the same.

[Mario]

Depends on the data-driven category. Number of processor cores. Database storage medium. Database size. User interactions. ...

[jch2103]

My question assumed the same criteria for a collection or a data-driven category.

[Mario]

There are no data-driven categories which analyze dimensions or position or relative size of face annotation rectangles. Totally different thing.

[plastikman]

I understand that getting the boundary box values of one or more annotations and calculating them relative to the overall image (and also potentially the user cropped value) could be (too) compute intensive.

This is what I do for most variable driven data-driven categories to begin with, running them manually. Most of them don't change that often and refreshing when you actually want to use them makes more sense to me.

And Mario: I am just sharing ideas without any insight/judgement in if/how/when it can be implemented sensibly judging the state of technology as well as the systems currently in use by customers. I understand that you (a) are a one-man team (b) are very busy (c) just shipped a new release (d) have other priorities right now that benefit more users than this one and (e) might decided you don't want this feature implemented.

I believe sharing ideas/insights is always good since it allows for creative thinking through new associations and cross-pollination knowledge and ideas.

[Mario]

I would say 60% of what was requested in the intiial FR can be achieved easily with data-driven categories based on person variables (like 1, 2, 3 persons or no people or group).
It depends on the database size, computer speed etc. if this drags your system down. Feel free to try it out.

Categories would definitely be the way to go. Or just a bunch of new People variables. Not collections.
I'm pondering whether to remove the collection concept from IMatch and replace it with something based on 'special' categories, similar to @Keywords. Like @Flags or @Bookmarks. This would simplify many things. Definitely something I will look into for IMatch 2021 
Variables would be the most versatile option of course.

Analytic clustering like "faces in photo close together" would require IMatch to determine all these metrics when face annotations are added or modified. And then use these for variables or collections or whatever. Doable in theory. Since the metrics only change when the face annotations change it can be calculated once and stored in the database.

But I wonder how many people would really need things like "Small faces e.g. all faces under Y x Z identified area" or "faces close together measures in % of the image width".
I'm sure there are some users who'd find this interesting, occasionally.

But how many users would really want something like this?
Let me know in the comments below and +1 if you want this.