Author Topic: Face Detect: File, Photo, Face Orientations  (Read 129 times)

StanRohrer

  • Sr. Member
  • **
  • Posts: 263
Face Detect: File, Photo, Face Orientations
« on: May 24, 2020, 01:33:19 AM »
ADMIN: Attachments of persons removed. Please do not post images with recognizable persons in public forums. This will violate laws in many countries, including Germany, where this community is hosted.

I suspect the face detection algorithms have problems with file orientations and detection orientations – especially when the user changes the face detect to an angled box. File 20180811_203704.jpg is my example source.

File AliceRohrer01 shows a confirmed face as black for a file visible in the file viewer which appears as expected orientation.
In Face Manager, selecting the black face file, and doing View is shown in file AliceRohrer02. Note that the image is rotated 90 degrees from expected and the face detection box is not on the face.

Viewing the suspect file in the Viewer shows correct image orientation and the Face box on the face, albeit rotated. This is attached image AliceRohrer03.

I suspect I have problems with the system detecting and identifying this face due to confirmed faces which apparently are not really faces. This is the first problem.

This first problem then seems to create a bigger mess across the Face portions of the database. The second problem is that trying to fix the above lost confirmed other faces. Just prior to working on the above, I had all people with status No Unconfirmed Faces in my database. I had managed zero Unknown Persons. Then I went to work on the above face detection by deleting the face box and restoring. Somewhere after my playing, Alice had 21 Unconfirmed Faces (which were not her) and Unknown Persons category had 294 listed.

So, trying to fix one faceless entry for one person has affected the status of many people.

A Database Diagnostic at this point has 1 warning:
Checking Annotation Objects:
      Containers:  2,082
Completed.
Checking Event Graph: Events:
    0 events loaded.
Checking People Graph: Events:
PersonManager Stats:
    Files with faces: 3232
    Persons: 119
    Faces: 3232
    Confirmed: 2854
    Unconfirmed: 21
    Unassigned: 357
    Warning: Ignored face records with confirmed/train/ignored state or with an entity oid. Corrected.
    Files: 3232
    Faces: 10468
    Entities: 8748. Confirmed: 8748
    Entity keywords: 0.
    Vector data size: 7530KB
    Thumbnail data size: 122154KB
    Annotations without face data: 482KB
Completed.

After the Diagnostic repair, I again have all people with status No Unconfirmed Faces in my database. However, I now have 305 Unknown Persons (started with zero).

Attached are the screen shots referenced above.

I was afraid to downsize the actual problem image file for fear the metadata would get changed as related to any possible orientation information. Since the file is too big for the forums, the source can be found at: https://drive.google.com/open?id=1dirGnfxMbuDtaEyD55dPnZIDs7Nzb0np

IM v2020.5.4 in use.
« Last Edit: May 24, 2020, 09:25:58 AM by Mario »

Mario

  • IMatch Developer
  • Administrator
  • *****
  • Posts: 25433
Re: Face Detect: File, Photo, Face Orientations
« Reply #1 on: May 24, 2020, 09:28:11 AM »
I'm not sure that I understand the problem? you are describing.

IMatch sends the images to the AI for detection and rotates the cache images before as needed.
If you manually change the face annotation rectangles in the Viewer (?) you may prevent the AI from detecting faces. Resizing the rectangle usually solves the problem. This is not always 100% 'logical' and just how AI technology works. The AI sees images different than you and figuring out what a face is from a mass of 40 million pixels is really tough.

StanRohrer

  • Sr. Member
  • **
  • Posts: 263
Re: Face Detect: File, Photo, Face Orientations
« Reply #2 on: May 24, 2020, 01:20:53 PM »
There ar a couple of problems here that I see....
1) Pictures AliceRohrer02 and AliceRohrer03 show views of face detect boxes and don't agree on the location of the face. Perhaps there is bad source data in the image source file for height/width/rotation and image characteristics that cause bad location calculations in some face detect formula.

2) When working with issue above, many previously confirmed face names get lost (returns to "?" tag) and the diagnostics report a warning that gets fixed. The diagnostic fix does indeed fix some face references (new unconfirmed gets fixed to be confirmed), generally for the person being worked on. The diagnostic seems not to fix, perhaps does make worse, other face references (I keep getting more and more Unknown Persons). What I don't know is if the above file and face detect concern is indeed related to this confirmed/unconfirmed face issue or if this issue is just a fact of working in the database with faces. I have worked for hours to tag and confirm faces and get my Unknown Persons list to quantity zero - only to go off and work on problem above and return to hundreds of entries in my Unknown Persons list. Seeing an increase of unconfirmed faces in a person reference or an increase of Unknown Persons indicates to me that the database is damaged and Diagnostics need run again.

Mario

  • IMatch Developer
  • Administrator
  • *****
  • Posts: 25433
Re: Face Detect: File, Photo, Face Orientations
« Reply #3 on: May 24, 2020, 02:40:31 PM »
Since the images were deleted for privacy reasons, I cannot comment on this.

Where these files created by an Apply device, by chance?
Apple devices detect faces and record them in the XMP record, but they mess up the coordinates (Apple knows about this) and this causes the face rectangles to be wrong when IMatch imports the XMP metadata.

Quote
many previously confirmed face names get lost (returns to "?" tag

Under which condition would that happen. Confirmed faces are locked to the face and a no longer touched by the AI.
Do you use the latest IMatch version? Important.
Do you manually move around face annotations in the Viewer? Do you copy & paste faces?  Do you use the annotation gallery?

StanRohrer

  • Sr. Member
  • **
  • Posts: 263
Re: Face Detect: File, Photo, Face Orientations
« Reply #4 on: May 24, 2020, 04:59:15 PM »
According to the metadata the image was captured on a Samsung Galaxy Note-5 (SAMSUNG-SM-N920A). I have no idea if/what software may have changed/edited orientation information. I likely got the image sent to me via Email (not known for sure anymore). I may or may not have done orientation changes in IM or Photoshop using various tools or settings. So any photo manipulations are not well known.

The IM Viewer looks as expected with correct image orientation and face detect box on the face. The image face is angled and the detect box was likely manually rotated to the angle of the face. I'm playing with the system and only been working with the People tools for a couple days.

In the IM Face Manager the image is 90 degrees rotated to the left. It has a different shape face box and it is not on the face. The box is not angled adjusted but is square with the picture.

I have maybe three files of this photo session that I think do similar things. I don't know if the manual angle of the face detect box is significant to the problem. I don't know if maybe the saving of unique face box data is related to other files getting changed - perhaps a data container storage problem within the database.

It will take me some time to fix (again) my 303 files with Unknown Persons. Maybe I'll then play again with this file and keep track of what step sequence is in play and when I note database problems. A zip file of the sample image and my screen grabs will be too big for Email. (Thanks for your steps in identity protection of my erroneous public face posting). Shall I provide a ZIP file via my dropbox?

What are you calling the "image gallery"? The screen labeled Face Manager and called by the smile face icon? I am learning my way around the all/confirmed/unconfirmed/trained/rejected faces and tools.

I am using iMatch 2020.5.4.

StanRohrer

  • Sr. Member
  • **
  • Posts: 263
Re: Face Detect: File, Photo, Face Orientations
« Reply #5 on: May 24, 2020, 09:43:31 PM »
I now suspect the Face detection problem is not a trigger for the Rejected/Unconfirmed database problems.

Attached is a log of my wondering through the Face Manager Unconfirmed/Rejected faces operations, while checking status and running diagnostics. There are some weird results. The results may be explained away if the face AI system is in play for each Reject or Unconfirm operation. If the AI is not involved then the conversion logic and numbers of faces converted just don't make sense in some conditions within my database. I hope Mario can follow my log and see my operations and concerns. Is there anything else I can do to help troubleshoot?

Mario

  • IMatch Developer
  • Administrator
  • *****
  • Posts: 25433
Re: Face Detect: File, Photo, Face Orientations
« Reply #6 on: May 25, 2020, 11:40:20 AM »
Sorry, I don't open Office documents from the 'outside' for security reasons.
Is this an IMatch log file or something else? IMatch log files are plain text.

If you have an image and steps which produce the problem you are seeing, feel free to send the image and the repro steps to support email address.
I can then try to reproduce this behavior. I'm at this point not 100% sure if I understand the problems you are experiencing.

When you confirm or reject a face, this imports all other unconfirmed/unassigned face.
Because this may change how the AI sees the person and this means it re-checks all 'free' faces (unassigned or unconfirmed) to see if it can find matches for the person. Or if another person now matches better.

StanRohrer

  • Sr. Member
  • **
  • Posts: 263
Re: Face Detect: File, Photo, Face Orientations
« Reply #7 on: May 25, 2020, 02:16:40 PM »
Attached is a -plain text- manual record of my steps through the Face Manager Unconfirmed/Rejected faces operations, while checking status and running diagnostics. There are some weird results. The results may possibly be explained away if the face AI system is in play for each Reject or Unconfirm operation - though I'm not fully convinced. If the AI is not involved then the conversion logic and numbers of faces converted just don't make sense in some conditions within my database. I've commented on what I did as well as where the results came back unexpected. The detail in the text file emphasizes my following overall concerns.

1) IM operators can certainly be allowed to do some stupid stuff in Face Manager (i.e. Reject a face that is really that person). But an IM operator should never be able to do something that damages the database and requires running a diagnostic to fix it. I have numerous sequences in the text of doing an operation and getting unexpected results. Running the diagnostic fixes a warning and that seems to fix my previous unexpected result.

2) I have cases near the end of the text where the Face Manager shows the spinning circle (working) and never terminates or comes to a conclusion. The processing is hung.