Incorrect Count (Collections => Bookmarks)

Started by Darius1968, January 26, 2020, 07:32:20 AM

Previous topic - Next topic

Darius1968

As you can see in my screenshot, I have no global filter(s) applied, nor is there any search in effect.  If you have a look at the Bookmarks section of my Collections, you will see a count of 2, which is incorrect, because I have no files bookmarked!  The File Window does correctly say that there are no files.  So what could be causing this false reading.  I've attached the log file in Debug Mode. 

Mario

I see a number of "invalid file object" reports in the log file. These are logged by the routines which sort files.
Did you run a database diagnosis to ensure the database is OK?
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Darius1968

#2
So, I did a DB diagnosis, which found errors (Metabase, Collections, Files) that were able to be fixed, resulting in my once again, getting accurate & correct readings.  I had just deleted around 250K files over the past day or so, via a Binary Identicals check.  Could this be the reason why the test found these errors?  What is an Invalid File Object? 

Mario

Usually such errors happen when IMatch crashes or when there is a hard power failure.
But, deleting a file requires about 200 actions, from category updates to collection updates to the final deletion of the file on disk. There are literally hundreds of things that may go wrong, under some conditions. The database diagnosis is able to detect and fix such rare issues. Deleting 250,000 files is not something people do often. Many 'pro' DAM systems cannot even handle 250,000 files...
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Darius1968

I didn't delete all 250K at once.  Instead, I deleted that lump sum in smaller groups of around 20K. 

Mario

-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Darius1968

Yep...  And, sometime, it's not how much, but it's WHAT is done that can cause something to not be right. 

Mario

I have no idea what happened. As I said, such an issue is pretty rare.
You could switch IMatch to debug mode and then delete files in batches of 100 files, then run a diagnosis to check.
If the problem happens again, make a copy of the IMatch log file. Most likely IMatch has logged a warning of sorts, something that tells us why somewhere in the database a reference to the deleted file was kept. Could be a category. Collection. Cache of sorts. Sort array for a custom sort used somewhere.

Did you keep the diagnosis log? I would like to see which issues were diagnosed and fixed.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Darius1968

Does this log have any relevant information, or is it too far after the diagnoses to be of any use? 

Mario

I would need the diagnosis log file (im5diag.txt) not the IMatch log file. See The Log File
You wrote that the diagnosis solved the problem so the log should explain what exactly was the problem and what was fixed.
Maybe this tells me more.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Darius1968

#10
Okay, I'm providing you my last database diagnosis log file.  I hope it has useful data, as I did run the database diagnosis once, after the first diagnosis that repaired the errors, to ensure that indeed, the problems were fixed, leaving the database error free. 

Mario

#11
The diagnosis log file is overwritten with each diagnosis run. So this log does not report any errors or warnings.
When you run into this issue again, run a diagnosis and keep send me the log file. Do not run the diagnosis again before copying the log file to keep all the data intact.

This may not help to solve this. It will give us some hints, though.
What we would need to figure out under which conditions the problem actually happens.
File operations involve dozens of operations, with many Windows system calls to deal with the file system, category updates, collection updates, sort array updates if you use custom sorts and lots more. What we would need is something like

I select a file in the File Window
Then I do xyz
Then I delete the file with this command
Then the diagnosis reports this error.

Even then, the problem may be only reproducible with your database - because it has a specific mix of categories, collections, folders etc. which cause this behavior...
The IMatch log in debug mode and the immediate diagnosis log might help. But may not.

As I said above, issues such as this are very rare and usually not reproducible.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Darius1968


JohnZeman

Recently I have seen this same problem once in a while too and like Darius1968 to solve it each time I just did a database diagnostics.

Never kept the diagnostics log though but I will the next time it happens.  I'll also make a note of exactly what I was doing at the time.