Adding Child Folder (Display In Order Not Automatic)

[Darius1968]

Given:
The following Windows File Hierarchy: 
A
---I
---II
---III
---VI

Now, if I take what is given, and insert a new folder with the name IV or V, for instance, it will then be placed underneath VI, despite the fact that chronologically speaking, it is not in the proper order.  The only ways to correct this idiosyncrasy is to either close out the database and reopen it, or issue a Refresh Command (F5).  The later solution has proven to require a substantial amount of time to complete.  Is this normal, or is it a bug, or is there a necessary setting in Imatch/the database that I don't have established? 

[Mario]

You insert a folder where and with which command?
<F5> just reloads the folder tree from the database. On my 50,000 files test database this takes about one second. What do you mean by "substantial amount of time"?
The only thing that can slow this down are the file system queries. <F5> also purges the file system cache in IMatch and IMatch will check each folder again to see if it exists. If you have slow network connections or Windows does not has its file system cache updated, this may take some extra time. But nothing IMatch can do about this.

Always attach a log file in debug mode when you report issues or performance problems. This will give us a lot more info to work with. See log file.

[Darius1968]

Okay, I've attached a "Debug Logging" mode log file.  It represents me operating on the following folder original structure:
Fantasy Island
---Season 1
---Season 2
---Season 4

First, I Simply added child folder, "Season 3", at which point, my folder structure was this:
Fantasy Island
---Season 1
---Season 2
---Season 4
---Season 3
This folder structure is not ordered.

To get the Folder structure to be in order, I issued <F5> command, which achieved what I wanted, this updated, ordered file structure:
Fantasy Island
---Season 1
---Season 2
---Season 3
---Season 4. 
The problem with <F5> (at least on my setup) is that the refresh takes at least 30 sec.  So how do I make the update, reflecting the added folder automatic or the issuing of the <F5> command to be faster? 

I've attached the log file for your review.  Thanks. 

[Mario]

Do you add the folder in IMatch or externally? Which command/workflow do you use?
IMatch does not necessarily resort the folder tree immediately. Not unless you press <F5>.
Did you see my note about slow file system and networked drives. Do you have indexed folders from networks?

Your database has almost 700,000 files and over 10,000 (!) folders.
Reloading that many folders will take a while, even on a fast machine. Even more so if you have spinning disks or remote folders.
IMatch is fast, but it cannot do miracles.

Please keep in mind that 700,000 files is a corporate-grade asset collection. And this usually means dedicated servers or at least a dedicated and purpose-built workstation (many CPU cores, super-fast SSD storage,...).
Looking at the log file, IMatch is performing quite well. E.g. loading your database in less than 14 seconds.

I see that many file-system operations are reported as "slow". Again, do you index files on slow media like networks or NAS?

[Darius1968]

I added the folder from within IMatch. (using the command available from the right-click menu)  Being that my log file was derived from Debug Logging Mode, did my adding the folder in IMatch register in that file?  No, I don't have anything indexed in IMatch that is network storage. 

My first 'come-to-mind' question was just why it takes 14 sec. to load the database, which that too, effectively reorders the file structure, but the <F5> command takes longer, >30 sec? 

[Mario]

<F5> does the same done during database load, but at once. 90% of the time is spend querying Windows for information about the file system objects (to learn which folders exist and are on-line, mostly).
14 seconds is loading the database. This does not include the time required to load the UI or other tasks.

[Darius1968]

Is there any way that I can have IMatch fetch from Windows 10, only the folder structure of "Fantasy Island" and its child nodes, instead of reloading/repainting the entire tree structure? 

[Mario]

IMatch fetches only what's needed ('visible' folders) and caches the result in memory.

Maybe your virus checker is interfering every file system call IMatch makes and slows this down badly?
Did you make IMatch an exception in your virus checker and also excluded the folder containing the database from on-access scans (but not normal scans)?
See IMPORTANT: Virus Checkers

[Darius1968]

Yes, I can say all such provisions have already been met!  I even disarmed my antivirus for a few minutes to see if that would change the outcome, and it didn't - <F5> still taking the same 30 sec. or so to redraw the file structure. 

[Mario]

How strange.

The first refresh of the tree takes about 1.7 seconds.
The same operation / code running later takes 35 seconds.

This is logged as an atomic operation without further details.
Because normally this is so fast (2 seconds for 10,000 folders) that it does not matter.

Do you have many folders expanded in your tree? This increases the number of file system operations IMatch has to perform. Try to collapse folders you don't need right now.

[Darius1968]

Actually, I thought of that - collapsing the tree structure (including the structures of the children, grandchildren, and so forth) - before I saw your suggesting it!  No, that has no effect on the equation, whatsoever. 

[Mario]

No further clues at this time.

What is also puzzling:

Under a fresh folder TEST I created 4 new folders: a d c b in that order. For each I selected TEST and pressed <Ins>. The folders show up sorted correctly: a b c d.
It also worked when I first created a and then used the "Create Multiple Folders" command to create d;b;c. The sequence in the tree is a b c d.

Do you use the standard sort mode (Gear menu in the M&F tree toolbar)?

[Darius1968]

I checked, and yes, I have the standard sort mode selected. 

[Mario]

Puzzling, indeed.

[Darius1968]

Could the problem here, possibly, be related to the text strings of the folder names themselves?  (ie., the length being too long, and/or certain characters that the engine - involved in calculating/drawing the folder structure - doesn't like)
Is there a way in IMatch, to export the tree-structure of my folder system, such that I could see it in tree-form (with all of the nodes), for the purpose of evaluating such things? 

[Mario]

IMatch does not care. Even folder names with over 50 characters on individual levels are no problem. Windows limits paths to 256 characters (in most versions).
There are no additional debugging aids for this, because they were never needed. This is a rather unique case that seems to happen only on your PC.
The names of all folders are included in the database diagnosis logs, if you need them.

[Darius1968]

I do recall a couple years back (probably, in the days of IMatch 5), I had a problem where IMatch was not adding folders that were over a certain # of characters.  It would just seemingly skip those folders.  Once, I simply shortened the character length of the names of those folders, IMatch magically responded to those folders, registering them, as well as indexing the contents. 

[Mario]

Windows limits paths to 256 characters by default. You can use longer paths and IMatch can handle them. But then maybe a WIC codec fails to load the file from the long path, ExifTool fails to handle the long path, FFMpeg or ID3lib fail to handle the path, or PDFLib fails to process PDF files from such long paths...

Keeping folder names and file names short and simple is the best way to guarantee optimal interoperability between platforms and applications.

[Darius1968]

And, after all, short folder/file names are meant to be because we have all the metadata to handle all the glorious details! 

[Darius1968]

I probably should have mentioned this before:  This problem is not just confined to my current system.  It started sometime, when I had my old PC.  My new PC has the media files from the old one, transferred over, but as far as Windows 10 goes, I purchased a new copy and did a fresh install.  Even with my IMatch database, even though it's the same as before (now, on an SSD), I didn't make use of any of the old settings; everything there is new.