Need help converting a file-import-and-rename script

[DavidOfMA]

I've looked at using the Renamer to rename my imported files, but in the long run it seems better, to me, to recreate the automated process I have been using in IMatch 3. However, I'm a little stuck on the first step of this part of the import/rename/categorize script I want to build. My newly imported files are structured like this:

F:\Photos\Captured\New_Photos\2014-07-01
F:\Photos\Captured\New_Photos\2014-07-02
F:\Photos\Captured\New_Photos\2014-07-03
...

and so on. A single level of subfolders contains all the files taken on the date the folder name specifies.

Question:  I haven't been able to figure out how to get Rescan to run so I can test it. What do I have to add to the following simple code to test Rescan? Using only this code doesn't give me an error, but it also doesn't scan the folder or subfolders, so I must be omitting something basic that I can't figure out from the Help or the sample scripts.

Code: [Select]

Sub Main
Database.Rescan Database.Folders("F:\Photos\Captured\New_Photos").Files, frmNormal
MsgBox "Done"
End Sub

Question: Once I get the Rescan running, would it recursively scan the folder F:\Photos\Captured\New_Photos and all its subfolders, or only the top-level New_Photos? If the latter, how could I get the script to walk the folder tree?  In my IMatch 3 import script, I had to use a kludge to walk the directory tree. I'm hoping IMatch 5 automatically does a recursive scan. 

Thanks in advance for your help,
David

[Mario]

The method you are using is document as:This method updates/rescans a given set of files.

It does not add new files to a folder or rescans a folder. You can use it to forcefully rescan some files.
IMatch automatically rescans folders when changes are made so there is no script method to run that manually.

[DavidOfMA]

Okay, I misunderstood that. In the UI, rescan includes adding new files so I thought the script method worked the same.

The automatic import process I am trying to emulate, which I have been using for years in Imatch 3, uses a batch file to:

• Import files from the SD card to New_Photos
• Convert RAW files to DNG files
  
• Start IMatch

This part usually takes quite a while because of the DNG conversion.

Then it uses a script, which the batch file sets up to be automatically launched on IMatch startup, to:

• Scan the new files into IMatch
• Rename the new files
• Assign default categories to the new files based on EXIF information and file extension
• Switch to the New_Photos category (a folder category) so I can finish categorizing the files

This method has allowed me to insert the SD card, start the batch file, and go off and do something else while the computer does the work. When I come back, all that's left is the additional categorizing.

Without a manual way to launch a scan of the New_Photos folder, how can I tell when IMatch 5 has finished rescanning in the background within a script? If there is a way to detect that Background processing or Background folder scanning has completed, then I could simply wait until scanning is complete to trigger launching the file-renaming and other parts of the script.

If that can't be detected within a script, do you have any suggestions for how to achieve this level of automation in IMatch 5?

Thanks,
David
With this method, I can insert the SD card, start the batch

[Mario]

Scanning the folder in IMatch is automatic, and asynchronous. There are no scripting events fired "after" a folder scan has completed, because this may happen at any time. Or new files are added while a scan is already running, then a new scan will be scheduled. A scan may be temporarily suspended, e.g. when the user is very busy in the UI, or there are write-backs to do (which always come first). This is a parallel process and files from various folders may be processed in no specific order. In parallel, utilizing all processors. There is no definitive endpoint. I'm not sure if I could add a "scan completed" event even when I wanted to. Maybe an "bakground processing queue" empty event. But that could be only temporary, because before your script could react, new entries are added to the queue and IMatch starts over...

The usual (non script way) for doing what you achieved with scripts is to let IMatch scan the folder, and then run a Metadata Template and a Renamer for renaming and distributing the files.

Categorizing files based on Metadata is usually more convenient by using a data-driven category. Unless your requirements are so special that not even IMatch's data-driven categories can handle it.

You can react on the "File Added" event by adding an event handler to the database script.

Private Sub Database_FileAdded(ByVal File As IMatch.File)

End Sub

This will be called for each file added, so you can put in your code there.

[DavidOfMA]

Thanks. If I ran my file-renaming script inside this event handler and specified it run on the New_Photos folder, would it start as soon as IMatch detected the first file added and then run repeatedly each time it detects a file (which could result in many runnings of this script for each batch of files I add)?
 
Also, is there somewhere in the Help where all the event handlers are described? I could find only a general description of event handlers.

My categorizing system is fairly simple and there's probably a way I could emulate it using data-driven categories, but as I have 100,000 files already categorized using my previous method, I'd prefer to port it to IMatch 5 as much as possible. I don't really need anything more than what it already does.

Thanks again,
David

[DavidOfMA]

Before I give up on my automated process, I am wondering if I can use the Rescan method in IMatch 5 the way Refreshfolders worked in IMatch 3?

Here is the code I used to get around Refreshfolders scanning only the top-level folder in IMatch 3. Can I adapt this to IMatch 5? Or is the only way, now, either to manually rescan the New_Photos folder or to see if adding my renaming and categorizing code to the File Added event handler works, as you suggest above?

Code: [Select]

' ========================================================
' Workaround: Scan the folder and subfolders
Dim start As String
Dim subdir As String
Dim subdirs As Variant
Dim x As Integer
Dim updfolder As IMFolder

Set db = Application.ActiveDatabase

subdirs = Array

Application.WaitDialogOpen("Scanning New_Photos...",1,1,True)
' the folder from where to start
start = "F:\Photos\Captured\New_Photos"
Set updfolder = db.GetFolder( start)
' Check if it is in the db, if not add it
If updfolder Is Nothing Then
Set updfolder = db.AddFolder( start)
' Debug.Print updfolder.Path
End If
' Refresh the content of the folder
db.RefreshFolder( updfolder, True)

' Fetch the list of subdirectories
ChDir(start)
subdirs = Find_Dir()
' Debug.Print Join(subdirs)

If subdirs(0) > 0 Then
' Only if subdirs exist,they will be scanned
For x = 1 To subdirs(0)
' create the path for the subdir
subdir = start & "\" & subdirs(x)

' get the info about the folder from the db
Set updfolder = db.GetFolder( subdir)

' Check if it is in the db, if not add it
If updfolder Is Nothing Then
Set updfolder = db.AddFolder( subdir)
' Debug.Print updfolder.Path
End If

' Update/scan the folder
db.RefreshFolder(updfolder, True)

Next
End If
' End Workaround
' ========================================================



[Mario]

You cannot rescan a folder. Rescan works only for files already in the database. You cannot access files not yet in the database.

[DavidOfMA]

Okay. I guess I'll have to make it a manual operation, unless the File Added event will emulate what I had been doing in IMatch 3.

By the way, I tried adding a new folder outside IMatch to the New_Photos folder, to see how long background scanning would take. After half an hour IMatch had not detected it. Only when I initiated a manual rescan was it detected. Background Indexing is on, so I'm not sure why the new folder is not being detected. Does Background Indexing also only work for folders that are already in the database?

Thanks,
David

[Mario]

IMatch by default does not add new folders. This is to prevent IMatch from adding folders to a database the user does not want to include, just because the user made a change in one of the parent folders.

You can check Edit > Preferences > Indexing to enable automatic inclusion of sub-folders.

[DavidOfMA]

Hi, Mario. I checked the box to include subfolders and then copied a folder of images to one of my test folders in the database. I then let IMatch sit idle for about 15 minutes, and it has still not scanned the folder I added. There don't seem to be other background processes going on -- at least, there is no status bar indicator that there is. Is this length of delay normal?  Info & Activities shows no entries in the queue and that the database has been idle for several minutes.

[Mario]

The option to enable is the advanced system monitoring. See the tooltip for this option and read the corresponding IMatch help to get details about what it does, and whether you want to enable it or not.

[DavidOfMA]

Hi, Mario. I had (and still have) all four checkboxes in the Indexing panel checked, including Advanced file system monitoring. Imatch does not find the folder I added to an existing folder in the database. Attached is a screen shot of the Indexing and Background Processing panels and the current IMATCH5_LOG.txt file. Debugging is on for the log file.

As of this message, IMatch ran for 15 minutes without detecting the folder last night, and it has been running 12 minutes without detecting it this morning. I have gotten the message that all background processing is completed, but the folder still doesn't show up.

Is there something I'm missing, or is this feature not working on my machine? If the latter, how do I get it to work, as this is an important feature for me.

Thanks,
David

[attachment deleted by admin]

[Mario]

Just tried that here, and it works.

I have c:\images\test in the database.
In Windows Explorer, I navigate to c:\images\test and add a new sub-folder "folder".
After maybe 10 seconds, the new folder shows up in the Media & Folders tree.

Please give me more details about your folder hierarchy and what you add where.

[DavidOfMA]

I added a folder of images called "Category Test" to F:\Photos\Captured\New_Photos_Test by copying it in Explorer, then started IMatch. I also tried adding another folder there while IMatch was open, and tried renaming the Category Test folder. IMatch did not detect any of these activities. Running Windows 7 Home Premium 64-bit on a circa 2009 dual-core Intel processor computer with 6GB RAM.

[Mario]

Works here. Please add a bug report with your log file and steps to reproduce. I will look into this in sequence.

[DavidOfMA]

Okay, I'll do that tonight or tomorrow.