Author Topic: IMatch Updater issue  (Read 364 times)

benoitf

  • New Members
  • *
  • Posts: 9
IMatch Updater issue
« on: October 31, 2018, 08:51:07 PM »
Hi,

Not sure if this falls into the bug section, as it might be related to my environment.
When I use the IMatch Updater, I run into the following situation:

- Help Menu: Check for Updates
- A new version is available: I hit Download and install now
- IMatch Updater shows up and is pre-filled with my credentials
- I hit Download update...

The download goes on smoothly, and then the following dialog pops up:
Preparation error
There was a problem preparing the downloaded file or while transferring it to IMatch

I got this in the past with previous updates, and I am getting it with the latest update also.
I tried to re-run the update, restart IMatch, reboot the system: same results.

A manual installation works perfectly fine.

Logs attached.

Thanks!
Benoit

Mario

  • IMatch Developer
  • Administrator
  • *****
  • Posts: 20692
    • photools.com
Re: IMatch Updater issue
« Reply #1 on: November 01, 2018, 08:16:38 AM »
This is a known issue, for some users. See the discussion in this thread: https://www.photools.com/community/index.php?topic=8409.msg59485#msg59485 (end of the thread) and similar problem reports.

I have no solution for now.

benoitf

  • New Members
  • *
  • Posts: 9
Re: IMatch Updater issue
« Reply #2 on: November 01, 2018, 01:29:19 PM »
Thanks Mario. And BTW this represents very little impact given that the manual installation works all fine.

lbo

  • Full Member
  • **
  • Posts: 132
Re: IMatch Updater issue
« Reply #3 on: December 05, 2018, 08:34:29 AM »
This is a known issue, for some users. See the discussion in this thread: https://www.photools.com/community/index.php?topic=8409.msg59485#msg59485 (end of the thread) and similar problem reports.

I have no solution for now.

All affected users have a user name with spaces or dots.

If I would earn something for solving this problem, I would guess that IMatch fails to properly construct a path for downloading the update.

Oliver

schwarzvogel

  • New Members
  • *
  • Posts: 41
Re: IMatch Updater issue
« Reply #4 on: December 05, 2018, 09:01:50 AM »
...
All affected users have a user name with spaces or dots.
If I would earn something for solving this problem, I would guess that IMatch fails to properly construct a path for downloading the update.
Oliver

Bin von dem Update-Problem auch betroffen.
Habe keinen Benutzernamen mit Leerzeichen oder Punkten.
mfG schwarzvogel

lbo

  • Full Member
  • **
  • Posts: 132
Re: IMatch Updater issue
« Reply #5 on: December 05, 2018, 09:29:16 AM »
dann war meine Vermutung wohl ein Griff ins ...

lbo

  • Full Member
  • **
  • Posts: 132
Re: IMatch Updater issue
« Reply #6 on: December 05, 2018, 08:50:15 PM »
Bin von dem Update-Problem auch betroffen.
Habe keinen Benutzernamen mit Leerzeichen oder Punkten.
mfG schwarzvogel

you seem to have a different problem.

Since I was already testing regarding the "jumping keyboard navigation" error, I checked this, and confirm that long user names break the updater.

The updater fails to store the downloaded file if the username doesn't fit 8.3

HTH

Oliver

Mario

  • IMatch Developer
  • Administrator
  • *****
  • Posts: 20692
    • photools.com
Re: IMatch Updater issue
« Reply #7 on: December 06, 2018, 09:07:12 AM »
What us a "long" user name?

The upload/save to disk is handled by embedded web server in IMatch, which is a reliable multi-platform component.
The only link between the user name and the upload I can see is the name of the TEMP folder, which is used by the web server component to store the uploaded file temporarily, before IMWS moves it to the target folder. IMWS uses the GetTempFileName() Windows API function to get a temporary file name for the web server .

Can you post the name of your TEMP folder so I can simulate that here?
« Last Edit: December 06, 2018, 10:44:13 AM by Mario »

Mario

  • IMatch Developer
  • Administrator
  • *****
  • Posts: 20692
    • photools.com
Re: IMatch Updater issue
« Reply #8 on: December 06, 2018, 01:52:50 PM »
I have tried with very long path names (up to 254 characters). I have tried path names containing . and blanks. Or both. 20 levels deep.
The app always worked.

Mario

  • IMatch Developer
  • Administrator
  • *****
  • Posts: 20692
    • photools.com
Re: IMatch Updater issue
« Reply #9 on: December 06, 2018, 02:55:10 PM »
For the user with the problem:

1. Switch Match to Debug Logging (Help menu > Support > Debug Logging) and restart it. Keep debug logging enabled when asked.
2. Run the downloader app again.
3. Open the TEMP folder on your system by typing

%TEMP%

into the Windows Explorer address bar.

Does the TEMP folder contain a file named "IMATCH_IMWS_ERRORLOG.TXT"? If so, please attach it.

lbo

  • Full Member
  • **
  • Posts: 132
Re: IMatch Updater issue
« Reply #10 on: December 07, 2018, 02:49:15 PM »
What us a "long" user name?

just tried it with a user name "langer.name" and it failed. The 8.3 name created by Windows was LANGER~1.NAM

IMATCH_IMWS_ERRORLOG.TXT contains just the line
Code: [Select]
[1544189356] [error] [client 127.0.0.1] POST /v1/upload: mg_handle_form_request: Cannot create file C:\Users\LANGER~1.NAM\AppData\Local\Temp\IMT76DB.tmp
The file C:\Users\LANGER~1.NAM\AppData\Local\Temp\IMT76DB.tmp exists and is 0 bytes long. It is writeable (at least after closing IMatch).

Switched to a short user name and the updater succeeded. The file was downloaded to C:\Users\xxx\AppData\Local\Temp

I have tried with very long path names (up to 254 characters). I have tried path names containing . and blanks. Or both. 20 levels deep.
The app always worked.

Really strange that you can't reproduce it!

My test was in a Win 7 64 VM,I used a snapshot from briefly after the Windows installation so there is nothing to interact.

Oliver

Mario

  • IMatch Developer
  • Administrator
  • *****
  • Posts: 20692
    • photools.com
Re: IMatch Updater issue
« Reply #11 on: December 07, 2018, 02:58:36 PM »
IMatch does not use a 8.3 file name for the temporary file! IMatch uses the original output from the Windows GetTempFileName() function, converts it into UTF8 and hands that to the web server function during upload processing.

I have looked into the web server source code and they do rather, eh, uncommon things for testing various Windows file-level things.

They convert the UTF8 path name to Unicode, and back and then do some tests to see if this is an 8.3 file name. And if this is the case, they refuse to process the file. I'm not quite sure why they do that. IN some branch they exist with an error if they think the mangled file name is 8.3. Not sure if this branch is active.

Don't you get a IMATCH_IMWS_ERRORLOG.TXT?
The web server should create this file when IMatch is set to debug logging and write all internal errors to it. IMatch can only see (and log) this from the 'outside'.

lbo

  • Full Member
  • **
  • Posts: 132
Re: IMatch Updater issue
« Reply #12 on: December 07, 2018, 03:37:58 PM »
IMatch does not use a 8.3 file name for the temporary file!

but it's mentioned in IMATCH_IMWS_ERRORLOG.TXT

Don't you get a IMATCH_IMWS_ERRORLOG.TXT?

you didn't read my (whole) posting, did you?

Oliver

Mario

  • IMatch Developer
  • Administrator
  • *****
  • Posts: 20692
    • photools.com
Re: IMatch Updater issue
« Reply #13 on: December 07, 2018, 06:48:08 PM »
I see, sorry.
Too many hats.

Mario

  • IMatch Developer
  • Administrator
  • *****
  • Posts: 20692
    • photools.com
Re: IMatch Updater issue
« Reply #14 on: December 08, 2018, 09:02:46 AM »
I think I have figured out what happens. It's a Windows thing, not an IMatch thing.

IMatch uses official Windows functions to create a unique temporary file in the Windows TEMP folder. It then hands this file name UTF-8 encoded over to the embedded web server as the target for the download file operation.

The web server code converts the path from UTF-8 to UNICODE, does some checks and then checks if this path name is a shortened path name in 8.3 format. If this check returns true, the download is silently aborted, because 8.3 file names may be a security risk.

The problem is, where does the 8.3 file name come from? IMatch just requests a temporary file name from the system. And why does this fail only on some computers?

After researching this, I've found this article from guru Raymond Chan: https://blogs.msdn.microsoft.com/oldnewthing/20041224-00/?p=36893/ which explains this.

Windows silently converts the TMP and TEMP environment variables entered via the control panel (or set by the system) internally silently into their 8.3 form. For compatibility reasons.

So, if the user name or the location of the TEMP folder produce a 8.3 file name (user name longer than 8 characters or spaces or blanks) that is different than the real file name, Windows will return the 8.3 file name to IMatch, and IMatch will give that to the embedded web server. Which will then reject the download. Sigh.

I have modified IMWS to enforce a "long" temp path name by using another API function that expands 8.3 into their normal form on top. This should do the trick.

Mees Dekker

  • Translators
  • *
  • Posts: 343
  • Mees
Re: IMatch Updater issue
« Reply #15 on: December 08, 2018, 10:15:50 AM »
Fantastic that you do so much research and spend so much time and effort to resolve a very minor problem for only a couple of users.

Effectively, this was not really a problem, because installing manually did the trick too and was the normal way to update/upgrade untill a couple of versions before the last one.

Chapeau !!

Mario

  • IMatch Developer
  • Administrator
  • *****
  • Posts: 20692
    • photools.com
Re: IMatch Updater issue
« Reply #16 on: December 08, 2018, 10:23:32 AM »
Thanks, Mees!

But this bugged me for a long time. I'v created the Updater app to make it easy and comfortable for users to install updates. Since the introduction of the app, the number of users who work with the latest version of IMatch has increased considerably.

I think that users these days are lazy ;) used to built-in automatic update functions and many don't bother to manually go to the customer portal and download updates. And this means that they miss out on new features, but, more importantly, may not get bug fixes for problems which may cause issues with the database or metadata. And I care for that a lot.

So, fixing this issue was important. For users and for me. Besides, I hate bugs.

lbo

  • Full Member
  • **
  • Posts: 132
Re: IMatch Updater issue
« Reply #17 on: December 08, 2018, 02:42:41 PM »
I think I have figured out what happens. It's a Windows thing, not an IMatch thing.

IMatch uses official Windows functions to create a unique temporary file in the Windows TEMP folder. It then hands this file name UTF-8 encoded over to the embedded web server as the target for the download file operation.

The web server code converts the path from UTF-8 to UNICODE, does some checks and then checks if this path name is a shortened path name in 8.3 format. If this check returns true, the download is silently aborted, because 8.3 file names may be a security risk.


Glad I could point you to the source of the problem (although it was not my problem).

Oliver

Mario

  • IMatch Developer
  • Administrator
  • *****
  • Posts: 20692
    • photools.com
Re: IMatch Updater issue
« Reply #18 on: December 08, 2018, 03:09:49 PM »
Your help is appreciated. Alto the feedback and log files from other users.