Appreciation of solid coding

Started by AppBox2984, September 29, 2016, 06:39:37 PM

Previous topic - Next topic

AppBox2984

Was just looking for another file and found this screenshot of my database info, just thought I'd compliment Mario on how far it's possible to push the old 3.6 version.  Way WAY beyond what it was ever intended to cope with, there are a few issues with some functionality but overall it still works well even with a database of this size.

Keep up the great work  :)

BanjoTom

Wow!  Really?  5 MILLION images in an IM 3.6 database?  Impressive!   ;D
— Tom, in Lexington, Kentucky, USA

Mario

5 million files. Wow. That's about how many Getty Images has  ::) ;)

I would not attempt that in IMatch 5, though. IMatch 5 does a lot more with metadata and has many advanced features which make working with image collections so much easier, more powerful and comfortable.

The backside is that there is  price to pay in terms of CPU resources.
Data-driven categories, filters, collections etc. all need plenty of CPU and memory. No problem for normal sized or even large databases (meaning about maybe 50,000 to 500,000 files per database). Which is already a lot more than you can manage in most DAMs out there (except the big ones which demand separate server farms etc.). But 5 million images per database is not feasible - and I doubt that a) many users will ever have that many files and b) try to manage them with a 100 dollar software  :)
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

AppBox2984

Quote from: Mario on September 29, 2016, 11:47:38 PM
5 million files. Wow. That's about how many Getty Images has  ::) ;)

Yes, I admit my collection might have gotten a little out of control ;)

Quote from: Mario on September 29, 2016, 11:47:38 PM
I would not attempt that in IMatch 5, though. IMatch 5 does a lot more with metadata and has many advanced features which make working with image collections so much easier, more powerful and comfortable.

The backside is that there is  price to pay in terms of CPU resources.
Data-driven categories, filters, collections etc. all need plenty of CPU and memory. No problem for normal sized or even large databases (meaning about maybe 50,000 to 500,000 files per database). Which is already a lot more than you can manage in most DAMs out there (except the big ones which demand separate server farms etc.). But 5 million images per database is not feasible - and I doubt that a) many users will ever have that many files and b) try to manage them with a 100 dollar software  :)

Agreed.  I gave it a go with the trial on 5 but wasn't getting anywhere, and saw a post where you said pretty much the same as above and realized that yes, in this case I would need to stick with this version. 

But that's ok, it does what I need it to!  Removing files from the DB individually doesn't really work any more, but adding is fine and removing an entire folder is usually ok too.  Sometimes it gets stuck in a bit of a loop on a folder update but killing and restarting the process will usually sort it.

Anyway yeah no real point to this, just thought you might be amused to see what is possible.  Although you never planned for this, your code is solid enough that it went far beyond what was anticipated and is incredible for a $100 piece of software so thankyou for being awesome.  And yes, I doubt many or any others have pushed it this far, I never really intended to but it just turned out that way :)

Aubrey

5 million files, total size of all images indexed 1 Tb ? That is 200 bytes per file??

sinus

Quote from: Aubrey on September 30, 2016, 06:34:48 AM
5 million files, total size of all images indexed 1 Tb ? That is 200 bytes per file??

Hm, I am not very good in counting such high numbers, but does this not give roughly 200 KB (Kilobytes)?

This would mean, that an average file is 200 KB big.
Make then sense for me, if AppBox2984 has not only images, but also other files like doc, pdf, txt and so on ... btw, like me.
Because IMatch is a real DAM, what can manage all sort of files.

Or the 5 millions are simply small files, like we had some years ago.
I have also still a lot of images, what are 100 - 500 KB big, and now in 2016 an average image is maybe, hm, a raw has 16 MB and a jpg about 1-5 MB roughly.
But this can vary of course a lot.

Would be interesting, what kind of files AppBox2984 has.

But anywhy, an amount of 5 Millions files in IM3, phew, very impressive.
And thanks, AppBox2984, for sharing this.
And congratulations for you ... and for Mario.  ;D
Best wishes from Switzerland! :-)
Markus

Aubrey

Sinus,
Where has my math gone? You are right it is approx 200 kB

Do you also keep pdf's in your database? I'm currently using 2 databases. one for technical documents and one for photographs.

I'm toying with the idea of moving away from itunes to IMatch. however, this needs serious thinking!

Aubrey.

sinus

Quote from: Aubrey on September 30, 2016, 04:11:43 PM
Sinus,
Where has my math gone? You are right it is approx 200 kB

Do you also keep pdf's in your database? I'm currently using 2 databases. one for technical documents and one for photographs.

I'm toying with the idea of moving away from itunes to IMatch. however, this needs serious thinking!

Aubrey.

Hi Aubrey
Well, maybe your math is gone for a  moment, but my math is usually really not good. I had quite some time to find out, that 200 KB should be right.  ::)

Yes, I keep a lot of formats in IMatch, including pdf.
Wait, easy, I can look in the data-driven format-categories, see attachement.

I had once also 2 DBs, with for example music-files and some others.
But as long as IMatch does so good in performance, I had no reason to change back again, I have only 1 big DB.

The very good thing for this is, I can search so quickly for some files and be sure, the searched file is not in the other DB.


Best wishes from Switzerland! :-)
Markus

Mario

Users who manage MP3 files in their IMatch database will be happy to hear that IMatch WebViewerTM (part of the soon to be released IMatch AnywhereTM) has a special file window layout that allows you to play MP3 files directly, even continuously.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook