An Introduction to NAS for Photo & Video Production

[jch2103]

The LensRentals blog has a new post written by Backblaze, An Introduction to NAS for Photo & Video Production https://www.lensrentals.com/blog/2018/10/an-introduction-to-nas-for-photo-video-production/. They not surprisingly promote their own product, but it's an interesting read nevertheless. This is the third of a three-part series.

[Mario]

Thanks for sharing.

Interestingly, with the new 16 TB disk drives, 100 TB (!) SSDs on the horizon and, at least if you live in a country with fast Internet, interesting cloud storage offerings, there are many ways to deal with long-time storage and archival of mass data these days. On the other hand, files become bigger. Many photographers now also do videos, and that means Gigabytes or even Terabytes of storage requirement for a shooting or project. Plus backup.

As a side research project I've just made some experiments with database technology that not only can handle the IMatch / IMatch Anywhere WebServices™ requirements, but also easily manages Terabytes or even Exabytes of storage - locally installed, server- or cloud-based. I'm a bit excited about my initial results, I have to admit...  I think I've made the right move with creating the IMWS technology for IMatch and IMatch Anywhere. Great path into the future.

[greg]

NAS storage is a good topic to bring up on this forum! Those taking the care to organize their photo images with IMatch are likely to be concerned with storage and backup of their images. Implementing a solid backup solution can easily slip to the back burner. For me it's multiple approaches to address hardware failure, theft and disasters. Gambling on the chance of hardware/software failure and losing irreplaceable photographs is not a good bet. I've used SSD drives and NAS in various configurations over the years. However, I recently have relegated SSD to fast editing and organizing tasks with Synology NAS as the primary local storage. While much more reliable than a single drive a NAS still needs to be backed up. I now use a second NAS to backup the primary NAS storage.  I use USB drives for offsite backups and online uploads are used for display and storage of the most valued images. This may seem like overkill, but this kind of redundancy is my approach to address hardware failure, theft and disaster threats. My multi-level backup with the help IMatch was just enough to save my collection of images from a recent SSD failure. IMatch thumbnails with offline indications and filtering were crucial in pulling everything back together. Luckily I had just enough backup to save the day, but  this incident caused me to reevaluate and improve my storage/backup processes.

Some recent NAS products offer BTRFS file systems with checksums and autocorrection that further increases the reliability of a NAS. Drive manufacturers offer drive reliability information that can work in conjunction with the NAS to determine drive health. For example Seagate's Ironwolf drives are designed for the NAS environment and offer this kind of capability. Today the performance and cost of NAS systems make them much more attractive than cloud for large amounts of storage. For those that dislike software subscription services, NAS is a very attractive alternative.

[sinus]

I do know almost nothing about NAS.
Is this a way to use IMatch Anywhere safely?

As far as I understand, using IMatch Anywhere over the internet is not safe (for the normal user).
One must understand, what to do.
And I do not understand this enough, hence I do not (more) use IMatch Anywhere.

Means, if my brother sits in Cologne (Köln) and my Sister in Paris and I want show them pictures with IMatch, I cannot do this, because it is not safe.

Could NAS be a possibility to solve this "security-problem"?

[Mario]

NAS just means "Network Attached Storage". Basically, in the simplest form, a more or less powerful server with a bunch of hard disks. 
This has nothing to do with IMatch / Anywhere, it's just a common way to store masses of data with protection against disk failure. See the article linked to by jch2103 for a good intro.

[sinus]

NAS just means "Network Attached Storage". Basically, in the simplest form, a more or less powerful server with a bunch of hard disks. 
This has nothing to do with IMatch / Anywhere, it's just a common way to store masses of data with protection against disk failure. See the article linked to by jch2103 for a good intro.

Thanks, Mario, for the clarification.
I TRIED to read the article, but I misunderstood there some phrases with the word "Internet".

I will read it again, but use a translator. 

[Jingo]

NAS can offer a solid solution especially in a shared use environment.. HOWEVER, not much beats the speed and reliability of local drives for single use scenarios.  I have over 40TB of drives hooked up locally to my PC with a 20TB NAS solution on the network.  My local drives are all backed up nightly to ... other local drives.. and my photo array is also backed up weekly to the NAS.  Redundant backup is so important these days.... I even back up "mission critical" data to external HD's which then get stored off-site.

While I love the idea of NAS... even with gigabit networks - having access to these files locally still gives me a performance boost.

[Mario]

NAS is especially good for streaming use and long-term archival. Nothing beats local spinning disks or even SSD when it comes to speed.
Pulling files over a network is 10 to 100 times slower.

IMWS running on a NAS would be great...

[greg]

While having files stored on a Linux NAS RAID system separate from the local Windows system provides extra security the network slows down access. A Gigabit LAN system helps, but as you say, NAS still is nowhere as fast as a local SSD. For sometime I thought local SSD with RAID backup would work best for me, but it doesn't scale well because of the cost of SSD. My recent USB3.1 SSD failure wasn't a dead device, but a loss of the file system on the SSD. Recovery looked to be possible but not of the directory organization.  I don't know what kind of hardware, software or human caused event caused the failure. Recovery was a huge disruption for several days, so that was the tipping point that flipped me to abandon SSD as a primary storage. Since I tend to work on just a few directories of files at a time, copying files from NAS to local SSD, working with them on SSD and then copying them back to NAS is my solution to get performance. I use IMatch for file copying which works cleanly with the database. I continue to be impressed with the way IMatch has been able to handle various working processes I've thrown at it.

[Mario]

I have never had any problems with a SSD, and I'm on my 10th SSD now, 4th generation.
Before I switched to SSD as main storage, I lost two hard disks per year due to failure.

Tip: I you want somebody to read your posts, use shorter paragraphs (press Return now and then). Your post is two screen pages of solid text block on a smart phone, very hard to read.

[greg]

 I didn't consider phones when talking about IMatch and storage, but will try to keep that in mind. Old school typing on a keyboard with a large display is far easier for me than hunt and peck on a phone.

My computer has  only internal SSDs with two external USB SSDs, but as with any hardware and software problems can occur. Perhaps the bigger issue is having all storage on Windows computers, with which I have minimal problems, but is such a big target for virus and hackers.

[Mario]

I often work mobile.

[loweskid]

I often work mobile.

I sometimes think you must work while you are asleep.....

[Jingo]

I didn't consider phones when talking about IMatch and storage, but will try to keep that in mind. Old school typing on a keyboard with a large display is far easier for me than hunt and peck on a phone.

My computer has  only internal SSDs with two external USB SSDs, but as with any hardware and software problems can occur. Perhaps the bigger issue is having all storage on Windows computers, with which I have minimal problems, but is such a big target for virus and hackers.

25+ year windows users... 2 viruses found in all those years.  Windows OS is rock solid for me and does everything I want and more... also, perhaps I am very lucky - but I've only lost 2 HDD in all that time as well... both Seagate and both used for a fair amount of time.  With the amount of disk reads/writes I put on all my media - I will NEVER complain if/when one fails.... easy, peasy to restore to a newer, faster and larger drive when I replace it as well thx for backup/restore software.  Life is good!

[Aubrey]

I've been using Backblaze for some 8+ years.
My setup does not use NAS, I use USB disks attached to a PC.
I initially back up to the USB disks from other PCs using Allwaysync, this automatically starts on a schedule. This also gives automatic local backup of important files

The PC running backblaze is the same as running IMWS so it is never shut down.

Works a charm for me 50$ per year. I have about 4Tb backed up.
Aubrey.