Which RAW image format is best?

One of the more confusing decisions that I had to make as a beginning Lightroom user was which raw format I should use. Of course, if you read all the Adobe literature and tutorials, it’s hard not to come away with the idea that DNG is the answer. But really, which is better? DNG or NEF (or cr2, or whatever proprietary raw format your camera of choice spits out)? This is question that I have recently begun to revisit.

DNG Logo

For the last two and a half years I have been a big advocate of the DNG format. As probably everyone knows by this point, DNG is a raw image format developed by Adobe, and is designed to be a long term, archival storage format for your digital originals. The concept behind DNG files is that, as an openly documented standard, software engineers currently or in the future could incorporate that standard into their cameras or editing software. This, in theory at least, should mean they these files should always be able to be opened. Sort of like a .txt file, PDF document, or a jpeg image. When’s the last time you had a problem opening a text file? How about a PDF? These are all widely adopted, open standards file formats.

My reasons for using DNG were twofold: first, I always like to err on the side of open formats. I don’t like the idea of thinking that in 10, 20, or 50 years, I won’t be able to access my digital files because they could only be opened by some arcane bit of software that the manufacturer no longer supports. Don’t think it can’t happen. We have actually had this problem at work with scanned images of old maps and documents-TIFF files no less! Whoever scanned these files back 15 years ago apparently just used whatever default settings were provided by the  scanning software, and saved these archival scans as tiff-tagged jpegs, using some weird, not-widely-supported encoding format. Depending on what software you open the files with, it will render the images pink, black, or just plain refuse to open them. These types of problems should be avoided by using the DNG format.

My second reason was that I liked the idea of having the metadata for the image written right into the .dng file. All raw images contain metadata: keywords, copyright information, location (GPS coordinates if your camera can do that), etc, etc. This metadata is what allows the  file to be read by software, and also to be “edited.”

All the major camera raw formats are largely undocumented (if at all). The engineers developing any third party editing software (read: Adobe Lightroom) have to sort of reverse engineer these proprietary raw formats, just to read the data out of them. If this doesn’t sound like a good formula for reliably being able to write edited metadata back into the proprietary files, you would be right. That’s how we wound up with XMP sidecar files. All the metadata that can be read out of the original raw file is recorded in these little sidecar files. After that, any keywording, copyright info, all metadata changes that you make to your images are recorded in that same XMP file. That leaves your original raw file safely untouched by the software. It’s a great system. The difference with DNG is, because there is no reverse engineering or guesswork involved, it is safe and easy for the software to record and edit all that metadata right into the .dng file. My feeling with this was that it would simplify file management. You wouldn’t have all those little XMP files running around, that have to stay with their associated image.

All this said, there are a few further facts about DNG’s that I have discovered along the way. These have made me seriously consider just sticking with the original raw file, rather than converting everything to .dng as I have been doing.

    1. The XMP metadata changes that are written into the .dng file do not include the full history of edits that were made to the file.This was one of the main reasons that I was writing the metadata to the file in the first place, sort of a backup of my catalog edits that would stay with the file no matter where it went. It is true that your edits get stored – whatever the most recent state of the file is in Lightroom, including snapshots. – but the edit history does not get recorded. Neither do your flags, ratings, or collections info. It kind of lessens the usefulness of the whole “write metadata into the dng” feature I addressed above.
    1. There are certain contests that require entries to include the original raw file that came out of the camera. That means that if you converted to dng, and did not retain your original image file, you are out of luck… There are ways to preserve the original file, DNGs even have the ability to embed the original raw when you do the conversion – but this is at the cost of taking up twice as much storage space! Storage is cheap, but not that cheap…
  1. DNG metadata writes sort of mess up my cloud backup workflow. Think about it, every time you make an edit to a dng, it writes those changes right into the file. My backup software looks at the image and says “Aha! The file has changed!”, and dutifully uploads it to cloud storage. All ~30MB of it. Now, say I edit 200 images in one sitting…see how that could be a problem? If you are using the camera manufacturers raw format, the only file that will have changed is that little XMP metadata file. Now we are only talking a few megabytes of data (at most) that have to be backed up, rather than 15GB.

These three problems are my main reasons for migrating my workflow towards just retaining the original .nef files as they come out of my camera. There are some other benefits as well, like not having to wait 20 minutes for Lightroom to convert everything to DNG as I am importing, but these are the main ones. I probably will adopt a policy at some point of ultimately converting a copy of my originals to .dng, for long term archival and backup purposes; however, for my working copies and ordinary backups, I am just going to leave them in their original format.

I hope this has been helpful to anyone trying to decide on a raw workflow, and I’d love to hear any opinions you might have on this subject!