OK, I saw a post on Mastodon about open-source tools for photo and video editing and it reminded me that I’ve been using DxO PhotoLab for more than 10 years and it is time to see if the open-source tools have caught up. I generally liked DxO over Lightroom because their defaults seem closer to reality. For a while, I was using their neutral contrast but finally just switched to the default color profile in the Sony R7 Mark II because it popped more than the natural tones. Sort of the way that Velvia is just brighter.
The other big reason I’ve stayed is that DxO has their proprietary lens correction and their proprietary noise reduction (which corrects for both vignette and optical flaws and gets rid of noise in the images). But with the arrival of the Lensfun database, these are now available to open-source photo editing programs. Also, I’m now shooting exclusively with prime lenses so the optical defects are really small (like tiny) and there is vignetting, but typically this is not that bad either. And also when I’m shooting I don’t see that much noise with modern cameras.
Anyway, first I tried Raw Therapee which I’ve tried a few times, but this project seems to have really slowed down. The latest version 5.9 shipped in 2022 after two years and the Mac version has not arrived. In looking at reviews, Darktable seems to be really coming up fast and it has library management as well.
There have traditionally been two categories of photographic programs. There is Lightroom which handles lots of photos and puts them into categories with basic corrections and then Photoshop which doesn’t deal with lots of photos, but has really deep corrections (eg you can make deep fakes).
I find I normally do not do much in the way of hard-core corrections, I like the photo to reflect what was actually there, but the distortions bother me (for the same reason). So I mainly shoot RAW and then process myself.
Net, net, with the emergence of darktable, Luminance HDR, and Hugin, there are fewer and fewer reasons to pay the Adobe tax. If you want to support small folks who are doing a good job, the DxO LightRoom is good (they do annual updates, but I find that skipping every other update works fine). Also if you are Mac-only, there are some good programs over there that I have to try particularly for iPad-only editing (which would be great!).
Darktable Tips and Tricks for a DxO User
OK, since I’m going from DxO, the question was how to change the workflow. The biggest difference between DxO and Darktable is that darktable has no menu bar at all, it is one of these new “hunt and peck” interfaces where everything is scattered in buttons everywhere.
In terms of understanding what it is doing, I do wish they had a tutorial on basic concepts, but the big ideas are there are a few different programs in a single interface. At the upper right, there is “lightroom” which is for managing collections, and then “darkroom” which is in-depth single image editing. Note that this is sort of like Lightroom and DxO where there is the concept of “Photo Library” and then “Customize”. So here are the big differences and confusing things if you are coming from DxO
However, there are lots of things that you can’t do in the lightroom area that only work in the Darkroom applications.
Ultimately I think I’ll probably move over to Darktable
Except for really difficult high-noise scenes, I’ll probably keep using DxO Photolab for those cases and try some iPad-only editing for the next iteration in 2023, but here are some confusing things and tips:
- The most important thing to realize is that this does not have layers like Lightroom instead it has an image-processing pipeline. On the left are the various status indications. but on the right below the histogram is a list of all the modules and from top to bottom how they are being run. The first tab is the most frequently used module. The next one is very useful, it shows just which modules are active so you can see what is being processed and the order. Then the next three tabs are in order Technical (which means things. you can do to the photograph like optical correction), then Grading (how you change the colors), then Effects (so special effects).
- Zooming in lightroom is not obvious, but on a Mac, the scroll wheel zooms in and out and then you use a two finger up and down swipe to zoom in and out.
- The modules that I normally use are output color profile, which is basically what range of colors. If you are producing photos mainly for the web, then I normally use sRGB which is a narrow range but works everywhere (or AdobeRGB), but if you are taking a RAW image that has way more than 8-bit color and producing say JPEG-2000 which is 12-bit JPEG (although not used a lot in the real world), then you can use ProPhotoRGB with a much wider range of colors or Gamut or you can use Rec2020 which are the HDR color profile used in files.
- When I’m shooting with a Sony camera like my Sony R7 Mark III or little shooter, you can use their Imaging Edge Mobile tool to provide the GPS from your phone and this gets pushed into your EXIF. And Darktable correctly exports this to JPEGs (but not to AVIF or JPEG-2000 see below).
- You should get automatic lens correction from a large lens database that includes most of this.
- Darktable includes a map module in the upper right go to the other tab and look at map module. This lets you find a location and then you can drag from the filmstrip below and it adds the EXIF data. Unfortunately, when I tried this, it generated invalid EXIF GPS data in Apple Photos and the image would not upload. I have to file a bug on this.
- There is a setting called filmic that lowers the saturation of the output quite a lot. This produces a more natural image and a module called velvia that increase the contrast and saturation.
- It has a huge number of cameras in its database with information on White Balance and Noise generated including my camera, the Sony R7 Mark II
- You cannot just export (what DxO calls Save As) in lightroom by selecting with (Command-A) all the images and then on the right you can click on export side, and set format options on file format and you can set this to JPEG 8-bit and then set to Adobe RGB with 30% more. And it correctly exports things. You can select as much as possible in that view. You can also export individual photos in the darkroom area with the export on the left. And make sure to set the filenames correctly.
Move from 8-bit JPEG to 12-bit AVIF or JPEG-2000 missing EXIF
OK, the world keeps moving and every year I try to figure out how to actually use the full 14-bit RAW files that a modern mirrorless camera generates. Things are looking way better this year because, with iOS 16, iPadOS 16, and MacOS 13, they finally support AVIF (which is an open source more highly compressed version that also supports up to 16 bits). Apple moved to HEIC containers and HEIF a few years ago, but these are proprietary formats and require licensing so open-source tools like darktable don’t support it.
But, now darktable 4.2 supports AVIF output and it does seem to work for 8-, 10- and 12-bit color and also the Rec. 2020 PQ which is a wide color gamut profile, so this is actually really great. So you get more color points with the bit per pixel and then with Rec 2020 PQ as the output intent, you get a wide gamut that is more can be represented. Most JPEGs are using a very narrow range of colors called sRGB. And with good cameras, you can get a 30% wider gamut (range of colors). Then there is the blizzard of profiles beyond that. I’ve used ProPhotoRGB for a long time, but with the advent of HDR in televisions, the more common formats are what are used in video which is traditional Standard Dynamic Range (SDR) called Rec. 709, then is DCI-P3 which came out in the 2000s, but the most modern is REC 2020 which has two flavors, the HLG and then the really high-quality PQ.
The main issue right now is that the EXIF information is not correctly generated, so the JPEGs properly generate GPS and other data, but this is not in AVIF.
I have to say it is very cool that Preview and Apple Photos can view JPEG-2000 and AVIF but JPEG-2000 does not export any EXIF data at all and AVIF. With JPEG-2000 you get no data at all, with AVIF, you get at least the color profile so I can see it says it says Rec. 2020 PQ.
As an aside while it sounds great to take when shooting uncompressed RAW you get 14-bit color, it is in reality hard to see any difference in a 12-bit RAW.
Fixing the Missing EXIF with exiftool
So given that pain it turns out there is a way for EXIF from a jpeg to avif with EXIFTOOL with the command:
exiftool -tagsfromfile _DSA7384_dt.jpg -exif:all _DSA7384_dt.avif
# for a directory its a simple loop where %.* removes the extension
for JPG in *.jpg; do
exiftool -tagsfromfile "$JPG" -exif:all "${JPG%.*}.avif"
doneAnd this really does work to transfer these tags. Note that when you do an import of AVIF files into Apple Photos, it takes some time for the EXIF data to be processed. I was at first thinking that it didn’t work, but after a minute or so the EXIF information pops up which is nice!
Setting the Lossy ratios on AVIF at 95% results in 1.5x files than JPEG but with Wide Gamut
One interesting thing is that AVIF has a lossless mode and a lossy mode. If you take a 14-bit uncompressed RAW file at 80MB, then if you run it through a 12-bit AVIF lossless compression, the file actually end up being 2x smaller to 1.1x bigger. So this format when used like Adobe DNG is not a huge win. Since Apple Photos how handles Sony RAW files directly, if you are going to do this you might as well just load RAW files directly into Apple Photos and all is good.
The only real use then for AVIF is in lossy mode as a smaller replacement for JPEG files. So experiments here show that with lossy compression set at 95% and 12-bit, an 80MB RAW file gets to be about 13-44 MB vs an 8MB JPEG set at the same 95%, so it goes only modestly but has the full 12-bit. This is a pretty good compromise of size to data and is 2-6x smaller than the RAW file which is by definition the archival format of choice as it has exactly what the camera is putting out.
