RawTherapee & Nikon DSLR Cameras
2019.12.22
Here are some tips for editing Nikon RAW NEF files using
RawTherapee, more specifically for Nikon DSLR camera models which
already have a camera model specific DCP color profile included
within RawTherapee.
If your Nikon DSLR camera is not already supported by a previously
submitted camera model specific DCP color profiles for the
RawTherapee project, you can easily create a camera model specific
DCP color profile by purchasing an X-Rite ColorChecker (Passport),
taking a few photos of the ColorChecker and submitting them to the
RawTherapee project. Knowing whether your DSLR camera model
already has a camera model specific DCP color profile submitted to
the RawTherapee project is not easily readily known, usually
requiring the user to query the RawTherapee project forums.
The RawTherapee program automatically chooses Automatic for DCP
support, and the program does not display whether the RawTherapee
program is using a camera model specific DCP profile or the basic
generic DCP profile.
For myself, the Nikon D5600 did not have a camera model specific DCP
color profile yet submitted to the RawTherapee project, and required
going through a brief process of submitting a bug report along with
my X-Rite ColorChecker Passport photos. Once performed, your
colors will look much better and comparative to Nikon's processed
photos.
Here are some tips or bugs I noticed when using RawTherapee 5.2, and
the following will either be resolved or will help you with
processing your Nikon NEF raw image files.
CURRENT STATUS: Most of these jottings are from my notes when trying
to initially attempting to acquire duplicate or similar results from
my Nikon camera's firmware or Nikon's Caputre NX-D software
results. Most of the below jottings are likely deprecated as
of 2019.05.11 enable "Exposure > Tone Mapping" and disable
"Detail > Noise Reduction". I'm just currently seeing
slightly reduced highlights and slightly less contrast, along with
not adjusting for lens anomalies with my current neutral-nikon-d5600.pp3
file. Lens profiles for Nikon's AF-P lenses can be
extracted from the latest Adobe Camera Raw softare. Try to use
Adobe Camera Raw 11.3+, as other AF-P profiles seem either
incomplete or lacking in some manner.
As time permits, I'll duplicate my efforts and cut all unnecessary
chatter within this page to a minimal.
NOTE: Switch-off Active D-Highlighting within the camera, as Nikon
ADL adjustments negatively effect the raw image file when importing
into third party RAW editors. (Active D-Highlighting is
apparently proprietary.) All other settings have little to no
effect on the raw image file when importing into third party RAW
editors.
NOTE: I prefer, Neutral picture control. I also tested and use
High ISO Noise Removal set to High, which is applied to ISO settings
greater than 800/1600. Most of these settings are geared
towards the JPEG processed image. The Nikon D5600 seems to
have noticeable noise at ISO settings above 1600. Once set,
the maximum ISO I use is 1600, reserving higher ISO values for when
the camera automatically uses those higher ISO values. During
my brief testing, High ISO NR had no noticeable effect on images
less than 1600, with only positive effects on ISO values 1600 and
higher when set to High.
NOTE: For still photos or photos I value, I prefer having the camera
save to RAW files only, as the Nikon RAW NEF file already includes
an embedded 2MB JPEG and having an extremely similar quality with
the Nikon D5600 6MB JPEG files. For photos requiring
continuous release mode resulting in many photos, such as moving
objects, I tend to prefer JPEG Fine. Use Linux/GNU Geeqie for
readily viewing NEF files, as Geeqie displays the embedded 2MB JPEG
image.
After reading the above section, most will just want to skip to the
bottom of the page and acquire the PP3 profiles within the My Files
section. Remember to first grab the lens profiles as
previously mentioned!
RawTherapee Processing Steps for Nikon RAW NEF Files
The following information will help provide identical results
using RawTherapee in comparison to using Nikon's in camera
processing (or Nikon's software processing) for creating images from
Nikon's NEF raw image files. If you want to skip all of this
and just want to try my working profile, here it is: neutral-nikon-d5600.pp3
Be aware this profile is not adjusted for lenses, which may include
vignetting or lens anomalies, for which steps 10 and above
address. (The lens profiles are probably hard-coded or within
an internal database within Nikon's camera firmware and Nikon's
proprietary software.)
Key steps are adjust the DCP profile and it's associated options,
and setting the RawTherapee processing profile. Then adjusting
exposure setting. The remaining options (eg. white
balance, noise reduction, ...) can likely be performed in any
order. Probably should reserve noise reduction alterations as
one of the final steps, unlike it's indicated listed order below.
1) Sensor with Bayer Matrix > Demoasaicing > Border
Check the RAW image resolution. The Nikon D5600 camera is
producing 6008x4008 resolution images, which will incur anomalies
when applying lens corrections. Setting the border to 8, will
provide 6000x4000 resolution images without enabling cropping or
resizing.
2) Color > Color Management > Input Profile > Color
Management > DCP Profile
Ensure your camera model has a camera model specific DCP included
within RawTherapee's code base, and this camera model specific
RawTherapee DCP profile will be automatically enabled upon image
loading. Ensuring "Auto DCP" is the default or is always
selected, this does not help users much from distinguishing whether
or not RawTherapee is using a camera specific DCP profile. (To
verify, post to the RawTherapee user forums will likely inform
whether or not your specific camera model already has an included
DCP profile, or is still using a generic DCP profile. Or grep
the code.)
3) Color > Color Management > Input Profile > Color
Management > DCP Options
The automatically detected (or your specified or embedded profile)
should likely already have the following key options enabled.
Base Table should already be selected, and in my case, Baseline
Exposure is checked and grayed-out. The following key options
should also be checked for Nikon (D5600) images, providing an
eye-pleasing image versus color toning not matching the original
camera processed image. By default within RawTherapee 5.2,
Tone Curve and Look Table are deactivated by default, regardless of
what the hover-over info states! Active "Tone Curve" and "Look
Table" regardless of the hover-over info stating these further
options are only activated if a DCP profile contains these
features. NOTE: This is the key option for making Nikon
D5600 raw NEF images eye-pleasing or comparable to the in camera
processed JPEG images! I struggled for months trying to
figure-out why in camera photos were not comparable to RawTherapee
images. Now you should be able to move forward in you image
processing more easily, providing images slightly or far better
than the in camera processed images!
4) Processing Profiles > Neutral
Use the NEUTRAL RawTherapee profile. If you use the Default
RawTherapee profile, I have little idea what you'll get at this
point, aside from having an image comparable to being post-processed
with Gimp's Color Auto Levels, having clipped whites and blacks with
an adjusted gamma, providing a more contrasting vivid image; versus
a (neutral) realistic duplicate of what you're naked-eye saw when
taking the photo. (The bundled Default profile for
RawTherapee does provide an image similar to the end results of my
instructions here, but I have some difficulty with color matching,
etc? I haven't re-verified if my results are still similar
after activating DCP "Tone Curve" and "Look Table" options
mentioned above.)
5) Might be a good idea to save the profile as a (eg.
NIKON-NEUTRAL-D5600) custom profile now, are just after the next
step, setting a minimal noise level.
6) Detail > "Noise Reduction" > Luminance & Luminance
Detail (NOT NEEDED UNLESS YOUR IMAGE IS DARK OR HAS
NOISE!) Enable Detail > "Noise Reduction" > Luminance
& Luminance Detail options, unless the photo was taken within
extremely bright conditions from my experience. Regardless,
enable noise reduction and I usually set 100% Luminance and 40-50%
for Luminance Detail. BIG NOTE: Sometimes noise
reduction does not immediately show within either the image OR when
using the Image Detail Window. (Middle right bottom icon,
indicated with a square with a plus sign in the upper right
corner.) (2019.04.24 I recently noticed detail was more blurry
than Nikon NEF embedded JPEG file, and boosting the Luminance
detail from 40-50% to 100% then displayed duplicate levels of
detail. Nikon's Neutral profile seems to boost detail, as such
the likely reason why I'm seeing better detail within the rasturized
NEF raw embedded JPEG file using the Nikon Neutral profile.) Almost
all the time after I save the image (eg. CTRL + S) and viewing
within The Gimp, I find noise reduction did perform extremely
well. This seems like a bug, as users should see noise
reduction always occur within the 100% image detail window, but in
reality sometimes just does not show noise reduction! Another
note, be careful not to over-do luminance detail, as this option can
create too much distracting color contrast. The only
detectable method is to compare the image to the original in camera
process (JPEG) image, and make sure colors are not too over
exaggerated or contrasting or overly vivid. One might
initially think saturation levels, but with these settings, we
should not be seeing any over-saturation. When reducing
luminance detail, will need to likely reduce luminance levels as
well. NOTE: At higher ISO levels per camera settings, High ISO
NR might be applied to only higher ISO's, or likely exposures using
> 1600 ISO's. I have no idea what is being performed when
Nikon's High ISO NR is being applied, but do notice I have a
particularly difficult time removing red/green/blue noise from dark
25600 ISO exposures. I have High ISO NR activated as I
usually try to only work with < 1600 ISO, not expecting to work
with the grainier >1600 ISO exposures.
a) High ISO NR: Or, High ISO Noise Remove is
something performed during in camera processing on Nikon cameras,
for photos using >= 1600 ISO. At 25600 ISO, High ISO NR
usually becomes extremely desirable. Unfortunately, this
processing does not carry over for raw exposures with third party
raw editors. I've have found noise can be further suppressed
by first using a 100% Luminance value while setting Luminance Detail
to 0%. Choose the Manual Chrominance option and increase the
Chrominance Master channel from 15 to 40%, or more. (Possibly,
Activate Median Filter, increase Median Filter from 3x3 to 9x9, at
the expense of processing time, although I've seen no difference
here at 3x's zoom) In brief, opting not to increase Luminance
Detail provides an overall softer image, while increasing the Master
Chrominance channel seems to resolve almost all noise.
Unfortunately, you'll likely need to first export/save the image in
order to see these specific changes rather than trying to use the
Detail Window! Probably save a sidecar profile and name it
"NEUTRAL-NIKON-D5600-HIGHISONR.pp3", and use instead of the usual
"NEUTRAL-NIKON-D5600.pp3" sidecar. I'm guessing there's like a
bit flipped within the proprietary tags, and the Nikon Capture NX-D
editor institutes a similar escalated noise removal such as this one
described here. (Almost looks as if some sort of Active-D
Highlighting might also be implemented within this High ISO NR
process when comparing images here.)
7) Exposure > Exposure Compensation
At this point, adjust the exposure of the image. I usually
find I always need to reduce the exposure by a .5 to 1.0
value. (Exposure compensation setting is apparently not the
job of the color DCP profile.) What I usually do is use Geeqie
to view my Nikon NEF raw images, in which Geeqie displays the 2MB
embedded NEF JPEG image, far quicker thank Windows does I might
add! Using the embedded NEF JPEG image as a reference image, I
then set the exposure compensation within RawTherapee to something
similar to the embedded NEF JPEG image, and further adjust the
exposure compensation to my liking. Typically I expose for 1/3
stop less on most exposures, so setting this to -0.33 or -0.66 (for
-1/3 or -2/3 stops) is the norm for me.
8) Exposure > Tone Mapping - Activate but leave at defaults.
9) Color > White Balance
If you took a white balance photo, you should probably apply a
specific white balance value to the image now using the dropper
within the Color > White Balance menu. (NOTE: The white
balance image should be white, or a very slight not noticeable gray,
while the middle 18% gray is noticeably grayer and used for setting
manual exposure instead of using the auto exposure shutter button
while taking the photo. However I commonly hear people using
the 18% gray patch for setting white balance too.) The set
White Balance value can be copied to the other photos within the
film strip displayed images. NOTE: Strongly suggest using a white
balance from a X-Rite ColorChecker Passport device or similar!
NOTE: RawTherapee Automatic White Balance seems to provide incorrect
values. Manual setting seems to provide best results by
setting to daylight, beyond both Automatic and Camera.
10) Saturation
Best to enable Vibrance within the Color menu, or play contrasts by
adjusting the scaling level within the Tone Mapping.
11) Transform > Lens Correction Profile (Using RawTherapee
Plexiglas trick exact removal of vignetting.)
Transform > Vignetting Correction (User manual adjust
vignetting... guess work!)
Depending whether you used a zoom or telephoto lens, you may need to
adjust for vignetting. RawTherapee has an article on how to
create your own lens profile for taking a series of lens specific
profile photo images using clouded Plexiglas, for automatically
removing vignetting according to the profiled used lens. Or,
just use the guesswork method. (DEPRECATED. LensFun and Adobe
Camera Raw now include Nikon AF-P lens profiles.)
12) Transform > ? (Correct barrel and pincushion distortion.)
If you're using a Wide lens, you'll likely need to remove the lens
distortion, commonly called barrel or pincushion warping. Not
sure how this is performed yet, but later Nikon models have an in
camera menu item for selecting automatically applying lens
correction to the raw image. Not sure if this is carried over
to RawTherapee imported images, but I would think this would
occur. (DEPRECATED. LensFun and Adobe Camera Raw now include
Nikon AF-P lens profiles.)
13) CTRL + S
Export the image to JPEG, TIFF or your preferred image file format
for importing for using within ImageMagick or The Gimp. Double
check to ensure noise reduction took effect and all your other image
corrections look good.
14) Experiment using Exposure > CIE Color Appearance Model
2002. Activate sub-option "Tone mapping using CIECAM02?
For RawTherapee, the default Color Appearance Model used is CIELAB
(1976) (eg. L*a*b*), while CIECAM02 (2002) is a more recent color
appearance model. Per Color Appearance Model Wikipedia for
CIECAM02, "It performs better and is simpler at the same time.
Apart from the rudimentary CIELAB model, CIECAM02 comes closest to
an internationally agreed upon 'standard' for a (comprehensive)
color appearance model." But on this note as of the year 2017
and only related so far as the term "international" is concerned, I
still prefer to using ASCII character sets on all my computers
instead of UTF-8. I reside within the US and have no need for
the extra UTF-8 characters, causing confusion and further bugs with
programs. (I tend to get better results using CIE 2002, but
not needed with my current PP3 profiles.)
Once you learn the above hurdles, you'll find RawTherapee does a
pretty darn good job alongside Nikon's proprietary results, if not
far better. Once your satisfied with the results, save the
profile within your RawTherapee processing profiles to something
such as "neutral-nikon-d5600.pp3".
If you're looking for results similar to Adobe LightRoom,
significantly increase the Tone Mapping scaling. Also can
experiment with activating Wavelets and increasing the contrast
levels from neutral.
Geeqie Tip
Geeqie will display the embedded JPEG image for your NEF raw
images files. (Nikon NEF raw images tend to have a 2MB
embedded JPEG having similar resolution in comparison to a full
resolution 10-15 MB JPEG.) As such, Geeqie tends to be
extremely quick when viewing Nikon NEF raw images, unlike the Nikon
Microsoft Windows plugin rasturizing each embedded raw image for
each Nikon NEF raw image file prior to displaying. There
are likely no drawbacks for Geeqie viewing the embedded JPEG image
when a user is just scanning through photos, so very likely
preferred for Geeqie to display the embedded JPEG image.
If you use Geeqie for viewing Nikon camera created images, should
probably add the following XMP tags to Geeqie's EXIF menu.
(eg. Right click within Exif menu, click "Add entry", and disable
"Show only if set" when testing.) The following additional
EXIF tags are not shown by default, and need to be created for
viewing within Geeqie's EXIF side menu:
Xmp.dc.creator
Xmp.dc.rights
formatted.GPSPosition
formatted.GPSAltitude
Xmp.xmp.Rating
Preferably set Xmp.xmp.Rating as editable, or likely any
previously mentioned tag can be set as editable. I like to
rate (1-5 stars) all photos I plan to keep and not trash/delete,
while all photos unrated (0 stars) indicate a lack of personal
value, such as photos of nuisance garbage. Using this method,
I can hopefully elect to move/delete all unrated (0 stars) nuisance
photos all at once, at any future time. I have not discerned
what to do with 2-4 star values, however five star rated images are
obviously keepers. Think I've previously used 2-4 star ratings
as a method of grouping photos during a session. Only one
problem with my plan, do not forget to rate (assign 1-5 stars to)
images prior to a move/delete operation!
My Files
Users can create a dynamic profile with filters for specific lenses
within RawTherapee. (The more proper method is to isolate the
lens correction settings for the lenses to their respected file,
with all other settings within the general base profile file.
Since I'm limited on time, all files contain duplicate redundant
settings.)
NOTE: The lens specific profiles contain a static location to my
folder containing Adobe Camera Raw lens profiles.
NOTE: The denoise PP3 profile is to be applied after one of the
previous lens specific PP3 profile have been applied. Open the
denoise PP3 file within your favorite text editor, you'll find the
file only contains denoise options for correcting grainy low-lit
photos, and will be applied without changing any other options.
Camera: NIKON D5600
Lens: Generic base profile
neutral-nikon-d5600.pp3
Camera: NIKON D5600
Lens: AF-P DX Nikkor 10-20mm f/4.5-5.6G VR
neutral-nikon_d5600-afpdxnikkor_10-20mm.pp3
Camera: NIKON D5600
Lens: AF-P DX Nikkor 18-55mm f/3.5-5.6G
neutral-nikon_d5600-afpdxnikkor_18-55mm.pp3
Camera: NIKON D5600
Lens: AF-P DX Nikkor 70-300mm f/4.5-6.3G ED VR
neutral-nikon_d5600-afpdxnikkor_70-300mm.pp3
Camera: All
Lens: All
neutral-nikon-d5600-denoise.pp3
RawTherapee Dynamic Profile
Copy to $USER/.config/RawTherapee/dynamicprofile.cfg or use this
file's data for creating your own dynamic profile.
dynamicprofile.cfg
Third Party Products
1) Adobe Camera Raw 11.3 package contains both lens profiles for
AF-P lenses as well as Nikon D5600 (*.dcp) color profiles. I
prefer using RawTherapee's DCP profile rather than Adobe's DCP
profiles. Regardless, I copy over the DCP files into
~/.config/RawTherapee/profiles folder in the event I do need
them. So far, I mostly rely on the Adobe Camera Raw lens
profiles for the Nikon AF-P lenses, as the LensFun AF-P lens
profiles (added at version lensfun-0.3.95) seems to have some
anomalies.
2) X-Rite ColorChecker Passport
To Do
1) Ensure proper consistent results are being obtained with the
Nikon 70-300mm telephoto lens. Most testing or comparing with
Nikon Capture NX-D or Adobe LightRoom results have been limited to
the 18-55mm kit lens.
Errata
2019.12.22 Removed Lens Correction "Geometric Distortion" &
"Vignetting" corrections. On some photos, the geometric
distortion was incorrectly modifying the image border and vigetting
correction was creating abnormal highlights. It could be a
fastmath compiler error, however, I do not think the Nikon OEM
software, Capture NX-D or the in camera firmware corrects for
distortion or vignetting. I have also included a denoise PP3
profile, for applying after the appropriate lens designated PP3 file
is applied.
2019.08.25 Modify Demosaicing Border from 4 to 8, due to the Nikon
NEF files being 6008x4008 in raw form. Doing so also
alleviates lens deformations as well. I've setup dynamic
profiles within RawTherapee for automatically applying lens
correction files using Adobe Camera Raw lens profiles. I'll
update my PP3 file shortly. Also, to create Adobe LightRoom
similar images, increase the Tone Mapping scaling to somewhere near
maximum. (I'm pretty sure Adobe LightRoom is applying a custom
curve, whether within tone mapping or wavelets, or both.
Shrugs, I prefer a little color versus increasing black and white
tonal contrasts.)
2019.08.22 Updated neutral-nikon-d5600.pp3. Enabled Lens
Deformation (Lens Fun) profiles set to automatic. Important, do not
enable/activate "Lens Deformation" (the function below the lens
profiles function) as I noticed this conflicts.
2019.05.11 Cleaned-up my neutral-nikon-d5600.pp3, disabling noise
reduction and enabling tone mapping within Exposure menu. I
haven't looked over my current neutral-nikon-d5600.pp3 with a text
editor, but you may find most options are set to defaults with the
exception of exposure & color profile adjustments, along with
enabling tone mapping.
2019.04.24 Added note within Luminance detail, to boost luminance
detail to 100% from 40-50%. Nikon's Neutral profile seems to
boost detail, when processing the original camera raw image
file. About the only item I'm noticing at this point, is
slightly clipped colors at the far right of the histogram, along
with slightly augment blue colors. Likely a slight adjustment
in exposure levels, and/or adjust in the blue color spectrum, other
than this the results are seemingly identical to Nikon's in camera
processing (or Nikon software processing of raw images.