DNG - WTF?!

This one got me really frustrated!
As usual, I was walking about, taking pictures with the LX7, in raw of course. And at the end of the day, I converted those raw files into digital negatives, as usual, using Adobe's DNG Converter.

Of course, at some stage, one starts working on those DNGs. And here's were the unpleasant surprise took place. Have a look:

Converted to DNG by Adobe DNG Converter

WTF?!
Here's how the raw file looks alike:

RW2 file

Both files are just scaled down and converted to PNG.
I do not mind the brightness shift too much. But what is this pink crap about in the DNG?!

Initially I thought that the conversion went wrong on one particular image, which would be bad enough. Hence, I repeated the conversion exercise. The result remained the same!
Since I am shooting +/-3Ev AEB, there are 2 more file to be checked. I will spare you with the result, but the other 2 frames also show the pink stuff! Which in a way is consistent...

At first, I put this incident into a "one of" box, however, it happened again, with other shots.

Now, this whole experience caused me having doubts about DNG and the software provided by Adobe. It seemed such a good idea to convert all files to digital negatives... not any longer.

To RAW or not to RAW?

Sure, RAW is cool, I use it all the times! However, there are some advantages of in-camera computations that should not be put aside.

The CHDK allows you to store, with your Canon camera, RAW files. Such files (I am sure you know all about this) represent the data as recorded by the sensor plus some meta-data such as camera settings.

RAW is good, it allows to manipulate image data whenever you want, e.g. adjusting white balance. There is not compression in the raw format, since it represent raw data. Which is good, because there will be no compression artifacts.

However, there are some down-sides to ignoring the camera's computations. Since CHDK records both, the processed JPG as well as the RAW (DNG - see below), we are able to see the differences ... fingers crossed that blogger does not "improve" the images.

The images below are results of the same photo, i.e. frame. Both files were taken from the memory card, scaled down and saved as 95% quality JPG.

Lets have a look at the raw image:

DNG (raw image) as recorded by the camera

Obviously there is some vignetting going on, cf. upper left corner. The image also shows a very strong distortion as known from wide-angle lenses... mind you, those buildings are all straight...
Colors are rich and there is a good contrast. The finer detail, e.g. fancy structure on the left building's roof, show clear definition.

On the JPG side of things:

JPG as processed by the camera

We are now loosing definition, very clearly. Also the colors are less vibrant. The image is clearly cropped down, c.f. the bicycle in the foreground.
However, there is some lens-distortion correction going on. It seems that the camera actually over-corrects. Note that the modern building bends outwards to the right in the upper right corner.

Correct me if I am wrong, in my perception, the over-correction of the camera is less offending than the distortion seen in the raw image. Of course, in GIMP (potentially also in photoshop, but I can't tell) one can correct for such distortions, however, in the daily life of a P&S-user, it might be even reasonable to just go with JPG in the end.
Mind you, even though it might not be advisable, stacking techniques, such as HDR, can be done using JPG.

Note: Pure raw-format changes from camera to camera, hence, sometimes it is useful to use DNG (digital negative) format to store raw sensor information. The DHCK offers the option to store raw images directly in DNG. Personally, I do make use of that option. Up to now, I had no issues with DNG, when created within the camera.

Where Photography Meets Hacking

"What's that all about?", I hear you thinking... Well, this is where thing a really getting cool.
CANON cameras, at least some, in particular the cheaper ones, all of a sudden, make sense to be preferred over other brands. Yep, that's right! This is how to get a camera shooting in RAW or even DNG for very little money.

Being honest, on this topic, I am just reporting, nothing bears any contribution from me, all honors to the guys who made the following available:
CHDK

No, this is not some sort of UNIX command such as chmod, chgrp, chown, etc. it is the Canon Hack Development Kit... great stuff!
Mind you, today's digital cameras are more like computers. They might even have the compute power that the world was dreaming of 20 years ago. CHDK makes use of this fact and open a new world to photographers.

I learned about the CHDK some months ago and I was a little scared applying it to my older generation EOS DSLR. Anyway, this camera does everything I wanted anyways, so why bother?
A more recent member of my camera family had to hold up for my curiosity, the G15. If you study the CHDK homepage, you will notice why I made that choice.
CHDK works fine with the G15, however, again, there was not much to gain from the kit. The G15 shoots in raw, has custom settings etc. So, yes, CHDK worked fine, but was not adding "much" at the end (more about that at the end of this post).

There is a catch to the G15, it is not a camera to carry about in your pocket. Neither is the Panasonic DMC-LX7 (presently my favorite camera), although doing a bit better in terms of dimensions, the LX7 is still to big for a camera to carry on a daily basis. Something smaller had to join the group.
Have a look:

Canon IXUS 140, Samsung S760, Panasonic DMC-LX7, Canon G15, (*)

The Samsung S760 (see earlier post) followed me quite a bit, in my pocket, it is however clear that neither the LX7 nor the G15 will be able to take that space.

Initially I though of a Panasonic DMC-XS3, for its compactness and the advanced sensor, however, for the good stuff provided by CHDK, a new Canon was in order, one that fits into any pocket w/o disturbing too much. My first choice fell on an IXUS 125, being on sale for just €66, however, the store was not local, which would cause additional cost of shipping. My second choice was the IXUS 140, which not only has a longer zoom than the IXU 125, but also employs WiFi. A local store asks €119 for this camera, so I bought it.

Here comes the good stuff. There is an alpha-version of CHDK available for the IXUS 140. Believing that this camera will be very popular, I am sure that this alpha will evolve into a full release (I am willing to help). 
Anyhow, for just above (or under) €100,- one can get a pocket compact camera that shoots in RAW! How cool is that?!
And here comes the magic, your inexpensive little Canon is not only shooting in RAW, it is even able to create DNG-files (Adobe digital negative).

Best things to come (this is where I close the loop to the G15), with CHDK suitable cameras can be automated to an extend beyond wildest dreams. CHDK offers 2 different ways for automating your Canon by scripts: Lua and uBASIC. At a first look, Lua reminds me of FORTRAN with influences of C. uBASIC looks like BASIC to me, fair enough.
Hence, due to scripting, the camera also can do things that only you can come up with and you were only able to dream about to be implemented into your camera.
And... thanks to the small form factor in which some Canon cameras are available today, all that fits in your pocket easily.



(*) Note the different setting of the mode dial: the S760 on manual for the recent HDR demo, the LX7 on C1 (handheld HDR mode) and the G15 on P (like P&S). And yes, I like to use the straps, preventing me from dropping stuff to destruction.

Manual HDR Photography

To be clear from the beginning, this is about manual, i.e. as not automated, HDR photography, not handheld. Although it could potentially be done handheld, for starters I recommend a tripod.

Outside it is raining cats and dogs with a lovely storm going on, hence, I decided to stay inside this Christmas eve. Consequently, there are no landscape or architecture photos accompanying this particular post.

Some years ago, I picked up a P&S camera for various low demand purposes.
Selection criteria: it needed to

• be inexpensive
• run on AA or AAA batteries
• use SD-cards
• have a tripod mount.

That was essentially it. The choice fell on a Samsung S760, which serves me well. Most of the photos shown on my other blog (draaggolf.blogspot.com) are actually shot with this particular camera.

You may have guessed from my blog, that I experimented with HDR for a few weeks. Fun stuff. However, the variety of AE-bracketing cameras is rather limited and placed at the higher price segment, cf. http://www.hdr-photography.com/aeb.html.
However, we know that all that is required are 3, maybe even only 2, images of the same scene, with the same aperture but different shutter speeds.
Of course it is very convenient if AEB is already built into the camera in an automated way, in particular if it is as speedy as the 11fps of the LX7. But, it can be done with almost every digital camera on the market, provided that the camera has got a possibility to adjust exposure compensation, in one or the other way, and preferably has got a tripod mount.

Here is where in S760 came in. The camera employ a manual mode, with a +/-2 exposure meter... very handy! And, in addition to that, this particular model is equipped with a 2s or 10s self-timer.

Here's my workflow for bracketed exposures using the Samsung S760:

1. Mount the camera on a tripod and set your scene.
2. Switch mode dial to "P".
3. Power up.
4. Half-press the shutter release, the aperture and the shutter speed for a "normal" exposure will be displayed; try to memorize the numbers.
5. Change mode dial to "M".
6. Hit the "Fn" button, aperture and shutter-speed will be displayed in red.
7. Introduce those values with the "5 function button". This will be you middle frame.
8. Hit "OK" to confirm the settings; aperture and shutter speed will be displayed in white.
9. Engage the 2s timer by pressing the right part of the "5 function button" twice.
10. Press the shutter release to start timer and step back!
11. From here on, steps 6 to 10 will be repeated with difference that in step 7, only the shutter speeds should be changed. Multiply the shutter time by 4 to obtain a +2EV frame, divide the shutter time by 4 for a -2 EV frame.

Quick and dirty through the window demonstration:

HDR by exposure fusion

I am not sure what blogger does, it seems that blogger tries to "improve" the images... the original of the over-exposed frame is much brighter and the under-exposed is much much darker. Anyway, you get the idea...

The only downside to the whole story is that this camera, like all in-expensive P&S cameras, does not write RAW files. Compression artifacts may cause problems under certain circumstances.

Driving the G15 to the Limits

Today I gave the Canon Powershot G15 a session. First during daylight, later after sundown.
Of course, I tested the HDR capabilities, i.e. handheld bracketed frames. Additionally, I took photos with bokeh and also tried the built-in HDR function. The composition of those photographs are not good, I admit, however, composition was not the point of the tests I was doing.

Bokeh
Lets have a look at the bokeh first. The following image is recorded in RAW (of course), scaled down and exported to a 100% quality JPG.

Canon G15, f/1.8, s/400 @ ISO80, EV-1

HDR
What about HDR then? Well, here comes a test. The following photos will show the same scene, shot whilst the camera was solidly stable on the handrail of a pedestrian bridge.

the scene, single shot f/1.8, s/1000 @ ISO 80, EV-2

The built-in HDR function takes 3 frames (no user control), chews on them and spits out some photo. This is the result:

Built-in HDR, f/2.8, s/100(?) @ ISO80 (according to the camera)

And here is that same scene, shot in exposure bracketed mode, actually, the "single shot" is the middle image of this HDR.

no tricks here, 3 frames, -4 (s/2000 ??), -2 (s/1000) and 0 (s/250) EV, f/1.8, ISO80

To me the result is pretty obvious, the built-in HDR function solves some problems, but lags behind traditional HDR methods. In favor of the built-in HDR it is to be noted that the 3 frames are taken in a shorter time in comparison to the exposure bracketing mode.
I have the impression that the exposure compensation dial has no influence in the built-in HDR mode at all.

Handheld HDR
Now we are talking handheld exposure bracketed frames combined using HDR techniques. The problem with the camera is that it takes those 3 shots with about 1fps, giving the photographer a lot of time to ever so slightly move that camera. Using the optical viewfinder helps to reduce such a drift, also the position of your arms are more stable that way.
I skip showing the middle image here.

middle frame data: f/4.0, s/320 @ ISO80, EV-0.3

Pretty decent performance, thanks to the available HDR software.

Extremes
Later that day, I decided to pay a visit to the harbor of Scheveningen. At the time I got there, daylight was completely gone, stars were showing in the sky.
So it was time to try the extreme, a handheld set of bracketed photos under extreme light conditions.
Have a look:

middle frame

final HDR

There we go, dark sky and super bright deck-lights shining directly into the camera. I am sure there are better ways to treat the frames, however, it is clear that this how far this camera can go, not a bit further...

Conclusion
The Canon G15 does a good job. Despite the bracketed mode being a bit slow, it is possible to use this camera for handheld HDR photography purposes. However, the built-in HDR mode should be avoided, in particular since there a no anti-ghosting provisions.

The bokeh of the lens is not too bad.

Using C1 for HDR

Short note using the custom settings dial position(s) of the Panasonic DMC-LX7 for HDR.

If you search the internet about the C1 and C2 custom settings position, folks seem to advice to setup those memories using the P mode. Not sure why that is, maybe those folks believe that the P stands for "programming"... Well it does not, it stands for "Program AE", which is a mode that automatically selects the shutter speed and the aperture value. This mode allows for a little bit more user control over the camera's settings, e.g. ISO, in comparison to the iA auto mode. That's all there is to P.

For my HDR purposes, I actually configured the camera to my likings in A mode, the mode used for exposure bracketing, and saved those setting into C1. Why C1? Very simple, C2 has got 3 sub-modes, which the user needs to select via a menu. Anything under C2 is therefore not directly selectable.


Here are some of the settings (non exhaustive list) I use and why I use those:

AUTO ISO / max ISO 400
In the very beginning I used max ISO 80, which is fine in pure daylight. However, if you allow AUTO ISO, this is what the camera will selected anyway (watch you ND-filter). Under low light, ISO 400 still produces reasonably low noise +3EVframes with handheld-compatible shutter speeds.
Don't go any higher however! My experience is that the camera will select ISO 80-160 for the 0EV and ISO 80 for the -3EV shot.

Stabilizer OFF
Well, I love the stabilizer for single frame shots, in particular in lower light and/or longer lenses. However, the O.I.S. (Optical Image Stabilizer) uses a mobile doublet, see image below, which will cause tiny changes to the image geometry.
There is nothing wrong with tiny changes in image geometry, unless you want to match image coordinates from different frames.
Consequently, I want OIS always engage, unless I am taking bracketed frames for HDR.

FN button to "Quality"
This might look funny since you would consider to shoot in RAW only. That is entirely true, however, occasionally it comes along handy to also have a JPG recorded. The LX7 is able to record a JPG file next to the RAW file. The content of this JPG file is actually configurable in the camera itself, even camera effects will have influence on this file. A push on the FN button now allows for a quick decision to include or not said JPG file.

Bokeh in Winter 2013

You would not believe it, calender says it was winter, nature lets it feel like spring.

Shortly before I wanted to return home, this is what I have seen, a tree full of blossoms!
Good opportunity to test the bokeh of my LX7.

f=24mm, f/1.4, s/125 @ ISO80

This image is converted from RAW into TIFF, scaled and saved to JPG (100% quality); no processing other than scaling whatsoever.

All in all I am satisfied with the result. Lighting was somewhat tricky. For a first attempt, the bokeh looks OKish to me. 

Anecdotal would be a hypothetical photograph someone else could have taken: "Man with camera staring at tree". Today was a windy/gusty day, the little branch kept swaying in and out of focus and framing respectively.

Magic hour in "Westland"

December 14th 2013 in South Holland during the magic hour obviously, facing NNW.
The photo was taken with my Panasonic L7 in bracketing mode +/- 3EV, handheld of course.
I tried to replicate as much as what I perceived, although some may comment that the colors would  look unnatural.

For those interested, here are images of the original exposures, note that the 0EV photo is actually not aligned with the other two exposures:

0EV for the most of the content

-3EV for the details in the sky

+3EV for the details in the shadows

One trick I would like to share, I use anti-ghosting, i.e. only take the information of the 0EV photo, on parts of the water showing those nice ripples. When not using anti-ghosting, the ripples will wash out.
In principle there is nothing wrong with washed out ripples, however, flat water would make the observer miss a mirror image of the windmill and the shot would look really wrong.

DSLR replacement?

Earlier, I mentioned that the particular DSLR I personally own, is not the most convenient camera to haul about on a daily basis. Never the less, I really like the experience of my Canon EOS-350D.
Looking for a decent small form factor replacement, I came across the Canon PowerShot G15. Not a bad replacement at all!

However, there are problems...

1. Bulb exposure, i.e. full manual, is not available on the G15.
2. RAW is somewhat different from the old RAW format and not supported by the open source software I am using.
3. There is no IR-remote sensor, hence my RC-5 is rendered useless.
4. The filesystem of the camera is not compatible with anything.

Those points are actually ranked to the importance they have to me.

1) Full manual control, for night shots, would be a very nice thing to have!

2) There are (free) converters for strange raw formats, however, one leaves the "free software" world using those. It would be nice to have some free software tools available for converting the new raw formats to more common standards.

3) Remote control units are really useful. Although there is not IR receiver in the G15, a wired remote control can be hooked up to this particular camera. With free designs available on the internet, this should not pose a major obstacle.

4) Of course one can use the camera as "card reader"... but still, it would be nice to be able to just put the memory card into a reader and go from there.

Other than that, the G15 can be used for bracketing, even using the timer... just like the EOS-350D. However, the camera is too slow, in this mode, to be useful in handheld HDR photography. Carry a tripod and the G15 will do an OK job under not too dark conditions. Mind you, the shutter is limited to 15s, which is OK for most applications other than night photography.


Likes:
The G15 offers some really interesting features.
- First of all, the exposure compensation selector, which is very convenient during a regular session.
- The optical view finder. Classic and still very useful! I love it!
- The front and rear programmable dials, which provide excellent control for various parameters.

Dis-Likes:
- the funny filesystem
- slow bracketing speed

Verdict:
Up to now, I personally really the like camera, in particular because of its handling. I intend to use this camera for "general purposes", e.g. for portraits w/ some speedlights. For less demanding jobs, this camera seems to offer all the feature you might desire. 
However, this is neither an EOS-350D, not it is an LX7. The G15 certainly has its place in my photography life, however, it seems to be a compromise in all aspects.
Hence, if you would like to cover all aspects of photography, admitting certain compromises, the G15 may just be your camera.

Handheld HDR Photography ...

... can it be done?!

Here I am, writing my first entry in my newly created photography blog. Of course I had to pick a rather controversial topic to start with, HDR photography.

History

The first encounter with HDR photography was in the photography club in my school, I was 14 years old to that time. We tried to learn and master Ansel Adams' technique which of course required a lot time spent in the darkroom playing with liquids. Of course, as soon as affordable digital cameras were made available, somewhat mid/end nineties, I had to have one... However, the fun was limited, since the quality was rather capped, etc.
Not having that much time any longer to spend shaking development tanks (my enlarger has not been used in more than 30 years by now), and seen the inferior lomography like quality of the digital images, photography turned into some "point and shoot" experience for the occasional snapshot.

Mid this year, I picked up my aging Canon EOS-350D (aka. digital rebel) and started playing with bracketed short in aperture priority mode. This experience is/was a mixed bag, however, a tripod appeared to be an absolute necessity. Of course, a tripod is somewhat cumbersome to carry about and also the camera, although one of the smaller DSLRs is not really handy for everyday use.
 

The Camera

Consequently, the decision was taken that a new camera was falling due. The choice fell on a Panasonic Lumix DMC-LX7, initially mainly for the reason that it is able to do ±3EV bracketing.
Without knowing it yet, the basis of my decision seems the criterion for shooting handheld HDR.
Of course, the very fast Leica lens with an F-number of f/1.4 and the neutral density filter were very welcome features (BOKEH!!!).

There are some additional technical features to this particular camera which help a lot taking  bracketed images w/o a tripod. The particularly interesting feature is the camera's ability to take 11fps bracketed shots in raw.

Now, lets have a look why this particular camera makes it possible to shoot bracketed images for HDR without a tripod.

Principles

What you are looking for in HDR is to collect the details in the lights and in the shadows, cf. Ansel Adams' "Zone System", which are lost in the your typical "correctly" exposed photo due to the limitation of the dynamic range of your recording medium.

In principle, what we are looking for in the over exposed image is totally blown out lights and good texture in the darks. Concerning the under exposed image, we are looking for entirely black shadow and good texture in the lights.

Experience

Mind you, I am relatively new to this. The photos shown below are at the beginning of my learning curve. Also, I try to display what I have seen myself, in other words, I try to keep colors at levels which I believed I have seen. Maybe, at some later stage, I might create those artsy color overloads which HDR is infamous for.

The following shots were taken with my handheld LX7 in "P" (ups! - details in the image captions). Shooting HDR brackets in "P"? Well... it was a mistake, I took some non-bracketed photos before and forgot to change all the settings. Of course, you want to shoot your images for HDR in "Av".
Those (raw) images are not treated at all, just scaled down and saved as jpg.

normal exposure - looks kinda odd (f/2.8, 1s/1600, ISO 80)

overexposed - as expected (f/2.8, 1s/200, ISO 80)

underexposed - not so much to see here (f/6.3, 1s/2500, ISO 80)

Actually, seen that the camera decided to stop down the under exposed image at a very fast shutter speed, the use of the ND-filter might have been a good idea. Luckily, the aperture change did not influence the geometry of the underexposed frame.

It is evident that those images show the textures mentioned above. Of course, this is a cliché shot for HDR photography and yes, the composition is a little on the boring side, I admit.
After having mingled together the 3 raw photos, this is what my result is:

final HDR photo

For obtaining the final result, anti-ghosting was used on the car. Other than that, I tweaked the colors, contrast and brightness a bit, using the GIMP. Essentially, I reduced color saturation and leveled out the image a bit.

Note the features highlighted in the following image. The details annotated in blue are results of the underexposed frame, while the features marked in red are a result of the overexposed contribution.

annotated final image

You may notice that the grass is a little bit greener on the result than on the over exposed shot. I wanted to show more texture on the trees' trunks, hence, I boosted the over exposed data a bit. Of course I would have done that in masked layers, however, I decided to not apply any "special treatment" to the photo, in order to demonstrate pure HDR photography.

Personally, I am pretty pleased with the result, in particular since this is handheld HDR photography, which most folks say was impossible.