DSLRBodies

Do you see a photograph or do you see pixels? 

Unfortunately, in digital photography a lot of people see pixels, not photographs. And this isn't a new phenomenon, we've encountered and fought over the "measuring syndrome" before, most notably in audio. Funny thing is, we could have had the same heated and highly visible discussions about film that we have today about sensors, but there just weren't enough user-accessible tools (nor the Web forums) to enable them. But I'm getting ahead of myself. Let's run the Wayback Machine and see where we land, shall we? (Eek, I've already had a Yogi Berra and Rocky & Bullwinkle reference and I haven't even gotten two paragraphs in; prepare for Eclectic Storms ahead. ;~)

When I was in High School, the High Fidelity arguments were sounding louder than a Hendrix solo. Audio went through a number of format changes over the years (45>33 1/3, AM>FM, Mono>Stereo, etc.), but in the late 50's and throughout the 60's technology changes came fast enough that each new iteration of equipment came with claims of "higher fidelity," "more power," "less noise," and a host of other things that ought to sound vaguely familiar to anyone following digital photography discussions. 

Moreover, the audio press at the time (mostly magazines, but a few fanzines as well) went through several iterations that should sound famliar: first "I know better sound when I hear it," then "When I measured X it tested better than Y," and finally "But measurements don't tell the whole story, Y sounds better." Eventually, the audio press spun into two distinct groups, the mouse marketeers, excuse me, mass marketers (ironically, High Fidelity magazine ended up a good example of these) and the audiophiles (Stereophile, amongst others). While stereo sales boomed as people replaced old technologies with new, the mass marketers also boomed. When most every living room ended up with an "adequate" system and the buying boom ended, only the audiophile group continued to thrive, albeit as a small niche.

But that thriving was just as contentious as before. When the CD came to market in the early 1980's, the measurement versus listening arguments were already in full force, and the crude initial digital sampling mechanisms on those first CD players just accelerated these discussions. Once again we had a boom market and the CD makers rapidly iterated technology seeking both real and marketing advantages over their competitors. Oversampling, ADC linearity, and a host of other innovations came one after another. And with each new generation, we got the same arguments: "it measures better than previous generations" versus "it sounds better/worse/same than previous generations." Plus let's not forget the famous "11 is better than 10" arguments.

One of the outgrowths of the audio "fidelity" debates was the notion of the "golden ear." (In photogrpahy we'd speak of a golden eye.) First, it ought to be obvious that some people have better hearing than others, both in the physical mechanism involved and in the interpretation of what they're hearing. Some ear canals are plugged with wax, some inner ears have critical hairs that have been destroyed, ear drums can be broken, and many more things impede the physical mechanism of hearing. On the interpretation side, musicians are trained to hear subtle differences and can therefore often describe exactly what they heard while someone that might never have been even exposed to a genre or technique might only be able to say "I like it" or "I don't like it" in reference to the sound. As Oliver Sacks points out in a recent book, Musicophilia (affiliate link), the brain also enters into the discussion, as it sometimes gets in the way, sometimes intercedes, and sometimes augments the musical experience.

So the contentious question in the audio field has been—and still remains—how do you review a component that produces music? Do you use measurements, a golden-eared reviewer, both, or something else? Can you reliably blind AB test differences (e.g. play examples to listeners randomly and have them accurately differentiate between A and B)? And the most famous one: if you can't explain the technical impact an apparently silly change might have but can hear, is there really a difference (e.g. mark the outer rim of a CD with a black or green marker, place a quarter on the stereo, use non-soldered connections versus soldered on wires, etc.)? Indeed, Heisenberg and Schroedinger would have a field day with audio reviewers ("If a tree falls in the forest and you hear it but can't find it did it fall?"). 

Which brings us to photography.

We've actually had the same problem as audio has, even back in the film days. Essentially all we do is switch sensing systems from ear to eye and sound waves to light waves and all the same crazy variables come to play. (Sometimes a wave is just a wave.)

No film was perfect at reproducing the visual spectrum. Indeed, some of the most favorite films were very imperfect. My favorite slide film, for example, was Fujifilm Provia F, not Velvia. Velvia, has two strong tendencies that Provia F did not. First, Velvia drops very rapidly to black in the shadow realm ("no toe" in the parlance of film designers), restricting its captured dynamic range, but making for a very clean and rich black in the deep shadows (as opposed to preserving shadow detail, which Provia F tends to do with it's more normal toe). Second, Velvia has a strong magenta component in sky colors (Provia F does too, but not nearly as strong). On the other hand, Provia F doesn't quite saturate colors quite as much as Velvia. So, is Velvia better than Provia F or vice versa? Good question, what shall we measure to find out?

And there you have the rub: "what shall we measure?"

Some people religiously believe "noise measurements" published by some Web sites. Indeed, I've been known to publish such numbers from time to time. But do you know what those numbers represent, and how they apply to what your ultimate photography might look like? It used to be that the only published numbers were middle gray patch average standard deviations. Only one problem with many of those measurements: the middle gray wasn't standardized to a common value. Moreover, the default contrast controls of some cameras, Nikons in particular, varies with subject, meaning that gray isn't necessarily grey. Or gray. Or grae ;~). And gray performance tells you nothing about how the groom and the bride are going to look, do they? 

It doesn't help that two identical numbers produced by a noise test can look different.

Which brings us to another thing I've noticed over the years: just how good are the eyes of the person making the judgment you trust (including your own)? That's actually one of the reasons that some people want to see numbers. After all, numbers don't lie and are reliable and repeatable, right? Sure, but what do they mean? 

For example, let's say that Test Site A reports that Camera X has an ISO 1600 noise value of 3 and Camera Y has an ISO value of 4. Why, Camera X must be 25% better than Camera Y, right? That would be 25% better at what, though? And were the detail levels held constant to report those numbers? Was that test performed at the "optimal settings" of the camera or default? 

The numbers game gets complex very fast. To get a strong handle on just noise, for example, you need to measure noise:

• at every stop in the recorded range for a given ISO.
• at all sharpness, contrast, and noise reduction settings.
• In different colors of light (incandescent light has low blue components, high shade low red).
• in both luminance and color (chroma) components.
• to find out whether it is patterned or random.

And that's just for JPEG. Things get more complicated when we get to raw files and additional options. Plus, even if we have all those numbers, just how do you anoint one camera "better" than the other? What if Camera A has more noise than Camera B in the shadows, but less in the highlights? And Camera A has more noise in incandescent light but Camera B has more in shade? How about a camera with Patterned Luminance noise versus one with Random Chroma noise? 

Guess what, either you have to wade through tons of data sets and come to your own conclusions or you need to trust someone else to make a judgement for you. What's a decent amount of noise and what's an obscene amount? We're basically back to the Golden Eye (cue Bond theme music). Or as Supreme Court Justice Potter Stewart wrote about another type of obscenity: "I know it when I see it." (Note: he later recanted that thought in Miller v. California, saying that viewpoint was untenable.) Do you know it when you see it? As good as my eyes are, I'm not sure I always do.

Yes, dear readers, I still do lots of numeric tests on new cameras, and yes I use my Golden Eye (actually more of a light brown) to make judgment calls on image quality. You'll get my best efforts in any camera review I might post. But I've also been writing the same statement now for almost a decade and it's getting more and more difficult to refute: "if you're not getting great shots at the maximum size any desktop inkjet printer can produce (13x19") from any of the current crop of DSLRs, it's not the camera." 

It's not the pixels, it's the photograph that counts. 

Sure, find out what's happening at the pixel level of your favorite camera and learn how to optimize that, but don't obsess on it. We can already shoot with better quality in lower light than we could in the days of 35mm slide film. That doesn't make the pictures I see any better. Moreover, I don't recall at any image review session at any workshop I've taught where I've said "you know, too bad you were shooting with a D86, otherwise you wouldn't have these lousy pixels to work with."

So relax. I'm sure the pixels on the latest Nikons are better than those on earlier Nikon bodies, but that's actually not the reason why I enjoy new cameras. Everyone together now: it's not the pixels. Sing it with me:

Sung to "It's Not Me" (3 Doors Down)

Never mind the lens you put on
In front of me
Never mind the quality
that you set on your D3

Cause every little thing you see
And every little thing you capture
Makes me doubt all of this

(chorus)
Look what you did
Is this the photo you wanted? Well...
It's not the pixels