DSLRBodies

(commentary)

I’m about to pop out a bunch of new lens reviews. They might not always seem to fully agree with what you’ve read on other sites. So it’s time to restate a few things. But first, if you haven’t read my previous article about lens reviews, please see About Lens Reviews on this site.

Some updated reminders:

• Reviews solely dependent upon flat field testing (and generally at close distances) don't tell you everything you need to know about a lens. While it’s wonderful to assign numbers to traits, doing so the way most sites do it tells you very little. Often, it provides you misleading information. You may recall that my very tardy review of the 200-400mm said something that none of the other reviews up to that point had: that lens is very weak at distance. All the testing others did was on test charts at short distances in labs/studios. One number—heck, not even the full set of numbers I see anyone else publishing—tells you the full character of a lens. Curiously, Nikon just released a video in which one of the quality control engineers mentioned this: “However simply having the same MTF values does not mean that the level of performance will be the same. We often found ourselves amazed at [the 24-70mm’s] excellent imaging characteristics despite the fact that its measured values were quite low."
• Lens makers design differently. If you haven’t read the article with Nikkor lens designers that Nikon recently posted, do so. The key statement there is "I wasn’t satisfied with the appraisal methods for camera lenses I studied in school. The traditional appraisal method for the most generally used photographic lenses begins and ends with the evaluation of the plane that matches the point of focus on the subject and the plane that appears in the picture. At Nikon we've come up with our own method of appraisal.” Zeiss designs to a different standard, Sigma designs to yet another standard, but very special to note, Nikon thinks that they’re designing to a completely different standard than anyone else, one that doesn’t have an academic history to it. 
• I shoot test charts. I shoot with lenses in the field. I have my own testing regime, and it takes time to run a lens through it, especially when I see something unexpected. But I also just like to see how a lens performs in actual use, as that’s how you and I will be using it ;~). Just like Nikon has its own standard of appraisal, so do I. If a lens works for what it is intended, I don’t care how it tests on charts. And vice versa: if a lens tests really well on charts with something like Imatest, I don’t care about that if in actual use the lens sucks wind. I stopped publishing the test chart shots and numeric results a long time ago, mainly because I don’t believe they tell you nearly enough about how good a lens is, and can give many a false sense of security. Moreover, I don’t generally show much in the way of JPEGs to demonstrate quality. Why? Because not only does my Web site have a re-imaging engine that re-interprets JPEGs based upon presentation size, so do a lot of the things you use to read my site with. You’re almost never looking at what I actually see in raw files. Remember, JPEG is a compression system throwing away information. Resizing and recompressing is just going to throw away more information, and often antialiases that information, as well.  
• Then there’s this [puts helmet on, buckles up bulletproof vest]: a lot of reviews put a great deal of trust on initial impressions. Within hours to days of getting the lens some reviewers are proclaiming it a winner or loser. One sample lens, one quick test session: bingo, reviewed! The reason for that tends to have to do with how affiliate links work. If you’re optimizing to get the highest income from a photography Web site, the earlier you can post an announcement and the earlier you can post a review, the more likely you get clicks that make you money. [disclosure: this site has a few somewhat buried Amazon affiliate links, but only uses paid advertising on individual announcement or review pages, currently exclusively from B&H]. Certainly all of us using new gear have snap opinions about it as we first use it. But such quick opinions are never nuanced, and almost never capture the full “flavor” of the product. 
• Lenses vary. If you test multiple samples of a lens you get multiple results. Indeed, it’s rare to get exactly the same results. The lens makers try to get all the product they make into a narrow margin of tolerance, but the operative word is “tolerance.” Stradivarius made a number of violins: they did not all sound the same. Nikon and others make a lot of lenses, they do not all perform the same. Nikon’s tolerances seem reasonably tight—I’ve found less sample variation in them and a smaller variation at that—though some third party lens makers have a reputation for much more variable performance between samples. 

So what does all this mean in practice? I was chatting with Roger Cicala at Lens Rentals about this, and he sees something similar to what I just related: that different standard Nikon is designing to must have a lot to do with astigmatisms, which chart test products like Imatest don’t really measure well, if at all. 

One of the net results to Nikon’s current design philosophy is that their lenses don’t seem to score quite as well as competitors’ lenses on test charts. Yet, in the field in actual use, those Nikkors often look as good if not better. Moreover, there are subtle things that come into play.

Clearly Nikon has chosen to do something that makes the focus-to-out-of-focus transition very smooth and natural looking. Some earlier lenses—I’m thinking in particular of the 70-300mm—had what I’d call “rough” near-out-of-focus areas. Indeed, with VR turned on, not only would a near-out-of-focus area look rough, it would tend to have what I ended up calling echo artifacts. I just don’t see that problem in current Nikon telephoto designs. 

The problem that I have with Nikon’s “new lens design philosophy” is that its description is currently just a bunch of Zen-like marketing mumbo-jumbo. There’s no meat on the bones they’ve given us, so there’s no way to assess whether they’re doing as good as they could possibly do with this new philosophy, or still have room to get better. Let alone what they’re doing and how they’re doing it. There’s nothing we can actually test, since there are no specifics. 

That’s made me put an even-higher-than-usual importance on studying actual field results with the lenses. What I tend to see is this:

• Better edge to edge acuity than the test charts suggest
• A very subtle and natural gradation from focus plane to out-of-focus areas
• Little additional artifacts in out-of-focus areas
• With primes, a bit lower longitudinal chromatic aberration than before
• In corners, less coma and artifacts caused by astigmatisms

I also see that Nikon isn’t as concerned about linear distortion and vignetting as they used to be. Considerably less concerned. That’s likely because they are correcting that in cameras and post processing now.

Now, that wasn't a perfect list I just presented, and I don’t see all of those things on every new Nikkor coming through my studio. But it sure seems like a trend. Moreover, the combination of these things tend to lead me to saying that new Nikkors tend to be “very well behaved in balancing all variables that are critical to a clean set of pixel data with high acuity and contrast, and less well behaved in variables that can easily be post processed out" (linearity and vignetting, for example).

Obviously, I’ll have individual comments about individual lenses. Still, since I’m kicking off a series of Nikkor reviews this week, I thought we should revisit the “About Lens Reviews” thing and give you these updates to my thinking.