DSLRBodies

This question comes up a lot, particularly when people get paranoid over Internet posts about sample variation and lens faults. 

So first, a word from our sponsor, the makers of Valium: chillax, digital dude or dudette.

Yes, sample variation does exist. Yes lens faults do exist. But let's be realistic about those. The former tends to be common and is often not problematic; the latter are uncommon and tend to be obvious.

Thus, the first thing I'd advise you to do is just shoot with a lens. If there's something dramatically wrong, it's very likely to show up immediately as you examine your images. Things that would trouble me out of the box are easily seen inconsistencies of exposure and a clear variation of sharpness between the two sides (long axis) when one wouldn't be expected.

Oh, before we move on: filters off! As I've written before, filters—especially low cost ones—have a strong tendency to reduce contrast, increase flare, damage acuity, and more. If you're evaluating a lens with a protective filter over it, you're crazy. Because you're not evaluating the lens, you're evaluating the lens/filter combination, and that filter will almost certainly turn out to be an issue.

Update: image stabilization should be off, too. Especially as you get into hand-held testing—but even on tripods as IS isn't perfect at recognizing "stable"—you don't want a lens element or sensor to be moving. In some lens designs with IS, the IS element isn't at the optical center of the lens and may move off center, meaning that at the edges you're getting closer to the edge of the image circle when you look at results. Likewise on-sensor IS, which may move the sensor so that it's looking at edge of image circle on one edge but not the other.

Notice in my first pass, above, I wouldn't be troubled by focus issues. We need to deal with those separately, particularly on DSLRs. We'll get to that in a moment. 

But first, just shoot as you'd normally shoot with the lens. If you aren't noticing issues, it's highly likely there aren't any. 

You didn't do that, did you? Instead, you took a shot of a flat object or test chart and you pixel peeped. And freaked out. 

Okay, at this early stage in your evaluation you should have noticed a few things. For example, how smooth were the zoom and focus rings? How noisy? Any free play or any restriction in turning is something I'd be immediately suspicious of, though note that there are lenses that have clear design issues here (the Nikkor 200-500mm f/5.6 zoom ring, for instance). Excessive noise, particularly squeaking, is not a good sign, though again there are certain Nikkors that are known for that when they haven't been used in a while). 

Did you notice if all the switches worked? Did you notice any scratches on the front or rear element? If it's a used lens, did you look through it to see if you could see any etching on the lens elements, a sure sign of fungus? 

Okay, we're up to our first real "test": focus. On DSLRs—or any other camera that relies purely on phase detect focus, which does include a few mirrorless cameras in certain focus modes—we really want to take out any mismatched tolerances via AF Fine Tuning. 

AF Fine Tuning is an article all in itself, and it's a bit of an art form to get dialed in right. That's because phase detect focus rarely repeats itself exactly. Usually there will be a small range that the camera/lens settles into for any given target/distance/focal length. It's why I tell people to test 10 times and take the median of their tests (not the average). There are other gotchas, as well: you can't test close up indoors in artificial light and expect that to be right for long distance outdoor work. Both distance and light color temperature are factors you have to account for in AF Fine Tuning (which is why it's another whole article in itself).

But assuming that you know how to AF Fine Tune, do that, and start shooting, we're ready for another set of evaluations. Use the central focus sensors and shoot some subjects.

Is the focus consistent? Is the acuity high? How close is the focus in normal shooting to that of Live View (where contrast detect is used on the DSLRs)? Note that I did not write "is the focus right?" Because if it's not right, it's much more likely that you didn't correctly AF Fine Tune the camera/lens than that there's a lens fault.

Okay. Focus is consistent, and you're getting high acuity in the central area. 

Here's where we start to lose quite a few people to their paranoia: they suddenly test the lens at near its close focus distance on a flat subject (test chart, newspapers on the wall, etc.). 

Holy moly is that filled with problems. First, this is almost always done indoors under artificial light. Second, it's done with lenses that weren't designed to capture flat fields. Third, it's done at closer focus ranges than you'd normally used the lens. Fourth, virtually no one correctly aligns the sensor to be parallel to the subject (test chart). 

Thus, it's almost certain that if you run the usual test chart test you'll find "problems" with your lens. 

Here's a better and easier (though still not perfect) choice: go outside where you have a clear view to the horizon and take a picture of said horizon slanted across diagonally (upper left corner to lower right corner, lower left corner to upper right corner). Focus in the center. Run that test at every aperture.

A great lens is going to reveal excellent acuity along the entire horizon, and it's going have equal acuity in each corner. If it's a telephoto lens and has a rotating tripod collar, rotate the lens 180° and shoot the same test. A lens that's decentered or not aligned properly internally will typically have any focus issue in the corner flip to the opposite corner when you do so. 

If you don't have a telephoto lens with a rotating collar, you can try Roger Cicala's method of using a slightly out of focus star chart. But a warning: you're going to do this at close distance on a flat surface, so all those problems I mentioned above come into play, too. Also note Roger's comment about consumer grade superzooms. 

Next, put your camera on a tripod and find something that is at a longish shooting distance (but not at anything near infinity as you just did with the diagonal) that has a lot of detail/structure. Center that and focus on the center. Now adjust the ballhead so that what was in the center is now at one edge and reshoot. Then rotate so that the detail is now at the opposite edge. Note, lenses with field curvature will show clear issues with focus with this test. But the two edge tests should be the same.

What do I look for in a lens?

• Mechanically sound. No ring issues, no switch issues, no problems mounting it, no obvious physical damage.
• Exposes repeatably. This is particularly something to watch for with lenses on Nikon DSLRs that use the mechanical aperture activation arm. At 12 fps, are the exposures all the same? At different aperture/shutter speed combos?
• Focuses repeatably. After AF Fine Tuning I want a lens to focus reliably where I tell it to. And I want that near 100% repeatable. I'd like little to no difference between live view and regular DSLR focus.
• Not decentered. Am I seeing edge performance issues that differ side-to-side?

Assuming those four things, I take another look at the third bullet. In particular, is this lens giving me higher acuity than my current lenses at that focal length? If the answer is no, I'm probably not going to use that lens. 

Some exceptions to the above exist. For instance, nothing beats my 200mm f/2 and 400mm f/2.8 Nikkors at those focal lengths. Nothing. Indeed, a lot of lenses don't even come close to what the exotics do, which is pretty much what you should expect from the price differential. 

So does that automatically mean I reject, say, the 80-400mm f/4-5.6? No. Every sample I've seen—and I've used many—is clearly far weaker at 400mm f/5.6 than my fast 400mm lens is, but the 80-400mm is also one-sixth the price and can be easily hand-held. Heck, it can make it into a kit that has to fit into some of the worst airline carry-on weights. So yes, the 80-400mm at 400mm is tolerable for many situations. 

Note that I haven't written about:

• Linear distortion — that can be corrected in camera and in post, though at the expense of very precise framing.
• Chromatic aberration — lateral CA can be corrected in camera and in post, with little penalty.
• Vignetting — again, something that can be corrected in camera or in post, though lenses with large amounts of vignetting can produce visible noise components in the corners when you do.

Nor have I written about "dust in the lens." First, a lot of the dust in the lens reports are incorrect, as we had with the 70-200mm f/2.8 VR II when it came out. You're not seeing what you think you're seeing. Second, a bit of dust internally doesn't bother me. A lot tells me that the lens needs to be cleaned by someone who knows what they're doing, as it can start to have veiling flare and contrast impacts.