The Value of User Data

(commentary)

First, head on over to the Reikan blog and look at their initial post on user data samples for focus. In particular I want you to look at the D800 + 24-70mm or 24-120mm and the Canon 5DIII + 24-105mm results. The reason for picking those samples is that there are a lot of data points already, so the data is more likely indicative of actual camera/lens populations.

reikan sample.jpeg

The D800 + 24-70mm chart is especially disturbing (part shown above). While we have a bell curve centered around zero for a lot of the tested cameras, we have a second bell curve at about -17 AF Fine Tune and a lot of outliers in the + AF Fine Tune realm, too. The D800 + 24-120mm is better, but the bell curve is still quite spread out (as Reikan notes, the "variability is significant"). 

Now look at the 345 tests (even more than the D800 tests) with the Canon 5DIII and 24-105mm: a near perfect bell curve almost at the center of the Fine Tune area. That's what we want to see: low variability and a bell-shaped curve. 

I've written several times in the past year that Nikon's quality control has slipped. This simple data set is an example of how that shows up in a hidden, but tangible way. When I tested a large number of D800 bodies early in the "left-sensor is wrong" crisis, I also noted that I was seeing an AF Fine Tune curve that wasn't bell-like in shape. I found cameras that were from -15 to +20 with the same lens, and that was with the central sensor, which generally wasn't the problem on those cameras (in the problem cameras, the left sensors were not "fixable" with AF Fine Tune). 

Data like my user surveys and the posting of multiple test results like Reikan's are starting to get to levels where we can closely predict from them. For example, Reikan's numbers for D800 bodies with the 24-70mm have a 95% confidence level an a confidence interval of 13.33. That's almost enough to make a serious statement about the entire population. The confidence interval on my left-sensor survey was 95%, so I was able to feel comfortable about my statements about early D800 models. The plus/minus interval was less than 5%, which is why I wrote that between 20 and 30% of the cameras shipped at that time had the problem.

One issue from the very beginning has been the "I've got no problem" folks. Those that have a perfectly functioning D800 have been quick to jump on the claims of those that are having problems with theirs. I'll remind everyone that absence of a problem on your sample doesn't mean that there wasn't a problem on other samples. Moreover, I happen to have two D800's that perform pretty much as expected; I didn't dismiss those claims by others: I investigated those claims with testing. It's only through aggregation of data that you can tell whether there are infrequent problems, common problems, or no problems at all. D800 focus issues appear to be a common problem, as the Reikan data suggests. 

Nikon can hide behind their wall of silence if they'd like, but real user data will out them in the end. The Internet has a lot of garbage to wade through, but it's also very good at aggregating real information when handled correctly. I call upon Nikon to make a definitive statement about the focus issues some users have been having, and what they're doing to make sure that this doesn't continue to be a problem into the future.

text and images © 2017 Thom Hogan
portions Copyright 1999-2016 Thom Hogan-- All Rights Reserved
Follow us on Twitter: @bythom, hashtags #bythom, #dslrbodies