Is J.D. Power IQS a Bunch of BS?

J.D. Power’s Initial Quality Study (IQS) is closely watched by automakers, media and the car-buying public. But does this highly influential report deserve the attention it receives?

IQS tracks issues during the first three-months of new-vehicle ownership. It covers traditional mechanical faults such as an inoperative power window or a defroster that doesn’t work, but it also includes design-related defects, which are features or controls that don’t function as intended or are difficult to use.

And that last part muddies the water. Arguably, J.D. Power is combining two separate issues: outright failures, which can be tracked, and personal issues that are difficult to quantify.

Michael Karesh, developer of TrueDelta.com used an interesting analogy to describe IQS. What if a survey tracked how smart people are as well as how tall they are? “We’re going to have one score that measures someone’s height and intelligence,” he said, “And we’re only going to give you one number.” He said a person could be tall, smart or both, but there’s no way to determine which from a single figure.

Postulating why J.D. Power would lump seemingly disparate things into a single study Karesh said, “One reason for combining the scores could be to keep the problems-per-100 scores high enough that people feel they’re worth paying attention to.” They’ve managed to keep the average right around 100, which is one issue per car, he said. For 2015 the mean was 112, a 3 percent year-over-year improvement.

SEE ALSO: Everything You Wanted to Know About the 2015 J.D. Power Initial Quality Study

“[IQS was] originally meant to gauge production issues that happened at the assembly-plant level,” said Dave Sullivan, manager of product analysis at AutoPacific. But since everyone has adopted Toyota’s manufacturing processes over the last 30 years mechanical faults have largely disappeared.

Sullivan said IQS has almost become too subjective. “People aren’t finding things like, ‘Hey, my engine died,’ or ‘I have oil leaking,’ or ‘This screw is loose.’” Instead, issues with entertainment and connectivity systems have moved to the fore and trouble with these technologies are tough to measure. In fact, voice recognition and Bluetooth pairing are top issues this year. “Trying to find [mechanical] errors in the first three months of service is getting really, really difficult,” said Sullivan.

How can you tell if the troubles associated with pairing a handset aren’t an issue with the phone? Sullivan said in his experience “they all just work,” and as an industry analyst he’s in and out of new vehicles all the time.

One reason why there are more reported problems with in-car electronics is that they’re becoming hugely more popular. According to J.D. Power, 67 percent of owners in this installment of IQS said they have in-vehicle voice-recognition technology, which is up from 57 percent just two years ago.

Echoing these sentiments Karesh said, “A lot of people tell me they’re more concerned with how a car does out of warranty,” when it’s older than three years. He said many individuals are only really concerned with serious, expensive repairs, which are “extremely rare” during the first few years of ownership and are a miniscule part of J.D. Power’s IQS and even its Vehicle Dependability Study (VDS), which tracks things gone wrong with three-year-old vehicles.

J.D. Power is easily recognized by consumers and is widely respected for its awards but, “Every year I seem to take it [IQS] with a larger grain of salt,” said Sullivan. “I think it’s definitely lost its significance over the years,” partly because percentage improvements can only go so far. “And now we’re down to nit picking basically … subjective things,” he added.

One reason Sullivan feels this way is that all new vehicles are pretty darn good. If they didn’t include the so-called design-related defects he said, “I think they’re running out of things … The showing would be too close between automakers,”

Karesh noted that with his survey, “We’re very clear about what we’re measuring.” He tracks how many repair trips a vehicle had in which a problem was corrected, specifically targeting “things that failed.” TrueDelta aggregates quarterly data from around 27,000 members and roughly 30,000 vehicles.

On possible alternative to traditional surveys and studies is social media. Sullivan said, “I think a lot of people rely on their friends, neighbors, coworkers, Facebook … people talk about how reliable their vehicles are and that speaks volumes.” It’s hard to argue with a personal recommendation from someone you know and trust.

J.D. Power IQS may be less than perfect but it’s still a hugely influential publication, one that OEMs and car buyers look forward to each year. The 2015 installment of this study is the 29th they’ve published so the firm has a longstanding track record monitoring initial quality. This year their findings are based on responses provided by more than 84,000 purchasers and lessees of new 2015 model-year vehicles.

Check out our Tips and Advice Section for more articles like this one.