Recently we decided that a major product test that we had undertaken could not be published.
It was with great reluctance that we took the decision. AutoSpeed contributor Graham Pring spent many hours testing the product, a process which was extraordinarily intensive and prolonged. We even involved the local distributor of the product in the testing process (one major test was even carried out on the rep's own car!); we had a formal meeting with a technical representative of the manufacturer; and pages and pages of correspondence with the manufacturer were entered into.
In addition, we talked to as many other people who had fitted the product as we could find (including some who loved it), and even totally re-evaluated our testing procedure after the first series of tests, in case our results were artefacts of the test methodology. We read the SAE engineering paper prepared on the product by the manufacturer (though not, it appears, published), looked at specifications of competing products, and talked to others professionally skilled in the testing of similar devices. We also followed to the letter the instructions provided with the product - ironically, something that we later discovered many people who bought the thing didn't bother doing!
We then edited our material until it could be fitted into a short series of articles. They were incredibly detailed (amongst other data, we were going to make downloadable in spreadsheet form the hundreds of measurements that had been made), and included road and dyno testing on two quite different cars.
Our testing complete, we believed that there were very major concerns with the product. The problems seemed so great that at this stage we contacted the company for a response, and - as a courtesy - showed them draft versions of our stories. We were saddened when instead of rebutting the detailed findings that had been made - ie having a logical debate with us about the product and our testing of it - they responded by making scarcely veiled threats of legal action to be taken should the articles be published.
I have been involved in many magazine product tests over the years and I can say with certainty that few readers have any idea of the legal and other pressures that are applied by companies whose products are analysed unfavourably.
I remember - when a freelance contributor to a number of magazines - testing a chip. The chip company immediately had their lawyer try to slap an injunction on the magazine that was taking the story. In addition, the chip company had their dealers - all well known car modification workshops - ring the editor of the magazine complaining of the poor quality of other stories that I had written. It was a quite deliberate attempt to harm me financially - to a freelancer, cutting off a magazine might cost him or her half of their income. The editor - awake to the sleaze that was going on - brushed them off. Also as a result of that episode, a meeting of all of the editors in that quite major publishing house (it was a company producing dozens of magazines) was held. The editors were told to be very careful in running stories that were negative of products... The company was scared - and it was an emotion that time has shown it's sound to have.
I remember, this time as the editor of a magazine, receiving a phone call from a major muffler company's legal secretary making it absolutely clear that the multinational would ruin the company for which I worked unless we changed a muffler comparison story. My publisher - the owner of a company with a turnover of perhaps just a few million dollars a year - had absolutely no choice but to ask me to pull that aspect of the story. A similar thing happened when I tested - with atrocious results - a particular cat converter. The only thing that we could do that time was to remove the converter entirely from the comparison.
Statements like "any magazine can publish fair reviews" and "if you tell the truth you'll be alright" are simply naïve and simplistic. I was staggered by the gall of one editor who recently wrote that there were no problems at all in publishing negative product reviews - that, from a man who works for a major publishing company known to sue others at the drop of a hat! I mean, nothing like having a host of experienced company lawyers to draw upon, is there?
After it became clear that we were going to have legal difficulties if we published our product analysis, AutoSpeed looked very, very carefully at the Trade Practices Legislation (our attention had been thoughtfully drawn to it in correspondence from the company that manufactured the product being reviewed...), talked to the Australian Competition and Consumer Commission, and sought and received legal advice from other sources.
All suggested that it would unwise for a small company like AutoSpeed to publish the stories, irrespective of the evidence available to us through our testing.
You see, our legal system allows anyone to initiate legal action against anyone else. That seems fair enough, but when it is a large company taking on an individual or a small company, the bully's lawyers can easily make such legal action extremely protracted. And of course, enormous legal bills continue to accrue during the whole of that extended procedure. So even if you're the party that is finally determined to be in the right, it is quite likely that by the time it's all over, you'll be too bankrupt to buy the champagne....
Many readers fail to understand that companies that feel threatened do not respond with a smile - they hit back in a manner commensurate with having a fighting fund of perhaps $1 million to defend a product. And of course, the poorer the product and the larger the investment made in it, the more likely they are to take very expensive and prolonged legal action against those who might criticise it - especially if the criticisms are valid. I used to believe that companies would be appreciative of having product inadequacies pointed out... but not a bit of it.
Some people may believe that the product test that we cannot bring to you was carried out in a slipshod manner - so full of untruths and distortions that the reasons it was pulled are based on points other than those I have outlined here. Let's make things quite clear - it was one of the most detailed and comprehensive test procedures that I have ever witnessed a magazine undertake on a product, and all the test results were assessed only against the claims made by the manufacturer. Graham Pring is a man of utmost credibility and honesty, a person who had absolutely no vested interest in the results. A qualified mechanic in addition to being technically skilled in a very wide variety of fields, he re-doubled his testing efforts whenever I or anyone else asked for further justification of his conclusions.
Product testing is one area where it's really, really easy to be a sidelines expert. Like the guy from the car club who told me all about his plans for a comparison of programmable ECUs. He seemed pretty genuine and I spent some time discussing with him the legal implications of his proposed test. I could see that he simply didn't believe me (if the tests were fair, what was the problem?), but I think by now he's probably a little more aware of them. The club magazine's editor didn't want to publish the story for fear of legal action... and that's just in a club magazine.
Imagine how long a court battle could be that was seeking damages against the author and publisher of a story with a potential worldwide audience....