Criticism and the Armchair Fallacy

Last updated 4/10/15.

One often hears the claim that criticism is easy and cheap. Steve Jobs once attacked a critic by saying "What have you done that’s so great? Do you create anything, or just criticize others [sic] work and belittle their motivations?" Yet Jobs was no stranger to dishing out his own criticism. The New Yorker revealed that "Jobs’s taste for merciless criticism was notorious."

Similarly, the creator of Speak Up and Brand New, Armin Vit, had this bit of anti-intellectualism to share:

I have very little patience for criticism these days. Either do the work or shut up. Critique with action, not words. Words are so twentieth century. There may be a few people that still enjoy reading that kind of stuff, but I think for the most part design criticism (at least in graphic design) is dead, and that’s not saying much as it never really lived much.

This view espoused by Jobs, Vit and those like them can be understood by what I call the 'armchair fallacy.' Those engaging in the armchair fallacy make a false distinction between theory and practice as forms of argumentation. For them, the only valid way to critique something is to create something different and better, and it must be a fully baked rebuttal in the form of a competing product (redesigns, of course, are not permitted. Those who invoke the armchair fallacy posit that actions speak louder than words. Yet this is an incredibly limited understanding of criticism.

Public criticism is just as much action as the creation of products is. The creation of products may speak loudly. But words that prompt widespread action literally speak louder. In any case, for a product to truly resonate, it must be written or spoken about, recorded or transmitted in some fashion.

When we combine the camp of detractors who would have critics make instead of write with the camp of those like Mike Monteiro who argue that redesigns are forbidden, there is no room for criticism at all. Which is, it would seem, the goal here for both groups.

Engaging in criticism is hardly a matter to take lightly—it comes with major risks and takes significant time and effort to produce. But criticism is essential because it forms the bedrock that sustains quality and craft in practitioners. Any community that embraces rigorously examined judgment and critical thought is a flourishing and self-aware one. Further, it is profoundly anti-intellectual to proclaim that theory and studied efforts at understanding what makes good design are unimportant. This is tantamount to admitting that one goes about releasing user-facing products without a second thought about their impact.

Given its importance, I am proposing a preliminary framework for how we as a community might both engage in and engage with criticism. I am open to suggestions for additions.

Sources of Criticism

Anyone can provide criticism. Individual feedback by a layperson speaks to the variety of people who may engage with a design.
Each time a user expresses 'I don't like this' that is a data point. 'I don't like this' is the first step towards getting something more actionable from a user, eg. 'I don't like this because.' It is up to the creator to ask 'why?'
It follows that mass feedback, taken in aggregate, is a weighty form of criticism to be taken seriously.
Still, it helps to have some domain knowledge. While everyone is capable of criticism, an excellent critic brings something more to the table, whether through historical, philosophical or technical analysis. These elements serve to enhance and contextualize criticism.
For unsurprising, though unfortunate reasons, criticism by peers, colleagues and journalists is rarely aired publicly, and when it is offered, it is done so reluctantly. Given this, the absence of criticism is not in the least-bit evidence of a well-designed product.
The more context one has as to the nature of the process that led to a product, the better one can determine a point of failure.
However, to engage in criticism one need not have worked at a company and know the internal operations that resulted in a particular design. In fact, if criticism were only permitted by those with access to protected internal information, it would very rarely occur.
Alberto Brandolini's Bullshit Asymmetry Principle states that "the amount of energy necessary to refute bullshit is an order of magnitude bigger than that necessary to produce it." Criticism is nonetheless a worthy goal.

Quality of Critique

Criticism may be in the form of a verbal or written interpretation, but it can also be in the form of a redesign or even a competing product.
Innovative products take a critical stance on their competitors. They say 'we can make this better than you.' Redesigns publicly request better from creator.
For criticism to have weight, it does not need to provide an answer or solution. It is valid to simply point out a glaring, or even a minor, flaw in a design.
Both visceral responses and thorough examination are encouraged, with the latter generally being more convincing.
Apologia is not criticism, but it is nonetheless another useful tool of examination. However, there is a difference between unwavering fanaticism and intellectually honest argumentation. An honest apologist will admit to the flaws of their position, while a fanatic will not.
Criticism is more readily accepted when deferential, but it need not be polite in order to be valid.
Direct negation is not the only type of criticism. Advocacy through creation, curation and the promotion of designs are often forms of criticism. In the praising of a preferred method, product or idea one can implicitly critique its opposition.
However, in the face of overwhelming consensus, these forms of critical praise are largely insufficient to make a credible case against a movement.
In such cases, one may use negative criticism and snark in order to point out flaws and contradictions. In this way it is constructive to give negative criticism, because in doing so one creates an opening for an alternative.
As unfortunate as it is, in the age of social media readers and the critiqued are increasingly sensitive, even when faced with thorough and measured criticism.
When readers say "You're not wrong, you're just an asshole" they entirely miss the point. In making such a statement, they only admit that they cannot handle the truth. Instead, they decide to invoke the Tu Quoque fallacy, with the intent of reversing the focus onto the critic. In such an environment where critical assessments of imperfection (let alone findings of project mishandling) are met with accusations of being an “asshole” it is not the critic who should be examined, but the community unwilling to look honestly at itself.
Calling out a practitioner's work or a theorist's writing is not meant to disparage them personally. It is simply meant to discourage the continuation of a harmful practice or ideology they are engaged in.
Apologists sometimes accuse critics of engaging in the fallacy of cherry picking quotes. In these cases, 'You took that out of context' very often means 'I wish you hadn't exposed what that person said and meant.'
Hanlon's razor states that one should "Never attribute to malice that which is adequately explained by stupidity." On the other hand, one must consider the Sykes Corollary, "any sufficiently advanced form of ignorance is indistinguishable from malice." Furthermore, Rao's Corollary holds that "Sufficiently enlightened self-interest is indistinguishable from altruism. Sufficiently clueless altruism is indistinguishable from self-interest." Thus it is crucial to expose toxic designs and design philosophies. We must do this regardless of their origin, whether born of malicious intent, self-interest or mere ignorance.

The middle ground fallacy does not get you closer to truth, it just leads to compromised mediocrity.

Hedging One's Criticism

A position of neutrality in a debate necessarily means one accepts the status quo. As it has been said, "You can't stand still on a moving train."
To criticize a work, one may cede ground and attempt to embrace the constraints of the original project, but this may lead to conclusions that lack rigor or are unimaginative because they carry the flawed assumptions of the original design.
Similarly, truth is not found in compromise. It is important not to fall for the middle ground fallacy. There may be a third option apart from two opposing sides that takes the best of both prior options. But any compromise is simply a watering-down of the available solutions.
It is also not useful to argue from the nihilistic perspective of artistic or moral relativism. An example might be if one argued that because a design is better than its competitors, or its predecessor, that it must be a good design.
A work's profitability, or lack thereof, is not the only nor is it the best measure of its value to its audience.
The fact that a design or method is universally adopted does not make it the optimal solution. Design practitioners consistently bandwagon as part of a cargo cult of trend-followers. They also often take the easy way out by getting caught in the Abilene Paradox, in which everyone forms a consensus so as to not look like the odd one out. Add to that the common situation of being forced into a set of practices because of lock-in with a particular set of technologies or with a company one might not want to publicly disagree with. In any of these cases, that a practice is popular does not make it right.
Short deadlines and client misdirection are not a sort of alchemy that makes bad design good. If leadership does not make strategic decisions for ideological reasons or otherwise, that is something that can and should be discussed by critics. But any designs in question may nonetheless be criticized for their qualities.
Appeals to features are fallacious. Features do not make design, design makes features. Software design must be judged on the design implementation of functionality, not on the mere existence of functionality.
Appeals to bugs are similarly not to be considered. Attempting to discredit criticism by claiming that a design philosophy's results are merely bugs is effectively moving the goalposts in order to excuse designs patterns that would otherwise be considered unacceptable.
Appealing to user evolution assumes that users have grown in sophistication, ability and skill with computers. Very often it is not the user who has done this, but the designers themselves. It is wise to be skeptical of a designer who claims that 'all of a sudden paradigms have changed.' It is most likely they cannot see past their own narrow design mental model.
Appeals to empty and vague virtues are just that, empty.

The Critiqued

That positions are held by big companies and smart, prominent or wealthy practitioners does not make them immune from criticism.
Appeals to omniscience involve claiming that big corporations and their employees 'know better' than the rest of us. That they are already on the case mending any faulty design they have allowed in software. Unfortunately intelligence hardly guarantees that these employees and their managers have prioritized fixing their designs or analyzing their design languages critically enough to notice said flaws. Very often they are quite convinced the design issues are not flaws, but are instead pushing the design world forward.
Criticism of a piece of work is usually a sign that it has made a wide impact. At the very least it means one's work holds weight for the critic, whether or not they happen to like it. It is a therefore good publicity to receive criticism in the sense that 'any publicity is good publicity.' The underlying benefit of having a valuable brand is that people notice, care about and use one's products. In the end, very often even the critic pays up and buys the product.
For an individual, company or community to be healthy and spur its intellectual growth, it must not censor criticism, but encourage it.