Research & Ethnography
What is unspoken
A nested classificatory hierarchy
Unfinished
Record them all
Recommendations for field notes
In Defense of Browsing
The illustrated guide to a Ph.D.
When users never use the features they asked for
An Article by Austin Z. HenleyWe deployed our tool. Almost no one used it.
The handful that did use it, used it once or twice and barely interacted with it. After a few days, zero people were using it.
Why did they tell me they wanted these features?
Why Most Published Research Findings Are False
A Research Paper by John P.A. IoannidisThere is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance.
How can we develop transformative tools for thought?
A Research Paper by Andy Matuschak & Michael NielsenConventional tech industry product practice will not produce deep enough subject matter insights to create transformative tools for thought.
...The aspiration is for any team serious about making transformative tools for thought. It’s to create a culture that combines the best parts of modern product practice with the best parts of the (very different) modern research culture. You need the insight-through-making loop to operate, whereby deep, original insights about the subject feed back to change and improve the system, and changes to the system result in deep, original insights about the subject.
A hypothesis is a liability
A Research Paper by Itai Yanai & Martin LercherThere is a hidden cost to having a hypothesis. It arises from the relationship between night science and day science, the two very distinct modes of activity in which scientific ideas are generated and tested, respectively [1, 2]. With a hypothesis in hand, the impressive strengths of day science are unleashed, guiding us in designing tests, estimating parameters, and throwing out the hypothesis if it fails the tests. But when we analyze the results of an experiment, our mental focus on a specific hypothesis can prevent us from exploring other aspects of the data, effectively blinding us to new ideas.
Keep digging
An Article by Ryan SingerThe hardest thing about customer interviews is knowing where to dig. An effective interview is more like a friendly interrogation. We don’t want to learn what customers think about the product, or what they like or dislike — we want to know what happened and how they chose... To get those answers we can’t just ask surface questions, we have to keep digging back behind the answers to find out what really happened.
Fast Path to a Great UX – Increased Exposure Hours
An Article by Jared SpoolAs we’ve been researching what design teams need to do to create great user experiences, we’ve stumbled across an interesting finding. It’s the closest thing we’ve found to a silver bullet when it comes to reliably improving the designs teams produce.
The solution? Exposure hours. The number of hours each team member is exposed directly to real users interacting with the team’s designs or the team’s competitor’s designs. There is a direct correlation between this exposure and the improvements we see in the designs that team produces.
Weighing up UX
An Article by Jeremy KeithMetrics come up when we’re talking about A/B testing, growth design, and all of the practices that help designers get their seat at the table (to use the well-worn cliché). But while metrics are very useful for measuring design’s benefit to the business, they’re not really cut out for measuring user experience.
Monkeys testing random designs
A Tweet by Jared SpoolA/B testing is an effective approach to use science to design and deliver deeply-frustrating user experiences.
A/B testing without upfront research is just random monkeys testing random designs to see which of those designs do “best” against random criteria.
If drug testing was actually implemented like most A/B tests, you’d give 2 drugs to 2 groups of people and pick the “winner” by whichever group had fewer deaths.
You and Your Research
A Speech by Richard HammingThis talk centered on Hamming's observations and research on the question "Why do so few scientists make significant contributions and so many are forgotten in the long run?"
The Sense of Style
Classic style
The guiding metaphor of classic style is seeing the world. The writer can see something that the reader has not yet noticed, and he orients the reader’s gaze so that she can see it for herself. The purpose of writing is presentation, and its motive is disinterested truth. It succeeds when it aligns language with the truth, the proof of success being clarity and simplicity.
The truth can be known, and is not the same as the language that reveals it; prose is a window onto the world.
The assumption of equality
Classic writing, with its assumption of equality between writer and reader, makes the reader feel like a genius. Bad writing makes the reader feel like a dunce.
Nominalization
The English language provides bad writers with a dangerous weapon called nominalization: making something into a noun.
Instead of affirming an idea, you effect its affirmation; rather than postponing something, you implement a postponement.
"Comprehension checks were used as exclusion criteria” would be better said as “we excluded people who failed to understand the instructions.”
“There is not any anticipation there will be a cancellation” would be better as “I don’t anticipate that I will have to cancel.”
Zombie sounds, unlike the verbs whose bodies they snatched, can shamble around without subjects. That is what they have in common with the passive constructions that also bog down these examples.
The curse of knowledge
The better you know something, the less you remember about how hard it was to learn.
The curse of knowledge is the single best explanation I know of why good people write bad prose. It simply doesn’t occur to the writer that her readers don’t know what she knows - that they haven’t mastered the patois of her guild, can’t divine the missing steps that seem too obvious to mention, have no way to visualize a scene that to her is as clear as day. And so she doesn’t bother to explain the jargon, or spell out the logic, or supply the necessary detail.
Structural parallelism
If the new phrase has the same structure as the preceding one, its words can be slotted into the waiting tree, and the reader will absorb it effortlessly. The pattern is called structural parallelism, and it is one of the oldest tricks in the book for elegant (and often stirring) prose.
“He maketh me to lie down in green pastures; he leadeth me beside the still waters.”
You live only once
The logician would argue, You only live once should be rewritten as You live only once, with only next to the thing it qualifies, once.
The logician would be unbearably pedantic, but there is a grain of good taste in the pedantry. Writing is often clearer and more elegant when a writer pushes an only or a not next to the thing that it quantifies. In 1962 John F. Kennedy declared, “We choose to go to the moon not because it is easy but because it is hard.” That sounds a lot classier than “We don’t choose to go to the moon because it is easy but because it is hard."
Such tortuous syntax
How does a writer manage to turn out such tortuous syntax? It happens when he shovels phrase after phrase onto the page in the order which each one occurs to him.
The problem is that the order in which thoughts occur to the writer is different from the order in which they are easily discovered by a reader. It’s a syntactic version of the curse of knowledge. The writer can see the links among the concepts in his internal web of knowledge, and has forgotten that a reader needs to build an orderly tree to decipher them from his string of words.