Research & Ethnography
What is unspoken
A nested classificatory hierarchy
Unfinished
Record them all
Recommendations for field notes
In Defense of Browsing
The illustrated guide to a Ph.D.
When users never use the features they asked for
An Article by Austin Z. HenleyWe deployed our tool. Almost no one used it.
The handful that did use it, used it once or twice and barely interacted with it. After a few days, zero people were using it.
Why did they tell me they wanted these features?
Why Most Published Research Findings Are False
A Research Paper by John P.A. IoannidisThere is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance.
How can we develop transformative tools for thought?
A Research Paper by Andy Matuschak & Michael NielsenConventional tech industry product practice will not produce deep enough subject matter insights to create transformative tools for thought.
...The aspiration is for any team serious about making transformative tools for thought. It’s to create a culture that combines the best parts of modern product practice with the best parts of the (very different) modern research culture. You need the insight-through-making loop to operate, whereby deep, original insights about the subject feed back to change and improve the system, and changes to the system result in deep, original insights about the subject.
A hypothesis is a liability
A Research Paper by Itai Yanai & Martin LercherThere is a hidden cost to having a hypothesis. It arises from the relationship between night science and day science, the two very distinct modes of activity in which scientific ideas are generated and tested, respectively [1, 2]. With a hypothesis in hand, the impressive strengths of day science are unleashed, guiding us in designing tests, estimating parameters, and throwing out the hypothesis if it fails the tests. But when we analyze the results of an experiment, our mental focus on a specific hypothesis can prevent us from exploring other aspects of the data, effectively blinding us to new ideas.
Keep digging
An Article by Ryan SingerThe hardest thing about customer interviews is knowing where to dig. An effective interview is more like a friendly interrogation. We don’t want to learn what customers think about the product, or what they like or dislike — we want to know what happened and how they chose... To get those answers we can’t just ask surface questions, we have to keep digging back behind the answers to find out what really happened.
Fast Path to a Great UX – Increased Exposure Hours
An Article by Jared SpoolAs we’ve been researching what design teams need to do to create great user experiences, we’ve stumbled across an interesting finding. It’s the closest thing we’ve found to a silver bullet when it comes to reliably improving the designs teams produce.
The solution? Exposure hours. The number of hours each team member is exposed directly to real users interacting with the team’s designs or the team’s competitor’s designs. There is a direct correlation between this exposure and the improvements we see in the designs that team produces.
Weighing up UX
An Article by Jeremy KeithMetrics come up when we’re talking about A/B testing, growth design, and all of the practices that help designers get their seat at the table (to use the well-worn cliché). But while metrics are very useful for measuring design’s benefit to the business, they’re not really cut out for measuring user experience.
Monkeys testing random designs
A Tweet by Jared SpoolA/B testing is an effective approach to use science to design and deliver deeply-frustrating user experiences.
A/B testing without upfront research is just random monkeys testing random designs to see which of those designs do “best” against random criteria.
If drug testing was actually implemented like most A/B tests, you’d give 2 drugs to 2 groups of people and pick the “winner” by whichever group had fewer deaths.
You and Your Research
A Speech by Richard HammingThis talk centered on Hamming's observations and research on the question "Why do so few scientists make significant contributions and so many are forgotten in the long run?"
Design Thinking
Ducks and decorated sheds
A duck is a building whose confirmation is a complete symbol or icon. A decorated shed is a building to which symbols, often commonplace signs, have been attached.
Clinging to ideas
Another aspect of design thinking that was evident in the foregoing case studies is the tenacity with which designers will cling to major design ideas and themes in the face of what at times might seem insurmountable odds. Often the concept the designer has in mind can only come to fruition if a large number of apparently countervailing conditions can be surmounted.
Even when severe problems are encountered, a considerable effort is made to make the initial idea work, rather than to stand back and adopt a fresh point of departure.
A concept of style
It is a concept based not on the classification of various physical features of architecture and urban design but on the problem-solving process itself. We have seen that the final outcome of a design process is strongly determined by at least three aspects of that process:
- the subject matter of the organizing principles which are adopted,
- the manner in which these principles are interpreted and reinterpreted in the context of the problem at hand, and
- the sequent of applying such organizing principles.
Consistency in style among the output of designers can thus be understood as a habitual way of doing things, of solving problems.
Form and figure
Form applies to “a configuration with natural meaning or none at all,” whereas figure applies to “a configuration whose meaning is given by culture."
Design skirmishes
it is apparent that the unfolding of the design process assumed a distinctly episodic structure, which we might characterize as a series of related skirmishes with various aspects of the problem at hand.
As the scope of the problem became more determined and finite for the designer, the episodic character of the process seems to have become less pronounced. During this period a systematic working out of issues and conditions took hold within the framework that had been established. This phenomenon is not at all surprising when we consider the fundamental difference between moments of problem solving when matters are poorly defined and those with clarity and sufficiency of structure.
Within the episodic structure of the process, the problem, as perceived by the designer, tends to fluctuate from being rather nebulous to being more specific and well-defined. Furthermore, moments of "blinding" followed by periods of backtracking take place, where blinding refers to conditions in which obvious connections between various considerations of importance go unrecognized by a designer.
Such plans were deemed efficient
The terrain of cities was subdivided along the lines of distinct and discrete patterns of use, with very little opportunity for mixing (separation and concentration of functions). After all, the home environment should be just that…while places of work should be aggregated and serviced with their appropriate supporting functions.
Such plans were deemed efficient.
Constrained by the medium
The inevitable reciprocation that occurs between the act of drawing and the thinking associated with it. The hand moves, the mind becomes engaged, and vice versa. We might ask: How much does the medium of expression actually constrain a design process?
A medium has a way of constraining our choices, and this influence may not involve conscious choice at all. The planner, in the end, sees and understands only those things for which they can provide expression.
Autonomous constraints
Autonomous or independent constraints do not derive from the problem as given and understood…there was nothing in the problem statement, or brief, that required any reference be made to it. This constraint, introduced by the designers, usefully transcended the givens of the problem situation.
Unless the entire problem at hand can be solved using strictly problem-oriented constraints, we have to step outside the known problem context in order to continue problem solving activity.
The strange familiar and the familiar strange
The problem solver, when confronted with a new and yet unsolved problem, overlays the structure of the unsolved problem with an apparently similar problem with which he or she is experienced.
Making the strange familiar and the familiar strange are also principally based on the use of analogy.