Research & Ethnography
What is unspoken
A nested classificatory hierarchy
Unfinished
Record them all
Recommendations for field notes
In Defense of Browsing
The illustrated guide to a Ph.D.
When users never use the features they asked for
An Article by Austin Z. HenleyWe deployed our tool. Almost no one used it.
The handful that did use it, used it once or twice and barely interacted with it. After a few days, zero people were using it.
Why did they tell me they wanted these features?
Why Most Published Research Findings Are False
A Research Paper by John P.A. IoannidisThere is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance.
How can we develop transformative tools for thought?
A Research Paper by Andy Matuschak & Michael NielsenConventional tech industry product practice will not produce deep enough subject matter insights to create transformative tools for thought.
...The aspiration is for any team serious about making transformative tools for thought. It’s to create a culture that combines the best parts of modern product practice with the best parts of the (very different) modern research culture. You need the insight-through-making loop to operate, whereby deep, original insights about the subject feed back to change and improve the system, and changes to the system result in deep, original insights about the subject.
A hypothesis is a liability
A Research Paper by Itai Yanai & Martin LercherThere is a hidden cost to having a hypothesis. It arises from the relationship between night science and day science, the two very distinct modes of activity in which scientific ideas are generated and tested, respectively [1, 2]. With a hypothesis in hand, the impressive strengths of day science are unleashed, guiding us in designing tests, estimating parameters, and throwing out the hypothesis if it fails the tests. But when we analyze the results of an experiment, our mental focus on a specific hypothesis can prevent us from exploring other aspects of the data, effectively blinding us to new ideas.
Keep digging
An Article by Ryan SingerThe hardest thing about customer interviews is knowing where to dig. An effective interview is more like a friendly interrogation. We don’t want to learn what customers think about the product, or what they like or dislike — we want to know what happened and how they chose... To get those answers we can’t just ask surface questions, we have to keep digging back behind the answers to find out what really happened.
Fast Path to a Great UX – Increased Exposure Hours
An Article by Jared SpoolAs we’ve been researching what design teams need to do to create great user experiences, we’ve stumbled across an interesting finding. It’s the closest thing we’ve found to a silver bullet when it comes to reliably improving the designs teams produce.
The solution? Exposure hours. The number of hours each team member is exposed directly to real users interacting with the team’s designs or the team’s competitor’s designs. There is a direct correlation between this exposure and the improvements we see in the designs that team produces.
Weighing up UX
An Article by Jeremy KeithMetrics come up when we’re talking about A/B testing, growth design, and all of the practices that help designers get their seat at the table (to use the well-worn cliché). But while metrics are very useful for measuring design’s benefit to the business, they’re not really cut out for measuring user experience.
Monkeys testing random designs
A Tweet by Jared SpoolA/B testing is an effective approach to use science to design and deliver deeply-frustrating user experiences.
A/B testing without upfront research is just random monkeys testing random designs to see which of those designs do “best” against random criteria.
If drug testing was actually implemented like most A/B tests, you’d give 2 drugs to 2 groups of people and pick the “winner” by whichever group had fewer deaths.
You and Your Research
A Speech by Richard HammingThis talk centered on Hamming's observations and research on the question "Why do so few scientists make significant contributions and so many are forgotten in the long run?"
Contrasting Concepts of Harmony in Architecture
The realm of feeling
Alexander: If you say: "Well, look, you're a feeling type, and I'm a thinking type, so let's not discuss that because we are always going to be on different sides", then it removes from this discussion what I feel to be the absolute heart and soul of the matter when it comes to buildings. Now I don't want to deny at all what you are saying about personalities. But I really cannot conceive of a properly formed attitude towards buildings, as an artist or a builder, or in any way, if it doesn't ultimately confront the fact that buildings work in the realm of feeling.
Panic
Alexander: It never occurred to me that someone could so explicitly reject the core experience of something like Chartres. It's very interesting to have this conversation. If this weren't a public situation, I'd be tempted to get into this on a psychiatric level. I'm actually quite serious about this. What I'm saying is that I understand how one could be very panicked by these kinds of feelings. Actually, it's been my impression that a large part of the history of modern architecture has been a kind of panicked withdrawal from these kinds of feelings, which have governed the formation of buildings over the last 2000 years or so.
Why that panicked withdrawal occurred, I'm still trying to find out. It's not clear to me. But I've never heard somebody say, until a few moments ago, someone say explicitly: "Yes, I find that stuff freaky. I don't like to deal with feelings. I like to deal with ideas.”
The pitched roof
Eisenman: I would argue that the pitched roof is – as Gaston Bachelard points out – one of the essential characteristics of "houseness". It was the extension of the vertebrate structure which sheltered and enclosed man. Michel Foucault has said that when man began to study man in the 19th century, there was a displacement of man from the center. The representation of the fact that man was no longer the center of the world, no longer the arbiter, and, therefore, no longer controlling artifacts, was reflected in a change from the vertebrate-center type of structure to the center-as-void.
The trick of little machines
Alexander: Up until about 1600, most of the world views that existed in different cultures did see man and the universe as more or less intertwined and inseparable ... either through the medium of what they called God or in some other way. But all that was understood. The particular intellectual game that led us to discover all the wonders of science forced us to abandon temporarily that idea. In other words, in order to do physics, to do biology, we were actually taught to pretend that things were like little machines because only then could you tinker with them and find out what makes them tick. That's all fine. It was a tremendous endeavor, and it paid off.
But it may have been factually wrong. That is, the constitution of the universe may be such that the human self and the substance that things made out of, the spatial matter or whatever you call it, are much more inextricably related than we realized. Now, I am not talking about some kind of aboriginal primitivism. I am saying that it may actually be a matter of fact that those things are more related than we realize. And that we have been trained to play a trick on ourselves for the last 300 years in order to discover certain things. Now, if that's true – there are plenty of people in the world who are beginning to say it is, by the way, certainly in physics and other related subjects – then my own contribution to that line of thought has to do with these structures of sameness that I have been talking about.
In other words, the order I was sketching out last night is ultimately, fundamentally an order produced by centers or wholes which are reinforcing each other and creating each other. Now, if all of that is so, then the pitched roof would simply come about as a consequence of all that – not as an antecedent. It would turn out that, in circumstances where one is putting a roof on a building, in the absence of other very strong forces that are forcing you to do something different, that is the most natural and simple roof to do. And, therefore, that kind of order would tend to reappear – of course, in a completely different, modern technological style – simply because that is the nature of order, not because of a romantic harkening back to past years.
Merely a building
Eisenman: Le Corbusier once defined architecture as having to do with a window which is either too large or too small, but never the right size. Once it was the right size it was no longer functioning. When it is the right size, that building is merely a building. The only way in the presence of architecture that is that feeling, that need for something other, when the window was either too large or too small.
Fucking up the world
City Hall by Rafael Moneo, Logroño, La Rioja, Spain.
Alexander : At least my experience tells me, that when a group of different people set out to try and find out what is harmonious, what feels most comfortable in such and such a situation, their opinions about it will tend to converge, if they are mocking up full-scale, real stuff. Of course, if they're making sketches or throwing out ideas, they won't agree. But if you start making the real thing, one tends to reach agreement. My only concern is to produce that kind of harmony.
The thing that strikes me about your friend's building – if I understood you correctly – is that somehow in some intentional way it is not harmonious. That is, Moneo intentionally wants to produce an effect of disharmony. Maybe even of incongruity.
Eisenman: That is correct.
Eisenman: I find that incomprehensible. I find it very irresponsible. I find it nutty. I feel sorry for the man. I also feel incredibly angry because he is fucking up the world.
A good table
Eisenman: Tolstoy wrote about the man who had so many modern conveniences in Russia that when he was adjusting the chair and the furniture, etc., that he was so comfortable and so nice and so pleasant that he didn't know – he lost all control of his physical and mental reality. There was nothing. What I'm suggesting is that if we make people so comfortable in these nice little structures of yours, that we might lull them into thinking that everything's all right, Jack, which it isn't. And so the role of art or architecture might be just to remind people that everything wasn't all right.
Alexander: I can't, as a maker of things, I just can't understand it. I do not have a concept of things in which I can even talk about making something in the frame of mind you are describing. I mean, to take a simple example, when I make a table I say to myself: "All right, I'm going to make a table, and I'm going to try to make a good table." And of course, then from there on I go to the ultimate resources I have and what I know, how well I can make it. But for me to then introduce some kind of little edge, which starts trying to be a literary comment, and then somehow the table is supposed to be at the same time a good table, but it also is supposed to be I don't know what; a comment on nuclear warfare, making a little joke, doing various other things... I'm practically naive; it doesn't make sense to me.