Research & Ethnography
What is unspoken
A nested classificatory hierarchy
Unfinished
Record them all
Recommendations for field notes
In Defense of Browsing
The illustrated guide to a Ph.D.
When users never use the features they asked for
An Article by Austin Z. HenleyWe deployed our tool. Almost no one used it.
The handful that did use it, used it once or twice and barely interacted with it. After a few days, zero people were using it.
Why did they tell me they wanted these features?
Why Most Published Research Findings Are False
A Research Paper by John P.A. IoannidisThere is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance.
How can we develop transformative tools for thought?
A Research Paper by Andy Matuschak & Michael NielsenConventional tech industry product practice will not produce deep enough subject matter insights to create transformative tools for thought.
...The aspiration is for any team serious about making transformative tools for thought. It’s to create a culture that combines the best parts of modern product practice with the best parts of the (very different) modern research culture. You need the insight-through-making loop to operate, whereby deep, original insights about the subject feed back to change and improve the system, and changes to the system result in deep, original insights about the subject.
A hypothesis is a liability
A Research Paper by Itai Yanai & Martin LercherThere is a hidden cost to having a hypothesis. It arises from the relationship between night science and day science, the two very distinct modes of activity in which scientific ideas are generated and tested, respectively [1, 2]. With a hypothesis in hand, the impressive strengths of day science are unleashed, guiding us in designing tests, estimating parameters, and throwing out the hypothesis if it fails the tests. But when we analyze the results of an experiment, our mental focus on a specific hypothesis can prevent us from exploring other aspects of the data, effectively blinding us to new ideas.
Keep digging
An Article by Ryan SingerThe hardest thing about customer interviews is knowing where to dig. An effective interview is more like a friendly interrogation. We don’t want to learn what customers think about the product, or what they like or dislike — we want to know what happened and how they chose... To get those answers we can’t just ask surface questions, we have to keep digging back behind the answers to find out what really happened.
Fast Path to a Great UX – Increased Exposure Hours
An Article by Jared SpoolAs we’ve been researching what design teams need to do to create great user experiences, we’ve stumbled across an interesting finding. It’s the closest thing we’ve found to a silver bullet when it comes to reliably improving the designs teams produce.
The solution? Exposure hours. The number of hours each team member is exposed directly to real users interacting with the team’s designs or the team’s competitor’s designs. There is a direct correlation between this exposure and the improvements we see in the designs that team produces.
Weighing up UX
An Article by Jeremy KeithMetrics come up when we’re talking about A/B testing, growth design, and all of the practices that help designers get their seat at the table (to use the well-worn cliché). But while metrics are very useful for measuring design’s benefit to the business, they’re not really cut out for measuring user experience.
Monkeys testing random designs
A Tweet by Jared SpoolA/B testing is an effective approach to use science to design and deliver deeply-frustrating user experiences.
A/B testing without upfront research is just random monkeys testing random designs to see which of those designs do “best” against random criteria.
If drug testing was actually implemented like most A/B tests, you’d give 2 drugs to 2 groups of people and pick the “winner” by whichever group had fewer deaths.
You and Your Research
A Speech by Richard HammingThis talk centered on Hamming's observations and research on the question "Why do so few scientists make significant contributions and so many are forgotten in the long run?"
Infrastructure: A Guide to the Industrial Landscape
Savage, hostile, and cruel
Some may find puzzling or distasteful the parallel I am drawing between the study of nature and the study of technology. After all, nature is good and good for you, whereas everyone knows that technology is ugly, evil, and dangerous.
A few centuries ago—say, on the American western frontier—a quite different view prevailed. Nature was seen as savage, hostile, cruel. Mountains and forests were barriers, not refuges. The lights of civilization were a comforting sight. We took our charter from the book of Genesis, which grants mankind dominion over the beasts, and felt it was both our entitlement and our duty to tame the wilderness, fell the trees, plow the land, and dam the rivers.
Nature undisturbed
My chief aim is simply to describe and explain the technological fabric of society, not to judge whether it is good or bad, beautiful or ugly. And yet I would not argue that technology is neutral or value-free. Quite the contrary: I suggest that the signs of human presence are the only elements of the landscape that have and moral or aesthetic significance at all. In nature undisturbed, a desert is not better or worse than a forest or a swamp; there is simply no scale on which to rank such things unless it is a human scale of utility or beauty. Only when people intervene in nature is there any question of right or wrong, better or worse.
The raw materials of society
A big hole in the ground: This is where most of the raw materials of an industrial society come from. To appreciate the scale of this excavation, note that the bright blue object on a shelf near the center of the image is a Porta Potti.
The dragline
A dragline is the largest of the machines used to strip away the overburden and mine the ore layer at an open-cast mine. A bucketload for this particular dragline, one of the world's largest, is 220 cubic yards. Note the school bus, which would easily fit in the bucket.
Dark satanic steel
When poet William Blake wrote of "dark satanic mills", he couldn't have been looking at a steel mill because there were none in 1804. Nevertheless, when I visit a steel mill, Blake's phrase always comes to mind. With the heat and the pounding noise, the dust and smoke, and the red glow against the night sky, it's hard not to see these places as infernal. And yet the process of making steel also produces some of the most hauntingly beautiful images found anywhere in the world of industry.
Strength from both mass and form
Hoover Dam has the shape of an arch dam, but it is actually a hybrid structure, gathering strength from both mass and form. The dam is often ranked as one of the most exquisite of all engineered structures. It is fitted to its site so well that the gnarly canyon wall looks like an organic growth engulfing the mass of concrete.
Warmed by the afternoon sun
Textbooks on water-system engineering state that supply mains are generally installed on the north side of the street in the Northern Hemisphere and on the south side in the Southern Hemisphere, so that the sun will warm them. In both hemispheres they are supposed to be on the east side of north-south streets, on the premise that the afternoon sun is warmer than the morning sun.
Trompe l'oeil fantasies
In residential neighborhoods some sewage-pumping stations are trompe l'oeil fantasies, dressed up to look like the split-level or colonial houses that surround them. If you look closely, it's not hard to spot these disguised pumphouses: the heavy-duty power connections, the big ventilating fans, and the diesel generator in the backyard are all tip-offs. Furthermore, the windows are often fakes, with sash and shutters adorning a blank wall.
Quaker Square Inn
The modernist architect Le Corbusier was an admirer of American grain elevators, suggesting that their regularity and modularity could serve as a model for other kinds of buildings. At least one later architect took the suggestion seriously. The Quaker Square Inn in Akron, Ohio, occupies the shell of a former elevator. If you're in town for the night, you can rent a round room in one of the silos.
A metropolis for hydrocarbons
An oil refinery suggests the image of a metropolis for hydrocarbons, the pipe manifolds like expressways, the distillation towers like skyscrapers.
A haunting, syncopated music
"Rappers" on the roof of the electrostatic precipitator knock the accumulated dust free, letting it fall into the storage hopper. Each rapper is the size and shape of a baseball bat. Inside is an electromagnet that pulls a steel plunger upward, then allows it to fall again, producing a sharp knock. The rappers are energized at seemingly random intervals, producing a haunting, syncopated music. (The rhythm seemed more modern jazz than rap.)
Safety cut rope axe man
In the first nuclear reactor, constructed by Enrico Fermi in 1942 under the bleachers of the University of Chicago football stadium, the control rods were held up by a manila rope. A man with an axe was told to cut the rope if the reactor got out of hand. This "safety cut rope axe man" is supposedly the origin of the term SCRAM for an emergency shutdown procedure.
Pylons
Not all the towers along a transmission line are identical. Look closely at a tower where the line makes a sharp turn and you will likely find it is wider and beefier than other towers along the route. The added strength and weight are needed to resist the unbalanced pull of the conductors, which might overturn an ordinary tower. These special towers are called deviation or angle towers.
The transmission-line tower everybody knows is an Erector Set latticework of steel girders and diagonal braces. The techniques for designing and building these towers are the same ones used in constructing steel bridge trusses or crane booms. The individual pieces can be made cheaply from rolled steel and then bolted together on the site. This last point is more important than it might seem: transporting a fully assembled tower 100 feet tall is an awkward and expensive business.
Color coding
Telephone wires erupt in a multicolor cascade in a ground-level, pedestal-type splice case on a city street.
Dividing the set of 10 colors into two contrasting groups of 5 allows for exactly 25 combinations with one color from each group; thus, each pair in a bundle can be uniquely colored. A similar color code is applied to the ribbons that bind together all the pairs in a bundle, and to those of the superbundles. The result is highly festive! A specific wire might be identified as the blue-red conductor within the orange-black bundle within the brown-yellow superbundle.
The Iridium System
Several Low-Earth-Orbit (LEO) networks were proposed, but only one got off the ground: the Iridium system. The original Iridium proposal called for a "constellation" of 77 satellites, which gave the plan its name: the element iridium has atomic number 77, meaning that an iridium atom has 77 orbiting electrons. Before the satellites were launched, the constellation was scaled back to 66 active satellites, but no one wanted to change the name to Dysprosium.
Roads to nowhere
Among real-estate developers, straight lines and right angles went out of fashion sometime in the middle of the twentieth century. If you look at a town or a residential neighborhood laid out since then, you are more likely to find sinuous, serpentine roads—whether or not the topography offers any excuse for such curves. Many of these roads go nowhere: they are loops that bring you back to where you started, or they are cul-de-sacs. Making it easy to find your way through the network of streets is obviously not a high priority. This is an interesting development in urban geography: having redesigned the city to accommodate the automobile, we now search for ways to discourage people from driving on the streets.
A gradual refinement
The steel rail is an artifact whose form has been carefully optimized. This gradual refinement of the design was done not by a single brilliant engineer but by more than a century of industrial evolution. The rail was never meant to be an object of beauty, but its cross section has all the elegance of fine typography.
Hyperart: U.S. Rail
The steepest grade on U.S. main-line track is at the small town of Saluda, on a Norfolk Southern route between Spartanburg, South Carolina, and Asheville, North Carolina. The grade goes on for three miles at a slope of 4 or 5 percent. Trains have not been running on the line since 2001, but the tracks are still maintained.
An emblem of friendship
Bridges make connections; they bring people together—a role that has made them a traditional emblem of friendship. Consider the town of Mostar in Bosnia-Herzegovina. When fighting between ethnic factions broke out there in the 1990s, nothing symbolized the social disintegration more clearly than the destruction of a sixteenth-century stone-arch bridge that had linked the two parts of the town on opposite banks of the Neretva River. And the emblem of efforts to heal the divisions is a rebuilt bridge, opened with fireworks and fanfare in July of 2004.
Routine design
When we think of bridges, it is the dramatic and monumental long spans that come to mind first, especially the lithe suspension bridges such as the Golden Gate and the pure geometric arches such as Sydney Harbour. But the majority of bridges are not such spectacular structures. Most of them are ordinary overpasses, with spans of 30 or 40 feet, carrying roadways or rails across other thoroughfares or over small streams. You see such bridges by the dozen on any drive down the Interstate. They may be lacking in glamour, but they are more representative of a bridge builder's art.
The engineering and construction of girder bridges are pretty routine these days, but the bridges are not quite standard items you order from a catalogue. The girders, whether of steel or concrete, are custom-build for each bridge, then trucked to the site and hoisted into place with a crane. The designer still has scope for variation and creativity, and it shows out on the highways: some overpasses are prettier than others.
No Smoking
What's that big electrical cord that also connects the ship to the dock? It's not a power cord to run machinery aboard ship; it's a grounding strap, to prevent sparks from static electricity. Something else you're sure to notice on a tanker is a big warning sign about the fire hazard. Frederick Allen, the editor of American Heritage of Invention & Technology, has remarked that all tankers seem to be named No Smoking.
Port furniture
All the miscellaneous fittings and fixtures on wharves and piers and elsewhere in nautical neighborhoods are known by the charming term port furniture.
The mirror-image economy
When we enter the world of refuse and waste, we cross over into a mirror-image economy. In the "normal" world, we pay to acquire things; on the other side of the looking glass, we pay to get rid of them. Junk isn't merely worthless; it has negative value.
A chemical engineer once told me about a recent improvement in a manufacturing process; by fine-tuning a chemical synthesis he had increased the yield of a certain commodity from 98 percent to 99 percent. I congratulated him, but I couldn't help remarking that this seemed like a rather paltry improvement. "Ah, you miss the important point," he said. "The amount of waste goes from 2 percent down to 1 percent. It's cut in half. We save tremendously on disposal costs."
NIMBY, BANANA, NOPE
Waste-disposal facilities of all kinds—landfills, incinerators, even transfer stations—are sure bets for generating the NIMBY response: not in my backyard. In its most cynical form, NIMBY is the attitude of citizens who acknowledge the need for a facility, somewhere, but who oppose a plan for building it simply because the selected site is too close to their own property. But opposition to landfills and many other kinds of development goes well beyond cynical NIMBY. Another catch phrase for this phenomenon is BANANA: build absolutely nothing anywhere near anybody. Or else it's NOPE: not on planet earth.
Creations of human artifice
In the twenty-first century, the question most of us ask when disaster strikes is not "How could God let that happen?" but "Who screwed up?" This is a salutary development: We take responsibility for the world we live in. Whether or not our world is the best of all possible worlds, it is a world we have made for ourselves. We live in an engineered landscape, on an engineered planet. Our cities and farms, our dwellings and vehicles, our power plans and communication networks—these are all creations of human artifice. If we don't like it here, we have only ourselves to blame.
What kind of world it's going to be
The wonderful thing about living in a world of our own creation is that we get to choose what kind of world it's going to be—at least in principle. But the promise is meaningful only if a broad enough "we" can be engaged in the process. At present, mechanisms and democratic institutions for making collective decisions about the deployment of technology are hopelessly cumbersome. How can anyone make a sensible choice without being able to weigh one alternative against another?