Our Quantified Life
The primary measure of progress
SAFe is oriented around volume, not value
Outcomes decide
Design as an engineering problem
Obsessed with absolute numbers
Lost purposes
The case against heatmaps
It's All Over
An Essay by Justin E. H. SmithIt has come to seem to me recently that this present moment must be to language something like what the Industrial Revolution was to textiles. A writer who works on the old system of production can spend days crafting a sentence, putting what feels like a worthy idea into language, only to find, once finished, that the internet has already produced countless sentences that are more or less just like it, even if these lack the same artisanal origin story that we imagine gives writing its soul. There is, it seems to me, no more place for writers and thinkers in our future than, since the nineteenth century, there has been for weavers.
Time-based analytics
An Article by Ryan SingerAnalytics apps don't tell you much about usage behavior. You might be able to see how many users performed an event, or how many times they did it. But none of the analytics packages out there are good at showing you how often people do things. Are they using to-dos once a week? Every day? Only signing into the app once a month but happily paying for years?
Time matters. You can't understand usage without time.
Fast Path to a Great UX – Increased Exposure Hours
An Article by Jared SpoolAs we’ve been researching what design teams need to do to create great user experiences, we’ve stumbled across an interesting finding. It’s the closest thing we’ve found to a silver bullet when it comes to reliably improving the designs teams produce.
The solution? Exposure hours. The number of hours each team member is exposed directly to real users interacting with the team’s designs or the team’s competitor’s designs. There is a direct correlation between this exposure and the improvements we see in the designs that team produces.
Metrics have a strange hold on the imagination
A Fragment by Shawn WangOnce in place, metrics have a strange hold on the imagination: I've seriously had a CTO carelessly reject my genuine idea out of hand because "it doesn't help OKRs", the same OKRs we previously agreed should not describe all that we do.
I agree with Amir Shevat that we should "do the right things over the easy to measure things."
How would you feel if you could no longer use the product?
An ArticleThe product/market fit definitions I had found were vivid and compelling, but they were lagging indicators — by the time investment bankers are staking out your house, you already have product/market fit. Instead, Ellis had found a leading indicator: just ask users “how would you feel if you could no longer use the product?” and measure the percent who answer “very disappointed.”
Site performance is potentially the most important metric
A Fragment by Kealan ParrSite performance is potentially the most important metric. The better the performance, the better chance that users stay on a page, read content, make purchases, or just about whatever they need to do. A 2017 study by Akamai says as much when it found that even a 100ms delay in page load can decrease conversions by 7% and lose 1% of their sales for every 100ms it takes for their site to load which, at the time of the study, was equivalent to $1.6 billion if the site slowed down by just one second.
The McNamara fallacy
A DefinitionThe McNamara fallacy, named for Robert McNamara, the US Secretary of Defense from 1961 to 1968, involves making a decision based solely on quantitative observations (or metrics) and ignoring all others. The reason given is often that these other observations cannot be proven.
The fallacy refers to McNamara's belief as to what led the United States to defeat in the Vietnam War—specifically, his quantification of success in the war (e.g., in terms of enemy body count), ignoring other variables.
Artifice, blindness, and suicide
A QuoteThe first step is to measure whatever can be easily measured. This is OK as far as it goes. The second step is to disregard that which can't be easily measured or to give it an arbitrary quantitative value. This is artificial and misleading. The third step is to presume that what can't be measured easily really isn't important. This is blindness. The fourth step is to say that what can't be easily measured really doesn't exist. This is suicide.
Weighing up UX
An Article by Jeremy KeithMetrics come up when we’re talking about A/B testing, growth design, and all of the practices that help designers get their seat at the table (to use the well-worn cliché). But while metrics are very useful for measuring design’s benefit to the business, they’re not really cut out for measuring user experience.
Predicted Mean Vote
A DefinitionThe predicted mean vote (PMV) was developed by Povl Ole Fanger at Kansas State University and the Technical University of Denmark as an empirical fit to the human sensation of thermal comfort. It was later adopted as an ISO standard. It predicts the average vote of a large group of people on the a seven-point thermal sensation scale where:
- +3 = hot
- +2 = warm
- +1 = slightly warm
- 0 = neutral
- -1 = slightly cool
- -2 = cool
- -3 = cold
What Do Metrics Want? How Quantification Prescribes Social Interaction on Facebook
A Research Paper by Benjamin GrosserWhat are the effects of this enumeration, of these metrics that count our social interactions? In other words, how are the designs of Facebook leading us to act, and to interact in certain ways and not in others? For example, would we add as many friends if we weren’t constantly confronted with how many we have? Would we “like” as many ads if we weren’t told how many others liked them before us? Would we comment on others’ statuses as often if we weren’t told how many friends responded to each comment?
In this paper, I question the effects of metrics from three angles. First I examine how our need for personal worth, within the confines of capitalism, transforms into an insatiable “desire for more.” Second, with this desire in mind, I analyze the metric components of Facebook’s interface using a software studies methodology, exploring how these numbers function and how they act upon the site’s users. Finally, I discuss my software, born from my research-based artistic practice, called Facebook Demetricator (2012-present). Facebook Demetricator removes all metrics from the Facebook interface, inviting the site’s users to try the system without the numbers and to see how that removal changes their experience. With this free web browser extension, I aim to disrupt the prescribed sociality produced through metrics, enabling a social media culture less dependent on quantification.
Rationality: From AI to Zombies
The Tao of rationality
If you would learn to think like reality, then here is the Tao:
Since the beginning
not one unusual thing
has ever happened.Everyone sees themselves as behaving normally
To understand why people act the way they do, we must first realize that everyone sees themselves as behaving normally.
Argue against the best
To argue against an idea honestly, you should argue against the best arguments of the strongest advocates.
It’s all too easy to argue that someone is exhibiting Bias #182 in your repertoire of fully generic accusations, but you can’t settle a factual issue without closer evidence. If there are biased reasons to say the sun is shining, that doesn’t make it dark out.
Let the meaning choose the word
What is above all needed is to let the meaning choose the word, and not the other way around.
Probably it is better to put off using words as long as possible and get one’s meaning as clear as one can through pictures and sensations.
People can stand for what is true, for they are already enduring it
Knowing the design
Knowing the design can tell you much about the designer; and knowing the designer can tell you much about the design.
Cutting through to the truth
The essential thing in the art of epistemic rationality is to understand how every rule is cutting through to the truth in the same movement.
Your sword has no blade. It has only your intention. When that goes astray you have no weapon.
Joy in the merely attainable
But the really fundamental problem with desiring the unattainable is that as soon as you actually get it, it stops being unattainable. If we cannot take joy in the merely available, our lives will always be frustrated.
I have failed my art
The novice goes astray and says, “The art has failed me.”
The master goes astray and says, “I have failed my art."Taboo your words
Albert says that people have “free will.” Barry says that people don’t have “free will.” Well, that will certainly generate an apparent conflict. Most philosophers would advise Albert and Barry to try to define exactly what they mean by “free will,” on which topic they will certainly be able to discourse at great length. I would advise Albert and Barry to describe what it is that they think people do, or do not have, without using the phrase “free will” at all.
Your map of reality
Reality is very large—just the part we can see is billions of lightyears across. But your map of reality is written on a few pounds of neurons, folded up to fit inside your skull. I don’t mean to be insulting, but your skull is tiny. Comparatively speaking. Inevitably, then, certain things that are distinct in reality, will be compressed into the same point on your map. But what this feels like from inside is not that you say, “Oh, look, I’m compressing two things into one point on my map.” What it feels like from inside is that there is just one thing, and you are seeing it.
What you're trying to swim
There is an art to using words; even when definitions are not literally true or false, they are often wiser or more foolish. Dictionaries are mere histories of past usage; if you treat them as supreme arbiters of meaning, it binds you to the wisdom of the past, forbidding you to do better. Though do take care to ensure (if you must depart from the wisdom of the past) that people can figure out what you’re trying to swim.
Applause lights
Most applause lights are much more blatant, and can be detected by a simple reversal test.
For example, suppose someone says: We need to balance the risks and opportunities of AI.
If you reverse this statement, you get: We shouldn’t balance the risks and opportunities of AI.Since the reversal sounds abnormal, the unreversed statement is probably normal, implying it does not convey new information. There are plenty of legitimate reasons for uttering a sentence that would be uninformative in isolation. “We need to balance the risks and opportunities of AI” can introduce a discussion topic; it can emphasize the importance of a specific proposal for balancing; it can criticize an unbalanced proposal. Linking to a normal assertion can convey new information to a bounded rationalist—the link itself may not be obvious. But if no specifics follow, the sentence is probably an applause light.
Reality just seems to go on crunching
I once met a fellow who thought that if you used General Relativity to compute a low-velocity problem, like an artillery shell, General Relativity would give you the wrong answer—not just a slow answer, but an experimentally wrong answer—because at low velocities, artillery shells are governed by Newtonian mechanics, not General Relativity. This is exactly how physics does not work. Reality just seems to go on crunching through General Relativity, even when it only makes a difference at the fourteenth decimal place, which a human would regard as a huge waste of computing power. Physics does it with brute force. No one has ever caught physics simplifying its calculations—or if someone did catch it, the Matrix Lords erased the memory afterward.
Mystery exists in the mind
Mystery exists in the mind, not in reality. If I am ignorant about a phenomenon, that is a fact about my state of mind, not a fact about the phenomenon itself. All the more so if it seems like no possible answer can exist: Confusion exists in the map, not in the territory. Unanswerable questions do not mark places where magic enters the universe. They mark places where your mind runs skew to reality.
Lost purposes
There’s chocolate at the supermarket, and you can get to the supermarket by driving, and driving requires that you be in the car, which means opening your car door, which needs keys. If you find there’s no chocolate at the supermarket, you won’t stand around opening and slamming your car door because the car door still needs opening. I rarely notice people losing track of plans they devised themselves.
It’s another matter when incentives must flow through large organizations—or worse, many different organizations and interest groups, some of them governmental. Then you see behaviors that would mark literal insanity, if they were born from a single mind. Someone gets paid every time they open a car door, because that’s what’s measurable; and this person doesn’t care whether the driver ever gets paid for arriving at the supermarket, let alone whether the buyer purchases the chocolate, or whether the eater is happy or starving.