Agile Design and Development
So that you can get feedback on it and make it better
The most rewarding iterations
Building is never a straight line
Product owner vs. product manager
A Product Owner is focused on output i.e. how quickly can we build these features?
Product Management, on the other hand, is focused on outcomes i.e. why are we building these features in the first place?
Good design is redesign
Good design is redesign. It's rare to get things right the first time. Experts expect to throw away some early work. They plan for plans to change.
It helps to have a medium that makes change easy. When oil paint replaced tempera in the fifteenth century, it helped painters to deal with difficult subjects like the human figure because, unlike tempera, oil can be blended and overpainted.
Finish designing as close to the end of a sprint as possible
The traditional process of delivering design, vs. delivering design just in time.
Designers are often working at least one sprint ahead of engineers. While one sprint might not seem like much of a lag, a typical product team learns a lot after the design hand-off. ...Instead of working ahead, we should finish designing as close to the end of a sprint as possible: just-in-time design.
We optimize what we measure
Scrum does not say “only focus on output”, but, unfortunately, humans will optimize for what they measure.
If you worry about story points & hitting your estimations, that’s what is going to consume your attention. That is what you and your team will optimize for.
And that is the core critique of Scrum as it is practiced: That it focuses a product team’s attention so heavily on delivery — on building lots of features quickly & efficiently — that teams fail to focus on spending time to discover what the right thing to build is.
How we can do better
It actually doesn't matter whether you actually have a formal retrospective. It doesn't matter whether you have four or five labels of things on your retro board, or exactly how you do the retro. What does matter is the notion of thinking about what we're doing and how we can do better, and it is the team that's doing the work that does this, that is the central thing.
The 'date scrum' anti-pattern
Date Scrum is an R&D pattern where developers are asked to estimate software project requirements upfront for the entirety of the project. After the project is green lighted and the budget is set based on the final estimates, the team then holds daily scrums to status and manage risk as they “iterate” the solution toward the release date. To some, this approach is described as doing Waterfall in sprints.
The fundamental problem with Date Scrum is that the team is de-focused from discovering the best solution. Instead they are heavily focused on delivering Something™ by the Date™. Engineers are problem solvers, and if the primary problem becomes delivering Something™ that will pass QA by the Date™, they will, with enough pressure, solve that exact problem.
That which requires caring
Today's real world of technology is characterized by the dominance of prescriptive technologies.
The temptation to design more or less everything according to prescriptive and broken-up technologies is so strong that it is even applied to those tasks that should be conducted in a holistic way. Any tasks that require caring, whether for people or nature, any tasks that require immediate feedback and adjustment, are best done holistically. Such tasks cannot be planed, coordinated, and controlled the way prescriptive tasks must be.
Prescriptive technologies eliminate the occasions for decision-making and judgment in general and especially for the making of principled decisions. Any goal of the technology is incorporated a priori in the design and is not negotiable.
Manifesto for Agile Software Development
A DefinitionWe are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.
Agile Scrum is not working
The Agile founders had it right, one size doesn't fit all. What the founders perhaps didn’t foresee, or couldn’t agree on, is that in order for the world to scale and consume their wisdom, it had to be packaged as concrete practices, not as abstract classes with virtual methods to be defined in context. And to the proponents of Agile Scrum, give them their due, for their part, they made it concrete – Agile Scrum has been packaged and delivered. Yet much work remains to realize the promise of Agile, which in summary is, the realization of wise use of lightweight development practices and workflows that flexibly adapt to the changing and evolving needs of customers.
Driving engineers to an arbitrary date is a value destroying mistake
An Article by Gandalf HudlowWhat happens when you apply date pressure to software engineers working on high value software projects? The engineers will focus on delivering Something™ by the Date™! This fatal flaw results in delivery of a Something™ full of chaos and features that nobody really wants or needs.
Beware SAFe, an Unholy Incarnation of Darkness
An Article by Sean DexterThe Lean Portfolio Management function that controls funding, are given sole authority to approve which Portfolio Epics move into each stream. Epics are not explanations about a problem that needs to be solved. They are pre-formed ideas about how best to solve those problems.
Right away we can see signs of the old-school mindset of viewing teams as a “delivery” function instead of a strategic one. The high level thinkers come up with ideas, and the low level doers execute on those ideas. Ignored is the possibility that those closest to the work might be best equipped to make decisions about it. Escaping from this misguided mindset is a core goal of Agile thinking that SAFe fails to remotely accomplish.
Why Scrum is killing your product
An Article by Henry LathamDesign Systems, Agile, and Industrialization
An Article by Brad FrostI’ve come to the conclusion that “enterprise web development” is just regular web development, only stripped of any joy or creativity or autonomy. It’s plugging a bunch of smart people into the matrix and forcing them to crank out widgets and move the little cards to the right.
In these structures, people are stripped of their humanity as they’re fed into the machine. It becomes “a developer resource is needed” rather than “Oh, Samantha would be a great fit for this project.” And the effect of all this on individuals is depressing. When people’s primary motivation is to move tickets over a column, their ability to be creative or serve a higher purpose are almost completely quashed. Interaction with other humans seems to be relegated to yelling at others to tell them they’re blocked.
Reading “AS PER THE REQUIREMENTS” in tickets makes me dry heave. How did such sterile, shitty language seep into my everyday work?
The value-destroying effect of arbitrary date pressure on code
An Article by Gandalf HudlowThe mandate from above is clear, just get it done! Avoid everything that's in the way: all advice, all expertise, all discovery efforts that detract from hitting the Date™!
What these organizations don't realize is that all software change can be modeled as three components: Value, Filler and Chaos. Chaos destroys Value and Filler is just functionality that nobody wants. When date pressure is applied to software projects, the work needed to remove Chaos is subtly placed on the chopping block. Work like error handling, clear logging, chaos & load testing and other quality work is quietly deferred in favor of hitting the Date™.
Agile is Dead (Long Live Agility)
An Article by Dave ThomasThe word “agile” has been subverted to the point where it is effectively meaningless, and what passes for an agile community seems to be largely an arena for consultants and vendors to hawk services and products.
…Let’s abandon the word agile to the people who don’t do things. Instead, let’s use a word that describes what we do. Let’s develop with agility.
- You aren’t an agile programmer—you’re a programmer who programs with agility.
- You don’t work on an agile team—your team exhibits agility.
- You don’t use agile tools—you use tools that enhance your agility.
/
Individuals and Interactions over Processes and Tools
Working Software over Comprehensive Documentation
Customer Collaboration over Contract Negotiation, and
Responding to Change over Following a PlanTraditional companies are losing because they mismanage software engineers
An Article by Emma WattersonInnovation is messy, and frankly Anti-Steve [Jobs] can’t figure out why you wouldn’t just tell people the right thing to build and skip all the trial and error that comes with innovation. Anti-Steve and his board of directors that keep him in place fundamentally believe that they know what needs to be built. Or at least that they can hire the messiah that will come down off the mountain and tell everyone what to build. There is no such messiah.
Why we stopped breaking down stories into tasks
An Article by Adam SilverThe Scrum process says to break down stories into tasks to make estimation easier, encourage collaboration and to be able to show more granular progress during a sprint.
But after a few sprints, we decided to do the next sprint without creating tasks. As a result we drastically increased our velocity and never went back. Here I'll jot down some of the reasons we decided to do this:
- Breaking down stories into tasks is time consuming
- The tasks we came up with invariably would change as we worked on the stories
- Tasks are repetitive
- Tasks were often carried out in parallel
- Our estimates didn't improve
- It decluttered our task board
- It encouraged collaboration throughout the sprint
While we started our process by following Scrum to the letter, we soon realised that breaking down stories into tasks was something that wasn’t worthwhile for us. In the end we realised that it was overplanning and poor use of our time. In the end we used that time to get on with the work and deliver at a significantly faster pace.
Why We Don't Do Daily Stand-Ups at Supercede
An Article by Jezen ThomasYesterday I worked on the widget.
Today I will work on the widget.
I have no blockers.Are you asleep yet? The developers are. You promise them an intellectually stimulating work environment and what they end up with is drudgery.
What value can be had from these meetings anyway? Using “alignment” for justification is so nebulous that it is essentially meaningless. Engineers align themselves. They talk. Especially if you hire good ones (which, you know, you’ll struggle to if you have a culture of coercing them into this kind of busywork). Where does the real discussion happen? It’s written down.
Software that nobody wants
An Article by Gandalf HudlowFinding value is the result of enabling individual and group-level discovery attempts. It's not the result of everyone following one leader's gut.
What just happened is a new software product/feature was created that no customer wanted. This happens way too often. In fact, most hyper important software projects that must be done by date certain or else, have deep flaws that cause some variation of this phenomenon, flaws that include:
- Not wanted - Company specified a solution to a problem that customers don't actually have
- No Rarity - Company is pursuing an iKnockoff of existing products. The market already has two scaled competitors with working solutions, customers naturally spend budget on products that are already successful to avoid risk
- Incorrect Packaging - Customers need a website, but the company created an iOS app instead
- Incorrect Pricing - Customers need SaaS pricing, but the company created a shrink wrapped, on-premise solution with CapEx and maintenance agreements instead
Making sense of MVP
An ArticleHenrik Kniberg:
The top scenario (delivering a front tire) sucks because we keep delivering stuff that the customer can’t use at all. If you know what you’re doing – your product has very little complexity and risk, perhaps you’ve built that type of thing hundreds of times before – then go ahead and just do big bang. Build the thing and deliver it when done.
Doing It Right
An Article by Brad FrostDoing it right requires a different pace of working and a much broader thought process than “ok, let’s get this thing out the door.” Which is super tough because most workplaces place a huge emphasis on getting things out the door, and fast. Little agile tickets that are expected to be completed in micro sprints to me seem to be antithetical to doing it right.
Planning doesn't make for better software
A Fragment by Robin RendleMy own time in a Silicon Valley startup has proved this much to be true; planning doesn’t make for better software. In fact today our design systems team doesn’t have sprints, we don’t have tickets or a daily standup. Each day we come to work, figure out what’s the most important thing that we could be doing, and then we—gasp!—actually do it.
Watching so many other teams slowly flail about whilst they plan for quarter 3.2 of subplan A, whilst our team produces more work in a week than they all do combined in a quarter has been shocking to me.
After four years of working in a large startup, I know what I always assumed was true: you don’t need a plan to make a beautiful thing. You really don’t. In fact, there’s a point where overplanning can be a signal of inexperience and fear and bullshit. The scrum board and the sprints and the inane meetings each and every day are not how you build another Super Mario 64.
Instead all you have to do is hire smart people, trust them to do their best work, and then get the hell out of their way.
Agile as Trauma
An Essay by Dorian TaylorThe Agile Manifesto is an immune response on the part of programmers to bad management.
Yagni
A Definition by Martin FowlerYagni originally is an acronym that stands for "You Aren't Gonna Need It". It is a mantra from Extreme Programming that's often used generally in agile software teams. It's a statement that some capability we presume our software needs in the future should not be built now because "you aren't gonna need it".
The State of Agile Software in 2018
A Talk by Martin FowlerOn the surface, the world of agile software development is bright, since it is now mainstream. But the reality is troubling, because much of what is done is faux-agile, disregarding agile's values and principles. The three main challenges we should focus on are: fighting the Agile Industrial Complex and its habit of imposing process upon teams, raising the importance of technical excellence, and organizing our teams around products (rather than projects).
Product vs. Feature Teams
An Article by Marty CaganThis article is certain to upset many people.
The Art of Doing Science and Engineering: Learning to Learn
The Art of Doing Science and Engineering is the full expression of what "You and Your Research" outlined. It's a book about thinking; more specifically, a style of thinking by which great ideas are conceived.
Gifts of knowledge to humanity
There are many commonalities we can admire in these endeavors: the dazzling leap of imagination, the broad scope of applicability, the founding of a new paradigm. But let’s focus here on their form of distribution. These are all things that are taught. To “use” them means to learn them, understand them, internalize them, perform them with one’s own hands. They are free to any open mind.
In Hamming’s world, great achievements are gifts of knowledge to humanity.
Hamming-greatness
Hamming-greatness is tied, inseparably, with the conception of science and engineering as public service. This school of thought is not extinct today, but it is rare, and doing such work is not impossible, but fights a nearly overwhelming current.
It cannot be taught in words
How to be a great painter cannot be taught in words; one learns by trying many different approaches that seem to surround the subject. Art teachers usually let the advanced student paint, and then make suggestions on how they would have done it, or what might also be tried, more or less as the points arise in the student’s head—which is where the learning is supposed to occur!
Preparing for problems
I firmly believe in Pasteur’s remark, “Luck favors the prepared mind.” In this way I can illustrate how the individual’s preparation before encountering the problem can often lead to recognition, formulation, and solution.
Student's future, not teacher's past
Teachers should prepare the student for the student’s future, not for the teacher’s past.
If you know what you are doing
In science, if you know what you are doing, you should not be doing it.
In engineering, if you do not know what you are doing, you should not be doing it.The homogeneity of knowledge
The standard process of organizing knowledge by departments, and sub-departments, and further breaking it up into separate courses, tends to conceal the homogeneity of knowledge, and at the same time to omit much which falls between the courses.
Another goal of the course is to show the essential unity of all knowledge rather than the fragments which appear as the individual topics are taught. In your future anything and everything you know might be useful, but if you believe the problem is in one area you are not apt to use information that is relevant but which occurred in another course.
An information service society
Society is steadily moving from a material goods society to an information service society. At the time of the American Revolution, say 1780 or so, over 90% of the people were essentially farmers—now farmers are a very small percentage of workers.
What will the situation be in 2020? As a guess I would say less than 25% of the people in the civilian workforce will be handling things; the rest will be handling information in some form or other. In making a movie or a tv program you are making not so much a thing, though of course it does have a material form, as you are organizing information.
From hands to machines
It has rarely proved practical to produce exactly the same product by machines as we produced by hand. Mechanization requires you produce an equivalent product, not identically the same one.
A matter of choice and balance
More than ever before, engineering is a matter of choice and balance rather than just doing what can be done. And more and more it is the human factors which will determine good design—a topic which needs your serious attention at all times.
Central planning gives poor results
Central planning has been repeatedly shown to give poor results (consider the Russian experiment, for example, or our own bureaucracy). The persons on the spot usually have better knowledge than can those at the top and hence can often (not always) make better decisions if things are not micromanaged.
"Real programmers"
At the time the Symbolic Assembly Program (SAP) first appeared I would guess about 1% of the older programmers were interested in it—using SAP was “sissy stuff,” and a real programmer would not stoop to wasting machine capacity to do the assembly.
Using your own expertise
Almost all professionals are slow to use their own expertise for their own work. The situation is nicely summarized by the old saying, “The shoemaker’s children go without shoes.”
History tends to be charitable
History tends to be charitable. It gives credit for understanding what something means when we first do it. But there is a wise saying, “Almost everyone who opens up a new field does not really understand it the way the followers do.”
The reason this happens so often is the creators have to fight through so many dark difficulties, and wade through so much misunderstanding and confusion, they cannot see the light as others can, now the door is open and the path made easy.
Mass production of variable products
Computers have opened the door much more generally to the mass production of a variable product, regardless of what it is: numbers, words, word processing, making furniture, weaving, or what have you. They enable us to deal with variety without excessive standardization, and hence we can evolve more rapidly to a desired future!
Thinking is a matter of degree
Perhaps “thinking” is not a yes/no thing, but maybe it is a matter of degree.
Making coal miners into programmers
Many humans at present are not equipped to compete with machines—they are unable to do much more than routine jobs. There is a widespread belief (hope?) that humans can compete, once they are given proper training. However, I have long publicly doubted you could take many coal miners and make them into useful programmers.
A minimum size to fish
There is the famous story by Eddington about some people who went fishing in the sea with a net. Upon examining the size of the fish they had caught, they decided there was a minimum size to the fish in the sea! Their conclusion arose from the tool used and not from reality.
Spelled with a lowercase letter
I used to tease John Tukey that you are famous only when your name was spelled with a lowercase letter such as watt, ampere, volt, fourier (sometimes), and such.
Why it can't be done
Moral: when you know something cannot be done, also remember the essential reason why, so later, when the circumstances have changed, you will not say, “It can’t be done.”
When you decide something is not possible, don’t say at a later date it is still impossible without first reviewing all the details of why you originally were right in saying it couldn’t be done.
Intellectual shelf life
Let lab equipment lie idle for some time, and suddenly it will not work properly! This is called “shelf life,” but it is sometimes the shelf life of the skills in using it rather than the shelf life of the equipment itself! I have seen it all too often in my direct experience. Intellectual shelf life is often more insidious than is physical shelf life.
Beware of jargon
Beware of jargon—learn to recognize it for what it is, a special language to facilitate communication over a restricted area of things or events. But it also blocks thinking outside the original area it was designed to cover. Jargon is both a necessity and a curse.
You cannot consume what is not produced
The only law of economics that I believe in is Hamming’s law: “You cannot consume what is not produced.” There is not another single reliable law in all of economics I know of which is not either a tautology in mathematics or else sometimes false.
I walked the crest of the dune
Thus piece by piece I walked the crest of the dune, and each time the solution slipped on one side or the other I knew what to do to get back on the track.
God loved sand
“God loved sand, He made so much of it.” I heard, inside myself, that we were already having to exploit lower-grade copper mines, and could only expect to have an increasing cost for good copper as the years went by, but the material for glass is widely available and is not likely to ever be in short supply.
The Hawthorne effect
At the Hawthorne plant of Western Electric, long, long ago, some psychologists were trying to improve productivity by making various changes in the environment. They painted the walls an attractive color, and productivity rose. They made the lighting softer, and productivity rose. Each change caused productivity to rise. One of the men got a bit suspicious and sneaked a change back to the original state, and productivity rose! Why? It appears that when you show you care, the person on the other end responds more favorably than if you appear not to care. The workers all thought the changes were being made for their benefit and they responded accordingly. In the field of education, if you tell the students you are using a new method of teaching, then they respond by better performance, and so, incidentally, does the professor. A new method may or may not be better, indeed it may be worse, but the Hawthorne effect, which is not small in the educational area, is likely to indicate that here is a new, important, improved teaching method. It hardly matters what the new method is; its trial will produce improvements if the students perceive it as being done for their benefit.
What you learn for yourself
What you learn from others you can use to follow;
What you learn for yourself you can use to lead.The unreasonable effectiveness of mathematics
We now see that all this “truth” which is supposed to reside in mathematics is a mirage. It is all arbitrary, human conventions.
But we then face the unreasonable effectiveness of mathematics. Having claimed there was neither “truth” nor “meaning” in the mathematical symbols, I am now stuck with explaining the simple fact that mathematics is used and is an increasingly central part of our society, especially in science and engineering. We have passed from absolute certain truth in mathematics to the state where we see there is no meaning at all in the symbols—but we still use them! We put the meaning into the symbols as we convert the assumptions of the problem into mathematical symbols, and again when we interpret the results. Hence we can use the same formula in many different situations—mathematics is sort of a universal mental tool for clear thinking.
What can be put into words
It is not evident, though many people, from the early Greeks on, implicitly act as if it were true, that all things, whatsoever they may be, can be put into words—you could talk about anything: the gods, truth, beauty, and justice. But if you consider what happens in a music concert, then it is obvious that what is transmitted to the audience cannot be put into words—if it could, then the composer and musicians would probably have used words. All the music critics to the contrary, what music communicates cannot (apparently) be put into words. Similarly, but to a lesser extent, for painting. Poetry is a curious field where words are used but the true content of the poem is not in the words!
Initial psychological distance
Creativity seems, among other things, to be “usefully” putting together things which were not perceived to be related before, and it may be the initial psychological distance between the things which counts most.
Tossing an idea around
Can we do anything to increase creativity? There are training courses, and books, as well as “brainstorming sessions” which are supposed to do this. Taking the brainstorming sessions first, while they were very fashionable at one time, they have generally been found to be not much good when formally done, when a brainstorming session is carefully scheduled. But we all have had the experience of “tossing an idea around” with a friend, or a few friends (but not a large group, generally), from which insight, creativity, or whatever you care to call it, arises and we make progress.
Prepare your mind for the future
Probably the most important tool in creativity is the use of an analogy. Something seems like something else which we knew in the past. Wide acquaintance with various fields of knowledge is thus a help—provided you have the knowledge filed away so it is available when needed, rather than to be found only when led directly to it. This flexible access to pieces of knowledge seems to come from looking at knowledge while you are acquiring it from many different angles, turning over any new idea to see its many sides before filing it away. This implies effort on your part not to take the easy, immediately useful “memorizing the material” path, but to prepare your mind for the future.
Stuck with a problem
If you cannot drop a wrong problem, then the first time you meet one you will be stuck with it for the rest of your career.
Experts and impossibility
If an expert says something can be done he is probably correct, but if he says it is impossible then consider getting another opinion.
Always time to fix it later
As the saying goes:
There is never time to do the job right, but there is always time to fix it later,
especially in computer software!
The average adult
Averages are meaningful for homogeneous groups (homogeneous with respect to the actions that may later be taken), but for diverse groups averages are often meaningless. As earlier remarked, the average adult has one breast and one testicle.
Solution to evaluation and back again
A second reason the systems engineer’s design is never completed is the solution offered to the original problem usually produces both deeper insight and dissatisfactions in the engineers themselves.
Furthermore, while the design phase continually goes from proposed solution to evaluation and back again and again, there comes a time when this process of redefinement must stop and the real problem be coped with—thus giving what they realize is, in the long run, a suboptimal solution.
The heart of systems engineering
While the client has some knowledge of his symptoms, he may not understand the real causes of them, and it is foolish to try to cure the symptoms only. Thus while the systems engineers must listen to the client, they should also try to extract from the client a deeper understanding of the phenomena. Therefore, part of the job of a systems engineer is to define, in a deeper sense, what the problem is and to pass from the symptoms to the causes.
Just as there is no definite system within which the solution is to be found, and the boundaries of the problem are elastic and tend to expand with each round of solution, so too there is often no final solution, yet each cycle of input and solution is worth the effort. A solution which does not prepare for the next round with some increased insight is hardly a solution at all.
I suppose the heart of systems engineering is the acceptance that there is neither a definite fixed problem nor a final solution, rather evolution is the natural state of affairs. This is, of course, not what you learn in school, where you are given definite problems which have definite solutions.