To build a folly is essentially to do something a second time, something at an inopportune moment. That something is always the memory of something forgotten, about which we can paradoxically say "There it is again."
Follies were misunderstood, purposeless constructions. They were often only small, extravagant gestures in a garden, easily whisking off the imagination to distant lands, a sort of time capsule built to awaken the memory and induce surprise in passers-by. They marked locations, organized secondary paths in a park, or simply predicted the arrival of better times—a demarcation, a sacred spot, a mysterious trail, a hill whose tragic rocky nature begged for a tower, a party, or the arrival of summer.
“It is demonstrably true that things cannot be other than as they are. For, everything having been made for a purpose, everything is necessarily for the best purpose.” — Professor Pangloss
In Architecture Without Architects, Bernard Rudofsky documented the ways in which most cities were mostly built by improvisation, following no consistent formal design. Building was added to building, street to street, their forms adapting to different site conditions in the process of extension.
Rudofsky thought that this hidden order is how most settlements of poor people develop and that the work of improvising street order attaches people to their communities, whereas 'renewal' projects, which may provide a cleaner street, pretty houses, and large shops, give the inhabitants no way to mark their presence on the space.