To build a folly is essentially to do something a second time, something at an inopportune moment. That something is always the memory of something forgotten, about which we can paradoxically say "There it is again."
Follies were misunderstood, purposeless constructions. They were often only small, extravagant gestures in a garden, easily whisking off the imagination to distant lands, a sort of time capsule built to awaken the memory and induce surprise in passers-by. They marked locations, organized secondary paths in a park, or simply predicted the arrival of better times—a demarcation, a sacred spot, a mysterious trail, a hill whose tragic rocky nature begged for a tower, a party, or the arrival of summer.
“It is demonstrably true that things cannot be other than as they are. For, everything having been made for a purpose, everything is necessarily for the best purpose.” — Professor Pangloss
The demand for “originality”—with the implication that the reminiscence of other writers is a sin against originality and a defect in the work—is a recent one and would have seemed quite ludicrous to poets of the Augustan Age, or of Shakespeare’s time. The traditional view is that each new work should be a fresh focus of power through which former streams of beauty, emotion, and reflection are directed. This view is adopted, and perhaps carried to excess, by writers like T. S. Eliot, some of whose poems are a close web of quotations and adaptations, chosen for their associative value, or like James Joyce, who makes great use of the associative value of sounds and syllables.