The start of Little, Big introduces readers to an American architect named John Drinkwater. John thinks that there are worlds within his world (and even larger worlds within the world within John's world). At the center of those planets, John discovers, is the realm of fairies - a realm which his wife can interact with, but he cannot. In an effort to get all of his thoughts on paper and to inform people of his thoughts and premonitions.
Shortly thereafter, John begins to design and build a house (which he later called Edgewood) in the middle of nowhere, New York. John's new home is no doubt one of the unique homes in terms of its style; it is disorienting yet protective. To that end, John wants to make his new house a sampler for potential clients of his firm.
The novel then flashes forward to a time after the death of John and his wife. Readers now follow Daily Alice, the couple's great-great-granddaughter. Daily Alice falls in love with a man named ″Smoky″ Barnable, an intelligent and interesting man.
In a flashback, it is revealed that most of the people who live around John's Edgewood home are descended from John and his wife, Violet thanks to a pact he made with the aforementioned fairies. Meanwhile, many years later, Smoky and Alice begin to have issues in their marriage and Smoky begins to cheat. His cheating ultimately results in him impregnating another woman.
Meanwhile, Alice and others around her begin to learn how to use Tarot cards, which she thinks will help her tell the future. The story then moves on to Auberon, Smoky and Alice's child, who has settled in Manhattan and fallen in love with a Puerto Rican woman named Sylvie - who quickly leaves him after being lured away.
At the same time, a politician named Russell Eigenblick becomes President of the United States and advocated for a civil war to start in the country. This drives the faires - and many other people in the U.S. - into hiding, as they feared for their lives. This, along with his losing Sylvie, causes Auberon to sink into a deep depression and alcoholism. But Auberon and his sister are determined to stop Eigenblick, who they see as a grave threat to humanity.
Their struggle to stop Eigenblick comes to a head, and the novel ends with Edgewood, having seen many generations, falling into disrepair and decay - returning to nature one final time. The house, author John Crowley tells readers, became a legend along the way.