The story starts with a town reeling from a collective hangover. It is a midsummer Sunday afternoon and everyone drank too much the previous night. Neddy Merrill, however, is unperturbed. He reclines by the pool at Donald and Helen Westerhazy's house, full of pleasure at the beautiful day. All of a sudden, it occurs to him that he could swim from the Westerhazys' to his own house, which is about 8 miles away, by traversing the pools of his neighbors. Neddy decides that this is a grand and worthy voyage; naming his route the "Lucinda River" after his wife, he sets off in just his swimming trunks.
Neddy is full of youthful energy, and he swims vigorously through several pools to start his journey. He is familiar with all of his neighbors, and his interactions with them as he swims their pools come to define the social world of the story. At first, his neighbors are gracious and happy to see him, but about halfway through his journey, a big storm comes. He waits out the rain in a friend's gazebo and then continues on, but the route has changed. Now, he is tired and feels a cold wind penetrate his bones. He seems to have aged, shrunken, or lost weight somehow. Most importantly, the people who he assumes will receive him happily to their homes and maybe offer him a gin before he swims their pool now treat him rudely or gossip about some vague misfortunes of his behind his back. Neddy cannot understand why he is being treated this way. He does not remember any of the events alluded to—selling his house or a misfortune befalling his daughters, for example. Nevertheless, something seems to have happened. Each pool seems to bring Neddy closer to winter, old age, and social death.
Finally, nearly overcome with exhaustion, Neddy reaches his home. As he staggers up the driveway, he notices the house is slightly dilapidated, and no lights are on. When he reaches its doors, they are locked and no one is home. He cannot understand where his wife and daughters are. Only at the very end of the text, in the last line, does he realize the house itself is completely empty. The misfortunes he heard whispers of on his voyage were true.