A Christmas Memory is told from the first-person perspective of a seven-year-old boy who is called "Bubby" by his older cousin, who also plays a significant part in A Christmas Memory. Buddy's older cousin, who is never referred to by name, lives with Buddy in a house with several different relatives and a family dog named Queenie.
Buddy and his family have very little money and have to be industrious with what little money they do have. Still, Buddy and his cousin look forward to Christmas every year; they also look forward to their Christmas traditions. One of their favorite traditions involves them saving their money to buy the ingredients to make a fruit cake, including pecans and whisky even though they are living during Prohibition, a time in which selling and consuming alcohol was illegal within the United States. To that end, the two must buy their whisky from a bootlegger called Haha Jones, who is friendly and seemingly harmless. But once the two make their cakes, they don't give them to friends and family. Instead, they give them to people they have never met and people who have only met a few times.
After they finish making the fruit cakes, Buddy and his cousin decide to celebrate by finishing off the bottle of whisky the two bought. They quickly become drunk. And the other people get angry with Buddy's older cousin for letting him drink alcohol much too young. Buddy and his older cousin, however, don't take to their housemates' anger very well and Buddy rushes to comfort his older cousin with thoughts of their Christmas rituals.
The following day, Buddy and a friend venture out to cut down a Christmas tree for Buddy's house, something which they can do. After Buddy brings the tree home, he and his cousin spend time decorating their home and their new tree with Christmas decorations, another one of their traditions.
The following morning, Buddy, his cousin, and the rest of their family open their presents, but Buddy is very disappointed with his gifts. Buddy makes his cousin a kite (as she does for him every year), which the cousin loves. The next day, the two go to a meadow and fly the woman's kite together. That day, as it turns out, was one of the last times that the two saw each other. Buddy was quickly shipped off to a military school. But as Buddy was away, his cousin developed dementia and quickly forgot who he was. She then died shortly thereafter.
