The story is composed of three distinct and separate narratives.
Francis Barber was captured in his village in Africa and brought to England as a slave in the 18th century. He works tirelessly to earn his freedom, eventually finding the hospitality of Samuel Johnson, the historical author. Johnson hires Barber as a servant, and they become devoted friends. Narrated by a third party, seemingly a British gentleman of the time, Barber's story is continually interrupted by social commentary. The narrator voices sympathy for men like Barber who suffered from suspicion just because of their race, but he also vents his own racial prejudices about Barber and Johnson's amicable relationship.
Raised in poverty by his single mother, Randolf Turpin, then called "Randy," looked up to his older brothers. When they started boxing, he followed. He proved astonishingly good at it and quickly became famous and wealthy. He earns the people's admiration during his up-and-coming career in the 1940s, but people start taking advantage of Randy's ignorance. They manipulate him in order to take advantage of his newfound wealth. At the height of his fame he wins the middleweight world championship. As time goes on, however, he mismanages his finances and allows his temper to dictate his interactions. Known to abuse women, Randy loses his popularity to scandal and lives the rest of his life in quiet despair and poverty.
David Oluwale's story is narrated in brief interviews with many different people who met him. Their descriptions of his character vary dramatically, but Phillips takes pains to outline how some of the most critical witnesses have compromised their opinions with racial prejudice. Escaping violence in Nigeria, Oluwale sneaks onto a ship bound for England. In 1949 he finds work in Leeds, participating in the great industrial boom at the time. As racist tension in the region increases, Oluwale starts fearing for his life. Two policemen in particular continue to target him, eventually beating him nearly to death in 1969. He's arrested multiple times on false charges, which hurts his ability to find and keep work. Finally Oluwale spends period in a mental institute, after which he winds up on the streets having lost all of his former ambition.