Look at the last paragraph of the chapter. Why do you think Jem says that Boo wants to stay inside?

Jem has matured throughout the novel. In Chapter 23, he comes to realize that the actions of the community and the ugliness of rumors have been the reason for Boo's isolation. Boo simply doesn't want to be the object of gossip, nor does he want to take part.

Jem has matured throughout the novel. In Chapter 23, he comes to realize that the actions of the community and the ugliness of rumors have been the reason for Boo's isolation.