what worry keeps coming up for the children?

They worry that equality amongst people and races may never be realized.

I thought they keep worrying that Bob Ewell will hurt Atticus.

"Scout realizes that the decision to see the world fairly can only occur within each individual's heart, and that there is no way to reach a person who has not become personally convinced of the equality of all races and the virtue of following a moral course of action."

"The roly-poly incident is yet another example of Jem's increasing maturity. Having witnessed Tom's trial and his family's reaction of his death, Jem has an even greater sense of the need to protect the innocent. Therefore, the roly-poly bug is a symbol of the weak and oppressed who are often "stomped on" by society. Jem believes in the equality of all people, and his choice to protect the roly-poly demonstrates how deeply ingrained this value is. Jem is becoming a young man of honor and moral virtue, just like his father."

I'd go with that too........ but I think the results of the trial and Tom's death are weighing more heavily on their minds at this point. The kids pretty much go about their business here and don't seem to be isolating themselves over fear of Bob Ewell. But then again, you could be right!