How does bud not buddy find out herman isnt his father but his grandfather

Bud finds out on his trip back from one of the "Dusky Devastators" performances. Bud was in the car with Herman when he saw Herman had the same rocks that he had. Bud said that he got them from his momma. Bud states his moms name and Herman dropped down to the floor in awe. Herman says that she was his daughter.