What announcement causes the dwarves to moan and Bilbo to weep?

Gandalf's announcement that he must leave the group causes this reaction.

"I always meant to see you all safe (if possible) over the mountains," said the wizard, "and now by good management and good luck I have done it. Indeed we are now a good deal further east than I ever meant to come with you, for after all this is not my adventure. I may look in on it again before it is all over, but in the meanwhile I have some other pressing business to attend to."

The dwarves groaned and looked most distressed, and Bilbo wept.