What sign does Gandalf put on Bilbo's door?

It was a symbol though not anything specific that is expalined:

And I assure you there is a mark on this door—the usual one in the trade, or used to be. Burglar wants a good job, plenty of Excitement and reasonable Reward, that's how it is usually read.