How did Jim's father realize that the captain is trying to avoid seafaring men whom he talking about?

Jim's father realized that the Captain is anxious to avoind travelers and seafaring men from the start.... when the Captain arrives.

‘This is a handy cove,’ says he at length; ‘and a pleasant sittyated grog-shop. Much company, mate?’
My father told him no, very little company, the more was the pity.
‘Well, then,’ said he, ‘this is the berth for me. Here you, matey,’ he cried to the man who trundled the barrow; ‘bring up alongside and help up my chest. I’ll stay here a bit,’ he continued.