Why wouldn't anyone help Jim and his mother defend the inn against the Captain's enemies?

In essence, they refused to help because they were afraid.

For--you would have thought men would have been ashamed of themselves--no soul would consent to return with us to the Admiral Benbow. The more we told of our troubles, the more--man, woman, and child-- they clung to the shelter of their houses. The name of Captain Flint, though it was strange to me, was well enough known to some there and carried a great weight of terror. Some of the men who had been to field-work on the far side of the Admiral Benbow remembered, besides, to have seen several strangers on the road, and taking them to be smugglers, to have bolted away; and one at least had seen a little lugger in what we called Kitt's Hole. For that matter, anyone who was a comrade of the captain's was enough to frighten them to death. And the short and the long of the matter was, that while we could get several who were willing enough to ride to Dr. Livesey's, which lay in another direction, not one would help us to defend the inn.