What leads Jim to conclude that the ship is deserted?

Jim concludes the ship is either deserted or the crew is drunk because no one was steering.

To and fro, up and down, north, south, east, and west, the HISPANIOLA sailed by swoops and dashes, and at each repetition ended as she had begun, with idly flapping canvas. It became plain to me that nobody was steering. And if so, where were the men? Either they were dead drunk or had deserted her, I thought, and perhaps if I could get on board I might return the vessel to her captain.