Answer
This speeds up the Earth's rotation, shortening the day by a tiny, tiny amount.
Work Step by Step
This situation is similar to the one shown in Figure 8.52. If everyone moved to the poles, they would be closer to the Earth’s rotational axis. Moving mass toward the axis of rotation decreases the rotational inertia of the system.
No external torques act on the system (Earth + inhabitants), so angular momentum is conserved. The product of rotational inertia and rotational speed stays constant. Since the first went down, the second must go up. The rotational speed increases, making the day shorter.