Answer
Some of the $Ca(OH)_2$ would start to precipitate, forming $Ca(OH)_2(s)$, which is a white solid.
Work Step by Step
This is the equilibrium reaction:
$Ca(OH)_2(s) \lt -- \gt Ca^{2+} (aq) + 2OH^-(aq)$
When the pH increases, the $[OH^-]$ is increasing too, and the common ion effect will act. So, the reaction will be more reactant-favored.
That follows the Le Chatelier's principle. Since we are raising the concentration of one of the products, the equilibrium will try to reduce this concentration, to reduce the change.
Since the original solution of $Ca(OH)_2$ is saturated, if the equilibrium moves to the solid's side, the precipitation will occur.