Answer
The combined balls reach a height of $~~2.6~m~~$ above the collision point.
Work Step by Step
Let "up" be the positive direction.
We can use conservation of momentum to find the velocity of the combined balls immediately after the collision:
$p_f = p_i$
$(5.0~kg)~v_f = (3.0~kg)(20~m/s)+(2.0~kg)(-12~m/s)$
$v_f = \frac{(3.0~kg)(20~m/s)+(2.0~kg)(-12~m/s)}{5.0~kg}$
$v_f = 7.2~m/s$
For the next part of the solution, we can let $v_0 = 7.2~m/s$
We can find the height above the collision point reached by the combined balls:
$v_f^2 = v_0^2+2ay$
$y = \frac{v_f^2 - v_0^2}{2a}$
$y = \frac{0 - (7.2~m/s)^2}{(2)(-9.8~m/s^2)}$
$y = 2.6~m$
The combined balls reach a height of $~~2.6~m~~$ above the collision point.