Answer
$M = 5.0~kg$
Work Step by Step
We can use Equation (9-68) to find $M$:
$v_{2f} = \frac{2~m_1}{m_1+M}~v_{1i}$
$(m_1+M)~v_{2f} = 2~m_1~v_{1i}$
$M~v_{2f}= 2~m_1~v_{1i} - m_1~v_{2f}$
$M = \frac{2~m_1~v_{1i} - m_1~v_{2f}}{v_{2f}}$
$M = \frac{(2)(3.0~kg)~(8.0~m/s) - (3.0~kg)~(6.0~m/s)}{6.0~m/s}$
$M = 5.0~kg$
