Answer
$(a)\space Mdv=V_{ex}dm$
$(b)\space V_{f}-V_{i}=V_{ex}ln(\frac{m_{i}}{m_{f}})$
Work Step by Step
Please see the attached image below.
(a) Here we use the principle of conservation of momentum.
$ m_{1}u_{1}+m_{2}u_{2}=m_{1}V_{1}+m_{2}V_{2}$
Let's take, the speed of the rocket after ejection = u
Let's apply the principle of conservation of momentum to the system in the vertical direction.
$\uparrow m_{1}u_{1}=m_{2}V_{2}+m_{3}V_{3}$
Let's plug known values into this equation.
$MV=(M-dm)u+dm(-V_{ex})-(1)$
Since dm is very small,$\space M-dm\approx M$
$(1)=\gt\space MV=Mu-dmV_{ex}$
$dmV_{ex}=M(u-v)= MdV$
$Mdv=V_{ex}dm-(2)$
(b) We can write, $dv=V_{ex}\times \frac{1}{M}dm$
Let's integrate this equation,
$\int_{V_{i}}^{V_{f}}dv=V_{ex}\int_{M_{f}}^{M_{i}}\frac{1}{M}dm$
$x\biggr |_{V_{i}}^{V_{f}}=V_{ex}\times lnM\biggr |_{M_{f}}^{M_{i}}$
$V_{f}-V_{i}=V_{ex}(lnM_{i}-lnM_{f})$
$V_{f}=V_{i}+V_{ex}(ln\frac{M_{i}}{M_{f}})$