Answer
$(a)\space 18\space aj/nm^{2}$
(b) Yes
Work Step by Step
$(a)\space U(x)=ax^{2}-bx^{4}-(1)$
We know that, at the equilibria $\frac{dU}{dx}=0$
Let's differentiate (1) by x.
$\frac{dU(x)}{dx}=2ax-4bx^{3}$
$0=2x(a-2bx^{2})$
$x=0$
$\space \space \space \space or$
$\space 0=a-2bx^{2}$
$x^{2}=\frac{a}{2b}=\gt x=\pm \sqrt {\frac{a}{2b}}$
Let's plug known values into this equation.
$\pm\space3nm=\pm \sqrt {\frac{a}{2\times1\space aj/nm^{4}}}=\gt9\space nm^{2}=\frac{a}{2\space aj/nm^{4}}$
$18\space aj/nm^{2}=a$
(b) Yes. at x = 0, There is a metastable equilibrium