Answer
$$-\frac{x}{\sqrt{x^{2}-1}}+\ln |x+\sqrt{x^{2}-1}|+C$$
Work Step by Step
Given $$ \int \frac{x^{2} d x}{\left(x^{2}-1\right)^{3 / 2}}$$ Let $$x=\sec u\ \ \ \ \ \ dx=\sec u\tan udu $$ Then \begin{align*} \int \frac{x^{2} d x}{\left(x^{2}-1\right)^{3 / 2}}&= \int \frac{\sec^3 u\tan udu }{\left(\sec^{2}u-1\right)^{3 / 2}}\\ &= \int \frac{\sec^3 u\tan udu}{\left(\tan ^2u\right)^{3 / 2}}\\ &= \int \frac{\sec^3 u\tan udu}{ \tan ^3u }\\ &=\int \frac{\sec u(1+\tan^2 u)du}{ \tan ^2u }\\ &= \int \frac{\sec u d u}{\tan ^{2} u}+\int \frac{\tan ^{2} u \sec u d u}{\tan ^{2} u} \\ &=\int \csc u \cot u \, d u+\int \sec udu\\ &= -\csc u+\ln |\sec u+\tan u|+C\\ &= -\frac{x}{\sqrt{x^{2}-1}}+\ln |x+\sqrt{x^{2}-1}|+C \end{align*}
