Answer
\[x(\sin^{-1}x)^2+2\sqrt{1-x^2}\,\sin^{-1}x-2x+C\]
Work Step by Step
Let \[I=\int (\arcsin x)^2\,dx=\int (\sin^{-1}x)^2\,dx\;\;\;\ldots (1)\]
Using integration by parts
\[I=(\sin^{-1}x)^2\int dx-\int \left(\left\{(\sin^{-1}x)^2\right\}'\int dx\right)dx\]
\[I=x(\sin^{-1}x)^2-\int\frac{2x\sin^{-1}x}{\sqrt{1-x^2}}dx\]
\[I=x(\sin^{-1}x)^2+\int\left(\frac{-2x}{\sqrt{1-x^2}}\right)\sin^{-1}x\,dx+C_1\;\;\;\ldots (2)\]
Where $C_1$ is constant of integration
Let \[I_1=\int\left(\frac{-2x}{\sqrt{1-x^2}}\right)\sin^{-1}x\,dx\]
Using integration by parts
\[I_1=\sin^{-1}x\int\frac{-2x}{\sqrt{1-x^2}}dx-\int\left((\sin^{-1}x)'\int\frac{-2x}{\sqrt{1-x^2}}dx\right)dx\]
\[I_1=2\sqrt{1-x^2}\sin^{-1}x-\int\frac{2\sqrt{1-x^2}}{\sqrt{1-x^2}}dx\]
\[I_1=2\sqrt{1-x^2}\sin^{-1}x-2x+C_2\;\;\;\ldots (3)\]
Where $C_2$ is constant of integration
Using (3) in (2)
\[I=x(\sin^{-1}x)^2+2\sqrt{1-x^2}\,\sin^{-1}x-2x+C\]
Where $C=C_1+C_2$
Hence \[I=x(\sin^{-1}x)^2+2\sqrt{1-x^2}\,\sin^{-1}x-2x+C.\]
