Answer
$$\int\left(x^{5} e^{x^{2}}\right) d x=\frac{1}{2}\left( x^4e^{x^2}-2(x^2e^{x^2}-e^{x^2})\right)+c$$
Work Step by Step
$$
\int\left(x^{5} e^{x^{2}}\right) d x
$$
Let $z=x^{2} \Rightarrow d z=2 x d x$, then
$$
\int\left(x^{5} e^{x^{2}}\right) d x=\frac{1}{2}\int\left(z^{2} e^{z}\right) dz
$$
Using integration by parts, since
\begin{align*}
u=&z^2\ \ \ \ \ \ \ \ dv=e^zdz\\
du=&2zdz\ \ \ \ \ \ \ v=e^z
\end{align*}
Hence
\begin{align}
\int\left(x^{5} e^{x^{2}}\right) d x&=\frac{1}{2}\int\left(z^{2} e^{z}\right) dz\\
&=\frac{1}{2}\left( z^2e^z-2\int ze^zdz\right)
\end{align}
Using integration by parts again to integrate $\int ze^zdz$
\begin{align*}
u=&z \ \ \ \ \ \ \ \ dv=e^zdz\\
du=& dz\ \ \ \ \ \ \ v=e^z
\end{align*}
Then
\begin{align}
\int ze^zdz&= ze^z-\int e^zdz\\
&=ze^z-e^z
\end{align}
Hence
\begin{align}
\int\left(x^{5} e^{x^{2}}\right) d x&=\frac{1}{2}\left( z^2e^z-2(ze^z-e^z)\right)\\
&=\frac{1}{2}\left( x^4e^{x^2}-2(x^2e^{x^2}-e^{x^2})\right)+c
\end{align}
