Answer
$$ - \frac{1}{4}{x^2}\sqrt {2 - 4{x^4}} + \frac{1}{4}{\sin ^{ - 1}}\left( {\sqrt 2 {x^2}} \right) + C$$
Work Step by Step
$$\eqalign{
& \int {\frac{{4{x^5}}}{{\sqrt {2 - 4{x^4}} }}} dx,\,\,\,\,u = 2{x^2} \cr
& = \int {\frac{{{{\left( {2{x^2}} \right)}^2}}}{{\sqrt {2 - {{\left( {2{x^2}} \right)}^2}} }}\left( {4x} \right)} dx \cr
& {\text{Using the given substitution}} \cr
& u = 2{x^2},\,\,\,\,\,\,\,du = 4xdx,\,\,\,\,\,\,dx = \frac{1}{{4x}}du \cr
& {\text{write in terms of }}u \cr
& \int {\frac{{{{\left( {2{x^2}} \right)}^2}}}{{\sqrt {2 - {{\left( {2{x^2}} \right)}^2}} }}\left( {4x} \right)} dx = \int {\frac{{{u^2}}}{{\sqrt {2 - {u^2}} }}\left( {\frac{1}{4}} \right)} du \cr
& = \frac{1}{4}\int {\frac{{{u^2}}}{{\sqrt {{{\left( {\sqrt 2 } \right)}^2} - {u^2}} }}} du \cr
& {\text{Use the Endpaper Integral Table to evaluate the integral}} \cr
& {\text{By formula 81}} \cr
& \left( {81} \right):\,\,\,\,\,\,\int {\frac{{{u^2}du}}{{\sqrt {{a^2} - {u^2}} }} = - \frac{u}{2}\sqrt {{a^2} - {u^2}} + \frac{{{a^2}}}{2}{{\sin }^{ - 1}}\left( {\frac{u}{a}} \right) + C} \cr
& {\text{take }}a = \sqrt 2 \cr
& \frac{1}{4}\int {\frac{{{u^2}}}{{\sqrt {{{\left( {\sqrt 2 } \right)}^2} - {u^2}} }}} du = - \frac{u}{8}\sqrt {{{\left( {\sqrt 2 } \right)}^2} - {u^2}} + \frac{{{{\left( {\sqrt 2 } \right)}^2}}}{8}{\sin ^{ - 1}}\left( {\frac{u}{{\sqrt 2 }}} \right) + C \cr
& {\text{simplifying}} \cr
& = - \frac{u}{8}\sqrt {2 - {u^2}} + \frac{1}{4}{\sin ^{ - 1}}\left( {\frac{u}{{\sqrt 2 }}} \right) + C \cr
& {\text{write in terms of }}x{\text{, and replace }}2{x^2}{\text{ for }}u \cr
& = - \frac{{2{x^2}}}{8}\sqrt {2 - {{\left( {2{x^2}} \right)}^2}} + \frac{1}{4}{\sin ^{ - 1}}\left( {\frac{{2{x^2}}}{{\sqrt 2 }}} \right) + C \cr
& = - \frac{1}{4}{x^2}\sqrt {2 - 4{x^4}} + \frac{1}{4}{\sin ^{ - 1}}\left( {\sqrt 2 {x^2}} \right) + C \cr} $$
