Answer
$$\frac{1}{5}{\left( {\sec x} \right)^5} - \frac{1}{3}{\left( {\sec x} \right)^3} + C $$
Work Step by Step
$$\eqalign{
& \int {{{\sec }^3}x} {\tan ^3}xdx \cr
& {\text{We write }}{\sec ^3}x{\text{ as }}{\sec ^2}x\sec x{\text{ and }}{\tan ^3}x{\text{ as ta}}{{\text{n}}^2}x\tan x \cr
& = \int {{{\sec }^2}x\sec x} {\tan ^2}x\tan xdx \cr
& = \int {{{\sec }^2}x} {\tan ^2}x\left( {\sec x\tan x} \right)dx \cr
& {\text{use the fundamental identiy ta}}{{\text{n}}^2}\theta = {\sec ^2}\theta - 1 \cr
& = \int {{{\sec }^2}x} \left( {{{\sec }^2}x - 1} \right)\left( {\sec x\tan x} \right)dx \cr
& = \int {\left( {{{\sec }^4}x - {{\sec }^2}x} \right)} \left( {\sec x\tan x} \right)dx \cr
& {\text{Integrate by substitution method}} \cr
& {\text{Let }}u = \sec x,\,\,\,\,du = \sec x\tan xdx,\,\,\,\,dx = \frac{{du}}{{\sec x\tan x}} \cr
& {\text{Then}}{\text{,}} \cr
& = \int {\left( {{u^4} - {u^2}} \right)} \left( {\sec x\tan x} \right)\left( {\frac{{du}}{{\sec x\tan x}}} \right) \cr
& {\text{Cancel common factor }}\sec x\tan x \cr
& = \int {\left( {{u^4} - {u^2}} \right)} du \cr
& {\text{integrating}} \cr
& = \frac{1}{5}{u^5} - \frac{1}{3}{u^3} + C \cr
& {\text{write in terms of }}x,\,\,\,u = \sec x \cr
& = \frac{1}{5}{\left( {\sec x} \right)^5} - \frac{1}{3}{\left( {\sec x} \right)^3} + C \cr} $$
