Answer
$$\frac{1}{4} \tan x \sec ^{3} x-\frac{1}{8} \tan x \sec x-\frac{1}{8} \ln |\sec x+\tan x|+C$$
Work Step by Step
\begin{align*}
\int \tan^2x\sec^3xdx&= \int (\sec^2x-1)\sec^3 xdx\\
&=\int (\sec^5x-\sec^3 x)dx\\
\end{align*}
Use
$$
\int \sec ^{m} x d x=\frac{\tan x \sec ^{m-2} x}{m-1}+\frac{m-2}{m-1} \int \sec ^{m-2} x d x
$$
Then
\begin{align*}
\int \tan ^{2} x \sec ^{3} x d x&=\frac{1}{4} \tan x \sec ^{3} x+\frac{3}{4} \int \sec ^{3} x d x-\int \sec ^{3} x d x\\
&=\frac{1}{4} \tan x \sec ^{3} x-\frac{1}{4} \int \sec ^{3} x d x\\
&= \frac{1}{4} \tan x \sec ^{3} x-\frac{1}{4}\left(\frac{1}{2} \tan x \sec x+\frac{1}{2} \int \sec x d x\right)\\
&=\frac{1}{4} \tan x \sec ^{3} x-\frac{1}{8} \tan x \sec x-\frac{1}{8} \ln |\sec x+\tan x|+C
\end{align*}
