Answer
$x=x'(\frac{4}{5})-y'(\frac{3}{5})=\frac{1}{5}(4x'-3y')$ and
$y=x'(\frac{3}{5})+y'(\frac{4}{5})=\frac{1}{5}(3x'+4y')$
Work Step by Step
Step 1. From the given equation, we have $A=34, B=-24, C=41$. Thus we have $cot2\theta=\frac{A-C}{B}=\frac{34-41}{-24}=\frac{7}{24}$. Thus $2\theta$ is in Quadrant I and $cos2\theta=\frac{7}{\sqrt {7^2+24^2}}=\frac{7}{25}$ (form a right triangle with sides $7,24,25$ here).
Step 2. We can find values of $sin\theta$ and $\cos\theta$ as $sin\theta=\sqrt {\frac{1-cos2\theta}{2}}=\sqrt {\frac{1-(\frac{7}{25})}{2}}=\frac{3}{5}$ and $cos\theta=\sqrt {1-sin^2\theta}=\frac{4}{5}$
Step 3. Using the transformation formula $x=x'cos\theta-y'sin\theta$ and $y=x'sin\theta+y'cos\theta$, we have
$x=x'(\frac{4}{5})-y'(\frac{3}{5})=\frac{1}{5}(4x'-3y')$
$y=x'(\frac{3}{5})+y'(\frac{4}{5})=\frac{1}{5}(3x'+4y')$
