Answer
a. $9x'^2-y'^2=9$
b. $\frac{x'^2}{1}-\frac{y'^2}{9}=1$
c. See graph.
Work Step by Step
a. From the given equation, we have $A=C=4, B=10$, thus $cot\theta=\frac{A-C}{B}=0$, $2\theta=90^\circ$ and $\theta=45^\circ$.
Use the transformation formulas
$x=x'cos\theta-y'sin\theta=x'cos45^\circ-y'sin45^\circ=\frac{\sqrt 2}{2}(x'-y')$
$y=x'sin\theta+y'cos\theta=x'sin45^\circ+y'cos45^\circ=\frac{\sqrt 2}{2}(x'+y')$
The original equation becomes
$4x^2+10xy+4y^2-9=4(\frac{\sqrt 2}{2}(x'-y'))^2+10(\frac{\sqrt 2}{2}(x'-y'))(\frac{\sqrt 2}{2}(x'+y'))+4(\frac{\sqrt 2}{2}(x'+y'))^2-9=2(x'^2-2x'y'+y'^2)+5(x'^2-y'^2)+2(x'^2+2x'y'+y'^2)-9=9x'^2-y'^2-9=0$
or
$9x'^2-y'^2=9$
b. We can express the equation involving x′ and y′ in the standard form of a conic section as $\frac{x'^2}{1}-\frac{y'^2}{9}=1$
c. See graph.