Answer
a. $-x'^2+4y'^2=16$
b. $\frac{y'^2}{4}-\frac{x'^2}{16}=1$
c. see graph
Work Step by Step
a. Step 1. Given
$3x^{2}-10xy+3y^{2}-32=0$
we have $A=C=3, B=-10$.
The rotation angle is
$cot(2\theta)=\frac{A-C}{B}=0, 2\theta=\frac{\pi}{2}, \theta=\frac{\pi}{4}$
Step 2. We have the transformations:
$x=\frac{\sqrt 2}{2}(x'-y')$ and $y=\frac{\sqrt 2}{2}(x'+y')$
Step 3. The equation becomes
$3x^{2}-10xy+3y^{2}-32=3(\frac{\sqrt 2}{2}(x'-y'))^{2}-10(\frac{\sqrt 2}{2}(x'-y'))(\frac{\sqrt 2}{2}(x'+y'))+3(\frac{\sqrt 2}{2}(x'+y'))^{2}-32=-2x'^2+8y'^2-32=0$
or
$-x'^2+4y'^2=16$
b. In standard form, we have
$\frac{y'^2}{4}-\frac{x'^2}{16}=1$
which represents a hyperbola.
c. We can graph the equation as shown in the figure.
