Answer
a) $\frac{-\pi}{4}$
b) $\frac{119}{169}$
Work Step by Step
a)
$\arcsin(\sin{\frac{5\pi}{4}})$
Using the unit circle:
$\arcsin(-\frac{\sqrt{2}}{2})$
Sine of what gives us $-\frac{\sqrt{2}}{2}$? If we look at the unit circle, we see that it is $\frac{-\pi}{4}$.
b) Draw a right triangle with a leg length of $5$ and a hypotenuse of $13$. The angle $\theta$ should be across the $5$. Using the Pythagorean Theorem ($a^2+b^2=c^2$), we find that the length of the missing leg is $12$.
$\cos(2\sin^{-1}(\frac{5}{13}))$
According to our triangle, $\sin^{-1}(\frac{5}{13})$ is our $\theta$, so now, $\cos(2\sin^{-1}(\frac{5}{13}))=\cos(2\theta)$.
Using double angle identities ($\cos(2\theta)=\cos^2(x)-sin^2(x)$):
$\cos(2\theta)=\cos^2(x)-sin^2(x)$
According to our triangle, $\cos(x)=\frac{12}{13}$ and $\sin(x)=\frac{5}{13}$
$\cos^2(x)-sin^2(x)=(\frac{12}{13})^2-(\frac{5}{13})^2=\frac{144}{169}-\frac{25}{169}=\frac{119}{169}$