Answer
$cos(A+B) = -\frac{56}{65}$
Work Step by Step
If the hypotenuse is 13, and the opposite side is 5, then the length of the adjacent side is $\sqrt{13^2-5^2} = 12$
If the hypotenuse is 5, and the adjacent side is 3, then the length of the opposite side is $\sqrt{5^2-3^2} = 4$
$cos(A+B) = cos~A~cos~B-sin~A~sin~B$
$cos(A+B) = (\frac{12}{13})(-\frac{3}{5})-(\frac{5}{13})(\frac{4}{5})$
$cos(A+B) = -\frac{36}{65}-\frac{20}{65}$
$cos(A+B) = -\frac{56}{65}$
