Answer
$$\eqalign{
& \left( {\text{a}} \right)\frac{1}{5}{\cos ^4}x\sin x + \frac{4}{{15}}{\cos ^2}x\sin x + \frac{8}{{15}}\sin x + C \cr
& \left( {\text{b}} \right)\frac{5}{{32}}\pi \cr} $$
Work Step by Step
$$\eqalign{
& \left( {\text{a}} \right)\int {{{\cos }^5}x} dx \cr
& {\text{Use the reduction formula }} \cr
& \int {{{\cos }^n}x} dx = \frac{1}{n}{\cos ^{n - 1}}x\sin x + \frac{{n - 1}}{n}\int {{{\cos }^{n - 2}}x} dx \cr
& n = 5 \cr
& \int {{{\cos }^5}x} dx = \frac{1}{5}{\cos ^4}x\sin x + \frac{4}{5}\int {{{\cos }^3}x} dx \cr
& n = 3 \cr
& \int {{{\cos }^5}x} dx = \frac{1}{5}{\cos ^4}x\sin x + \frac{4}{5}\left( {\frac{1}{3}{{\cos }^2}x\sin x + \frac{2}{3}\int {\cos x} dx} \right) \cr
& \int {{{\cos }^5}x} dx = \frac{1}{5}{\cos ^4}x\sin x + \frac{4}{{15}}{\cos ^2}x\sin x + \frac{8}{{15}}\int {\cos x} dx \cr
& \int {{{\cos }^5}x} dx = \frac{1}{5}{\cos ^4}x\sin x + \frac{4}{{15}}{\cos ^2}x\sin x + \frac{8}{{15}}\sin x + C \cr
& \cr
& \left( {\text{b}} \right)\int_0^{\pi /2} {{{\cos }^6}x} dx \cr
& {\text{Use the reduction formula }} \cr
& \int {{{\cos }^n}x} dx = \frac{1}{n}{\cos ^{n - 1}}x\sin x + \frac{{n - 1}}{n}\int {{{\cos }^{n - 2}}x} dx \cr
& n = 6 \cr
& \int {{{\cos }^6}x} dx = \frac{1}{6}{\cos ^5}x\sin x + \frac{5}{6}\int {{{\cos }^4}x} dx \cr
& n = 4 \cr
& \int {{{\cos }^6}x} dx = \frac{1}{6}{\cos ^5}x\sin x + \frac{5}{6}\left( {\frac{1}{4}{{\cos }^3}x\sin x + \frac{3}{4}\int {{{\cos }^2}x} dx} \right) \cr
& \int {{{\cos }^6}x} dx = \frac{1}{6}{\cos ^5}x\sin x + \frac{5}{{24}}{\cos ^3}x\sin x + \frac{{15}}{{24}}\int {{{\cos }^2}x} dx \cr
& n = 2 \cr
& \int {{{\cos }^6}x} dx = \frac{1}{6}{\cos ^5}x\sin x + \frac{5}{{24}}{\cos ^3}x\sin x \cr
& {\text{ }} + \frac{{15}}{{24}}\left( {\frac{1}{2}\cos x\sin x + \frac{1}{2}\int {dx} } \right) \cr
& {\text{ }} = \frac{1}{6}{\cos ^5}x\sin x + \frac{5}{{24}}{\cos ^3}x\sin x + \frac{{15}}{{48}}\cos x\sin x + \frac{{15}}{{48}}\int {dx} \cr
& {\text{ }} = \frac{1}{6}{\cos ^5}x\sin x + \frac{5}{{24}}{\cos ^3}x\sin x + \frac{{15}}{{48}}\cos x\sin x + \frac{{15}}{{48}}x + C \cr
& {\text{Therefore,}} \cr
& \int_0^{\pi /2} {{{\cos }^6}x} dx \cr
& {\text{ }} = \left[ {\frac{1}{6}{{\cos }^5}x\sin x + \frac{5}{{24}}{{\cos }^3}x\sin x + \frac{{15}}{{48}}\cos x\sin x + \frac{{15}}{{48}}x} \right]_0^{\pi /2} \cr
& \left[ {\frac{1}{6}{{\cos }^5}\left( {\frac{\pi }{2}} \right)\left( 0 \right) + \frac{5}{{24}}{{\cos }^3}\left( {\frac{\pi }{2}} \right)\left( 0 \right) + \frac{{15}}{{48}}\cos \left( {\frac{\pi }{2}} \right)\sin \left( 0 \right) + \frac{{15}}{{48}}\left( {\frac{\pi }{2}} \right)} \right] \cr
& - \left[ {\frac{1}{6}{{\cos }^5}\left( 0 \right)\left( 0 \right) + \frac{5}{{24}}{{\cos }^3}\left( 0 \right)\left( 0 \right) + \frac{{15}}{{48}}\cos \left( 0 \right)\sin \left( 0 \right) + \frac{{15}}{{48}}\left( 0 \right)} \right] \cr
& = \frac{5}{{32}}\pi \cr} $$