Answer
Both methods come out with the result $$F'(x)=2x-5-\frac{3\sqrt x}{2x^3}$$
Work Step by Step
$$F(x)=\frac{x^4-5x^3+\sqrt x}{x^2}$$
1) Using the Quotient rule: $$F'(x)=\frac{x^2\frac{d}{dx}(x^4-5x^3+\sqrt x)-(x^4-5x^3+\sqrt x)\frac{d}{dx}(x^2)}{(x^2)^2}$$
$$F'(x)=\frac{x^2(4x^3-15x^2+\frac{1}{2}x^{-1/2})-(x^4-5x^3+\sqrt x)\times2x}{x^4}$$
$$F'(x)=\frac{4x^5-15x^4+\frac{1}{2}x^{3/2}-2x^5+10x^4-2x\sqrt x}{x^4}$$
$$F'(x)=\frac{2x^5-5x^4+\frac{1}{2}x\sqrt x-2x\sqrt x}{x^4}$$
$$F'(x)=2x-5+\frac{-\frac{3}{2}x\sqrt x}{x^4}$$
$$F'(x)=2x-5-\frac{3\sqrt x}{2x^3}$$
2) Simplifying first: $$F(x)=x^2-5x+\frac{x^{1/2}}{x^2}$$
$$F(x)=x^2-5x+x^{-3/2}$$
So, $$F'(x)=\frac{d}{dx}(x^2-5x+x^{-3/2})$$
$$F'(x)=2x-5-\frac{3}{2}x^{-5/2}$$
$$F'(x)=2x-5-\frac{3}{2}(x^{1/2}\times x^{-3})$$
$$F'(x)=2x-5-\frac{3\sqrt x}{2x^3}$$
The results are equivalent.
Which method one prefers is of personal choice. However, in this exercise, I prefer the second method.