Answer
$F$ is a continuous function of $r$.
Work Step by Step
Using the functions of the problem:
$\lim\limits_{r \to R^{-}} F(r) = \frac{GM(R)}{R^{3}} = \frac{GM}{R^{2}}$
$\lim\limits_{r \to R^{+}} F(r) = \frac{GM}{r^{2}} = \frac{GM}{R^{2}}$
Both limits as they approach $R$ from the left and right side are equal, so that means that $\lim\limits_{r \to R} F(r) = \frac{GM}{R^{2}}$ is continuous.
