What is the meaning of the phrase “unfair which fairly doth excel” as it is used in Shakespeare’s “Sonnet 5”?

The speaker is referring to the paradox of time. Although time sculpted the lovely face of the speaker's muse, time will surely destroy it.