Answer
The big rock has more inertia.
Work Step by Step
Mass is a measure of inertia. Your foot must exert more force on the bigger rock to get it moving, and in turn, the bigger rock exerts a larger force on your foot. Yes, it will hurt more to kick the larger rock.