Incest is shameful regardless of whether it is executed.
Although James might have seemed fairly sly to himself, the truth is that Materia realized he had sexual feelings for his daughter way before anything actually happens. This causes a sense of shame and confusion within the family.
Embarrassment can be a destructive emotion.
By hiding his attractions from his wife, James only exacerbates his isolation in life, and he bears the full weight of his shame instead of seeking help for his mental health issues, since sexual perversion is often associated with serious mental health issues.
