The genius of Roosevelt's speech is that by bringing attention to the people's moral obligation to work hard in life, he manages to change the mood of his speech. He maintains authority and power, while defending himself against the common cultural complaints. Roosevelt is defending himself to a populace who is highly annoyed with him, and most people are still rattling from the Great Depression. Roosevelt ends his speech with the most important part, mentioning that America's role in WWII, although highly unpopular, is both moral and right.
By bringing attention to the question of hard work, Roosevelt seems to turn the question on its head. Typically, people would criticize the president for a bad economy, but if one thinks through Roosevelt's arguments completely, he seems to be criticizing the people who could work, but don't. He reminds his people that even if they cannot work with their bodies, because they are averse to manual labor, or unable to do it, that they can work in the arts and sciences.
He encourages the American people to consider life as a toil, and he turns that into an ethical question, suggesting that those who work hard in life are inherently blessed, and those who do not work hard in life do not see the same meaning in life that those who work hard often do. Therefore, his argument is that if a person could have riches and luxury, it would bring him less value than if he just worked hard in life and made a destiny for himself. Then, subtly, Roosevelt turns that into a political statement, saying that he is doing with America's army what he encourages Americans to do: taking the more meaningful path, even if it's difficult.