Answer
$$\int x^2e^{-x}dx=-x^2e^{-x}-2xe^{-x}-2e^{-x}+C$$
Work Step by Step
$$A=\int x^2e^{-x}dx$$
Set $u=x^2$ and $dv=e^{-x}dx$
Then we would have $du=2xdx$ and $v=-e^{-x}$
Using the formula $\int udv= uv-\int vdu$:
$$A=-x^2e^{-x}-\int(-e^{-x})2xdx$$ $$A=-x^2e^{-x}+2\int xe^{-x}dx$$
Set $u=x$ and $dv=e^{-x}dx$
Then we would have $du=dx$ and $v=-e^{-x}$
Using the formula $\int udv= uv-\int vdu$:
$$A=-x^2e^{-x}+2\Big(-xe^{-x}-\int-e^{-x}dx\Big)$$ $$A=-x^2e^{-x}+2\Big(-xe^{-x}+\int e^{-x}dx\Big)$$ $$A=-x^2e^{-x}+2\Big(-xe^{-x}-e^{-x}\Big)+C$$ $$A=-x^2e^{-x}-2xe^{-x}-2e^{-x}+C$$