Answer
The global minimum is $f\left( {1,2} \right) = - 5$ and the global maximum is $f\left( {0,0} \right) = f\left( {3,3} \right) = 0$.
Work Step by Step
We have $f\left( {x,y} \right) = {x^2} + {y^2} - 2x - 4y$ on the domain $x \ge 0$, $0 \le y \le 3$, $y \ge x$.
Step 1. Find the critical points on the domain and evaluate $f$ at these points
The partial derivatives are
${f_x} = 2x - 2$, ${\ \ \ }$ ${f_y} = 2y - 4$
To find the critical points we solve the equations ${f_x} = 0$ and ${f_y} = 0$:
${f_x} = 2x - 2 = 0$, ${\ \ }$ ${f_y} = 2y - 4 = 0$
So, there is only one critical point at $\left( {1,2} \right)$. The extreme value of $f$ is $f\left( {1,2} \right) = - 5$.
Step 2. Check the boundaries
We restrict the function $f$ along the edges and find the minimum and maximum values. The results are given in the following table:
$\begin{array}{*{20}{c}}
{}&{{\rm{Restriction{\ }of}}}&{}&{f\left( {x,y} \right)}&{f\left( {x,y} \right)}\\
{{\rm{Edge}}}&{f\left( {x,y} \right){\rm{to{\ }Edge}}}&{}&{{\rm{min}}{\rm{.}}}&{{\rm{max}}{\rm{.}}}\\
{Top:0 \le x \le 3,y = 3}&{h\left( x \right) = f\left( {x,3} \right) = {x^2} - 2x - 3}&{h' = 2x - 2}&{ - 4}&0\\
{Left:x = 0,0 \le y \le 3}&{m\left( y \right) = f\left( {0,y} \right) = {y^2} - 4y}&{m' = 2y - 4}&{ - 4}&0\\
{Right:0 \le x \le 3,y = x}&{n\left( x \right) = f\left( {x,x} \right) = 2{x^2} - 6x}&{n' = 4x - 6}&{ - \frac{9}{2}}&0
\end{array}$
Notes:
1. The critical point of $h\left( x \right)$ is $x=1$, so the extreme value $h\left( 1 \right) = - 4$.
2. The critical point of $m\left( y \right)$ is $y=2$, so the extreme value $m\left( 2 \right) = - 4$.
3. The critical point of $n\left( x \right)$ is $x = \frac{3}{2}$, so the extreme value $n\left( {\frac{3}{2}} \right) = - \frac{9}{2}$.
Step 3. Compare the results
Comparing the values of $f$ in Step 1 and the values in this table, we obtain the global minimum $f\left( {1,2} \right) = - 5$ and the global maximum $f\left( {0,0} \right) = f\left( {3,3} \right) = 0$.