Answer
The given system is inconsistent for $a=1\text{ or }a=3$.
Work Step by Step
The augmented matrix for the given system will be,
$\left[ \begin{matrix}
1 & 3 & 1 \\
2 & 5 & 2a \\
1 & 1 & {{a}^{2}} \\
\end{matrix}\left| \begin{matrix}
{{a}^{2}} \\
0 \\
-9 \\
\end{matrix} \right. \right]$
Now the necessary condition for inconsistency is that the determinant should be zero,
So,
$\left| \begin{matrix}
1 & 3 & 1 \\
2 & 5 & 2a \\
1 & 1 & {{a}^{2}} \\
\end{matrix} \right|=0$
Solve the determinant:
Expanding along row 1,
$\begin{align}
& 1\left( 5{{a}^{2}}-2a \right)-3\left( 2{{a}^{2}}-2a \right)+1\left( 2-5 \right)=0 \\
& 5{{a}^{2}}-2a-6{{a}^{2}}+6a-3=0 \\
& -{{a}^{2}}+4a-3=0 \\
& {{a}^{2}}-4a+3=0
\end{align}$
Solve the above quadratic equation as below:
$\begin{align}
& {{a}^{2}}-3a-a+3=0 \\
& a\left( a-3 \right)-\left( a-3 \right)=0 \\
& \left( a-3 \right)\left( a-1 \right)=0 \\
& a=1,3
\end{align}$
So, the equation may be inconsistent for $a=1\text{ or }a=3$
When, $a=1$
$\begin{align}
& x+3y+z=1 \\
& 2x+5y+2z=0 \\
& x+y+z=-9 \\
& \text{Augmented matrix,} \\
\end{align}$
$\left[ \begin{matrix}
1 & 3 & 1 \\
2 & 5 & 2 \\
1 & 1 & 1 \\
\end{matrix}\left| \begin{matrix}
1 \\
0 \\
-9 \\
\end{matrix} \right. \right]$
Using Gaussian elimination,
$\text{Replace row 2 by }{{\text{R}}_{2}}-2{{R}_{1}},$
$\left[ \begin{matrix}
1 & 3 & 1 \\
0 & -1 & 0 \\
1 & 1 & 1 \\
\end{matrix}\left| \begin{matrix}
1 \\
-2 \\
-9 \\
\end{matrix} \right. \right]$
$\begin{align}
& \text{Replace row 2 by -1}{{\text{R}}_{2}},\text{ row 3 by }{{\text{R}}_{3}}-{{R}_{1}} \\
& \left[ \begin{matrix}
1 & 3 & 1 \\
0 & 1 & 0 \\
0 & 1 & 0 \\
\end{matrix}\left| \begin{matrix}
1 \\
2 \\
-10 \\
\end{matrix} \right. \right] \\
\end{align}$
$\begin{align}
& \text{Replace row 3 by }{{\text{R}}_{3}}-{{R}_{2}}, \\
& \left[ \begin{matrix}
1 & 3 & 1 \\
0 & 1 & 0 \\
0 & 0 & 0 \\
\end{matrix}\left| \begin{matrix}
1 \\
2 \\
-12 \\
\end{matrix} \right. \right] \\
\end{align}$
The equation will be
$\begin{align}
& x+3y+z=1 \\
& y=2 \\
& 0x+0y+0z=-12
\end{align}$
So, the last equation can never be a true statement; hence, the system has no solution or the system is inconsistent.
When $a=3$
$\begin{align}
& x+3y+z=9 \\
& 2x+5y+6z=0 \\
& x+y+9z=-9
\end{align}$
Augmented matrix,
$\left[ \begin{matrix}
1 & 3 & 1 \\
2 & 5 & 6 \\
1 & 1 & 9 \\
\end{matrix}\left| \begin{matrix}
9 \\
0 \\
-9 \\
\end{matrix} \right. \right]$
Using Gaussian elimination,
$\begin{align}
& \text{Replace row 2 by }{{\text{R}}_{2}}-2{{R}_{1}}, \\
& \left[ \begin{matrix}
1 & 3 & 1 \\
0 & -1 & 4 \\
1 & 1 & 9 \\
\end{matrix}\left| \begin{matrix}
9 \\
-18 \\
-9 \\
\end{matrix} \right. \right] \\
\end{align}$
$\begin{align}
& \text{Replace row 2 by -1}{{\text{R}}_{2}},\text{ row 3 by }{{\text{R}}_{3}}-{{R}_{1}} \\
& \left[ \begin{matrix}
1 & 3 & 1 \\
0 & 1 & -4 \\
0 & -2 & 8 \\
\end{matrix}\left| \begin{matrix}
9 \\
18 \\
-18 \\
\end{matrix} \right. \right] \\
\end{align}$
$\begin{align}
& \text{Replace row 3 by }{{\text{R}}_{3}}+2{{R}_{2}}, \\
& \left[ \begin{matrix}
1 & 3 & 1 \\
0 & 1 & -4 \\
0 & 0 & 0 \\
\end{matrix}\left| \begin{matrix}
9 \\
18 \\
18 \\
\end{matrix} \right. \right] \\
\end{align}$
Therefore, the equation will be,
$\begin{align}
& x+3y+z=-9 \\
& y-4z=18 \\
& 0x+0y+0z=18 \\
\end{align}$
So, the last equation can never be a true statement; hence, the system has no solution or the system is inconsistent.
So, the system is inconsistent for $a=1\text{ or }a=3$