Answer
(a) we show that the net force is
$F = k\rho Q\mathop \smallint \limits_{\theta = - \pi /2}^{\pi /2} \mathop \smallint \limits_{r = a}^b \left( {\frac{{\cos \theta }}{{{r^2}}}} \right)r{\rm{d}}r{\rm{d}}\theta $
(b) $F = 2k\rho Q\ln \frac{b}{a}$
Work Step by Step
(a) Let ${\cal R} \subset {\cal D}$ be a small polar rectangle of size $\Delta r \times \Delta \theta $ located at distance $r$ from the origin.
Since the charge density $\rho$ is uniform, the charge of the small polar rectangle of size $\Delta r \times \Delta \theta $ is
$q = \rho r\Delta r\Delta \theta $
The particle $P$ at the origin has charge $Q$ coulombs. Thus, by Coulomb's Law, the small polar rectangle exerts a force on $P$ given by
${F_P} = k\frac{{qQ}}{{{r^2}}}$
Since ${\cal D}$ is symmetric with respect to the $x$-axis, the vertical components of force on $P$ cancel out. Thus, the force on $P$ acts in the horizontal direction, namely along the $x$-axis.
To find the horizontal force ${F_x}$ of ${F_P}$, we project ${F_P}$ onto the $x$-axis. Thus,
${F_x} = {F_P}\cos \theta = k\frac{{qQ}}{{{r^2}}}\cos \theta $
Since $q = \rho r\Delta r\Delta \theta $, so
${F_x} = k\rho Q\left( {\frac{{\cos \theta }}{{{r^2}}}} \right)r\Delta r\Delta \theta $
To find the total force on $P$, we decompose the annular region ${\cal D}$ into an $N \times M$ grid of small polar rectangles ${{\cal R}_{ij}}$. Now, each ${{\cal R}_{ij}}$ exerts a force $k\rho Q\left( {\frac{{\cos \theta }}{{{r^2}}}} \right){r_{ij}}\Delta r\Delta \theta $ on $P$. Thus, the total force by ${\cal D}$ is the Riemann sum:
$k\rho Q\mathop \sum \limits_{i = 1}^N \mathop \sum \limits_{j = 1}^M \left( {\frac{{\cos \theta }}{{{r^2}}}} \right){r_{ij}}\Delta r\Delta \theta $
As $N,M \to \infty $, we obtain the double integral for the total force on $P$:
$F = k\rho Q\mathop \smallint \limits_{\theta = - \pi /2}^{\pi /2} \mathop \smallint \limits_{r = a}^b \left( {\frac{{\cos \theta }}{{{r^2}}}} \right)r{\rm{d}}r{\rm{d}}\theta $
(b) Compute F:
$F = k\rho Q\left( {\mathop \smallint \limits_{\theta = - \pi /2}^{\pi /2} \cos \theta {\rm{d}}\theta } \right)\left( {\mathop \smallint \limits_{r = a}^b \frac{1}{r}{\rm{d}}r} \right)$
$ = k\rho Q\left( {\sin \theta |_{ - \pi /2}^{\pi /2}} \right)\left( {\ln r|_a^b} \right)$
$ = 2k\rho Q\left( {\ln b - \ln a} \right)$
So, $F = 2k\rho Q\ln \frac{b}{a}$.