"In those years, people will say, we lost track"
Rich's poetry often focuses on political and societal structures. This first line imagines a future where people will look back on their past and realize that they were lost in their isolation into "I". Therefore, "those years" is a paradoxical description of the present in which the poet is writing the poem. Looking at it in this way, the poem seems to have a rather hopeful tone, as the speaker believes that those in the future will be able to recognize the failings of her contemporary moment (the 1990s).
"through the rags of fog /where we stood, saying I"
To be able to progress into the future, we must look back to the past. This is what the second stanza of the poem is saying with the metaphorical image of the dark birds of history. "Rags of fog" is a barrier we make so that others are unable to reach us, but the great dark birds still manage to penetrate that barrier with their beaks and pinions and remind us that seeing only the "I" has never quite worked out.
"we were trying to live a personal life/ and yes, that was the only life/ we could bear witness to"
Rich explains that, in the historical era she discusses, people could only bear witness to their own personal lives. The two small words "and yes" suggest an admission of guilt, as well as perhaps exasperation, as though people felt that was all they had the bandwidth for. Either way, though, this has negative effects, and people are unable to bear witness to others, and thus they let suffering go unacknowledged.