"It seems to me then as if all the moments of our life occupy the same space, as if future events already existed and were only waiting for us to find our way to them at last, just as we have accepted an invitation we duly arrive in a certain house at a given time."
To Austerlitz, the past is his obsession. It's strange to hear him speak of the future in such a familiar way because he rarely lives it. He reveals his perspective here, though, explaining that he profoundly believes in fate. Anything that will be is already destine to occur in just the right way at just the right time.
". . .the darkness does not lift but becomes yet heavier as I think how little we can hold in mind, how everything is constantly lapsing into oblivion with every extinguished life, how the world is, as it were, draining itself, in that the history of countless places and objects which themselves have no power or memory is never heard, never described or passed on."
Memory is a tricky subject for Austerlitz. He experienced a lot in his young childhood, but he couldn't remember the majority of it because of the radical change of direction which happened to him when he was sent to Wales. In his experience, memory is a muscle. If left unused for too long, it'll fade away into nothingness. All those stories will die if people don't tell them.
"No one can explain exactly what happens within us when the doors behind which our childhood terrors lurk are flung open."
As he watches Austerlitz' personal journey into the past, the narrator learns a great deal. It's painful to see. It makes him think of his own history and what he'd really find if he bothered to research his past extensively.
"I felt that the decrepit state of these once magnificent buildings, with their broken gutters, walls blackened by rainwater, crumbling plaster revealing the coarse masonry beneath it, windows boarded up or clad with corrugated iron, precisely reflected my own state of mind. . ."
Austerlitz knows that he had so much richness of life that was stolen from him. To him, it feels like his present is a crumbling old building. He's weathered too many storms. Whatever he's feeling on the inside, it's tired and worn.