The Speaker
The speaker has a descriptive and voyeuristic relationship to the world. Though the speaker addresses her beloved in the second person, she is unable to speak directly to her and instead watches in awe the man who is actually able to do so. The speaker is dramatically enamored with that beloved and finds herself, paradoxically, unable to speak, unable to exist as a body, almost unable to live, because of how strong her passion is when she looks at her beloved.
The man
The only character whose gender is specified explicitly, the man is introduced in the very first line, before the subject of the poem, the speaker's lover. He is a foil to the speaker because he sits near to the beloved and "listens close" to her speech, while the speaker is far from her beloved and struck dumb with the thought of her.
The beloved
The speaker doesn't tell us a lot of specifics about this beloved, and focuses rather on her own emotional response to seeing her from afar. The speaker does emphasize her beloved's "sweet" and "lovely" voice and laughter, and we know that the speaker, at least, finds her to be extremely beautiful, because she is struck dumb whenever she looks at her.