I don't think you can know which members said what about you in reputation unless they put their name in the comment. When I give rep, good or bad, I always put my name just because I want them to know it was me. Of course, I could be wrong, and there may be some way to figure it out. Maybe a mod knows? Edited to say: To check your rep comments, click on User CP at the top of the page.
Maybe you don't have any rep comments? I'll give you one now! Edited: Oh wait, you said someone gave you one... Hmm... I'll still give you one, anyway!