I feel back for my kitten Ollie... what do I do?

Ollie is kind of anti-social in comparison to his brother Bean. They are from the same litter and loves to play with each other. Bean is snuggly and wants to be pet and loved all the time. But Ollie doesn't want to be pet, or snuggle or show any signs of affection. I am afraid that I scarred Ollie by mistake. They are about 12 weeks and Ollie was in the kitchen with me and I didn't realize he was behind the fridge door and I mistakenly hit him in the eye with the fridge door. I felt so badly and picked him up and tried to make him feel better, any way that I could (I even cried about it) but he seemed fine. He was playing around with his brother and totally aloof to the injury. I am afraid however that he has yet to forgive me, and I have lost his trust. What should I do?