Robots that express human’s social norms, like empathy, are perceived as more friendly, understanding, and caring. However, appropriate human-like empathic behaviors cannot be defined in advance, instead, they must be learned through daily interaction with humans in different situations. Additionally, to learn and apply the correct behaviors, robots must be able to perceive and understand the affective states of humans. This study presents a framework to enable cognitive empathy in social robots, which uses facial emotion recognition to perceive and understand the affective states of human users. The perceived affective state is then provided to a reinforcement learning model to enable a robot to learn the most appropriate empathic behaviors for different states. The proposed framework has been evaluated through an experiment between 28 individual humans and the humanoid robot Pepper. The results show that by applying empathic behaviors selected by the employed learning model, the robot is able to provide participants comfort and confidence and help them enjoy and feel better.