Something important to keep in mind;
Empathy = You know what a person is feeling. If you see someone stuck outside in the cold you can imagine how terrible that must be and how much it would suck to be in that situation. Maybe you were once stuck in the cold yourself and can draw on that experience.
Sympathy = You don't know what a person is feeling but you believe them when they tell you. Maybe you've never been stuck out in the cold before and can't imagine what it's like, but when they explain how much it sucks you accept that they are suffering. You believe that they are having a bad time even if what they're going through isn't something you yourself can understand.
Compassion = Regardless of how you know, be it through empathy, sympathy or otherwise, you are aware that the person in front of you is cold. Acting on this information, you give them a blanket and direct them somewhere warm.
The presence of empathy does not make someone a good person, and the lack of empathy does not make someone bad.
Empathetic people can still be cruel; being able to tell at a glance that someone is suffering doesn't stop an empathetic person from deciding that the victim must deserve it.
Unempathetic people can still be kind; being unable to tell that someone is suffering doesn't prevent them from trying to help, and it doesn't prevent them from saying, "wow, that sucks. I'm sorry you're having a bad time."
Empathy has no bearing on a person's ability to love and care for others, and it should not be a method of judging whether someone is safe or not.