It's a rare non-transsexual person who can truly have empathy for someone who was born with transsexualism. It's probably easier to imagine having been born with just about any other congenital condition.
One of the problems that the "out and proud" cause those who have dealt with their birth defect (or are in the process of doing so) is that people think they can speak freely about the condition. Yet would someone say, "I remember when you had cleft lip," or "I saw this movie about someone with cleft lip"? Would a discussion of your birth defect be considered a topic for casual conversation? But people do that about transsexualism.*
They don't get it. And given just how unusual this condition is, they really shouldn't be expected to get it.
At the same time, those who have put their birth defect behind them have to protect themselves going forward. Those little casual slips and intrusions hurt, and they hurt all the more because the one who makes them doesn't know how hurtful the slips and intrusions are. It's no wonder some people born transsexual still try to go stealth, or at the very least avoid those who knew them before. Many if not most of those people will never be able to get the wrong image and however many years of memories out of their heads. Even if they never slip, even if they never say anything inappropriate, they will probably never see you as you really are.
To some extent, how well a person who knew you before will do probably depends on how present-focused they are. Those for whom past is past might do better than those who are very tied to old memories. Some people see what is in front of them. Some people can never get beyond what they used to see.
Just as most people don't understand how you feel, they probably won't understand if you avoid them or cut them off. They'll probably feel hurt, because they're sure that they're doing all the right things. But unless they are extraordinary human beings, strongly observant and very present-focused, they will continue to hurt you unintentionally.
*To be fair, people do similar things to those with other conditions. I recently saw a list of things that hearing people say to those who are hearing impaired. It was appalling.