The Wall's magic and it's effect on familiars
The Wall, the largest structure known to man, is a conglomeration of ice, magic, stone, blood (!), and something eerily alive. It wasn't actually built by men, but a legendary line-up of three races who had been at war with each other for centuries prior.
Whether the legends are true or not, it is plain that the First Men and the children of the forest (and even the giants, if we take the word of the singers) feared something enough that it drove them to begin raising the Wall. And this great construction, as simple as it is, is justly accounted among the wonders of the world.
-TWOIAF: The Night's Watch
According to Ygritte, the wildlings have a different view of it (although of course the threats of the Others is greatest against them):
"You know nothing, Jon Snow. Daughters are taken, not wives. You're the ones who steal. You took the whole world, and built the Wall t' keep the free folk out."
Regardless the point of this post is about how the Wall manages to weaken connections between humans and their animal familiars, whether these be wargs or dragonriders as we've seen:
The direwolf had run off three times as they climbed, twice returning reluctantly to Jon's whistle. The third time, the Lord Commander lost patience and snapped, "Let him go, boy. I want to reach the crest before dusk. Find the wolf later."
But when they reached the ringfort, Ghost balked again. He padded forward warily to sniff at the gap in the stones, and then retreated, as if he did not like what he'd smelled. Jon tried to grab him by the scruff of his neck and haul him bodily inside the ring, no easy task; the wolf weighed as much as he did, and was stronger by far. "Ghost, what's wrong with you?" It was not like him to be so unsettled. In the end Jon had to give it up. "As you will," he told the wolf. "Go, hunt." The red eyes watched him as he made his way back through the mossy stones.
Though it was summer and the Wall was weeping, the chill of the ice could still be felt whenever the wind blew, and every gust would make the dragon hiss and snap. “Thrice I flew Silverwing high above Castle Black, and thrice I tried to take her north beyond the Wall,” Alysanne wrote to Jaehaerys, “but every time she veered back south again and refused to go. Never before has she refused to take me where I wished to go. I laughed about it when I came down again, so the black brothers would not realize anything was amiss, but it troubled me then and it troubles me still.”
-Fire & Blood: Jaehaerys & Alysanne
Is it that the dragon couldn't go, or wouldn't? Ghost certainly could but direwolves still exist beyond the Wall (per Benjen) as do wargs of whom most commonly warg into wolves/direwolves. The Wall doesn't sever these ties because Alysanne continues to ride Silverwing and Ghost returns to Jon.
“Gods, wolf, where have you been?” Jon said when Ghost stopped worrying at his forearm. “I thought you’d died on me, like Robb and Ygritte and all the rest. I’ve had no sense of you, not since I climbed the Wall, not even in dreams.”
Notably Arya experiences wolf dreams with Nymeria all the way from Jupiter so the Wall is the factor of separation, not the distance.
Varamyr also steers Orell's eagle above the Wall and there seems to be no problem (although he's killed by Melisandre's magic):
"Once a beast's been joined to a man, any skinchanger can slip inside and ride him. Orell was withering inside his feathers, so I took the eagle for my own. But the joining works both ways, warg. Orell lives inside me now, whispering how much he hates you. And I can soar above the Wall, and see with eagle eyes."
What I think is interesting is that the Wall is built on magic from men, the Children (who forced the separation of the Arm of Dorne & the Neck), the First Men, and giants alike. It locks away the Others, presumably, but it also seems to have mutual safeguards against each other built in. Whatever Coldhands is, he can't travel to the other side of the Wall. The Black Gate that Bran travels through only allows Black Brothers in and out. Giants can only enter through one door, too large and heavy to climb, and that door is right beside Castle Black. The Wall defends itself, the Wall has moods of it's own, the Wall is weeping. These are all Jon Snow's descriptions of the beast.
This is where we get into my personal theory.
The King's Crown was the Cradle, to hear her tell it; the Stallion was the Horned Lord; the red wanderer that septons preached was sacred to their Smith up here was called the Thief.
"It is not that we fear these barbarians," Illyrio would explain with a smile. "The Lord of Light would hold our city walls against a million Dothraki, or so the red priests promise… yet why take chances, when their friendship comes so cheap?"
"Ghost." Melisandre made the word a song.
The direwolf padded toward her. Wary, he stalked about her in a circle, sniffing. When she held out her hand he smelled that too, then shoved his nose against her fingers.
Jon let out a white breath. "He is not always so…"
"…warm? Warmth calls to warmth, Jon Snow." Her eyes were two red stars, shining in the dark. At her throat, her ruby gleamed, a third eye glowing brighter than the others. Jon had seen Ghost's eyes blazing red the same way, when they caught the light just right. "Ghost," he called. "To me."
The direwolf looked at him as if he were a stranger.
Jon frowned in disbelief. "That's… queer."
"You think so?" She knelt and scratched Ghost behind his ear. "Your Wall is a queer place, but there is power here, if you will use it. Power in you, and in this beast. You resist it, and that is your mistake. Embrace it. Use it."
The magic in the Wall is a mix of things, but the fire magic Melisandre wields is clearly part of it. Her touching Ghost mimics the separation effect Jon experiences when Ghost is beyond the Wall. It's one of the "hinges of the world" according to her, as in, it both strengthens her and... confuses her.
She was stronger at the Wall, stronger even than in Asshai. Her every word and gesture was more potent, and she could do things that she had never done before.
Melisandre had practiced her art for years beyond count, and she had paid the price. There was no one, even in her order, who had her skill at seeing the secrets half-revealed and half-concealed within the sacred flames.
Yet now she could not even seem to find her king. I pray for a glimpse of Azor Ahai, and R'hllor shows me only Snow.
Melisandre is as at home with her magic as Jon is with Ghost (Ghost is a part of me). Yet both of them feel those bonds changed by the magic that exists in the Wall, just like Alysanne Targaryen did with Silverwing.
The Magnar nodded. "Even in far Thenn we know the tale of Arson Iceaxe and his tunnel."
Jon knew the tale as well. Arson Iceaxe had been halfway through the Wall when his tunnel was found by rangers from the Nightfort. They did not trouble to disturb him at his digging, only sealed the way behind with ice and stone and snow.
Here's something both wildlings and Northerners know: the Wall will swallow you whole just to keep you out.
The Wall is an exclusionary force by nature. Ayesha A. Siddiqi said: every border implies the violence of it's maintenance. The magic in it isn't only about the Others; it creates binaries on every axis and then tears the two apart - Northerners from wildlings, human from animal, Others from humans. To even join the Night's Watch as stewards of the Wall, you have to swear off all family, effectively severing yourself.
The Wall survives on the chaos of separation because that's how it defends itself. It was raised by a coalition never seen before and it refuses to let that happen again.
The Wall is alive and that is a threat.