No, it actually would have been really funny if Cas had brought both Sam and Adam’s soulless bodies back in season 6. You know, as a sort of cover in case Dean ever came sniffing around. Cas would just stand there with a straight face, claiming the cage must have just spat them both back out because it wasn’t made to hold humans.
Meanwhile Sam and Adam are roaming around being the most amoral stoic people ever, creeping Samuel out, and Dean would momentarily wonder if this is just how Adam is as a person, because he never got to know his youngest brother, and for all he knows Adam’s just rubbing off on Sam.
Later, Dean will still choose Sam to get his soul back when he has his meeting with Death. Largely because he really can’t tell the difference between soulless Adam and Adam-Adam, and the rest of the series would just have Sam and Dean constantly bickering over whose turn it is to babysit Adam, who’ll just be there wandering around inadvertently causing havoc in the background of their cases.
Soulless Adam and Amara meeting would be so funny!
The Winchesters are running away because she might eat their souls, and Adam is just standing there. Then Amara would squint at him, talking out loud, “Did I already…?”
Meanwhile in the impala, Sam and Dean are fleeing, when Dean suddenly hits the breaks. “WE FORGOT ADAM!”
“AGAIN?”
Rest of the season, Adam’s just chilling in the background of Amara’s scenes.
And come season 15, this could have allowed for some fun foreshadowing for Michael.
Chuck snaps his fingers in season 14, and we the audience see Adam just sort of freeze or startle in the background, as if he’s experiencing something no one else is. Then staright away in season 15, Adam’s sort of acting different. But no one really notices at first because there’s so much else going on with the escaped souls from Hell.
But then they capture Michael in a ring of fire, and as they’re standing there giving their little “you’re coming with us” speech, Adam comes in with a fire extinguisher, and we see Michael’s face light up.
Because Michael thought he’d lost Adam forever when they got separated in the crowd of paranormal entities fleeing Hell, and with those symbols that Cas carved on Adam’s bones back in season 5 still in place, Michael had been flying all over the world trying to find his little human friend.
(Naturally, Michael swoons on the spot, because he is not accustomed to being rescued.)