The thing about exorsexism is that it's ingrained in our language and culture. We lack the language to describe being non/abinary/genderqueer because our culture enforces a binary. So I generally support any efforts to improve exorsexism in language, even if I personally wouldn't use it.
It's true that "nonbinary" does not refer to one single gender. But I would argue that "woman" and "man" are vaster categories than we give them credit for. There are people who are attracted to certain kinds of man/womanhood performance but not others, especially when it comes to queerness & ethnic cultures. Not every lesbian is necessarily attracted to the same womanhood performance, and the same goes for gay men & straight men & straight women.
And "woman" and "man" are also not synonymous with presentation. You cannot tell how someone identifies by how they present themselves. But we do use gender coding to call on cultural associations & send a message of what gender stuff we wanna be associated with. A gay trans man may dress much differently than a straight trans man, because he is trying to perform gay manhood. Similarly, an agender person may also perform things associated with gay manhood even if they don't identify at all as a man, because they want queer men to notice them. But the same things could also be done by a straight man who just enjoys things traditionally associated with gay manhood And, going off that, there are ways that nonbinary/genderqueer people signal ourselves as NB/GQ as well. Someone with brightly dyed hair, wearing lipstick with a mustache, in a skirt with a dress shirt, is performing a genderfuck-hood that can signal genderqueerness to others. Personally, I like presenting in a purposefully genderqueer way so that other queer people recognize me as "family" and will be attracted to me because of my genderqueer performance.
So, considering all that... I appreciate language that let's us express love and attraction and appreciation for genderqueerness and non/abinarity. Arguments against ceterosexuality/skoliosexuality tends to go back to "nonbinary isn't a third gender, and some nonbinary people are okay being aligned with a gender, so anything that ever centers nonbinary people and recognizes abinarity is transphobic!!!" which always leaves a bad taste in my mouth. I've seen some people say you can use them but only Ina t4t way, which I also dislike; I am t4t and gq4gq but I don't like the idea that people who are cis aren't "allowed" to express attraction towards abinarity/genderqueerness because they must be chasers.
Also, I have the theory that genderqueer language always faces more stigma and is held to a higher standard than binary language. Partially because it's newer, but also because being genderqueer/nonbinary is viewed by transphobic society as unserious and ridiculous, and has also been associated with people assigned female & "weird femininity," which brings in misogyny (even moreso than transphobia already does). This is not to say people who personally don't want to use genderqueer language are doing anything wrong– but I feel like everytime someone comes up with a term to improve the lack of nonbinary visibility in language, it is immediately lambasted for being "cringe" and "infantilizing" and "just call me a slur" which I feel, on some level, comes from the association of binary things with normality & neutrality & adulthood, and nonbinary things with childishness & queerness.
The end goal of exorsexism is to smother any sign of sex/gender transgression it finds. I feel like the criticisms of nonbinary/genderqueer-focused sexuality are just another expression of this, but done in a "progressive" way. The underlying message is that nonbinary/genderqueer gender isn't as real as binary gender; our validity as people only comes from our identification with binary systems on some level; NB/GQ people always confirm to binary gender performance and we have no ways of communicating our genderqueerness on purpose; attraction to NB/GQ people must be dirty and objectifying and it can only be expressed via the language of binary attraction, even if the object of your affection is hurt by that.