If you emphasize on spiritual aspect of yoga then I am afraid,non hindus will object to it. Best way is to present it as exercise for the well being of mind and body.
The only spiritual aspect in yoga is chanting of OM. Muslims can chant whatever they feel like. This way both Hindus and muslims can enjoy the spiritual aspect of yoga too.
There is much more than chanting just OM in Yoga,OM is just 5-10% of the spiritual aspect Do you know about Kundalini shakti and how it rises,falls or is controlled etc.
These are all part of Maharishi Patanjali Yoga Sutras who formulated the rules of Yoga for the first time.
It is upto the Non-Hindus if they want to object,it's not like we are imposing it on them are we?They are coming to learn Yoga.
The non-Hindus should give credit where its due, they can't take our Yoga and say we want to remove Hinduism,Spirituality aspect from Yoga , because Bharat is the birthplace of Yoga invented by our Hindu Ancestors who toiled Thousands of years to know its secrets.
Just like a trademark and patent, you are supposed to follow some rules not remove or add as you wish without even paying the royalty.
The Non-Hindus have a bad habit of taking everything from other nations and then claiming they invented that this is true in case of Western Civilizations,that is how their history is, they want to Patent Ayurveda,Yoga,Basmati rice,Turmeric,Neem and then call it as a Invention of West. We had enough of these Westerners looting our ideas,concepts, products through the ages.