if you must use wireless, then you must have a dedicated access point and setup QoS for all the voice traffic. Voice traffic is "real time" traffic transmitted using UDP which means that if a packet is lost or delayed in transmission, it is simply dropped as there is no time to retransmit the data (the time to play the "sound" it represents has past!) which means you immediately hear the effects of traffic congestion.
You need to have a wired NIC on the internal interface of your Core, connected to a switch that does some form of QoS, then an access point connected to that, that also does QoS. Once QoS is configured, you will at least then stand a chance that other internal network traffic may not kill your VoIP traffic. Also try different codecs. I got my SIP wireless handset working quite well even under network traffic loads... but don't expect it to be bullet proof!