ZWave is designed to route messages between the devices, so if one device is unplugged or there is a temporary radio communication problem, you will get a failed transmission. By default, a failed transmission will cause the ZWave driver to delay, to allow the network time to re-route the message. If the message is still failed, it will try to re-send it, and once again cause a delay if it fails. This delay can be tuned up or down, but experience has shown that some USB sticks require a high delay(30-40s) or they will lock up and need to be unplugged and replugged.
The best way to avoid this is to make sure your devices all know about their neighbour devices, so that they are well connected. (The messages are routed through a device's known neighbours). If you use LMCE 1204, there is the new Advanced ZWave page (in the Automation menu) that will display this clearly and provide commands to fix such problems (Heal Node / Heal Network).
Adding more devices between the usb controller and the problematic device can also help better routing to that device.