Problem to Solve – In my VoIP environment, we have users complaining that they cannot hear the person on the other end of the call.    How can I troubleshoot  “One Way Audio” issues?

Contributing Author – Robert Wright, Network Engineer with 15+ years experience

“I like talking to a brick wall – it’s the only thing in the world that never contradicts me!”            ― Oscar Wilde

My day job provides me with both pre and post sales situations multiple times per week. On a recent visit to a customer’s site, I provided guidance with the most feared and experienced issue witnessed in VoIP environments today, one-way audio.

The Issue with One-Way Audio

The issue of one way audio isn’t new, and most of you will likely be reflecting on your past experiences while reading this blog entry. This particular customer had begun installing their new UC platform and followed every recommendation set forth by the vendor. Approximately six months after the initial install and certification of the environment, voice calls started dropping at random intervals. Various systems were checked, and blame was passed between numerous teams.

Senior management quickly grew tired of this issue as it went into the second and third week. Shortly thereafter I was engaged by the customer’s senior management to investigate. When I arrived I utilized the customer’s network and application performance management tool set which also contained a dedicated voice/video component. Utilizing its “one-way filter”, I quickly identified the problematic streams in less than 15 minutes of being on site including the normal introductions. Utilizing these details we were able to narrow down our problem area and ask the right questions of the right team. At the end of the day we had a device periodically dropping voice traffic.

So that’s great if you have a product which supports such functionality. But how would you perform troubleshooting if you didn’t have the “Easy Button”? Below I tackle that same problem but with normal network performance management toolsets.

Where Do I Start ????

Let’s assume the same criteria and problem statement. We first should determine if any common network metrics contained deltas significantly off from normal. Remember, at present, outside of “my calls are randomly dropping”, I have limited details about their environment. So I start to build a foundation of knowledge.

• Is QoS Enabled?
  • Why does this matter? QoS permits your business to prioritize traffic based on business criticality and application requirements. Without QoS, your customers may experience intermittent voice quality issues.
• Are we tagging calls appropriately?
  • Why does this matter? If QoS is not deployed uniformly across your enterprise, it can cause delays and confusion during troubleshooting exercises.
• What is the queue utilization broken down by application?
  • Why does this matter? By breaking out queue utilization we can determine how network resources are being consumed.
• Is the queue utilization being used properly? (i.e. no backups in express forwarding)
  • Why does this matter? If queues typically assigned to voice traffic are being consumed by non-UC traffic, it can quickly cause intermittent issues as non-compliant application usage fluctuates up and down.  Or one often overlooked, do we have higher than expected call volumes?
• Are there any discards?
  • Why does this matter? When QoS is not properly configured, or calls are greater than expected, often the result is valid communications are not permitted to its original destination. Instead, it is discarded to the bit bucket.
• If we are seeing discards, which queues and is it significant?
  • Why does this matter? Enterprises have the ability to utilize multiple different queues depending on their needs. A best effort queue isn’t as critical to you as expedited forwarding (where your voice should be). So where the discards exist will determine your priority and response.
• What codec(s) are being used and where?
  • Why does this matter? Different codec’s tolerate varying degrees of network conditions. Each have their role depending on the area of network where voice is being deployed.
• What MOS values are being witnessed per Codec?
  • Why does this matter? By reviewing the MOS values broken by codec we can quickly determine if the wrong codec is being utilized in a particular area of our network.
• Are different areas of the network (with the same codec) differing greatly in regards to MOS?
  • Why does this matter? This can point to the wrong codec being used, or more likely a negative network condition present.
• Are we seeing packet loss? (We can narrow into only areas via MOS)
  • Why does this matter? If we experience significant packet loss, our voice will experience intermediate periods of loss of audio.
• What sort of jitter is being witnessed? (Again using MOS so we don’t boil the ocean.)
  • Why does this matter? High amounts of jitter results in dropouts of audio.
• What is the call flow? (Is this isolated to conference calls?)
  • Why does this matter? By understanding how our call flows between your endpoints you are able to understand which components should have been reviewed in the previous steps.
• Does this call pass through any firewalls? If so does traffic flow freely in both directions?
  • Why does this matter? Firewalls will often block just one direction of the voice traffic resulting in one-way audio.

Up the Creek ?

Your NPM/APM solution should provide you with all of these data sets. If it doesn’t, then you will find yourself up a creek in a hurry. Remember voice streams are bidirectional, should we see differing paths we may be encountering different tagging, logic devices (firewalls) and speeds. All of which can open us up to additional risk.

By utilizing your NPM/APM solution you can narrow down the possible causes of one way audio and focus your companies’ resources appropriately. If budget permits, review the tools out there today which supports functionality such as filtering on one way audio as it will make your life easier..

Points to Ponder

• Ever had a VoIP phone call with a Mime?
• Have any other real world one way audio use cases that you had to troubleshoot?
• Have any other methodologies that you have used effectively to solve this issue?