Problem to Solve:  When we encrypt our traffic to be PCI compliant, how can we still troubleshoot application and network performance issues?

Contributing Author – Robert Wright, Network Engineer with 15+ years experience

We are almost half way through fighting the forest fire, which is PCI DSS. In the past sections we saw that we can utilize our APM / NPM tools inside our CDE (Cardholder data environment) and where to place them. So at this point we procured our tools, and installed them so we can call this job complete ….. right? Wrong! But wait, I use encryption; so NOW I am done!?!? Nope. Read on to discover the truth.

Tree – Protect Stored Cardholder Data

Protection methods such as encryption, truncation, masking, and hashing are critical components of cardholder data protection – PCI DSS v3.0 Requirement 3 Page 24

As we start to review requirement three, it actually calls out the various frameworks that we can utilize to maintain compliance. Many APM/NPM tool sets today offer the ability to take appropriate measures to remove card holder data from being stored inside their system. So the next time you see your vendor, start asking the tough questions. How can your tool help prevent my business from being referenced on CNN or Fox News??

For example, methods for minimizing risk include not storing cardholder data unless absolutely necessary – PCI DSS v3.0 Requirement 3 Page 24

I have reiterated this point numerous times throughout this series. The above quote should be burned into your memory and referenced often in your meetings. Folks troubleshooting network and application performance problems simply do not need access to cardholder data. So why increase your overall risk of being on the evening news? This is especially important when considering recent industry trends which permit the ability to extract critical application data while removing payload, cardholder data in our case.

Now someone out there may say… Robert, I have business requirement X, which absolutely requires that I store content which falls under PCI. Is there anything we can do regarding our APM/NPM tool sets? As a general disclaimer, I remind you this blog post is purely my opinion. Here are a few key points I would follow:

• Ensure your tool set provides audit logging of all packet capture activities
• Establish procedures with a third party to review those audit logs to monitor for abuse
• Establish rules and processes on how this data is viewed. I.E., Can the technician save packet captures to their PC?
• Establish procedures to routinely remove old packet captures that are no longer required for troubleshooting.
• Review the availability of disk based encryption with your vendor(s)
• Consult with your management chain and legal departments and save this communication for a future reference
• Reference PCI DSS Requirement 3 in great detail as it spells out what may and may not be stored should a business requirement exist
• Schedule repeating meetings (yearly?) to review if business requirements have changed.

Tree – Encrypt Transmission of Cardholder Data Across Open, Public Networks.

When you start talking about encryption some folks will view it as Pandora’s Box, while others believe it’s the cure for any aliment. There are fundamental problems with both lines of thinking, and PCI is going to bring encryption to your environment should your business decide to comply with PCI. Your organization needs to be prepared with the appropriate resources. Are your teams and tools ready and able to address these problems?

The fourth requirement of PCI 3.0 states the obvious when laying out the use of strong cryptography and security protocols to protect cardholder data as its transmitted across open and public networks.  But how will your teams troubleshoot these application protocols once they are encrypted should they be suspected of failure?

Your technicians will be able to validate that the security protocol (IE SSL/TLS, IPSEC, SSH) was transmitted, received, and routed appropriately. They will also be able to determine if those transport protocols sustained any problems during its life. However, what about the encapsulated protocol which actually contains the card holder data? Well your technicians can’t see that information because it’s encrypted! Many tool sets today can provide you the ability to perform decryption. But be warned it does take significant CPU resources to perform such a task and appropriate capacity planning must be performed. So for example, should your web server or load balancer start spitting out 500 error messages, your existing tool set won’t see these errors unless they are able to perform decryption. In the same breath, should an attacker utilize these same protocols to commit an attack, how will your IDS/IPS be able to detect it? Or better yet how will you perform forensics?

Requirement 4.1 has significant focus on the use of trusted certificates which calls into the use of a certificate authority. Depending on the use case, this very well may lead to the use of Private Key Infrastructure (PKI), which we have covered in another article found here on the Problem Solver Blog: https://problemsolverblog.czekaj.org/troubleshooting/detection-invalid-certificates-packet-analysis/

Requirement 4.1.1 goes into detail around the security technologies we should be utilizing should we have wireless networks transmitting cardholder data. When wireless networks are present and conforming to standards such as 802.11i, we can rest assured that RADIUS is most likely present. Suddenly our wireless network and authentication is thrust into a business critical role. Should our wireless connected credit card reader be unable to authenticate against the wireless network, we can no longer sell product.

Our APM/NPM tools now must not just be able to troubleshoot typical three tiered applications which are encrypted, but we must also focus on tools which support the troubleshooting of authentication.

PCI 3.0 Requirement 4.2 states that we should never send PANs (Credit card numbers) out through messaging such as email, IM or chat. The typical APM/NPM tool set is not one that utilizes signature based anomaly detection. That role and skill set falls under security tools such as IDS, IPS, and DLP. However as the owner of APM/NPM tools we are most likely the owner of or at least contributor to, the purchase of an aggregation switch.

These solutions (IDS/IPS/DLP) still require the same data set (packets) that your APM/NPM tools need. So the next time you are considering the deployment of APM/NPM tools, remember that the same data you’re looking to troubleshoot and protect is the same as the folks running Data Loss Prevention (DLP).

As we close out another chapter of our journey through PCI, remember that many vendors offer tools which will make your job easier. All you need to do is ask the question. We are speeding through each of these requirements so if you believe I have missed something, or only touched the surface on one of the requirements; please feel free to comment on this post!