AWS Metrics often underlooked when checking for performance issues

AWS provides many metrics to see resource consumptions by the workload or application. Out of these metrics we will shed some lights on some of the important metrics that are often underlooked. Often developers see cpu and memory consumption and forget about other metrics that might be really important to be checked. Although memory metrics is not available by default you need check through top command or may need to install cloudwatch agent on EC2 instance and send those metrics on AWS. Thorough this blog post we will try to cover other important metrics such as Network In, Network Out, and Queue Length.

1. Network In / Network Out

When requests to your AWS EC2 instance takes time in responding or times out it is really important to see Network In and Network Out monitoring graphs. These graph shows the data in the network. Although AWS instances have good amount of network throughput but the EBS volume may impact on this throughput. The EBS volumes have their own through put which depends on size. Let us show you through a pictures below from aws simple monthly calculator, when you add EBS it shows you network throughput.

EBS Throughput

Now take a look at throughput provided by AWS instances:

AWS Instance throughput

If the network in and network out are on high consumption equaling to EBS throughput new request or out may suffer from certain latency. This will slow the system performance. You can find network and network out graphs under the monitoring tab, when you click on a running EC2 instance at the bottom you can see multiple tabs like Description, Status Checks, Monitoring, Tags. Click on monitoring tab and you can see many graphs listed there. You can find Network In and Network out graphs in that list.

You can update these graphs on time range like 1 week, 2 week, period like 5 minute, 1 minute and statistic like sum, average, maximum etc.

EC2 Network In Graph
EC2 Network Out

Ways to improve performance if you are suffering from EBS throughput bottleneck?

You can improve performance in two ways:

  • You can increase EBS volume size, but there is certain limit on maximum network throughput of volume and you cannot further increase it by increasing volume size which you can clearly see in pictures below.
  • Max EBS Throughput1
    Max Ebs Throughput2

  • You can setup a raid array of volumes to increase performance.

2. Queue Length

Another important metric is queue length, the queue length is dependent on number of IOPS. If you choose general purpose EBS volume the iops are counted as = volume size x 3 so for 100 GB volume you get 300 iops (input output operations per second). Queue length is taken in the ratio of 200, that means if you have 300 iops queue length lesser than 1.5 is ok, if it is getting more than 1.5 your request may suffer from latency. The iops operation are really very costly and we should try to save to make our application consume less iops, if the application performs db operation it consumes iops, if application ready image file on server it consumes iops. You can check queue length monitoring under volumes. You can click on the volume attached with a running aws instance. Select the volume that is in use on instance. At the boot you can see monitoring tab, click on it and you can see queue length.


Help us Grow

Leave a Reply

Your email address will not be published. Required fields are marked *