AWS custom metrics with python boto3

For server performance monitoring we use AWS metrics and alarms for CPU & memory utilization but for webservers sometimes apache process itself and for dB server mysql/SqlServer itself consumes almost full memory and cpu also sometime.

We will use python psutil library with boto3 to send custom metrics for memory used by apache and mysql .

With psutil (python system and process utilities) library we can use for retrieving information on running processes and system utilization (CPU, memory, disks, network) in Python.

Below example use Top 3 processes which consumed the most CPU time:

We use two function for which retrieve memory info for apache and mysql process & send to cloudwatch metrics,full source code link.

def memUsedByApache():
return round(sum([p.info['memory_info'].rss for p in psutil.process_iter(attrs=['name','memory_info']) if 'httpd' in p.info['name']]) / (1024*1024), 1)

def memUsedByMysql():
return round(sum([p.info['memory_info'].rss for p in psutil.process_iter(attrs=['name','memory_info']) if 'mysqld' in p.info['name']]) / (1024*1024), 1)

And finally we will send this memory information to main function

In cloudwatch , you can search for custom metrics

and you will get metrics like below

You can create alarms based on performance and recommendation.