Sentiment Analysis on the Launch of Amazon Q using Amazon Comprehend and Sagemaker Studio

#awscommunitybuilders #aws #python #programming

Sentiment analysis is a powerful tool that allows companies to analyze customer opinions and sentiments towards their products or services. At AWS Re:Invent in Las Vegas on November 28, 2023, Amazon unveiled Amazon Q, an innovative AI-based assistant tailored for workplace use. This cutting-edge tool is designed to deliver quick responses to inquiries, create content, and enable actions using data from customer information databases, codebases, and corporate systems. Amazon Q provides personalized interactions to streamline processes and speed up decision-making while also fostering a culture of creativity and innovation in the workplace. The launch of Amazon Q marks a significant advancement in the field of AI-powered assistants for work.

This mini-project came as a result of my curiosity with the whole hype of AI which seems to be dominating the technology landscape. Also as a Product Manager this is of importance in order to determine customer satisfaction and engagement and ultimately make data driven decision in order to scale the product/service. As a Data Analyst, just like how Duet AI is to Google Looker, that is how Amazon Q is to AWS Quicksight. This tool integrated with Quicksight provides real time and fast analysis of data in seconds. When a new product is released, there is a need to determine the business value of its customers. Sentiment analysis can help identify the overall sentiment towards the new product by analyzing customer opinions and emotions expressed in online reviews and social media posts.

Getting Started

In this tutorial, I will show how to perform a Sentiment Analysis using 2 major tools from AWS: AWS Sagemaker Studio and Amazon Comprehend.

Architectural Diagram

Data Collection: data was extracted from twitter using Python. This data contained tweets from users. Here are some samples of the tweet that were scraped for this project:

Using #AmazonQ in the IDE is reeediculous! I know AI isn't always right but it has been absolutely amazing. It even overcomes my HORRIBLE spelling! So Sick!
reInvent2023 unveiled #AmazonQ as a standout highlight. At Caylent, we're excited to streamline the integration of Amazon Q into MeteorAI, making adopting GenAI solutions that much faster and easier.
Do I know anyone at AWS who can comment on when the Amazon Q Code Transformation for .NET will be available? If this works, it would be amazing for some legacy applications we need to modernize at work.

Data Transformation: most of the transformation done here was data cleaning, this is because the data contained certain special characters such as "#" and "@". This was done using Excel Sheet.

IDE: Amazon Sagemaker Studio served as the coding environment to run and perform the task

Sentiment Analysis: AWS offers a range of services with strong NLP capabilities, including entity recognition, key phrase extraction, and sentiment analysis. Amazon Comprehend is one of such service. It utilizes machine learning to uncover insights and connections within text, enabling the detection of sentiments.

This article is best suited for programmers familiar with Python.

Scraping of Tweets
I have a detailed article on Web Scraping here, so ensure to follow the link to get started. After this is done, the tweets are saved in the Excel Sheet and uploaded to S3 Bucket

Sentiment Analysis
Import the CSV File needed to run the analysis in Amazon Sagemaker Studio.

import boto3
import pandas as pd

comprehend_client = boto3.client('comprehend', region_name='us-east-1')

# Read CSV file
amazon_q_df = pd.read_csv('amazon_q.csv')
texts = amazon_q_df['Text']  
len(texts)

Perform the sentiment analysis using detect_sentiment in Amazon Comprehend to determine different sentiments of users tweets. Read more on how it works here

sentiments = []

# Analyze text using Amazon Comprehend
for text in texts:
    sentiment_response = comprehend_client.detect_sentiment(Text=text, LanguageCode='en')
    sentiment_scan = sentiment_response['Sentiment']

    sentiments.append({'Text': text, 'Sentiment': sentiments})

sentiment_df = pd.DataFrame(sentiments)
print(sentiment_df.head())

Output:

Visualize these texts by sentiments (mixed, neutral, positive, negative)

import seaborn as sns
import matplotlib as plt
percentage = (sentiment_df.groupby(['Sentiment']).size())
percentage

# Adjusting the explode parameter to match the number of unique sentiment categories in the data
unique_sentiments = sentiment_df['Sentiment'].nunique()
explode = [0.1 if i == 0 else 0 for i in range(unique_sentiments)]

plt.figure(figsize=(8, 8))
plt.pie(percentage_new['Percentage'], labels=percentage_new['Sentiment'], autopct='%1.1f%%', startangle=140, explode=explode)
plt.title('Percentage of Tweets by Sentiment')

# Show the plot
plt.show()

The data collected were only specific to Twitter and the tweets were quite limited too, which may skew the overall findings and limit generalizability.

This shows:

A large majority of tweets, 74.4%, are classified as Neutral, indicating no particular positive or negative emotion.
20.9% of tweets are Positive, suggesting a favorable or optimistic sentiment of the Amazon Q.
A small proportion, 2.3%, are Negative, implying a pessimistic or unfavorable sentiment.
An equally small proportion, another 2.3%, are Mixed, indicating the presence of both positive and negative sentiments within the same content.

This analysis is useful for gauging public opinion or reaction to the launch of Aamzin Q on social media platforms like Twitter.

Amazon Comprehend and Amazon Sagemaker Studio played a crucial role in carrying out the Sentiment Analysis of Amazon Q. With Amazon Comprehend's advanced NLP capabilities such as entity recognition, key phrase extraction, and sentiment analysis, we were able to uncover valuable insights from the tweets extracted from Twitter. The use of Amazon Sagemaker Studio also provided an efficient coding environment for running and performing these important tasks. These tools not only facilitated the analysis process but also enhanced my understanding of public sentiments towards Amazon Q. The use of Amazon Comprehend and Amazon Sagemaker Studio showcased the power and effectiveness of AI-powered tools in sentiment analysis.

Thank you for taking the time to read my article. Don't forget to follow me on here and feel free to reach out to me. Looking forward to connecting again soon!