Sentiment analysis, also known as ‘opinion mining’, is basically the process of identifying the emotions or emotional tone behind a series of words by using Artificial Intelligence.
Sentiment analysis helps analysts understand the current emotion, opinion or attitude of the general population behind a certain topic or set of words mentioned online.
Understanding the current emotional status behind certain topics helps organizations correctly align their proposition, messaging, or product to match trending opinion.
For example, ahead of the 2012 presidential election, the Obama administration leveraged sentiment analysis to gauge the public’s opinion towards campaign messages and policy announcements.
By studying forum posts, comments, and responses to related articles, the team was able to plan strategies and messages to suit the current public mood.
Several studies conclude that up to 90% of the decisions we make are based on emotion.
This plays a major impact when brands are propositioning their products or general advertising. As it is an AI algorithm, you can program sentiment analysis in python (learning Python? Check out the 5 Best Python Projects to Make Your Skills Stronger), or sentiment analysis with r.
Examples & Cases
An advertisement by Expedia Canada was receiving steady negative feedback online.
Through sentiment analysis, the brand was able to determine that the advertisement was getting negative reviews quickly, and it was the violin music, in particular, that was annoying the public.
They quickly made a turn around by releasing a new advert within a few weeks, which showed the violin being smashed.
Download Detailed Brochure and Get Complimentary access to Live Online Demo Class with Industry Expert.
Sentiment analysis uses AI, machine learning and deep learning concepts (which can be programmed using AI programming languages: sentiment analysis in python, or sentiment analysis with r) to determine current emotion, but it is something that is easy to understand on a conceptual level. Consider the following tweet:
Uber’s sentiment analysis algorithm is capable of categorizing this tweet into the price category, and show that the sentiment is negative (anger and disappointment). It can also pull out keywords like ‘flat rate’ to give analysts an idea of the crux of the tweet.
By mapping each category, they can get a clear view of what is the general opinion towards Uber at the moment, what category (price, service, app, etc.) is being talked about, etc. They can then take decisions on improving their future messaging and services to cater to these sentiments.
Consider this example which makes the result of sentiment analysis clearer. A company analyzed 4,000+ online reviews to project a histogram view of public opinion about their reliability, usability, functionality, customer service, and pricing. The graph looked like this:
It is quite clear that the public is more positive than negative about the pricing, but vice versa on the customer service. This company can now easily take action to remediate the situation and improve its brand and consequently, sales.
By developing a program via sentiment analysis in python, or sentiment analysis with r, companies quickly set in place a functioning system.
Types of Sentiment Analysis & How They Work
There are different methods of segregating the types of sentiment analysis, these are the most important ones you need to know about:
Fine-Grained Sentiment Analysis
It drills further down from the common positive, neutral, and negative sentiment tags. This can be achieved by multi-starred ratings (5-star ratings) or a score out of 10 or 100. Sentiments can be categorized as:
- Very Positive
- Positive
- Neutral
- Negative
- Very Negative
A further level of categorization can be achieved by defining the state of each sentiment along with the rating. For example, an emotion can be 4-negative and can be described as ‘angry’ for further segmentation.
2. Emotion Detection
This typically involves identifying the public’s emotional state towards a topic, categorizing emotions like happiness, frustration, anger, sadness, etc. This is often achieved by either studying emoticons used to describe a product, or mainly via text.
For example, a comment saying ‘Absolutely frustrated with XYZ’s customer service!!’ will be categorized as an angry comment under customer service. The downside of using this method is that lingo differs based on culture, location, and even depending on the situation.
For example, saying ‘XYZ’s product is killing me’ is negative but saying ‘XYZ killed it with their latest product’ is positive according to modern lingo, and can be confusing to an AI algorithm.
3. Intent Analysis
In an intent analysis, the AI algorithm aims to identify what is required rather than just what is felt. For example, consider the following comments:
Is there a way to only order the charger for this product?
I’m wondering if there’s a store in London
In both cases, there’s an emotion, but there’s also a question that needs answering. In an intent analysis, the algorithm learns to identify the needs, which in particular would be whether they can order only the charger and if there’s a store in London.
4. Multilingual Sentiment Analysis
This type is basically analyzing comments made in different languages around the world to decide the general opinion about a topic. It comes in handy for brands that sell globally and want to learn from feedback irrespective of language. Because of the complexity of this type, many brands resort to allowing only English comments and reviews.
5. Aspect-Based Sentiment Analysis or Contextual Semantic Search(CSS)
Aspect-based Sentiment Analysis gives an idea about the emotion, and also the feature that is being commented upon. For example, an aspect-based sentiment analysis AI algorithm will read the following comment: ‘I wish these shoes came in size 11’ and register the emotion as unhappy, AND the category that is the size.
This is the most important and most used type of sentiment analysis, as it not only depicts the emotion but also gives brands deeper insights into what is working and what is not.
In a contextual analysis, the AI program is trained not to filter based on selected terms, but also to look out for context-based terms. For example, consider the following comment:
Now although the comment specifically mentions the word ‘price’, a CSS algorithm will be able to categorize this into the price section by picking up on context-based words like ‘paid’, ‘charged’, ‘fare’ etc.
Use Cases and Applications (Sentiment Analysis of Twitter, Facebook, & News Portals)
Before we get into the core application of sentiment analysis, consider the following example of sentiment analysis based on Uber’s available social media data.
This analysis was done based on studying the following data:
- Facebook: 34,173 Comments
- Twitter: 21,603 Tweets
- News: 4,245 Articles
Data was segregated by emotion for the following categories: Cancelation, Payment, Price, Safety, and Service. The following results were derived by using the Contextual Semantic Search algorithm:
Facebook: Sentiment Analysis
From just this one result, we can see that apart from the price category, all categories have more negative feedback than positive. To further understand the basis of these emotions, the comments were divided based on intent:
In order to get more accurate, cleaner results, unwanted comments (like spam) were ignored, and the sentiment analysis graph changed to:
Now what you see is a more accurate representation of the public’s opinion about the different categories of Uber’s service. The above process was repeated for the tweets made on Twitter.
Sentiment Analysis of Twitter
After ignoring junk, spam, and other similar unwanted comments:
And finally, the analysis made on the news data that was collated:
News: Sentiment Analysis
From Facebook and sentiment analysis of twitter results, it is quite evident that the service needs immediate attention, and safety is the most concerning topic being discussed in the news sections and forums.
With this data in hand, Uber can set about making changes to internal policies, roll out discounts, make the app more user-friendly, set stricter driver policies, etc. to improve public sentiment.
Applications of Sentiment Analysis
With this use case in mind, let’s look at some of the applications
1. Social Media Monitoring
The most obvious use of sentiment analysis is social media monitoring. According to Oberlo, there are 3.2 billion social media users worldwide. Most of these users take to social media when they want to portray an emotion towards a brand or a product.
This is large because it is easier to find like-minded individuals going through the same experience, to gain traction, and to gain the attention of a brand. Here’s how it can be applied on social media sites:
(i) Track facebook posts and Twitter tweets to extract opinions and categories.
(ii) Track mentions on different platforms and runs it on these comments.
(iii) Automate response or assign a real-time team to cater to posts ad mentions.
(iv) Implement analytics solutions to get insights on collected data
Using social media analysis, you can get quick feedback because topics usually catch on quickly, giving you a lot of data in a short time span. You can use his quickly mitigate risks or leverage-positive movement.
2. Brand Monitoring
Brands can get a plethora of information by tracking brand mentions not only on social media but across the internet like on forums, blogs, news articles, etc. You can use basic tools like Moz to find brand mentions, which, however, do not help with it but can be used nonetheless to track brand popularity over time.
By putting a proper sentiment analysis AI solution in place, brands can monitor their reputation over time, and also get insights on what actions are impacting the change in brand reputation, be it positive or negative.
3. Voice of Customer (VoC)
While social media and online mentions are offered by users, you can voluntarily collect customer feedback to learn and improve, or to accelerate the movement of positive feedback.
The advantage of customer feedback is that it is more authentic. Setting up an NPS (Net Promoter Score) system can be more than enough to get useful insights. A NOS survey involves some basic questions like: Would you recommend this company, product, and/or service to a friend or family member? And why?
By studying the response and running it on the comments provided, brands can get an idea of how well their products are being received.
An example of using VoC is the McKinsey City Voices project. The federal public spending rose by 156% in Brazil from the year 2007 to 2015, while the public’s satisfaction level with the public services continually decreased. The Urban-planning Department hired the services of McKinsey to provide insights on improvements.
McKinsey developed a tool called City Voices, which basically allowed them to conduct customer (or citizen) surveys across over 150 metrics.
They then ran sentiment analysis over the results to come to a conclusion on what people were most desperately in need of- a safer bus system was one, for example, and provided local corporations and the government as a whole with insights.
4. Customer Service
It can be used to study incoming customer requests and assign them appropriately. By studying the emotions of an incoming message, an automated system can route the complaint to appropriate teams and personnel within teams to help connect the customer with the right person even before they ask for it.
For example, an automated message could ask “Hi, how may I help you today?”. If the user’s response is “I just got an SMS saying your order has been delivered, but I haven’t received the order yet!”, the bot can immediately recognize anger and assign the user to a human in the logistics team to sort out the issue.
5. Market Research
It can be used in market research to understand public opinion about different products and services. This can help brands improve their own products, and also help the public identify good products.
For example, a sentiment analysis program can be run against TripAdvisor’s reviews to group hotels based on good, neutral and bad with respect to visitor’s sentiments.
The Bottom Line
In today’s world of quick judgement and rapid information sharing, it is important for companies to stay on top of things to protect their brand and product/service reputation.
The advantage of a fast-paced information-sharing world is that companies can get feedback about their brand or product faster, sometimes in a matter of hours or days.
This also makes it a lucrative field for developers and programmers to venture into, and learn sentiment analysis in python, or sentiment analysis with r as a career choice.
You may also enroll in a Data Science Using Python course for more lucrative career options in Data Science.
