Extracting Reality from Data
Go to aylien.com

Why is Sentiment Analysis important from a business perspective?

What is Sentiment Analysis?

Sentiment essentially relates to feelings; attitudes, emotions and opinions. Sentiment Analysis refers to the practice of applying Natural Language Processing and Text Analysis techniques to identify and extract subjective information from a piece of text. A person’s opinion or feelings are for the most part subjective and not facts. Which means to accurately analyze an individual’s opinion or mood from a piece of text can be extremely difficult. With Sentiment Analysis from a text analytics point of view, we are essentially looking to get an understanding of the attitude of a writer with respect to a topic in a piece of text and its polarity; whether it’s positive, negative or neutral.  

In recent years there has been a steady increase in interest from brands, companies and researchers in Sentiment Analysis and its application to business analytics. The business world today, as is the case in many data analytics streams, are looking for “business insight.”

Sentiment Analysis Interest Over Time: Google Trends

So what “insights” are we talking about?

Well, in relation to sentiment analysis, we’re talking about insights into consumer behaviour, what our customers want, what are customers like and dislike about our products, what their buying signals are, what their decision process looks like etc…

Analyzing Sentiment for business insight:

Screen shot 2014-08-21 at 6.31.48 PM.png

As more and more content is created and shared online, through Social Channels, Blogs, Review Sites etc. we are becoming more and more vocal and open about our experiences online. In a recent study carried out by Zendesk it was noted that 45% of people share bad customer service experiences and 30% share good customer service experiences via social media. Which again, highlights the need and desire for businesses to mine this information to gain business insight from it has also increased.

Businesses are trying to unlock the hidden value of text in order to understand their customers’ opinions and needs and make better, more informed, business decisions. Traditionally businesses relied on surveys, workshops and focus groups to gain insight into their customers opinions and feelings, but today with modern technology we are able to harness the power of Machine Learning and Artificial Intelligence to extract meaning from text and dive into opinions of customers and see them outside of the often controlled environment of a survey.

What can we Analyze?  

There is a wealth of information out there hidden in individuals comments, emails, tweets, form submissions, reviews and the challenge is wrangling all of this info and extracting value from it. Below are some examples of detecting an individual’s opinion about a certain topic or product using tweets and reviews:

A tweet about Facebook’s messenger app:

"Literally ur facebook message app is useless, you only want it to increase profit. Please fix yourself. Its sad @facebook"

In this case we are going to detect the sentiment of the tweet in terms of polarity (positive or negative) and subjectivity (subjective or objective).

Result:

Screen shot 2014-08-29 at 10.54.42 AM.png

A product review:

“I really enjoyed using the Canon Ixus in Madrid. The Panasonic Lumix is a poor camera, but the Canon Ixus is really sleek. The Canon Ixus is much better than the Panasonic Lumix. All I want to do with a camera is point it and then just press the button. The Canon Ixus is perfect for that. You will soon get great photos with very little effort. I had previously returned a Panasonic Lumix because the pictures were not of the quality I expected. Spending the money on the Canon was a smart move. I would recommend the Ixus to anyone without hesitation. Two small issues to be pointed out: firstly, the battery life is not fantastic. Also, the zoom is not very powerful. If your subject is far away, you should not expect good results. Review made by John Irving from Florida.”

Result:

Screen shot 2014-08-29 at 2.48.52 PM.png

There is a couple of things you can do with this sort of information. We can act on them quite easily for example by responding in a timely manner to negative tweets and dealing with them head on or showing your love for a positive review by saying thanks!  

While it is somewhat interesting to look at each of these examples in isolation the true “business” value comes when you combine more and more tweets or reviews at scale. Looking at the bigger picture allows you to truly listen to the voice of your customer, identify trends, pinpoint problems and essentially extract more “business” value.

Interested in analyzing the sentiment of text, tweets, comments or reviews? Check out our Text Analysis API.

Multilingual Text Analysis - Minor Product Update

At AYLIEN we only make promises we can keep. Back in May we announced support for two of our endpoints across multiple languages, Hashtag Suggestion and Concept Extraction. We also promised you we would extend that support to the rest of our endpoints in due course.

We’re not 100% there yet but the good news is, we are well and truly on our way to being fully functional across 6 different languages. From today, we have added multilingual support for a further 5 end points Extraction, Summarization, Classification and Language detection which is available to all users across German, French, Italian, Spanish and Portuguese.

Let’s say we want to classify this Article:

All we need to do is:

curl -H "X-Mashape-Authorization: YOUR_MASHAPE_KEY" --data-urlencode "url=http://www.zeit.de/politik/ausland/2014-08/waffenlieferung-kurden" -d "language=auto" https://aylien-text.p.mashape.com/classify

Which returns:

{
  "text": "...",
  "language": "de",
  "categories": [
    {
      "label": "Verteidigung - Militär",
      "code": "11001004",
      "confidence": 1
    }
  ]
}

Cool, Huh?

So, 6 out of 8 endpoints ain’t bad, but we’re not stopping there. We are currently working hard on multilingual support for two of our most popular and complex endpoints; Sentiment Analysis and Entity Extraction. Which we hope to rollout in the next couple of months, so stay tuned for more updates.

What language would you love to see us support?

Text Analysis; 10 business use cases you may not have thought of…

One of the coolest things about analysing text is, it’s everywhere! Irrespective of industry, companies & individuals want to make better informed business decisions based off trackable and measurable insight. With advancements in Text Analysis, companies can now mine text to uncover insights and improve their service or offering to prosper in their market.

So far at AYLIEN, our Text Analysis API has had great success in the news and media space. But this is just the tip of the Text Analytics iceberg. There are countless numbers of other industries that can gain the same value from such insights. As we don’t have a countless amount of time, let’s stick with a Top 10 list of use cases for Text Analytics.

1. Sports trading - One of the most popular sports to bet on, particularly in Europe, is football (soccer). The top sports traders gather data from the mainstream media and have a deep understanding of the game and it’s politics at a local level. If you live in England and you bet on English football, irrespective of the division, it’s relatively easy to understand your market. You can successfully bet on a local second division English team because you speak the language, read the local newspapers and may even follow some of the team members on Twitter. But what if you’d like to do the same for a similar team in Spain and you don’t speak a word of Spanish? A Text Analysis API capable of understanding Spanish would allow you to extract meaning from local Twitter feeds, giving you insights into what the local fans are saying about their team. These people understand the squad dynamics at a local level. If, for example, the star striker of Real Club Deportivo Mallorca has an argument with his wife the night before his cup game, is he as likely to be the top scorer on match day?

2. Financial Trading - As with sports trading, having an insight into what is happening at a local level can be very valuable to a financial trader. Domain-specific sentiment analysis/classification can add real value here. The same way in which fans have their own distinct vocab based on the sport, so too do traders in particular markets. Intent recognition and Spoken Language Understanding services for detecting user intents (e.g. “buy”, “sell”, etc) from short utterances can help to guide traders in deciding what to trade, how much and how quickly.

3. Voice of the customer (VOC) - VOC applications are primarily used by companies to determine what a customer is saying about a product or service. Sources of such data include emails, surveys, call center logs and social media streams like blogs, tweets, forum posts, newsfeeds, and so on. For example, a telecom company could use voice of customer text analysis to scan Twitter for customer gripes about their broadband internet services. This would would give them an early warning when customers were annoyed with the performance of the service and allow them to intercept the issue before it involved the customer calling to officially complain or request contract cancellation.

4. Fraud - Whether it’s workers claiming false compensation or a motorist disclosing a false home address, fraudulent activity can be discovered much more quickly when those investigating can join the dots together, faster. In the latter case, for example, the guilty party may give an address that has many claims associated with it or the driven vehicle may have been involved in other claims. Having the ability to capture this information saves the insurer time and gives them greater insight into the case.

5. Manufacturing or warranty analysis - In this use case, companies examine the text that comes from warranty claims, dealer technician lines, report orders, customer relations text, and other potential information using text analytics to extract certain entities or concepts (like the engine or a certain part). They can then analyze this information, looking at how the entities cluster and to see if the clusters are increasing in size and whether they are a cause for concern, for example.

6. Customer service routing - In this use case, companies can use text analytics to route requests to customer service representatives. For example, say you’ve sent an email to a company while on hold to one of their reps. You might have a question or a complaint about one of their products. The company can use text analytics for intelligent routing of that email to the appropriate person at the company. This could also be possible in a call center situation, provided you have sufficiently accurate speech-to-text software.

7. Lead generation - As was the case with the VOC application, taking timely action on a piece of Social Media information can be used to both retain and gain new customers. For example, if a person tweets that they are interested in a certain product or service, text analytics can discover this & feed this info to a sales rep who can then pursue this prospect and convert them into a customer.

8. TV advertising & audience analysis - TV shows or live televised events are some of the most talked-about topics on Twitter. Marketers and TV producers can both benefit from using Text Analytics in two distinct ways. If producers can get an understanding of how their audience ‘feels’ about certain characters, settings, storylines, featured music etc they can make adjustments in a bid to appease their viewers and therefore increase the audience size and viewers ratings. Marketers can dig in to social media streams to analyse the effectiveness of product placement and commercials aired during the breaks. For example, the TV character ‘Cersei’ from Game of Thrones is becoming a fashion icon amongst fans, who regularly Tweet about her latest frock. High street retailers that want to take advantage of this trend could release a line of ‘Queen of Westeros’ style clothing and align their commercials with shows like Game of Thrones. Text Analytics could also be used by TV Executives looking to sell to advertisers. For example, a TV company could mine viewers tweets & forum activity to profile their audience more accurately. So instead of merely pitching the size of their audience to advertisers, they could wow them by identifying their gender, location, age etc and their feelings towards certain products.

9. Recruitment - Text Analysis could be used in both the search and selection phases of recruitment. The most basic application would be identifying the skills of a potential hire. In the recruitment industry, the real value comes from identifying prospects before they become active on the job market. For example, it would be very powerful to know if somebody tweets about disliking their job or expresses an interest in working in a different field, larger/smaller company, different location etc. Once you have identified such a prospect, you could use Text Analytics to analyse the suitability of this person based on what others say about them. Mining news and blog articles, forum postings and other sources could help to evaluate potential hires.

10. Review Sites - Companies like Expedia have millions of reviews on their website, from travellers all over the world. Given the nature of the site and the fact that their users are looking for a stress free experience, having to sift through hundreds of reviews to find a place to stay can be a real turn off. Text Analysis can be used here to build tools that can summarize multiple properties in 2-3 word phrases. Instead of scrolling through a list of hotel features like heated pool, massage therapy, buffet breakfast etc, you could simply say “Luxurious Hotel and Spa”.

Did you like our top 10 use cases? If you work in an industry that’s not mentioned above and have an idea of how Text Analytics could help you, please let us know!

Subscribe to our blog and keep an eye out for our next post on how Text Analytics can add value to your business.

Drop us an email @mention us on Twitter

Text Analytics meets 2014 World Cup tweets - Part 1

The FIFA World Cup is without doubt the biggest sporting event in the World, with millions of fans and viewers from all around the globe who use Social Media to share their thoughts and emotions about the games, teams and players and thus creating massive amounts of content on Social Media by doing so.

Throughout the tournament, Facebook saw a record-breaking 3 billion interactions and Twitter saw a whopping 672 million tweets about the World Cup.

That’s why at AYLIEN we decided to collect some of this data using Twitter’s Streaming API and analyzed tweets related to the world cup, looking for interesting insights and correlations.

We are going to explore how you can use text analysis techniques to dig into some of this data in a series of blog posts.

In Part 1 of the series, we’re going to get a high-level view of our data, and also to look for some basic data insights about the tournament.

Data and Tools

Data: datasets used in this blog post are as follows:

  • tweets.csv: Around 30 million Tweets (80 million including retweets - which are omitted) collected between June 6th and July 14th using the Twitter Streaming API, and filtered by some of the official World Cup hashtags (e.g. “#WorldCup” and “#Brazil2014”), as well as team code hashtags (e.g. “#ARG” and “#GER”) and Twitter usernames of teams and players. (Note: we’re assuming that Twitter samples the tweets in a uniform fashion and without any major side effect on their distribution)
  • matches.csv: Information about the 64 matches, such as match time and results, obtained using the World Cup json project.
  • events.csv: Information about match events such as goals, substitutions and cards, obtained using the World Cup json project.

Tools: For these posts we will use AYLIEN Text Analysis API for Sentiment Analysis, RapidMiner for data processing and Tableau for interactive visualizations.

Matches

Let’s start our quest by taking a look at the matches and their events, such as goals, substitutions and red and yellow cards:

Things to note:

  • The number of matches with 5 or more yellow cards tends to increase in later stage games, possibly due to higher sensitivity and intensity of these matches.

Tweet languages

Now let’s take a look at a breakdown of the most popular languages used in our tweets dataset:

Things to note:

  • English, followed by Spanish and Portuguese are the three most used languages in our tweets dataset.

Tweet locations

Next we’ll have a look at the distribution of geo-tagged tweets over different countries around the globe, along with their languages:


Tweets and events

Plotting the total volume of tweets over time shows a repeating pattern of spikes appearing at match times and also at times when a major event has occurred (such as elimination of a team, qualification for the next round, or shocking results). Let’s have a look at a few examples:

1. Tweet volume by Language

In these examples, we’re going to see how the volume of tweets in a language is affected by the matches and critical events related to teams from countries where that language is spoken (also note the trend lines in black):

Note: double click on the charts to zoom, click and hold to pan.

Teams: USA, England, Australia, Cameroon and Nigeria.

Teams: Germany and Switzerland.

Teams: France, Belgium, Algeria, Cameroon and Côte d’Ivoire.

Teams: Spain, Argentina, Mexico, Uruguay, Chile, Costa Rica, Ecuador, Honduras and Colombia.

Teams: Italy.

Teams: Brazil and Portugal.

2. Tweet volume during matches

A similar pattern can be observed at a smaller scale during matches, with spikes appearing for each goal or major event. Let’s see an example from the Brazil - Germany match:


3. Tweet volumes for different teams

Finally, let’s take a look at how the volume of tweets that mention a team changes over time for the four teams that qualified for the semi-finals round (for each team we are counting mentions of the team’s full name e.g. “Germany” as well as its team code hashtag e.g. “#GER”):

Subscribe to our blog and stay tuned for part 2, where we use Text Analytics to dig deep into the tweets’ contents.

Got some cool use cases of text analysis? We would love to hear about them. Get in touch below.

Drop us an email @mention us on Twitter

Welcome on board to our 3 new team members!

Kevin Boyle - COO
Mike Waldron - VP Sales and Marketing
Mike O’Gorman - Sales and Marketing Director


We have the pleasure of introducing Kevin Boyle as our new COO, Mike Waldron as our VP of Sales and Marketing and Mike O’Gorman as our Sales and Marketing Director.

Welcome on board, folks!