Extracting Reality from Data
Go to aylien.com

Why Text Analytics is essential in the ever changing Publishing industry

The publishing industry has changed dramatically. Mainstream newspapers and magazines have given way to desktop publishing and the Internet as economics have changed the game.

Let’s look at the main drivers behind this change.

More competition - Self-publishing has moved into mainstream online channels. The increase of entrants into the market means more choice and much of it is free.

The introduction of Apps - Apps create a more engaging and effective way to interact with an audience. The ever increasing ownership and usage of mobile devices mean that more readers can be reached.

Real-time social sharing - It can be argued that Facebook and Twitter provide the most up-to-date news channels. The sharing dimension can also be very appealing to readers who want to contribute to reporting the news, as opposed to passively receiving it.

Shift from mass to a niche market - Before the inception of the internet, successful newspapers and magazines appealed to the general public. Today, however, digital publishing has far lower production costs and a far greater reach to service niche markets.

image

According to Ofcom, use of the internet to consume news has increased for computers, laptops, tablets and mobiles since 2013, while TV has seen a small decrease from 78 to 75 percent. Use of any type of an online platform to consume news increased from 32 to 41 percent this year, and is now higher than the use of newspapers (40 percent) and radio (36 percent).

This shift in how we consume news has forced publishers to change their strategies in order to compete. More specifically, publishers understand that their content needs to be more relevant, richer, interactive, timely and discoverable.

An Example

Let’s say an editor hears about a bus crashing near a major school, close to a fire station. The editor wants to write about the story and they want to include historical information about the cause of bus crashes (e.g. time of day, time of year, equipment malfunction, driver error etc based on other bus crashes for the past 30 years) to give the story more depth and context. In most cases, a journalist would have tagged documents with dates and keywords. This is generally a manual process and therefore documents could easily be left untagged due to human error. Tags may be missed if different individuals are involved in the process. Some people may also not be as thorough as others. For instance, if somebody simply tags the document “bus crash”, it might be very difficult to find similar stories, much less analyze what happened in other relevant crashes.

Enter Text Analytics

By incorporating text analysis software, historical data can be culled for relevant concepts, entities, sentiments and relationships to produce a far richer tagging system. Information about the bus crash such as the type of bus involved, location, times, dates and causes could be extracted from the text. These entities would be kept as metadata about the articles and used when needed.

The Benefits

Text Analysis software can ‘understand’ the relationships between articles and provide suggestions to similar content. This benefits the editor as he or she can be far more productive as they navigate easily through a complete dataset. Research is, therefore, easier, a lot of time is saved and the end product is often richer, as the editor can reference similar events and give more depth and context to their article.

Richer, more relevant content can improve user engagement, meaning more page views by a narrower market, which can increase the potential for generating advertising revenue. A consumer that is more engaged with their content is far more likely to subscribe to niche newsletters, which can allow publishers to develop these relationships further and upsell their service to their consumers.

Our conclusion

Text analytics is essential in the publishing industry because it saves time when gathering data, allows you to produce richer content to attract more readers in narrower markets, where consumers are often more loyal.

Learn more about our API Drop us an email @mention us on Twitter

Sentiment Analysis: Going Beyond Positive and Negative

Apple Live probably got off to the worst start possible on Wednesday. Most of us who tried to log on to watch the much-anticipated launch were first, forced to watch the live feed in Safari and second, greeted with the TV Truck Schedule Screen…

image

To add to this Apple also made a complete mess of the audio. We were left sitting refreshing the page, waiting for the stream to start while being subjected to an audio visual nightmare, described brilliantly by this “fan” below:

Analyzing Tweets

At AYLIEN, we gathered 11 million+ tweets mentioning ‘Apple’, ‘iPhone’, ‘iOS’, ‘iPad’, ‘Mac’, ‘iPod’, ‘Macbook’, ‘iCloud’, ‘OS X’, ‘iWatch’ and ‘#AppleLive’ from the 4th of September to the 10th of September with a view of analyzing the tweets to gain insight into the voice of Apple Followers.

The Tweets over time graph covers tweets from 4pm on launch day to 12am that night. The top half of the graph displays the volume of tweets by journalists and the bottom displays tweets by the general public.

What’s interesting here is the difference in what the two groups got excited and most vocal about. The general public was most vocal on Twitter in anticipation of the iPhone updates, not the Keynote, not the Apple Watch, not Apple Pay or not even one of the biggest rock bands in the world announcing a free album giveaway to over 500 million people, no, just an announcement to say the iPhone is going to be bigger! The journalists however, as a group were far less biased towards any one announcement and tweeted about what they thought were the biggest announcements of the day and not surprisingly, given recent hype in the tech industry around payments, they were most vocal about Apple Pay.

Tweet Polarity

The disappointment of #AppleLive was evident on Twitter especially at the start of the launch.

Running Sentiment Analysis on the tweets we can see from the graph above that the overall polarity of tweets took a sharp dive into the negative in the build up to the event. The public weren’t afraid of expressing their opinion as to how frustrated they were to be missing out on the action and Apple’s Audiovisual team must have been scrambling to make things right.

The graph above shows the polarity of tweets throughout the launch. It’s pretty clear the difficulty people had streaming the live cast put a dampener on the mood overall with the sharp switch from positive to negative around 5 pm. However, Apple did manage to turn things round with some well received announcements which brought them back into the green. While it’s interesting to see the overall polarity of the tweets, things start to get more interesting when we look at what aspects in particular about the event were p*ssing people off?

Aspect-Level Sentiment Analysis

Whether you are analyzing Tweets, Articles or Reviews the overall sentiment and knowing whether it is positive, negative or neutral is cool and useful to know. However, Sentiment Analysis gets a lot more valuable and interesting when we can identify what aspects of entities in particular are positive or negative. Aspect Level Sentiment Analysis refers to identifying opinions or sentiments expressed on different features or aspects of entities; a phone, a camera, a bank. The aspects or features of entities would be say, the screen of a phone or the battery of a watch.

Consider the following tweet as an example: “The iPhone 5s is amazing, I love the camera and the OS is so fast. The battery is terrible though.”

The entity here is the iPhone and the aspects are the Camera, OS and Battery. If you were to focus on tweet-level sentiment for this particular example it would most likely be tagged as a positive tweet and you would most probably miss out on the negativity related to the battery.

The word cloud below highlights what in particular were the negative aspects of the Tweets about the launch. This information would be important to Apple in analyzing their brand, the overall success of the event and to get an understanding of what in particular people liked and disliked about the event. The same could also be done for particular products that were announced.

Extracting Insight

For the purpose of this blog we decided to focus on one aspect of the launch in particular from the Apple Live event: the Apple Watch.

The word clouds relating to the Apple watch highlight some key points which give a strong insight into the public reaction to the Apple Watch.

Positive:
  • People “want” one!
  • They think it “looks” “pretty”
  • They like that it comes in “34 styles”
  • They are “excited” by it.
Negative:
  • Some think it looks “ugly”
  • They are wary of the “battery life”

In saying that it’s also pretty interesting that both the positive and negative tweets tell us that most people just care about how it looks over any of its fancy features!

Conclusion

Sentiment Analysis doesn’t have to stop at high level understanding of whether or not a sample of tweets, reviews, emails, NPS scores etc are positive or negative. The ability to dive deeper into the “what” and “why” of positive or negative sentiment allows us to gain a better understanding of public opinion.

Whether we are looking for indications of what features people like and dislike in a product or offering or what aspects they place most importance on we can gain a deeper insight into opinions by focusing on aspects in Sentiment Analysis.

Learn more about our API Drop us an email @mention us on Twitter

Introducing Text Analysis for Google Sheets

Text analysis newbie? Seasoned data junkie? Do you spend time classifying and analyzing text documents? Are you trying to listen to the voice of your customers online?

We are excited to launch a super cool and powerful tool that brings Text Analysis capabilities to Google Sheets.

 With AYLIEN Text Analysis for Google Sheets you can:

  • Perform Sentiment Analysis on Social Media streams to identify what is positive, negative or neutral about your company or brand
  • Extract mentions of entities and concepts such as people, organizations, places and products from documents
  • Summarize long chunks of text and articles into a few key sentences
  • Detect the language of a document
  • Find the best hashtags for your content for better discoverability on Social Media
  • Classify your documents or links into more than 500 categories
  • Extract the full text of an article, as well as its author name, embedded media, etc.

We are really excited about this announcement as it brings the power of Text Analysis to data junkies, marketers and analysts who may not have the programming or technical expertise to use an API. This easy to use tool plugs straight into your spreadsheet and provides valuable insight from text in minutes.

Screenshot: Sentiment Analysis of Tweets



The AYLIEN Text Analysis add-on is the first and only text analysis tool available for Google Sheets. It is the easiest way to perform sophisticated text analysis on documents without leaving your Google Sheets environment. Powered by the AYLIEN Text Analysis API, this tool brings an easy to use package of Natural Language Processing and Machine Learning tools to your spreadsheets. It allows you to transform your spreadsheet into a powerful yet easy-to-use Text Analysis tools with no coding or text analytics experience needed to get started.

Getting started is easy, check out our videos and walkthrough guides to see how easy it is to get up and running on our tutorial page. Or you can watch our intro video below.



The AYLIEN Text Analysis Add-on is free to use for up to 1000 credits. Essentially 1 call = 1 credit. For users utilising more that 1000 credits credits can be topped up easily in the Chrome add-on store and your credits won’t expire.

Download your add-on today and start extracting reality from your data in your spreadsheets.

Text Analysis in Mobile App Development; Telerik/AYLIEN partnership

We are happy to announce that as part of our strategic partnership with Telerik our Text Analysis API is now even easier for mobile developers to incorporate into their apps and development process.

Telerik Platform

Telerik provides a platform on which developers can build Mobile apps quickly and efficiently. The platform allows you to design, build, connect, test, deploy, manage and measure sophisticated mobile apps across multiple platforms including android, iOS and windows. The advantages of the platform mean it is easier for developers to add features and capabilities to native apps that would otherwise entail some heavy lifting and domain expertise.

Text Analysis and Mobile Apps

There is no denying mobile device usage has been on the rise in recent years and according to a study carried out by Flurry more and more of us are spending our browsing time in native apps and moving away from mobile web browsing. As much as 86% of web use on mobile devices is spent in native apps today. This has meant mobile development has risen high in priority as companies scramble to kick their mobile projects into gear.

What can be accomplished today with Text Analytics and the insight it can provide to developers and business stakeholders alike, has meant that there has also been a steady increase in adoption and application of Text Analysis practices in mobile app development.

Providing a quick and efficient way for mobile developers to harness and utilize the powers of Text Analysis in their apps and development process is something we have been focusing on in the past couple of months.

Our Partnership

Telerik recently announced the launch of an industry first verified plugin marketplace. This Cordova/PhoneGap verified marketplace aims to showcase and promote custom plugins which allow developers to extend the functionality of their mobile apps.

This Plugin marketplace provided an interesting opportunity for us at AYLIEN to bring the power of Text Analysis straight to the development process of mobile apps. The plugin allows sophisticated text analysis capabilities to be easily incorporated into a mobile app without any text analytics knowledge or expertise.

Our AYLIEN Plug-in is now available in the Telerik Plugin Marketplace. This plugin allows you to extract structured information and insights from text. It performs Summarization, Sentiment Analysis and Hashtag Suggestion for any piece of text (tweets, articles, form submissions, documents etc.).

To get started with the plugin:

Step 1. Grab your free AYLIEN Text Analysis API Key here.

Step 2. Download your plugin from the plugin marketplace here

Or alternatively, you can utilise the prebuilt demo app available in the Telerik Plug-in Marketplace to test it out first.



Learn more about our API Drop us an email @mention us on Twitter

Text Analytics meets 2014 World Cup tweets - Part 2

In the first part of our World Cup 2014 blog series, we analyzed 30 million tweets collected between June 6th and July 14th about the biggest sporting event in the world, FIFA World Cup 2014, and we looked at some high-level associations and insights about the tournament: in a nutshell, we observed a repeating pattern of spikes appearing in tweet volumes around match times and important events. In the second part of the series, we’re going to dive deep into the tweets and analyze their content using our very own Text Analysis API and Rapidminer to get a more in-depth view of the data.

Data

We’re using the same datasets that were used in part 1 (tweets.csv) plus a new dataset called tweets-sentiment.csv, which contains the sentiment polarity and subjectivity results obtained using our Sentiment Analysis API in tweet mode.

Top hashtags and mentions

Let’s start our analysis by finding the most popular hashtags and @ mentions from the tournament, by tokenizing tweets and sorting the tokens by frequency:

Sentiment Analysis

We’re now going to look into the polarity values (“positive” or “negative”) of these tweets to see what these values are for different entities and how they change over time, as a result of various events.

Note: we are only analyzing English tweets for the following examples, which introduces a sampling bias. The following charts and insights are based on the opinions of the English-speaking Twitter users.

Sentiment over time

Different events concerning players or a teams affect how people think and talk about them. Using polarity analysis, we can get an idea of people’s reaction to various events, which can provide valuable insight. Let’s look at two major talking points from the tournament as examples: Luis Suarez and the Brazil’s shocking performance.

1. On June 24th, Argentine Luis Suarez was largely accused of biting Italy defender Giorgio Chiellini, which was followed by a big wave of negative comments and feedback from Social Media. Suarez issued an apology on June 30th, which seems to have been satisfactory for the Twitter community (take note PR people!):

2. Brazil had arguably one of its worst performances in World Cup history during the 2014 tournament. This is pretty evident when you analyze the sentiment of tweets about #BRA after every lost match or controversial win:

Before the 3rd place playoff game between Brazil and Netherlands, people were hopeful that the catastrophic loss against Germany might bring the best out of the Brazilians. However, a few minutes into the game it’s pretty clear this was no longer the case:

Popularity by sentiment

We can use the average polarity measures for various entities to see how positively or negatively people talk about them.

Teams

Average polarity for the 16 teams that qualified for the second round:

Players

Average polarity for top 10 scorers as well as two noteworthy players, Tim Howard of USA and Luis Suarez of Argentina:

Most ‘polar’ hashtags and mentions

Finally, let’s look at some of the most positive and negative hashtags and mentions:



Final notes

Analyzing the sentiment of tweets gives an extraordinary view into the opinions of the public in relation to a certain topic or event. Listening to “social chatter” allows you to extract detailed insight into opinions and trends on brand, companies, events, football teams etc. and how they change over time, with say, the launch of a product, a company announcement, a crisis event or in the case above a footballer biting another player.

In Suarez’s case his “brand” took a major hit and “social chatter” about him turned pretty sour following the biting incident, however, his PR teams involvement and his deal at Barcelona allowed him to bounce back quite quickly, shown quite clearlyin the switch in polarity of tweets about him.

To learn more about Sentiment Analysis check out our recent blog posts. If you are Interested in analyzing the sentiment of text, tweets, comments or reviews you can get free access to our Text Analysis API.

Learn more about our API Drop us an email @mention us on Twitter