Good Contents Are Everywhere, But Here, We Deliver The Best of The Best.Please Hold on!
Your address will show here +12 34 56 78

Introduction

Super Bowl 51 had us on the edge of our seats. A dramatic comeback and a shocking overtime finish meant the 111.3 Million Americans who tuned into the event certainly got what they came for. Even though TV viewership was down on previous years, the emotional rollercoaster that was Sunday’s game will certainly go down as one of the greatest.

As with any major sporting event, the Super Bowl creates an incredible amount of hype, particularly on Social Media. All of the social chatter and media coverage around the Super Bowl means it’s a fantastic case study in analyzing the voice of fans and their reactions to the event. Using advanced Machine Learning and Natural Language Processing techniques, such as Sentiment Analysis, we are able to understand how fans of both the Patriots and the Falcons collectively felt at any given moment throughout the event.

Not familiar with Sentiment Analysis? Sentiment Analysis is used to detect positive or negative polarity in text and can help you understand the split in opinion from almost any body of text, website or document.

Our process

We used the Twitter Streaming API to collect a total of around 2.2 million tweets that mentioned a selection of game and team-related keywords, hashtags and handles. Using the AYLIEN Text Analysis API, we analyzed each of these tweets and visualized our results using Tableau. In particular, we were interested in uncovering and investigating the following key areas:

  • Volume of tweets before, during and after the game
  • Sentiment of tweets before, during and after the game
  • Team-specific fan reactions
  • The most tweeted players
  • The most popular Super Bowl hashtag

Keyword selection

We focused our data collection on keywords, hashtags and handles that were related to Super Bowl 51 and the two competing teams, including;

#SB51, #superbowl, #superbowlLI, #superbowl51, #superbowl2017, #HouSuperBowl, #Patriots, #NEPatriots, #newenglandpatriots, #Falcons, #AtlantaFalcons.

Once we collected all of our tweets, we spent a bit of time cleaning and prepping our data set, first by disregarding some of the metadata which we felt we didn’t need. We kept key indicators like time stamps, tweet ID’s and the raw text of each tweet. We also removed retweets and tweets that contained links. From previous experience, we find that tweets containing links are mostly objective and generally don’t hold any author opinion towards the event.

Tools we used

Visualizations

Like with many of our data-driven blog posts, we used Tableau to visualize our results. All visualizations are interactive and you can hover your mouse over each one to dive deeper into the key data points from which they are generated.

We began our analysis of Super Bowl 51 by looking at the overall volume of tweets in the lead up and during the game.

Tweet volume over time: all tweets

The graph below represents minute-by-minute fluctuations in tweet volumes before during and after the game. For reference, we’ve highlighted some of the key moments throughout the event with the corresponding spikes in tweet volume.

As you can see, there is a definite and steady increase in tweet volume in the period leading up to the game. From kickoff, it is then all about reactions to in-game highlights, as seen by the sharp spikes and dips in volumes. We’ve also highlighted the halftime period to show you the effect that Lady Gaga’s performance had on tweet volumes.

Let’s now take a closer look at the pre-game period and in particular, fan predictions.

Pre-game tweet volume: #PatriotsWin vs. #FalconsWin

For the past 13 years, video game developers EA Sports have been using their football game ‘Madden NFL’ to simulate and predict the winner of the Super Bowl each year. They now have a 10-3 success-failure rate, in case you were wondering! In recent times, they have also been inviting the Twittersphere to show their support for their team by using a certain hashtag in their tweets. For 2017, it was #PatriotsWin vs. #FalconsWin.
So, which set of fans were the most vocal in the 2017 #MyMaddenPrediction battle? We listened to Twitter in the build up to the game for mentions of both hashtags, and here’s what we found;

58.57% of tweets mentioned #FalconsWin while 41.43% went with #PatriotsWin. While the Patriots were firm pre-game favorites, it is likely that the neutral football fan on Twitter got behind the underdog Falcons as they chased their first ever Super Bowl win, in just their second appearance.

Tweet volume over time by team

Now that we’ve seen the overall tweet volume and the pre-game #MyMaddenPrediction volumes, let’s take a look at tweet volumes for each individual team before, during and after the game.
The graph below represents tweet volumes for both teams, with the New England Patriots in the top section and the Atlanta Falcons in the bottom section.

Talk about a game of two halves! That vertical line you can see between the two main peaks represents halftime, and as you can see, Falcons fans were considerably louder in the first half of the game, before the Patriots fans brought the noise in the second half as their team pulled off one of the greatest comebacks in Super Bowl history.

Sentiment analysis of tweets

While tweet volumes relating to either team can be a clear indicator of their on-field dominance during various periods of the game, we like to go a step further and look at the sentiment of these tweets to develop an understanding of how public opinion develops and fluctuates.

The charts below are split into two sections;

Top: Volume of tweets over time, by sentiment (Positive / Negative)

Bottom: Average sentiment polarity over time (Positive / Negative)

New England Patriots

What’s immediately clear from the chart above is that, for the majority of the game, Patriots fans weren’t too happy and it seems had given up hope. However, as you can see by the gradual increase in positive tweets sentiment and volume in the final third, their mood clearly and understandably changes.

Atlanta Falcons

In stark contrast to the Patriots chart, Falcons fans were producing high volumes of positive sentiment for the majority of the game, until the Patriots comeback materialized, and their mood took a turn for the worse, as indicated by the drop of sentiment into negative.

Most tweeted individuals

To get an understanding of who people were talking about in their tweets, we looked at the top mentioned individuals. Unsurprisingly, Tom Brady was heavily featured after his 5th Super Bowl triumph.However, the most mentioned individual had no part to play in the actual game.

All notable players and scorers (and even Brady himself) were shrugged aside when it came to who the viewers were talking about and reacting to most on Twitter, as halftime show performer Lady Gaga dominated. To put the singer’s domination into perspective, she was mentioned in nearly as many tweets as Brady and Ryan were combined!

To get an idea of the scale of her halftime performance, check out this incredible timelapse;


Interestingly, national anthem singer Luke Bryan was tweeted more than both the Patriots’ Head Coach Bill Belichick and catch-of-the-game winner Julian Edelman. Further proof, if needed, that the Super Bowl is not just about the game of football, but that it is becoming more and more of an entertainment spectacle off the field.

Most popular Super Bowl hashtags

We saw a variety hashtags emerge for the Super Bowl this year, so we decided to see which were the most used. Here are the top 5 most popular Super Bowl hashtags, which we have visualized with volumes below;

#SuperBowl

#SB51

#SuperBowl2017

#SuperBowlLI

#SuperBowl51

Despite the NFL’s best efforts to get Twitter using #SB51, the most obvious and simple hashtag of #SuperBowl was a clear winner.

Conclusion

There is no other event on the planet that creates as much hype in the sporting, advertising and entertainment worlds. But the Super Bowl as we know it today, is far less about the football and more about the entertainment factor and commercial opportunity. With big brands spending a minimum $5 Million for a 30 second commercial, competition for viewers eyes and more importantly viewers promotion through shares and likes on social media, the Super Bowl has become big business.

In our next installment, we’ve analyzed the chatter around Super Bowl 51 from a branding point of view. We collected and analyzed Twitter data and news and media coverage of the event to pinpoint which brands and commercials joined the Patriots as Super Bowl 51 champions.





Text Analysis API - Sign up




0

One of our API users Farhad, from Taskulu recently published a blog post on how he uses AYLIEN Text Analysis API for lead generation on Twitter.


As Farhad puts it himself; “There are many people out there literally asking you to introduce your product to them so they can become your customers, but the problem is that it’s very difficult to find them!”

The idea behind the app he created is simple; discover, understand and get involved in conversations about management platforms on twitter, as they happen.

With about 50 lines of code, a free AYLIEN subscription and a Google spreadsheet, Taskulu are now actively mining twitter in real time for “Tweeleads.” Tweets made by users who are looking for alternative project management platforms. In the first 8 hours of having the App live, Fahad had managed to gather about 35 “Tweeleads” in a spreadsheet. He has since been following up on each lead and has been having some interesting conversations with prospects since.

For those of you who want to try it out, they’ve published the code for generating Tweeleads on Github. You can also find the documentation and instructions for setting it up on the Github page.

The whole process doesn’t take longer than 5 minutes! If you want to give it a go, you’ll need to do the following:

Farhad Hedayati is co-founder and CEO of Taskulu, a management platform that lets you keep all the stakeholders and resources of your project inside one project – making project managers lives, team management and communication a lot easier. He regularly writes about growth hacking, marketing techniques, startups, and occasionally programming, you can Follow him on Twitter: @farhad_hf

If you have an interesting application for Text Analysis or our API then let us know. We love to see our users hack simple and useful apps together off our API’s.

 

0

At AYLIEN, we do our best to make sure our users get up and running and calling our API in the shortest time possible. As part of a new initiative, we are going to be sharing use case ideas, source code and fully functional apps to help you get the most out of our API. For this edition of the blog, we are going to focus on a pretty common use case that a lot of users want to use our API for, analyzing Tweets.

There is a wealth of insight that can be extracted from Tweets. You can read more on Analyzing Tweets and Social data in our previous blog; Why is Sentiment Analysis important from a business perspective.

Today, we’re going to provide you with the source code for a functioning app that mines Twitter for keywords, extracts Tweets and analyzes the text in the Tweets. As part of the process, we’ll run two analysis endpoints on each Tweet, Sentiment Analysis on all of the Tweets and Hashtag Suggestion on Tweets that contain a URL.

What you’ll need to get going:

 

  • Twitter API access: Get your api_key, consumer_secret key and access token to make calls to the Twitter API here.
  • Node.js running on your machine: If you don’t already have Node.js on your machine you can download it here.
  • Twit: A Twitter API client library for Node.js. You can download it here.
  • A text editor: You can use any editor, We recommend Sublime Text and you can download it here.
  • AYLIEN Text Analysis API access: Get your Application ID and Application Key here. See our ‘getting started’ blog for details on how to sign up.

 

Overview of the code in action

To give an overview of what can be achieved, we will first look at the code in action. The complete code snippet is given at the end of this blog for you to copy and paste.

Step1. Setup your Environment

Ensure Node.js is running on your machine, download the twit client library from GitHub, get access to the Twitter API and finally, open an AYLIEN Text Analysis API account.

Step 2. Copy the Code

Open your text editor and copy and paste the code snippet (provided at the bottom of this blog) and save the file as, tweetsentiment.js. Next, open command prompt and Navigate to the folder where you saved the code snippet.

The application takes two command line parameters which you chose; a keyword for the Twitter query and the number of Tweets the query should return.

Step 3. Run your Code

Run the code by typing “tweetsentiment websummit 3”. In this case we are querying the keyword “websummit” and asking for 3 Tweets to be returned.

Once the Tweets are returned by the Twitter API they are fed to AYLIEN Text Analysis API, where the polarity will be determined and where the optimal Hashtags for URL’s will be generated.

Note: Ensure you replace the YOUR_APP_ID and YOUR_APP_KEY placeholders in the code with the application id and application key which you received when you signed up for the AYLIEN API. You will also need to fill in your specific Twitter API credentials that you received from Twitter. All going well you should see an output on the command line similar to that shown below.

Results:

Tweet Text           :  RT @IndoBusiness: The #WebSummit is drawing to a close. #Bono up soon. Watch live here: http://t.co/onpdYvoIy4 Or follow our blog here: htt.
Sentiment Polarity   :  neutral
Polarity Confidence  :  0.9702560119839743
Hashtags :  [
  '#RyderCup',
  '#PeterThiel',
  '#Davos',
  '#AdrianGrenier',
  '#PaulMcGinley',
  '#FoundersFund'
]
Tweet Text           :  RT @FierceClever: I swear to god, if I hear the word "di
srupt" one more time... #websummit
Sentiment Polarity   :  negative
Polarity Confidence  :  0.8947368421052632
Hashtags :  No Hastags available as no Url specified in the Tweet
Tweet Text           :  Having a great day at the #websummit . Were at stand ECM 243 in the village if anyone would like to pop over before closing!
Sentiment Polarity   :  positive
Polarity Confidence  :  0.9230769230769231
Hashtags :  No Hastags available as no Url specified in the Tweet

Taking the first result as an example, you can see that the Tweet itself is displayed followed by the “sentiment polarity” of the Tweet (positive, neutral or negative) and the “polarity confidence” i.e. the confidence that the sentiment returned was correct (a number between 0 and 1). Finally, if the Tweet contained a URL embedded in the Tweet a list of optimal hashtags is generated for that webpage/article.

How the code works

It’s worth looking at the two parts of the solution that do most of the heavy lifting :

1. Querying Twitter

Querying Twitter is very straight forward using the twit client and requires just one line of code:

T.get('search/tweets', {
	q: process.argv[2],
	count: process.argv[3]
}, function(err, data, response) {
  data.statuses.forEach(function(s) {
...

The above line of code uses the supplied command line arguments to query Twitter, it then passes the returned results one by one to the function that will feed the body of the Tweet and the embedded URL (if any) to the AYLIEN API endpoints for analysis.

2. Analyzing the Tweets that are returned.

The function below, takes the following arguments, the AYLIEN endpoint to call (Sentiment, Hashtags, Entities etc.) the parameters which the endpoint should work on (i.e. we indicate whether we are passing a piece of text or a URL for analysis and we also pass the actual text or URL) and a callback function to call when the analysis is complete.

function call_api(endpoint, parameters, callback) {
  var postData = querystring.stringify(parameters);
  var request = https.request({
    host: 'api.aylien.com',
    path: '/api/v1/' + endpoint,
    headers: {
      'Accept':                             'application/json',
      'Content-Type':                       'application/x-www-form-urlencoded',
      'Content-Length':                     postData.length,
      'X-AYLIEN-TextAPI-Application-ID':    APPLICATION_ID,
      'X-AYLIEN-TextAPI-Application-Key':   APPLICATION_KEY,
    }
  }, function(response) {
    var data = "";
    response.on('data', function(chunk) {
      data += chunk;
    });
    response.on('end', function() {
      callback(JSON.parse(data));
    });
  });
  request.write(postData);
  request.end();
}

The examples we have used, analyze the Tweets for Sentiment and Hashtag Suggestions. It’s up to you what endpoints you wish to use. Maybe you want to extract entities or concepts from the Tweets as well. A full list of our endpoints can be found in our documentation.

The Code Snippet

var Twit = require('./node_modules/twit')  //Twitter API client library
var https = require('https'), querystring = require('querystring');

//AYLIEN API Credentials
const APPLICATION_KEY = YOUR_APPLICATION_KEY,
      APPLICATION_ID = YOUR_APPLICATION_ID;


//Twitter API Credentials
var T = new Twit({
    consumer_key:         YOUR_TWITTER_CONSUMER_KEY
  , consumer_secret:      YOUR_TWITTER_CONSUMER_SECRET
  , access_token:         YOUR_TWITTER_ACCESS_TOKEN
  , access_token_secret:  YOUR_TWITTER_ACCESS_TOKEN_SECRET
})


var analysisResults = {};
var parameters;
var i =  process.argv[3] * 2; //Counter to track when Asynchronous API call have completed

console.log("Processing your request. Please wait...")
console.log("n");

T.get('search/tweets', { q: process.argv[2], count: process.argv[3] }, function(err, data, response) {
  data.statuses.forEach(function(s) {
    var returnedUrls = s.entities.urls;
    analysisResults[s.id] = {};
    analysisResults[s.id].text = s.text;
  	parameters = {'text': s.text};
    callAylienAPIs(parameters,outputResults);

    function callAylienAPIs(parameters, callback) {

      call_api('sentiment', parameters, function(result) {
        var a = {};
        a.endpoint = 'sentiment';
        a.polarity = result.polarity;
        a.polarity_confidence = result.polarity_confidence;
        analysisResults[s.id].sentiment = a;
        i--;
        if (i == 0)  {
          callback();
        }
      })

      if (returnedUrls.length > 0 ) {
          var url_paramaters = {'url' : returnedUrls[0].expanded_url };
          call_api('hashtags', url_paramaters, function(result) {
            var a = {};
            a.endpoint = 'hashtags';
            a.hashtags = result.hashtags;
            analysisResults[s.id].hashtags = a;
            i--;
            if (i == 0)  {
              callback();
            }
          })
        } else {
            var a = {};
            a.endpoint = 'hashtags';
            a.hashtags = 'No Hastags available as no Url specified in the Tweet';
            analysisResults[s.id].hashtags = a;
            i--;
            if (i == 0)  {
              callback();
            }
        }
    }
  });

})


function outputResults() {

    for (var key in analysisResults) {
    console.log("Tweet Text           : ", analysisResults[key].text);
    console.log("Sentiment Polarity   : ", analysisResults[key].sentiment.polarity);
    console.log("Polarity Confidence  : ", analysisResults[key].sentiment.polarity_confidence);
    console.log("Hashtags : ", analysisResults[key].hashtags.hashtags);
    console.log("n");
    }
}




function call_api(endpoint, parameters, callback) {
  var postData = querystring.stringify(parameters);
  var request = https.request({
    host: 'api.aylien.com',
    path: '/api/v1/' + endpoint,
    headers: {
      'Accept':                             'application/json',
      'Content-Type':                       'application/x-www-form-urlencoded',
      'Content-Length':                     postData.length,
      'X-AYLIEN-TextAPI-Application-ID':    APPLICATION_ID,
      'X-AYLIEN-TextAPI-Application-Key':   APPLICATION_KEY,
    }
  }, function(response) {
    var data = "";
    response.on('data', function(chunk) {
      data += chunk;
    });
    response.on('end', function() {
      callback(JSON.parse(data));
    });
  });
  request.write(postData);
  request.end();
}





Text Analysis API - Sign up




1