Product

Text Analysis and Python: Getting Started with AYLIEN Text Analysis API and Python

This is a continuation of our, “Getting up and running with AYLIEN Text Analysis API” blog series. Our last edition walked you through how to start making calls with our Text Analysis API and Java. For this blog, we are going to focus on working with the API using Python.

We’re going to perform some basic Text Analysis processes like, detecting what language a piece of text is written in, analyzing the sentiment of a piece of text and finally generating some hashtags for a URL.

As an overview and to showcase what can be achieved, we’re first going to look at the code in action. We’ll then go through the code, section by section, to investigate each of the endpoints used in the code snippet.

As a simple example, we are going to do the following:

    • Detect what language the following text is written in: “What language is this sentence written in?”
    • Analyze the sentiment of the following statement: “John is a very good football player!”
    • Generate hashtags for the following URL: “http://www.bbc.com/news/health-29912877”Note: We’re using Python 3 compatible code for this blog, the getting started page on our website has Python 2 and Python 3 compatible code snippets for you to try.

Overview of the code in action

The complete code snippet is given at the end of this blog for you to copy and paste. To run it, open a text editor of your choice and copy and paste the snippet. Before running the code, ensure you replace the YOUR_APP_ID and YOUR_APP_KEY placeholders in the code with your own application id and application key which would have been sent to you upon signing up to the API.

Save the file as TextAPISample.py and then open command prompt. Navigate to the folder where you saved the code snippet and run the code by typing “python TextAPISample.py”.

Note: You will need to have python installed to run this example, you can download it here, if you haven’t done so already.

Once you run it, you should receive the output as shown below:

In this case we have detected that the first piece of text is written in English, the sentiment or polarity of the second statement is positive and we have generated hashtags for the URL, that points to a BBC story on how the brain processes different tastes.

The detail above shows the code running in its entirety, but to highlight each feature/endpoint we will now go through the code snippet, section by section.

Language Detection

Using the Language Detection endpoint you can analyze a piece of text or a URL. In the piece of code we have used in this demo code, the “parameter” variable controls whether the call is made specifying the text directly or as a URL.

parameters = {"text": "What language is this sentence written in?"}
language = call_api("language", parameters)

In this case we have specified that it should analyze the following text “What language is this sentence written in?” and as you can see from the output below, it determined that the text is written in English and it gave a 0.999993 confidence score that the language was detected correctly. Note: For all of the endpoints, the API returns the text which was analysed for reference and we have included it in the results in each case.

Result:

Text: What language is this sentence written in?
Language: en (0.999993)

 

 

Sentiment Analysis

Similarly, the Sentiment Analysis endpoint can take a piece of text or a URL and analyze it to determine whether it is positive, negative or even neutral.

parameters = {"text": "John is a very good football player!"}
sentiment = call_api("sentiment", parameters)

In this case, we have specified that it should analyze the text “John is a very good football player!”. The API has determined that the sentiment of the piece of text is positive, we can also be pretty sure it’s correct based on the confidence score returned of 0.999999.

Result:

Text: John is a very good football player!
Sentiment: positive (0.999999)

 

 

Hashtag Suggestions

Finally, the Hashtag Suggestion endpoint, analyses a URL and generates a list of hashtag suggestions which can be used to ensure your content or URL’s are optimally shared on social media:

parameters = {"url": "http://www.bbc.com/news/health-29912877"}
hashtags = call_api("hashtags", parameters)

For hashtag suggestions, we have used an article about how the brain processes tastes published on the BBC news website http://www.bbc.com/news/health-29912877. The hashtag endpoint first extracts the text from the URL (which is returned for reference by the call and the start of which I have shown below) and then analyses that text and generates hashtag suggestions.

Result:

Hashtags: ['#Umami', '#Neuron', '#ColumbiaUniversity', '#Brain', '#Sugar', '#Carnivore', '#Species', '#VampireBat', '#TasteBud', '#StemCell', '#BBCNews', '#Fluorescence', '#Road', '#TonicWater']
Text: Brain's taste secrets uncovered
The brain has specialist neurons for each of the five taste categories - salty, bitter, sour, sweet and umami - US scientists have discovered.
The study, published in the journal Nature, should settle years of debate on how the brain perceives taste.
The Columbia University team showed the separate taste sensors on the tongue had a matching partner in the brain...

For more getting started guides and code snippets to help you get up and running with our API, visit our Getting Started page on our website. If you haven’t already done so you can get free access to our API on our sign up page.

    The complete code snippet:

    import json
    import urllib.request, urllib.error, urllib.parse
    
    APPLICATION_ID = YOUR_APP_ID
    APPLICATION_KEY = YOUR_APP_KEY
    
    def call_api(endpoint, parameters):
      url = 'https://api.aylien.com/api/v1/' + endpoint
      headers = {
          "Accept":                             "application/json",
          "Content-type":                       "application/x-www-form-urlencoded",
          "X-AYLIEN-TextAPI-Application-ID":    APPLICATION_ID,
          "X-AYLIEN-TextAPI-Application-Key":   APPLICATION_KEY
      }
      opener = urllib.request.build_opener()
      request = urllib.request.Request(url,
        urllib.parse.urlencode(parameters).encode('utf-8'), headers)
      response = opener.open(request);
      return json.loads(response.read().decode())
    
    parameters = {"text": "What language is this sentence written in?"}
    language = call_api("language", parameters)
    
    parameters = {"text": "John is a very good football player!"}
    sentiment = call_api("sentiment", parameters)
    
    parameters = {"url": "http://www.bbc.com/news/health-29912877"}
    hashtags = call_api("hashtags", parameters)
    
    print("n")
    print("Text: %s " % (language["text"]))
    print("Language: %s (%F)" % (language["lang"], language["confidence"]))
    print("n")
    print("Text: %s " % (sentiment["text"]))
    print("Sentiment: %s (%F)" % (sentiment["polarity"], sentiment["polarity_confidence"]))
    print("n")
    print("Hashtags: %s " % (hashtags["hashtags"]))
    print("n")
    print("Text: %s " % (hashtags["text"]))
    

 




Text Analysis API - Sign up




Author


Avatar

Mike Waldron

Head of Marketing & Sales @ AYLIEN A legal convert with a masters degree from Smurfit Business School, Mike runs our Sales and Marketing at AYLIEN. Mike gathered his Sales and Marketing experience with technology companies in Sydney and Dublin before getting the startup itch and joining the team at AYLIEN. Twitter: @MikeWallly