Product

Getting Started With Image Tagging for AYLIEN API

Just this week, we added Image Tagging capabilities, to our API. The result of a strategic partnership with Imagga, our Image Tagging endpoint is the first step in providing a Hybrid Text and Image analysis API.

So how does it work?

Our Image Tagging, uses advanced image recognition and deep learning technology, to recognize and identify objects in an image. From a dataset of over 6000 objects, it then suggests candidate tags for that image along with confidence scores.

The Image Tagging feature automates image annotation, categorization and tagging, a task that is often time-consuming and laborious. It’s designed to reduce the heavy lifting involved in dealing with images.

The quickest way to get up and running with this endpoint is to grab an SDK of your choice and check out our detailed documentation.

Analyzing and Tagging images

For this blog post, we’re going to run through a couple of simple examples to showcase the image tagging endpoint using Node.js.

As a simple example, let’s analyze the following image and see what type of tags the API suggests for it. It’s pretty clear to the human eye that it’s a house, but what exactly will the API actually recognize in the image? Let’s find out!

image

Image: http://suburbanfinance.com/wp-content/uploads/2013/04/streetinfo.jpg

Code:


textapi.imageTags(
    'http://suburbanfinance.com/wp-content/uploads/2013/04/streetinfo.jpg',
    function(err, response, ratelimits) {
        if (err !== null) {
            console.log("Error: " + err);
        } else {
            response.tags.forEach(function(t) {
                console.log("Tag : ", t.tag + " , " + "Confidence : ", t.confidence);
            });
        }
    });

Results:


Tag :  mansion , Confidence :  0.47509114790445056
Tag :  driveway , Confidence :  0.4552631524330092
Tag :  house , Confidence :  0.4304875785176193
Tag :  architecture , Confidence :  0.33942988236938837
Tag :  building , Confidence :  0.3094331548632239
Tag :  home , Confidence :  0.2697299038194921
Tag :  estate , Confidence :  0.23534657602532882
Tag :  dwelling , Confidence :  0.23327752776762856
Tag :  structure , Confidence :  0.22539033224250088
Tag :  residential , Confidence :  0.2150613973041477
Tag :  residence , Confidence :  0.19566665898999108
Tag :  housing , Confidence :  0.19091773340752613...

The results will be returned displaying a number of candidate tags and a confidence score for each. The confidence score tells you how sure the API is that the tag it suggested is correct. It faired pretty well with this image and returned some pretty accurate results.

Let’s try something a little different this time, the last image was a little easy. Let’s say we want to analyze the image below.

image

Image: http://www.tencate.com/amer/Images/American%20football29-15343.jpg

Code:


textapi.imageTags(
    'http://www.tencate.com/amer/Images/American%20football29-15343.jpg',
    function(err, response, ratelimits) {
        if (err !== null) {
            console.log("Error: " + err);
        } else {
            response.tags.forEach(function(t) {
                console.log("Tag : ", t.tag + " , " + "Confidence : ", t.confidence);
            });
        }
    });

Results:

Tag :  football , Confidence :  1
Tag :  stadium , Confidence :  0.24106947076803517
Tag :  back , Confidence :  0.17014749275097063
Tag :  structure , Confidence :  0.12840672955565677
Tag :  game , Confidence :  0.1186865418441251
Tag :  sport , Confidence :  0.10223847715591786
Tag :  field , Confidence :  0.09827027407477171
Tag :  team , Confidence :  0.09808334899832503
Tag :  ball , Confidence :  0.09400727111771828
Tag :  people , Confidence :  0.09279897917842066...

Again, the API returned a number of accurate and relevant tags. This time it recognized it was a picture related to Football and Sport and returned a variety of results, based on what it saw.

The Image Tagging feature, is not just able to spot objects, but it can also recognize humans and certain aspects in a photo of a person. Let’s see what it recognizes in this photo, of pop singer, Beyonce.

image

Image:https://ronehiphopnc2.files.wordpress.com/2013/05/beyonce-knowles-closeup-1024x576.jpg

Code:


textapi.imageTags(
    'https://ronehiphopnc2.files.wordpress.com/2013/05/beyonce-knowles-closeup-1024x576.jpg',
    function(err, response, ratelimits) {
        if (err !== null) {
            console.log("Error: " + err);
        } else {
            response.tags.forEach(function(t) {
                console.log("Tag : ", t.tag + " , " + "Confidence : ", t.confidence);
            });
        }
    });

Results


Tag :  attractive , Confidence :  0.3641480251093146
Tag :  portrait , Confidence :  0.3570435098374641
Tag :  adult , Confidence :  0.3419036072980034
Tag :  model , Confidence :  0.3381432174429411
Tag :  person , Confidence :  0.32852624615621456
Tag :  pretty , Confidence :  0.3228676609396025
Tag :  complexion , Confidence :  0.3164959079957736
Tag :  face , Confidence :  0.313449951118162...

The results when it comes to an image like the one above are a little bit more advanced or intelligent. The API is confident that the image of Beyonce is an attractive portrait of a model, which isn’t far off.

So, that’s how easy it is to start analyzing images with AYLIEN. Keep an eye out for our next blog on, common and not so common use cases, for hybrid text and image analysis.

Author


Avatar

Mike Waldron

Head of Marketing & Sales @ AYLIEN A legal convert with a masters degree from Smurfit Business School, Mike runs our Sales and Marketing at AYLIEN. Mike gathered his Sales and Marketing experience with technology companies in Sydney and Dublin before getting the startup itch and joining the team at AYLIEN. Twitter: @MikeWallly