Product

How to Create Your Own Personalized TL;DR Newswire

We like to share simple and useful examples of how we use our Text Analysis API to automate everyday tasks and business processes. What could be seen as a life hack of sorts, we’ve put together a useful script you can use, to stay on top of your favorite news resources or content outlets.

If you’re like us you’ll have a morning routine of checking various news sites, blogs and forums for interesting news or content, either to share or just for personal interest. Trawling through news sites and different outlets can be time consuming, jumping from site to site and skim reading articles can be a bit of a pain and using a news aggregator often doesn’t provide the personalization of content it promises.

With this in mind, we decided hack together a useful script in Node.js to do the hard work for you. This script, available on GitHub , will track an RSS feed or feeds that you specify and email you a personalized daily digest. I know what you’re thinking, another daily digest! But this is different, using Artificial Intelligience, we’ll automatically categorize your articles and summarize them into a TL;DR, so you’ll know immediately which articles to read, what you should share and what to ignore in no time at all.

At AYLIEN, we use this hack to stay on top of useful and interesting content gathered from a range of different outlets. It means we don’t have to spend time checking up on the various news sites and blogs we curate content from. The content we gather is either used internally or we share it through other channels if we think it’s of interest to our customers and followers.

We’re going to take you through how the script works so that you can customize it to monitor a feed or feeds of your choice.

What you’ll need

For this script to run, you’re going to need the following:

  • A Text Analysis API Key (free)
  • A Mailgun account (free)
  • Node.js
  • An RSS feed or multiple feeds to monitor

Setup

Step 1.

Get your accounts with AYLIEN and Mailgun setup and note your AYLIEN_APP_ID, AYLIEN_APP_KEY and MAILGUN_API_KEY which you’ll need to run the script later.

Step 2.

Find the URL(s) for the feed or feeds you want like to monitor. For this example, we’re going to use http://www.kdnuggets.com/feed an interesting blog on all things data and analytics related.

Step 3.

Copy and paste the javascript code given at the end of this blog into your text editor or grab the code from GitHub

Step 4.

At the top of the script, there’s a section to fill in your specific AYLIEN and Mailgun credentials. As below:

const AYLIEN_APP_ID = '123456789';
const AYLIEN_APP_KEY = '123456789;
const MAILGUN_API_KEY = '12345678911';
const SENDING_EMAIL_ACCOUNT = 'sending@example.com';
const RECIPIENTS = 'Recipient 1 <recipient1@example.com>';
const NUM_STORRIES_PER_RSS_FEED = 3;
var rssFeeds = ["http://www.kdnuggets.com/feed", "http://news.ycombinator.com/rss"];

Step 5.

Save the script with the name rssToEmail.js or whatever you choose. Next, open command prompt/terminal, navigate to the folder where you saved the script and run it.


node rssToEmail.js

If you’re running the script in a windows environment you’ll see a response similar to the following.


C:srcjavascript>node rssToEmail.js
Processing rss feed - http://www.kdnuggets.com/feed...
Processing rss feed - http://news.ycombinator.com/rss...
Preparing Digest email...
Success

C:srcjavascript>

Results

Provided your code and setup are running OK, you should now have an email in the inbox you specified titled ‘Your Daily RSS Digest’. The email will resemble the one below:

As you can see in the sample image above, each story will have a Title extracted from the URL, a two-tier classification, a link to the original article and most importantly a TL;DR of the article to give you quick hit of info. Which should give you enough information to decide whether you want to read, share or ignore it.

Try adding Hashtag Suggestions if you’re hoping to use it as a content curation tool. That will include optimal hashtags, along wth the other data, for you to use if your resharing the content ensuring you’ll get maximum exposure on social sites.

Code Snippet

 

//**************************************
//Configuration details specific to you.
//**************************************
const AYLIEN_APP_ID = YOUR_AYLIEN_APP_ID;
const AYLIEN_APP_KEY = YOUR_AYLIEN_APP_KEY;
const MAILGUN_API_KEY = YOUR_MAILGUN_API_KEY;
const SENDING_EMAIL_ACCOUNT = 'sending@example.com';
const RECIPIENTS = 'Recipient 1 ';
const NUM_STORRIES_PER_RSS_FEED = 3;
var rssFeeds = [
  "http://www.kdnuggets.com/feed",
  "http://news.ycombinator.com/rss"
];

//**************************************
//End of configuration section
//**************************************


var AYLIENTextAPI = require('aylien_textapi'),
  request = require('request'),
  xml2js = require('xml2js'),
  Mailgun = require('mailgun').Mailgun;

var textapi = new AYLIENTextAPI({
  application_id: AYLIEN_APP_ID,
  application_key: AYLIEN_APP_KEY
});

var mg = new Mailgun(MAILGUN_API_KEY);
var emailBody = '';
var numStories = 0;

rssFeeds.forEach(function(rssFeed) {
  setTimeout(function() {
    request(rssFeed, function(error, response, body) {
      if (error === null) {
        parser.parseString(body);
      } else {
        console.log('rss error : ', error);
      }
    });
    var parser = new xml2js.Parser();
    console.log("Processing rss feed - " + rssFeed + "...");
    parser.addListener('end', function(result) {
      var items = result.rss.channel[0].item;
      items.slice(0, NUM_STORRIES_PER_RSS_FEED).forEach(function(
        item) {
        var title, link;
        title = item.title[0];
        link = item.link[0];
        textapi.classify(link, function(error, result) {
          if (error === null && result.categories[0]) {
            var label = result.categories[0].label;
            var code = result.categories[0].code;
            textapi.summarize({
              url: link,
              sentences_number: 3
            }, function(err, resp) {
              if (err === null) {
                var summary = "";
                for (var i = 0; i & lt; resp.sentences.length; i++) {
                  summary += resp.sentences[i] + " ";
                }
                var story = 'nnTitle : ' + title +
                  'n' + 'Classification : ' + label +
                  'n' + 'Link : ' + link +
                  'nSummary : ' + summary;
                emailBody += story;
                numStories++;
                if (numStories ==
                  NUM_STORRIES_PER_RSS_FEED * rssFeeds.length
                ) {
                  console.log(
                    "Preparing Digest email...");
                  mg.sendText(SENDING_EMAIL_ACCOUNT, [
                      RECIPIENTS
                    ],
                    'Your Daily RSS Digest',
                    emailBody,
                    'noreply@example.com', {},
                    function(err) {
                      if (err) console.log(
                        'Error sending message to mailgun...: ' +
                        err);
                      else console.log('Success');
                    });
                }
              }
            });
          } else {
            console.log('classify error : ', error);
            console.log('classify error ', result.categories[
              0]);
            numStories++;
          }
        });
      });
    });
  }, 3000);
});

Author


Avatar

Mike Waldron

Head of Marketing & Sales @ AYLIEN A legal convert with a masters degree from Smurfit Business School, Mike runs our Sales and Marketing at AYLIEN. Mike gathered his Sales and Marketing experience with technology companies in Sydney and Dublin before getting the startup itch and joining the team at AYLIEN. Twitter: @MikeWallly