News API Feature Update: Real-time monitoring

News API Feature Update: Real-time monitoring

With our News API, our goal is to make the world’s news content easier to collect, monitor and query, just like a database. We leverage Machine Learning and Natural Language Processing to process, normalize and analyze this content to make it easier for our users to gain access to rich and high quality metadata, and use powerful filtering capabilities that will ultimately help you to find precise and targeted stories with ease.

To this end, we have just launched a cool new feature, Real-time monitoring. Real-time monitoring allows you to further automate your collection and analysis of the world’s news content by creating tailored searches that source and automatically retrieve highly-relevant news stories, as soon as they are published.

real-time monitoring

You can read more about our latest feature – which is now also available in our News API SDKs – below.

Real-time monitoring

With Real-time monitoring enabled you can automatically pull stories as they are published, based on your specific search query. Users who rely on having access to the latest stories as soon as they are published, such as news aggregators and news app developers for example, should find this new feature particularly interesting.

The addition of this powerful new feature will help ensure that your app, webpage or news feed is bang up to date with the latest and most relevant news content, without the need for manual searching and updating.

Newly published stories can be pulled every minute (configurable), and duplicate stories in subsequent searches will be ignored. This ensures you are only getting the most recent publications, rather than a repeat of what has come before.

Usage

We have created code in seven different programming languages to help get you started with Real-time monitoring, each of which can be found below, as well as in our documentation.

import time
import datetime
import aylien_news_api
from aylien_news_api.rest import ApiException

def fetch_new_stories(params={}):
  print('------------')

  fetched_stories = []
  stories = None

  while stories is None or len(stories) > 0:
    try:
      response = api_instance.list_stories(**params)
    except ApiException as e:
      if ( e.status == 429 ):
        print('Usage limit are exceeded. Wating for 60 seconds...')
        time.sleep(60)
        continue

    stories = response.stories
    params['cursor'] = response.next_page_cursor

    fetched_stories += stories
    print("Fetched %d stories. Total story count so far: %d" %
      (len(stories), len(fetched_stories)))

  return fetched_stories


# Configure API key authorization: app_id
aylien_news_api.configuration.api_key['X-AYLIEN-NewsAPI-Application-ID'] = 'YOUR_APP_ID'
# Configure API key authorization: app_key
aylien_news_api.configuration.api_key['X-AYLIEN-NewsAPI-Application-Key'] = 'YOUR_APP_KEY'

# create an instance of the API class
api_instance = aylien_news_api.DefaultApi()

params = {
  'title': 'Apple OR Amazon OR Facebook OR PayPal OR Microsoft OR Ebay OR Cisco OR Netflix OR IBM',
  'language': ['en'],
  'published_at_start': 'NOW-1HOUR',
  'published_at_end': 'NOW',
  'cursor': '*',
  'per_page': 5,
  'sort_by': 'published_at',
  'sort_direction': 'desc'
}

while True:
  stories = fetch_new_stories(params)

  print('************')
  print("Fetched %d stories which were published between %s and %s" %
    (len(stories), params['published_at_start'], params['published_at_end']))

  if len(stories) > 0:
    last_fetched_at = stories[0].published_at + datetime.timedelta(seconds=1)
    params['published_at_start'] = last_fetched_at.isoformat()[:-6] + 'Z'
    params['cursor'] = '*'

  print('Sleep for 60 seconds until next poll...')
  print('------------')
  time.sleep(60)

NB: Real-time monitoring will only work when you set the sort_by parameter to published_at and sort_direction to desc.

Benefit

The main benefit of this cool new feature is that you can be confident you are receiving the very latest stories and insights, without delay, by creating an automated process that will continue to retrieve relevant content as soon as it is published online. By automating the retrieval of content in real-time, you can cut down on manual input and generate feeds, charts and graphs that will automatically update in real-time.

We hope that you find this new update useful, and we would love to hear any feedback you may have.

To start using our News API for free and query the world’s news content easily, click the image below.

 

 


News API - Sign up



 

 

Let's Talk