Twingly Blog LiveFeed API

Introduction

Blog LiveFeed API is a web service based SOAP API used for receiving a continuous real-time stream of blog data. To be able to retrieve data through LiveFeed, an API key issued by Twingly must be used. The API key then grants access to blog data for one or more languages.

It is possible to query the LiveFeed API for data up to 30 days back in time. The responsibility of retrieving the data in a correct manner so that nothing is missed lies entirely on the client. To facilitate this, the LiveFeed API exposes four different methods, where the most commonly used method is GetDataByPostCountAndTimespan.

HTTP options

Methods

Service Description (WSDL): https://api.twingly.com/livefeed4.asmx?WSDL

Note that all input timestamps relates to when Twingly first made the data available through the LiveFeed API.

GetData

This method will return all posts that have entered Twingly’s systems and that the API key grants access to. However, there are limitations built in so that the time window created by timestamp and the time when the call was made cannot be greater than 2 hours. This means that only data newer than 2 hours can be fetched via this method.

Parameters

Parameter Type Notes
apiKey string The API key used for authentication
timestamp dateTime The timestamp from when to fetch data

GetDataByPostCount

This method has the same semantics as GetData. The only difference is that it will return at most maxPosts items. So, if an invocation of GetDataByPostCount will return 1000 items, and maxPosts is set to 100, then only the 100 oldest items will be included in the result set.

Parameters

Parameter Type Notes
apiKey string The API key used for authentication
timestamp dateTime The timestamp from when to fetch data
maxPosts int The maximum number of posts to retrieve

GetDataByTimespan

The GetDataByTimespan is used to retrieve data between two timestamps. The time window created by the two timestamps is restricted to at most 2 hours. If a wider time window is given the window will be shrunk by reducing the to timestamp.

Parameters

Parameter Type Notes
apiKey string The API key used for authentication
from dateTime The timestamp from when to fetch data
to dateTime The timestamp to when to fetch data

GetDataByPostCountAndTimespan

This method is the most commonly used method for fetching a continuous stream of data. This is because it lets the client control both the window of time and the max number of posts to handle at each time.

Parameters

Parameter Type Notes
apiKey string The API key used for authentication
from dateTime The timestamp from when to fetch data
to dateTime The timestamp to when to fetch data
maxPosts int The maximum number of posts to retrieve

Status

Used for ensuring that the web service is up.

Response

The response is an XML document containing one or more posts. Below is an explanation of the various elements and attributes that can be present in the result.

Note that in older versions of the LiveFeed API, the result can look different. Only the current version is documented here. The structure of the XML document is as follows:

<twinglydata
  ts=""
  tsMs=""
  from=""
  fromMs=""
  to=""
  toMs=""
  type=""
  noOfPosts=""
  maxNumberOfPosts=""
  firstPost=""
  firstPostMs=""
  lastPost=""
  lastPostMs="">
  <post>
    <id></id>
    <url></url>
    <title></title>
    <summary></summary>
    <languageCode></languageCode>
    <date></date>
    <blogName></blogName>
    <blogUrl></blogUrl>
    <blogRank></blogRank>
    <approved></approved>
    <authority></authority>
  </post>
</twinglydata>

Elements and attributes:

Within the root element, <twinglydata>, there can be zero or more <post> elements.

Example response

A response from the GetDataByPostCountAndTimespan method with maxPosts set to 1, note that the summary field content has been shortened in this example:

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
   <soap:Body>
      <GetDataByPostCountAndTimespanResponse xmlns="http://twingly.com/">
         <GetDataByPostCountAndTimespanResult>
            <twinglydata ts="2016-01-18T10:46:51.5405916Z" tsMs="540" from="2016-01-18T10:00:00Z" fromMs="0" to="2016-01-18T10:10:00Z" toMs="0" type="stream" noOfPosts="1" maxNumberOfPosts="1" firstPost="2016-01-18T10:00:00.167Z" firstPostMs="167" lastPost="2016-01-18T10:00:00.167Z" lastPostMs="167" xmlns="">
               <post>
                  <id>8950519844709962941</id>
                  <url>http://www.nouw.com/akermanevelinn/gar-in-i-en-ny-vecka-22529882</url>
                  <title>Går in i en ny vecka</title>
                  <summary>Då var...</summary>
                  <languageCode>sv</languageCode>
                  <date>2016-01-18T09:59:12</date>
                  <blogName>akermanevelinn blogg</blogName>
                  <blogUrl>http://www.nouw.com/akermanevelinn</blogUrl>
                  <blogRank>1</blogRank>
                  <approved>false</approved>
                  <authority>0</authority>
               </post>
            </twinglydata>
         </GetDataByPostCountAndTimespanResult>
      </GetDataByPostCountAndTimespanResponse>
   </soap:Body>
</soap:Envelope>

Best practices

Full C# integration example

A very simple full example (without parsing of the result and error handling) follows below. Note that the LiveFeed class instantiated on the 3rd line is the proxy object for the web service.

public void LivefeedExample()
{
    LiveFeed client = new LiveFeed();

    DateTime from = DateTime.Now.ToUniversalTime().AddMinutes(-60);
    DateTime to = from.AddMinutes(10);

    int maxNoOfPosts = 1000;

    while (true)
    {
        // fetch the data
        XmlNode data = client.GetDataByPostCountAndTimespan(
            "the api key",
            from,
            to,
            maxNoOfPosts);

        // parse the result and do something with it
        // (not implemented here).

        // set the to and from paramaters to new values
        // based on the timestamp for the last (newest)
        // post in the result.
        DateTime lastPostTs = DateTime.Parse(data.Attributes["lastPost"].InnerText).ToUniversalTime();
        DateTime lastPostMs = Double.Parse(data.Attributes["lastPostMs"].InnerText);

        from = lastPostTs.AddMilliseconds(lastPostMs + 1);
        to = from.AddMinutes(10);

        Thread.Sleep(10 * 60 * 1000); // sleep ten minutes
    }
}

With this approach you will get a continuous stream of all the data that the API key grants access to. Of course, error handling must be added to handle network outages, server failure et cetera.


Available data

As soon as we get new blog content it will be available through the LiveFeed API. Most often it takes just a few seconds for data to flow through our system, it may be a few minutes during maintenance though.

Some notes:


Deprecations


Known issues


Clients


Documentation changelog

API changelog