Banner

Using the Twitter API

Back to Blog Listing


Much to the dismay of developers and website owners everywhere Twitter has now shut off its RSS feed and deprecated the API 1.0. This proved problematic for a few of our websites and we needed a solution to this. However we were not prepared to swap it out for another third party provider, which would add another layer of complexity to our solutions. We also wanted a server side solution so that it wouldn't rely on the client browser to be rendered on the page.

To achieve this we had to learn what happened and how to rectify it to get our customer's tweets back, taking in to consideration that this would need to be a scalable solution. Learning what had happened was the easy part, as many websites seemed to have fallen foul of the same issue.

 

The Problem

This basic method of authentication is now discontinued and can no longer be used. With Basic Authentication, your username and password had to be stored in your application and sent over the internet each time you accessed twitter, unless of course you just wanted to access the RSS feed.   Say, for example, your customer had to change the password because of security concerns. The application would stop working.

The security problems seem like common sense, right? Unless all you want is an RSS list of all of the latest tweets.  No authentication was required for this feed and, because it is publicly accessible information, you would have thought that it would have been moved over to API 1.1.  This was not the case and, from what I can see, everyone now has to use the Oauth model to gain access to any of the twitter functionality.

Twitter has suggested in various tweets and articles on their site that they have given ample notice for developers to re-code their websites (https://dev.twitter.com/blog/api-v1-is-retired).  This is ok but if you are a web company that builds twitter into all of their sites then this could be a large job and a hard client sell. 

The RSS feed address for C2Software is below and you can see when you submit this that you get the following error returned: "The Twitter REST API v1 is no longer active. Please migrate to API v1.1. https://dev.twitter.com/docs/api/1.1/overview."

https://api.twitter.com/1/statuses/user_timeline.rss?screen_name=c2software

 

The Solution

The Principles of the API 1.1

The OAuth model in my opinion makes sense if you have to authenticate and are working on several sites across many industries and sectors.  There is no real need for you to hold the username and password, other than if required to create the application on dev.twitter.com.

It means that you can avoid the responsibility and costs of storing a client’s username and password.  In some cases, we have found that organisations regard this information as highly sensitive and we have left them to create the application and send us the keys that we require.  There are also no consequences on your web application to the user changing this password on a daily basis, if they so choose.  It also allows the client to gain ultimate control of their twitter account.  Rights to access tweets can be revoked by them at any time.

From dev.twitter.com you can create multiple applications with varying permissions.  For example, say you have many websites that require read-only access for your twitter feed and only one that requires write access.   You can create a twitter application for each website, giving you the correct access writes for each site. This gives you the flexibility to change permissions as the business requirements change and the control over how each individual website gets access to your data.  Please note that if you change permissions you will have to generate a new set of keys and this will involve a code or configuration file update.  This may not be such a big deal if you design your twitter integration to be scalable in the first instance.

How Create a Twitter application

The Twitter application is really easy to create and can be done fairly easily by someone who is relatively confident with surfing the internet. Simply go to https://dev.twitter.com/apps/new.  From here you can create a new application. 

Once you have created your application you can click on "create my access token". The four keys that you need are the consumer key, consumer secret key, access token and the access token secret. It is these you will use to contact the twitter API.

Scalability

We did not just need a solution that fixed one site, we needed to roll this solution out across our sites.  This cut our development time, clients costs and future integration time significantly.  Our solution was to create an ASCX (ASP.NET User Control File).  This file would contain two classes.  The first one would be the normal user control class.  The second class would deal with the authenticating between twitter and our client’s website (whoever that might be), it would also format the text to include the links, @ signs and # tags.  Finally it calculates how long it has been since each Twitter feed had been posted.  This is then passed back to the first class as a dataset.

The OATH CLASS

I set the oath class so that it has public properties which are set to receive the Consumer secret, Consumer key, Access token, Access token secret, Number of tweets to return, whether to Include re-tweets, and the Screen name

This enables us to transport the ASCX file from site to site and get it to authenticate and pass back a dataset structure that has the basic elements of a tweet in it, such as the title, username and published date.

The ASCX Class

The reason that we kept this file separate was that no two websites have the data formatted in the same way, i.e. locations, surrounding html tags or IDs/class names.

This allows development and design teams to focus on their professional areas.

Preparing for Authentication

First of all you need to create your public properties inside the Oauth class as mentioned above.  To do this, you need to supply the mandatory parameters and any optional parameters.  The example below simply prepares the request to return the basic status, but you could expand this to use any of the other twitter API functionality.  

oauth_ consumer_key

This is the consumer key that was generated by your twitter application.

oauth_nonce

This is used to determine whether each request is unique.  The last thing you want is multiple instances of the same status updates because of an impatient user.  You can use any random string of alphanumeric characters.  It should be noted, however, that this has to be unique per request.   The example below uses the ticks method to produce a unique number that is one ten-millionth of a second.  It then converts this number to a base64 string.  

 

Dim oauth_nonce As String = Convert.ToBase64String(New ASCIIEncoding().GetBytes(DateTime.Now.Ticks.ToString()))


oauth_signature_method

The signature method is the type of encryption that twitter requires and this is HMAC-SHA1

Dim oauth_signature_method  As String = "HMAC-SHA1"

 

oauth_timestamp
This is as it suggests, a timestamp and should be the number of seconds since the UNIX epoch (01/01/1970). You would convert this using the following methods.

Dim TimeSpan As TimeSpan = DateTime.UtcNow - New DateTime(1970, 1, 1, 0, 0, 0, 0, DateTimeKind.Utc)
 
Dim oauth_timestamp As String = Convert.ToInt64(TimeSpan.TotalSeconds).ToString()

 

HINT: This is easily generated through code, but a word of warning, if your server clock is not sitting at the right time then your twitter feeds will fail. 

oauth_Token
oauth_token is the token as supplied by your twitter application.

oauth_version
The version is 1.0 and is in relation to the OAuth authentication and not the twitter API.

Dim oauth_version As String = "1.0"

oauth_ signature
To create a signature you need to combine all of the other request parameters, including the optional ones. These will then be encrypted using your secret keys as the salt.

'creating your signature
 Dim baseFormat As String = "oauth_consumer_key={0}&oauth_nonce={1}&oauth_signature_method={2}&oauth_timestamp={3}&oauth_token={4}&oauth_version={5}&screen_name={6}"
 
'Adding your parameters too it
Dim baseString As String = String.Format(baseFormat, oauth_consumer_key, oauth_nonce, oauth_signature_method, oauth_timestamp, oauth_token, oauth_version, Uri.EscapeDataString(screen_name))
 
'the signature including resource URL and string un-encrypted
baseString = String.Concat("GET&", Uri.EscapeDataString(resource_url), "&", Uri.EscapeDataString(baseString))
 
'the composite key is made from the concatenation of the consumer secret and the token secret from your twitter application
Dim compositeKey As String
compositeKey = oauth_consumer_secret & "&" & oauth_token_secret
 
 
'add the composite key as a salt to your hash request
Dim hasher As New HMACSHA1(ASCIIEncoding.ASCII.GetBytes(compositeKey))
 
Dim oauth_signature As String
 
'run your basestring through the signing algorithm
oauth_signature =
Convert.ToBase64String(hasher.ComputeHash(ASCIIEncoding.ASCII.GetBytes(baseString)))

 

Authentication Header

The authentication header is created as a list of key value pairs with a comma separated delimiter. Note the use of quotation marks (“”) around the value of the key value pairs. 

Dim headerTemplate As String = "OAuth oauth_nonce=""{0}"", oauth_signature_method=""{1}"", oauth_timestamp=""{2}"", oauth_consumer_key=""{3}"", oauth_token=""{4}"", oauth_signature=""{5}"", oauth_version=""{6}"""
 
Dim authHeader As String = String.Format(headerTemplate, oauth_nonce, oauth_signature_method, oauth_timestamp, oauth_consumer_key, oauth_token, Uri.EscapeDataString(oauth_signature), oauth_version)

Although only done on the signature it is probably best practice to wrap all of your parameters in an Uri.EscapeDataString, otherwise your request could fail. Remember and do the same with your signature as you do not want inconsistencies to appear between your signature and your header.

GET request

Creating your get request is relatively easy in VB using the HttpWebRequest method. You need to first instantiate the request and assign the header and method. 

Dim url As String = "https://api.twitter.com/1.1/statuses/home_timeline.json" & screen_name
 
 
Dim request As HttpWebRequest = CType(WebRequest.Create(url), HttpWebRequest)
request.Headers.Add("Authorization", authHeader)
request.Method = "GET"
request.ContentType = "application/x-www-form-urlencoded"
Dim WebResults As WebResponse = request.GetResponse()

Hint: some error trapping to ensure that you have not received an error from the twitter would be a good idea.

Turning text into Workable object

The following example I have is based on a .net 3.5 site and therefore there are newer methods of retrieving your JSON Object. This is how I did it using our web application. I read the response data into a variable responseData and then de-serialized it into and object, which I can then loop over.

Dim responseData As String = New StreamReader(WebResults.GetResponseStream()).ReadToEnd()
     
 
Dim jss As New JavaScriptSerializer
Dim jsonString As Object = jss.Deserialize(Of Object)(responseData)
 
For Each i As Object In jsonString
             
{do what you want with the object in here i.e i("entities")("user_mentions")}
 
Next

ENJOY!

 


13 Nov 2013

About the Author

James CTwo is our blogging alter ego for the Dynamics CRM and Web experts here at C2. We have a passion for anything CRM, Web Design or Social Media. We love writing about it! Find us on YouTube & Twitter.

comments powered by Disqus

Our White Papers

Discover the content that matters to you on business transformation. Explore how you can drive customer engagement, empower your team, optimise your businesses and revolutionise your products with the next digital revolution. For more information visit our white paper library.

Library

C2 Newsletter

Our newsletter is sent on a quarterly basis, offering detailed insights into all aspects of business transformation with Microsoft cloud technologies. We also promote upcoming events and special offers throughout the year. Sign up to get involved today.

Dynamics 365 Office 365 Power BI Silver Partner