Setting up the Twitter R package for text analytics

The Twitter R package by Jeff Gentry is a great way to get started in Text Analytics. This tutorial is about how to get setup to pull twitter data into R so that you can do some text analytics with it. Basically it is only the first step of getting data in a text analytics Workflow. I should also give a shout out here to Ted Kawartler for getting me interested in learning more about text analytics.

The first thing you need to do is downloadd the twitteR package and make it available in your R session.

library(twitteR)

Now on the Twitter side you need to do a few things to get setup if you have not done them already. I will go through this step by step so no one gets lost. It drives me crazy when people go too fast on the setup so I can't use the tool.



You need to have a twitter account. if you do not have one go to http://twitter.com/signup and set one up. Also you need to have a modile number as part of this account.

Now that you have created a twitter account you need to go to https://apps.twitter.com and sign on with your twitter account.

Once you have signed in you should see the following screen, and simply click on the button that says "Create New App".

Once you click on the "Create New App" button you will go to the Create an Application screen. There are three fields, a click box and a button you need to click on this page. The three fields are Name, Description and Website. The name of the application must be unique so this may take a few tries. The description needs to be at least 10 character long, and put in a website. If you do not have one you can use https://bigcomputing.blogspot.com. Now click the "Yes, I agree" box for the license agreement and click the "Create your Twitter application".

Once you successfully create an appplication you will be taken to the application page. Once there click on the "key and access token" tab. From that page you are going to need four things.

1. Consumer Key (API Key)

2. Consumer Secret (API Secret)

click the "Create my access token" button.

3. Access Token

4. Access Token Secret

Now re-open your R session and enter the following code using those four pieces of information.

consumer_key <- "your_consumer_key"

consumer_secret <- "your_consumer_secret"

access_token <- "your_access_token"

access_secret <- "your_access_secret"

setup_twitter_oauth(consumer_key, consumer_secret, access_token, access_secret)

Now you are set up on the Twitter side and the R side you should be ready to go. I will cover doing some basic data pulls from Twitter in my next post

n

Twitter Sentiment

I know I continue to beat the Twitter sentiment thing to death, but there is just something about it that just does not sit right with me. I think there has been good research that finds Twitter feeds can be text mined for words like "colds" or "flu" and predict local flu outbreaks. I am not sold that Twitter is a good source for sentiment gathering on investment strategy. I do not think it works because mircoblogs like twitter are a random stream of consciousness provided by a population only loosely connected to the markets that investors are trying to predict. I believe a more relevant population with proof of action on their sentiment would be a much better indicator of trends. That is why I am a much bigger fan of using sources like Reddit. I wonder how the Twitter Sentiment Fund is doing. I can not find any numbers, but that is not really a surprise given that it is a private fund.

I will close this short post with a video called Twitter for Math Nerds . I wonder if the Twitter Sentiment Fund is heavily invested in Justin Beiber.

Using Social Media to predict Marcoeconomic tends

While I am very leery of using Twitter to predict economic trends particularly in time to anticipate trading trends in the financial markets, I do believe they may be useful in terms of predicting marcoeconomic trends.

Google introduced this idea in 2009 with a paper by Hal Varian. This work was based on the mining of Google searches, and it proved to be pretty effective. I believe similar or maybe more accurate results can be obtained by mining social media. This is because many tweets and posts are richer than simple Google searches. If you are looking for a good R base twitter client try twitteR. The same author of that package wrote an R based Stack Exchange client called RStackExchange. RStackExchange may provide some foreshadowing of technology and software trends