Everything old in Data Science is new again - Graphs

20 years ago when I was still in school we used graphs heavily before we tried to fit a model. The data sets were pretty small, but we used the graphs to help select the model we were going to fit. The process of fitting a model at the time was time consuming and tedious so it was important to minimize the number of inappropriate models that you might try. I even remember having templates of graphs of data which basically said if your data looks like this do this.

As time went on, I continued to use graphs, but I confess I spent less time graphing the data in various ways then I did back then and more time experimenting with a suite of models. The result was that I lost some of the connection to the data that is the art and feel side of data science.

Lately I have been playing with RStudio's shiny app ( along with everyone else). The result has been that it brought back to me the importance of exploring your data graphically and introduced me to a lot of types of graphs that I had never seen or used. Many of these plots provide powerful insight into the structure and the nature of the data that I was working with that I do not believe I could have gotten any other way.

So I thought I might spend the next couple of weeks doing relative short blog posts on graph different types of data set with different types of graphs. I hope this simple review of one of the core elements of data science will be as helpful to you as it has been to me.

For more detail you can always depend on Hadley. His stuff is always great

Predicting the Spread of Ebola

I found this piece on the predicting the spread of Ebola, and I think think it is such a wonderful piece that I am sharing on every outlet I have.
http://grantbrown.github.io/Ebola-2014-Analysis-Archive/Aug_28_2014/Ebola2014/Ebola2014.html

Introduction to using Random Forests for the Kaggle Titanic Data Set

During the summer a number of the members of the Connecticut R User Group decided to work on a Kaggle competition data set to improve our R programming skills. The first data set we tried was the Titanic data set. This is a fairly simple data set from which we are trying to predict who will survive and who will not. The task for the team was to simply load the data set and the Random Forest package and run the basic model. We completed that task last week. The R code to do that is below:

#load in Package randomForest
install.packages("randomForest")
library(randomForest)
## Read in Titanic Data training and test set
train.data<-read.csv("~/titanic/train(2).csv")
test.data<-read.csv("~/titanic/test.csv")
## Convert data into a simpler dataframe
train<<- data.frame(survived=train.data$Survived,
                    age=train.data$Age,
                    fare=train.data$Fare,
                    pclass=train.data$Pclass,
                    sex=as.integer(factor(train.data$Sex))
                    )
test<<- data.frame( age=test.data$Age,
                    fare=test.data$Fare,
                    pclass=test.data$Pclass,
                    sex=as.integer(factor(test.data$Sex))
                    )
## now we need to get rid of the NAs and make
train$fare[ is.na( train$fare) ]   <- 0
train$age[ is.na( train$age) ]     <- 30
test$fare[ is.na( test$fare) ]   <- 0
test$age[ is.na( test$age) ]     <- 30
labels <- as.factor(train[,1])
train <- train[,-1]
# fit a random forest and make a prediction
rf <- randomForest(train, labels, xtest=test, ntree=100,do.trace=TRUE)
results<<-predictions <- levels(labels)[rf$test$predicted]


As we do more work on the model and try other approached I will update on my blog.

Taco Bell's Waffle Taco a novelty that needs to go the way of the pet rock.

This morning I decided to try the Waffle Taco at Taco Bell. What a mistake! It was terrible. A tasteless frozen waffle with rubbery pseudo food inside. Do not ever even bother trying this because you will just regret it like I do. No wonder they use old people in their commericals for this item.Their taste buds are already shot.

Reading a large number of files into R

I know this is a fairly basic topic, but it is one that caused me problems lately. Normally I only have to read in one data file at a time or I read in a few tables separately.

If I am reading in a single file would do the following

>read.table("file")

 or if it is online

>read.table("url")

If it is a csv file

read.csv("file")

Now the problem arose because I needed to read in 400 files from a directory, but the files were not numerically indexed. So to solve this problem I used the functions list.files and paste.

>names<-list.files("~/directory/")
>complete_names<-paste( "~directory", names, sep="")
>for (i in sep_along(names){
         monitors<-rbind(monitors, read.csv(complete_names[i]))

It was slow, but got the job done.

Everything old is new again in Video Games

They say in fashion every comes back in style about every 20 years. I am beginning to think the same applies to video games. The online games of facebook and mobile games have the same look and feel of early PC games. The only thing I am glad is that command line games like Zork or Galtrader did not make a comeback because I was glad to see them go. Now I am hearing more and more about table video games returning to bars. It is like the 80s have come back in fashion.

In the 80s video games had there heyday in bars across America. Games like Pong, Pac-Man and Donkey Kong gain fame and fortune not on the home gaming screens, but on table video game units in bars. I remember walking through these smoke filled ( yes they still smoked in bars then) by table after table of video games being played with beers and quarters stacked on top of them. By the 1990s they had all disappeared to make way for things like Karaoke.


Now it seems that video games and beer will once again be joined together in a happy partnership of something that requires fine motor skill and something that impairs motor skills. Recently I saw a video of a new video games for Grasshopper NYC. The Game and its developer make no bones about its purpose. The games is called Don't F**K Up: A video game for drunk people. Here is a link to the Video.

ROIA - A great new Restaurant in New Haven

I met Avi, the owner of ROIA, about a year ago as he was working on getting his restaurant open.  He is a very passionate man and great to talk to. I was invited to the opening. The place and the food were outstanding! I remember thinking someone needs to write about this place to get the word out. I have eaten there a number of times since, and every time I keep thinking someone needs to write an article about this place because it is awesome. Well, the New Haven Register finally did. Go to ROIA you will love with the place and the food.

Register review of ROIA