## Data visualisation talk: Presentation using reports package

Why I used html5 for my today’s talk?   My last presentation was in html5. This time I wanted to do my slides in something new.  I prepared  first few slides in Jessyink. Then I got to know that my friend (Trinker) developed one R package, reports,  for creating an amazing presentations including  beamer and reveal.js template.  This is kind of cool!!! Create presentation in R for R talk. I could write everything in R markdown and convert into beautiful html5 slides.

You could find my presentation just below this blog.  If any of you wanted to recreate the same, here is the R Markdown file Rmd.  I did bit modification in the style file and some tags. Otherwise you could generate the same. Ensure you have all required packages (check Rmd file) before running the file. The options required for slide creation very well described in the help (reports package).

Short version of slide creation is as follows:

1. Install reports package (install.packages(“reports”) or take it from github).
2. Call the presentation(“your_folder”) function. This create Rnw and Rmd files inside your_folder/PRESENTATION.
3. Add contents in Rmd file. Then click knit HTML if you are using RStudio or call knit2html. This would create the html file.
4. Finally run reveal.js() function. This would create the html5 presentation slides inside your_folder/PRESENTATION/reveal.js(). You could customize the path or theme or transition… I have used theme as beige and transition as cube.

This process wouldn’t be smoother for an R beginner who uses Windows. One should make sure all dependencies are set.  Github page provides more help resources and it is helpful to complete successful creation of your presentation.

Apart from reports one can try slidify  package for creating html5 slides. This got multiple  html5 presentation templates including shower, io2012, …

One thing I am bit sad  that Emacs+ess doesn’t have an Rmd mode. I guess somebody is working on that.  Now it is time for my research work.

 Note:Use firefox or chrome browser to view my presentation. left, right, up*, and down* arrows for transition.

PS:  As per audience response I should spend more time on ggplot2

Posted in data visualisation, data visualization, Graphics, R | Tagged , , , , , | Leave a comment

## Fun with R graphics: A raptor and a cake

I like to explore the graphics features of R.  Base graphics, ggplot2 are my favourites. On  other graphics packages, I haven’t used plotrix much. I am currently exploring grid package.

## Raptor

First one is using ggplot2. (you could also do this using base graphics).

It would be more fun if we can do animation on raptors. You can find animated version of raptor here. Rcode.

## Cake

Here is the easy one. This one is a cake using plotrix package.

I have already discussed this graphs in my favourite forum talkstats.  I dedicated both the graphs to my friends in the forum. You can find more fun graphs in this forum.

Posted in Graphics, R | Tagged , , , , | 8 Comments

## R Graphics with ggplot2

ggplot2 is one of the most elegant R package for data analysis and visualization.  Recently I gave a tutorial on ggplot2 package.  You could find my ggplot2 notes here(click the image below).

You could find my presentation slide below. The presentation made in html5. Use your left/right key to go through the presentation.

Posted in data visualization, Graphics, R | Tagged , , | 6 Comments

## A heuristic enhancement of optimisation algorithm

The majority of the world’s problem deal with directly or indirectly some kind of optimisation. Instance of optimisation of resources or utility function can be seen our daily life. Here I am talking about standard optimisation problem in statistics, maximum likelihood estimate. This blog is a small episode of my recent work. I used R for this optimisation exercise.

There are packages available for optimisation in R. The mostly used are: optim() and  optimize() are in stats package. A dedicated function mle() available in stats4 package. This is very useful most of the mean part modeling(eg: glm). What we need for this optimisation is to prepare function for the (log)likelihood and gradient (or hessian). We can also specify the algorithm(methods) as any of the following: “Nelder-Mead”, “BFGS”, “CG”, “L-BFGS-B”, “SANN”, and   “Brent”. This opimisation output give the necessary information for the model estimation.

The case I was doing is an extension of bivariate garch model. It includes more than 20 parameters. let me focus pure garch model, garch(1,1), as a prototype to explain the algorithm. Here the model equation is in the variance part. Also the parameter has constraint to ensure the positive variance. The   model equation is given below.

$x_t = z_t \sqrt{h_t}; z_t \sim \mathcal{N}(0,1)$
$h_t = \omega + \alpha x_{t-1}^2 + \beta h_{t-1}; \omega >0 , \alpha,\beta \geq 0$

The above discussed optimisation function may work for garch(1,1) estimation. But as the number of parameters increases there is not much use with optim functions. The main reason for failing the algorithm is the nonconvex surface with sensitive boundaries. So the solution to write the codes for newton raphson algorithm or BHHH or other modified algorithms.

I wrote BHHH algorithm for my model. I was not able solve it fully, it was not giving the optimum. The reason is the direction for each iteration when it reaches sensitive area is not properly scaled. ie. either it overshoot in some dimension or there is no significant improvement in any direction. So to get appropriate multiplier on optimal direction fails here.

Now I will discuss about the heuristic part of the modification in the algorithm. In this heuristic, certain number of dimensions are randomly selected and perturbed. Remaining dimensions are left unchanged. This way we can overcome the scaling issue. This given me optimal solution but it is not optimised in the execution perspective. It take slightly more time.( I compared few simple models with my heuristic and existing garch estimation.)

This approach looks very silly for regression type problems. But in garch like process tweaking like this is very much require. I am still looking for the better way. Anyway the blogging helped me to relook at my approach again. Please let me know if you have any suggestions.

Posted in OR, R, Time Series | Tagged , , , | Leave a comment

## Why Emacs is important to me? : ESS and org-mode

I cannot believe that I lived without emacs!  Now I use emacs more than any application. The usage  of emacs is going to increase as the days go. A non-emacs user  may think that why an editor should get a credit like “emacs is an integral part of my life”. Emacs users may find nothing unusual about my statement. Well! I will brief my emacs story. I will start with my programming life.

### My programming life

My programming life is summarised as below. Even though I learnt multiple languages I have included the languages where I spend more than 30 days(or more than 200 hours) roughly.

• 2000-2001: Basic and FORTRAN                ->Windows days
• 2002: C and C++                              ->Windows days
• 2003: VC++ and OpenGL                ->Windows days
• 2004: C and C++                              ->Here onwards Linux is my favourite desktop OS
• 2005-2006: JAVA, VB and VFoxpro ( I don’t think anybody uses this one)
• 2007-2009: SAS and Shell Script
• 2010-2011: R
• 2012 : R, Python and elisp

Out of this I consider 2002, 2003 and 2012 are my best years of programming. I should consider my first few years were my golden days of programming. Those days I didn’t had to do anything (No responsibility) other than coding and some mathematics. At that time I never understood the importance of skill of programming. I treated programming as one subject of mathematics. I did programming mainly for fun. I thank my best friend Jais for my consistent interest in coding. We had lot of intellectual discussion and debates. We spent lot of hours to create graphics applications, screen savers, hacking, etc. Now after a long break I am back to real programming lifestyle.

When I worked for a software company(2005-2006) I did more cut-copy-paste and trial & error methods. This way I never learnt anything deep. Money and movies where other distractions. Weekends were mainly window shopping and parties. The main reason why I didn’t do any real programming was I didn’t get a good friend who is crazy enough on geek stuff.

### What motivated to learn Emacs

I always like simple applications. When I say simple it means quick execution, transparent functionality and customisable. I really like vi/vim editor and it was my favourite editor till 2011(I don’t consider Windows era). I tried emacs in 2004 and I didn’t like it. It was nightmare at that time because to remember hell number of commands. Again I tried emacs in 2010-2011. This time the motivation was  emacs  topped as the favourite editor for most of the stat tech guys. But I didn’t learn anything other than C-x C-f and C-x C-c. Now what happened? I was searching a good R editor. I got to know about ESS and ESS is the first reason to learn emacs. So I started using emacs only for R coding. Then I started exploring shortcuts. I found it is very easy to customise. I have copied few shortcuts and started customising my own. Fortunately stackoverflow is very helpful on this. Some of the customisations are

• auto complete
• execution shortcuts
• folding mode

First two features are very common ins standard editor but, not the third one. I have seen folding mode in Rstudio. After experimenting on customisation I started learning emacs lisp. This made me to think one editor for all programming.

The main reason to learn emacs is Org-mode. I guess this should be one of the key addon to emacs. Like ggplot2 in R. The advantage of org-mode is Integrating emacs with:

• connect with other applications( HTML, latex beamer, google calendar)
• table creation
• Good for draft/notes preparation, etc.

I am mainly a statistician and coding is my next role. But still for all my data modelling and technical report I depend on emacs.  Now I use emacs for latex, python, R, daily planning and blogs(including this one).

One of the main reason I switched to linux is constant learning. Emacs will ensure the constant learning at best.

Emacs is good enough for preparing daily planning, latex, R, Python, and other programming. With just a console I can use emacs and do most of my work. I am exploring on other applications emacs like reading mail,news feeds,twitting, and other automation. Already the .el files are available on above but, looking for perfect one. I started exploring on elisp. I will post few of my elisp experiment in future.

Posted in emacs, Statistics | Tagged , , | 4 Comments

## The Curious Incident of the Dog in the NightTime

Hi! This is Richie again after a long time, precisely after 1 year, 1 month and 27 days. During the last one year I had very less free time compare to the previous years. Weekends, I was watching so many movies and occasionally, reading novels and stories. So never felt like writing a blog. This blog is mainly about a book I read few weeks ago. The book’s name is the same as the title of the blog, written by Mark Haddon that won the 2003 Whitbread Book of the Year.

Last Wednesday morning my bro called me and gave the idea of writing a blog about the book. At first I wondered, because I had given this book to him one month before and he said he had not found it interesting.

The hero of the novel is Christopher, a 15-year-old autistic boy living in Swindon (somewhere in UK). The story is written in the first-person narrative of the boy. So I give all credit to the boy. I am not going to explain the story because it will be boring if you read the novel without suspense. Why I liked this book is because of a mathematical touch in the storyline and remarkable funny stuffs. When I read this novel I actually thought about my boyhood days and I found a minor similarity in my thinking and that of Christopher’s.

I will list out some of the interesting stuff i liked in the book

Prime Numbers. He likes prime number( me too) and in this book the chapter are given in this order 2,3,5,7,11 and so on. He considers that prime numbers are like life “ even though they are logical, it’s impossible to work out the rules”.

Monty Hall Problem. This is a famous puzzle problem. And he has used this example to shows that intuition can sometimes get things wrong.

The problem is

Suppose you’re on a game show, and you’re given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what’s behind the doors, opens another door, say No. 3, which has a goat. He then says to you, “Do you want to pick door No. 2?” Is it to your advantage to switch your choice?

I heard this problem 1.5 years back. One of the professor in IISc asked me this question. I didn’t know it was a famous question till I read this book.

Sherlock homes. He likes sherlock homes stories(me too). This could be the reason for the title of the book( Because sherlock homes mentioned this sentence, “To the curious incident of the dog in the night-time.” Book: “The Memoirs of Sherlock Holmes” Story: Silver Blaze). This may be the reason he is pursing the mystery of the dead dog. He has mentioned that his favorite story is “The hound of Baskervilles”. And he has explained the clues and red herrings of this sherlock holmes book.

Map. He draws map whenever he goto new places. I also do the similar thing. When I was in Kolkata, I had a big map of the city in the wall. And the same habit I continued in Hyd, Chennai and Bangalore.

White lie. A white lie is not a lie at all. It is where you tell the truth but you do not tell all of the truth. He used this lie to his father many time in the book. Because his father don’t like his detective attempt on dead dog.

* * **

There are a couple of new stories I wanted to tell you. Once I finish all clint eastwood’s movies, I will start writing those. Till that time. Bye bye