Tools for programming in applied statistics

A general-purpose programming language (chosen to be Python) and a statistics and graphics package (chosen to be ROOT) are essential to any serious course on applied statistics, as we will be using it for all data analysis. Much of the analyses involved is hard/impossible to do using less powerfull programming languages (e.g. matlab).

Python Python is a popular high-level general-purpose programming language. It emphasizes code readability and easy development.

ROOT ROOT is a software library (developed at CERN), which enables fast and easy reading of data, advanced stastistical analysis, model fitting, and graphical output of results. ROOT is written in C++, but it contains a python module which makes it usable in from python.

Note: Don't worry if you are not able to get it working yourself. We will figure it out during the Python/ROOT setup intro day 10:15 the 27th of August. Or, if you can't make it that day (rustur?), write us a mail and swing by.

1 Installation instructions

1.1 Installing Python 2.7

We will use Python 2.7.X in this course.

Linux

Python is available in all major linux distributions. If it is not installed already install it using your favorite package manager (e.g. Software Center in Ubuntu). The package is typically called python or python2.7.

OS X

Python is typically part of newer OS X installations. In that case you can skip this step. See 1.1.1 for checking if you have a working python installation. If you need to install python you can find .dmg files here.

Windows

If you don't have python already, download the installation file for your version of windows from this page and install the contents. However, note that it should be the 32 bit version for it to be compatible with ROOT.
After installing Python you need add a few things to your enviromental variables, in order make your computer recognize the python extentions (.py). To do this on windows 7 press start->computer->system properties->Advanced System settings->Enviromental Variables. In the bottom list, under "System variables" find the variable called "Path". Press edit, and add ";C:\Python27" (If have installed Python at another location, enter that instead). Press ok. Next, find the variable called PATHEXT. Here add ";.PY". Now your computer should recognize python.

1.1.1 Testing your python installation

(Logging out and in again may be required on OS X after installing python)
(In windows the terminal is opened by clicking on the Start button and opening the "Command prompt", which can be found under "All Programs > Accessories" or by searching for "Powershell" in the start menu)
Open a terminal and type:
python --version
Python is installed if the output is similar to:
Python 2.7.8

1.2 Installing ROOT

We recommend ROOT 5.34 for this course.

Windows

If you don't have ROOT already, download the installation file for for windows from this page. Click on the link to the page for the 5.34 version. Then scroll down to the Windows section in the Binaries section, and choose the "Windows Visual Studio 12" executable, which you should install. When installing, choose the option "Add ROOT to the system PATH for current user", and the ROOT installer automatically adds the relevant paths and extentions to the environmental variables. However, you also need to add "PYTHONPATH" to your environmental variables, and it should have the same value as the ROOT part of your "PATH". This might have to be done for both user and system variables.
If this doesn't work, try restarting your machine.

Linux

ROOT is available in all major linux distributions. If it is not installed already install it using your favorite package manager (e.g. Software Center in Ubuntu). The package is typically called root or libroot. ROOT may be split into many packages whose names start with libroot or root. We want to install the python bindings for ROOT. In distributions that are Debian derivatives (like Ubuntu) this package is typically called libroot-bindings-python5.34. If your distribution does not split up ROOT into several packages the python bindings are included in the normal ROOT package.

OS X

If you don't have ROOT already, download the installation file from this page. If you have OS X 10.9, click on the link to the page for the 5.34 version, otherwise try 5.32. Then scroll down to the OS X section in the Binaries section. Download the file. Move the downloaded .tar.gz file to the folder where you want to install ROOT (We recommend to use your user directory). Now open a terminal and follow these instructions:
  1. Navigate to the folder where you put the .tar.gz file you downloaded. You start in your user directory. Commands are executed by typing them into the terminal and pressing enter. You can always check in which folder you are with the command pwd. Entering a subfolder with name NAME in the current directory is done with the command cd NAME. Entering the parent folder to the folder you are in is done with the command cd ..  . If you put the downloaded file directly in your user directory, you can skip this step.
  2. Extract the content of the downloaded file:
    tar -zxvf root_v5.34.19.macosx64-10.9-i386.tar.gz
    (The file may have a slightly different name, adjust is accordingly)
  3. Add a line to your .bash_profile file in your user directory by executing:
    printf "\nsource "`pwd`"/root/bin/thisroot.sh\n" >> ~/.bash_profile
If this for some reason doesn't work, then try to install ROOT by compiling it yourself, as described here.

1.1.1 Testing your ROOT installation

(Logging out and in again may be required on OS X after installing ROOT)
This assumes that you installed python successfully and checked wether it works or not as described in 1.1.1. Open a terminal and start python:
python
You will get output similar to this:
Python 2.7.8 (default, Jul 4 2014, 13:08:34)
[GCC 4.9.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

Now load the ROOT module by executing this python command:
import ROOT
If there is no output (especially no error messages), you have a working ROOT installation.
However, if this does not work (especially in El Capitain on mac), then try from your shell the following command:
"for foo in $ROOTSYS/lib/*.so; do echo $foo && install_name_tool -add_rpath $ROOTSYS/lib $foo; done"
Nevermind what it does... just fold your hands and pray :-)

2 Executing a test script

Now that you have a working Python/ROOT setup, you should be able to download and execute this python script, which uses ROOT (If you have trouble downloading the file, try clicking the link with the right mouse button and select Save As... from the appearing context menu). Download the script, open a terminal and navigate to the folder where you saved the script (See 1.2 First step of the OS X instructions on how to navigate in the terminal. In windows use echo %cd% instead of pwd). Execute the script with this command:
python CalcPrimeNumbers.py
A version of the script with more detailed documentation is available here.

3 Writing your own script

To write your own scripts, or edit the downloaded ones you can simply open them in a texteditor. New scripts can be written in a new document with the file extention .py at the end of their name.
More advanced editors for Windows (and OS X) are available at notepad++(Windows only) and sublime text. They include syntax highlighting and many other features.

Most default editors (kwrite, gedit) in Linux distributions already have these features and any of them is a good choice, if you are looking for an advanced and easy to use editor, try kate.