Statistics using Python

Porting Code from Matlab to Python - 2017

10 October 2017 | Rajalekshmi Deepu

Member of the Helmholtz Association
 Statistics in Matlab and python
Ø Matlab:
- Proprietary software.
- Need “Statistics” toolbox. (extra cost )
Ø Python:
- Opensource
- Extended with a fantastic ecosystem of
data-centric packages like: numpy,scipy,
Member of the Helmholtz Association

matplotlib, scikit-learn,pandas, …

10. October 2017 Porting Code from Matlab to Python 3

 Numpy and Statistics(Descriptive)
Ø Contains in-built statistical functions like Mean,
Median, Standard Deviation and Variance.
Matlab Python (using Numpy)
>> load [Link]
>>> import numpy as np
% Mean
>>> X = [16.92, 96.10, 11.82, 44.32,
>> Mean_value = mean(wages)
55.66, 10.75]
% Median
>>> mean = [Link](X)
>> med_value = median(wages)
>>> median = [Link](X)
% Standard deviation
>>> sd = [Link](X)
>> std_value = std(wages)
>>> variance = [Link](X)
Member of the Helmholtz Association

% Variance
>> var_value = var(wages)

10. October 2017 Porting Code from Matlab to Python 4

 ScientificPython (SciPy)
Ø Scientific Computing Package for Python.
>>> help(scipy)
Ø Built on top of Numpy and uses Numpy arrays and data types.
Ø Scipy package is organized into several sub-packages.
Ø Imports all functions in the Numpy package, and several commonly
used functions from sub-packages, into the top level namespace.
e.g: [Link] and [Link]
both refers to function var in module [Link]
[Link] and [Link]
both refers to built-in function array in module [Link]
Member of the Helmholtz Association

10. October 2017 Porting Code from Matlab to Python 5

 SciPy and Statistics (Inferential)

Ø SciPy offers an extended collection of statistical

tools such as distributions (continuous and discrete)
and functions.
Ø Few sub packages for statistics are:
[Link] --- Vector Quantization / Kmeans
[Link] --- Statistical Functions
[Link].t --- Student’s T test

Remember: Subpackages requires an explicit import

Member of the Helmholtz Association

e.g: >>> import [Link]

>>> from scipy import stats

10. October 2017 Porting Code from Matlab to Python 6

 [Link]
Member of the Helmholtz Association

10. October 2017 Porting Code from Matlab to Python 7

Member of the Helmholtz Association

10. October 2017 Porting Code from Matlab to Python 8

 Scipy and Matlab

Ø [Link] - Utilities for dealing with MATLAB

files.
Ø Included functions:
• [Link] - Load MATLAB file. Returns dictionary
with variable names as keys, and loaded matrices as values.
• [Link] - Save a dictionary of names and arrays
into a MATLAB-style .mat file.
• [Link] - List variables inside a MATLAB file.
Member of the Helmholtz Association

Jupyter Notebook: Load_List_Save_MAT_files

10. October 2017 Porting Code from Matlab to Python 9

 Matplotlib
Ø [Link]
Numerical python functions written for compatibility
with MATLAB commands with the same names.

MATLAB compatible functions

:func:`cohere` Coherence (normalized cross spectral density)
:func:`csd` Cross spectral density using Welch's average periodogram
:func:`detrend` Remove the mean or best fit line from an array
:func:`find` Return the indices where some condition is true; [Link] is
similar but more general.
:func:`griddata` Interpolate irregularly distributed data to a regular grid.
:func:`prctile` Find the percentiles of a sequence
Member of the Helmholtz Association

:func:`prepca` Principal Component Analysis

:func:`psd` Power spectral density using Welch's average periodogram
:func:`rk4` A 4th order runge kutta integrator for 1D or ND systems
:func:`specgram` Spectrogram (spectrum over segments of time)

10. October 2017 Porting Code from Matlab to Python 10

 Principal Component Analysis (PCA)

Ø Way of identifying patterns and expressing the data

to highlight their similarities and differences.

Ø Powerful tool for analyzing high dimensional data.

Ø Enables data compression without much loss of

information by reducing the number of dimensions.
Member of the Helmholtz Association

10. October 2017 Porting Code from Matlab to Python 11

 Matlab code for PCA (An example)
rd = load_untouch_nii('[Link]');
rd = double([Link]);
sz = size(rd)
nrows = sz(1)
ncols = sz(2)
nslcs = sz(4)
s = reshape(rd,nrows*ncols,nslcs);
[coeff,score] = pca(s);
s = reshape(score,nrows,ncols,nslcs);
n = make_nii(s);
save_nii(n,'results/[Link]')
Member of the Helmholtz Association

Ref: [Link]
[Link]
produces-different-results-for-pca

10. October 2017 Porting Code from Matlab to Python 12

 PCA using Python ([Link])

Ø Hint:
- Use [Link]
- Imported as given below:
from [Link] import PCA
- Dataset: [Link]
- Ref:
[Link]
Member of the Helmholtz Association

[Link]
Jupyter Notebook

10. October 2017 Porting Code from Matlab to Python 13

 Scikit-learn or sklearn
Ø Meant for machine learning in Python
Ø [Link]
Ø ‘[Link]’ module includes matrix
decomposition algorithms, including among others
PCA, NMF or ICA.
e.g. modules:
- [Link] - Non-negative matrix factorization
- [Link] - Principal Component Analysis

Ø Most of the algorithms of this module can be

Member of the Helmholtz Association

regarded as dimensionality reduction techniques.

10. October 2017 Porting Code from Matlab to Python 14

 PCA using Python (sklearn)

Ø Hint:
- Use [Link]
- Imported as given below:
from [Link] import PCA
- Dataset: [Link]
- Ref:
[Link]
[Link]
Member of the Helmholtz Association

[Link]

Jupyter Notebook Optional

10. October 2017 Porting Code from Matlab to Python 15

 Other Python modules for Statistics
Ø Seaborn : Statistical data visualization
[Link]

Ø Statsmodels : Library for statistical and econometric

analysis in Python.
[Link]
Member of the Helmholtz Association

Jupyter Notebook : seaborn_savefig

10. October 2017 Porting Code from Matlab to Python 16

 References
The Python Language Reference: [Link]
The Python Standard Library: [Link]
[Link]
[Link]
[Link]
[Link]
[Link]
PEP 20 -- The Zen of Python :[Link]
[Link]
[Link]
Member of the Helmholtz Association

10. October 2017 Porting Code from Matlab to Python 17