Name: [6237] unpingco
Member: 71 months
Authored: 40 videos
Description: Python for scientific and large-scale computing. ...

Scientific and parallel computing using IPython [ID:720]

a series of video-tutorials by unpingco

This series introduces scientific and parallel computing using IPython with emphasis on IPython on a Windows PC. We discuss best practices for effectively using IPython with numpy, scipy, and matplotlib, as well has using IPython for interactive parallel computation.

Video Tutorials

1. Introducing and setting up IPython on a Windows PC

We discuss effective ways to set up and configure IPython on a Windows PC including handy tips for making the most out of IPython in a Windows environment.

2. Getting around and using the IPython interpreter

We discuss some of the 'magic' commands, how to use the command history, how to use the editor, some useful keystrokes, and how to get interactive help.

3. Using on-line documentation for a short case study on computing integrals

Scipy and numpy are too vast to discuss their constituent components. Instead, we take a simple case study where we want to compute the integral of sampled data and illustrate how to use the on-line documentation effectively.

4. Subclassing and using the Python language for scientific computing

in this segment, we discuss how to subclass an existing object from the scipy module, how to use an external editor with IPython, and how to add customized functions to an existing object.

5. Using doctest and docstrings in customized classes

In this segment, we continue with our example of subclassing objects from the scipy module and we discuss how to set up and use docstrings and doctests to provide documentation and testing.

6. Python Language Features for Large Data Sets

Here we discuss how generators can be used to process large data sets, which may be too resource-consuming to pull into memory all at once. We further introduce the itertools module as a way of extending the functionality of generators.

7. Debugging using IPython

We discuss the following debugging methods: using %run -d -b using %pdb to catch errors using pdb.set_trace() in source code

8. Profiling in IPython

Basic profiling methods available in IPython.

9. Introducing Sympy -- Symbolic Mathematics within IPython

Introducing Sympy for symbolic mathematics within IPython. Basic operations and advice for Windows users.

10. Getting started with Matplotlib

Matplotlib is one of the best, if not the best, plotting package for scientific Python. In this segment, we discuss how to best use the Matplotlib website to get started with plotting with Matplotlib.

11. Getting Started with Parallel IPython

IPython provides excellent interactive parallel computing tools. In this segment, we discuss how to set up remote computational engines using ipcluster and how to use special magic commands in IPython to use these remote engines. Here we show how to run the IPython interpreter on the Windows PC (as usual), but connect to and interact with remote computational engines.

12. Basic Parallel Computing Using IPython

In this segment, we discuss cover importing modules for the remote nodes and how to distribute data (i.e. scatter, gather) to and from these nodes.

13. Distributing Parallel Computations Using IPython

In this segment, we discuss the map feature to distribute computations across the available nodes, how to use targets to specify which nodes to use, and how to use blocking to avoid waiting for long computations to finish.

14. Finer Control for Parallel Computing in IPython

In this segment, we discuss the push, pull, and push_function, features of IPython that allow finer control over individual namespaces of remote nodes and over the computations that run on them.

15. Case Study: Robust Regression Methods Using Python

In this case study, we pull together everything we have learned so far in order to compute the least MEDIAN squared estimate of a line given data with outliers present. This segment describes the fundamental problem.

16. Case Study: Robust Regression Methods Using Python continued

In this segment, we discuss how the least-median estimate can be computed from the sample dataset using generators.

17. Case Study: Robust Regression Methods Using Python and Parallel IPython

In this segment, we continue with the case study of robust regression methods, but here we show how to use generators and the parallel computing features of IPython to accelerate the computation.

18. Closing remarks and using dictionaries effectively for scientific computing

In this final segment, we highlight concepts important for new scientific Python programmers, especially those coming from a Matlab environment. We discuss effective use of Python dictionaries and the timeit profiling module. We also discuss the defaultdict object for quick and easy database creation useful for keeping track of data in multiple parameter runs.

Showmedo is a peer-produced video-tutorials and screencasts site for free and open-source software (FOSS)- with the exception of some club videos, the large majority are free to watch and download.

how to help » about » faq »

Educating the Open-source Community With Showmedo

Although as important as the software it supports, education and documentation are relatively neglected in the Open-source world. Coders love to code, and explaining how best to use or improve the software tends to be deferred or even sidelined.

At Showmedo we believe the community can play a vital role here and also say thanks for the tools and software that make our lives easier. If you have a piece of software you love or a programming langugage you are enthusiastic about, why not make a screencast showing others how to use it? All the stuff you wish you'd been told, the tips, tricks, insights that would have saved you time and frustration.

Screencasting is easier than you think, and we're happy to help you. You can emailus for advice or just use some of the how-to screencasts on the site. This screencasting learning-pathis a good place to start.

Kudos and Thanks for unpingco



Showmedo's development is fairly rapid and bugs will inevitably creep in. If you have any problems please drop us a line using the contact address below. Likewise, any suggestions for improvements to the site are gratefully received.