Name:
[6237] unpingco
Member:
61 months
Authored:
40 videos
Description:
Python for
scientific and large-scale computing. ...
Scientific and parallel computing using IPython [ID:720]
a series of video-tutorials by unpingco
This series introduces scientific and parallel computing using IPython with emphasis on IPython on a Windows PC. We discuss best practices for effectively using IPython with numpy, scipy, and matplotlib, as well has using IPython for interactive parallel computation.
Video Tutorials
1. Introducing and setting up IPython on a Windows PC
We discuss effective ways to set up and configure IPython on a Windows PC including handy tips for making the most out of IPython in a Windows environment.
16 votes
2. Getting around and using the IPython interpreter
We discuss some of the 'magic' commands, how to use the command history, how to use the editor, some useful keystrokes, and how to get interactive help.
7 votes
3. Using on-line documentation for a short case study on computing integrals
Scipy and numpy are too vast to discuss their constituent components. Instead, we take a simple case study where we want to compute the integral of sampled data and illustrate how to use the on-line documentation effectively.
10 votes
4. Subclassing and using the Python language for scientific computing
in this segment, we discuss how to subclass an existing object from the scipy module, how to use an external editor with IPython, and how to add customized functions to an existing object.
2 votes
5. Using doctest and docstrings in customized classes
In this segment, we continue with our example of subclassing objects from the scipy module and we discuss how to set up and use docstrings and doctests to provide documentation and testing.
3 votes
6. Python Language Features for Large Data Sets
Here we discuss how generators can be used to process large data sets, which may be too resource-consuming to pull into memory all at once. We further introduce the itertools module as a way of extending the functionality of generators.
7 votes
7. Debugging using IPython
We discuss the following debugging methods:
using %run -d -b
using %pdb to catch errors
using pdb.set_trace() in source code
3 votes
9. Introducing Sympy -- Symbolic Mathematics within IPython
Introducing Sympy for symbolic mathematics within IPython. Basic operations and advice for Windows users.
4 votes
10. Getting started with Matplotlib
Matplotlib is one of the best, if not the best, plotting package for scientific Python. In this segment, we discuss how to best use the Matplotlib website to get started with plotting with Matplotlib.
4 votes
11. Getting Started with Parallel IPython
IPython provides excellent interactive parallel computing tools. In this segment, we discuss how to set up remote computational engines using ipcluster and how to use special magic commands in IPython to use these remote engines.
Here we show how to run the IPython interpreter on the Windows PC (as usual), but connect to and interact with remote computational engines.
4 votes
12. Basic Parallel Computing Using IPython
In this segment, we discuss cover importing modules for the remote nodes and how to distribute data (i.e. scatter, gather) to and from these nodes.
9 votes
13. Distributing Parallel Computations Using IPython
In this segment, we discuss the map feature to distribute computations across the available nodes, how to use targets to specify which nodes to use, and how to use blocking to avoid waiting for long computations to finish.
0 votes
14. Finer Control for Parallel Computing in IPython
In this segment, we discuss the push, pull, and push_function, features of IPython that allow finer control over individual namespaces of remote nodes and over the computations that run on them.
3 votes
15. Case Study: Robust Regression Methods Using Python
In this case study, we pull together everything we have learned so far in order to compute the least MEDIAN squared estimate of a line given data with outliers present. This segment describes the fundamental problem.
1 votes
16. Case Study: Robust Regression Methods Using Python continued
In this segment, we discuss how the least-median estimate can be computed from the sample dataset using generators.
2 votes
17. Case Study: Robust Regression Methods Using Python and Parallel IPython
In this segment, we continue with the case study of robust regression methods, but here we show how to use generators and the parallel computing features of IPython to accelerate the computation.
2 votes
18. Closing remarks and using dictionaries effectively for scientific computing
In this final segment, we highlight concepts important for new scientific Python programmers, especially those coming from a Matlab environment. We discuss effective use of Python dictionaries and the timeit profiling module. We also discuss the defaultdict object for quick and easy database creation useful for keeping track of data in multiple parameter runs.
6 votes
nice tool...longing to start using it.