Dask Tutorial

This is a forked from https://github.com/dask/dask-tutorial

Dask is a parallel and distributed computing library that scales the existing Python and PyData ecosystem. Dask can scale up to your full laptop capacity and out to a cloud cluster.

Prepare

1. You should clone this repository

git clone http://github.com/esarrazin/dask-tutorial

2. Install via pixi

Install pixi

In the main repo directory

cd dask-tutorial
pixi install 
pixi shell

Launch Jupyter

From the repo directory

jupyter lab

You are welcome to use Jupyter notebook if you prefer, but we'll be using lab in the live tutorial.

Links

• Reference
  • Docs
  • Examples
  • Code
  • Blog
• Ask for help
  • dask tag on Stack Overflow, for usage questions
  • github issues for bug reports and feature requests
  • discourse forum for general, non-bug, questions and discussion
  • Attend a live tutorial

Outline

0. Overview - dask's place in the universe.

1. Dataframe - parallelized operations on many pandas dataframes spread across your cluster.

2. Array - blocked numpy-like functionality with a collection of numpy arrays spread across your cluster.

3. Delayed - the single-function way to parallelize general python code.

4. Deployment/Distributed - Dask's scheduler for clusters, with details of how to view the UI.

5. Distributed Futures - non-blocking results that compute asynchronously.

6. Machine learning - use dask for machine learning.