Learning Machine Learning

It’s often hard to get into new fields like machine learning or data science. One reason for it being hard, lies in the fact that we don’t even know what do not know.

This list is a compilation of sources that will help you, dear reader, to fill the gaps towards using data science and machine learning. Here, we will not define what these things are, or why exactly you need them. They are more geared towards something to come back to, when you find gaps in your education, while delving into machine learning.

Many readers will be students or come from underprivileged backgrounds. Here I have tried my best to curate free resources, or resources that in my opinion are worth the investment. You will see that some links are monetized, but these links were added after I have curated the content, you will not see any funny business here.

However, although they are arranged in the conventional “start here, then move on” way, as this is the easiest way to structure text, I believe this is not the best way to learn. I would suggest to you, to dive in deep and coming back to this resource, once you find roadblocks. You will find the fast.ai course is very interesting and will throw you right into the fascinating world of deep learning. Sometimes you will realize that maybe your math or your python skills are lacking, then this can be used as a central resource to find a way to fill the gap.

Or read it back to back, I’m not your mother after all.

Mathematics

Mathematics lies at the core of  machine learning. The main components that you have to understand are linear algebra and calculus. Linear algebra is about understanding how neural networks themselves work and calculus is all about training neural networks. If you had troubles in school with math, like me, you may want to start at getting the basics right. Having a good foundation will make your life easier in every subsequent step.

Precalculus and General applications in Math

Khan Academy is fantastic at replacing the bad math education you endured in school. If you find you are out of your depth with the videos below, start where school left you with Khan Academy then use both 3Blue 1Brown and Khan Academy to learn the math necessary for modern machine learning.

Linear Algebra

This is the best introduction to vectors and matrix calculations I have ever seen.

Calculus

Just as well this is the best overview of calculus I have ever seen.

Coding Focused Linear Algebra

This course focuses on translating math to the text editor. It’s good to understand how you can actually put vector logic into your computer. This course has written material and video content, as touching the code yourself is always the best way to learn.

Fast AI course on Computation Linear Algebra

 

Optimization

Numerical optimization is a very general skill, but it lies at the basis of every machine learning application. Be it support vector machines or neural networks, they all minimize an objective function. These functions are extremely important if you want to start doing some work in machine learning. There is a nice introductory video series available. Additionally, you can check out the other two books detailing all of optimization and focusing on convex problems.

Metaheuristics essentials book

Convex optimization book

Programming in Python

I am a Python fan. It’s an extremely readable and high-level programming language. Python itself is very slow, but it can utilize almost any other programming language. When we use the deep learning framework Tensorflow in Python, it is just an abstraction of the real C++ Tensorflow that is extremely optimized and fast. Learning at least some Python will take you a long way in data science. Data has to be cleaned, transformed and feed into the hand-crafted algorithms. Learn the basics in Python and it will take you a long way. Pandas is a Python library that you will find very helpful, when working with data, therefore Pandas is included in this list.

Some code on an expensive Laptop and a book with hipster aesthetic

Coding can have such a nice aesthetic

Codecademy Python interactive course

Automate the boring stuff – Educational Python book (Affiliate Link)

Introduction to Pandas

Statistics

Machine learning is just statistics at scale. Knowing statistical basics is very important to not make a fool of ourselves when throwing a learning machine at data. There is a free classic book and several free courses. I like the following to give specific insights into descriptive and inferential statistics as well as a general basic stats course.

A keyboard and some pretty statistics

Pretty Statistics available now!

Elements of Statistical learning book

Descriptive Statistics – Udacity course

Inferential Statistics – Udacity course

Basic Statistics – Coursera – University of Amsterdam

Design Thinking

Design thinking may be the odd one out. However, this thought process has helped me immensely in improving both my code and my interaction with colleagues. Ego is the Enemy is a book about stoic philosophy that isn’t directly Design thinking, but a good reminder to keep ourselves in check, even when we are doing cutting edge research.

Woman sitting in a cluttered creative space programming away

Design thinking in a design space

Intro to Design thinking – Microsoft course

Empathy: an essential skill in Software Development

Intro course into Design thinking from the d.school itself

Ego is the Enemy – Ryan Holiday

Reproducibility

Making your research reproducible is very important. However, just sharing your code often isn’t enough to make it easily reproducible.

Let’s be honest here, we scientists are mostly hacking together our code and sometimes we don’t know ourselves why it’s working. The following resources will help you create documentation and bundle your code together with the software to run and share. Jupyter is fantastic for exploring data with code. Docker is fantastic at bundling your code and the necessary software for it to be run by peers.

Shipping in Containers is much easier.

Shipping in Containers is much easier.

Jupyter Notebook course

Introduction to Virtual Machines, Containers and specifically Docker

Docker introduction

Data Science

Data Science is the category where machine learning fits into. Analyzing data in various forms is sort of an art and takes practice. Kaggle stands uncommented, as it’s a resource you have to know in this field. And of course, following the Way of the Geophysicist.

Dashboards and Coffe, that's where the magic happens

Dashboards and Coffe, that’s where the magic happens

Kaggle.com

Intro to Data Analysis – Udacity course

Tableau Software — Data Analysis Out of the Box Software

Spotfire Software — Data Analysis Out of the Box Software

Machine Learning

There is classic machine learning that includes methods such as logistic regression and support vector machines. While deep learning is very shiny, often times a simpler model is preferable, learn these first.

This model needs more training.

This model needs more training.

Fantastic visual introduction to Decision Trees

Andrew Ng course on Coursera

Scikit-Learn Tutorials

CS231N Stanford course – Machine Learning Examples

Deep Learning

All the rage at the moment. Three free courses, of which my favorite are the Fast.AI courses. The deep learning book is free and fantastic as well.

Deep Learning Machine - CC-BY Kyle McDonald

Deep Learning Machine – CC-BY Kyle McDonald

Fast AI course Part 1

Fast AI course Part 2

https://www.deeplearningbook.org/

CS231 Stanford course – Deep Learning track

Cloud Computing

Cloud Computing and HPC on clusters is at the heart of deep learning. Many do not have a desktop PC anymore, therefore flocking to Google Cloud or AWS is natural. On Google Colab you will get a free GPU to use, but the setup can be tricky for all of them. These resources try to get you on your way.

The Cloud can be tough.

The Cloud can be tough.g

Google Colab Free GPU

CS231n Amazon Web Services Setup

Coursera Google Cloud Setup

Google Cloud Getting Started

Paperspace Cloud computing platform

Graph Theory

Once you have these concepts down, you can dive off the deep end. Neural networks can mathematically be described as acyclic directed graphs. In the case of some recurrent neural networks, the topology is actually a cyclic directed graph. Additionally, a new type of network that is promised to be changing everything about neural nets are capsule networks. These are only possible due to dynamic routing and stray from the basic linear concepts that come with standard convolutional neural networks. To understand these concepts, graph theory comes in very handy.

Communities in graphs overlapping.

Communities in graphs overlapping.

Intro to Graph Conv Nets

Coursera Course on Algorithms on Graphs

Python Package NetworkX for graph calculations

Gephi – Open source Graph visualization with great open online tutorials

Johns Hopkins Uni script on adaptive routing essential for caps nets and helpful for graph convolutional nets.

Useful Tips and Tricks

These are links that don’t quite fit in other categories, but are concepts and resources that are important.

ML Cheatsheet – Visual examples of everyday (and not so everyday) ML concepts

Neural Network Zoo – Most common architectures

Nash Equilibrium on Khan Academy – concept from game theory that is important for generative adversarial networks

Stay Up to Date

A field that evolves at this kind of speed is hard to keep up with. Here are three resources I use to learn about the newest flavors of deep learning.

Small Megaphone, considering the size of the field.

Small Megaphone, considering the size of the field.

Google Scholar Alerts

Arxiv Sanity by Karpathy

Reddit Machine Learning

2 Minute Papers – Youtube Channel

 

The following two tabs change content below.
... is a geophysicist by heart. He works at the intersection of machine learning and geoscience. He is the founder of The Way of the Geophysicist and a deep learning enthusiast. Writing mostly about computational geoscience and interesting bits and pieces relevant to post-grad life.

Latest posts by Jesper Dramsch (see all)