Notes on learning AI & ML22 Nov 2021
Hello Mati! I'm really really sorry I didn't respond to this sooner, I was really happy to see your message! To be honest I wasn't sure exactly the best way to introduce you to the concepts you wanted to learn about. Here's my best attempt.
Tom this is far too long just give me links
I know there's a lot here. I've summarised where I think you should start, and then I guess this page might be helpful to you as a reference.
Quick note before the list: ML & AI is mostly statistics. That can make it seem daunting, but most of the concepts can be explained very well visually, so youtube and interactive websites are the best way to learn, especially as a non-programmer (but that's how CS people learn this, too!). That makes citing hard in your situation. I'd recommend learning what actually matters to you — probably neural nets and some of the problems we try to solve with them, like clustering and classification — and some other high-level concepts, such as Optimisation, explainable AI, and supervised vs unsupervised learning. Once you know what matters to you, find some intro textbooks that cover what you care about, skim them to make sure you understand, and just cite those. Honestly, it's your best bet.
I'll send you any .pdfs you need (if you don't have access to anything I likely will through Glasgow Uni) and if I think of anything useful for you to cite academically, I'll let you know.
- Learn about Neural Nets. They're the most popular modern ML technique and there are many explanations about how they work.
- They're likely what you think of when you think of modern things made with ML & AI
- I recommend 3Blue1Brown as a first place to learn.
- I recommend staying away from textbooks, they're generally awful for this stuff.
- AI & ML is mostly statistics tricks. Here's a good place to learn some stats — if you're still stuck, try youtube — and here's a similar site for machine learning.
- Learn Anything and metacademy can show you where to learn specific things that you want to go deeper on
- A good place to start might be their deep learning from the bottom up, which is probably what this should have been in the first place…whoops!
- Data Skeptic is a brilliant podcast for learning more about how these things work.
- They have a whole playlist of AI & ML concepts
- Their "mini" episodes are usually easy to understand explainers on different topics. That might be quite useful: some of the other episodes get a bit complicated for begineers at times, so look for those especially.
- Some other useful episodes:
- Reinforcement Learning
- Perceptrons (these are mini neural nets!)
- If you're not a podcast person, I recommend Pocket Casts as a good place to listen. Way better than spotify.
What do we mean by AI & ML?
The first thing we need to do is to work out what we actually mean by AI and ML. When you ask to learn about these things I think what you're getting at is along the lines of, "Computers are getting better and better at making 'decisions'. This has legal implications! How do they do that?" — and the answer isn't always AI or ML. And sometimes it's kind of AI or ML: they're both really just advanced statistics and clever observations. Is "stats" AI? Is "a program" ML? Where do we draw the line?
The truth is that there are more things that we use to make decisions than AI or ML, and some of them don't really have names.
The best way to approach this stuff is to learn some of the important concepts/tricks at a high level, and then go as deep into any particular approach using those tricks as you'd like. For example, one trick that's often used is "classification". Classification is our name for being shown an example of something and having to decide what "kind" of thing it is, labelling it. I have a student currently using this to take images of salmon and identify whether the salmon is a specific species that's currently causing problems in Scottish rivers. How do you classify / label it? There are all sorts of methods. But the problem comes up lots, and knowing that, you'll begin to realise that every AI / ML technique is really trying to solve only one problem out of a handful we care about. Mathematical concepts can also be useful: there are only a few that matter and everything else is really a special version of those. I'll list some below.
How should I learn them?
Don't read books or academic materials to learn from. They're useful to cite, but they're horrible for learning this stuff. Really.
- Youtube works really well for this.
- If you only use one thing, 3Blue1Brown does amazing videos on stuff like this. Bet his ML ones are good. Here's a link.
- CS people actually mostly read blogs to learn, not books or academic text.
- We also often write code. You might find it useful to watch youtube videos of ML / AI explained by non-programmers who learned to program for this reason. For example, this person who taught a computer to make pasta using machine learning.
- We write code because it's interactive; the equivalent for non-programmers is probably "explorable explanations". Here's a few:
- Once you've got the gist of how they work, I'd go to Google Scholar and just find citations for intro textbooks for whatever you've learned about. Look for ones with high citation counts, and scan through the chapter that's of interest to make sure you get it. If you do, you might as well just cite that, right?
- If anything like this is useful, I'll reference it below.
Things you might want to learn
With that in mind, here's some things you might want to learn about.
AI & ML techniques
- Neural networks
- The most important ML method by far.
- Most of the things you'll think of when you think of ML are probably made using this today, although the field is evolving rapidly and many new techniques exist. In practice they're quite stats-y though. Youtube vids will be great for these.
- The first of these was the "Perceptron". It's simple, so probably a good place to start.
- The modern ones you should care about are "Convolutional Neural Nets" and "Recurrant Neural Nets", in my opinion. There are many others.
- "Deep learning" is also important. It basically means a very very big and complicated neural net.
- Operational Research
- This is a set of mathematical techniques for making decisions
- Markov Chains are a particularly useful and important example (simple explanation on the Simple English Wiki)
- "SAT solvers" are used to solve whole categories of problems, called "NP" problems. Interesting ones are sudoku puzzles and constraint satisfaction. I'll add a reference here when I get chance but a quick youtube search will tell you lots. Look for "P vs NP explained" or something similar.
- My favourite of these is "simulated annealing". The problem it solves is to find some biggest / optimal value ("Hillclimbing", see below). It does this by simulating the atoms in a hot metal cooling down: it tries many values that are very different / extreme to find the best one, and over time, tries things that are closer and closer together, where they're all near the best value it's seen. Eventually they're not changing at all and it's found what it thinks is the best option, but it's done this by trying only a few.
- Genetic algorithms
- Simulates genes swapping to "evolve" an optimal solution to a problem. This is surprisingly efficient.
- It's not very popular today, but a couple of decades ago it was all the rage! It's also very interesting to learn about.
- This is of legal significance, I think, because it might offer good ways of us explaining how or why a computer came to the decision it came to. One big problem for AI & ML folk is that they can't tell you how they arrived at a decision. Genetic algorithms are also hard to trace the decisions of, but they're much easier than neural nets, and this could be useful in answering some philosophically challenging questions when algorithms are applied in a legal context.
- This is the original paper describing genetic algorithms and it's a good place to start.
- I'm happy to explain these in detail if they're interesting to you — I've actually done a bit of research on them 😎 (and they're the only ones I fully understand in this list!)
- Constraint satisfaction
- Optimisation / Hillclimbing
- Supervised & unsupervised learning
Good things to cite
Much like somebody with a pretentious bookcase, I know these are important works in the philosophy of AI & ML, but I can't claim I've actually read them. Still, if they're useful to you, they'd make excellent citations in your dissertation.