Understanding Artificial Intelligence through Algorithmic Information Theory

Overview

What you'll learn

• How to measure information through compression
• How to compare algorithmic information with Shannon’s information
• How to detect languages through joint compression
• How to use the Web to compute meaning similarity
• How probability and randomness can be defined in purely algorithmic terms
• How algorithmic information sets limits to the power of AI (Gödel’s theorem)
• A criterion to make optimal hypotheses in learning tasks
• A method to solve analogies and detect anomalies
• A new understanding of machine learning as a way to achieve compression
• Why unexpected means abnormally simple
• Why coincidences are unexpected
• Why subjective information and interest are due to complexity drop and why relevance, aesthetics, emotional intensity and humour rely on coding.

Syllabus 11

1. Complexity as code length
2. Conditional Complexity
3. Complexity and frequency
4. Meaning distance
5. Algorithmic probability, Randomness
6. Gödel’s theorem
7. Universal induction - MDL
8. Analogy & Machine Learning as complexity minimization
9. Simplicity & coincidences
10. Subjective probability
11. Relevance

Skills you'll practise

PlanningArtificial Neural NetworksBasic MathMathematicsMachine LearningCognitive ScienceAestheticsProbability TheoriesArtificial IntelligenceTuring MachineComputer ScienceInformation TheoryInnovationProbability

Advantages and disadvantages

Some points apply to every course of this kind; see how we rank.

Before you start

Examples of what you should know before embarking on this course what a convex curve looks like, that log(7^n) is n times log(7) that rational numbers have finite or periodic expansion, that rational numbers are countable, but that real numbers are not, that the probability of "A and B" is the probability of "A knowing B" times the probability of B, that 65 is 1000001 is in base 2 and 41 in base 16, how to compute the sum of a finite geometric series, that {'a':1, 'i':0} is a Python dictionary and why list('ab'*4)[::2] yields ['a','a','a','a'], that k-means is a clustering method, what Bayes’…

Taught by

• Jean-Louis DessallesAssociate Professor
• Nils HolzenbergerAssociate Professor