Lecture 3: Basic Knowledge in Probability Theory
Machine learning is essentially about finding methods for making decisions, and the best way to make decisions is based on assessing the probability (or likelihood) of potential outcomes. Therefore, probability theory has undoubtedly become a fundamental tool in this field.
In this lecture, we will cover the fundamental concepts of probability theory. To make this note self-contained, I will start from scratch. The benefit of this approach is that we can gradually build on each concept in a coherent manner. However, there may be some topics that we won’t use in this course, so I will list the most important aspects separately at the end. If you have already studied probability theory, you can treat this as a quick review. I will do my best to provide you with an intuitive picture of probability theory. If you feel this isn’t necessary, you can also skip ahead to the exercises.
Before we begin, I want to emphasize something important: probability models exist solely within our rational world. They are summaries of patterns in observational data, rather than replicas of real-world phenomena. No probability model can perfectly replicate reality, but it can help us solve real-world problems. As the statistician Box famously said, “All models are wrong, but some are useful.” I highly recommend you take a moment to read a mini note, it may open a door to a whole new perspective for you.
(WBPA) The tools provided in this lesson are a type of soft tool. They may not help you directly with how to use the software, but they can offer you a way of thinking. A good understanding of them will make things easier for your studies not only in this course but also for all your data science knowledge.
Outline:
- 3.1 From Frequence to Probability
- 3.2 Random Variable and Distribution
- 3.3 Expected Value and Variance
- 3.4 Two Dependent Variables
- 3.5 Continuous Variables
- 3.6 Likelihood Analysis
