自用的概率论笔记

@[TOC]

随机事件

$$\begin{aligned} & A\cap B=AB,A\cup B=A+B \\ & P(A+B)=P(A)+P(B)-P(AB) \\ & P(A-B)=P(A\overline{B}) \\ & P(A+B-C)\neq P(A-C+B),P(A+B-C)\neq P(A+(B-C)) \\ & P(A(B-C))\neq P(AB-AC) \\ \end{aligned}$$

事件运算法则

$$\begin{aligned} & A+AB=A,A-B=A\overline{B} \\ & A(B+C)=AB+AC,A+BC=(A+B)(A+C) \\ & A+B+C=A+(B+C) \\ & \overline{A+B}=\overline{A}\overline{B},\overline{AB}=\overline{A}+\overline{B} \\ \end{aligned}$$

独立$P(AB)=P(A)P(B)$，不相容$P(A+B)=P(A)+P(B)$，$AB=\emptyset$

一元线性回归

$$\begin{aligned} & y(n)=a^*x(n)+b^* \\ & J(a,b)=\sum_{n=1}^N(y(n)-ax(n)-b)^2 \\ \end{aligned}$$ $$\begin{aligned} & \frac{\partial J}{\partial b} = -2\sum_{n=1}^N[y(n)-ax(n)-b]=0 \\ & \sum_{n=1}^Ny(n)-a\sum_{n=1}^Nx(n)-bn=0 \\ & b=\frac{1}{N}\left(\sum_{n=1}^Ny(n)-a\sum_{n=1}^Nx(n)\right)=\bar{y}-a\bar{x} \\ & \frac{\partial J}{\partial a} = -2\sum_{n=1}^Nx(n)[y(n)-ax(n)-b]=0 \\ & \sum_{n=1}^Nx(n)y(n)-a\sum_{n=1}^Nx^2(n)-b\sum_{n=1}^Nx(n)=0 \\ & \sum_{n=1}^Nxy-a\sum x^2-\sum x \left(\sum y-a\sum x\right)=0 \\ & \sum xy-\sum x\sum y =a\sum x^2-a\sum x\sum x \\ &\begin{cases} a=\frac{\displaystyle\sum_{n=1}^Nx(n)y(n)-\sum_{n=1}^Nx(n)\sum_{n=1}^Ny(n)} {\displaystyle\sum_{n=1}^Nx^2(n)-\left(\sum_{n=1}^Nx(n)\right)^2} \\ b=\bar{y}-a\bar{x} \end{cases} \end{aligned}$$

分布律

二项分布

$X\sim B(n,p)$
设事件X发生的概率为p，进行n次实验，事件X发生k次的概率为$P(X=k)=C_n^kp^k(1-p)^{n-k}$
$E(X)=np,DX=np(1-p)$

几何分布

$$X\sim G(p)$$

设事件X发生的概率为p，事件X首次发生时的实验次数k满足$P(X=k)=p(1-p)^{k-1}$

超几何分布

$$X\sim H(n,M,N)$$

N件产品中有m件次品，从中任取n件产品不放回，有X件次品的概率为

$$P(X=k)=\frac{C_M^k\cdot C_{N-M}^{n-k}}{C_N^n}$$

泊松分布

$$X\sim P(\lambda)$$ $$\begin{aligned} & P(X=k)=\frac{\lambda^k\text{e}^{-\lambda}}{k!} \\ & \lim_{N\to+\infty}\frac{C_M^k\cdot C_{N-M}^{n-k}}{C_N^n}=C_n^kp^k(1-p)^{n-k} \\ & \lim_{n\to+\infty}C_n^kp^k(1-p)^{n-k}=\frac{\lambda^k\text{e}^{-\lambda}}{k!} \\ \end{aligned}$$

证明如下

$$\begin{aligned} & P(X=k)=\text{C}_n^kp^k(1-p)^{n-k} \\ & \text{E}(x)=np=\lambda \\ &\lim_{n\rightarrow+\infty}\text{C}_n^kp^k(1-p)^{n-k} \\ =&\lim_{n\rightarrow+\infty}\frac{n(n-1)(n-2)\cdots(n-k+1)}{k!}\left(\frac{\lambda}{n}\right)^k\left(1-\frac{\lambda}{n}\right)^{n-k} \\ =&\frac{\lambda^k}{k!}\lim_{n\rightarrow+\infty}1(1-\frac{1}{n})(1-\frac{2}{n})\cdots(1-\frac{k}{n})\left(1-\frac{\lambda}{n}\right)^n\left(1-\frac{\lambda}{n}\right)^{-k} \\ =&\frac{\lambda^k}{k!}\lim_{n\rightarrow+\infty}\left(1-\frac{\lambda}{n}\right)^n \\ =&\frac{\lambda^k\text{e}^{-\lambda}}{k!} \end{aligned}$$

均匀分布

$$X\sim U[a,b]$$

指数分布

$$X\sim e(\lambda)$$ $$f(x)=\begin{cases} \lambda\text{e}^{-\lambda x} &, x\ge 0 \\ 0 &, x<0 \end{cases}$$

指数分布与泊松分布的关系

$$\begin{aligned} & P(Y>t)=P(X=0,t)=\frac{\left(\lambda t\right)^0}{0!}e^{-\lambda t}=e^{-\lambda t},\quad t\ge 0 \\ & F(x)=1-\text{e}^{-\lambda x},x\ge 0 \\ \end{aligned}$$

正态分布

$$X\sim N(\mu,\sigma) \\ f(x)=\frac{1}{\sqrt{2\pi}\sigma} \text{exp}\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\} \\ \Phi(x)=\int_{-\infty}^x\frac{1}{\sqrt{2\pi}}\text{e}^{-\frac{t^2}{2}}\text{d}t$$

数学期望和方差

$$\begin{aligned} \text{Cov}(X,X) &= \text{D}X \\ \text{Cov}(X,Y) &= \text{E}[(X-\text{E}X)(Y-\text{E}Y)] =\text{E}(XY)-\text{E}X\cdot\text{E}Y \\ \rho_{xy} &= \frac{\operatorname{Cov}(X, Y)}{\sqrt{\text{D} X} \sqrt{\text{D} Y}} \\ \text{E}(X\pm Y) &= \text{E}X\pm \text{E}Y \\ \text{D}(X\pm Y) &= \text{D}X+\text{D}Y\pm 2\text{Cov}(X,Y) \\ \text{Cov}(X,Y_1\pm Y_2) &= \text{Cov}(X,Y_1)\pm\text{Cov}(X,Y_2) \\ \text{Cov}(X,kY) &= k\text{Cov}(X,Y) \\ \end{aligned}$$

切比雪夫不等式

马尔可夫不等式

$$\text P\{X>\varepsilon\}\le\frac{\text EX}{\varepsilon}$$

其中$X\ge0$

$$\begin{aligned} \text Ex &=\int_{0}^{\infty} x f(x) d x \\ & \geq \int_{\varepsilon}^{\infty} x f(x) d x \\ &\ge\varepsilon\int_{\varepsilon}^{\infty} f(x) d x \\ &=P\{x \geq \varepsilon\} \end{aligned}$$

切比雪夫不等式

$$\text P\{|X-\text EX|\geq\varepsilon\}\leq\frac{\text DX}{\varepsilon^2}$$

样本方差的无偏估计

$$\begin{aligned} \frac{1}{n-1}\sum_{i=1}^{n}(x_{i}-\bar{x})^{2} &= \frac{1}{n-1}\sum_{i=1}^{n}[(x_{i}-\mu)-(\bar{x}-\mu)]^{2} \\ &=\frac{1}{n-1} \sum_{i=1}^{n}(x_{i}-\mu)^{2}-\frac{2}{n-1}\sum_{i=1}^{n}(x_{i}-\mu)(\bar{x}-\mu) +\frac{1}{n-1}\sum_{i=1}^{n}(\bar{x}-\mu)^{2} \\ &= \frac{1}{n-1} \sum_{i=1}^{n}(x_{i}-\mu)^{2}-\frac{2(\bar{x}-\mu)}{n-1}(n \bar{x}-n \mu)+\frac{n}{n-1}(\bar{x}-\mu)^{2} \\ &= \frac{1}{n-1} \sum_{i=1}^{n}(x_{i}-\mu)^{2}-\frac{n}{n-1}(\bar{x}-\mu)^{2} \\ \text{E}\left[\frac{1}{n-1}\sum_{i=1}^{n}(x_{i}-\bar{x})^{2}\right] &= \text{E}\left[\frac{1}{n-1} \sum_{i=1}^{n}(x_{i}-\mu)^{2}-\frac{n}{n-1}(\bar{x}-\mu)^{2}\right] \\ &=\frac{n}{n-1}\text{D}x-\frac{n}{n-1}\cdot\frac{\text{D}x}{n}=\text{D}x \end{aligned}$$

高维正态随机变量

二维正态随机变量$(X_1,X_2)$的概率密度为

$$\begin{aligned} f(x_1,x_2)=& \frac{1}{2\pi\sigma_1\sigma_2\sqrt{1-\rho^2}}\exp\left\{\frac{-1}{2(1-\rho^{2})}\left[\frac{(x_1-\mu_1)^2}{\sigma_1^2} -2 \rho \frac{\left(x_{1}-\mu_{1}\right)\left(x_{2}-\mu_{2}\right)}{\sigma_{1} \sigma_{2}}+\frac{\left(x_{2}-\mu_{2}\right)^{2}}{\sigma_{2}^{2}}\right]\right\} \\ =& \frac{1}{2\pi\text{det}^{1/2}\mathbf{C}}\exp\left\{-\frac{1}{2} \left[\begin{matrix} x_1-\mu_1 \\ x_2-\mu_2 \end{matrix}\right]^\top\mathbf{C}^{-1}\left[\begin{matrix} x_1-\mu_1 \\ x_2-\mu_2 \end{matrix}\right]\right\} \end{aligned}$$

其中的协方差矩阵$\mathbf{C}$为

$$\mathbf{C}=\left[\begin{matrix} \sigma_1^2 & \rho\sigma_1\sigma_2 \\ \rho\sigma_1\sigma_2 & \sigma_2^2 \\ \end{matrix}\right] =\left[\begin{matrix} \text{D}X_1 & \text{Cov}(X_1,X_2) \\ \text{Cov}(X_1,X_2) & \text{D}X_2 \\ \end{matrix}\right]$$

n维正态随机变量的概率密度为

$$f(\boldsymbol{x})=\frac{1}{(2 \pi)^{n/2}\text{det}^{1/2}\mathbf{C}}\exp \left\{-\frac{1}{2}(\boldsymbol{x}-\boldsymbol{\mu})^{\top}\mathbf{C}^{-1}(\boldsymbol{x}-\boldsymbol{\mu})\right\}$$

条件数学期望

边缘分布

$$\begin{aligned} & f(x)=\int_{a(x)}^{b(x)} f(x,y)\text{d}y \\ & f(y)=\int_{a(y)}^{b(y)} f(x,y)\text{d}x \\ \end{aligned}$$

其中 $a(x),b(x)$ 或 $a(y),b(y)$ 是与 $x$ 或 $y$ 有关的上下界，概率论教材中用的是 $\infty$。
条件分布

$$f(y|x)=\frac{f(x,y)}{f(x)}$$

条件期望

$$\text{E}(x|y)=\int_{a(y)}^{b(y)} yf(x|y)\text{d}x$$

性质1： $\text{E}(\text{E}(X|Y))=\text{E}X$
证明：

$$\begin{aligned} \text Ex =& \int_{a_x}^{b_x}xf(x)\text dx \\ =& \int_{a_x}^{b_x}x\left[\int_{a_y(x)}^{b_y(x)}f(x,y)\text dy\right]\text dx \\ =& \int_{a_x}^{b_x}\int_{a_y(x)}^{b_y(x)}xf(x,y)\text dy\text dx \\ =& \int_{a_x}^{b_x}\int_{a_y(x)}^{b_y(x)}xf(y)\frac{f(x,y)}{f(y)}\text dy\text dx \\ =& \int_{a_y}^{b_y}\int_{a_x(y)}^{b_x(y)}xf(y)\frac{f(x,y)}{f(y)}\text dx\text dy \\ =& \int_{a_y}^{b_y}f(y) \left[\int_{a_x(y)}^{b_x(y)}x\frac{f(x,y)}{f(y)}\text dx\right]\text dy \\ =& \int_{a_y}^{b_y}f(y) \left[\int_{a_x(y)}^{b_x(y)}xf(x|y)\text dx\right]\text dy \\ =& \int_{a_y}^{b_y}\text E(x|y)f(y)\text dy \\ =& \text E[\text E(x|y)] \\ \end{aligned}$$

性质2： $\text{E}[g(X)h(Y)|Y]=h(Y)\text{E}[g(X)|Y]$
证明：