Type I and Type II errors, significance and power

In short

• Type I error: rejecting a true null, a false positive. Its probability is the significance level $\alpha$.
• Type II error: failing to reject a false null, a false negative. Its probability is $\beta$.
• Confidence level = $1 - \alpha$. Power = $1 - \beta$, the probability of correctly rejecting a false null.
• With the sample size fixed, cutting $\alpha$ raises $\beta$ and lowers power. Only a larger sample reduces both errors at once.
• Choose $\alpha$ by weighing what each mistake would cost.