Populations, samples and sampling error

In short

• A population is every member of the group we care about; a parameter (e.g. $\mu$) describes it. A sample is a subset; a statistic (e.g. $\bar X$) describes the sample and estimates the parameter.
• Probability sampling gives every member an equal chance of selection and tends to produce a representative sample. Non-probability sampling relies on judgment or convenience and risks a non-representative one.
• Simple random sampling suits a homogeneous population. Systematic sampling (every $k$th member of a list) is a practical way to get an approximately random sample.
• Sampling error = statistic − parameter. It exists because only part of the population is observed, even when the sample is drawn perfectly.
• A statistic is a random variable. Its sampling distribution is the distribution of all the values it can take across samples of the same size from the same population.