Bootstrap and jackknife resampling

In short

• Resampling repeatedly draws samples from the observed sample to make inferences about population parameters.
• Bootstrap: draw many resamples of the same size as the original sample, with replacement, and compute the statistic in each. The spread of those values estimates the standard error, and their distribution gives confidence intervals.
• The bootstrap works for statistics with no simple formula, such as the median, and is model-free (non-parametric): it assumes no distribution for the population.
• Jackknife: leave out one observation at a time, without replacement. A sample of $n$ needs $n$ repetitions and gives the same result every run. Often used to reduce bias, and also for standard errors and confidence intervals.
• Bootstrap results vary from run to run because the draws are random, and the analyst chooses the number of resamples $B$.