The Simple and Infinite Joy of Mathematical Statistics, by J.N. Corcoran, is a classical introduction to mathematical statistics that has been written for a varied audience of graduate and undergraduate students with diverse backgrounds and skill sets. It has been in use in draft form for over 20 years in statistics departments where students may or may not have had advanced mathematical fluency, and in mathematics departments where students may or may not have had any statistical background. "Joy" has been written in a way that tries to bridge the divide between these groups of students. The text begins with an optional review of basic probability concepts, and then covers basic tools needed before diving into the world of estimation and inference. Topics include

convergence concepts for sequences of random variables
order statistics
maximum likelihood estimation with asymptotic properties
pivotal quantities and confidence intervals
method of moments estimation
sufficiency, completeness, and minimum variance unbiased estimation
uniformly most powerful hypothesis tests
generalized likelihood ratio tests and Wilks' Theorem

Lose the assumptions of normality and learn to develop your own statistical tests!

convergence concepts for sequences of random variables
order statistics
maximum likelihood estimation with asymptotic properties
pivotal quantities and confidence intervals
method of moments estimation
sufficiency, completeness, and minimum variance unbiased estimation
uniformly most powerful hypothesis tests
generalized likelihood ratio tests and Wilks' Theorem

Lose the assumptions of normality and learn to develop your own statistical tests!