 |
| 2 users recommended this comment. What they said: |
|
tel Mar 16, 2010 | on: Math Books Organized by Area of Mathematics | context
To quote justokay quoting Mike Jordan http://news.ycombinator.com/item?id=1055389 I've read a number of books from that sequence and they are definitely the core you need to start hitting for advanced stat. |
|
dr-neptune Aug 16, 2022 | on: Ask HN: In 2022, what is the proper way to get into… | context
> By getting into machine or deep learning I mean building upto a stage to do ML/DL research. > The target ability: > 1. To understand the theory behind the algorithms > 2. To implement an algorithm on a dataset of choice. (Data cleaning and management should also be learned) > 3. Read research publications and try to implement them. There are many different ways that people do ML/DL research these days. Some people do more theory-work which will necessarily be more focused on mathematics, and others do more of an applied approach which will be more focused on coding and iterating. For theory-driven work, I think Michael I Jordans list is still pretty solid: > https://news.ycombinator.com/item?id=1055389 I would focus on the fundamentals first though: 1. get a solid background in mathematics 2. get a solid foundation in statistical machine learning 3. get a baseline understanding of deep learning If you get through to this last step, you are probably solid enough to get a job building models. If that's the route you want, then begin iterating on learning about new approaches in papers (look for papers with code / data) and implementing them. If you want to go the academic route, you have enough of a view of the field to begin specializing further. Choose a sub-domain and dig deep if you want to do more deep learning work. Maybe revisit Michael I Jordan's list if you're still confused about where to go. A lot of those books will feel a lot more familiar. Best of luck! |
Mike Jordan at Berkeley sent me his list on what people should learn for ML. The list is definitely on the more rigorous side (ie aimed at more researchers than practitioners), but going through these books (along with the requisite programming experience) is a useful, if not painful, exercise.
I personally think that everyone in machine learning should be (completely) familiar with essentially all of the material in the following intermediate-level statistics book:
1.) Casella, G. and Berger, R.L. (2001). "Statistical Inference" Duxbury Press.
For a slightly more advanced book that's quite clear on mathematical techniques, the following book is quite good:
2.) Ferguson, T. (1996). "A Course in Large Sample Theory" Chapman & Hall/CRC.
You'll need to learn something about asymptotics at some point, and a good starting place is:
3.) Lehmann, E. (2004). "Elements of Large-Sample Theory" Springer.
Those are all frequentist books. You should also read something Bayesian:
4.) Gelman, A. et al. (2003). "Bayesian Data Analysis" Chapman & Hall/CRC.
and you should start to read about Bayesian computation:
5.) Robert, C. and Casella, G. (2005). "Monte Carlo Statistical Methods" Springer.
On the probability front, a good intermediate text is:
6.) Grimmett, G. and Stirzaker, D. (2001). "Probability and Random Processes" Oxford.
At a more advanced level, a very good text is the following:
7.) Pollard, D. (2001). "A User's Guide to Measure Theoretic Probability" Cambridge.
The standard advanced textbook is Durrett, R. (2005). "Probability: Theory and Examples" Duxbury.
Machine learning research also reposes on optimization theory. A good starting book on linear optimization that will prepare you for convex optimization:
8.) Bertsimas, D. and Tsitsiklis, J. (1997). "Introduction to Linear Optimization" Athena.
And then you can graduate to:
9.) Boyd, S. and Vandenberghe, L. (2004). "Convex Optimization" Cambridge.
Getting a full understanding of algorithmic linear algebra is also important. At some point you should feel familiar with most of the material in
10.) Golub, G., and Van Loan, C. (1996). "Matrix Computations" Johns Hopkins.
It's good to know some information theory. The classic is:
11.) Cover, T. and Thomas, J. "Elements of Information Theory" Wiley.
Finally, if you want to start to learn some more abstract math, you might want to start to learn some functional analysis (if you haven't already). Functional analysis is essentially linear algebra in infinite dimensions, and it's necessary for kernel methods, for nonparametric Bayesian methods, and for various other topics. Here's a book that I find very readable:
12.) Kreyszig, E. (1989). "Introductory Functional Analysis with Applications" Wiley.