While the two fields named in the title seem unrelated, there is a strong link between them. Indeed, random matrix theory makes predictions in both fields, and some of these predictions can be verified rigorously on the number theory side. This amazing connection came to life during a meeting between Freeman Dyson and Hugh Montgomery at the Institute for Advanced Study. Random matrices are now known to predict many statistics about zeta functions, such as moments, low-lying zeros and correlations between zeros. The goal of this talk is to discuss this connection, focusing on number theory.