The integrated sensing and communication (ISAC), in which the sensing and communication share the same frequency band and hardware, has emerged as a key technology in future wireless systems due to two main reasons. First, many important application scenarios in fifth generation (5G) and beyond, such as autonomous vehicles, Wi-Fi sensing and extended reality, requires both high-performance sensing and wireless communications. Second, with millimeter wave and massive multiple-input multiple-output (MIMO) technologies widely employed in 5G and beyond, the future communication signals tend to have high-resolution in both time and angular domain, opening up the possibility for ISAC. As such, ISAC has attracted tremendous research interest and attentions in both academia and industry. Early works on ISAC have been focused on the design, analysis and optimization of practical ISAC technologies for various ISAC systems. While this line of works are necessary, it is equally important to study the fundamental limits of ISAC in order to understand the gap between the current state-of-the-art technologies and the performance limits, and provide useful insights and guidance for the development of better ISAC technologies that can approach the performance limits. In this paper, we aim to provide a comprehensive survey for the current research progress on the fundamental limits of ISAC. Particularly, we first propose a systematic classification method for both traditional radio sensing (such as radar sensing and wireless localization) and ISAC so that they can be naturally incorporated into a unified framework. Then we summarize the major performance metrics and bounds used in sensing, communications and ISAC, respectively. After that, we present the current research progresses on fundamental limits of each class of the traditional sensing and ISAC systems. Finally, the open problems and future research directions are discussed.