Taking a tour through the mines in Potosi, Bolivia, was enough to convince me to never enter an enclosed, underground space again. I can't imagine how these people deal with the claustrophobia, oppressive heat, and painfully conspicuous lack of oxygen their work entails.
Caves and mines are different. Most caves in northern US are at 46-49F all year around which is quite comfortable for strenuous activities. There is never lack of oxygen. Caving is likely the second most demanding physical activity I was involved in (most demanding being mountaineering). For most people being in complete dark, surrounded by rock on all sides is probably even bigger psychological challenge than physical one. I have known few cavers who likes to spent entire week inside caves exploring, mapping, reflecting. It certainly requires different type of personality. One caver described being in cave as getting back to womb and coming out of Earth as re-born just like baby comes out squeezing in through birth canal. My own experiences are varied depending on caves. Caves are some of the very rare places where you can feel completely utterly almost impossibly disconnected from rest of the world. BTW, I'm not talking about those commercial gigantic caves, I'm talking about those little holes on Earth that are unknown outside caving communities such as NSS and local grottos. I can say that caving is likely one of 5 most important things I got in to. A very important part of caving is the people (you never go caving alone!). Mountaineering folks are typically very introverts but cavers are just some of the wildest awesome bunch out there. Look up organizations called "grottos" in your state or country and go out on beginners trip.
A hard-core caving buddy of mine once took me on a couple cave expeditions of increasing difficulty, though they were easy for him. As a newbie, I was more than a little uncomfortable. It is a totally new way of seeing the world around you, and by no means a natural experience. I have much respect for those who do it regularly. Just make sure you don't call them spelunkers, which is a much tamer tourists-only approach. When doing real caving, even the "simple" caves, the psychological and physical requirements are much more than you'd at first expect.
> For most people being in complete dark, surrounded by rock on all sides is probably even bigger psychological challenge than physical one
Just last week, while riding to the airport we went through a large and long tunnel. I suddenly felt trapped (a first), and then my brain got confused by the lack of light thinking the black 'ceiling' was a night open sky. I imagine being in a cave might trigger similar sensations.