Because of its small size and close proximity to the Sun, we didn't know very much about Mercury until NASA's MESSENGER (Mercury Surface, Space Environment, GEochemisrty, and Ranging) probe began its approach and orbit of the planet in 2008. But what we did know didn't seem to suggest that there had been much geological activity—and boy, were we wrong about that.
We now know that Mercury has been home to at least 51 volcanoes, the oldest of which probably erupted about 3.5 billion years ago and the most recent of which may have erupted a billion years ago. None of this says very much about the planet's geology now, but the fact that it was once a geologically active planet also suggests that it was once hotter than it is now and has undergone a gradual cooling process with the eruption of each volcano. Volcanoes often make planets warmer by creating or contributing to a greenhouse effect, but Mercury—having very little gravity to speak of—would have simply spewed the contents into the vacuum of space.
This evidence of past volcanic activity also suggests that if there's any life in Mercury's most habitable polar regions, it's either very new or very hardy. But given Mercury's weak atmosphere and volatile surface temperature, any life we might find there would already need to be ludicrously durable by Earth standards.