Without caching, WordPress builds every page from scratch for every visitor: it runs PHP, queries the database, assembles the HTML and only then sends it. A caching plugin saves the finished page and serves that copy instead. On most sites, this is the single biggest speed improvement you can make in five minutes.
The right plugin depends heavily on your hosting, though. The best cache on a LiteSpeed server is not the best cache on Nginx, and some managed hosts do not want you running one at all. This best WordPress caching plugins guide covers both the plugins and that decision.