Bilingual website

I would like to create a bilingual website. I do NOT mean that I would like to incorporate a Google translate API to translate content automatically: I mean that I would like to be able to incorporate a language toggle on each page, the clicking of which would serve up a version of the page in a different language (in this case, French, and I shall create the French-language versions of each page myself).
A good example of the kind of configuration I am trying to implement can be seen on this page: https://www.lexikall.com/ (this is NOT my website).

Does this kind of functionality need to be implemented via a theme? Or via a plug-in?
Please excuse the na?veté of my question.
Any help would be gratefully received.

The page I need help with: [log in to see the link]