Hi there

I also like to able to browse through folders. Luckily for you, it is actually possible in Jellyfin itself! In your administrator dashboard > Libraries > Display (or view idk) at the top, check the first box. This adds an extra library of some sort to your home screen in which folders are shown.

Hope this helps !