You might want to check here for more info: https://www.reddit.com/r/LocalLLaMA/.

This website and server will only be accessible within my network

If this is the case then your security is already a done deal. No real need for ufw (though you could argue that it is a good thing to have either way). You have no need for fail2ban if it is internal server, nobody can connect to it outside of your internal network, meaning you might only manage to ban yourself by accident.