Not serious does not equal shameful. You should not feel ashamed to have fun.

Edit: You should feel very ashamed if you hurt someone while having fun, but that’s a different story.