I use them interchangeably and I’ve never had a layperson get that glassy eyed stare they get when I talk about IT stuff they don’t understand.

I call the top level hierarchy directories directories. Eg /, bin, boot, lib, var, etc. All directories.

I generally refer to directories within them as folders.