It’s about which forces act on them. Other than neutrons, every particle here will interact with charged particles (e.g. electrons or protons), which makes it an EM interaction. In the world of particles, EM is long range. Whereas the force that acts on neutrons is very, very short range. There’s no actual touch the way we think of it, outside exotic conditions like in supernovae. Think of the size of a nucleus vs an atom. Those are the two forces keeping them together.

The way concrete stops neutrons is similar to a pinball machine. It doesn’t need to be a solid material, it just needs to be dense. It just so happens that solid materials tend to be denser. But for example, older nuclear reactor designs use water to slow the neutrons.

Btw this is a very clever use of water and by no means dinosaur tech. We just have more clever materials now, e.g. molten salt.