This:

Problem is when you get passed because other people aren’t driving legally.

Is what made me assume that they at least desire to block traffic by driving on the left at the speed limit. People breaking the speed limit might be a problem, but it isn’t your problem.