Also, another issue with what you’re suggesting is that people have to memorize several conversion factors as well. Inherently, you only have to be able to convert inches -> cm and pounds -> kg, but unless you want to do even more math in your head, you also have to remember feet -> cm, yards -> cm, miles -> cm, square feet -> square meters, cubic feet -> cubic meters (phew, that’s just all the length conversions), pounds -> kg, ounces -> grams, pounds -> grams, cups -> grams (for every fluid you might want to measure), litres -> gallons, litres -> pints, etc.

Or you could just go through the one-time effort of actually using the metric system so you don’t have to carry this mental burden with you everywhere you go…