You don’t need to understand a system to see what it produces.

This is actually exactly how neural networks work currently.

All we know is that for a given input it creates a given output. The actual formula it’s using to calculate those is massively obviscated.

For example they are using machine learning to predict fluctuations in magnetic fields. We don’t know the equations just it’s starting state and ending state. The AI can still do the calculation even though we can’t.

If we create an AI that performs as we want we don’t need to understand it’s internal workings.

The same way we have effective therapy even though we don’t fully understand how the brain actually works.

It won’t be less. Computers and machines already outperform in a huge array of tasks.

Computers massively outperform us at doing maths. Cars outperform us in speed of travel.

It’s the whole point of technology.

We will one day be capable of creating system that think and understand humans better than we do.