Your knowledge of data engineering may be limited. SQL is predominant in data processing nowadays. FOSS tools such as DBT allows to write efficient data processing pipelines with SQL and some YAML config without the need for a general purpose coding language.
Why would anyone want that? Because SQL has the interesting property of describing the result you want rather than describing how to compute it. So you can put inside the database, a query engine with decades of optimizations, that will make a much better job at finding the best execution plan than the average developer.
It also means it’s easier to train people for data processing nowadays.
Your knowledge of data engineering may be limited. SQL is predominant in data processing nowadays. FOSS tools such as DBT allows to write efficient data processing pipelines with SQL and some YAML config without the need for a general purpose coding language.
Why would anyone want that? Because SQL has the interesting property of describing the result you want rather than describing how to compute it. So you can put inside the database, a query engine with decades of optimizations, that will make a much better job at finding the best execution plan than the average developer.
It also means it’s easier to train people for data processing nowadays.
Learning DBT was pretty easy for me as a data analyst. Now I’m contributing to my company’s data warehouse instead of just pulling existing data.