- cross-posted to:
- machine_learning
- cross-posted to:
- machine_learning
December 28 2023 Pankaj Singh writes:
In big data processing and analytics, choosing the right tool is paramount for efficiently extracting meaningful insights from vast datasets. Two popular frameworks that have gained significant traction in the industry are Apache Spark and Presto. Both are designed to handle large-scale data processing efficiently, yet they have distinct features and use cases. As organizations grapple with the complexities of handling massive volumes of data, a comprehensive understanding of Spark and Presto’s nuances and distinctive features becomes essential. In this article, we will compare Spark vs Presto, exploring their performance and scalability, data processing capabilities, ecosystem, integration, and use cases and applications.
Strange. I was able to view the article without either. I also tried in a private browser on both mobile and desktop.
I’m getting this paywall that cannot be skipped anyhow.
I don’t know what’s bypassing it on my setups, but it’s on the wayback machine.
https://web.archive.org/web/2/https://www.analyticsvidhya.com/blog/2023/12/spark-vs-presto-a-comprehensive-comparison/
Thank you! The conclusion is quite good, like use spark as ETL and Presto (Trino) for analytical queries but the article looks very outdated.
Spark is not about RDDs. Today the most usage of Spark is via DataFrame API. And it is not just syntax. The Catalyst itslef provide a lot of performance optimizations, like predicate pushdown on the level of orc/parquet reading, automatic skew joins detection, prunning, etc.
Also Presto in this case should be called as Trino because there was a rebranding in 2020
I was a questioning the quality of the source, thanks for confirming that it’s not a top quality article.