If you care about performance, you may want to avoid CSV files. But since our data sources are often like our family, we can’t make a choice, we’ll see in this blog post how to process a CSV file as fast as possible.

  • NostraDavid
    link
    fedilink
    arrow-up
    3
    arrow-down
    1
    ·
    8 months ago

    Also, regarding better formats: parquet is relatively nice. Smaller files, though not human readable. Use parquet if you read often, or have IO issues (file “too large” as CSV).