It is common knowledge that pickle is a serious security risk. And yet, vulnerabilities involving that serialisation format keep happening. In the article I shortly describe the issue and appeal to people to stop using pickle.
You must log in or # to comment.
The thing is, none of the suggested alternatives can do what pickle does, and the article focuses on a narrow (albeit ubiquitous) use case: serialisation of untrusted data.
There are still legitimate use cases for pickle, especially when storing, caching, or comparing objects that can’t easily be serialised with say, JSON or TOML. It’s a question of using the right thing for the right job is all, and pretending like JSON is a comparable alternative to pickle doesn’t help anyone.
If you’re serialising trusted data, you can define schema for it and use Protocol Buffers which will not only by safer but also faster. Pretending that you need to be able to serialise arbitrary data hurts everyone.
I second this.
If you need to
pickleyour ML model, just use JobLib instead.If you want to save a polars or pandas df, save files as parquet.
Both ways you can also use compression, so you’ll save space as well. Use
zstdif you need decent compression, orlz4if you write and read speeds.Joblib has the same drawback as
pickle. From the documentation:joblib.dump() and joblib.load() are based on the Python pickle serialization model, which means that arbitrary Python code can be executed when loading a serialized object with joblib.load().
joblib.load() should therefore never be used to load objects from an untrusted source or otherwise you will introduce a security vulnerability in your program.
