There seem to be a few of these in Github, found this one recently it seems to do a reasonable job. The real problem is its bash script you need WSL2 to run it on windows.
What I like is the output is CSV files that are spreadsheet friendly and can be used to analyze and remove files in bulk.
https://github.com/Jim-JMCD/Duplicate-File-Finder
Czkaaka (CLI version) output file can be used to delete stuff in bulk but doesn’t list directories separately you have to through files individually.
The real problem is its bash script you need WSL2 to run it on windows.
Eh?
WSL2 is one way to run a Linux kernel (and thus native Linux executable binaries) in Windows.
And while bash is definitely very common on Linux, it has never by any means been a strictly Linux program.
It can be used on all kinds of operating systems – mostly unix-like operating systems, but also including Windows using a POSIX compatibility shim like Cygwin.
People were using bash in 1989, years before Linux became the beginning of a thing. And folks have been using it on Windows since at least 1995, or maybe even earlier – decades before WSL.
WSL2 is still the easiest way to get bash imo.
For basic duplicates, I find rmlint to be a superior alternative.
I use VisiPics. It’s very old, but the algorithm is very good.
Just tried this! It’s either so efficient my CPU isn’t working too hard or it focuses most of its work onto one CPU core. That said, I appreciate being able to identify the highest res version of a file, that helps a lot!
Yep, it’s ancient but the algorithm is still good in 2023.
I thought it was open source but it seems it’s just freeware? I was hoping someone would build a new front-end…unfortunately I lack the skills to do so.
Your connection is WiFi?
The latest WiFi standards provide faster theoretical throughput than what most motherboards have onboard for ethernet.
lol. lmao, even
Lol.
This was /r/apple’s excuse for why iPhones arill have USB 2.
Completely different scenario.
Most motherboards only come with a single 1Gbps port, maybe 2.5Gbps if you’re lucky. The latest WiFi standards far exceed 1Gbps, so I’m not sure why people are up in arms about my comment when it’s factual.
Revolutionized the way I organize my files. There is no better tool.
Revolutionized
got a link?
Anti-Twin (in pixel recognition mode) takes a long time, but finds more dupes.
i am looking for a tool to find similar videos i had downloaded before … its big like 10 tb in 2 two drives…
Czkawka, dupeGuru, and VisiPics are my go-to for non-exact photo duplicates. I typically run all three, since they don’t all find the same duplicates. VisiPics runs fine under wine.
But Czkawka’s real strength is non-exact video duplicates. The only other tool I’ve found that does that is videoduplicatefinder.
Always loved Picasa unfortunately stopped development. And it crashed if you compare over 100k images
I love it, but can never remember how to pronounce it so I can’t recommend it to people very easily.
I was just thinking about this tool the other day while I was driving. “duplicate file finder with polish name”
Any suggestions for a tool like this i could run natively on my QNAP NAS?
I need similar tool but for music. Name of the files might be different. Size might be slightly different as well but im not sure that its count as duplicates
Czkawka has something for finding duplicate music, though I’ve never used it.
This tool works for audio, too! I haven’t tried it for audio but the options are there.
I’ve removed at least 100GB of duplicates using this tool. Running every few months is a good way to cleanup things when my download queues get unruly.
This utility helps you find duplicate files, empty folders, images, videos, and music that are similar to one another, and much more. And it’s free. Here’s a link to the repository.
I used to use a program called NoClone, but it appears that somebody may have purchased it a while back and stopped development and made it kind of shady.
My old licensed version doesn’t work very well, so I was looking for a replacement and as I have learned more and more about this program, I’ve been able to use it more and more effectively.
Edit: I should mention I’m a visual person, so I do use the GUI.