Tool that will remove duplicate lines from a file? [closed]

Closed. This question is seeking recommendations for books, tools, software libraries, and more. It does not meet Stack Overflow guidelines. It is not currently accepting answers.

---

We don’t allow questions seeking recommendations for books, tools, so开发者_Python百科ftware libraries, and more. You can edit the question so it can be answered with facts and citations.

Closed 4 years ago.

Improve this question

Is there a mac file comparison tool that can detect duplicates within a file? This is very close to what I'm looking for except it is for Windows: http://www.rlvision.com/dupli/about.asp. Also, I need to compare duplicate content across two files. If nothing else, I can just append file2 content to the bottom of file1.

---

I don't know of a GUI app to do it, but it's fairly easy at the terminal:

cat file1.txt file2.txt | sort | uniq -d

The -d will make it show only duplicate lines. If you just want the output without duplicates, just leave off the -d.