How to remove duplicates from Collections or Stream in Java? Stream distinct() Example

Hello guys, if you wonder how to remove duplicates from Stream in Java, don't worry. You can use the Stream.distinct() method to remove duplicates from a Stream in Java 8 and beyond. The distinct() method behaves like the distinct clause of SQL, which eliminates duplicate rows from the result set. The distinct() is also a standard method, which means it will return a new Stream without duplicates, which can be used for further processing. Like other methods of Stream class, I mean, map(), flatmap(), or filter(), distinct() is also lazy, and it will not remove duplicate elements until you call a terminal method on Streams like collect or forEach().  

Yes, this is an important concept to know about Java Stream that the actual processing of the Stream pipeline starts only after calling terminal methods like collect() or forEach(). 

If you are interested in learning more about how Stream processing works or internal details of Stream, I will encourage you to go through these best Java Stream courses, which cover them nicely, along with other Java 8 concepts.

Java 8 Stream.distinct() Example

Here is an example of Stream.distinct() method to remove duplicate elements from Stream in Java 8. 

In this example, I have a list of duplicate integer values, then I will create a stream from that List and remove duplicates using the distinct() method. The final Stream will have no duplicates and result will be collected again in a List to show that no duplicates are present in the List. 

package tool;

import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;

/**
* 
* A simple Java Program to to remove duplicates from Stream in Java 8
* This example uses Stream.distinct() method to remove
* duplicates. 
*/
public class Hello {

public static void main(String args[]) {

List<Integer> withDupes = Arrays.asList(10, 10, 20, 20, 30, 30, 40, 50);

System.out.println("List with duplicates: " + withDupes);

List<Integer> withoutDupes = withDupes.stream()
                                      .distinct()
                                      .collect(Collectors.toList());

System.out.println("List without duplicates: " + withoutDupes);

}
}


Output
List with duplicates: [10, 10, 20, 20, 30, 30, 40, 50]
List without duplicates: [10, 20, 30, 40, 50]

You can see that the resulting List doesn't have a duplicate because we have collected it from the Stream after calling the distinct method, which removes the duplicate elements 10, 20, and 30. In other words, this is a new way to remove duplicate elements from a List in Java.

If you are interested in learning new features of Java 8, I encourage you to check out these Java 8 to Java 13 feature courses, which cover essential Java 8 changes in exquisite detail, including Stream and Lambda expression.

Important points

1. The distinct() method belongs to java.util.stream.Stream class.

2. The distinct() method represents an intermediate Stream operation, which returns a new Stream without duplicates.

3. Like other stream methods, distinct is lazy and will not start working unless you call it a terminal method. If you are interested in learning the internal details of Stream, I will encourage you to read Core Java Volume 1 and 2 by Cay S Horstman.

4. You can also use the distinct method to remove duplicates from List or ArrayList by first converting them to Stream, removing duplicates, and then collecting results back to Stream.

That's all about how to remove duplicate values from Stream in Java 8. As seen in the above example, you can use the distinct() method of the Stream class to remove duplicate elements from a given Stream. Whenever you want to work with unique values of Stream, you can use distinct; it works with all kinds of Stream, like Stream of String or Integer.

Other Java 8 tutorials and Resources for further learning

• Top 5 Courses to Learn Java 8 Programming (courses)
• 5 Books to Learn Java 8 from Scratch (books)
• How to use Stream class in Java 8 (tutorial)
• How to use forEach() method in Java 8 (example)
• 10 Java Date, Time, and Calendar based Questions from Interviews (questions)
• How to change the date format of String in Java 8? (tutorial)
• Top 5 Courses to learn Lambda Expression and Stream (courses)
• How to format/parse the date with LocalDateTime in Java 8? (tutorial)
• How to join String in Java 8 (example)
• How to use filter() method in Java 8 (tutorial)
• Top 5 courses to become a full-stack Java developer (courses)
• Java 8 map + filter + stream example (tutorial)
• 10 Examples to format and parse Date in Java 8? (tutorial)
• How to convert Timestamp to Date in Java? (example)
• 20 Examples to learn new Date and Time API in Java 8 (example)
• How to compare two Dates in Java 8? (example)
• 5 Free Courses to learn Java 8 and 9 (courses)

Thanks for reading this article so far. If you like this article, then please share it with your friends and colleagues. If you have any questions or feedback, then please drop a note.

P. S. - If you are looking for a free online course to learn lambda expression and Stream API, I suggest you check out this Java Programming, Lambdas, and More course on Udemy. It's a completely free course, and you will also learn about new Java features from JDK 9, 10, 11, 12, and Java 13.