List java убрать дубликаты

Русские Блоги

Java: удалить повторяющиеся значения в коллекции List (четыре простых метода)

В недавних проектах необходимо иметь дело с дублирующимися значениями в коллекции списков, большинство из которых используют два метода: один — перебрать коллекцию списков, а затем назначить ее другой коллекции списков, другой — присвоить ее коллекции наборов и вернуть в список. набор.
Но после назначения коллекции наборов, поскольку коллекция наборов неупорядочена, исходный порядок нарушается. Поэтому я подумал, смогу ли я использовать функции набора для дедупликации, не нарушая порядок?
Также можно попробовать, несколько методов имеют свои преимущества в разных ситуациях. Теперь напишите код и сравните его.

 // Набор дедуплицируется, не нарушая порядок public static void main(String[] args) < Listlist = new ArrayList(); list.add("aaa"); list.add("bbb"); list.add("aaa"); list.add("aba"); list.add("aaa"); Set set = new HashSet(); List newList = new ArrayList(); for (String cd:list) < if(set.add(cd))< newList.add(cd); >> System.out.println ("Коллекция после дедупликации:" + newList); > 

 // После прохождения, судья назначить другому списку коллекции public static void main(String[] args) < Listlist = new ArrayList(); list.add("aaa"); list.add("bbb"); list.add("aaa"); list.add("aba"); list.add("aaa"); List newList = new ArrayList(); for (String cd:list) < if(!newList.contains(cd))< newList.add(cd); >> System.out.println ("Коллекция после дедупликации:" + newList); > 

 // установить дедупликацию public static void main(String[] args) < Listlist = new ArrayList(); list.add("aaa"); list.add("bbb"); list.add("aaa"); list.add("aba"); list.add("aaa"); Set set = new HashSet(); List newList = new ArrayList(); set.addAll(list); newList.addAll(set); System.out.println ("Коллекция после дедупликации:" + newList); > 

 // установить дедупликацию (сокращено до одной строки) public static void main(String[] args) < Listlist = new ArrayList(); list.add("aaa"); list.add("bbb"); list.add("aaa"); list.add("aba"); list.add("aaa"); List newList = new ArrayList(new HashSet(list)); System.out.println ("Коллекция после дедупликации:" + newList); > 

Hashset не сортирует, есть еще один способ - использовать treeset, дедупликацию и упорядочить в естественном порядке, просто изменив hashset на treeset. (Исходный порядок изменен, только в алфавитном порядке)

// Дедупликация и упорядочение в естественном порядке List newList = new ArrayList(new TreeSet(list)); 

Источник

Removing All Duplicates From a List in Java

The Kubernetes ecosystem is huge and quite complex, so it’s easy to forget about costs when trying out all of the exciting tools.

To avoid overspending on your Kubernetes cluster, definitely have a look at the free K8s cost monitoring tool from the automation platform CAST AI. You can view your costs in real time, allocate them, calculate burn rates for projects, spot anomalies or spikes, and get insightful reports you can share with your team.

Connect your cluster and start monitoring your K8s costs right away:

We rely on other people’s code in our own work. Every day.

It might be the language you’re writing in, the framework you’re building on, or some esoteric piece of software that does one thing so well you never found the need to implement it yourself.

The problem is, of course, when things fall apart in production — debugging the implementation of a 3rd party library you have no intimate knowledge of is, to say the least, tricky.

Lightrun is a new kind of debugger.

It’s one geared specifically towards real-life production environments. Using Lightrun, you can drill down into running applications, including 3rd party dependencies, with real-time logs, snapshots, and metrics.

Learn more in this quick, 5-minute Lightrun tutorial:

Slow MySQL query performance is all too common. Of course it is. A good way to go is, naturally, a dedicated profiler that actually understands the ins and outs of MySQL.

The Jet Profiler was built for MySQL only, so it can do things like real-time query performance, focus on most used tables or most frequent queries, quickly identify performance issues and basically help you optimize your queries.

Critically, it has very minimal impact on your server’s performance, with most of the profiling work done separately — so it needs no server changes, agents or separate services.

Basically, you install the desktop application, connect to your MySQL server, hit the record button, and you’ll have results within minutes:

DbSchema is a super-flexible database designer, which can take you from designing the DB with your team all the way to safely deploying the schema.

The way it does all of that is by using a design model, a database-independent image of the schema, which can be shared in a team using GIT and compared or deployed on to any database.

And, of course, it can be heavily visual, allowing you to interact with the database using diagrams, visually compose queries, explore the data, generate random data, import data or build HTML5 database reports.

The Kubernetes ecosystem is huge and quite complex, so it’s easy to forget about costs when trying out all of the exciting tools.

To avoid overspending on your Kubernetes cluster, definitely have a look at the free K8s cost monitoring tool from the automation platform CAST AI. You can view your costs in real time, allocate them, calculate burn rates for projects, spot anomalies or spikes, and get insightful reports you can share with your team.

Connect your cluster and start monitoring your K8s costs right away:

We’re looking for a new Java technical editor to help review new articles for the site.

1. Introduction

In this quick tutorial, we’re going to learn how to clean up the duplicate elements from a List. First, we’ll use plain Java, then Guava, and finally, a Java 8 Lambda-based solution.

This tutorial is part of the “Java – Back to Basic” series here on Baeldung.

2. Remove Duplicates From a List Using Plain Java

We can easily remove the duplicate elements from a List with the standard Java Collections Framework through a Set:

public void givenListContainsDuplicates_whenRemovingDuplicatesWithPlainJava_thenCorrect() < ListlistWithDuplicates = Lists.newArrayList(5, 0, 3, 1, 2, 3, 0, 0); List listWithoutDuplicates = new ArrayList<>( new HashSet<>(listWithDuplicates)); assertThat(listWithoutDuplicates, hasSize(5)); assertThat(listWithoutDuplicates, containsInAnyOrder(5, 0, 3, 1, 2)); >

As we can see, the original list remains unchanged.

In the example above, we used HashSet implementation, which is an unordered collection. As a result, the order of the cleaned-up listWithoutDuplicates might be different than the order of the original listWithDuplicates.

If we need to preserve the order, we can use LinkedHashSet instead:

public void givenListContainsDuplicates_whenRemovingDuplicatesPreservingOrderWithPlainJava_thenCorrect() < ListlistWithDuplicates = Lists.newArrayList(5, 0, 3, 1, 2, 3, 0, 0); List listWithoutDuplicates = new ArrayList<>( new LinkedHashSet<>(listWithDuplicates)); assertThat(listWithoutDuplicates, hasSize(5)); assertThat(listWithoutDuplicates, containsInRelativeOrder(5, 0, 3, 1, 2)); >

Further reading:

Java Collections Interview Questions

Java — Combine Multiple Collections

How to Find an Element in a List with Java

3. Remove Duplicates From a List Using Guava

We can do the same thing using Guava as well:

public void givenListContainsDuplicates_whenRemovingDuplicatesWithGuava_thenCorrect() < ListlistWithDuplicates = Lists.newArrayList(5, 0, 3, 1, 2, 3, 0, 0); List listWithoutDuplicates = Lists.newArrayList(Sets.newHashSet(listWithDuplicates)); assertThat(listWithoutDuplicates, hasSize(5)); assertThat(listWithoutDuplicates, containsInAnyOrder(5, 0, 3, 1, 2)); >

Here also, the original list remains unchanged.

Again, the order of elements in the cleaned-up list might be random.

If we use the LinkedHashSet implementation, we’ll preserve the initial order:

public void givenListContainsDuplicates_whenRemovingDuplicatesPreservingOrderWithGuava_thenCorrect() < ListlistWithDuplicates = Lists.newArrayList(5, 0, 3, 1, 2, 3, 0, 0); List listWithoutDuplicates = Lists.newArrayList(Sets.newLinkedHashSet(listWithDuplicates)); assertThat(listWithoutDuplicates, hasSize(5)); assertThat(listWithoutDuplicates, containsInRelativeOrder(5, 0, 3, 1, 2)); >

4. Remove Duplicates From a List Using Java 8 Lambdas

Finally, let’s look at a new solution, using Lambdas in Java 8. We’ll use the distinct() method from the Stream API, which returns a stream consisting of distinct elements based on the result returned by the equals() method.

Additionally, for ordered streams, the selection of distinct elements is stable. This means that for duplicated elements, the element appearing first in the encounter order is preserved:

public void givenListContainsDuplicates_whenRemovingDuplicatesWithJava8_thenCorrect() < ListlistWithDuplicates = Lists.newArrayList(5, 0, 3, 1, 2, 3, 0, 0); List listWithoutDuplicates = listWithDuplicates.stream() .distinct() .collect(Collectors.toList()); assertThat(listWithoutDuplicates, hasSize(5)); assertThat(listWithoutDuplicates, containsInAnyOrder(5, 0, 3, 1, 2)); >

There we have it, three quick ways to clean up all the duplicate items from a List.

5. Conclusion

In this article, we demonstrated how easy it is to remove duplicates from a list using plain Java, Google Guava, and Java 8.

The implementation of all of these examples and snippets can be found in the GitHub project. This is a Maven-based project, so it should be easy to import and run.

Slow MySQL query performance is all too common. Of course it is. A good way to go is, naturally, a dedicated profiler that actually understands the ins and outs of MySQL.

The Jet Profiler was built for MySQL only, so it can do things like real-time query performance, focus on most used tables or most frequent queries, quickly identify performance issues and basically help you optimize your queries.

Critically, it has very minimal impact on your server’s performance, with most of the profiling work done separately — so it needs no server changes, agents or separate services.

Basically, you install the desktop application, connect to your MySQL server, hit the record button, and you’ll have results within minutes:

Источник

Remove Duplicate Items from a List in Java

Learn to remove duplicate elements from a List in Java using Collection.removeIf(), LinkedHashSet and Stream APIs.

1. Using Collection.removeIf()

The removeIf() method removes all of the elements of this collection that satisfy a specified Predicate. Each matching element is removed using Iterator.remove(). If the collection’s iterator does not support removal, then an UnsupportedOperationException will be thrown on the first matching element.

In the following code, the predicate adds the current element to a HashSet. As HashSet does not allow duplicate items, the add() method returns false for them. All such duplicate items are removed from the List, and finally, the List contains only the unique items.

List items = new ArrayList<>(Arrays.asList(1, 1, 2, 3, 3, 3, 4, 5, 6, 6, 6, 7, 8)); Set set = new HashSet<>(items.size()); items.removeIf(p -> !set.add(p)); System.out.println(items); //[1, 2, 3, 4, 5, 6, 7, 8]

We can use the Java 8 Stream. distinct() method which returns a stream consisting of the distinct elements compared by object’s equals() method. Finally, collect all district elements as List using Collectors.toList().

ArrayList numbersList = new ArrayList<>(Arrays.asList(1, 1, 2, 3, 3, 3, 4, 5, 6, 6, 6, 7, 8)); List listWithoutDuplicates = numbersList.stream().distinct().collect(Collectors.toList()); System.out.println(listWithoutDuplicates); //[1, 2, 3, 4, 5, 6, 7, 8]

The LinkedHashSet is another good approach for removing duplicate elements in an ArrayList. LinkedHashSet does two things internally :

In the given example, items in the ArrayList contain integers; some are duplicate numbers e.g. 1, 3 and 5. We add the list to LinkedHashSet, and then get back the content back into the list. The result arraylist does not have duplicate integers.

ArrayList numbersList = new ArrayList<>(Arrays.asList(1, 1, 2, 3, 3, 3, 4, 5, 6, 6, 6, 7, 8)); LinkedHashSet hashSet = new LinkedHashSet<>(items); ArrayList listWithoutDuplicates = new ArrayList<>(hashSet); System.out.println(listWithoutDuplicates); //[1, 2, 3, 4, 5, 6, 7, 8]

Drop me your questions related to how to remove duplicate objects in arraylist in Java.

Источник