Python - Remove Columns of Duplicate Elements
Last Updated :
22 Feb, 2023
Given a Matrix, write a Python program to remove whole column if duplicate occurs in any row.
Examples:
Input : test_list = [[4, 3, 5, 2, 3], [6, 4, 2, 1, 1], [4, 3, 9, 3, 9], [5, 4, 3, 2, 1]]
Output : [[4, 3, 5], [6, 4, 2], [4, 3, 9], [5, 4, 3]]
Explanation : 1 has duplicate as next element hence 5th column is removed. 3 occurs as 2nd and 4th index, hence 4th index is removed.
Input : test_list = [[6, 4, 2, 1, 1], [4, 3, 9, 3, 9], [5, 4, 3, 2, 1]]
Output : [[6, 4, 2], [4, 3, 9], [5, 4, 3]]
Explanation : 1 has duplicate as next element hence 5th column is removed. 3 occurs as 2nd and 4th index, hence 4th index is removed.
Approach 1: Using list comprehension + set() + chain.from_iterable() + generator + loop
In this, indices which are duplicate elements are extracted as first step using generator function and set(). In the second step, all the required columns are excluded while reconstruction of Matrix.
Python3
# Python3 code to demonstrate working of
# Remove Columns of Duplicate Elements
# Using list comprehension + set() +
# chain.from_iterable() + generator + loop
from itertools import chain
def dup_idx(sub):
memo = set()
for idx, ele in enumerate(sub):
# adding element if not there
if ele not in memo:
memo.add(ele)
else:
# return index is Duplicate
yield idx
# initializing list
test_list = [[4, 3, 5, 2, 3], [6, 4, 2, 1, 1],
[4, 3, 9, 3, 9], [5, 4, 3, 2, 1]]
# printing original list
print("The original list is : " + str(test_list))
# initializing K
K = 3
# passing each row to generator function
# flattening indices at end
temp_idxs = set(chain.from_iterable(dup_idx(sub) for sub in test_list))
# extracting columns with only non-duplicate indices values
res = [[ele for idx, ele in enumerate(
sub) if idx not in temp_idxs] for sub in test_list]
# printing result
print("The filtered Matrix : " + str(res))
Output:
The original list is : [[4, 3, 5, 2, 3], [6, 4, 2, 1, 1], [4, 3, 9, 3, 9], [5, 4, 3, 2, 1]] The filtered Matrix : [[4, 3, 5], [6, 4, 2], [4, 3, 9], [5, 4, 3]]
Time Complexity: O(n*n)
Auxiliary Space: O(n*n)
Approach 2: Using operator.countOf() method
Python3
# Python3 code to demonstrate working of
# Remove Columns of Duplicate Elements
from itertools import chain
import operator as op
def dup_idx(sub):
memo = set()
for idx, ele in enumerate(sub):
# adding element if not there
if op.countOf(memo, ele) == 0:
memo.add(ele)
else:
# return index is Duplicate
yield idx
# initializing list
test_list = [[4, 3, 5, 2, 3], [6, 4, 2, 1, 1],
[4, 3, 9, 3, 9], [5, 4, 3, 2, 1]]
# printing original list
print("The original list is : " + str(test_list))
# initializing K
K = 3
# passing each row to generator function
# flattening indices at end
temp_idxs = set(chain.from_iterable(dup_idx(sub) for sub in test_list))
# extracting columns with only non-duplicate indices values
res = [[ele for idx, ele in enumerate(
sub) if op.countOf(temp_idxs, idx) == 0] for sub in test_list]
# printing result
print("The filtered Matrix : " + str(res))
OutputThe original list is : [[4, 3, 5, 2, 3], [6, 4, 2, 1, 1], [4, 3, 9, 3, 9], [5, 4, 3, 2, 1]]
The filtered Matrix : [[4, 3, 5], [6, 4, 2], [4, 3, 9], [5, 4, 3]]
Time Complexity:O(N*N)
Auxiliary Space: O(N*N)
Similar Reads
Python | Remove duplicates based on Kth element tuple list Sometimes, while working with records, we may have a problem in which we need to remove duplicates based on Kth element of a tuple in the list. This problem has application in domains that uses records as input. Let's discuss certain ways in which this problem can be solved. Method #1: Using loop Th
8 min read
Python | Remove Initial K column elements Sometimes, while working with Matrix data, we can have stray elements that attached at front end of each row of matrix. This can be undesired at times and wished to be removed. Letâs discuss certain ways in which this task can be performed. Method #1 : Using loop + del + list slicing The combination
4 min read
Remove Unordered Duplicate Elements from a List - Python Given a list of elements, the task is to remove all duplicate elements from the list while maintaining the original order of the elements.For example, if the input is [1, 2, 2, 3, 1] the expected output is [1, 2, 3]. Let's explore various methods to achieve this in Python.Using setWe can initialize
3 min read
Python - Remove Duplicates from a List Removing duplicates from a list is a common operation in Python which is useful in scenarios where unique elements are required. Python provides multiple methods to achieve this. Using set() method is most efficient for unordered lists. Converting the list to a set removes all duplicates since sets
2 min read
Python | Remove duplicates in Matrix While working with Python Matrix, we can face a problem in which we need to perform the removal of duplicates from Matrix. This problem can occur in Machine Learning domain because of extensive usage of matrices. Let's discuss certain way in which this task can be performed. Method : Using loop This
2 min read