Skip to content

BUG: max()/min() don't work on DataFrames constructed from scipy sparse matrix #40921

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
melkonyan opened this issue Apr 13, 2021 · 6 comments · Fixed by #41159
Closed

BUG: max()/min() don't work on DataFrames constructed from scipy sparse matrix #40921

melkonyan opened this issue Apr 13, 2021 · 6 comments · Fixed by #41159
Labels
Bug Reduction Operations sum, mean, min, max, etc. Sparse Sparse Data Type
Milestone

Comments

@melkonyan
Copy link

melkonyan commented Apr 13, 2021

Example:

import pandas as pd
from scipy import sparse as sc
import numpy as np

np.random.seed(42)
a = np.random.randint(0, 10, size=(200, 5))
a_sp = sc.coo_matrix(a)
a_sp_pd = pd.DataFrame.sparse.from_spmatrix(a_sp)
a_sp_pd.max()

Expected output:

0    9
1    9
2    9
3    9
4    9
dtype: int64

Actual output:

Series([], dtype: float64)

System info:

  - numpy [required: >=1.17, installed: 1.20.2]
  - scipy [required: >=0.19.1, installed: 1.6.2]
  - pandas [required: >=1.0.0,<=1.1.4, installed: 1.2.4]
@melkonyan melkonyan added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Apr 13, 2021
@phofl
Copy link
Member

phofl commented Apr 13, 2021

This raises

Traceback (most recent call last):
  File "/home/developer/.local/share/JetBrains/Toolbox/apps/PyCharm-P/ch-0/211.6693.115/plugins/python/helpers/pydev/pydevd.py", line 1483, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/home/developer/.local/share/JetBrains/Toolbox/apps/PyCharm-P/ch-0/211.6693.115/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/developer/.config/JetBrains/PyCharm2021.1/scratches/scratch_4.py", line 527, in <module>
    a_sp = sc.sparse.coo_matrix(a)
AttributeError: module 'scipy' has no attribute 'sparse'

Process finished with exit code 1

for me. Could you check your example?

@melkonyan
Copy link
Author

My bad, updated

@phofl
Copy link
Member

phofl commented Apr 13, 2021

This is not implemented. The error is caught in the call stack and an empty result is returned.

@phofl phofl added Sparse Sparse Data Type Reduction Operations sum, mean, min, max, etc. and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Apr 13, 2021
@taytzehao
Copy link
Contributor

take

@taytzehao
Copy link
Contributor

I will be adding a max & min method pandas\core\arrays\sparse\array.py in the sparsearray class

@taytzehao
Copy link
Contributor

Take

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Reduction Operations sum, mean, min, max, etc. Sparse Sparse Data Type
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants