Skip to content

replace method with comma separator does not work in pandas 0.24.1 #25259

@mr-yoo

Description

@mr-yoo

The latest version (0.24.1) does not replace comma separator, but 0.23.4 does.

import pandas as pd

df = pd.DataFrame({'one': ["1,000", "2,000"], 'two': ["3,000", "4,000"]})
df = df.replace({',': ''}, regex=True)

print(pd.__version__)
print(df)
0.23.4
    one   two
0  1000  3000
1  2000  4000
0.24.1
     one    two
0  1,000  3,000
1  2,000  4,000

If other characters are used with a comma, it works normally

df = df.replace({'1,': ''}, regex=True)
     one    two
0    000  3,000
1  2,000  4,000

Activity

PetitLepton

PetitLepton commented on Feb 11, 2019

@PetitLepton
Contributor

Hi, it is not related to commas. In your example @mr-yoo, if you try to replace ,0, it also fails. The replacement works only if the pattern is the beginning of the string

df.replace({',0': ''}, regex=True)
0.23.4
   one  two
0  100  300
1  200  400
0.24.1
     one    two
0  1, 000  3,000
1  2,000  4,000

Can it be related to

op = np.vectorize(lambda x: bool(re.match(b, x)) if isinstance(x, str)
                  else False)

in managers.py?

re.match checks only if the beginning of the string matches. Should it be re.search? From my tests, it seems to reproduce the previous behaviour.

WillAyd

WillAyd commented on Feb 11, 2019

@WillAyd
Member

@PetitLepton seems logical, so maybe introduced as part of #20656

Would certainly welcome a PR to fix if you are up for one

added
RegressionFunctionality that used to work in a prior pandas version
StringsString extension data type and string data
on Feb 11, 2019
added this to the 0.24.2 milestone on Feb 11, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugRegressionFunctionality that used to work in a prior pandas versionStringsString extension data type and string data

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

      Development

      Participants

      @WillAyd@PetitLepton@mr-yoo

      Issue actions

        replace method with comma separator does not work in pandas 0.24.1 · Issue #25259 · pandas-dev/pandas