Series.asof not preserving dtype

[fabrice-deschatres]

Code Sample, a copy-pastable example if possible

from pandas import Series
from datetime import date

if __name__ == "__main__":
    series  = Series([True, False, True], index=[date(2017,1,1), date(2017,2,1), date(2017,3,1)])

    print series.dtype, series.asof([date(2017,4,1)]).dtype

Problem description

With Pandas 0.19.2, I found that dtype was not preserved with asof (bool -> float64), whereas older version of Pandas (ex: version 0.18.0), was preserving bool type

Expected Output

bool, bool

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 2.7.12.final.0 python-bits: 64 OS: Windows OS-release: 7 machine: AMD64 processor: Intel64 Family 6 Model 44 Stepping 2, GenuineIntel byteorder: little LC_ALL: None LANG: None LOCALE: None.None

pandas: 0.19.2
nose: 1.3.7
pip: 8.1.2
setuptools: 23.0.0
Cython: 0.23.4
numpy: 1.12.1
scipy: 0.19.0
statsmodels: 0.8.0
xarray: None
IPython: 4.0.1
sphinx: None
patsy: 0.4.1
dateutil: 2.5.3
pytz: 2016.4
blosc: None
bottleneck: None
tables: None
numexpr: 2.6.2
matplotlib: 2.0.0
openpyxl: 2.3.4
xlrd: None
xlwt: None
xlsxwriter: None
lxml: 3.6.1
bs4: None
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.8
boto: None
pandas_datareader: None
None

[jreback]

so this works if the incoming is already object dtype, IOW you have NaN with a non-native dtype (like bool).

bool with-nan is not a first class type, so not surpising this coerces.

In [13]:     series  = Series([True, False, np.nan, True], index=[date(2017,1,1), date(2017,2,1), date(2017, 2, 2), date(2017,3,1)])

In [14]: series
Out[14]: 
2017-01-01     True
2017-02-01    False
2017-02-02      NaN
2017-03-01     True
dtype: object

In [15]: series.asof([date(2017,4,1)])
Out[15]: 
2017-04-01    True
dtype: object

[fabrice-deschatres]

Yes, this is the right output:

In [15]: series.asof([date(2017,4,1)])
Out[15]:
2017-04-01 True
dtype: bool

... and that was used to be returned by pandas before version 0.19 (ex: version 0.18.0)

Why would you expect an empty series? The last good value of the original series is True (for 2017/3/1).

[fabrice-deschatres]

Ok, I don't mind deleting my reply too