Formatting Exception in io/formats/terminal.py

[MTKnife]

The Code

from pandas import DataFrame
df = DataFrame([[1,2,3], [4,5,6]])
print df

What I get when I try to print "df":

Traceback (most recent call last):

  File "C:\Anaconda2\lib\site-packages\IPython\core\formatters.py", line 699, in __call__
    printer.pretty(obj)

  File "C:\Anaconda2\lib\site-packages\IPython\lib\pretty.py", line 403, in pretty
    return _repr_pprint(obj, self, cycle)

  File "C:\Anaconda2\lib\site-packages\IPython\lib\pretty.py", line 703, in _repr_pprint
    output = repr(obj)

  File "C:\Anaconda2\lib\site-packages\pandas\core\base.py", line 78, in __repr__
    return str(self)

  File "C:\Anaconda2\lib\site-packages\pandas\core\base.py", line 58, in __str__
    return self.__bytes__()

  File "C:\Anaconda2\lib\site-packages\pandas\core\base.py", line 70, in __bytes__
    return self.__unicode__().encode(encoding, 'replace')

  File "C:\Anaconda2\lib\site-packages\pandas\core\frame.py", line 633, in __unicode__
    line_width=width, show_dimensions=show_dimensions)

  File "C:\Anaconda2\lib\site-packages\pandas\core\frame.py", line 713, in to_string
    line_width=line_width)

  File "C:\Anaconda2\lib\site-packages\pandas\io\formats\format.py", line 428, in __init__
    self._chk_truncate()

  File "C:\Anaconda2\lib\site-packages\pandas\io\formats\format.py", line 444, in _chk_truncate
    (w, h) = get_terminal_size()

  File "C:\Anaconda2\lib\site-packages\pandas\io\formats\terminal.py", line 41, in get_terminal_size
    tuple_xy = _get_terminal_size_tput()

  File "C:\Anaconda2\lib\site-packages\pandas\io\formats\terminal.py", line 103, in _get_terminal_size_tput
    cols = int(output[0])

ValueError: invalid literal for int() with base 10: ''

Traceback (most recent call last):

  File "C:\Anaconda2\lib\site-packages\IPython\core\formatters.py", line 338, in __call__
    return method()

  File "C:\Anaconda2\lib\site-packages\pandas\core\frame.py", line 667, in _repr_html_
    show_dimensions=show_dimensions, notebook=True)

  File "C:\Anaconda2\lib\site-packages\pandas\core\frame.py", line 2253, in to_html
    render_links=render_links)

  File "C:\Anaconda2\lib\site-packages\pandas\io\formats\format.py", line 428, in __init__
    self._chk_truncate()

  File "C:\Anaconda2\lib\site-packages\pandas\io\formats\format.py", line 444, in _chk_truncate
    (w, h) = get_terminal_size()

  File "C:\Anaconda2\lib\site-packages\pandas\io\formats\terminal.py", line 41, in get_terminal_size
    tuple_xy = _get_terminal_size_tput()

  File "C:\Anaconda2\lib\site-packages\pandas\io\formats\terminal.py", line 103, in _get_terminal_size_tput
    cols = int(output[0])

ValueError: invalid literal for int() with base 10: ''

Problem description

I get this exception when running the code quoted above in the iPython console in the latest version of Anaconda Spyder (3.3.2); the Python version is Anaconda 2.7.15.

Apparently, the ValueError has existed all along, but it was previously masked by the try/except block that wraps line 103 in terminal.py.

The function in 0.23 looked like this:

def _get_terminal_size_tput():
    # get terminal width
    # src: http://stackoverflow.com/questions/263890/how-do-i-find-the-width
    # -height-of-a-terminal-window
    try:
        import subprocess
        proc = subprocess.Popen(["tput", "cols"],
                                stdin=subprocess.PIPE,
                                stdout=subprocess.PIPE)
        output = proc.communicate(input=None)
        cols = int(output[0])
        proc = subprocess.Popen(["tput", "lines"],
                                stdin=subprocess.PIPE,
                                stdout=subprocess.PIPE)
        output = proc.communicate(input=None)
        rows = int(output[0])
        return (cols, rows)
    except:
return None

In 0.24, it looks like this:

def _get_terminal_size_tput():
    # get terminal width
    # src: http://stackoverflow.com/questions/263890/how-do-i-find-the-width
    # -height-of-a-terminal-window
    try:
        import subprocess
        proc = subprocess.Popen(["tput", "cols"],
                                stdin=subprocess.PIPE,
                                stdout=subprocess.PIPE)
        output = proc.communicate(input=None)
        cols = int(output[0])
        proc = subprocess.Popen(["tput", "lines"],
                                stdin=subprocess.PIPE,
                                stdout=subprocess.PIPE)
        output = proc.communicate(input=None)
        rows = int(output[0])
        return (cols, rows)
    except OSError:
        return None

The change here is in the "except" line: it's gone from a blanket "except" (which is, admittedly, bad practice), to an "OSError". I believe the "except" needs to look like this:

    except (OSError, ValueError):

Yes, the underlying issue will still be there (it always has been), but nonetheless it displays just fine in my Spyder console window. I'd submit a PR myself, but I'm not confident I know enough about the pandas codebase that my fix is what's needed.

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 2.7.15.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 94 Stepping 3, GenuineIntel
byteorder: little
LC_ALL: None
LANG: en
LOCALE: None.None

pandas: 0.24.0
pytest: 4.1.1
pip: 18.1
setuptools: 40.6.3
Cython: 0.29.2
numpy: 1.15.4
scipy: 1.2.0
pyarrow: None
xarray: None
IPython: 5.8.0
sphinx: 1.8.2
patsy: 0.5.1
dateutil: 2.7.5
pytz: 2018.9
blosc: None
bottleneck: 1.2.1
tables: 3.4.4
numexpr: 2.6.9
feather: None
matplotlib: 2.2.3
openpyxl: 2.5.12
xlrd: 1.2.0
xlwt: 1.3.0
xlsxwriter: 1.1.2
lxml.etree: 4.3.0
bs4: 4.7.1
html5lib: 1.0.1
sqlalchemy: 1.2.16
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
gcsfs: None

[TomAugspurger]

Rather than catching OSError, ValueError I think we should

1. Do the subprocess stuff in a try / except OSError
2. Do the parsing (cols = int(output[0]), etc.) in a try / except ValueError

[EternalLearner42]

Seems interesting. I'm a bit of a newbie to open source, but I wouldn't mind taking a look at it. Do I have your go-ahead?

[paplio]

Hi! I'm new to open-source as well but I think I could handle this well. Saw the issue was still open, and was wondering if you would accept a PR?

[gfyoung]

@paplio : Thanks for your interest! I think we already have a PR in the works to address this (see #25088). That being said, you are more than welcome to look at any of our other open issues (those tagged with "good first issue" are definitely good ones to start with)!