Fix CRLF -> LF conversion on read for rb_io_fdopen & rb_file_open
When opening a file with File.open, and then setting the encoding with IO#set_encoding, it still correctly performs CRLF -> LF conversion on
Windows when reading files with a CRLF line ending in them (in text
mode).
However, the file is opened instead with either the rb_io_fdopen or rb_file_open APIs from C, the CRLF conversion is NOT set up
correctly; it works if the encoding is not specified, but if IO#set_encoding is called, the conversion stops happening. This seems
to be because the encflags never get ECONV_DEFAULT_NEWLINE_DECORATOR
set in these codepaths.
Concretely, this means that the conversion doesn't happen in the
following circumstances:
When loading ruby files with require (that calls rb_io_fdopen)
When parsing ruuby files with RubyVM::AbstractSyntaxTree (that calls
rb_file_open).
This then causes the ErrorHighlight tests to fail on windows if git has
checked them out with CRLF line endings - the error messages it's
testing wind up with literal \r\n sequences in them because the iseq
text from the parser contains un-newline-converted strings.
This commit fixes the problem by copy-pasting the relevant snippet which
sets this up in rb_io_extract_modeenc (for the File.open path) into
the relevant codepaths for rb_io_fdopen and rb_file_open.
Fix CRLF -> LF conversion on read for rb_io_fdopen & rb_file_open
When opening a file with
File.open, and then setting the encoding withIO#set_encoding, it still correctly performs CRLF -> LF conversion onWindows when reading files with a CRLF line ending in them (in text
mode).
However, the file is opened instead with either the
rb_io_fdopenorrb_file_openAPIs from C, the CRLF conversion is NOT set upcorrectly; it works if the encoding is not specified, but if
IO#set_encodingis called, the conversion stops happening. This seemsto be because the encflags never get ECONV_DEFAULT_NEWLINE_DECORATOR
set in these codepaths.
Concretely, this means that the conversion doesn't happen in the
following circumstances:
rb_file_open).
This then causes the ErrorHighlight tests to fail on windows if git has
checked them out with CRLF line endings - the error messages it's
testing wind up with literal \r\n sequences in them because the iseq
text from the parser contains un-newline-converted strings.
This commit fixes the problem by copy-pasting the relevant snippet which
sets this up in
rb_io_extract_modeenc(for the File.open path) intothe relevant codepaths for
rb_io_fdopenandrb_file_open.[Bug #20101]