[#111472] [Ruby master Bug#19274] Error installing ruby 3.2.0 on RH 8 — "aalllop (Alberto Allegue) via ruby-core" <ruby-core@...>
Issue #19274 has been reported by aalllop (Alberto Allegue).
5 messages
2022/12/28
[#111508] Data support for versions before 3.2.0 — Eustáquio Rangel via ruby-core <ruby-core@...>
I was wondering that every piece of code (gems, etc) that use the new Data =
3 messages
2022/12/29
[ruby-core:111255] [Ruby master Bug#19192] IO has third data mode, document is incomplete.
From:
"alanwu (Alan Wu)" <noreply@...>
Date:
2022-12-11 20:41:10 UTC
List:
ruby-core #111255
Issue #19192 has been updated by alanwu (Alan Wu). Ugh, it's quite weird. The `:crlf_newline` option is an [encoding option](https://docs.ruby-lang.org/en/master/encodings_rdoc.html#label-Encoding+Options), but `File.open` uses it in a way different from `String#encode`. With `String#encode`, it replaces `\n` with `\r\n`: ```ruby p "\n".encode(crlf_newline: true) # => "\r\n" ``` With `File.open`, it controls DOS TEXT mode, which is Windows-specific and does the inverse conversion when reading. Also, when used in conjunction with encoding conversion, it doesn't do any newline conversion on Windows: ```ruby require 'tempfile' content = "\x1a \r \r\n \n".freeze tmp = Tempfile.new.binmode tmp.write(content) tmp.flush File.open(tmp.path, "r:US-ASCII:UTF-8", crlf_newline: true) do read_content = _1.read p content p read_content p content == read_content end __END__ F:\> ruby.exe -v .\19192.rb ruby 3.2.0dev (2022-12-09T14:34:17Z master 12b5268679) [x64-mswin64_140] "\u001A \r \r\n \n" "\u001A \r \r\n \n" true ``` There seems to be no way to get just CRLF conversion without also getting special treatment for `0x1A`. This is probably because Ruby has to rely on the Windows system library to do the conversion. The `:universal_newline` option is built into Ruby so it works cross-platform. ---------------------------------------- Bug #19192: IO has third data mode, document is incomplete. https://bugs.ruby-lang.org/issues/19192#change-100549 * Author: YO4 (Yoshinao Muramatsu) * Status: Open * Priority: Normal * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN ---------------------------------------- The documentation on the mode parameter of File.open is incomplete, I would like to clarify IO's data mode actual behavior here. document says ``` To specify whether data is to be treated as text or as binary data, either of the following may be suffixed to any of the string read/write modes above: 't': Text data; sets the default external encoding to Encoding::UTF_8; on Windows, enables conversion between EOL and CRLF and enables interpreting 0x1A as an end-of-file marker. 'b': Binary data; sets the default external encoding to Encoding::ASCII_8BIT; on Windows, suppresses conversion between EOL and CRLF and disables interpreting 0x1A as an end-of-file marker. If neither is given, the stream defaults to text data. ``` But actually it's more complicated than that. There is three Data Mode * text mode. Can convert encoding and newline. * binary mode. Cannot convert encoding nor newline. Encoding is treated as Encoding::ASCII_8BIT. * third mode: DOS TEXT mode. That enables conversion between EOL and CRLF and enables interpreting 0x1A as an end-of-file marker. On Windows platform 't' textmode with universal newline conversion. 'b' binary mode. If neither is given, DOS TEXT mode. On other platforms 't' textmode with universal newline conversion. 'b' binary mode. If neither is given, textmode without newline conversion. On Windows, there are some special cases. If Encoding conversion is specified, DOS TEXT mode is ignored and universal newline conversion applied. If access mode is "a+", last (only one) EOF charactor is overwritten when DOS TEXT mode. There are more parameter combinations, see https://gist.github.com/YO4/262e9bd5e44a37a7a2fa9118e271b30b Is this all? I have not fully investigated. Since the topic of data mode spanned access mode and encoding conversion, I don't think my English skills will allow me to summarize this into rdoc without breaking something... -- https://bugs.ruby-lang.org/ ______________________________________________ ruby-core mailing list -- [email protected] To unsubscribe send an email to [email protected] ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/