[#118346] [Ruby master Bug#20586] Some filesystem calls in dir.c are missing error handling and can return incorrect results if interrupted — "ivoanjo (Ivo Anjo) via ruby-core" <ruby-core@...>
Issue #20586 has been reported by ivoanjo (Ivo Anjo).
13 messages
2024/06/19
[ruby-core:118197] [Ruby master Misc#20407] Question about applying encoding modifier to an interpolated Regexp
From:
"nobu (Nobuyoshi Nakada) via ruby-core" <ruby-core@...>
Date:
2024-06-06 08:22:52 UTC
List:
ruby-core #118197
Issue #20407 has been updated by nobu (Nobuyoshi Nakada).
I think:
1. If a `Regexp` source string contains non-US-ASCII chars, the source stri=
ng encoding is honored.
2. If the source string contains US-ASCII chars only, falls back to
a. an encoding option if given.
b. US-ASCII.
----------------------------------------
Misc #20407: Question about applying encoding modifier to an interpolated R=
egexp
https://bugs.ruby-lang.org/issues/20407#change-108684
* Author: andrykonchin (Andrew Konchin)
* Status: Open
----------------------------------------
I am wondering how Regexp encoding modifiers (u, s, e, n) interfere in enco=
ding negotiation of parts/fragments in an interpolated Regexp literal.
Examples #1
```ruby
# encoding: us-ascii
# Unicode: =D0=A4 - U+0424
# windows-1251: =D0=A4 - 0xD4
# without encoding modifier
puts /a #{ "\xd4".force_encoding("windows-1251") } c/.encoding # Windows=
-1251
puts /a #{ "b".encode("windows-1251") } c/.encoding # US-ASCI=
I
puts /a #{ "\u0424".force_encoding("UTF-8") } c/.encoding # UTF-8
puts /a #{ "\xc2\xa1".b } c/.encoding # ASCII-8=
BIT
# with encoding modifier
puts /a #{ "\xd4".force_encoding("windows-1251") } c/e.encoding # Windows=
-1251
puts /a #{ "b".encode("windows-1251") } c/e.encoding # EUC-JP
puts /a #{ "\u0424".force_encoding("UTF-8") } c/e.encoding # UTF-8
puts /a #{ "\xc2\xa1".b } c/e.encoding # ASCII-8=
BIT
# string interpolation
puts "a #{ "\xd4".force_encoding("windows-1251") } c".encoding # Windows=
-1251
puts "a #{ "b".encode("windows-1251") } c".encoding # Windows=
-1251
puts "a #{ "\u0424".force_encoding("UTF-8") } c".encoding # UTF-8
puts "a #{ "\xc2\xa1".b } c".encoding # ASCII-8=
BIT
```
Example #2
```ruby
# encoding: utf-8
# windows-1251: =D0=A4 - 0xD4
# unicode: =D0=A4 - U+0424
# without encoding modifier
puts /a #{ "\xd4".force_encoding("windows-1251") } c/.encoding # Windows=
-1251
puts /a #{ "b".encode("windows-1251") } c/.encoding # US-ASCI=
I
puts /a #{ "\u0424".force_encoding("UTF-8") } c/.encoding # UTF-8
puts /a #{ "\xc2\xa1".b } c/.encoding # ASCII-8=
BIT
# with encoding modifier
puts /a #{ "\xd4".force_encoding("windows-1251") } c/e.encoding # Windows=
-1251
puts /a #{ "b".encode("windows-1251") } c/e.encoding # EUC-JP
puts /a #{ "\u0424".force_encoding("UTF-8") } c/e.encoding # UTF-8
puts /a #{ "\xc2\xa1".b } c/e.encoding # ASCII-8=
BIT
# string interpolation
puts "a #{ "\xd4".force_encoding("windows-1251") } c".encoding # Windows=
-1251
puts "a #{ "b".encode("windows-1251") } c".encoding # UTF-8
puts "a #{ "\u0424".force_encoding("UTF-8") } c".encoding # UTF-8
puts "a #{ "\xc2\xa1".b } c".encoding # ASCII-8=
BIT
```
In the examples above the `e` modifier changes Regexp's encoding only in on=
e case when Regexp's encoding would be `US-ASCII` without the modifier:
```ruby
# encoding: us-ascii
puts /a #{ "b".encode("windows-1251") } c/.encoding =
# US-ASCII
puts /a #{ "b".encode("windows-1251") } c/e.encoding =
# EUC-JP
```
```ruby
# encoding: utf-8
puts /a #{ "b".encode("windows-1251") } c/.encoding =
# US-ASCII
puts /a #{ "b".encode("windows-1251") } c/e.encoding =
# EUC-JP
```
And the `e` modifier doesn't change Regexp's final encoding in all the othe=
r cases either Regexp's encoding without modifier is a file source encoding=
or `ASCII-8BIT`.
Looking at the following example:
```ruby
# encoding: us-ascii
# without modifier
p /\xc2\xa1 #{ "a" }\xc2\xa1/.encoding # AS=
CII-8BIT
p /a #{ "\xc2\xa1".force_encoding("EUC-JP") } b/.encoding # EU=
C-JP
p /a #{ "\xc2\xa1".b } b/.encoding # AS=
CII-8BIT
# with modifier
p /\xc2\xa1 #{ "a" }\xc2\xa1/e.encoding # EU=
C-JP
p /a #{ "\xc2\xa1".force_encoding("EUC-JP") } b/e.encoding # EU=
C-JP
p /a #{ "\xc2\xa1".b } b/e.encoding # AS=
CII-8BIT
```
we can notice that the `e` modifier changes `ASCII-8BIT` to `EUC-JP` in the=
first case (`/\xc2\xa1 #{ "a" }\xc2\xa1/`) but doesn't in the third one (`=
/a #{ "\xc2\xa1".b } b/`). So I assume that the `e` modifier could be appli=
ed to the Regexp fragments (`\xc2\xa1` and `\xc2\xa1`) before encoding nego=
tiation and not to the whole result after negotiation.
Could you please clarify how it works?
--=20
https://bugs.ruby-lang.org/