Refining module fails with `--format` and `--require spec_helper` on Ruby 2.7

[parkerfinch]

Refining module fails with --format and --require spec_helper on Ruby 2.7

I am attempting to test that a refinement to a module works as expected, but the refinement is seemingly not activated when using rspec with the --format and --require options on Ruby 2.7.1.

Your environment

Failing

• Ruby version: 2.7.1
• rspec-core version: 3.9.2

Succeeding

• Ruby version: 2.6.6
• rspec-core version: 3.9.2

Succeeding

• Ruby version 2.5.8
• rspec-core version: rspec-core 3.9.2

Steps to reproduce

I apologize for not including a single executable that can be run with ruby; this bug manifests only when using --format and --require options and I don't know how include those rspec options using autorun as the report template suggests. If there's a way to get this into a single executable then please let me know!

These files describe a minimal example, here's a tarfile with this structure:
minimal_refinement_failure.tar.gz.

# file: .rspec

--format documentation
--require spec_helper

# file: refinement_bug.rb
module RefinementBug
  refine Enumerable do
    def refined_method
      puts "Called #refined_method"
    end
  end
end

# file: spec/spec_helper.rb
require_relative "../refinement_bug"

# file: spec/test.rb

require_relative "../refinement_bug"

using RefinementBug

puts "Using Ruby #{RUBY_VERSION}"
[].refined_method

To reproduce, either create these files manually or download that tar file and run tar -xzf minimal_refinement_failure.tar.gz, which will create these in a directory named minimal_refinement_failure.

Expected behavior

# This is output when running on ruby 2.6.6, and is expected.
~/minimal_refinement_failure> rspec spec/test.rb
Using Ruby 2.6.6
Called #refined_method
No examples found.

Finished in 0.0004 seconds (files took 0.08706 seconds to load)
0 examples, 0 failures

Actual behavior

# This is the output when running on ruby 2.7.1.
~/minimal_refinement_failure> rspec spec/test.rb
Using Ruby 2.7.1

An error occurred while loading ./spec/test.rb.
Failure/Error: [].refined_method

NoMethodError:
  undefined method `refined_method' for []:Array
# ./spec/test.rb:8:in `<top (required)>'
No examples found.

Finished in 0.00003 seconds (files took 0.12287 seconds to load)
0 examples, 0 failures, 1 error occurred outside of examples

Changes to example that result in expected behavior

Doing any of the following results in the expected output when running on ruby 2.7:

1. Instead of refining Enumerable, refine Array directly. This suggests that it's an issue only with refining modules, not classes.
2. Removing the --require spec_helper from the .rspec file. (Or removing the require_relative "../refinement_bug" line from the spec_helper.rb file.) This suggests that it's an issue with how files are required by the spec helper, since requiring the file directly in the spec succeeds.
3. Removing the --format documentation from the .rspec file. I do not know why this would impact it, but it succeeds after doing this. (Notably, replacing with --format progress also causes a failure -- it seems like the presence of the --format flag causes the issue to manifest, rather than a specific value of --format.)

Output after doing any of the above:

~/minimal_refinement_failure> rspec spec/test.rb
Using Ruby 2.7.1
Called #refined_method
No examples found.

Finished in 0.00032 seconds (files took 0.06598 seconds to load)
0 examples, 0 failures

Ruby issue

Since it seems like this might involve an unintended change in behavior from ruby 2.6 to 2.7, I'll create an issue in the ruby bug tracker I created an issue in the ruby bug tracker here.

Let me know if I can provide any more information!

[pirj]

Wow, this would make an Interesting investigation.
Also changing refine Enumerable to refine Array makes the test pass.
--format progress also breaks the test.

I would say if it works in pre-2.7 and broke in 2.7, it's a Ruby bug. Don't think we're able to work this around.
Please post a link to a Ruby ticket.

[JonRowe]

This doesn't break if you just do ruby -r spec/spec_helper.rb spec/test.rb so I'd like to find out exactly how we break the refinement. We do have other workarounds for such Ruby features as the behaviour does change between versions.

So I'd like to keep this open until we've determined what, if anything we can do.

[parkerfinch]

A quick update on this: after poking around a little more I was surprised to find that this works if I refine Kernel. So far the only case I've found where this breaks is refining Enumerable.

@pirj I just created a ruby ticket here! And thanks for reopening @JonRowe -- this one is confusing and I'm not sure if it's a ruby issue or an rspec issue. The fact that bumping from ruby 2.6 to 2.7 causes this makes me think it's an underlying ruby issue, but the fact that removing the --format flag fixes this issue makes me think it's something with how rspec is handling the refinements.

[parkerfinch]

Update! This is reproducible without rspec, and it's been determined that the reason rspec hits this error is that it requires "stringio" which includes Enumerable. Definitely an underlying ruby issue though.

Thank you for looking at this, I'll close it out in favor of the ruby issue.

[JonRowe]

Great, I suspected it'd be something along those lines, thanks for coming back and closing this!