[#88240] [Ruby trunk Feature#14759] [PATCH] set M_ARENA_MAX for glibc malloc — sam.saffron@...
Issue #14759 has been updated by sam.saffron (Sam Saffron).
3 messages
2018/08/01
[#88251] Re: [ruby-alerts:8236] failure alert on trunk@P895 (NG (r64134)) — Eric Wong <normalperson@...>
[email protected] wrote:
3 messages
2018/08/01
[#88305] [Ruby trunk Bug#14968] [PATCH] io.c: make all pipes nonblocking by default — normalperson@...
Issue #14968 has been reported by normalperson (Eric Wong).
4 messages
2018/08/06
[#88331] [Ruby trunk Feature#13618] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid — samuel@...
Issue #13618 has been updated by ioquatix (Samuel Williams).
3 messages
2018/08/08
[#88342] [Ruby trunk Feature#14955] [PATCH] gc.c: use MADV_FREE to release most of the heap page body — ko1@...
Issue #14955 has been updated by ko1 (Koichi Sasada).
3 messages
2018/08/08
[#88433] [Ruby trunk Feature#13618] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid — ko1@...
SXNzdWUgIzEzNjE4IGhhcyBiZWVuIHVwZGF0ZWQgYnkga28xIChLb2ljaGkgU2FzYWRhKS4KCgpX
4 messages
2018/08/10
[#88467] Re: [Ruby trunk Feature#13618] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid
— Eric Wong <normalperson@...>
2018/08/14
a28xQGF0ZG90Lm5ldCB3cm90ZToKPiBJc3N1ZSAjMTM2MTggaGFzIGJlZW4gdXBkYXRlZCBieSBr
[#88475] [Ruby trunk Misc#14937] [PATCH] thread_pthread: lazy-spawn timer-thread only on contention — ko1@...
Issue #14937 has been updated by ko1 (Koichi Sasada).
3 messages
2018/08/14
[#88491] Re: [ruby-cvs:71466] k0kubun:r64374 (trunk): test_function.rb: skip running test — Eric Wong <normalperson@...>
[email protected] wrote:
5 messages
2018/08/15
[#88495] Re: [ruby-cvs:71466] k0kubun:r64374 (trunk): test_function.rb: skip running test
— Takashi Kokubun <takashikkbn@...>
2018/08/15
SSBzZWUuIFBsZWFzZSByZW1vdmUgdGhlIHRlc3QgaWYgdGhlIHRlc3QgaXMgdW5uZWNlc3Nhcnku
[#88496] Re: [ruby-cvs:71466] k0kubun:r64374 (trunk): test_function.rb: skip running test
— Eric Wong <normalperson@...>
2018/08/15
Takashi Kokubun <[email protected]> wrote:
[#88523] [Ruby trunk Bug#14999] ConditionVariable doesn't reacquire the Mutex if Thread#kill-ed — eregontp@...
Issue #14999 has been updated by Eregon (Benoit Daloze).
4 messages
2018/08/17
[#88524] Re: [Ruby trunk Bug#14999] ConditionVariable doesn't reacquire the Mutex if Thread#kill-ed
— Eric Wong <normalperson@...>
2018/08/18
[email protected] wrote:
[#88549] [Ruby trunk Bug#14999] ConditionVariable doesn't reacquire the Mutex if Thread#kill-ed — eregontp@...
Issue #14999 has been updated by Eregon (Benoit Daloze).
3 messages
2018/08/19
[#88676] [Ruby trunk Misc#15014] thread.c: use rb_hrtime_scalar for high-resolution time operations — ko1@...
Issue #15014 has been updated by ko1 (Koichi Sasada).
5 messages
2018/08/27
[#88677] Re: [Ruby trunk Misc#15014] thread.c: use rb_hrtime_scalar for high-resolution time operations
— Eric Wong <normalperson@...>
2018/08/27
[email protected] wrote:
[#88678] Re: [Ruby trunk Misc#15014] thread.c: use rb_hrtime_scalar for high-resolution time operations
— Koichi Sasada <ko1@...>
2018/08/27
On 2018/08/27 16:16, Eric Wong wrote:
[#88716] Re: [ruby-dev:43715] [Ruby 1.9 - Bug #595] Fiber ignores ensure clause — Eric Wong <normalperson@...>
Koichi Sasada wrote:
4 messages
2018/08/29
[#88723] [Ruby trunk Bug#15041] [PATCH] cont.c: set th->root_fiber to current fiber at fork — ko1@...
Issue #15041 has been updated by ko1 (Koichi Sasada).
3 messages
2018/08/29
[#88767] [Ruby trunk Bug#15050] GC after forking with fibers crashes — ko1@...
Issue #15050 has been updated by ko1 (Koichi Sasada).
7 messages
2018/08/31
[#88769] Re: [Ruby trunk Bug#15050] GC after forking with fibers crashes
— Eric Wong <normalperson@...>
2018/08/31
Koichi Sasada <[email protected]> wrote:
[#88770] Re: [Ruby trunk Bug#15050] GC after forking with fibers crashes
— Koichi Sasada <ko1@...>
2018/08/31
[#88773] Re: [Ruby trunk Bug#15050] GC after forking with fibers crashes
— Eric Wong <normalperson@...>
2018/08/31
Koichi Sasada <[email protected]> wrote:
[#88774] Re: [ruby-alerts:8955] failure alert on trunk@P895 (NG (r64594)) — Eric Wong <normalperson@...>
[email protected] wrote:
3 messages
2018/08/31
[ruby-core:88661] [Ruby trunk Feature#15022] Oneshot coverage
From:
samuel@...
Date:
2018-08-27 04:23:59 UTC
List:
ruby-core #88661
Issue #15022 has been updated by ioquatix (Samuel Williams).
Did you take a look at https://github.com/ioquatix/covered - I'd be interested in your feedback. Can we discuss further, and also how to improve performance? There is some more discussion here: https://bugs.ruby-lang.org/issues/14888
----------------------------------------
Feature #15022: Oneshot coverage
https://bugs.ruby-lang.org/issues/15022#change-73718
* Author: mame (Yusuke Endoh)
* Status: Assigned
* Priority: Normal
* Assignee: mame (Yusuke Endoh)
* Target version: 2.6
----------------------------------------
I'd like to introduce a new feature to the coverage library, namely, "oneshot coverage".
## Synopsis
The following is a sample target program "test-cov.rb":
```
1: def foo
2: :foo
3: end
4:
5: def bar
6: :bar
7: end
```
The following program measures line coverage with oneshot mode:
```ruby
require "coverage"
# Start the measurement of line coverage with oneshot mode
Coverage.start(oneshot_lines: true)
# Load the target file
load "test-cov.rb"
# Get all executed lines so far:
p Coverage.peek_result #=> {"/.../test-cov.rb"=>{:oneshot_lines=>[1, 5]}}
# This means that Lines 1 (def foo) and 5 (def bar) were executed
# Clear the counters
Coverage.clear
# Run one of the target functions:
foo()
# Get newly executed lines:
p Coverage.peek_result #=> {"/.../test-cov.rb"=>{:oneshot_lines=>[2]}}
# This means Line 2 (the body of foo) was newly executed
# Clear the counters
Coverage.clear
# Run the other target function:
bar()
# Get newly executed lines:
p Coverage.peek_result #=> {"/.../test-cov.rb"=>{:oneshot_lines=>[6]}}
# This means Line 6 (the body of foo) was newly executed
# Clear the counters
Coverage.clear
# Again, run the target functions:
foo()
bar()
# Get newly executed lines:
p Coverage.peek_result #=> {"/.../test-cov.rb"=>{:oneshot_lines=>[]}}
# This means that no new lines were executed
```
## What this is
Traditional coverage tells us "how many times each line was executed". However, it is often enough just to know "whether each line was executed at least once, or not". In this case, the counting just bring unneeded overhead.
Oneshot coverage records only the first execution of each line, and returns line numbers of newly executed lines. It contains less information than traditional coverage, but still useful in many use cases. The hook for each line is executed just once, so after it was fired, the program can run with zero-overhead.
## Why this is needed
I expect two use cases:
### coverage measurement in production
In Ruby, it is difficult to determine if some code is a dead code or not. To check this, some people insert a logging code to the possibly-dead code, run it in production for a while, and check if the log is not emitted.
Oneshot coverage can be used to make this process automatic, comprehensive, and non-invasive.
### CPU-intensive programs
It is known that traditional coverage measurement brings about 20x overhead at worst. It does not matter when testing IO-intensive programs (like Rails application), however, it sometimes matters for CPU-intensive programs.
Oneshot coverage brings the same (or a bit worse) overhead when each line is first executed, but brings no overhead for second and later executions.
## Proposal
Oneshot coverage consists of three parts of APIs.
### API 1: `Coverage.start(oneshot_lines: true)`
This enables the measurement of line coverage with oneshot mode.
In this mode, `Coverage.peek_result` and `result` returns the following format:
```
{ "/path/to/file.rb" => { :oneshot_lines => [an array of executed line numbers] } }
```
### API 2: `Coverage.clear`
This clears all the internal counters of coverage, but keeps the measuring target.
```
p Coverage.peek_result #=> {"/.../test-cov.rb"=>{:oneshot_lines=>[1,5]}}
foo()
p Coverage.peek_result #=> {"/.../test-cov.rb"=>{:oneshot_lines=>[1,5,2]}}
Coverage.clear
p Coverage.peek_result #=> {"/.../test-cov.rb"=>{:oneshot_lines=>[]}}
bar()
p Coverage.peek_result #=> {"/.../test-cov.rb"=>{:oneshot_lines=>[6]}}
```
You can also use this not only for oneshot coverage, but also for traditional one:
```
Coverage.start(lines: true)
load "test-cov.rb"
p Coverage.peek_result #=> {"/.../test-cov.rb"=>{:lines=>[1,0,nil,nil,1,0,nil,nil]}}
Coverage.clear
p Coverage.peek_result #=> {"/.../test-cov.rb"=>{:lines=>[0,0,nil,nil,0,0,nil,nil]}}
foo()
p Coverage.peek_result #=> {"/.../test-cov.rb"=>{:lines=>[0,1,nil,nil,0,0,nil,nil]}}
```
I'm not fully comfortable with this API because it returns incomplete and strange result. However, there have been some requests about restarting coverage (#4796, #9572, #12480), and I think this solves a part of the problem. Actually, it is useful to take coverage per test:
```
Coverage.start(lines: true)
load "target.rb"
each_test do |test|
Coverage.clear
test.run
Coverage.peek_result #=> coverage of each test
end
```
The same result can be get by subtraction between two `peek_result`s, but it is faster and easier.
### API 3: `Coverage.line_stub(filename)`
This is just a simple helper function that returns the "stub" of line coverage from a given source code:
```
Coverage.line_stub("test-cov.rb") #=> [0, 0, nil, nil, 0, 0, nil]
```
This is needed because oneshot coverage tells "which line was executed" but does not tell nothing about other lines.
We need to distinguish that other lines are just "not executed yet" or "not a measuing target (because it is non-significant lines like empty line)".
I don't like the name "line_stub". Counterproposal is welcome.
## Benchmark
As a micro benchmark, I timed the period which it took to run a 10M-line function with three modes: no coverage measurement, oneshot_lines, and lines.
|mode |1st call |2nd call |
|-------------|----------|---------|
|no coverage |0.035 sec |0.035 sec|
|oneshot_lines|0.618 sec |0.034 sec|
|lines |0.405 sec |0.425 sec|
The first call under oneshot_lines is slow, but the second call is as fast as no coverage measurement. On the other hand, lines mode is always slow.
As a relatively bigger CPU-intensive benchmark, I run [optcarrot](https://github.com/mame/optcarrot) with the three modes.
|mode |time |
|-------------|--------|
|no coverage | 5.5 sec|
|oneshot_lines| 5.5 sec|
|lines | 22 sec|
Oneshot lines mode is as fast as no coverage.
## Limitation
Currently, oneshot coverage supports only line coverage. It is theoretically possible to implement it for branch coverage. But, it was difficult for me to design the API, and I think line coverage is enough in many cases.
Due to implementation limitation, traditional line coverage and oneshot one cannot be enabled simultaneously: `Coverage.start(lines: true, oneshot_lines: true)` raises an exception.
---Files--------------------------------
oneshot-coverage.patch (15 KB)
--
https://bugs.ruby-lang.org/
Unsubscribe: <mailto:[email protected]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>