From: Hiroshi Shirosaki Date: 2011-11-30T23:24:39+09:00 Subject: [ruby-core:41411] [ruby-trunk - Feature #5562] Improvement of Windows IO performance Issue #5562 has been updated by Hiroshi Shirosaki. File win_io_for_r33908.patch added I updated a patch against trunk@33908. Patched ruby passed "make test" with no failures. Here are test-all results. # patched ruby OpenSSL::TestConfig#test_constants = 0.05 s = F TestDir_M17N#test_filename_as_bytes_extutf8 = 0.45 s = E TestDir_M17N#test_filename_extutf8_inteucjp_unrepresentable = 0.72 s = E TestGDBM#test_reorganize = 0.12 s = F TestGDBM#test_s_open_create_new = 0.11 s = F TestGDBM#test_s_open_error = 0.08 s = F TestGemInstaller#test_generate_bin_bindir_with_user_install_warning = 0.57 s = F TestModule#test_ancestors = 0.00 s = F TestModule#test_included_modules = 0.02 s = F 10300 tests, 1921758 assertions, 7 failures, 2 errors, 95 skips ruby -v: ruby 2.0.0dev (2011-11-30 trunk 33908) [i386-mingw32] # trunk ruby OpenSSL::TestConfig#test_constants = 0.02 s = F TestDir_M17N#test_filename_as_bytes_extutf8 = 0.42 s = E TestDir_M17N#test_filename_extutf8_inteucjp_unrepresentable = 0.67 s = E TestGDBM#test_reorganize = 0.11 s = F TestGDBM#test_s_open_create_new = 0.06 s = F TestGDBM#test_s_open_error = 0.06 s = F TestGemInstaller#test_generate_bin_bindir_with_user_install_warning = 0.42 s = F TestIO_M17N#test_s_pipe_undef = 0.06 s = E TestModule#test_ancestors = 0.00 s = F TestModule#test_included_modules = 0.00 s = F 10300 tests, 1921739 assertions, 7 failures, 3 errors, 95 skips ruby -v: ruby 2.0.0dev (2011-11-30 trunk 33908) [i386-mingw32] I added two tests for binmode method. This patch contains two tests. I preserved original newline conversion behavior of stdin and pipe read for binmode works properly. So stdin and pipe read remain slow. I assume these performance wouldn't be so important. I confirmed patched ruby's file write and read still faster. Trunk ruby has one more error, but make test-all TESTS="ruby/test_io_m17n.rb" passed with no errors. 105) Error: test_s_pipe_undef(TestIO_M17N): Errno::EBADF: Bad file descriptor c:/Users/hiroshi/work/ruby_trunk/test/ruby/test_io_m17n.rb:309:in `read' c:/Users/hiroshi/work/ruby_trunk/test/ruby/test_io_m17n.rb:309:in `block in test_s_pipe_undef' c:/Users/hiroshi/work/ruby_trunk/test/ruby/test_io_m17n.rb:27:in `call' c:/Users/hiroshi/work/ruby_trunk/test/ruby/test_io_m17n.rb:27:in `block in pipe' c:/Users/hiroshi/work/rubyinstaller/sandbox/ruby19_build/.ext/common/win32ole.rb:13:in `call' c:/Users/hiroshi/work/rubyinstaller/sandbox/ruby19_build/.ext/common/win32ole.rb:13:in `block in initialize' About OpenSSL failure, below "C:/projects/rubyinstaller-git" directory doesn't exist in my environment. Is this rubyinstaller's issue? 100) Failure: test_constants(OpenSSL::TestConfig) [c:/Users/hiroshi/work/ruby/test/openssl/test_config.rb:20]: Exception raised: <#>. I checked patched ruby on Mac OSX. Build succeeded. "make test" passed. "make test-all" had below two failures. TestModule#test_ancestors TestModule#test_included_modules Please evaluate this patch. Thank you for your consideration. ---------------------------------------- Feature #5562: Improvement of Windows IO performance http://redmine.ruby-lang.org/issues/5562 Author: Hiroshi Shirosaki Status: Assigned Priority: Normal Assignee: Usaku NAKAMURA Category: core Target version: =begin I suggest a patch to improve Windows IO performance. Ruby's text mode IO is much slower than binary mode. On Windows text mode is default, so Windows IO is slow. I assume that's mainly because of CRLF linefeed code conversion. My idea to improve IO performance is as below. - Change default linefeed conversion from Universal newline to CRLF newline on Windows - Use binary mode process with OS's text mode if only CRLF conversion is needed - Use Ruby's text mode with universal newline conversion if encoding conversion is needed Although that causes io.c code to be more complicated, IO with CRLF conversion performance seems to be improved much. I confirmed "make test-all TEST=ruby" have been passed. There was 3 errors, but ruby without this patch had same errors. I think this patch doesn't affect other OS. Line endings of "p" or "puts" writing is LF on trunk, but CRLF on 1.8.7 or 1.9.2. This patch reverts to CRLF. Here is #1332 benchmark test and results. time = [Time.new] c = '' 'aaaa'.upto('zzzz') {|e| c << e} 4.times { c << c } time << Time.new File.open('out.file','w') { |f| f.write(c) } time << Time.new c = File.open('out.file','r') { |f| f.read } time << Time.new 0.upto(time.size - 2) {|i| p "#{i} #{time[i+1]-time[i]}" } - Result ruby 1.8.7 (2011-06-30 patchlevel 352) [i386-mingw32] "0 0.78125" "1 0.6875" "2 0.5625" ruby 2.0.0dev (2011-11-03) [i386-mingw32] "0 0.59375" "1 1.09375" "2 1.296875" ruby 2.0.0dev (2011-11-03 trunk 33615) [i386-mingw32] with this patch "0 0.625" "1 0.65625" "2 0.34375" =end -- http://redmine.ruby-lang.org