From: hongli@... Date: 2017-10-12T20:18:13+00:00 Subject: [ruby-core:83243] [Ruby trunk Bug#14009] macOS High Sierra and “fork” compatibility Issue #14009 has been updated by hongli (Hongli Lai). Perhaps it helps if I further clarify this issue for those who are unfamiliar with it. As you probably already know, forking (but without exec'ing) in a multithreaded environment is inherently dangerous and the environment must be carefully written to support such a thing. Apple's Objective-C libraries have traditionally not supported being called in a forked (but not exec'd) child process at all, but since High Sierra 10.13 they've tried to add limited support for this. However in doing so they've also defined rules on what is *not* allowed after forking. One of the rules state that it is not allowed to call the `initialize` function of certain Objective-C classes after forking; that may only happen before forking. Makes sense so far. The problem occurs because of a combination of three things: 1. Ruby itself is not linked to any Objective-C libraries, and so does not initialize Objective-C classes by itself. 2. The user may use gems that do link to Objective-C libraries. Due to how these gems are used, it can occur that these gems end up calling Objective-C initializers after the app server has forked. 3. The new Apple-enforced rule checks then abort the process with a warning like this: ~~~ objc[81924]: +[__NSPlaceholderDictionary initialize] may have been in progress in another thread when fork() was called. objc[81924]: +[__NSPlaceholderDictionary initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug. ~~~ By itself, Apple's error check makes sense. Forking is dangerous. But all these factors combined make less sense. Adding a workaround in Ruby (in the form of ensuring that Objective-C initializers are called before forking) will at least ensure that we return to pre-High Sierra behavior. Adding a workaround inside Ruby would only be a part of the whole solution. In order to fully fix the problem, cooperation from the wider Ruby community is required: all the gem authors will also have to ensure that native libraries don't spawn any threads until the app server has forked. But I do believe that having a workaround inside Ruby is an essential part of the entire fix, because updating all gems takes a lot of time and effort. ---------------------------------------- Bug #14009: macOS High Sierra and ���fork��� compatibility https://bugs.ruby-lang.org/issues/14009#change-67194 * Author: ticky (Jessica Stokes) * Status: Open * Priority: Normal * Assignee: * Target version: * ruby -v: ruby 2.4.2p198 (2017-09-14 revision 59899) [x86_64-darwin17] * Backport: 2.3: UNKNOWN, 2.4: UNKNOWN ---------------------------------------- This was originally discussed on the issue tracker for Puma (https://github.com/puma/puma/issues/1421), however, it is possible that it would make more sense for inclusion in the Ruby implementation itself. macOS High Sierra has changed the behaviour of the fork syscall such that initialising Objective-C APIs in forked processes are treated as errors. (see http://sealiesoftware.com/blog/archive/2017/6/5/Objective-C_and_fork_in_macOS_1013.html for more details) This means that many applications which use forking to process concurrently will forcibly crash if the forked process calls out to any Objective-C library when Objective-C was not already initialised in the host process. This includes Puma, Unicorn, iodine and Passenger. A workaround I proposed for Puma was to implicitly load the Objective-C runtime before performing any forks (https://github.com/puma/puma/issues/1421#issuecomment-332650703). This causes forked processes using other Objective-C APIs to not crash. The workaround (specific to Puma���s DSL) was: ~~~ ruby # Work around macOS 10.13 and later being very picky about # `fork` usage and interactions with Objective-C code # see: if /darwin/ =~ RUBY_PLATFORM before_fork do require 'fiddle' # Dynamically load Foundation.framework, ~implicitly~ initialising # the Objective-C runtime before any forking happens in Puma Fiddle.dlopen '/System/Library/Frameworks/Foundation.framework/Foundation' end end ~~~ A similar fix has now been included in Passenger (https://github.com/phusion/passenger/blob/2a55a84e5de721d8bd806a8fea0bcedf27583c29/src/ruby_supportlib/phusion_passenger/loader_shared_helpers.rb#L84-L105). It was, however, proposed that it might make more sense for Ruby on macOS High Sierra and onward to implicitly initialise the Objective-C framework itself, so that forked processes work roughly as expected even if they intend to use Objective-C APIs. I understand that this is a heavy-handed move, but it seems to me that this relatively common technique will remain broken in Ruby unless everyone deploys a workaround (iodine has already expressed disinterest in doing so) or Ruby adopts one at the higher level. This issue is also applicable to all Ruby versions which support fork and run on macOS High Sierra. Thank you for your time. :) -- https://bugs.ruby-lang.org/ Unsubscribe: