From: "turboladen (Steve Loveless)" <steve.loveless@...>
Date: 2012-05-31T08:15:07+09:00
Subject: [ruby-core:45328] [ruby-trunk - Bug #6306] Ripper lexes :on_kw when it should find :on_ident


Issue #6306 has been updated by turboladen (Steve Loveless).


If this is how the lexer works, then I guess there's not much use in pleading my case, but FWIW, I'll explain more...

I'm the author of http://rubygems.org/gems/tailor and I'm trying to parse code that looks, for example, like:

  my_symbol = :def

I'm (conceptually) expecting to have Ripper tell me that ':def' is a symbol, plain and simple--not that ':' is the beginning of a symbol and the text used to describe the symbol is the same text that also happens to be a keyword.  Using ':def' like above doesn't behave as a keyword, so from where I sit, it's odd to be told that it is a keyword.  I'd expect similar behavior as:

  Ripper.lex('def') #=> [[[1, 0], :on_tstring_beg, "'"], [[1, 1], :on_tstring_content, "def"], [[1, 4], :on_tstring_end, "'"]] 

The tailor gem uses Ripper to, in this case, determine that good style means you should indent (in most cases) the next line after you start defining a method using 'def'.  Since ':def' lexes as a symbeg + kw, but doesn't behave as a keyword, this means I have to use extra logic to figure out if it's being used as a symbol or an actual keyword--which is the main reason I opted to use Ripper in the first place: it tells me what context text like "def" and "class" and "self" are being used in.  While I understand the reason for saying it's not a bug, as a Ripper _user_ I find this behavior inconsistent.
----------------------------------------
Bug #6306: Ripper lexes :on_kw when it should find :on_ident
https://bugs.ruby-lang.org/issues/6306#change-26910

Author: turboladen (Steve Loveless)
Status: Rejected
Priority: Normal
Assignee: 
Category: 
Target version: 
ruby -v: ruby 1.9.3p125 (2012-02-16 revision 34643) [x86_64-darwin11.3.0]


=begin
  1.9.3-p125 :001 > require 'ripper'
  => true 
  1.9.3-p125 :002 > Ripper.lex %Q{:def}
  => [[[1, 0], :on_symbeg, ":"], [[1, 1], :on_kw, "def"]]

I'd expect (({[[1, 1], :on_kw, "def"]})) to be (({[[1, 1], :on_ident, "def]})).  Sure, "def" is a keyword, but since it's prefaced by a ':', it's not necessarily being used in that context.  The same behavior pertains to all other keywords that are used as Symbols.
=end


-- 
http://bugs.ruby-lang.org/