This class provides a compatibility layer between prism and Ripper. It functions by parsing the entire tree first and then walking it and executing each of the Ripper callbacks as it goes. To use this class, you treat Prism::Translation::Ripper effectively as you would treat the Ripper class.
Note that this class will serve the most common use cases, but Ripper’s API is extensive and undocumented. It relies on reporting the state of the parser at any given time. We do our best to replicate that here, but because it is a different architecture it is not possible to perfectly replicate the behavior of Ripper.
The main known difference is that we may omit dispatching some events in some cases. This impacts the following events:
-
on_assign_error
-
on_comma
-
on_ignored_nl
-
on_ignored_sp
-
on_kw
-
on_label_end
-
on_lbrace
-
on_lbracket
-
on_lparen
-
on_nl
-
on_op
-
on_operator_ambiguous
-
on_rbrace
-
on_rbracket
-
on_rparen
-
on_semicolon
-
on_sp
-
on_symbeg
-
on_tstring_beg
-
on_tstring_end
A list of all of the Ruby binary operators.
This array contains name of all ripper events.
This array contains name of parser events.
This contains a table of all of the parser events and their corresponding arity.
This array contains name of scanner events.
This contains a table of all of the scanner events and their corresponding arity.
The current column in bytes of the parser.
The filename of the source being parsed.
The current line number of the parser.
The source that is being parsed.
# File tmp/rubies/ruby-master/lib/prism/translation/ripper.rb, line 71
def self.lex(src, filename = "-", lineno = 1, raise_errors: false)
result = Prism.lex_compat(src, filepath: filename, line: lineno, version: "current")
if result.failure? && raise_errors
raise SyntaxError, result.errors.first.message
else
result.value
end
end
Tokenizes the Ruby program and returns an array of an array, which is formatted like [[lineno, column], type, token, state]. The filename argument is mostly ignored. By default, this method does not handle syntax errors in src, use the raise_errors keyword to raise a SyntaxError for an error in src.
require "ripper"
require "pp"
pp Ripper.lex("def m(a) nil end")
#=> [[[1, 0], :on_kw, "def", FNAME ],
[[1, 3], :on_sp, " ", FNAME ],
[[1, 4], :on_ident, "m", ENDFN ],
[[1, 5], :on_lparen, "(", BEG|LABEL],
[[1, 6], :on_ident, "a", ARG ],
[[1, 7], :on_rparen, ")", ENDFN ],
[[1, 8], :on_sp, " ", BEG ],
[[1, 9], :on_kw, "nil", END ],
[[1, 12], :on_sp, " ", END ],
[[1, 13], :on_kw, "end", END ]]
# File tmp/rubies/ruby-master/lib/prism/translation/ripper.rb, line 482
def initialize(source, filename = "(ripper)", lineno = 1)
if source.is_a?(IO)
@source = source.read
elsif source.respond_to?(:gets)
@source = +""
while line = source.gets
@source << line
end
else
@source = source.to_str
end
@filename = filename
@lineno = lineno
@column = 0
@result = nil
end
Create a new Translation::Ripper object with the given source.
# File tmp/rubies/ruby-master/lib/prism/translation/ripper.rb, line 45
def self.parse(src, filename = "(ripper)", lineno = 1)
new(src, filename, lineno).parse
end
# File tmp/rubies/ruby-master/lib/prism/translation/ripper.rb, line 393
def self.sexp(src, filename = "-", lineno = 1, raise_errors: false)
builder = SexpBuilderPP.new(src, filename, lineno)
sexp = builder.parse
if builder.error?
if raise_errors
raise SyntaxError, builder.error
end
else
sexp
end
end
Parses src and create S-exp tree. Returns more readable tree rather than Ripper.sexp_raw. This method is mainly for developer use. The filename argument is mostly ignored. By default, this method does not handle syntax errors in src, returning nil in such cases. Use the raise_errors keyword to raise a SyntaxError for an error in src.
require "ripper"
require "pp"
pp Ripper.sexp("def m(a) nil end")
#=> [:program,
[[:def,
[:@ident, "m", [1, 4]],
[:paren, [:params, [[:@ident, "a", [1, 6]]], nil, nil, nil, nil, nil, nil]],
[:bodystmt, [[:var_ref, [:@kw, "nil", [1, 9]]]], nil, nil, nil]]]]
# File tmp/rubies/ruby-master/lib/prism/translation/ripper.rb, line 428
def self.sexp_raw(src, filename = "-", lineno = 1, raise_errors: false)
builder = SexpBuilder.new(src, filename, lineno)
sexp = builder.parse
if builder.error?
if raise_errors
raise SyntaxError, builder.error
end
else
sexp
end
end
Parses src and create S-exp tree. This method is mainly for developer use. The filename argument is mostly ignored. By default, this method does not handle syntax errors in src, returning nil in such cases. Use the raise_errors keyword to raise a SyntaxError for an error in src.
require "ripper"
require "pp"
pp Ripper.sexp_raw("def m(a) nil end")
#=> [:program,
[:stmts_add,
[:stmts_new],
[:def,
[:@ident, "m", [1, 4]],
[:paren, [:params, [[:@ident, "a", [1, 6]]], nil, nil, nil]],
[:bodystmt,
[:stmts_add, [:stmts_new], [:var_ref, [:@kw, "nil", [1, 9]]]],
nil,
nil,
nil]]]]
# File tmp/rubies/ruby-master/lib/prism/translation/ripper.rb, line 90
def self.tokenize(...)
lex(...).map { |token| token[2] }
end
Tokenizes the Ruby program and returns an array of strings. The filename and lineno arguments are mostly ignored, since the return value is just the tokenized input. By default, this method does not handle syntax errors in src, use the raise_errors keyword to raise a SyntaxError for an error in src.
p Ripper.tokenize(“def m(a) nil end”) # => [“def”, “ ”, “m”, “(”, “a”, “)”, “ ”, “nil”, “ ”, “end”]
# File tmp/rubies/ruby-master/lib/prism/translation/ripper.rb, line 3505
def compile_error(msg)
end
This method is called when the parser found syntax error.
# File tmp/rubies/ruby-master/lib/prism/translation/ripper.rb, line 3520
def dedent_string(string, width)
whitespace = 0
cursor = 0
while cursor < string.length && string[cursor].match?(/\s/) && whitespace < width
if string[cursor] == "\t"
whitespace = ((whitespace / 8 + 1) * 8)
break if whitespace > width
else
whitespace += 1
end
cursor += 1
end
string.replace(string[cursor..])
cursor
end
This method is provided by the Ripper C extension. It is called when a string needs to be dedented because of a tilde heredoc. It is expected that it will modify the string in place and return the number of bytes that were removed.
# File tmp/rubies/ruby-master/lib/prism/translation/ripper.rb, line 505
def error?
result.failure?
end
True if the parser encountered an error during parsing.
# File tmp/rubies/ruby-master/lib/prism/translation/ripper.rb, line 510
def parse
result.comments.each do |comment|
location = comment.location
bounds(location)
if comment.is_a?(InlineComment)
on_comment(comment.slice)
else
offset = location.start_offset
lines = comment.slice.lines
lines.each_with_index do |line, index|
bounds(location.copy(start_offset: offset))
if index == 0
on_embdoc_beg(line)
elsif index == lines.size - 1
on_embdoc_end(line)
else
on_embdoc(line)
end
offset += line.bytesize
end
end
end
result.magic_comments.each do |magic_comment|
on_magic_comment(magic_comment.key, magic_comment.value)
end
unless result.data_loc.nil?
on___end__(result.data_loc.slice.each_line.first)
end
result.warnings.each do |warning|
bounds(warning.location)
if warning.level == :default
warning(warning.message)
else
case warning.type
when :ambiguous_first_argument_plus
on_arg_ambiguous("+")
when :ambiguous_first_argument_minus
on_arg_ambiguous("-")
when :ambiguous_slash
on_arg_ambiguous("/")
else
warn(warning.message)
end
end
end
if error?
result.errors.each do |error|
location = error.location
bounds(location)
case error.type
when :alias_argument
on_alias_error("can't make alias for the number variables", location.slice)
when :argument_formal_class
on_param_error("formal argument cannot be a class variable", location.slice)
when :argument_format_constant
on_param_error("formal argument cannot be a constant", location.slice)
when :argument_formal_global
on_param_error("formal argument cannot be a global variable", location.slice)
when :argument_formal_ivar
on_param_error("formal argument cannot be an instance variable", location.slice)
when :class_name, :module_name
on_class_name_error("class/module name must be CONSTANT", location.slice)
else
on_parse_error(error.message)
end
end
nil
else
result.value.accept(self)
end
end
Parse the source and return the result.
# File tmp/rubies/ruby-master/lib/prism/translation/ripper.rb, line 3496
def warn(fmt, *args)
end
This method is called when weak warning is produced by the parser. fmt and args is printf style.
# File tmp/rubies/ruby-master/lib/prism/translation/ripper.rb, line 3501
def warning(fmt, *args)
end
This method is called when strong warning is produced by the parser. fmt and args is printf style.