Prism::Translation::Ripper

Class

This class provides a compatibility layer between prism and Ripper. It functions by parsing the entire tree first and then walking it and executing each of the Ripper callbacks as it goes. To use this class, you treat Prism::Translation::Ripper effectively as you would treat the Ripper class.

Note that this class will serve the most common use cases, but Ripper’s API is extensive and undocumented. It relies on reporting the state of the parser at any given time. We do our best to replicate that here, but because it is a different architecture it is not possible to perfectly replicate the behavior of Ripper.

The main known difference is that we may omit dispatching some events in some cases. This impacts the following events:

  • on_assign_error

  • on_comma

  • on_ignored_nl

  • on_ignored_sp

  • on_kw

  • on_label_end

  • on_lbrace

  • on_lbracket

  • on_lparen

  • on_nl

  • on_op

  • on_operator_ambiguous

  • on_rbrace

  • on_rbracket

  • on_rparen

  • on_semicolon

  • on_sp

  • on_symbeg

  • on_tstring_beg

  • on_tstring_end

Constants

PARSER_EVENT_TABLE

This contains a table of all of the parser events and their corresponding arity.

SCANNER_EVENT_TABLE

This contains a table of all of the scanner events and their corresponding arity.

PARSER_EVENTS

This array contains name of parser events.

SCANNER_EVENTS

This array contains name of scanner events.

EVENTS

This array contains name of all ripper events.

KEYWORDS

A list of all of the Ruby keywords.

BINARY_OPERATORS

A list of all of the Ruby binary operators.

Attributes

source

Read

The source that is being parsed.

filename

Read

The filename of the source being parsed.

lineno

Read

The current line number of the parser.

column

Read

The current column number of the parser.

Class Methods

lex(src, filename = "-", lineno = 1, raise_errors: false)
::

Tokenizes the Ruby program and returns an array of an array, which is formatted like [[lineno, column], type, token, state]. The filename argument is mostly ignored. By default, this method does not handle syntax errors in src, use the raise_errors keyword to raise a SyntaxError for an error in src.

require "ripper"
require "pp"

pp Ripper.lex("def m(a) nil end")
#=> [[[1,  0], :on_kw,     "def", FNAME    ],
     [[1,  3], :on_sp,     " ",   FNAME    ],
     [[1,  4], :on_ident,  "m",   ENDFN    ],
     [[1,  5], :on_lparen, "(",   BEG|LABEL],
     [[1,  6], :on_ident,  "a",   ARG      ],
     [[1,  7], :on_rparen, ")",   ENDFN    ],
     [[1,  8], :on_sp,     " ",   BEG      ],
     [[1,  9], :on_kw,     "nil", END      ],
     [[1, 12], :on_sp,     " ",   END      ],
     [[1, 13], :on_kw,     "end", END      ]]

new(source, filename = "(ripper)", lineno = 1)
::

Create a new Translation::Ripper object with the given source.

parse(src, filename = "(ripper)", lineno = 1)
::

Parses the given Ruby program read from src. src must be a String or an IO or a object with a gets method.

sexp(src, filename = "-", lineno = 1, raise_errors: false)
::

Parses src and create S-exp tree. Returns more readable tree rather than Ripper.sexp_raw. This method is mainly for developer use. The filename argument is mostly ignored. By default, this method does not handle syntax errors in src, returning nil in such cases. Use the raise_errors keyword to raise a SyntaxError for an error in src.

require "ripper"
require "pp"

pp Ripper.sexp("def m(a) nil end")
  #=> [:program,
       [[:def,
        [:@ident, "m", [1, 4]],
        [:paren, [:params, [[:@ident, "a", [1, 6]]], nil, nil, nil, nil, nil, nil]],
        [:bodystmt, [[:var_ref, [:@kw, "nil", [1, 9]]]], nil, nil, nil]]]]

sexp_raw(src, filename = "-", lineno = 1, raise_errors: false)
::

Parses src and create S-exp tree. This method is mainly for developer use. The filename argument is mostly ignored. By default, this method does not handle syntax errors in src, returning nil in such cases. Use the raise_errors keyword to raise a SyntaxError for an error in src.

require "ripper"
require "pp"

pp Ripper.sexp_raw("def m(a) nil end")
  #=> [:program,
       [:stmts_add,
        [:stmts_new],
        [:def,
         [:@ident, "m", [1, 4]],
         [:paren, [:params, [[:@ident, "a", [1, 6]]], nil, nil, nil]],
         [:bodystmt,
          [:stmts_add, [:stmts_new], [:var_ref, [:@kw, "nil", [1, 9]]]],
          nil,
          nil,
          nil]]]]
Instance Methods

bounds(location)
#

This method is responsible for updating lineno and column information to reflect the current node.

This method could be drastically improved with some caching on the start of every line, but for now it’s good enough.

command?(node)
#

Returns true if the given node is a command node.

compile_error(msg)
#

This method is called when the parser found syntax error.

dedent_string(string, width)
#

This method is provided by the Ripper C extension. It is called when a string needs to be dedented because of a tilde heredoc. It is expected that it will modify the string in place and return the number of bytes that were removed.

error?
#

True if the parser encountered an error during parsing.

parse
#

Parse the source and return the result.

result
#

Lazily initialize the parse result.

trailing_comma?(left, right)
#

Returns true if there is a comma between the two locations.

visit_alias_global_variable_node(node)
#

alias $foo $bar ^^^^^^^^^^^^^^^

visit_alias_global_variable_node_value(node)
#

Visit one side of an alias global variable node.

visit_alias_method_node(node)
#

alias foo bar ^^^^^^^^^^^^^

visit_alternation_pattern_node(node)
#

foo => bar | baz ^^^^^^^^^

visit_and_node(node)
#

a and b ^^^^^^^

visit_arguments(elements)
#

Visit a list of elements, like the elements of an array or arguments.

visit_arguments_node(node)
#

foo(bar) ^^^

visit_array_node(node)
#

[] ^^

visit_array_pattern_node(node)
#

foo => [bar] ^^^^^

visit_assoc_node(node)
#

{ a: 1 } ^^^^

visit_assoc_splat_node(node)
#

def foo(); bar(); end ^^

{ **foo } ^^^^^

visit_back_reference_read_node(node)
#

$+ ^^

visit_begin_node(node)
#

begin end ^^^^^^^^^

visit_begin_node_clauses(location, node, allow_newline)
#

Visit the clauses of a begin node to form an on_bodystmt call.

visit_block_argument_node(node)
#

foo(&bar) ^^^^

visit_block_local_variable_node(node)
#

foo { |; bar| } ^^^

visit_block_node(node)
#

Visit a BlockNode.

visit_block_parameter_node(node)
#

def foo(&bar); end ^^^^

visit_block_parameters_node(node)
#

A block’s parameters.

visit_body_node(location, node, allow_newline = false)
#

Visit the body of a structure that can have either a set of statements or statements wrapped in rescue/else/ensure.

visit_break_node(node)
#

break ^^^^^

break foo ^^^^^^^^^

visit_call_and_write_node(node)
#

foo.bar &&= baz ^^^^^^^^^^^^^^^

visit_call_node(node)
#

foo ^^^

foo.bar ^^^^^^^

foo.bar() {} ^^^^^^^^^^^^

visit_call_node_arguments(arguments_node, block_node, trailing_comma)
#

Visit the arguments and block of a call node and return the arguments and block as they should be used.

visit_call_operator_write_node(node)
#

foo.bar += baz ^^^^^^^^^^^^^^^

visit_call_or_write_node(node)
#

foo.bar ||= baz ^^^^^^^^^^^^^^^

visit_call_target_node(node)
#

foo.bar, = 1 ^^^^^^^

visit_capture_pattern_node(node)
#

foo => bar => baz ^^^^^^^^^^

visit_case_match_node(node)
#

case foo; in bar; end ^^^^^^^^^^^^^^^^^^^^^

visit_case_node(node)
#

case foo; when bar; end ^^^^^^^^^^^^^^^^^^^^^^^

visit_class_node(node)
#

class Foo; end ^^^^^^^^^^^^^^

visit_class_variable_and_write_node(node)
#

@@foo &&= bar ^^^^^^^^^^^^^

visit_class_variable_operator_write_node(node)
#

@@foo += bar ^^^^^^^^^^^^

visit_class_variable_or_write_node(node)
#

@@foo ||= bar ^^^^^^^^^^^^^

visit_class_variable_read_node(node)
#

@@foo ^^^^^

visit_class_variable_target_node(node)
#

@@foo, = bar ^^^^^

visit_class_variable_write_node(node)
#

@@foo = 1 ^^^^^^^^^

@@foo, @@bar = 1 ^^^^^ ^^^^^

visit_constant_and_write_node(node)
#

Foo &&= bar ^^^^^^^^^^^^

visit_constant_operator_write_node(node)
#

Foo += bar ^^^^^^^^^^^

visit_constant_or_write_node(node)
#

Foo ||= bar ^^^^^^^^^^^^

visit_constant_path_and_write_node(node)
#

Foo::Bar &&= baz ^^^^^^^^^^^^^^^^

visit_constant_path_node(node)
#

Foo::Bar ^^^^^^^^

visit_constant_path_operator_write_node(node)
#

Foo::Bar += baz ^^^^^^^^^^^^^^^

visit_constant_path_or_write_node(node)
#

Foo::Bar ||= baz ^^^^^^^^^^^^^^^^

visit_constant_path_target_node(node)
#

Foo::Bar, = baz ^^^^^^^^

visit_constant_path_write_node(node)
#

Foo::Bar = 1 ^^^^^^^^^^^^

Foo::Foo, Bar::Bar = 1 ^^^^^^^^ ^^^^^^^^

visit_constant_path_write_node_target(node)
#

Visit a constant path that is part of a write node.

visit_constant_read_node(node)
#

Foo ^^^

visit_constant_target_node(node)
#

Foo, = bar ^^^

visit_constant_write_node(node)
#

Foo = 1 ^^^^^^^

Foo, Bar = 1 ^^^ ^^^

visit_def_node(node)
#

def foo; end ^^^^^^^^^^^^

def self.foo; end ^^^^^^^^^^^^^^^^^

visit_defined_node(node)
#

defined? a ^^^^^^^^^^

defined?(a) ^^^^^^^^^^^

visit_destructured_parameter_node(node)
#

Visit a destructured positional parameter node.

visit_else_node(node)
#

if foo then bar else baz end ^^^^^^^^^^^^

visit_embedded_statements_node(node)
#

“foo #{bar}” ^^^^^^

visit_embedded_variable_node(node)
#

“foo #@bar” ^^^^^

visit_ensure_node(node)
#

Visit an EnsureNode node.

visit_false_node(node)
#

false ^^^^^

visit_find_pattern_node(node)
#

foo => [, bar, ] ^^^^^^^^^^^

visit_flip_flop_node(node)
#

if foo .. bar; end ^^^^^^^^^^

visit_float_node(node)
#

1.0 ^^^

visit_for_node(node)
#

for foo in bar do end ^^^^^^^^^^^^^^^^^^^^^

visit_forwarding_arguments_node(node)
#

def foo(…); bar(…); end ^^^

visit_forwarding_parameter_node(node)
#

def foo(…); end ^^^

visit_forwarding_super_node(node)
#

super ^^^^^

super {} ^^^^^^^^

visit_global_variable_and_write_node(node)
#

$foo &&= bar ^^^^^^^^^^^^

visit_global_variable_operator_write_node(node)
#

$foo += bar ^^^^^^^^^^^

visit_global_variable_or_write_node(node)
#

$foo ||= bar ^^^^^^^^^^^^

visit_global_variable_read_node(node)
#

$foo ^^^^

visit_global_variable_target_node(node)
#

$foo, = bar ^^^^

visit_global_variable_write_node(node)
#

$foo = 1 ^^^^^^^^

$foo, $bar = 1 ^^^^ ^^^^

visit_hash_node(node)
#

{} ^^

visit_hash_pattern_node(node)
#

foo => {} ^^

visit_heredoc_node(parts, base) { |result, on_tstring_content(map(&:content).join)| ... }
#

Visit a string that is expressed using a <<~ heredoc.

visit_heredoc_node_whitespace(parts)
#

Ripper gives back the escaped string content but strips out the common leading whitespace. Prism gives back the unescaped string content and a location for the escaped string content. Unfortunately these don’t work well together, so here we need to re-derive the common leading whitespace.

visit_heredoc_string_node(node)
#

Visit a heredoc node that is representing a string.

visit_heredoc_x_string_node(node)
#

Visit a heredoc node that is representing an xstring.

visit_if_node(node)
#

if foo then bar end ^^^^^^^^^^^^^^^^^^^

bar if foo ^^^^^^^^^^

foo ? bar : baz ^^^^^^^^^^^^^^^

visit_imaginary_node(node)
#

1i ^^

visit_implicit_node(node)
#

{ foo: } ^^^^

visit_implicit_rest_node(node)
#

foo { |bar,| } ^

visit_in_node(node)
#

case foo; in bar; end ^^^^^^^^^^^^^^^^^^^^^

visit_index_and_write_node(node)
#

foo &&= baz ^^^^^^^^^^^^^^^^

visit_index_operator_write_node(node)
#

foo += baz ^^^^^^^^^^^^^^^

visit_index_or_write_node(node)
#

foo ||= baz ^^^^^^^^^^^^^^^^

visit_index_target_node(node)
#

foo, = 1 ^^^^^^^^

visit_instance_variable_and_write_node(node)
#

@foo &&= bar ^^^^^^^^^^^^

visit_instance_variable_operator_write_node(node)
#

@foo += bar ^^^^^^^^^^^

visit_instance_variable_or_write_node(node)
#

@foo ||= bar ^^^^^^^^^^^^

visit_instance_variable_read_node(node)
#

@foo ^^^^

visit_instance_variable_target_node(node)
#

@foo, = bar ^^^^

visit_instance_variable_write_node(node)
#

@foo = 1 ^^^^^^^^

visit_integer_node(node)
#

1 ^

visit_interpolated_match_last_line_node(node)
#

if /foo #{bar}/ then end ^^^^^^^^^^^^

visit_interpolated_regular_expression_node(node)
#

/foo #{bar}/ ^^^^^^^^^^^^

visit_interpolated_string_node(node)
#

“foo #{bar}” ^^^^^^^^^^^^

visit_interpolated_symbol_node(node)
#

:“foo #{bar}” ^^^^^^^^^^^^^

visit_interpolated_x_string_node(node)
#

foo #{bar} ^^^^^^^^^^^^

visit_it_local_variable_read_node(node)
#

-> { it } ^^

visit_it_parameters_node(node)
#

-> { it } ^^^^^^^^^

visit_keyword_hash_node(node)
#

foo(bar: baz) ^^^^^^^^

visit_keyword_rest_parameter_node(node)
#

def foo(**bar); end ^^^^^

def foo(**); end ^^

visit_lambda_node(node)
#

-> {}

visit_local_variable_and_write_node(node)
#

foo &&= bar ^^^^^^^^^^^

visit_local_variable_operator_write_node(node)
#

foo += bar ^^^^^^^^^^

visit_local_variable_or_write_node(node)
#

foo ||= bar ^^^^^^^^^^^

visit_local_variable_read_node(node)
#

foo ^^^

visit_local_variable_target_node(node)
#

foo, = bar ^^^

visit_local_variable_write_node(node)
#

foo = 1 ^^^^^^^

visit_match_last_line_node(node)
#

if /foo/ then end ^^^^^

visit_match_predicate_node(node)
#

foo in bar ^^^^^^^^^^

visit_match_required_node(node)
#

foo => bar ^^^^^^^^^^

visit_match_write_node(node)
#

/(?<foo>foo)/ =~ bar ^^^^^^^^^^^^^^^^^^^^

visit_missing_node(node)
#

A node that is missing from the syntax tree. This is only used in the case of a syntax error.

visit_module_node(node)
#

module Foo; end ^^^^^^^^^^^^^^^

visit_multi_target_node(node)
#

(foo, bar), bar = qux ^^^^^^^^^^

visit_multi_target_node_targets(lefts, rest, rights, skippable)
#

Visit the targets of a multi-target node.

visit_multi_write_node(node)
#

foo, bar = baz ^^^^^^^^^^^^^^

visit_next_node(node)
#

next ^^^^

next foo ^^^^^^^^

visit_nil_node(node)
#

nil ^^^

visit_no_keywords_parameter_node(node)
#

def foo(**nil); end ^^^^^

visit_number_node(node) { |slice| ... }
#

Visit a node that represents a number. We need to explicitly handle the unary - operator.

visit_numbered_parameters_node(node)
#

-> { 1 + 2 } ^^^^^^^^^^^^^^

visit_numbered_reference_read_node(node)
#

$1 ^^

visit_optional_keyword_parameter_node(node)
#

def foo(bar: baz); end ^^^^^^^^

visit_optional_parameter_node(node)
#

def foo(bar = 1); end ^^^^^^^

visit_or_node(node)
#

a or b ^^^^^^

visit_parameters_node(node)
#

def foo(bar, *baz); end ^^^^^^^^^

visit_parentheses_node(node)
#

() ^^

(1) ^^^

visit_pattern_node(node)
#

Visit a pattern within a pattern match. This is used to bypass the parenthesis node that can be used to wrap patterns.

visit_pinned_expression_node(node)
#

foo => ^(bar) ^^^^^^

visit_pinned_variable_node(node)
#

foo = 1 and bar => ^foo ^^^^

visit_post_execution_node(node)
#

END {} ^^^^^^

visit_pre_execution_node(node)
#

BEGIN {} ^^^^^^^^

visit_program_node(node)
#

The top-level program node.

visit_range_node(node)
#

0..5 ^^^^

visit_rational_node(node)
#

1r ^^

visit_redo_node(node)
#

redo ^^^^

visit_regular_expression_node(node)
#

/foo/ ^^^^^

visit_required_keyword_parameter_node(node)
#

def foo(bar:); end ^^^^

visit_required_parameter_node(node)
#

def foo(bar); end ^^^

visit_rescue_modifier_node(node)
#

foo rescue bar ^^^^^^^^^^^^^^

visit_rescue_node(node)
#

begin; rescue; end ^^^^^^^

visit_rest_parameter_node(node)
#

def foo(*bar); end ^^^^

def foo(*); end ^

visit_retry_node(node)
#

retry ^^^^^

visit_return_node(node)
#

return ^^^^^^

return 1 ^^^^^^^^

visit_self_node(node)
#

self ^^^^

visit_shareable_constant_node(node)
#

A shareable constant.

visit_singleton_class_node(node)
#

class << self; end ^^^^^^^^^^^^^^^^^^

visit_source_encoding_node(node)
#

ENCODING ^^^^^^^^^^^^

visit_source_file_node(node)
#

FILE ^^^^^^^^

visit_source_line_node(node)
#

LINE ^^^^^^^^

visit_splat_node(node)
#

foo(*bar) ^^^^

def foo((bar, *baz)); end ^^^^

def foo(); bar(); end ^

visit_statements_node(node)
#

A list of statements.

visit_statements_node_body(body)
#

Visit the list of statements of a statements node. We support nil statements in the list. This would normally not be allowed by the structure of the prism parse tree, but we manually add them here so that we can mirror Ripper’s void stmt.

visit_string_content(part)
#

Visit an individual part of a string-like node.

visit_string_node(node)
#

“foo” ^^^^^

visit_super_node(node)
#

super(foo) ^^^^^^^^^^

visit_symbol_node(node)
#

:foo ^^^^

visit_token(token, allow_keywords = true)
#

Visit the string content of a particular node. This method is used to split into the various token types.

visit_true_node(node)
#

true ^^^^

visit_undef_node(node)
#

undef foo ^^^^^^^^^

visit_unless_node(node)
#

unless foo; bar end ^^^^^^^^^^^^^^^^^^^

bar unless foo ^^^^^^^^^^^^^^

visit_until_node(node)
#

until foo; bar end ^^^^^^^^^^^^^^^^^

bar until foo ^^^^^^^^^^^^^

visit_when_node(node)
#

case foo; when bar; end ^^^^^^^^^^^^^

visit_while_node(node)
#

while foo; bar end ^^^^^^^^^^^^^^^^^^

bar while foo ^^^^^^^^^^^^^

visit_words_sep(opening_loc, previous, current)
#

Dispatch a words_sep event that contains the space between the elements of list literals.

visit_write_value(node)
#

Visit a node that represents a write value. This is used to handle the special case of an implicit array that is generated without brackets.

visit_x_string_node(node)
#

foo ^^^^^

visit_yield_node(node)
#

yield ^^^^^

yield 1 ^^^^^^^

void_stmt?(left, right, allow_newline)
#

Returns true if there is a semicolon between the two locations.

warn(fmt, *args)
#

This method is called when weak warning is produced by the parser. fmt and args is printf style.

warning(fmt, *args)
#

This method is called when strong warning is produced by the parser. fmt and args is printf style.